Loop devices exhausted on fedora-vm02

[Eyal Edri]

----- Original Message -----
> From: "Fabian Deutsch" <fabiand at redhat.com>
> To: "Mike Burns" <mburns at redhat.com>
> Cc: "Eyal Edri" <eedri at redhat.com>, infra at ovirt.org
> Sent: Monday, March 18, 2013 6:48:58 PM
> Subject: Re: Loop devices exhausted on fedora-vm02
> 
> Am Montag, den 18.03.2013, 09:34 -0400 schrieb Mike Burns:
> > On 03/18/2013 08:25 AM, Eyal Edri wrote:
> > >
> > >
> > > ----- Original Message -----
> > >> From: "Fabian Deutsch" <fabiand at redhat.com>
> > >> To: infra at ovirt.org
> > >> Sent: Monday, March 18, 2013 1:08:12 PM
> > >> Subject: Loop devices exhausted on fedora-vm02
> > >>
> > >> Hey,
> > >>
> > >> it seems that all loop devices on fedora-vm02
> > >> are taken:
> > >> http://jenkins.ovirt.org/view/ovirt_node/job/ovirt-node-iso/671/console
> > >>
> > >> Could someone look into this?
> > >
> > > can you explain why it happens? (ovirt-node jobs?)
> > > maybe the job should add a cleanup step at end to release unused
> > > loop devices?
> > 
> > Looking here [1] there are 2 iso build jobs that could be running
> > on the
> > host at the same time.
> 
> Good catch!
> 
> > We could throttle the jobs so that they can't run concurrently.  We
> > can
> > also simply add something like max_loops=256 on the kernel
> > commandline
> > so that there are extra loop devices.

is this still needed? 

i don't see a /etc/modprobe.conf file on the f18 slave:

i imagine this needs to be added:

#vim /etc/modprobe.conf
...
options loop max_loop=256.
...


> 
> The Node specififc job is throttled but I didn't take into account
> that
> another job might be editing/creating livecds.
> 
> > We've seen this happen before, and livecd-tools is pretty good
> > about
> > cleaning up, even in failure scenarios (as long as you don't ctrl-c
> > or
> > kill -9 the job).  But a single job uses a number of loop devices
> > (3,4,5,6, not actually sure the number).  It also uses lazy umounts
> > in
> > some cases meaning that there could be a couple loop devices in use
> > that
> > get cleaned up at a later point.
> > 
> > The simplest fix for this would be to increase the default number
> > of
> > loop devices.
> 
> +1 for this solution from me too.
> It seems that two concurrent livecd jobs just need to many loop
> devices.
> 
> - fabian
> 
> 
> 
>