Loop devices exhausted on fedora-vm02

[Fabian Deutsch]

Am Montag, den 18.03.2013, 09:34 -0400 schrieb Mike Burns:
> On 03/18/2013 08:25 AM, Eyal Edri wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Fabian Deutsch" <fabiand at redhat.com>
> >> To: infra at ovirt.org
> >> Sent: Monday, March 18, 2013 1:08:12 PM
> >> Subject: Loop devices exhausted on fedora-vm02
> >>
> >> Hey,
> >>
> >> it seems that all loop devices on fedora-vm02
> >> are taken:
> >> http://jenkins.ovirt.org/view/ovirt_node/job/ovirt-node-iso/671/console
> >>
> >> Could someone look into this?
> >
> > can you explain why it happens? (ovirt-node jobs?)
> > maybe the job should add a cleanup step at end to release unused loop devices?
> 
> Looking here [1] there are 2 iso build jobs that could be running on the 
> host at the same time.

Good catch!

> We could throttle the jobs so that they can't run concurrently.  We can 
> also simply add something like max_loops=256 on the kernel commandline 
> so that there are extra loop devices.

The Node specififc job is throttled but I didn't take into account that
another job might be editing/creating livecds.

> We've seen this happen before, and livecd-tools is pretty good about 
> cleaning up, even in failure scenarios (as long as you don't ctrl-c or 
> kill -9 the job).  But a single job uses a number of loop devices 
> (3,4,5,6, not actually sure the number).  It also uses lazy umounts in 
> some cases meaning that there could be a couple loop devices in use that 
> get cleaned up at a later point.
> 
> The simplest fix for this would be to increase the default number of 
> loop devices.

+1 for this solution from me too.
It seems that two concurrent livecd jobs just need to many loop devices.

- fabian