Re: Loop devices exhausted on fedora-vm02

On 03/18/2013 08:25 AM, Eyal Edri wrote: > > > ----- Original Message ----- >> From: "Fabian Deutsch" <fabiand(a)redhat.com&gt; >> To: infra(a)ovirt.org >> Sent: Monday, March 18, 2013 1:08:12 PM >> Subject: Loop devices exhausted on fedora-vm02 >> >> Hey, >> >> it seems that all loop devices on fedora-vm02 >> are taken: >> http://jenkins.ovirt.org/view/ovirt_node/job/ovirt-node-iso/671/console >> >> Could someone look into this? > > can you explain why it happens? (ovirt-node jobs?) > maybe the job should add a cleanup step at end to release unused loop devices? Looking here [1] there are 2 iso build jobs that could be running on the host at the same time.

We could throttle the jobs so that they can't run concurrently. We can also simply add something like max_loops=256 on the kernel commandline so that there are extra loop devices.

We've seen this happen before, and livecd-tools is pretty good about cleaning up, even in failure scenarios (as long as you don't ctrl-c or kill -9 the job). But a single job uses a number of loop devices (3,4,5,6, not actually sure the number). It also uses lazy umounts in some cases meaning that there could be a couple loop devices in use that get cleaned up at a later point. The simplest fix for this would be to increase the default number of loop devices.