On Tue, Jun 17, 2014 at 10:33:24AM +0200, David Caro wrote:
On Tue 17 Jun 2014 10:24:49 AM CEST, Dan Kenigsberg wrote:
On Tue, Jun 17, 2014 at 03:40:51AM -0400, Vered Volansky wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Vered Volansky" <vered@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Monday, June 16, 2014 11:29:42 AM Subject: Re: Bad setup code in vdsm_master_storage_functional_tests_localfs_gerrit
On Sun, Jun 15, 2014 at 04:11:53AM -0400, Vered Volansky wrote:
The job with this issue is gone, let me know if it's risen again.
The fragile code is still in http://jenkins.ovirt.org/view/All/job/vdsm_master_storage_functional_tests_l... why not make it more robust before /var/log/vdsm disappears and make it break again?
because I don't understand the issue. The file is only created if missing. The directory should be there.
It was fixed by me some time ago (I added the mkdir -p before the touch, just in case)
sudo mkdir -p /var/log/vdsm sudo chown vdsm:kvm sudo sh -c 'echo "" > /var/log/vdsm/vdsm.log' sudo sh -c 'echo "" > /var/log/vdsm/supervdsm.log'
However, apparently it was not there, which made the echo fail, which led to the job failing. We should understand why it disappeared.
dcaro, eedri - do you have any idea?
Totally agree, and, if it was meant to be there, I'll remove the mkdir to make the test fail if it's not there. But for what I see on the job, there's nothing that ensures you that the directory will be there, vdsm might never have been installed on that machine, or might have been properly cleaned at some point (removing logs and leftovers). So, In my opinion the issue is that we are not cleaning up properly after the vdsm jobs and leaving the logs behind.
We can never be sure that the former job running on a particular thread has finished properly. It might have aborted before it cleaned the log. Thus, it's prudent to clear the log before a new job starts. But my original question begins - what could possibly remove the directory between the mkdir and the echo? Are you aware of anything that removes the directory (removal of vdsm.rpm does not) Whatever it was, it must not be running while a functional test takes place!