<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 19, 2017 at 9:22 AM, Michal Skrivanek <span dir="ltr"><<a href="mailto:michal.skrivanek@redhat.com" target="_blank">michal.skrivanek@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div><div class="h5"><blockquote type="cite"><div>On 19 Dec 2017, at 07:53, Eyal Edri <<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>> wrote:</div><br class="m_-3080504837579530323Apple-interchange-newline"><div><div dir="auto"><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Dec 19, 2017 00:20, "Michal Skrivanek" <<a href="mailto:michal.skrivanek@redhat.com" target="_blank">michal.skrivanek@redhat.com</a>> wrote:<br type="attribution"><blockquote class="m_-3080504837579530323quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="m_-3080504837579530323elided-text"><br>
> On 18 Dec 2017, at 13:21, Milan Zamazal <<a href="mailto:mzamazal@redhat.com" target="_blank">mzamazal@redhat.com</a>> wrote:<br>
><br>
> Yedidyah Bar David <<a href="mailto:didi@redhat.com" target="_blank">didi@redhat.com</a>> writes:<br>
><br>
>> On Mon, Dec 18, 2017 at 10:17 AM, Code Review <<a href="mailto:gerrit@ovirt.org" target="_blank">gerrit@ovirt.org</a>> wrote:<br>
>>> Jenkins CI posted comments on this change.<br>
>>><br>
>><br>
>>> View Change<br>
>>><br>
>>> Patch set 3:Continuous-Integration -1<br>
>>><br>
>>> Build Failed<br>
>>><br>
>>> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/2882/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/o<wbr>virt-system-tests_master_check<wbr>-patch-el7-x86_64/2882/</a><br>
>>> : FAILURE<br>
>><br>
>> Console output of above job says:<br>
>><br>
>> 08:13:34 # migrate_vm:<br>
>> 08:16:37 * Collect artifacts:<br>
>> 08:16:40 * Collect artifacts: Success (in 0:00:03)<br>
>> 08:16:40 # migrate_vm: Success (in 0:03:06)<br>
>> 08:16:40 # Results located at<br>
>> /dev/shm/ost/deployment-basic-<wbr>suite-master/default/006_migra<wbr>tions.py.junit.xml<br>
>> 08:16:40 @ Run test: 006_migrations.py: Success (in 0:03:50)<br>
>> 08:16:40 Error occured, aborting<br>
>><br>
>> The file 006_migrations.py.junit.xml [1] says:<br>
>><br>
>> <failure type="exceptions.AssertionErro<wbr>r" message="False != True after<br>
>> 180 seconds"><br>
><br>
> Reading the logs, I can see the VM migrates normally and seems to be<br>
> reported to Engine correctly. When Engine receives end-of-migration<br>
> event, it sends Destroy to the source (which is correct), calls dumpxmls<br>
> on the destination in the meantime (looks fine to me) and then calls<br>
<br>
</div>looks like a race between getallvmstats reporting VM as Down (statusTime: 4296271980) being processed, while there is a Down/MigrationSucceeded event arriving (with notify_time 4296273170) at about the same time<br>
Unfortunately the vdsm.log is not in DEBUG level so there’s very little information as to why and what exactly did it send out.<br>
@infra - can you enable debug log level for vdsm by default?<br></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">How do you enable debug mode for vdsm ?</div></div></div></blockquote><div><br></div></div></div>Can be permanently set in /etc/vdsm/logger.conf</div><div>I suppose enabling for all loggers would be useful</div><div><br></div><div>see <a href="https://access.redhat.com/articles/2919931" target="_blank">https://access.redhat.com/<wbr>articles/2919931</a></div></div></blockquote><div><br></div><div><br></div><div>Since this file probably is created only after host is added, I assume it can be set only after add host test, and not in deployment scripts.</div><div>So might be needed to added as a test or step after the 'verify add host' step.</div><div><br></div><div>I'll open a ticket on it and we'll see who can assist with it.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div>Thanks,</div><div>michal</div><div><div class="h5"><div><br><blockquote type="cite"><div><div dir="auto"><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="m_-3080504837579530323quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
It does look like a race to me - does it reproduce?<br>
<div class="m_-3080504837579530323elided-text"><br>
> Destroy on the destination, which is weird and I don't understand why<br>
> the Destroy is invoked.<br>
><br>
> Arik, would you like to take a look? Maybe I overlooked something or<br>
> maybe there's a bug. The logs are at<br>
> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/2882/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-006_migrations.py/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/o<wbr>virt-system-tests_master_check<wbr>-patch-el7-x86_64/2882/artifac<wbr>t/exported-artifacts/basic-<wbr>suite-master__logs/test_logs/<wbr>basic-suite-master/post-006_<wbr>migrations.py/</a><br>
> and the interesting things happen around 2017-12-18 03:13:43,758-05.<br>
><br>
>> Can someone please have a look? Thanks.<br>
>><br>
>> As a side note, if indeed this is the cause for the failure for the<br>
>> job, it's misleading to say "migrate_vm: Success".<br>
>><br>
>> [1]<br>
>> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/2882/artifact/exported-artifacts/basic-suite-master__logs/006_migrations.py.junit.xml" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/o<wbr>virt-system-tests_master_check<wbr>-patch-el7-x86_64/2882/artifac<wbr>t/exported-artifacts/basic-<wbr>suite-master__logs/006_migrati<wbr>ons.py.junit.xml</a><br>
>><br>
>>><br>
>>> To view, visit change 85177. To unsubscribe, visit settings.<br>
>>><br>
>>> Gerrit-Project: ovirt-system-tests<br>
>>> Gerrit-Branch: master<br>
>>> Gerrit-MessageType: comment<br>
>>> Gerrit-Change-Id: I7eb386744a2a2faf0acd734e0ba44<wbr>be22dd590b5<br>
>>> Gerrit-Change-Number: 85177<br>
>>> Gerrit-PatchSet: 3<br>
>>> Gerrit-Owner: Yedidyah Bar David <<a href="mailto:didi@redhat.com" target="_blank">didi@redhat.com</a>><br>
>>> Gerrit-Reviewer: Dafna Ron <<a href="mailto:dron@redhat.com" target="_blank">dron@redhat.com</a>><br>
>>> Gerrit-Reviewer: Eyal Edri <<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>><br>
>>> Gerrit-Reviewer: Jenkins CI<br>
>>> Gerrit-Reviewer: Sandro Bonazzola <<a href="mailto:sbonazzo@redhat.com" target="_blank">sbonazzo@redhat.com</a>><br>
>>> Gerrit-Reviewer: Yedidyah Bar David <<a href="mailto:didi@redhat.com" target="_blank">didi@redhat.com</a>><br>
>>> Gerrit-Comment-Date: Mon, 18 Dec 2017 08:17:11 +0000<br>
>>> Gerrit-HasComments: No<br>
> ______________________________<wbr>_________________<br>
</div>> Devel mailing list<br>
> <a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/devel</a><br>
<div class="m_-3080504837579530323elided-text">><br>
><br>
<br>
______________________________<wbr>_________________<br>
Infra mailing list<br>
<a href="mailto:Infra@ovirt.org" target="_blank">Infra@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/infra" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/infra</a><br>
</div></blockquote></div><br></div></div></div>
</div></blockquote></div><br></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table border="0" style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: +972-9-7692018<br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div></div></div>
</div></div>