<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 24, 2018 at 10:29 AM, Dan Kenigsberg <span dir="ltr"><<a href="mailto:danken@redhat.com" target="_blank">danken@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tue, Apr 24, 2018 at 5:09 PM, Ravi Shankar Nori <<a href="mailto:rnori@redhat.com">rnori@redhat.com</a>> wrote:<br>
><br>
><br>
> On Tue, Apr 24, 2018 at 9:47 AM, Dan Kenigsberg <<a href="mailto:danken@redhat.com">danken@redhat.com</a>> wrote:<br>
>><br>
>> On Tue, Apr 24, 2018 at 4:36 PM, Ravi Shankar Nori <<a href="mailto:rnori@redhat.com">rnori@redhat.com</a>><br>
>> wrote:<br>
>> ><br>
>> ><br>
>> > On Tue, Apr 24, 2018 at 9:24 AM, Martin Perina <<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >><br>
>> >><br>
>> >> On Tue, Apr 24, 2018 at 3:17 PM, Ravi Shankar Nori <<a href="mailto:rnori@redhat.com">rnori@redhat.com</a>><br>
>> >> wrote:<br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> On Tue, Apr 24, 2018 at 7:00 AM, Dan Kenigsberg <<a href="mailto:danken@redhat.com">danken@redhat.com</a>><br>
>> >>> wrote:<br>
>> >>>><br>
>> >>>> Ravi's patch is in, but a similar problem remains, and the test<br>
>> >>>> cannot<br>
>> >>>> be put back into its place.<br>
>> >>>><br>
>> >>>> It seems that while Vdsm was taken down, a couple of getCapsAsync<br>
>> >>>> requests queued up. At one point, the host resumed its connection,<br>
>> >>>> before the requests have been cleared of the queue. After the host is<br>
>> >>>> up, the following tests resume, and at a pseudorandom point in time,<br>
>> >>>> an old getCapsAsync request times out and kills our connection.<br>
>> >>>><br>
>> >>>> I believe that as long as ANY request is on flight, the monitoring<br>
>> >>>> lock should not be released, and the host should not be declared as<br>
>> >>>> up.<br>
>><br>
>> Would you relate to this analysis ^^^ ?<br>
>><br>
><br>
> The HostMonitoring lock issue has been fixed by<br>
> <a href="https://gerrit.ovirt.org/#/c/90189/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>90189/</a><br>
<br>
</span>Is there still a chance that a host moves to Up while former<br>
getCapsAsync request are still in-flight?<br>
<div class="HOEnZb"><div class="h5"><br></div></div></blockquote><div><br></div><div>Should not happen. Is there a way to execute/reproduce the failing test on Dev env?<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">
><br>
>><br>
>> >>>><br>
>> >>>><br>
>> >>><br>
>> >>><br>
>> >>> Hi Dan,<br>
>> >>><br>
>> >>> Can I have the link to the job on jenkins so I can look at the logs<br>
>> >><br>
>> >><br>
>> >><br>
>> >> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/346/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>ovirt-system-tests_standard-<wbr>check-patch/346/</a><br>
>> >><br>
>> ><br>
>> ><br>
>> > From the logs the only VDS lock that is being released twice is<br>
>> > VDS_FENCE<br>
>> > lock. Opened a BZ [1] for it. Will post a fix<br>
>> ><br>
>> > [1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1571300" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/<wbr>show_bug.cgi?id=1571300</a><br>
>><br>
>> Can this possibly cause a surprise termination of host connection?<br>
><br>
><br>
> Not sure, from the logs VDS_FENCE is the only other VDS lock that is being<br>
> released<br>
</div></div></blockquote></div><br></div></div>