<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 11, 2017 at 10:26 AM, Francesco Romani <span dir="ltr"><<a href="mailto:fromani@redhat.com" target="_blank">fromani@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi all<br>
</p><span class="gmail-">
<br>
<div class="gmail-m_-5785517619226444427moz-cite-prefix">On 01/11/2017 08:52 AM, Eyal Edri
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Adding Tomas from Virt.</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jan 10, 2017 at 10:54 AM, Piotr
Kliczewski <span dir="ltr"><<a href="mailto:piotr.kliczewski@gmail.com" target="_blank">piotr.kliczewski@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On Tue, Jan 10, 2017 at 9:29 AM, Daniel Belenky
<<a href="mailto:dbelenky@redhat.com" target="_blank">dbelenky@redhat.com</a>>
wrote:<br>
> Hi all,<br>
><br>
> test-repo_ovirt_experimental_m<wbr>aster (link to
Jenkins) job failed on<br>
> basic_sanity scenario.<br>
> The job was triggered by <a href="https://gerrit.ovirt.org/#/c/69845/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/6<wbr>9845/</a><br>
><br>
> From looking at the logs, it seems that the reason is
VDSM.<br>
><br>
> In the VDSM log, i see the following error:<br>
><br>
> 2017-01-09 16:47:41,331 ERROR (JsonRpc
(StompReactor)) [vds.dispatcher] SSL<br>
> error receiving from <yajsonrpc.betterAsyncore.Disp<wbr>atcher
connected ('::1',<br>
> 34942, 0, 0) at 0x36b95f0>: unexpected eof
(betterAsyncore:119)<br>
</span></blockquote>
</div>
</div>
</blockquote>
<br></span>
Daniel, could you please remind me the jenkins link? I see something
suspicious on the Vdsm log.<br></div></blockquote><div><br></div><div>Please use my live system:</div><div style="font-size:12.8px">ssh <a href="mailto:mini@ykaul-mini.tlv.redhat.com" target="_blank"><span class="gmail-m_1130033870917217609gmail-m_-7038708660695866355gmail-il"><span class="gmail-m_1130033870917217609gmail-il">mini</span></span>@<span class="gmail-m_1130033870917217609gmail-m_-7038708660695866355gmail-il">ykaul</span>-<span class="gmail-m_1130033870917217609gmail-m_-7038708660695866355gmail-il"><span class="gmail-m_1130033870917217609gmail-il">mini</span></span>.tlv.redhat<wbr>.com</a> (qum5net)</div><div style="font-size:12.8px">then run a console to the VM:</div><div><span style="font-size:12.8px">lagocli --prefix-path /dev/shm/run/current shell engine</span> </div><div><br></div><div>(or 'host0' for the host)</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
Most notably, Vdsm received SIGTERM. Is this expected and part of
the test?</div></blockquote><div><br></div><div>It's not.</div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><span class="gmail-"><br>
<br>
<blockquote type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
><br>
<br>
</span>This issue means that the client closed connection
while vdsm was<br>
replying. It can happen at any time<br>
when the client is not nice with the connection. As you can
see the<br>
client connected locally '::1'.<br>
<span><br>
><br>
> Also, when looking at the MOM logs, I see the the
following:<br>
><br>
> 2017-01-09 16:43:39,508 - mom.vdsmInterface - ERROR -
Cannot connect to<br>
> VDSM! [Errno 111] Connection refused<br>
><br>
<br>
</span>Looking at the log at this time vdsm had no open
socket.</blockquote>
</div>
</div>
<br>
</blockquote>
<br></span>
Correct, but IIRC we have a race on startup - that's the reason why
MOM retries to connect. After the new try, MOM seems to behave<br>
correctly:<br>
<br>
2017-01-09 16:44:05,672 - mom.RPCServer - INFO - ping()<br>
2017-01-09 16:44:05,673 - mom.RPCServer - INFO - getStatistics()</div></blockquote><div><br></div><div>But there are multiple other disconnections, without anything in mom log:</div><div><br></div><div>[root@lago-basic-suite-master-host0 vdsm]# grep SSL /var/log/vdsm/vdsm.log |grep "::1"</div><div>2017-01-11 02:29:46,310 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49046, 0, 0) at 0x39bf1b8>: unexpected eof (betterAsyncore:119)</div><div>2017-01-11 02:29:51,089 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49048, 0, 0) at 0x39d1ea8>: unexpected eof (betterAsyncore:119)</div><div>2017-01-11 02:29:51,392 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49050, 0, 0) at 0x39d1710>: unexpected eof (betterAsyncore:119)</div><div>2017-01-11 02:29:51,700 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49052, 0, 0) at 0x39d1128>: unexpected eof (betterAsyncore:119)</div><div>2017-01-11 02:29:52,008 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49054, 0, 0) at 0x39d1128>: unexpected eof (betterAsyncore:119)</div><div>2017-01-11 02:37:34,019 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 49032, 0, 0) at 0x380cef0>: unexpected eof (betterAsyncore:119)</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><span class="gmail-HOEnZb"><font color="#888888"><br>
<br>
<pre class="gmail-m_-5785517619226444427moz-signature" cols="72">--
Francesco Romani
Red Hat Engineering Virtualization R & D
IRC: fromani</pre>
</font></span></div>
<br>______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/devel</a><br></blockquote></div><br></div></div>