Re: [ovirt-devel] suspend_resume_vm fail on master experimental

On Wed, Jan 11, 2017 at 9:29 AM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: > > > On Wed, Jan 11, 2017 at 10:26 AM, Francesco Romani <fromani(a)redhat.com&gt; > wrote: >> >> Hi all >> >> >> On 01/11/2017 08:52 AM, Eyal Edri wrote: >> >> Adding Tomas from Virt. >> >> On Tue, Jan 10, 2017 at 10:54 AM, Piotr Kliczewski >> <piotr.kliczewski(a)gmail.com&gt; wrote: >>> >>> On Tue, Jan 10, 2017 at 9:29 AM, Daniel Belenky <dbelenky(a)redhat.com&gt; >>> wrote: >>> > Hi all, >>> > >>> > test-repo_ovirt_experimental_master (link to Jenkins) job failed on >>> > basic_sanity scenario. >>> > The job was triggered by https://gerrit.ovirt.org/#/c/69845/ >>> > >>> > From looking at the logs, it seems that the reason is VDSM. >>> > >>> > In the VDSM log, i see the following error: >>> > >>> > 2017-01-09 16:47:41,331 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] >>> > SSL >>> > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected >>> > ('::1', >>> > 34942, 0, 0) at 0x36b95f0>: unexpected eof (betterAsyncore:119) >> >> >> Daniel, could you please remind me the jenkins link? I see something >> suspicious on the Vdsm log. > > > Please use my live system: > ssh mini(a)ykaul-mini.tlv.redhat.com (qum5net) > then run a console to the VM: > lagocli --prefix-path /dev/shm/run/current shell engine > > (or 'host0' for the host) > >> Most notably, Vdsm received SIGTERM. Is this expected and part of the >> test? > > > It's not. > > >> >> >> >>> > >>> >>> This issue means that the client closed connection while vdsm was >>> replying. It can happen at any time >>> when the client is not nice with the connection. As you can see the >>> client connected locally '::1'. >>> >>> > >>> > Also, when looking at the MOM logs, I see the the following: >>> > >>> > 2017-01-09 16:43:39,508 - mom.vdsmInterface - ERROR - Cannot connect to >>> > VDSM! [Errno 111] Connection refused >>> > >>> >>> Looking at the log at this time vdsm had no open socket. >> >> >> >> Correct, but IIRC we have a race on startup - that's the reason why MOM >> retries to connect. After the new try, MOM seems to behave >> correctly: >> >> 2017-01-09 16:44:05,672 - mom.RPCServer - INFO - ping() >> 2017-01-09 16:44:05,673 - mom.RPCServer - INFO - getStatistics() > > > But there are multiple other disconnections, without anything in mom log: > > [root@lago-basic-suite-master-host0 vdsm]# grep SSL /var/log/vdsm/vdsm.log > |grep "::1" > 2017-01-11 02:29:46,310 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49046, 0, 0) at 0x39bf1b8>: unexpected eof (betterAsyncore:119) > 2017-01-11 02:29:51,089 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49048, 0, 0) at 0x39d1ea8>: unexpected eof (betterAsyncore:119) > 2017-01-11 02:29:51,392 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49050, 0, 0) at 0x39d1710>: unexpected eof (betterAsyncore:119) > 2017-01-11 02:29:51,700 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49052, 0, 0) at 0x39d1128>: unexpected eof (betterAsyncore:119) > 2017-01-11 02:29:52,008 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49054, 0, 0) at 0x39d1128>: unexpected eof (betterAsyncore:119) > 2017-01-11 02:37:34,019 ERROR (JsonRpc (StompReactor)) [vds.dispatcher] SSL > error receiving from <yajsonrpc.betterAsyncore.Dispatcher connected ('::1', > 49032, 0, 0) at 0x380cef0>: unexpected eof (betterAsyncore:119) > If there is no hosted engine this could be only mom. This is indication of connection closure.

> >> >> >> >> -- >> Francesco Romani >> Red Hat Engineering Virtualization R & D >> IRC: fromani >> >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > > > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel