--Apple-Mail=_97DB397B-9795-4701-8247-5CCB5C98A77F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8
On 5 Apr 2018, at 15:50, Sandro Bonazzola <sbonazzo@redhat.com> wrote: =20 Adding devel mailing list and some relevant people.
=20 2018-04-05 15:41 GMT+02:00 Simone Tiraboschi <stirabos@redhat.com = <mailto:stirabos@redhat.com>>: The first error is here: =20 =20 13:25:37 [ INFO ] TASK [Wait for the engine to come up on the target = VM] 13:37:01 [ ERROR ] fatal: [localhost]: FAILED! =3D> {"attempts": 120, = "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], = "delta": "0:00:00.316574", "end": "2018-04-05 09:37:00.826888", "rc": 0, = "start": "2018-04-05 09:37:00.510314", "stderr": "", "stderr_lines": [], = "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": =
should pass now with https://gerrit.ovirt.org/#/c/89873/ = <https://gerrit.ovirt.org/#/c/89873/> (under verification) Thanks, michal true, \"extra\": = \"metadata_parse_version=3D1\\nmetadata_feature_version=3D1\\ntimestamp=3D= 1838 (Thu Apr 5 09:36:51 = 2018)\\nhost-id=3D1\\nscore=3D0\\nvm_conf_refresh_time=3D1838 (Thu Apr = 5 09:36:51 = 2018)\\nconf_on_shared_storage=3DTrue\\nmaintenance=3DFalse\\nstate=3DEngi= neUnexpectedlyDown\\nstopped=3DFalse\\ntimeout=3DWed Dec 31 19:31:35 = 1969\\n\", \"hostname\": \"lago-he-basic-ansible-suite-master-host0\", = \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", = \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, = \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": = \"72e3e0e9\", \"local_conf_timestamp\": 1838, \"host-ts\": 1838}, = \"global_maintenance\": false}", "stdout_lines": ["{\"1\": = {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": = \"metadata_parse_version=3D1\\nmetadata_feature_version=3D1\\ntimestamp=3D= 1838 (Thu Apr 5 09:36:51 = 2018)\\nhost-id=3D1\\nscore=3D0\\nvm_conf_refresh_time=3D1838 (Thu Apr = 5 09:36:51 = 2018)\\nconf_on_shared_storage=3DTrue\\nmaintenance=3DFalse\\nstate=3DEngi= neUnexpectedlyDown\\nstopped=3DFalse\\ntimeout=3DWed Dec 31 19:31:35 = 1969\\n\", \"hostname\": \"lago-he-basic-ansible-suite-master-host0\", = \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", = \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, = \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": = \"72e3e0e9\", \"local_conf_timestamp\": 1838, \"host-ts\": 1838}, = \"global_maintenance\": false}"]}
13:37:01 [ ERROR ] Failed to execute stage 'Closing up': Failed = executing ansible-playbook =20 Anso so we have to check VDSM logs where we can find: 2018-04-05 09:27:01,083-0400 ERROR (vm/9bb2deab) [virt.vm] = (vmId=3D'9bb2deab-f686-4629-acee-90e2774a66b6') The vm start process = failed (vm:945) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 874, = in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2832, = in _run domxml =3D hooks.before_vm_start(self._buildDomainXML(), File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2258, = in _buildDomainXML return vmxml.format_xml(dom, pretty=3DTrue) File "/usr/lib/python2.7/site-packages/vdsm/virt/vmxml.py", line 71, = in format_xml stream, encoding=3D'utf-8', xml_declaration=3DTrue) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 820, in = write serialize(write, self._root, encoding, qnames, namespaces) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 939, in = _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 939, in = _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 939, in = _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 932, in = _serialize_xml v =3D _escape_attrib(v, encoding) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1092, in = _escape_attrib _raise_serialization_error(text) File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1052, in = _raise_serialization_error "cannot serialize %r (type %s)" % (text, type(text).__name__) TypeError: cannot serialize 0 (type int) 2018-04-05 09:27:01,084-0400 INFO (vm/9bb2deab) [virt.vm] = (vmId=3D'9bb2deab-f686-4629-acee-90e2774a66b6') Changed state to Down: = cannot serialize 0 (type int) (code=3D1) (vm:1685) =20 It seams another regression with the libvirt XML generated by the = engine. =20 =20 =20 On Thu, Apr 5, 2018 at 3:37 PM, <jenkins@jenkins.phx.ovirt.org = <mailto:jenkins@jenkins.phx.ovirt.org>> wrote: Project: = http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mas= ter/ = <http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-ma= ster/> Build: = http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mas= ter/143/ = <http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-ma= ster/143/> Build Number: 143 Build Status: Still Failing Triggered By: Started by user Simone Tiraboschi =20 ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #139 [Gal Ben Haim] he: Share control.sh between all the non-ansible suites =20 [Daniel Belenky] Workaround for a race in gerrit-trigger plugin =20 =20 Changes for Build #140 [Gal Ben Haim] he: Share control.sh between all the non-ansible suites =20 =20 Changes for Build #141 [Gal Ben Haim] he: Share control.sh between all the non-ansible suites =20 =20 Changes for Build #142 [Yaniv Kaul] Disable 006_migrations.prepare_migration_attachments_ipv6 =20 =20 Changes for Build #143 [Yaniv Kaul] Disable 006_migrations.prepare_migration_attachments_ipv6 =20 =20 =20 =20 ----------------- Failed Tests: ----------------- No tests ran. =20 =20 =20 =20 --=20 SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat=C2=A0EMEA <https://www.redhat.com/> sbonazzo@redhat.com <mailto:sbonazzo@redhat.com> =20 =20 <https://red.ht/sig>=09 <https://redhat.com/summit>
--Apple-Mail=_97DB397B-9795-4701-8247-5CCB5C98A77F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On 5 Apr 2018, at 15:50, Sandro Bonazzola <<a = href=3D"mailto:sbonazzo@redhat.com" class=3D"">sbonazzo@redhat.com</a>>= wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"ltr" style=3D"font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant-caps: normal; font-weight: normal; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px;" class=3D"">Adding devel mailing list = and some relevant people.</div></div></blockquote><div><br = class=3D""></div>should pass now with <a = href=3D"https://gerrit.ovirt.org/#/c/89873/" = class=3D"">https://gerrit.ovirt.org/#/c/89873/</a> (under = verification)</div><div><br = class=3D""></div><div>Thanks,</div><div>michal</div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = class=3D"gmail_extra" style=3D"font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant-caps: normal; font-weight: normal; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px;"><br class=3D""><div = class=3D"gmail_quote">2018-04-05 15:41 GMT+02:00 Simone Tiraboschi<span = class=3D"Apple-converted-space"> </span><span dir=3D"ltr" = class=3D""><<a href=3D"mailto:stirabos@redhat.com" target=3D"_blank" = class=3D"">stirabos@redhat.com</a>></span>:<br class=3D""><blockquote = class=3D"gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; = border-left-width: 1px; border-left-style: solid; border-left-color: = rgb(204, 204, 204); padding-left: 1ex;"><div dir=3D"ltr" class=3D"">The = first error is here:<div class=3D""><br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D""><pre = class=3D"m_410505539926204415gmail-console-output" style=3D"box-sizing: = border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; = color: rgb(51, 51, 51); font-size: 14px; font-style: normal; = font-variant-ligatures: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; word-spacing: 0px;"><span = class=3D"m_410505539926204415gmail-timestamp" style=3D"box-sizing: = border-box;"><b style=3D"box-sizing: border-box;" class=3D"">13:25:37</b> = </span>[ INFO ] TASK [Wait for the engine to come up on the target VM] <span class=3D"m_410505539926204415gmail-timestamp" style=3D"box-sizing: = border-box;"><b style=3D"box-sizing: border-box;" class=3D"">13:37:01</b> = </span>[ ERROR ] fatal: [localhost]: FAILED! =3D> {"attempts": 120, = "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], = "delta": "0:00:00.316574", "end": "2018-04-05 09:37:00.826888", "rc": 0, = "start": "2018-04-05 09:37:00.510314", "stderr": "", "stderr_lines": [], = "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": = true, \"extra\": \"metadata_parse_version=3D1\\<wbr = class=3D"">nmetadata_feature_version=3D1\\<wbr class=3D"">ntimestamp=3D183= 8 (Thu Apr 5 09:36:51 2018)\\nhost-id=3D1\\nscore=3D0\\<wbr = class=3D"">nvm_conf_refresh_time=3D1838 (Thu Apr 5 09:36:51 = 2018)\\nconf_on_shared_<wbr class=3D"">storage=3DTrue\\nmaintenance=3D<wbr= class=3D"">False\\nstate=3D<wbr class=3D"">EngineUnexpectedlyDown\\<wbr = class=3D"">nstopped=3DFalse\\ntimeout=3DWed Dec 31 19:31:35 1969\\n\", = \"hostname\": \"lago-he-basic-ansible-suite-<wbr = class=3D"">master-host0\", \"host-id\": 1, \"engine-status\": = {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": = \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": = false, \"maintenance\": false, \"crc32\": \"72e3e0e9\", = \"local_conf_timestamp\": 1838, \"host-ts\": 1838}, = \"global_maintenance\": false}", "stdout_lines": ["{\"1\": = {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": = \"metadata_parse_version=3D1\\<wbr = class=3D"">nmetadata_feature_version=3D1\\<wbr class=3D"">ntimestamp=3D183= 8 (Thu Apr 5 09:36:51 2018)\\nhost-id=3D1\\nscore=3D0\\<wbr = class=3D"">nvm_conf_refresh_time=3D1838 (Thu Apr 5 09:36:51 = 2018)\\nconf_on_shared_<wbr class=3D"">storage=3DTrue\\nmaintenance=3D<wbr= class=3D"">False\\nstate=3D<wbr class=3D"">EngineUnexpectedlyDown\\<wbr = class=3D"">nstopped=3DFalse\\ntimeout=3DWed Dec 31 19:31:35 1969\\n\", = \"hostname\": \"lago-he-basic-ansible-suite-<wbr = class=3D"">master-host0\", \"host-id\": 1, \"engine-status\": = {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": = \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": = false, \"maintenance\": false, \"crc32\": \"72e3e0e9\", = \"local_conf_timestamp\": 1838, \"host-ts\": 1838}, = \"global_maintenance\": false}"]} <span class=3D"m_410505539926204415gmail-timestamp" style=3D"box-sizing: = border-box;"><b style=3D"box-sizing: border-box;" class=3D"">13:37:01</b> = </span>[ ERROR ] Failed to execute stage 'Closing up': Failed executing = ansible-playbook</pre><div class=3D""><br class=3D""></div><div = class=3D"">Anso so we have to check VDSM logs where we can = find:</div><div class=3D""><pre style=3D"font-style: normal; = font-variant-ligatures: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; word-spacing: 0px;" class=3D"">2018-04-05 = 09:27:01,083-0400 ERROR (vm/9bb2deab) [virt.vm] = (vmId=3D'9bb2deab-f686-4629-<wbr class=3D"">acee-90e2774a66b6') The vm = start process failed (vm:945) Traceback (most recent call last): File "/usr/lib/python2.7/site-<wbr class=3D"">packages/vdsm/virt/vm.py",= line 874, in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-<wbr class=3D"">packages/vdsm/virt/vm.py",= line 2832, in _run domxml =3D hooks.before_vm_start(self._<wbr = class=3D"">buildDomainXML(), File "/usr/lib/python2.7/site-<wbr class=3D"">packages/vdsm/virt/vm.py",= line 2258, in _buildDomainXML return vmxml.format_xml(dom, pretty=3DTrue) File "/usr/lib/python2.7/site-<wbr = class=3D"">packages/vdsm/virt/vmxml.py", line 71, in format_xml stream, encoding=3D'utf-8', xml_declaration=3DTrue) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 820, in write serialize(write, self._root, encoding, qnames, namespaces) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 939, in _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 939, in _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 939, in _serialize_xml _serialize_xml(write, e, encoding, qnames, None) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 932, in _serialize_xml v =3D _escape_attrib(v, encoding) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 1092, in _escape_attrib _raise_serialization_error(<wbr class=3D"">text) File "/usr/lib64/python2.7/xml/<wbr class=3D"">etree/ElementTree.py", = line 1052, in _raise_serialization_error "cannot serialize %r (type %s)" % (text, type(text).__name__) TypeError: cannot serialize 0 (type int) 2018-04-05 09:27:01,084-0400 INFO (vm/9bb2deab) [virt.vm] = (vmId=3D'9bb2deab-f686-4629-<wbr class=3D"">acee-90e2774a66b6') Changed = state to Down: cannot serialize 0 (type int) (code=3D1) = (vm:1685)</pre><br class=3D""></div><div class=3D"">It seams another = regression with the libvirt XML generated by the engine.</div><div = class=3D""><div class=3D"h5"><div class=3D""><br class=3D""></div><br = class=3D""><div class=3D"gmail_extra"><br class=3D""><div = class=3D"gmail_quote">On Thu, Apr 5, 2018 at 3:37 PM,<span = class=3D"Apple-converted-space"> </span><span = class=3D"Apple-converted-space"> </span><span dir=3D"ltr" = class=3D""><<a href=3D"mailto:jenkins@jenkins.phx.ovirt.org" = target=3D"_blank" class=3D"">jenkins@jenkins.phx.ovirt.org</a><wbr = class=3D"">></span><span = class=3D"Apple-converted-space"> </span>wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Project:<span = class=3D"Apple-converted-space"> </span><a = href=3D"http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-s= uite-master/" rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://jenkins.ovirt.org/job/o<wbr = class=3D"">virt-system-tests_he-basic-ans<wbr = class=3D"">ible-suite-master/</a><br class=3D"">Build:<span = class=3D"Apple-converted-space"> </span><a = href=3D"http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-s= uite-master/143/" rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://jenkins.ovirt.org/job/o<wbr = class=3D"">virt-system-tests_he-basic-ans<wbr = class=3D"">ible-suite-master/143/</a><br class=3D"">Build Number: 143<br = class=3D"">Build Status: Still Failing<br class=3D"">Triggered By: = Started by user Simone Tiraboschi<br class=3D""><br = class=3D"">------------------------------<wbr class=3D"">-------<br = class=3D"">Changes Since Last Success:<br = class=3D"">------------------------------<wbr class=3D"">-------<br = class=3D"">Changes for Build #139<br class=3D"">[Gal Ben Haim] he: Share = control.sh between all the non-ansible suites<br class=3D""><br = class=3D"">[Daniel Belenky] Workaround for a race in gerrit-trigger = plugin<br class=3D""><br class=3D""><br class=3D"">Changes for Build = #140<br class=3D"">[Gal Ben Haim] he: Share control.sh between all the = non-ansible suites<br class=3D""><br class=3D""><br class=3D"">Changes = for Build #141<br class=3D"">[Gal Ben Haim] he: Share control.sh between = all the non-ansible suites<br class=3D""><br class=3D""><br = class=3D"">Changes for Build #142<br class=3D"">[Yaniv Kaul] Disable = 006_migrations.prepare_migrati<wbr class=3D"">on_attachments_ipv6<br = class=3D""><br class=3D""><br class=3D"">Changes for Build #143<br = class=3D"">[Yaniv Kaul] Disable 006_migrations.prepare_migrati<wbr = class=3D"">on_attachments_ipv6<br class=3D""><br class=3D""><br = class=3D""><br class=3D""><br class=3D"">-----------------<br = class=3D"">Failed Tests:<br class=3D"">-----------------<br class=3D"">No = tests ran.</blockquote></div><br = class=3D""></div></div></div></div></div></blockquote></div><br = class=3D""><br clear=3D"all" class=3D""><div class=3D""><br = class=3D""></div>--<span class=3D"Apple-converted-space"> </span><br = class=3D""><div class=3D"gmail_signature" = data-smartmail=3D"gmail_signature"><div dir=3D"ltr" class=3D""><div = class=3D""><div dir=3D"ltr" class=3D""><div class=3D""><div dir=3D"ltr" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D""><div dir=3D"ltr" class=3D""><div class=3D""><div dir=3D"ltr" = class=3D""><div dir=3D"ltr" class=3D""><div dir=3D"ltr" class=3D""><div = dir=3D"ltr" class=3D""><div style=3D"font-family: overpass, sans-serif; = font-weight: bold; margin: 0px; padding: 0px; font-size: 14px; = text-transform: uppercase;" class=3D""><span = class=3D"">SANDRO</span> <span class=3D"">BONAZZOLA</span></div><p = style=3D"font-family: overpass, sans-serif; font-size: 10px; margin: 0px = 0px 4px; text-transform: uppercase;" class=3D""><span class=3D"">ASSOCIATE= MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION = R&D</span><span style=3D"color: rgb(170, 170, 170); margin: 0px;" = class=3D""></span></p><div style=3D"font-family: overpass, sans-serif; = margin: 0px; font-size: 10px; color: rgb(153, 153, 153);" class=3D""><a = href=3D"https://www.redhat.com/" target=3D"_blank" style=3D"color: = rgb(0, 136, 206); margin: 0px;" class=3D"">Red Hat <span = class=3D"">EMEA</span></a></div><p style=3D"font-family: overpass, = sans-serif; margin: 0px 0px 6px; font-size: 10px; color: rgb(153, 153, = 153);" class=3D""><span style=3D"margin: 0px; padding: 0px;" class=3D""><a= href=3D"mailto:sbonazzo@redhat.com" target=3D"_blank" style=3D"color: = rgb(0, 136, 206); margin: 0px;" class=3D"">sbonazzo@redhat.com</a> = </span></p><table border=3D"0" style=3D"font-family: overpass, = sans-serif; font-size: inherit;" class=3D""><tbody class=3D""><tr = class=3D""><td width=3D"100px" class=3D""><a href=3D"https://red.ht/sig" = target=3D"_blank" class=3D""><img = src=3D"https://www.redhat.com/files/brand/email/sig-redhat.png" = width=3D"90" height=3D"auto" class=3D""></a></td><td style=3D"font-size: = 10px;" class=3D""><div class=3D""><a href=3D"https://redhat.com/summit" = target=3D"_blank" class=3D""><img width=3D"70px" height=3D"auto" = src=3D"https://www.redhat.com/files/brand/email/sig-summit.png" = style=3D"outline: 0px;" = class=3D""></a></div></td></tr></tbody></table></div></div></div></div></d= iv></div></div></div></div></div></div></div></div></div></div></div></div=
</blockquote></div><br class=3D""></body></html>=
--Apple-Mail=_97DB397B-9795-4701-8247-5CCB5C98A77F--