Re: [ovirt-devel] Update: HC suites failing for 3 weeks ( was: [OST][HC] HE fails to deploy )

On Wed, Apr 25, 2018 at 3:54 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On Mon, Apr 23, 2018 at 6:28 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > >> >> On Mon, Apr 23, 2018 at 5:41 PM, Eyal Edri <eedri(a)redhat.com&gt; wrote: >> >>> Sahina, >>> Any update on this? >>> >> >> Sorry, haven't been able to spend any time on this. The last I checked >> the HE install was failing at task - Get Local VM IP. >> and there were no logs from HE VM to debug. >> >> Will spend sometime on this tomorrow >> > > https://gerrit.ovirt.org/#/c/89953/ - fixes the issue, atleast when I > tried this on my local setup. > The CI however still fails in the HE install with : TASK [Get local VM IP]", "[ ERROR ] fatal: [localhost]: FAILED! => {\"attempts\": 50, \"changed\": true, \"cmd\": \"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'\", \"delta\": \"0:00:00.043961\", \"end\": \"2018-04-25 05:51:34.226374\", \"rc\": 0, \"start\": \"2018-04-25 05:51:34.182413\", \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"\", \"stdout_lines\": []}" FWIW, my local setup , ost repo was at I3fc2976ab2400e5908760aadc3258 329c0ffdf4d

> > >> >>> On Wed, Apr 18, 2018 at 3:40 PM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >>> wrote: >>> >>>> >>>> >>>> 2018-04-18 9:37 GMT+02:00 Eyal Edri <eedri(a)redhat.com&gt;: >>>> >>>>> FYI, >>>>> >>>>> I've disabled the 4.2 and master HC suites nightly run on CI as they >>>>> are constantly failing for almost 3 weeks and spamming the mailing lists. >>>>> >>>> >>>> >>>> HC uses gdeploy 2.0.6 which was released in December and was based on >>>> ansible 2.4. >>>> ansible-2.5 landed 3 weeks ago in EPEL, my guess is that gdeploy is >>>> not supporting ansible-2.5 properly. >>>> I had no time to validate my guess with proof, so please Sahina cross >>>> check this. >>>> >>>> >>>> >>>>> >>>>> I think this should get higher priority for a fix if we want it to >>>>> provide any value, >>>>> Work can continue using the manual jobs or via check-patch. >>>>> >>>>> >>>>> On Mon, Apr 16, 2018 at 10:56 AM, Gal Ben Haim <gbenhaim(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Any update on https://gerrit.ovirt.org/#/c/88887/ ? >>>>>> The HC suites still failing and it's hard to understand why without >>>>>> the logs from the engine VM. >>>>>> >>>>>> On Sat, Apr 7, 2018 at 7:19 AM, Sahina Bose <sabose(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Apr 6, 2018 at 1:10 PM, Simone Tiraboschi < >>>>>>> stirabos(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Apr 6, 2018 at 9:28 AM, Sahina Bose <sabose(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> >>>>>>>>> 2018-04-05 20:46:52,773-0400 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:100 TASK [Get local VM IP] >>>>>>>>> 2018-04-05 20:55:28,217-0400 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:94 {u'_ansible_parsed': True, u'stderr_lines': [], u'cmd': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'end': u'2018-04-05 20:55:28.046320', u'_ansible_no_log': False, u'stdout': u'', u'changed': True, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'_raw_params': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'start': u'2018-04-05 20:55:28.000470', u'attempts': 50, u'stderr': u'', u'rc': 0, u'delta': u'0:00:00.045850', u'stdout_lines': []} >>>>>>>>> 2018-04-05 20:55:28,318-0400 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:98 fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.045850", "end": "2018-04-05 20:55:28.046320", "rc": 0, "start": "2018-04-05 20:55:28.000470", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} >>>>>>>>> >>>>>>>>> Both the 4.2 and master suites are failing on getting local VM IP. >>>>>>>>> Any idea what changed or if I have to change the test? >>>>>>>>> >>>>>>>>> thanks! >>>>>>>>> >>>>>>>> >>>>>>>> Hi Sahina, >>>>>>>> 4.2 and master suite non HC are correctly running this morning. >>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>>>>>>> rt-system-tests_he-basic-ansible-suite-master/146/ >>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>>>>>>> rt-system-tests_he-basic-ansible-suite-4.2/76/ >>>>>>>> >>>>>>>> I'll try to check the difference with HC suites. >>>>>>>> >>>>>>>> Are you using more than one subnet in the HC suites? >>>>>>>> >>>>>>> >>>>>>> No, I'm not. And we havent's changed anything related to network in >>>>>>> the test suite. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *GAL bEN HAIM* >>>>>> RHV DEVOPS >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Eyal edri >>>>> >>>>> >>>>> MANAGER >>>>> >>>>> RHV DevOps >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> >>>>> >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>> <https://redhat.com/trusted> >>>>> phone: +972-9-7692018 >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> SANDRO BONAZZOLA >>>> >>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >>>> >>>> Red Hat EMEA <https://www.redhat.com/> >>>> >>>> sbonazzo(a)redhat.com >>>> <https://red.ht/sig> >>>> <https://redhat.com/summit> >>>> >>> >>> >>> >>> -- >>> >>> Eyal edri >>> >>> >>> MANAGER >>> >>> RHV DevOps >>> >>> EMEA VIRTUALIZATION R&D >>> >>> >>> Red Hat EMEA <https://www.redhat.com/> >>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>> <https://redhat.com/trusted> >>> phone: +972-9-7692018 >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> >> >> >