Re: [ovirt-devel] Update: HC suites failing for 3 weeks ( was: [OST][HC] HE fails to deploy )

On Wed, Apr 25, 2018 at 1:53 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On Wed, Apr 25, 2018 at 3:54 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > >> >> >> On Mon, Apr 23, 2018 at 6:28 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >> >>> >>> On Mon, Apr 23, 2018 at 5:41 PM, Eyal Edri <eedri(a)redhat.com&gt; wrote: >>> >>>> Sahina, >>>> Any update on this? >>>> >>> >>> Sorry, haven't been able to spend any time on this. The last I checked >>> the HE install was failing at task - Get Local VM IP. >>> and there were no logs from HE VM to debug. >>> >>> Will spend sometime on this tomorrow >>> >> >> https://gerrit.ovirt.org/#/c/89953/ - fixes the issue, atleast when I >> tried this on my local setup. >> > > > The CI however still fails in the HE install with : > > TASK [Get local VM IP]", "[ ERROR ] fatal: [localhost]: FAILED! => {\"attempts\": 50, \"changed\": true, \"cmd\": \"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'\", \"delta\": \"0:00:00.043961\", \"end\": \"2018-04-25 05:51:34.226374\", \"rc\": 0, \"start\": \"2018-04-25 05:51:34.182413\", \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"\", \"stdout_lines\": []}" > > > > FWIW, my local setup , ost repo was at I3fc2976ab2400e5908760aadc3258 > 329c0ffdf4d > Any update? suites are still failing.

> > >> >> >>> >>>> On Wed, Apr 18, 2018 at 3:40 PM, Sandro Bonazzola <sbonazzo(a)redhat.com >>>> > wrote: >>>> >>>>> >>>>> >>>>> 2018-04-18 9:37 GMT+02:00 Eyal Edri <eedri(a)redhat.com&gt;: >>>>> >>>>>> FYI, >>>>>> >>>>>> I've disabled the 4.2 and master HC suites nightly run on CI as they >>>>>> are constantly failing for almost 3 weeks and spamming the mailing lists. >>>>>> >>>>> >>>>> >>>>> HC uses gdeploy 2.0.6 which was released in December and was based on >>>>> ansible 2.4. >>>>> ansible-2.5 landed 3 weeks ago in EPEL, my guess is that gdeploy is >>>>> not supporting ansible-2.5 properly. >>>>> I had no time to validate my guess with proof, so please Sahina cross >>>>> check this. >>>>> >>>>> >>>>> >>>>>> >>>>>> I think this should get higher priority for a fix if we want it to >>>>>> provide any value, >>>>>> Work can continue using the manual jobs or via check-patch. >>>>>> >>>>>> >>>>>> On Mon, Apr 16, 2018 at 10:56 AM, Gal Ben Haim <gbenhaim(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Any update on https://gerrit.ovirt.org/#/c/88887/ ? >>>>>>> The HC suites still failing and it's hard to understand why without >>>>>>> the logs from the engine VM. >>>>>>> >>>>>>> On Sat, Apr 7, 2018 at 7:19 AM, Sahina Bose <sabose(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Apr 6, 2018 at 1:10 PM, Simone Tiraboschi < >>>>>>>> stirabos(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Apr 6, 2018 at 9:28 AM, Sahina Bose <sabose(a)redhat.com&gt; >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> 2018-04-05 20:46:52,773-0400 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:100 TASK [Get local VM IP] >>>>>>>>>> 2018-04-05 20:55:28,217-0400 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:94 {u'_ansible_parsed': True, u'stderr_lines': [], u'cmd': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'end': u'2018-04-05 20:55:28.046320', u'_ansible_no_log': False, u'stdout': u'', u'changed': True, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'_raw_params': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'start': u'2018-04-05 20:55:28.000470', u'attempts': 50, u'stderr': u'', u'rc': 0, u'delta': u'0:00:00.045850', u'stdout_lines': []} >>>>>>>>>> 2018-04-05 20:55:28,318-0400 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:98 fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.045850", "end": "2018-04-05 20:55:28.046320", "rc": 0, "start": "2018-04-05 20:55:28.000470", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} >>>>>>>>>> >>>>>>>>>> Both the 4.2 and master suites are failing on getting local VM >>>>>>>>>> IP. >>>>>>>>>> Any idea what changed or if I have to change the test? >>>>>>>>>> >>>>>>>>>> thanks! >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Sahina, >>>>>>>>> 4.2 and master suite non HC are correctly running this morning. >>>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>>>>>>>> rt-system-tests_he-basic-ansible-suite-master/146/ >>>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>>>>>>>> rt-system-tests_he-basic-ansible-suite-4.2/76/ >>>>>>>>> >>>>>>>>> I'll try to check the difference with HC suites. >>>>>>>>> >>>>>>>>> Are you using more than one subnet in the HC suites? >>>>>>>>> >>>>>>>> >>>>>>>> No, I'm not. And we havent's changed anything related to network >>>>>>>> in the test suite. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *GAL bEN HAIM* >>>>>>> RHV DEVOPS >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Eyal edri >>>>>> >>>>>> >>>>>> MANAGER >>>>>> >>>>>> RHV DevOps >>>>>> >>>>>> EMEA VIRTUALIZATION R&D >>>>>> >>>>>> >>>>>> Red Hat EMEA <https://www.redhat.com/> >>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>>> <https://redhat.com/trusted> >>>>>> phone: +972-9-7692018 >>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> SANDRO BONAZZOLA >>>>> >>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >>>>> >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> >>>>> sbonazzo(a)redhat.com >>>>> <https://red.ht/sig> >>>>> <https://redhat.com/summit> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Eyal edri >>>> >>>> >>>> MANAGER >>>> >>>> RHV DevOps >>>> >>>> EMEA VIRTUALIZATION R&D >>>> >>>> >>>> Red Hat EMEA <https://www.redhat.com/> >>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>> <https://redhat.com/trusted> >>>> phone: +972-9-7692018 >>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>> >>> >>> >> > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)