[ovirt-devel] Update: HC suites failing for 3 weeks ( was: [OST][HC] HE fails to deploy )

Sahina Bose sabose at redhat.com
Mon Apr 30 12:59:09 UTC 2018 

On Mon, Apr 30, 2018 at 5:45 PM, Eyal Edri <eedri at redhat.com> wrote:

>
>
> On Wed, Apr 25, 2018 at 1:53 PM, Sahina Bose <sabose at redhat.com> wrote:
>
>>
>>
>> On Wed, Apr 25, 2018 at 3:54 PM, Sahina Bose <sabose at redhat.com> wrote:
>>
>>>
>>>
>>> On Mon, Apr 23, 2018 at 6:28 PM, Sahina Bose <sabose at redhat.com> wrote:
>>>
>>>>
>>>> On Mon, Apr 23, 2018 at 5:41 PM, Eyal Edri <eedri at redhat.com> wrote:
>>>>
>>>>> Sahina,
>>>>> Any update on this?
>>>>>
>>>>
>>>> Sorry, haven't been able to spend any time on this. The last I checked
>>>> the  HE install was failing at task - Get Local VM IP.
>>>> and there were no logs from HE VM to debug.
>>>>
>>>> Will spend sometime on this tomorrow
>>>>
>>>
>>> https://gerrit.ovirt.org/#/c/89953/ - fixes the issue, atleast when I
>>> tried this on my local setup.
>>>
>>
>>
>> The CI however still fails in the HE install with :
>>
>> TASK [Get local VM IP]", "[ ERROR ] fatal: [localhost]: FAILED! => {\"attempts\": 50, \"changed\": true, \"cmd\": \"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'\", \"delta\": \"0:00:00.043961\", \"end\": \"2018-04-25 05:51:34.226374\", \"rc\": 0, \"start\": \"2018-04-25 05:51:34.182413\", \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"\", \"stdout_lines\": []}"
>>
>>
>>
>> FWIW, my local setup , ost repo was at I3fc2976ab2400e5908760aadc3258
>> 329c0ffdf4d
>>
>
>
> Any update? suites are still failing.
>

The suites work locally, on the CI systems they fail with above error.
Unfortunately, no clue as to why this is so.
With the current run, are we able to get logs from the engine VM?


>
>>
>>
>>>
>>>
>>>>
>>>>> On Wed, Apr 18, 2018 at 3:40 PM, Sandro Bonazzola <sbonazzo at redhat.com
>>>>> > wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> 2018-04-18 9:37 GMT+02:00 Eyal Edri <eedri at redhat.com>:
>>>>>>
>>>>>>> FYI,
>>>>>>>
>>>>>>> I've disabled the 4.2 and master HC suites nightly run on CI as they
>>>>>>> are constantly failing for almost 3 weeks and spamming the mailing lists.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> HC uses gdeploy 2.0.6 which was released in December and was based on
>>>>>> ansible 2.4.
>>>>>> ansible-2.5 landed 3 weeks ago in EPEL, my guess is that gdeploy is
>>>>>> not supporting ansible-2.5 properly.
>>>>>> I had no time to validate my guess with proof, so please Sahina cross
>>>>>> check this.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I think this should get higher priority for a fix if we want it to
>>>>>>> provide any value,
>>>>>>> Work can continue using the manual jobs or via check-patch.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 16, 2018 at 10:56 AM, Gal Ben Haim <gbenhaim at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Any update on https://gerrit.ovirt.org/#/c/88887/ ?
>>>>>>>> The HC suites still failing and it's hard to understand why without
>>>>>>>> the logs from the engine VM.
>>>>>>>>
>>>>>>>> On Sat, Apr 7, 2018 at 7:19 AM, Sahina Bose <sabose at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Apr 6, 2018 at 1:10 PM, Simone Tiraboschi <
>>>>>>>>> stirabos at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 6, 2018 at 9:28 AM, Sahina Bose <sabose at redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> 2018-04-05 20:46:52,773-0400 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:100 TASK [Get local VM IP]
>>>>>>>>>>> 2018-04-05 20:55:28,217-0400 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:94 {u'_ansible_parsed': True, u'stderr_lines': [], u'cmd': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'end': u'2018-04-05 20:55:28.046320', u'_ansible_no_log': False, u'stdout': u'', u'changed': True, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'_raw_params': u"virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'start': u'2018-04-05 20:55:28.000470', u'attempts': 50, u'stderr': u'', u'rc': 0, u'delta': u'0:00:00.045850', u'stdout_lines': []}
>>>>>>>>>>> 2018-04-05 20:55:28,318-0400 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:98 fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:24:d3:63 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.045850", "end": "2018-04-05 20:55:28.046320", "rc": 0, "start": "2018-04-05 20:55:28.000470", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
>>>>>>>>>>>
>>>>>>>>>>> Both the 4.2 and master suites are failing on getting local VM
>>>>>>>>>>> IP.
>>>>>>>>>>> Any idea what changed or if I have to change the test?
>>>>>>>>>>>
>>>>>>>>>>> thanks!
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi Sahina,
>>>>>>>>>> 4.2 and master suite non HC are correctly running this morning.
>>>>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi
>>>>>>>>>> rt-system-tests_he-basic-ansible-suite-master/146/
>>>>>>>>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi
>>>>>>>>>> rt-system-tests_he-basic-ansible-suite-4.2/76/
>>>>>>>>>>
>>>>>>>>>> I'll try to check the difference with HC suites.
>>>>>>>>>>
>>>>>>>>>> Are you using more than one subnet in the HC suites?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> No, I'm not. And we havent's changed anything related to network
>>>>>>>>> in the test suite.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *GAL bEN HAIM*
>>>>>>>> RHV DEVOPS
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Eyal edri
>>>>>>>
>>>>>>>
>>>>>>> MANAGER
>>>>>>>
>>>>>>> RHV DevOps
>>>>>>>
>>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>>
>>>>>>>
>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>>>> <https://redhat.com/trusted>
>>>>>>> phone: +972-9-7692018
>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Devel mailing list
>>>>>>> Devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> SANDRO BONAZZOLA
>>>>>>
>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>>>>
>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>
>>>>>> sbonazzo at redhat.com
>>>>>> <https://red.ht/sig>
>>>>>> <https://redhat.com/summit>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Eyal edri
>>>>>
>>>>>
>>>>> MANAGER
>>>>>
>>>>> RHV DevOps
>>>>>
>>>>> EMEA VIRTUALIZATION R&D
>>>>>
>>>>>
>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>> <https://redhat.com/trusted>
>>>>> phone: +972-9-7692018
>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
>
> Eyal edri
>
>
> MANAGER
>
> RHV DevOps
>
> EMEA VIRTUALIZATION R&D
>
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20180430/dbcd15b1/attachment-0001.html>

More information about the Infra mailing list