[ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 01/08/2017 ] [add_secondary_storage_domains]

Eyal Edri eedri at redhat.com
Mon Aug 14 11:44:18 UTC 2017 

On Mon, Aug 14, 2017 at 2:31 PM, Marc Young <3vilpenguin at gmail.com> wrote:

> Ill try to get some detailed log files later, but fwiw I'm not running the
> hc suites (afaik, Im still getting inundated with the system
> tests/lago). The link I used for Jenkins was just to try to pull the latest
> 'passing' hash for ovirt-system-tests
>

I see, its a bit more complicated than that I'm afraid, the HASH you see is
probably on for OST code itself, while the suites can fail on any oVirt
project ( ovirt-engine/vdsm/host-deploy/etc.... ), so its more than just a
single hash, but rather a list of RPMs and their versions, together with
hash of OST ( for the tests themselves ) and also OS updates.


>
> The errors have all been from master branch on the basic suite for 4.1 via:
>
> $ ./run_suite.sh basic-suite-4.1/
>
>
OK, then we need to investigate it and understand why add host fails on
4.1, please provide logs when possible.

BTW, you can also try running the manual job [1], but you'll need to
provide it with custom yum repo URL with your built artifacts to test your
code.
There is a section on it on the OST readthedocs page.

[1]
http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/


>
> On Mon, Aug 14, 2017 at 12:29 AM, Eyal Edri <eedri at redhat.com> wrote:
>
>> I see you run the 'hc' suite, which means hyperconverged. This suite is
>> running an hosted engine on gluster storage, It's a more complex suite than
>> the basic one, and prone to more errors, however it should still work, but
>> if you don't require it specifically, I would recommend running the basic
>> suite, which should be easier to debug and also quicker to run.
>>
>> I'm also adding hc maintainer so she can check, can you share the link to
>> the Jenkins job you're running? Or the log files if you're running locally.
>>
>> On Aug 14, 2017 06:00, "Marc Young" <3vilpenguin at gmail.com> wrote:
>>
>>> Actually I spoke too soon, still fails:
>>>
>>> + lago ovirt runtest /home/myoung/repos/github/ovir
>>> t-system-tests/vagrant/test-scenarios/002_bootstrap.py
>>> @ Run test: 002_bootstrap.py:
>>> nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
>>>   # print_api_ver:
>>>   # print_api_ver: Success (in 0:00:00)
>>>   # add_dc:
>>>   # add_dc: Success (in 0:00:43)
>>>   # add_cluster:
>>>   # add_cluster: Success (in 0:00:03)
>>>   # add_hosts:
>>> dd
>>>     * Collect artifacts:
>>>     * Collect artifacts: Success (in 0:01:14)
>>>   # add_hosts: Success (in 0:16:36)
>>>   # Results located at /home/myoung/repos/github/ovir
>>> t-system-tests/deployment-vagrant/default/002_bootstrap.py.junit.xml
>>> @ Run test: 002_bootstrap.py: Success (in 0:17:26)
>>> Error occured, aborting
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 360,
>>> in do_run
>>>     self.cli_plugins[args.ovirtverb].do_run(args)
>>>   File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line
>>> 184, in do_run
>>>     self._do_run(**vars(args))
>>>   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in
>>> wrapper
>>>     return func(*args, **kwargs)
>>>   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in
>>> wrapper
>>>     return func(*args, prefix=prefix, **kwargs)
>>>   File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in
>>> do_ovirt_runtest
>>>     raise RuntimeError('Some tests failed')
>>> RuntimeError: Some tests failed
>>>
>>>
>>>
>>>
>>> On Sun, Aug 13, 2017 at 9:47 PM, Marc Young <3vilpenguin at gmail.com>
>>> wrote:
>>>
>>>> Edit: reply-all
>>>>
>>>> It does, every time.
>>>> I got it to pass by using the last passing revision from Jenkins (hash
>>>> 98ae6d0b452d098f2703a197deb082a091bba837 ), noted from
>>>> http://jenkins.ovirt.org/job/system-tests_hc-basic-suite-mas
>>>> ter/15/consoleFull
>>>>
>>>> Not sure if it's a true race condition, that build in Jenkins has
>>>> failed consistently since #15
>>>>
>>>> On Sun, Aug 13, 2017 at 2:03 AM, Eyal Edri <eedri at redhat.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 11, 2017 at 9:34 PM, Marc Young <3vilpenguin at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> What's the fix for this for those of us using ovirt-system-tests?
>>>>>>
>>>>>> Im trying to adapt some of the code for testing a third party tool,
>>>>>> but master is still failing on 002_bootstrap
>>>>>>
>>>>>
>>>>> It fails consistently?
>>>>> AFAIK this is a race condition that happens maybe once a week, can you
>>>>> share your logs? is it also failing on add_secondary_storage?
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> On Tue, Aug 1, 2017 at 9:20 AM, Benny Zlotnik <bzlotnik at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm not sure it's related since the LSM test runs much later, in 004
>>>>>>>
>>>>>>> On Tue, Aug 1, 2017 at 3:33 PM, Eyal Edri <eedri at redhat.com> wrote:
>>>>>>>
>>>>>>>> Adding Allon & Benny.
>>>>>>>>
>>>>>>>> Is it possible to verify this is related to the LSM issue we've
>>>>>>>> been handling in [1]?
>>>>>>>> If this is the case, we agreed to disable the test next time it
>>>>>>>> fails, as the current workarounds with sleep isn't enough.
>>>>>>>>
>>>>>>>> Can you confirm this is the case, and so we'll have to disable this
>>>>>>>> test until one of the RFEs described in [1] is merged?
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://gerrit.ovirt.org/#/c/78613/
>>>>>>>>
>>>>>>>> On Tue, Aug 1, 2017 at 2:45 PM, Barak Korren <bkorren at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On 1 August 2017 at 14:39, Nir Soffer <nsoffer at redhat.com> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Tue, Aug 1, 2017 at 2:34 PM Barak Korren <bkorren at redhat.com>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Test failed: [ 002_bootstrap.add_secondary_storage_domains ]
>>>>>>>>> >>
>>>>>>>>> >> Link to suspected patches:
>>>>>>>>> >> https://gerrit.ovirt.org/#/c/79974
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > This patch adds missing log when resizing an online disk - why
>>>>>>>>> do you think
>>>>>>>>> > it is related to the failure?
>>>>>>>>>
>>>>>>>>> Because it is the only patch participating in the test.
>>>>>>>>> (The test was equivalent to running the manual job with just this
>>>>>>>>> patch)
>>>>>>>>>
>>>>>>>>> Then again this may also one of the usual SD testing race
>>>>>>>>> conditions.
>>>>>>>>> Since the code in the patch seemed to be related to storage I
>>>>>>>>> didn't
>>>>>>>>> want to just assume that.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Barak Korren
>>>>>>>>> RHV DevOps team , RHCE, RHCi
>>>>>>>>> Red Hat EMEA
>>>>>>>>> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
>>>>>>>>> _______________________________________________
>>>>>>>>> Devel mailing list
>>>>>>>>> Devel at ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Eyal edri
>>>>>>>>
>>>>>>>>
>>>>>>>> ASSOCIATE MANAGER
>>>>>>>>
>>>>>>>> RHV DevOps
>>>>>>>>
>>>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>>>
>>>>>>>>
>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>>>>> <https://redhat.com/trusted>
>>>>>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Devel mailing list
>>>>>>> Devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Eyal edri
>>>>>
>>>>>
>>>>> ASSOCIATE MANAGER
>>>>>
>>>>> RHV DevOps
>>>>>
>>>>> EMEA VIRTUALIZATION R&D
>>>>>
>>>>>
>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>> <https://redhat.com/trusted>
>>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>
>>>>
>>>>
>>>
>


-- 

Eyal edri


ASSOCIATE MANAGER

RHV DevOps

EMEA VIRTUALIZATION R&D


Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170814/9f6c2c86/attachment-0001.html>

More information about the Devel mailing list