rpm -q python-ovirt-engine-sdk4
Mine is outdated: $ rpm -q python-ovirt-engine-sdk4 python-ovirt-engine-sdk4-4.1.5-2.el7.centos.x86_64 Ill update that and run again. Logs (better to send too many than not enough sometimes). Ovirt engine logs are there. myoung at dev-vm in ~/repos/github/ovirt-system-tests on (no branch)▲ $ for i in $(find test_logs/basic-suite-4.1/post-002_bootstrap.py -name *.log); do echo $i; cat $i | fpaste; echo ----; done test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/tuned/tuned.log Uploading (3.5KiB)... https://da.gd/LiG0 -> https://paste.fedoraproject.org/paste/wed7pYARTmHHTHcf0DvVAQ/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/audit/audit.log Uploading (401.2KiB)... https://da.gd/tgSw -> https://paste.fedoraproject.org/paste/WEqegOFiLpiNGCv0hvY0ww/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/yum.log Uploading (24.3KiB)... https://da.gd/BzoMg -> https://paste.fedoraproject.org/paste/qla7z~hRsym8kkCr6NlXFg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/setup-ds.log Uploading (37.4KiB)... https://da.gd/1HHKA -> https://paste.fedoraproject.org/paste/4nRRDoC8lrv5CTxbS6TCxQ/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170814074645-vzih31.log Uploading (5544.4KiB)... https://da.gd/xYpYl -> https://paste.fedoraproject.org/paste/goz7RQVvusLXjiiheLD5xg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/boot.log Uploading (1.3KiB)... https://da.gd/4Y6pE -> https://paste.fedoraproject.org/paste/OCqWe6tjGkrNOrTkqex3mg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/server.log Uploading (34.1KiB)... https://da.gd/PZm7d -> https://paste.fedoraproject.org/paste/HpZOi8LTrgcWhKqfPeowPA/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/engine.log Uploading (2724.0KiB)... https://da.gd/pIOO -> https://paste.fedoraproject.org/paste/ufbtnfZOfSmun8sbOCs1xg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine-dwh/ovirt-engine-dwhd.log Uploading (2.4KiB)... https://da.gd/Xf5R -> https://paste.fedoraproject.org/paste/MO-dlfNQXi19Tla3T~QXsA/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/snmptrapd.log Uploading (0.1KiB)... https://da.gd/cF48 -> https://paste.fedoraproject.org/paste/NYqECCL1QVihcBQV~VWDsQ/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/tuned/tuned.log Uploading (1.5KiB)... https://da.gd/G5oC -> https://paste.fedoraproject.org/paste/JgvhOLoA7cEO2Q3ESjP5Aw/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/audit/audit.log Uploading (183.1KiB)... https://da.gd/9bjYn -> https://paste.fedoraproject.org/paste/Ado1joqnZxkPL1us6UeOwQ/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/yum.log Uploading (18.3KiB)... https://da.gd/i5ki -> https://paste.fedoraproject.org/paste/9sxQziGLRqw1tLHKfBAi6Q/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/tuned/tuned.log Uploading (1.5KiB)... https://da.gd/5Iiu4 -> https://paste.fedoraproject.org/paste/RaqsqKQdfgcTP1o2Epo~Pg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/audit/audit.log Uploading (179.9KiB)... https://da.gd/MyZ7 -> https://paste.fedoraproject.org/paste/x7T7ZkIS3g5JE2dj028AVg/ ---- test_logs/basic-suite-4.1/ post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/yum.log Uploading (17.4KiB)... https://da.gd/Gmux -> https://paste.fedoraproject.org/paste/kORmR4ShCv1EVGtTWSu2Zw/ ---- test_logs/basic-suite-4.1/post-002_bootstrap.py/lago_logs/lago.log Uploading (1198.1KiB)... https://da.gd/vzVCA -> https://paste.fedoraproject.org/paste/TuQiMEro23I~OnFcM7wG-g/ ---- On Mon, Aug 14, 2017 at 1:11 PM, Eyal Edri <eedri@redhat.com> wrote:
On Mon, Aug 14, 2017 at 7:18 PM, Marc Young <3vilpenguin@gmail.com> wrote:
You should have all logs needed @/home/myoung/repos/github/ovi
rt-system-tests/test_logs/basic-suite-4.1/post-002_bootstrap .py/lago_logs
The one I pasted into fpaste was the only one in that directory, if there are more logs Im just unsure where they live. Heres all logs for basic suite 4.1 that I know how to find:
myoung at dev-vm in ~/repos/github/ovirt-system-tests on (no branch)▲
The engine and hosts logs are usually under the test_logs, which should be in the same subdir as lago_logs directory. Also, can you verify you have the latest python v4 SDK installled?
rpm -q python-ovirt-engine-sdk4
If you don't have it installed, you can get it directly from the build job ( which acts also as yum repo ):
el7 build - http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_4.1_ build-artifacts-el7-x86_64/lastSuccessfulBuild/artifact/ exported-artifacts/
fc26 build is available only for master [1] but should also work.
There are also fc24/25 builds, just follow the same pattern to find them on Jenkins.
[1] http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_ master_build-artifacts-fc26-x86_64/lastSuccessfulBuild/ artifact/exported-artifacts/
$ for i in $(find . -name 'lago_logs' | grep basic-suite-4.1); do cat $i/*.log | fpaste; done Uploading (1355.0KiB)... https://da.gd/OIlR -> https://paste.fedoraproject.or g/paste/9Xn2RuDgF2pY2I0CouBbGg/ Uploading (1154.0KiB)... https://da.gd/0Cti -> https://paste.fedoraproject.or g/paste/uJqvAiDSblnmN8OA6i5tXA/ Uploading (1377.8KiB)... https://da.gd/GpDIf -> https://paste.fedoraproject.or g/paste/~Cn6IJgbvateJfFulDNFCg/ Uploading (1176.7KiB)... https://da.gd/skhBy -> https://paste.fedoraproject.or g/paste/HIpEVm4TRPpH5nJCOyUfWA/ Uploading (1397.1KiB)... https://da.gd/fEtF -> https://paste.fedoraproject.or g/paste/cuycP~u9D~gJKNPsQ5NH2g/ Uploading (1198.1KiB)... https://da.gd/1zNm -> https://paste.fedoraproject.or g/paste/5~yEDwCSr~tdxHT~XUaLPg/
On Mon, Aug 14, 2017 at 10:14 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Mon, Aug 14, 2017 at 5:10 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Full log is just as uninteresting: https://paste.f edoraproject.org/paste/ZaZMcT5sg9SAGQ7OKC3-EA
I was actually hoping for the engine log. Y.
On Mon, Aug 14, 2017 at 8:03 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Mon, Aug 14, 2017 at 3:31 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Thanks for clarifying, that makes sense in hindsight since its testing things in their entirety. This is more of a rabbit hole for me as Im really just trying to learn Lago by using the OST project (since I'll be duplicating a lot of the setup).
stdout logs from a run: https://pastebin.com/KBDaCCYp
Can you send engine log so we'll understand why the host installation failed? You should have all logs needed @/home/myoung/repos/github/ovi rt-system-tests/test_logs/basic-suite-4.1/post-002_bootstrap .py/lago_logs
Y.
On Mon, Aug 14, 2017 at 6:44 AM, Eyal Edri <eedri@redhat.com> wrote:
> > > On Mon, Aug 14, 2017 at 2:31 PM, Marc Young <3vilpenguin@gmail.com> > wrote: > >> Ill try to get some detailed log files later, but fwiw I'm not >> running the hc suites (afaik, Im still getting inundated with the system >> tests/lago). The link I used for Jenkins was just to try to pull the latest >> 'passing' hash for ovirt-system-tests >> > > I see, its a bit more complicated than that I'm afraid, the HASH you > see is probably on for OST code itself, while the suites can fail on any > oVirt project ( ovirt-engine/vdsm/host-deploy/etc.... ), so its > more than just a single hash, but rather a list of RPMs and their versions, > together with hash of OST ( for the tests themselves ) and also OS updates. > > >> >> The errors have all been from master branch on the basic suite for >> 4.1 via: >> >> $ ./run_suite.sh basic-suite-4.1/ >> >> > OK, then we need to investigate it and understand why add host fails > on 4.1, please provide logs when possible. > > BTW, you can also try running the manual job [1], but you'll need to > provide it with custom yum repo URL with your built artifacts to test your > code. > There is a section on it on the OST readthedocs page. > > [1] http://jenkins.ovirt.org/view/oVirt%20system%20tests/job > /ovirt-system-tests_manual/ > > >> >> On Mon, Aug 14, 2017 at 12:29 AM, Eyal Edri <eedri@redhat.com> >> wrote: >> >>> I see you run the 'hc' suite, which means hyperconverged. This >>> suite is running an hosted engine on gluster storage, It's a more complex >>> suite than the basic one, and prone to more errors, however it should still >>> work, but if you don't require it specifically, I would recommend running >>> the basic suite, which should be easier to debug and also quicker to run. >>> >>> I'm also adding hc maintainer so she can check, can you share the >>> link to the Jenkins job you're running? Or the log files if you're running >>> locally. >>> >>> On Aug 14, 2017 06:00, "Marc Young" <3vilpenguin@gmail.com> wrote: >>> >>>> Actually I spoke too soon, still fails: >>>> >>>> + lago ovirt runtest /home/myoung/repos/github/ovir >>>> t-system-tests/vagrant/test-scenarios/002_bootstrap.py >>>> @ Run test: 002_bootstrap.py: >>>> nose.config: INFO: Ignoring files matching ['^\\.', '^_', >>>> '^setup\\.py$'] >>>> # print_api_ver: >>>> # print_api_ver: Success (in 0:00:00) >>>> # add_dc: >>>> # add_dc: Success (in 0:00:43) >>>> # add_cluster: >>>> # add_cluster: Success (in 0:00:03) >>>> # add_hosts: >>>> dd >>>> * Collect artifacts: >>>> * Collect artifacts: Success (in 0:01:14) >>>> # add_hosts: Success (in 0:16:36) >>>> # Results located at /home/myoung/repos/github/ovir >>>> t-system-tests/deployment-vagrant/default/002_bootstrap.py.j >>>> unit.xml >>>> @ Run test: 002_bootstrap.py: Success (in 0:17:26) >>>> Error occured, aborting >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line >>>> 360, in do_run >>>> self.cli_plugins[args.ovirtverb].do_run(args) >>>> File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", >>>> line 184, in do_run >>>> self._do_run(**vars(args)) >>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line >>>> 501, in wrapper >>>> return func(*args, **kwargs) >>>> File "/usr/lib/python2.7/site-packages/lago/utils.py", line >>>> 512, in wrapper >>>> return func(*args, prefix=prefix, **kwargs) >>>> File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line >>>> 99, in do_ovirt_runtest >>>> raise RuntimeError('Some tests failed') >>>> RuntimeError: Some tests failed >>>> >>>> >>>> >>>> >>>> On Sun, Aug 13, 2017 at 9:47 PM, Marc Young < >>>> 3vilpenguin@gmail.com> wrote: >>>> >>>>> Edit: reply-all >>>>> >>>>> It does, every time. >>>>> I got it to pass by using the last passing revision from Jenkins >>>>> (hash 98ae6d0b452d098f2703a197deb082a091bba837 ), noted from >>>>> http://jenkins.ovirt.org/job/system-tests_hc-basic-suite-mas >>>>> ter/15/consoleFull >>>>> >>>>> Not sure if it's a true race condition, that build in Jenkins >>>>> has failed consistently since #15 >>>>> >>>>> On Sun, Aug 13, 2017 at 2:03 AM, Eyal Edri <eedri@redhat.com> >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Fri, Aug 11, 2017 at 9:34 PM, Marc Young < >>>>>> 3vilpenguin@gmail.com> wrote: >>>>>> >>>>>>> What's the fix for this for those of us using >>>>>>> ovirt-system-tests? >>>>>>> >>>>>>> Im trying to adapt some of the code for testing a third party >>>>>>> tool, but master is still failing on 002_bootstrap >>>>>>> >>>>>> >>>>>> It fails consistently? >>>>>> AFAIK this is a race condition that happens maybe once a week, >>>>>> can you share your logs? is it also failing on add_secondary_storage? >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> On Tue, Aug 1, 2017 at 9:20 AM, Benny Zlotnik < >>>>>>> bzlotnik@redhat.com> wrote: >>>>>>> >>>>>>>> I'm not sure it's related since the LSM test runs much later, >>>>>>>> in 004 >>>>>>>> >>>>>>>> On Tue, Aug 1, 2017 at 3:33 PM, Eyal Edri <eedri@redhat.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Adding Allon & Benny. >>>>>>>>> >>>>>>>>> Is it possible to verify this is related to the LSM issue >>>>>>>>> we've been handling in [1]? >>>>>>>>> If this is the case, we agreed to disable the test next time >>>>>>>>> it fails, as the current workarounds with sleep isn't enough. >>>>>>>>> >>>>>>>>> Can you confirm this is the case, and so we'll have to >>>>>>>>> disable this test until one of the RFEs described in [1] is merged? >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] https://gerrit.ovirt.org/#/c/78613/ >>>>>>>>> >>>>>>>>> On Tue, Aug 1, 2017 at 2:45 PM, Barak Korren < >>>>>>>>> bkorren@redhat.com> wrote: >>>>>>>>> >>>>>>>>>> On 1 August 2017 at 14:39, Nir Soffer <nsoffer@redhat.com> >>>>>>>>>> wrote: >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > On Tue, Aug 1, 2017 at 2:34 PM Barak Korren < >>>>>>>>>> bkorren@redhat.com> wrote: >>>>>>>>>> >> >>>>>>>>>> >> Test failed: [ 002_bootstrap.add_secondary_storage_domains >>>>>>>>>> ] >>>>>>>>>> >> >>>>>>>>>> >> Link to suspected patches: >>>>>>>>>> >> https://gerrit.ovirt.org/#/c/79974 >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > This patch adds missing log when resizing an online disk >>>>>>>>>> - why do you think >>>>>>>>>> > it is related to the failure? >>>>>>>>>> >>>>>>>>>> Because it is the only patch participating in the test. >>>>>>>>>> (The test was equivalent to running the manual job with >>>>>>>>>> just this patch) >>>>>>>>>> >>>>>>>>>> Then again this may also one of the usual SD testing race >>>>>>>>>> conditions. >>>>>>>>>> Since the code in the patch seemed to be related to storage >>>>>>>>>> I didn't >>>>>>>>>> want to just assume that. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Barak Korren >>>>>>>>>> RHV DevOps team , RHCE, RHCi >>>>>>>>>> Red Hat EMEA >>>>>>>>>> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted >>>>>>>>>> _______________________________________________ >>>>>>>>>> Devel mailing list >>>>>>>>>> Devel@ovirt.org >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Eyal edri >>>>>>>>> >>>>>>>>> >>>>>>>>> ASSOCIATE MANAGER >>>>>>>>> >>>>>>>>> RHV DevOps >>>>>>>>> >>>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>>> >>>>>>>>> >>>>>>>>> Red Hat EMEA <https://www.redhat.com/> >>>>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>>>>>> <https://redhat.com/trusted> >>>>>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel@ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Eyal edri >>>>>> >>>>>> >>>>>> ASSOCIATE MANAGER >>>>>> >>>>>> RHV DevOps >>>>>> >>>>>> EMEA VIRTUALIZATION R&D >>>>>> >>>>>> >>>>>> Red Hat EMEA <https://www.redhat.com/> >>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>>> <https://redhat.com/trusted> >>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>> >>>>> >>>>> >>>> >> > > > -- > > Eyal edri > > > ASSOCIATE MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> TRIED. TESTED. TRUSTED. > <https://redhat.com/trusted> > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) >
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
ASSOCIATE MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)