- lago-devel - oVirt List Archives

Lago Demo
by Gal Ben Haim 19 May '16

19 May '16

Hi All, I wrote a document which demonstrates a basic use case of Lago. The reader should follow the instructions in the document, and thus get a sense about Lago's capabilities and ease of use. This document should guide the user in his first experience, and this is why I'm not exposing the advanced capabilities of Lago (we will have another document for this). I would like you to try out the document, and write back a review. Link to the document: https://github.com/gbenhaim/Lago-Demo Thanks ! Gal Ben Haim gbenhaim(a)redhat.com RHEV DevOps

2 1

[2016-05-18] Weekly lago release
by David Caro 18 May '16

18 May '16

Hi everyone!! Last week's release was delayed a little bit due to the holidays, so we are releasing today, the new 0.17 version of lago comes with some interesting improvements: 9253f807: Added possibility to use env vars on disk path and url c7ff5190: Removed the repo verification code - replaced by retries and workarounds 61813f2e: Fix wait_for_ssh logic - now the times and tries when waiting for ssh make more sense, and have better logging 1ec94d55: Added LAGO_INITFILE_PATH env var when running init 86a8bde1: Setting LAGO_WORKDIR_PATH too on init 7a661154: Copy inside the prefix the deploy scripts 499c9912: Expanding the disks path on vm xml generation c5f6cafb: Using LAGO_PREFIX_DIR for the disk images too So the most change is the possibility to use env variables in the init file, and the addition of the env vars: * LAGO_PREFIX_DIR * LAGO_WORKDIR_PATH * LAGO_INITFILE_PATH (only when running lago init) That allow you to better specify relative paths to the different files (disks, scripts, urls...). And that allow to keep the workdir functional even after renaming them (mv old_workdir new_workdir). For the updated docs and changelog: http://lago.readthedocs.io/en/0.17/ For the rpms: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Can't run tests with latest Lago & ovirt-system-tests
by Yaniv Kaul 16 May '16

16 May '16

I've updated to the latest ovirt-system-tests, and apparently commit ed759a62 was breaking things to me. So I've updated Lago to latest (from http://resources.ovirt.org/repos/lago/unstable/0.0/latest/rpm/fc23/noarch/ ) - which is 0.16.12-1 now, and now things break differently, possibly by Lago commit ba6dda1e92faf26bc6842a78143ced20ae0546ff @ Create prefix internal repo: ERROR (in 0:00:00) Error occured, aborting Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 248, in do_run self.cli_plugins[args.ovirtverb].do_run(args) File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run self._do_run(**vars(args)) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 595, in wrapper return func(*args, prefix=prefix, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 166, in do_ovirt_reposetup skip_sync=skip_sync, File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 615, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 423, in prepare_repo _sync_rpm_repository(rpm_repo, reposync_yum_config, repos) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 105, in _sync_rpm_repository ret, out, _ = utils.run_command(reposync_command) AttributeError: 'module' object has no attribute 'run_command Any idea?

3 3

[2016-05-09] Lago weekly meeting minutes
by David Caro 09 May '16

09 May '16

Agenda ======== * oVirt system tests status: - 3.6 working - Master seems to fail on add-hosts still (investigating) * Merged workaround for reposync issues * Lago tutorial on it’s way - Using the unstable/latest lago (with the new patches) - Latest version: http://jenkins.ovirt.org/job/lago_master_check-patch-el7-x86_64/51/artifact… * Gal released 0.16.0 last week, kudos! * Merged patch to copy the deploy scripts inside the prefix on init, that takes us a step closer to relocatable prefixes (and exportable prefixes too) Let's have a great week! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

[05-05-2016] New lago release - 0.16.0
by Gal Ben Haim 05 May '16

05 May '16

Hi everyone! There's a new lago release with version 0.16.0 ready for you to upgrade! One important change in version 0.16.0 is the support for the following environment variables: $LAGO_PREFIX_PATH = .lago/current $LAGO_WORKDIR_PATH = .lago it's now highly recommended to use those variables when configuring the path to the deployment scripts. Here are the other changes: * Wed May 04 2016 David Caro <dcaroest(a)redhat.com> - 0.16.0 45764a95: Fixed small typo in a string * Wed May 04 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.5 66dec63b: Merge pull request #231 from lago-project/fix_el7_dirlock ccf6dca4: Added lock class for el7 locking 9a6e370c: Use lago.utils.LockFile instead of lockfile's one 98322f01: Removed thread pool on reposync 167c8b7d: Moved some code out of a with block db836eda: Yapf fixes * Tue May 03 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.4 0e535676: Merge pull request #230 from lago-project/add_prefix_env_vars 17e9be31: tests:Modified deploy funct tests to use vars 1440adee: Add some lago extra vars to the env 577c442a: Expand vars and user contraptions on deploy scripts * Tue May 03 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.3 a1a57978: Merge pull request #229 from lago-project/extra_gitignore 2a5a3359: Added extra ignores to .gitignore * Fri Apr 29 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.2 25c91601: Merge pull request #227 from lago-project/fix_unhandled_exception 8eeb43cc: Fixed wrong error log on reposetup * Fri Apr 29 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.1 5c5bc197: Merge pull request #226 from lago-project/fix_repoverify_check c4380a9a: Fixed repoverify not behaving like reposync d8c9ac55: Added new deps for the tests * Sat Apr 23 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.0 4b98d690: Merge pull request #223 from lago-project/try_cleaning_reposync_caches_on_first_failure f87101ee: Reposync first retry with clean caches Enjoy! Gal Ben Haim gbenhaim(a)redhat.com RHEV-CI Team

2 1

Weekly bug status for ovirt-system-tests (using Lago) [ 01/05/16 ]
by Eyal Edri 01 May '16

01 May '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Improvements on the host install issue, some env are not reproducing anymore but not sure we're 100% clear yet. *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open * There are quite a few open, some are just temp for helping with the debug of 'stuck in installing state' *Jenkins jobs: (failing on known issues)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ *Recently merged patches:* commit 211e475c004a2635896e528cd3aed6cd7d0dc5fd Author: Sandro Bonazzola <sbonazzo(a)redhat.com> Date: Thu Apr 28 16:44:50 2016 +0200 common: yum-repos: update glusterfs repos - dropped Gluster community repo for Fedora23: the repo is empty and Gluster community is maintaining Fedora packages within Fedora repositories. For reference: http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fedora/What_About_… - on EL7 moved from Gluster community repo to CentOS Storage SIG repositories: we've been advised by Gluster community that their packaging effort is focused on CentOS Storage SIG and CentOS infrastructure is more reliable. Change-Id: I4d2c752be825fbbb9035222845728d10c5dd288c Signed-off-by: Sandro Bonazzola <sbonazzo(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit 19c102110279975d274c32477cc28c7c29e75fbc Author: David Caro <dcaroest(a)redhat.com> Date: Wed Apr 20 21:10:37 2016 +0200 Fixed answerfile for master A new question was added to the setup Change-Id: I6be958a70d67126d8067ff1855018125623b10c1 ---------------------------------------------------------------------------------------------------------- commit 5829862d17e92f91cfd04e7b9d5549cd775c800c Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Mon Apr 25 14:06:27 2016 +0300 Add 'device=eth0' to EL7 deploy scripts. Apparently, /etc/sysconfig/network-scripts/ifcfg-eth0 lacks it on EL7 images. This works with NetworkManager, but not without it. When something goes bad, VDSM fails to rollback properly as its missing. For now, add it. Change-Id: I1c4ca07499a3947022e6f943cf05842ee1def408 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit ddb42782968e5c6d411cdc747503fd90bed249d7 Author: David Caro <dcaroest(a)redhat.com> Date: Tue Mar 29 18:53:42 2016 +0200 Fix the master suite runs This is just copying over and adapting a bit (just so it works) the 3.6 suite into the master suite. Only for the 3.6 api, the newapi tests will be added later Still affected by the storage deactivation issues Change-Id: I55781a5d5a746aaf81e403eeeee40880beacb8a0 Signed-off-by: David Caro <dcaroest(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit c7d50cf8aebd6b566ae5c6ee8d6e3b8ec24ffc2b Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Tue Apr 26 13:57:43 2016 +0300 Remove snapshot and reverting as part of the ovirt-system-tests I currently did not find a use case for it. It slows down the tests and causes (real? false?) issues with SDs. Mainly their activation and deactivation. It dramatically reduced the time to run the whole suite for me. 3-4 minutes on my desktop, and >10 minutes on my laptop. Change-Id: I9f7e509220c83805df78abb94b0ddc390c4a7f54 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[2016-04-26] Lago weekly meeting minutes
by David Caro 26 Apr '16

26 Apr '16

Agenda ======== * Ovirt system tests 3.6 status: Gal testing 3.6.6 it locally, it’s expected to pass * Ovirt system tests master status: Affected by: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 https://bugzilla.redhat.com/show_bug.cgi?id=1329317 * Had some issues with jenkins.phx.ovirt.org last week, fixed already * Currently reposetup not working on el7 systems, using old version of lockfile https://github.com/lago-project/lago/issues/225 * Released 0.15 with some improvements: Rotating the dest dir on collect Generic ‘lago deploy’ command More verbose reposetup command In progress: * Lago tutorial * Lago init file spec * Lago high level desing doc * virt.py fixes Action items: * dcaro: fix the el7 lockfile issue * dcaro: review the tutorial to unblock gbenhaim Thanks!! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

2 2

Weekly bug status for ovirt-system-tests (using Lago) [ 24/04/16 ]
by Eyal Edri 24 Apr '16

24 Apr '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Still suffering from host add failures (stuck in installing, David has been debugging it constantly with Piotr last week) *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open *Jenkins jobs: (failing on known issues)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ *Recently merged patches:* (Thanks again to Yaniv K!) Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Wed Apr 20 23:51:55 2016 +0300 More accurate 3.6 repo 1. With this repo, both engine and hosts can be installed, without any external access, I hope. 2. Now we can test latest 3.6, not the released one. Change-Id: I9066af4f5d92e275586c29c65fe7ef7500fe5c20 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Tue Apr 12 23:00:16 2016 +0300 Fixes for FC23 local repo setup: 1. The interface may not be eth0. The new code hopefully work with any if name. Depends that the interface is the first, but let's hope it is. 2. Add 'cost=1' to ensure the local repo is preferred over other repos. Change-Id: If846d7bacdf517d1342464c210359e5a0e8cad6e Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Wed Mar 30 22:46:39 2016 +0300 Basic quota tests Add quota and set it to audit mode, not more than that. Change-Id: Ia35e9b2b65c01cdcca8da7f87ed7a38f8dc2bb17 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Thu Apr 14 01:04:28 2016 +0300 Fix mount / path issues with storage_nfs_el7 provisioning script Clearly was not tested well (because of the host installation, but that's an execuse). This patch fixes it, tested on 3.6 with adding all domains, finally. It also again simplifies it - but removes the option of not mounting the export mount. Note that the ISO mount is not completely done as well, I think. Will attend to in a follow up patch. Based on David's request, the whole setup is split into functions. Tested (properly) with 3.6, including the TEMPLATES domain attachment. (The test for it should still be excluded, IMHO). Change-Id: I261f3d3f3b3647cbbc506913478ffc6c36594519 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Sun Apr 10 17:10:16 2016 +0300 Simplification and enhancements to iSCSI and NFS servers deployment Removed a lot of not needed code from the setup. Mainly, just straightforward using the disks (in case of NFS), no need for LVM stuff. Disabled firewall, enabled 'noop' scheduler. Tested with both master and 3.6. Never got to test fully master, blocked by a host installation bug. 3.6 passed well. Change-Id: I7ec630ba5fa62529421b8e8f9aec993e7cdc84b6 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> Please comment / update on any additional info you have on this thread, Eyal.

1 0

Weekly lago release 0.15
by David Caro 23 Apr '16

23 Apr '16

Hi everyone!! It's been a couple weeks without releases, but we are back and 100% committed to keep up the pace. On this week's release: * Several reposetup command improvements (better logs, friendlier retry, ...) * Collect now rotates the destination folder if it exists * Skip vm bootstrap, what allows faster functional tests * Generic deploy subcommand, now you can use the deploy command with lago, just add a 'deploy-scripts' list entry to the 'metadata' key on any vm/domain definition, with a list of scripts and they will be run on 'lago deploy' For the full changelog, as always, you can go to: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt And for the latest rpms: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Can I define somewhere the collect artifacts?
by Yaniv Kaul 14 Apr '16

14 Apr '16

I'm extending the VDSM functional tests, and saving files under /tmp/vdsm-results/ - can I simply ask somehow to get all files from there, perhaps via the init.json.ini file would be the right place to add this? TIA, Y.

3 6

Lago + ovirt-system-tests run fail on collecting logs
by Eyal Edri 13 Apr '16

13 Apr '16

After running: ./run_suite.sh --cleanup basic_suite_3.6 ./run_suite.sh basic_suite_3.6 The tests failed at [1] Does it mean the cleanup stage failed? why would log collection fail on a duplicate file? I think it will be beneficial to support multiple logs and rotate it like we do in ovirt for e.g, Since a developer might run a few times without cleanning up the env. Do we have an open issue for it? , cause I didn't find anything similar. [1]. @ Collect artifacts: ERROR (in 0:00:00) Error occured, aborting Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 248, in do_run self.cli_plugins[args.ovirtverb].do_run(args) File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run self._do_run(**vars(args)) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper return func(*args, prefix=prefix, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 217, in do_ovirt_collect prefix.collect_artifacts(output) File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 598, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 928, in collect_artifacts os.makedirs(output_dir) File "/usr/lib64/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: '/home/eedri/lago/ovirt-system-tests/test_logs/basic_suite_3.6/post-001_initialize_engine.py' -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

3 7

configuring path for reposync
by Eyal Edri 12 Apr '16

12 Apr '16

Hi, When installing lago you get these default values in : [lago] log_level = debug template_store = /var/lib/lago/store template_repos = /var/lib/lago/repos First, what is the diff between the 2? store holds the templates, what is repos? Also, when running ovirt-system-tests i see under /var/lib/lago/reposync all the repos sycned, is there away to define this path to be somewhere else? is it in the ovirt-system-tests config (if so why is it under lago dir) if not, shouldn't it be in lago.conf? -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 2

repeating libvirt exception in vdsm check-merged job
by Yaniv Bronheim 11 Apr '16

11 Apr '16

Hi, we keep getting the following error in vdsm automation *12:16:47* + lago start vdsm_functional_tests_host-fc23*12:16:49* current session does not belong to lago group.*12:16:49* @ Start specified VMs: *12:16:49* # Start nets: *12:16:49* * Create network vdsm_functional_tests_lago: *12:16:50* * Create network vdsm_functional_tests_lago: Success (in 0:00:00)*12:16:50* # Start nets: Success (in 0:00:00)*12:16:50* # Start vms: *12:16:50* * Starting VM vdsm_functional_tests_host-fc23: *12:16:50* libvirt: QEMU Driver error : Cannot check QEMU binary /usr/libexec/qemu-kvm: No such file or directory*12:16:50* * Starting VM vdsm_functional_tests_host-fc23: ERROR (in 0:00:00)*12:16:50* # Start vms: ERROR (in 0:00:00)*12:16:50* # Destroy network vdsm_functional_tests_lago: *12:16:50* # Destroy network vdsm_functional_tests_lago: ERROR (in 0:00:00)*12:16:50* @ Start specified VMs: ERROR (in 0:00:00)*12:16:50* Error occured, aborting*12:16:50* Traceback (most recent call last):*12:16:50* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 661, in main*12:16:50* cli_plugins[args.verb].do_run(args)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run*12:16:50* self._do_run(**vars(args))*12:16:50* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper*12:16:50* return func(*args, **kwargs)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper*12:16:50* return func(*args, prefix=prefix, **kwargs)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 234, in do_start*12:16:50* prefix.start(vm_names=vm_names)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 778, in start*12:16:50* self.virt_env.start(vm_names=vm_names)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 193, in start*12:16:50* vm.start()*12:16:50* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 910, in start*12:16:50* self._env.libvirt_con.createXML(self._libvirt_xml())*12:16:50* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3611, in createXML*12:16:50* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)*12:16:50* libvirtError: Cannot check QEMU binary /usr/libexec/qemu-kvm: No such file or directory all jobs fail today with the same exception - http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/53/console . any idea why qemu-kvm is missing? -- *Yaniv Bronhaim.*

3 5

libvirt: QEMU Driver error : Guest agent is not responding: QEMU guest agent is not available due to an error
by Eyal Edri 10 Apr '16

10 Apr '16

Anyone run lago before on RHEL 7.2? Its possible we're hitting [1] with libvirt-1.2.17 https://bugzilla.redhat.com/show_bug.cgi?id=1090551 -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

reminder on lago khanban board
by Eyal Edri 10 Apr '16

10 Apr '16

FYI, Lago started using a khanban board [1] to prioritize and visualize the current issues which are being worked on, since it very hard to use GitHub to monitor and prioritize issues. Taiga is an open source project and was chosen (rather than Trello for e.g) because it had already integration with GitHub for syncing automatically issues to the board and moving statuses. If you want to ask for a specific issue prioritization, please use this list to communicate it, We will review issues on a weekly basis and the WIP will be shown on the board. [1] https://tree.taiga.io/project/david-caro-lago/kanban -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

Weekly bug status for ovirt-system-tests (using Lago) [ 10/04/16 ]
by Eyal Edri 10 Apr '16

10 Apr '16

A weekly reminder on open bugs & patches to keep track on blockers for oVirt testing: *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Still suffering from add host failures, pending a fix from network team: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 *Recently merged:* (Thanks Yaniv K!) - Multiple changes to deployment script https://gerrit.ovirt.org/#/c/55695/ - Fix IP Address of engine *https://gerrit.ovirt.org/#/c/55903/ <https://gerrit.ovirt.org/#/c/55903/>* - Improvements to the storage domain add tests - https://gerrit.ovirt.org/#/c/55233/ *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open - *Basic quota tests: https://gerrit.ovirt.org/#/c/55514/ <https://gerrit.ovirt.org/#/c/55514/>* - *Switched between hot-add disk and hot-add NIC tests * https://gerrit.ovirt.org/#/c/55740/ - (Attempt to reduce chance of hitting the deactivate storage bug, though the fix of this bug might be in already, so not sure its needed, Yaniv?) - WIP: Fix master (4.0) suite, still in progress https://gerrit.ovirt.org/#/c/55425/ - Merging the storage VMs into one, still in progress: https://gerrit.ovirt.org/#/c/54957/ If you have any updates on the bugs/patches please reply with more info. -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 1

In which order repos are searched for RPMs?
by Yaniv Kaul 05 Apr '16

05 Apr '16

If I have the RPM on the 'localsync' repo as well as externally (say, from CentOS 7 updates), which repo will it be taken from? - Simple solution: sync all that you need and disable the 'updates' repo. I'm in favor. Same for the 'base' repo and any other repo. Essentially, all that the VM needs should be in the 'localsync' repo. - Simple solution #2 : rename 'localsync' to '*a*localsync' - just to make sure it is the first looked at... Before: [root@lago_basic_suite_3_6_engine ~]# egrep "Updated|Installed" /var/log/yum.log | awk -F\: '{print $NF}' |xargs yum info |grep "From repo" |egrep -c "base|updates" 87 After the change: [root@lago_basic_suite_3_6_engine ~]# egrep "Updated|Installed" /var/log/yum.log | awk -F\: '{print $NF}' |xargs yum info |grep "From repo" |egrep -c "base|updates" 1

2 1

[2016-04-04] Lago weekly meeting minutes
by David Caro 04 Apr '16

04 Apr '16

Hi everyone, short meeting this week: Discussions: ============= * Ovirt system tests 3.6 status: - 3.6 tests running on jenkins (currently failing to add hosts) * Ovirt system tests master status: - Affected by: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 * Investigating about vagrant-lago - https://docs.google.com/spreadsheets/d/1PKIlmhTd-vxEfs7JZC-mLkvWG4cBE1oOnxR… - Have to schedule a meeting to push this forward Action Items ============= * Migrate to jenkins.phx.ovirt.orgt -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Weekly bug status for ovirt-system-tests (using Lago)
by Eyal Edri 03 Apr '16

03 Apr '16

A weekly reminder on open bugs & patches to keep track on blockers for oVirt testing: *Open bugs:* https://goo.gl/eUaPUY *Blocking bugs in potential:* - Bug 1311121 - oVirt system tests for 3.6 fail to deactivate domain on the first try - suspect race between OVF update and SPM stop (engine bug) - Patch provided, pending verification/testing on the tests *Open patches:* https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open - WIP:Fix the master suite runs - Improvements to storage domains add tests - Merged storage servers into one - Basic quote tests If you have any updates on the bugs/patches please reply with more info. -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 1

[2016-04-01] Weekly lago release, 0.14.0
by David Caro 01 Apr '16

01 Apr '16

Hi everyone! There's a new lago release, with mostly bug fixes but also some features, the most relevant are: * New default init file: Now you can use a LagoInitFile in the current dir as the default init configuration file (the usual init.json or init.yaml) so when running 'lago init' it will pick it up. * New '--set-current' option for the init command, to allow setting the prefix that's being created as the current prefix. For more detailed changelog you can go to: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt And the rpms, as usual are at: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

'Persistent' IP addresses for the hosts
by Yaniv Kaul 31 Mar '16

31 Mar '16

Can I ensure that in every run, host0 will get the same IP from the network IP pool? I'd imagine that they would, by the order of their init.json.in - but i'm not sure this is the case. TIA, Y.

3 8

Failing to run ovirt-system-tests using lago due to missing rpms
by Eyal Edri 29 Mar '16

29 Mar '16

Hi, Anyone encountered these errors when running 3.6 basic suite on RHEL 7.2? * STDERR Error: Package: ovirt-engine-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-iso-uploader >= 3.6.0 Error: Package: ovirt-engine-3.6.4.1-1.el6.noarch (localsync) Requires: slf4j >= 1.7.0 Available: slf4j-1.5.8-8.el6.noarch (base) slf4j = 1.5.8-8.el6 Error: Package: ovirt-engine-backend-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-host-deploy-java >= 1.4.0 Error: Package: ovirt-engine-vmconsole-proxy-helper-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-vmconsole-proxy Error: Package: ovirt-engine-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-image-uploader >= 3.6.0 Error: Package: ovirt-engine-backend-3.6.4.1-1.el6.noarch (localsync) Requires: ebay-cors-filter Error: Package: ovirt-engine-setup-base-3.6.4.1-1.el6.noarch (localsync) Requires: otopi >= 1.4.0 Error: Package: ovirt-engine-setup-plugin-ovirt-engine-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-engine-extension-aaa-jdbc Error: Package: ovirt-engine-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-engine-wildfly >= 8.2.1 Error: Package: ovirt-engine-setup-plugin-ovirt-engine-common-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-setup-lib Error: Package: ovirt-engine-backend-3.6.4.1-1.el6.noarch (localsync) Requires: vdsm-jsonrpc-java >= 1.1.8 Error: Package: ovirt-engine-backend-3.6.4.1-1.el6.noarch (localsync) Requires: ovirt-engine-extension-aaa-jdbc # [Thread-3] Deploy VM lago_basic_suite_3_6_engine: ERROR (in 0:01:59) Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 46, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 472, in _deploy_host host.name(), RuntimeError: /home/eedri/lago/ovirt-system-tests/basic_suite_3.6/deploy-scripts/setup_engine.sh failed with status 1 on lago_basic_suite_3_6_engine @ Deploy oVirt environment: ERROR (in 0:01:59) Error occured, aborting Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 248, in do_run self.cli_plugins[args.ovirtverb].do_run(args) File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run self._do_run(**vars(args)) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper return func(*args, prefix=prefix, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 174, in do_deploy prefix.deploy() File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 598, in wrapper func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 52, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 480, in deploy self._deploy_host, self.virt_env.get_vms().values() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 88, in invoke_in_parallel vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 46, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 472, in _deploy_host host.name(), RuntimeError: /home/eedri/lago/ovirt-system-tests/basic_suite_3.6/deploy-scripts/setup_engine.sh failed with status 1 on lago_basic_suite_3_6_engine -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

Reviews needed for ovirt-system-tests
by Eyal Edri 24 Mar '16

24 Mar '16

Hi, There has been progress with fixing a few issues on ovirt-system-tests and patches were submitted [1]. We would like to merge them ASAP so the ovirt tests for 3.6 and 4.0 can get one more step closer to working and being stable both for developers and for CI. Can you please review the pending patches so we can merge them TODAY? [1] https://gerrit.ovirt.org/#/c/53534 - needs a review https://gerrit.ovirt.org/#/c/53518/ - +1, probably can be merged. https://gerrit.ovirt.org/#/c/54738/ - +1, can get merged. https://gerrit.ovirt.org/#/c/54721/ - +1, can get merged https://gerrit.ovirt.org/#/c/51566/4 - +1, can get merged https://gerrit.ovirt.org/#/c/52681/ - not sure about this one -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[2016-21-03] Lago weekly meeting minutes
by David Caro 21 Mar '16

21 Mar '16

Discussions: --------------- * Ovirt system tests 3.6 status: - Removed template import related tests - Using el6-base and el7-base only templates - Running ok ~50% of the time, current issues: ~ Storage domain is not being deactivated https://bugzilla.redhat.com/show_bug.cgi?id=1311121 ~ Host does not get installed in time (15min) ~ Host vm gets into non-responsive (no ssh, no console, but vm is started) * Investigating about vagrant-lago https://docs.google.com/spreadsheets/d/1PKIlmhTd-vxEfs7JZC-mLkvWG4cBE1oOnxR… * Started using taiga for task prioritization: https://tree.taiga.io/project/david-caro-lago/kanba Last week action items: ------------------------ * [ovirt-system-tests] Open bug on moving 4.0 tests to 3.6 api code https://bugzilla.redhat.com/show_bug.cgi?id=1317610 * [ovirt-system-tests] Skip the template-relate * Open thread on vagrant integration - DONE -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

2 2

merge_repos is not picking the latest version of ioprocess?
by Yaniv Kaul 21 Mar '16

21 Mar '16

The latest I'm interested in is 0.15.0-4.el7: mini@ykaul-mini:~/ovirt-system-tests$ find /var/lib/lago/reposync -name "ioprocess*.rpm" /var/lib/lago/reposync/ovirt-3.6-el7/x86_64/ioprocess-0.15.0-2.el7.x86_64.rpm /var/lib/lago/reposync/epel-el7/i/ioprocess-0.15.0-4.el7.x86_64.rpm /var/lib/lago/reposync/epel-el6/ioprocess-0.14.0-1.el6.x86_64.rpm /var/lib/lago/reposync/ovirt-3.6-el6/x86_64/ioprocess-0.15.0-2.el6.x86_64.rpm But merge picks 0.15.0-2.el7 instead: 2016-03-21 13:14:11,458::merge_repos.py::merge::123::ovirtlago.merge_repos::DEBUG::Copying /var/lib/lago/reposync/ovirt-3.6-el7/x86_64/ioprocess-0.15.0-2.el7.x86_64.rpm to output directory 2016-03-21 13:14:12,224::merge_repos.py::merge::123::ovirtlago.merge_repos::DEBUG::Copying /var/lib/lago/reposync/ovirt-3.6-el7/noarch/python-ioprocess-0.15.0-2.el7.noarch.rpm to output directory Which causes the VM to take the right one from EPEL directly (and not via the localsync). Thoughts?

2 3