April 2016 - lago-devel - oVirt List Archives

[2016-04-26] Lago weekly meeting minutes
by David Caro 26 Apr '16

26 Apr '16

Agenda ======== * Ovirt system tests 3.6 status: Gal testing 3.6.6 it locally, it’s expected to pass * Ovirt system tests master status: Affected by: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 https://bugzilla.redhat.com/show_bug.cgi?id=1329317 * Had some issues with jenkins.phx.ovirt.org last week, fixed already * Currently reposetup not working on el7 systems, using old version of lockfile https://github.com/lago-project/lago/issues/225 * Released 0.15 with some improvements: Rotating the dest dir on collect Generic ‘lago deploy’ command More verbose reposetup command In progress: * Lago tutorial * Lago init file spec * Lago high level desing doc * virt.py fixes Action items: * dcaro: fix the el7 lockfile issue * dcaro: review the tutorial to unblock gbenhaim Thanks!! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

2 2

Weekly bug status for ovirt-system-tests (using Lago) [ 24/04/16 ]
by Eyal Edri 24 Apr '16

24 Apr '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Still suffering from host add failures (stuck in installing, David has been debugging it constantly with Piotr last week) *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open *Jenkins jobs: (failing on known issues)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ *Recently merged patches:* (Thanks again to Yaniv K!) Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Wed Apr 20 23:51:55 2016 +0300 More accurate 3.6 repo 1. With this repo, both engine and hosts can be installed, without any external access, I hope. 2. Now we can test latest 3.6, not the released one. Change-Id: I9066af4f5d92e275586c29c65fe7ef7500fe5c20 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Tue Apr 12 23:00:16 2016 +0300 Fixes for FC23 local repo setup: 1. The interface may not be eth0. The new code hopefully work with any if name. Depends that the interface is the first, but let's hope it is. 2. Add 'cost=1' to ensure the local repo is preferred over other repos. Change-Id: If846d7bacdf517d1342464c210359e5a0e8cad6e Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Wed Mar 30 22:46:39 2016 +0300 Basic quota tests Add quota and set it to audit mode, not more than that. Change-Id: Ia35e9b2b65c01cdcca8da7f87ed7a38f8dc2bb17 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Thu Apr 14 01:04:28 2016 +0300 Fix mount / path issues with storage_nfs_el7 provisioning script Clearly was not tested well (because of the host installation, but that's an execuse). This patch fixes it, tested on 3.6 with adding all domains, finally. It also again simplifies it - but removes the option of not mounting the export mount. Note that the ISO mount is not completely done as well, I think. Will attend to in a follow up patch. Based on David's request, the whole setup is split into functions. Tested (properly) with 3.6, including the TEMPLATES domain attachment. (The test for it should still be excluded, IMHO). Change-Id: I261f3d3f3b3647cbbc506913478ffc6c36594519 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> -------------------------------------------------------------------------------------------------------------------- Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Sun Apr 10 17:10:16 2016 +0300 Simplification and enhancements to iSCSI and NFS servers deployment Removed a lot of not needed code from the setup. Mainly, just straightforward using the disks (in case of NFS), no need for LVM stuff. Disabled firewall, enabled 'noop' scheduler. Tested with both master and 3.6. Never got to test fully master, blocked by a host installation bug. 3.6 passed well. Change-Id: I7ec630ba5fa62529421b8e8f9aec993e7cdc84b6 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> Please comment / update on any additional info you have on this thread, Eyal.

1 0

Weekly lago release 0.15
by David Caro 23 Apr '16

23 Apr '16

Hi everyone!! It's been a couple weeks without releases, but we are back and 100% committed to keep up the pace. On this week's release: * Several reposetup command improvements (better logs, friendlier retry, ...) * Collect now rotates the destination folder if it exists * Skip vm bootstrap, what allows faster functional tests * Generic deploy subcommand, now you can use the deploy command with lago, just add a 'deploy-scripts' list entry to the 'metadata' key on any vm/domain definition, with a list of scripts and they will be run on 'lago deploy' For the full changelog, as always, you can go to: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt And for the latest rpms: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Can I define somewhere the collect artifacts?
by Yaniv Kaul 14 Apr '16

14 Apr '16

I'm extending the VDSM functional tests, and saving files under /tmp/vdsm-results/ - can I simply ask somehow to get all files from there, perhaps via the init.json.ini file would be the right place to add this? TIA, Y.

3 6

Lago + ovirt-system-tests run fail on collecting logs
by Eyal Edri 13 Apr '16

13 Apr '16

After running: ./run_suite.sh --cleanup basic_suite_3.6 ./run_suite.sh basic_suite_3.6 The tests failed at [1] Does it mean the cleanup stage failed? why would log collection fail on a duplicate file? I think it will be beneficial to support multiple logs and rotate it like we do in ovirt for e.g, Since a developer might run a few times without cleanning up the env. Do we have an open issue for it? , cause I didn't find anything similar. [1]. @ Collect artifacts: ERROR (in 0:00:00) Error occured, aborting Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 248, in do_run self.cli_plugins[args.ovirtverb].do_run(args) File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run self._do_run(**vars(args)) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper return func(*args, prefix=prefix, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 217, in do_ovirt_collect prefix.collect_artifacts(output) File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 598, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 928, in collect_artifacts os.makedirs(output_dir) File "/usr/lib64/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: '/home/eedri/lago/ovirt-system-tests/test_logs/basic_suite_3.6/post-001_initialize_engine.py' -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

3 7

configuring path for reposync
by Eyal Edri 12 Apr '16

12 Apr '16

Hi, When installing lago you get these default values in : [lago] log_level = debug template_store = /var/lib/lago/store template_repos = /var/lib/lago/repos First, what is the diff between the 2? store holds the templates, what is repos? Also, when running ovirt-system-tests i see under /var/lib/lago/reposync all the repos sycned, is there away to define this path to be somewhere else? is it in the ovirt-system-tests config (if so why is it under lago dir) if not, shouldn't it be in lago.conf? -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 2

repeating libvirt exception in vdsm check-merged job
by Yaniv Bronheim 11 Apr '16

11 Apr '16

Hi, we keep getting the following error in vdsm automation *12:16:47* + lago start vdsm_functional_tests_host-fc23*12:16:49* current session does not belong to lago group.*12:16:49* @ Start specified VMs: *12:16:49* # Start nets: *12:16:49* * Create network vdsm_functional_tests_lago: *12:16:50* * Create network vdsm_functional_tests_lago: Success (in 0:00:00)*12:16:50* # Start nets: Success (in 0:00:00)*12:16:50* # Start vms: *12:16:50* * Starting VM vdsm_functional_tests_host-fc23: *12:16:50* libvirt: QEMU Driver error : Cannot check QEMU binary /usr/libexec/qemu-kvm: No such file or directory*12:16:50* * Starting VM vdsm_functional_tests_host-fc23: ERROR (in 0:00:00)*12:16:50* # Start vms: ERROR (in 0:00:00)*12:16:50* # Destroy network vdsm_functional_tests_lago: *12:16:50* # Destroy network vdsm_functional_tests_lago: ERROR (in 0:00:00)*12:16:50* @ Start specified VMs: ERROR (in 0:00:00)*12:16:50* Error occured, aborting*12:16:50* Traceback (most recent call last):*12:16:50* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 661, in main*12:16:50* cli_plugins[args.verb].do_run(args)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run*12:16:50* self._do_run(**vars(args))*12:16:50* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper*12:16:50* return func(*args, **kwargs)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper*12:16:50* return func(*args, prefix=prefix, **kwargs)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 234, in do_start*12:16:50* prefix.start(vm_names=vm_names)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 778, in start*12:16:50* self.virt_env.start(vm_names=vm_names)*12:16:50* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 193, in start*12:16:50* vm.start()*12:16:50* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 910, in start*12:16:50* self._env.libvirt_con.createXML(self._libvirt_xml())*12:16:50* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3611, in createXML*12:16:50* if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)*12:16:50* libvirtError: Cannot check QEMU binary /usr/libexec/qemu-kvm: No such file or directory all jobs fail today with the same exception - http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/53/console . any idea why qemu-kvm is missing? -- *Yaniv Bronhaim.*

3 5

libvirt: QEMU Driver error : Guest agent is not responding: QEMU guest agent is not available due to an error
by Eyal Edri 10 Apr '16

10 Apr '16

Anyone run lago before on RHEL 7.2? Its possible we're hitting [1] with libvirt-1.2.17 https://bugzilla.redhat.com/show_bug.cgi?id=1090551 -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

reminder on lago khanban board
by Eyal Edri 10 Apr '16

10 Apr '16

FYI, Lago started using a khanban board [1] to prioritize and visualize the current issues which are being worked on, since it very hard to use GitHub to monitor and prioritize issues. Taiga is an open source project and was chosen (rather than Trello for e.g) because it had already integration with GitHub for syncing automatically issues to the board and moving statuses. If you want to ask for a specific issue prioritization, please use this list to communicate it, We will review issues on a weekly basis and the WIP will be shown on the board. [1] https://tree.taiga.io/project/david-caro-lago/kanban -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

Weekly bug status for ovirt-system-tests (using Lago) [ 10/04/16 ]
by Eyal Edri 10 Apr '16

10 Apr '16

A weekly reminder on open bugs & patches to keep track on blockers for oVirt testing: *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Still suffering from add host failures, pending a fix from network team: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 *Recently merged:* (Thanks Yaniv K!) - Multiple changes to deployment script https://gerrit.ovirt.org/#/c/55695/ - Fix IP Address of engine *https://gerrit.ovirt.org/#/c/55903/ <https://gerrit.ovirt.org/#/c/55903/>* - Improvements to the storage domain add tests - https://gerrit.ovirt.org/#/c/55233/ *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open - *Basic quota tests: https://gerrit.ovirt.org/#/c/55514/ <https://gerrit.ovirt.org/#/c/55514/>* - *Switched between hot-add disk and hot-add NIC tests * https://gerrit.ovirt.org/#/c/55740/ - (Attempt to reduce chance of hitting the deactivate storage bug, though the fix of this bug might be in already, so not sure its needed, Yaniv?) - WIP: Fix master (4.0) suite, still in progress https://gerrit.ovirt.org/#/c/55425/ - Merging the storage VMs into one, still in progress: https://gerrit.ovirt.org/#/c/54957/ If you have any updates on the bugs/patches please reply with more info. -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 1

In which order repos are searched for RPMs?
by Yaniv Kaul 05 Apr '16

05 Apr '16

If I have the RPM on the 'localsync' repo as well as externally (say, from CentOS 7 updates), which repo will it be taken from? - Simple solution: sync all that you need and disable the 'updates' repo. I'm in favor. Same for the 'base' repo and any other repo. Essentially, all that the VM needs should be in the 'localsync' repo. - Simple solution #2 : rename 'localsync' to '*a*localsync' - just to make sure it is the first looked at... Before: [root@lago_basic_suite_3_6_engine ~]# egrep "Updated|Installed" /var/log/yum.log | awk -F\: '{print $NF}' |xargs yum info |grep "From repo" |egrep -c "base|updates" 87 After the change: [root@lago_basic_suite_3_6_engine ~]# egrep "Updated|Installed" /var/log/yum.log | awk -F\: '{print $NF}' |xargs yum info |grep "From repo" |egrep -c "base|updates" 1

2 1

[2016-04-04] Lago weekly meeting minutes
by David Caro 04 Apr '16

04 Apr '16

Hi everyone, short meeting this week: Discussions: ============= * Ovirt system tests 3.6 status: - 3.6 tests running on jenkins (currently failing to add hosts) * Ovirt system tests master status: - Affected by: https://bugzilla.redhat.com/show_bug.cgi?id=1322257 * Investigating about vagrant-lago - https://docs.google.com/spreadsheets/d/1PKIlmhTd-vxEfs7JZC-mLkvWG4cBE1oOnxR… - Have to schedule a meeting to push this forward Action Items ============= * Migrate to jenkins.phx.ovirt.orgt -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Weekly bug status for ovirt-system-tests (using Lago)
by Eyal Edri 03 Apr '16

03 Apr '16

A weekly reminder on open bugs & patches to keep track on blockers for oVirt testing: *Open bugs:* https://goo.gl/eUaPUY *Blocking bugs in potential:* - Bug 1311121 - oVirt system tests for 3.6 fail to deactivate domain on the first try - suspect race between OVF update and SPM stop (engine bug) - Patch provided, pending verification/testing on the tests *Open patches:* https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open - WIP:Fix the master suite runs - Improvements to storage domains add tests - Merged storage servers into one - Basic quote tests If you have any updates on the bugs/patches please reply with more info. -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

2 1

[2016-04-01] Weekly lago release, 0.14.0
by David Caro 01 Apr '16

01 Apr '16

Hi everyone! There's a new lago release, with mostly bug fixes but also some features, the most relevant are: * New default init file: Now you can use a LagoInitFile in the current dir as the default init configuration file (the usual init.json or init.yaml) so when running 'lago init' it will pick it up. * New '--set-current' option for the init command, to allow setting the prefix that's being created as the current prefix. For more detailed changelog you can go to: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt And the rpms, as usual are at: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0