May 2016 - lago-devel - oVirt List Archives

[2016-05-30] Lago Weekly Meeting Minutes
by David Caro 30 May '16

30 May '16

oVirt system tests status: ========================== * 3.6 working * Master working * Image Failing due to appliance known issue (engine is not getting installed on the appliance, fabiand is on it) * New suite on it's way: Hosted engine (lev is working on it) **AI**: create a readthedocs page Last release items: ==================== Lago VM, VMProvider and Service plugins merged and released Next relase items: ================== Lago + repoman integration patch was merged, next lago release will require repoman to be available (ovirt ci-tools repo has the latest version) Installation howto effort ========================== * Installed lago on rhel 7.2 (some difficulties with the deps, probably mixed channels) * For some reason lago requires ovirtlago on start/stop Cheers!! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Weekly bug status for ovirt-system-tests (using Lago) [ 29/05/16 ]
by Eyal Edri 29 May '16

29 May '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *New training videos:* overview of the framework running on Lago - https://bluejeans.com/s/9FaJ/ <https://mojo.redhat.com/external-link.jspa?url=https%3A%2F%2Fbluejeans.com%…> hands on session on installing and running OST - https://bluejeans.com/s/9Iy0/ <https://mojo.redhat.com/external-link.jspa?url=https%3A%2F%2Fbluejeans.com%…> *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Work is being done to allow running OST locally per patch and on CI per merge.. more news to come soon *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open * Some patches are open on simplification and cleanup of the tests, should be merged this week. * This is part of preparing ovirt-system-tests to support running tests per commit. *Jenkins jobs: (STABLE!)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ 3.6 & master tests are stable and running on nightlies - please watch out for any new errors, they probably mean there is a regression! 3.6 NGN job is failing - Tolik/David - do we know what the issue is? *Recently merged patches:* https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:merged+since… > -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[2016-05-26] Weekly lago release 0.19.0
by David Caro 27 May '16

27 May '16

Hi everyone! A new lago release is out, this one adds the possibility to extend lago with three new plugins: * VMPlugin: Allows to modify each and every VM action, useful though when you want to extend the actions a VM can take, or customize some high level behavior (like defining a different set of artifacts to collect) We have lago.vm.DefaultVM (default), ovirtlago.virt.EngineVM (ovirt-engine), ovirtlago.virt.NodeVM (ovirt-node) and ovirtlago.virt.HostVM (ovirt-host). You can specify which one your domain belongs to with the key 'vm-type' and one of the above values, like: domains: my_domain: vm-type: ovirt-engine ... * VMProvider: Allows to implement a different set of low-level methods, like snapshot, or start/stop a vm, to allow to use different virtualization (or not) systems (only LocalLibvirtVMProvider (local-libvirt) and SSHVMProvider (ssh) are implemented right now). You can specify a custom provider for your domains with the 'vm-provider' key, like: domains: my_domain: vm-provider: local-libvirt ... * ServicePlugin: This is the class that implements the service-handling functions for a vm, starting/stopping a service, checking if the service is enabled, etc. Current implementations are SystemdService (systemd), SysVInitService (sysvinit) and SystemdContainerService (systemdcontainer). You can specify a custom service provider for your domain with the key 'service_provider': domains: my_domain: service_provider: systemd ... Another important fixed bug is that now copying a directory to a vm works :) For the latest docs and changelog you can go to the docs page: http://lago.readthedocs.io/en/0.19/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

[23-05-2016] Lago weekly meeting minutes
by David Caro 24 May '16

24 May '16

23-05-2016 - Agenda * oVirt system tests status: * 3.6 working (had some storage setup timeout on one run, rerunning to try to reproduce) * Master working * Image working * Cleanup and json->yaml refactor patches: * https://gerrit.ovirt.org/57664 * https://gerrit.ovirt.org/57699 * https://gerrit.ovirt.org/57700 * https://gerrit.ovirt.org/57702 * https://gerrit.ovirt.org/57703 * https://gerrit.ovirt.org/57704 * https://gerrit.ovirt.org/57705 * https://gerrit.ovirt.org/57731 * https://gerrit.ovirt.org/57663 * https://gerrit.ovirt.org/57662 * AI: create a readthedocs page * Lago Tutorial * Splitted the docs on two sections * Getting started: In depth explanation of the tutorial same steps * Tutorial: Per-feature explanation with examples * Lago demo * Totally different project, with it’s own maintenance * Very simple introduction on how to generate the lago repos (quick tutorial) * Lago + repoman integration: * https://github.com/lago-project/lago/pull/251 * Have to think a bit how to distribute repoman (currently it’s in ovirt ci-tools repo) * Lago VM, VMProvider and Service plugins * https://github.com/lago-project/lago/pull/260 * Still working out on the documentation but is functionally complete Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

2 2

Lago and Vagrant
by Eric Helms 23 May '16

23 May '16

I recall seeing some discussion of Lago vs Vagrant and how there could be some leverage between the two. We are currently using Lago in our release testing pipeline, however, for years now we've been using Vagrant for deploying developer and production hand-on testing environments. There are benefits to how we test and build environments to centering on a singular technology. Thus, I had a few questions I was hoping to get some thoughts on: 1) What was the outcome of the Lago vs Vagrant discussions? 2) How to reconcile the built up base of Vagrant use within our group with Lago (or how to sell switching to Lago)? 3) Do you see them as co-existing with different application points? To give an example of a disconnect where centering technology would benefit, we do an installation test currently spinning up a Lago based environment. Users testing our builds are using our existing Vagrant setup to spin up environments. However, we aren't testing the setup and configuration users are using and we end up double building the techniques for setting up environments. Thanks for any thoughts, Eric

3 2

Weekly bug status for ovirt-system-tests (using Lago) [ 22/05/16 ]
by Eyal Edri 22 May '16

22 May '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Failure on DirectLun (not merged yet), actually found a blocker by a new test added by Yaniv K, see open patches [1] *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open * Some patches are open on simplification and cleanup of the tests, should be merged this week. * This is part of preparing ovirt-system-tests to support running tests per commit. *Jenkins jobs: (STABLE!)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ - stable for the past days. 3.6 & master tests are stable and running on nightlies - please watch out for any new errors, they probably mean there is a regression! *Recently merged patches:* https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:merged+since… [1] *Bug 1337909* <https://bugzilla.redhat.com/show_bug.cgi?id=1337909> - NPE when trying to a direct LUN disk -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

New 0.18 version is out!
by David Caro 21 May '16

21 May '16

New lago 0.18 version is out, as the last one was done just a couple of days ago, there are not many changes, just a couple of bugfixes: * Expanding any vars on any url * Pinning down the version of yapf used For the latest docs: http://lago.readthedocs.io/en/0.18/ Cheers! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Lago Demo
by Gal Ben Haim 19 May '16

19 May '16

Hi All, I wrote a document which demonstrates a basic use case of Lago. The reader should follow the instructions in the document, and thus get a sense about Lago's capabilities and ease of use. This document should guide the user in his first experience, and this is why I'm not exposing the advanced capabilities of Lago (we will have another document for this). I would like you to try out the document, and write back a review. Link to the document: https://github.com/gbenhaim/Lago-Demo Thanks ! Gal Ben Haim gbenhaim(a)redhat.com RHEV DevOps

2 1

[2016-05-18] Weekly lago release
by David Caro 19 May '16

19 May '16

Hi everyone!! Last week's release was delayed a little bit due to the holidays, so we are releasing today, the new 0.17 version of lago comes with some interesting improvements: 9253f807: Added possibility to use env vars on disk path and url c7ff5190: Removed the repo verification code - replaced by retries and workarounds 61813f2e: Fix wait_for_ssh logic - now the times and tries when waiting for ssh make more sense, and have better logging 1ec94d55: Added LAGO_INITFILE_PATH env var when running init 86a8bde1: Setting LAGO_WORKDIR_PATH too on init 7a661154: Copy inside the prefix the deploy scripts 499c9912: Expanding the disks path on vm xml generation c5f6cafb: Using LAGO_PREFIX_DIR for the disk images too So the most change is the possibility to use env variables in the init file, and the addition of the env vars: * LAGO_PREFIX_DIR * LAGO_WORKDIR_PATH * LAGO_INITFILE_PATH (only when running lago init) That allow you to better specify relative paths to the different files (disks, scripts, urls...). And that allow to keep the workdir functional even after renaming them (mv old_workdir new_workdir). For the updated docs and changelog: http://lago.readthedocs.io/en/0.17/ For the rpms: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Can't run tests with latest Lago & ovirt-system-tests
by Yaniv Kaul 17 May '16

17 May '16

I've updated to the latest ovirt-system-tests, and apparently commit ed759a62 was breaking things to me. So I've updated Lago to latest (from http://resources.ovirt.org/repos/lago/unstable/0.0/latest/rpm/fc23/noarch/ ) - which is 0.16.12-1 now, and now things break differently, possibly by Lago commit ba6dda1e92faf26bc6842a78143ced20ae0546ff @ Create prefix internal repo: ERROR (in 0:00:00) Error occured, aborting Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 248, in do_run self.cli_plugins[args.ovirtverb].do_run(args) File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run self._do_run(**vars(args)) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 595, in wrapper return func(*args, prefix=prefix, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 166, in do_ovirt_reposetup skip_sync=skip_sync, File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 615, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 423, in prepare_repo _sync_rpm_repository(rpm_repo, reposync_yum_config, repos) File "/usr/lib/python2.7/site-packages/ovirtlago/__init__.py", line 105, in _sync_rpm_repository ret, out, _ = utils.run_command(reposync_command) AttributeError: 'module' object has no attribute 'run_command Any idea?

3 3

[2016-05-09] Lago weekly meeting minutes
by David Caro 09 May '16

09 May '16

Agenda ======== * oVirt system tests status: - 3.6 working - Master seems to fail on add-hosts still (investigating) * Merged workaround for reposync issues * Lago tutorial on it’s way - Using the unstable/latest lago (with the new patches) - Latest version: http://jenkins.ovirt.org/job/lago_master_check-patch-el7-x86_64/51/artifact… * Gal released 0.16.0 last week, kudos! * Merged patch to copy the deploy scripts inside the prefix on init, that takes us a step closer to relocatable prefixes (and exportable prefixes too) Let's have a great week! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

[05-05-2016] New lago release - 0.16.0
by Gal Ben Haim 05 May '16

05 May '16

Hi everyone! There's a new lago release with version 0.16.0 ready for you to upgrade! One important change in version 0.16.0 is the support for the following environment variables: $LAGO_PREFIX_PATH = .lago/current $LAGO_WORKDIR_PATH = .lago it's now highly recommended to use those variables when configuring the path to the deployment scripts. Here are the other changes: * Wed May 04 2016 David Caro <dcaroest(a)redhat.com> - 0.16.0 45764a95: Fixed small typo in a string * Wed May 04 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.5 66dec63b: Merge pull request #231 from lago-project/fix_el7_dirlock ccf6dca4: Added lock class for el7 locking 9a6e370c: Use lago.utils.LockFile instead of lockfile's one 98322f01: Removed thread pool on reposync 167c8b7d: Moved some code out of a with block db836eda: Yapf fixes * Tue May 03 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.4 0e535676: Merge pull request #230 from lago-project/add_prefix_env_vars 17e9be31: tests:Modified deploy funct tests to use vars 1440adee: Add some lago extra vars to the env 577c442a: Expand vars and user contraptions on deploy scripts * Tue May 03 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.3 a1a57978: Merge pull request #229 from lago-project/extra_gitignore 2a5a3359: Added extra ignores to .gitignore * Fri Apr 29 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.2 25c91601: Merge pull request #227 from lago-project/fix_unhandled_exception 8eeb43cc: Fixed wrong error log on reposetup * Fri Apr 29 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.1 5c5bc197: Merge pull request #226 from lago-project/fix_repoverify_check c4380a9a: Fixed repoverify not behaving like reposync d8c9ac55: Added new deps for the tests * Sat Apr 23 2016 Lago CI bot <dcaroest+cibot(a)redhat.com> - 0.15.0 4b98d690: Merge pull request #223 from lago-project/try_cleaning_reposync_caches_on_first_failure f87101ee: Reposync first retry with clean caches Enjoy! Gal Ben Haim gbenhaim(a)redhat.com RHEV-CI Team

2 1

Weekly bug status for ovirt-system-tests (using Lago) [ 01/05/16 ]
by Eyal Edri 02 May '16

02 May '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Improvements on the host install issue, some env are not reproducing anymore but not sure we're 100% clear yet. *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open * There are quite a few open, some are just temp for helping with the debug of 'stuck in installing state' *Jenkins jobs: (failing on known issues)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ *Recently merged patches:* commit 211e475c004a2635896e528cd3aed6cd7d0dc5fd Author: Sandro Bonazzola <sbonazzo(a)redhat.com> Date: Thu Apr 28 16:44:50 2016 +0200 common: yum-repos: update glusterfs repos - dropped Gluster community repo for Fedora23: the repo is empty and Gluster community is maintaining Fedora packages within Fedora repositories. For reference: http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fedora/What_About_… - on EL7 moved from Gluster community repo to CentOS Storage SIG repositories: we've been advised by Gluster community that their packaging effort is focused on CentOS Storage SIG and CentOS infrastructure is more reliable. Change-Id: I4d2c752be825fbbb9035222845728d10c5dd288c Signed-off-by: Sandro Bonazzola <sbonazzo(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit 19c102110279975d274c32477cc28c7c29e75fbc Author: David Caro <dcaroest(a)redhat.com> Date: Wed Apr 20 21:10:37 2016 +0200 Fixed answerfile for master A new question was added to the setup Change-Id: I6be958a70d67126d8067ff1855018125623b10c1 ---------------------------------------------------------------------------------------------------------- commit 5829862d17e92f91cfd04e7b9d5549cd775c800c Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Mon Apr 25 14:06:27 2016 +0300 Add 'device=eth0' to EL7 deploy scripts. Apparently, /etc/sysconfig/network-scripts/ifcfg-eth0 lacks it on EL7 images. This works with NetworkManager, but not without it. When something goes bad, VDSM fails to rollback properly as its missing. For now, add it. Change-Id: I1c4ca07499a3947022e6f943cf05842ee1def408 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit ddb42782968e5c6d411cdc747503fd90bed249d7 Author: David Caro <dcaroest(a)redhat.com> Date: Tue Mar 29 18:53:42 2016 +0200 Fix the master suite runs This is just copying over and adapting a bit (just so it works) the 3.6 suite into the master suite. Only for the 3.6 api, the newapi tests will be added later Still affected by the storage deactivation issues Change-Id: I55781a5d5a746aaf81e403eeeee40880beacb8a0 Signed-off-by: David Caro <dcaroest(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit c7d50cf8aebd6b566ae5c6ee8d6e3b8ec24ffc2b Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Tue Apr 26 13:57:43 2016 +0300 Remove snapshot and reverting as part of the ovirt-system-tests I currently did not find a use case for it. It slows down the tests and causes (real? false?) issues with SDs. Mainly their activation and deactivation. It dramatically reduced the time to run the whole suite for me. 3-4 minutes on my desktop, and >10 minutes on my laptop. Change-Id: I9f7e509220c83805df78abb94b0ddc390c4a7f54 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0