- lago-devel - oVirt List Archives

time keeping
by Yaniv Kaul 14 Mar '16

14 Mar '16

I'm concerned with two (different) time keeping issues: 1. Timezone - we need to ensure all VMs have the same timezone (At least by default). Best option would be to edit the deploy scripts and set the timezone to the host timezone? should be fairly easy to add @TIMEZONE@ variable to the deploy script in a similar manner to what we do today in the domain XML. It'll be in a command like 'ln -s /usr/share/zoneinfo/@TIMEZONE@ /etc/localtime 2. Time drifting - I have a feeling there's a time drift when we do all the freeze-thaw and snap/revert... Perhaps we should run 'ntpdate' after such actions?

2 2

[11-03-2016] Weekly lago release v0.12.0
by David Caro 11 Mar '16

11 Mar '16

Hi everyone! This week we introduced one big change in how lago handles prefixes, we introduced the 'workdir' concept, that is, a directory that will store one or more prefixes, let me show with an example. Imagine that you have an init.yaml that creates an environment with 1 vm, you can run init like this: lago --prefix-name one_vm init init.yaml That will create a workdir at $PWD/.lago with one prefix inside named one_vm and a soft link named 'current' that points to it, so any further command will run on that one_vm prefix: lago status # shows the one_vm prefix path Until there nothing new, but, now you can create more prefixes, if for example, if you have another init config file named two_vms.yaml, you can create another prefix inside that same workdir with: lago --prefix-name two_vms init two_vms.yaml That will add a new prefix to $PWD/.lago, it will not update the current link though, so any command will still run by default on the previous prefix, but you can change the current prefix with: lago set-current two_vms That will make any command that does not specify the --prefix-name option to use that one by default. NOTE: lago init will create only workdirs from now on, but you can still use the old prefixes if you have any around, it will autodetect them as one or the other. Some other changes: Now it will show the libvirt status of the domains Remove the memory balloon device. We don't use it. Ignore zero sized RPM files when merging repos. Added ignore-warnings option To see the full changelog: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt And as usual, you can get the rpms from: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Cheers! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Fwd: [ovirt-devel] [VDSM] Moving to py.test
by Yaniv Kaul 10 Mar '16

10 Mar '16

We probably want to move to it in ovirt-system-tests as well, to align with both VDSM and Lago itself, no? ---------- Forwarded message ---------- From: Nir Soffer <nsoffer(a)redhat.com> Date: Thu, Mar 10, 2016 at 3:15 PM Subject: [ovirt-devel] [VDSM] Moving to py.test To: devel <devel(a)ovirt.org>, Yaniv Bronheim <ybronhei(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com>, Piotr Kliczewski <pkliczew(a)redhat.com>, Francesco Romani <fromani(a)redhat.com>, Adam Litke <alitke(a)redhat.com> Hi all, In the last vdsm call we discussed moving to py.test. Here is how to run the current tests with py.test: cd tests PYTHONPATH=../lib:../vdsm py.test -v *_test.py *_tests.py *Tests.py network Currently we have some failures and errors: 8 failed, 1662 passed, 101 skipped, 14 pytest-warnings, 10 error in 86.05 seconds Some py.test nice tricks: Finding the 10 slowest tests: PYTHONPATH=../lib:../vdsm py.test -v --durations=10 vmTests.py *_test.py *_tests.py *Tests.py network Checking code coverage: PYTHONPATH=../lib:../vdsm py.test -v --cov=../vdsm --cov=vdsm *_test.py *_tests.py *Tests.py network Once we finish moving vdsm code to lib/vdsm, renaming the tests to *_test.py, and add pytest.ini, running the test will be simply: py.test Once all the tests pass, we can: - Drop all the special asserts (assertXXX), py.test uses plain asserts https://pytest.org/latest/assert.html#asserting-with-the-assert-statement - Get better error reports https://pytest.org/latest/example/reportingdemo.html - Remove unneeded classes, or inherit from any class https://pytest.org/latest/getting-started.html - Replace @permutations with more powerfull @pytest.mark.parametrize https://pytest.org/latest/parametrize.html#parametrize-basics - Replace slowtest and stresstest with @pytest.mark https://pytest.org/latest/mark.html - Replace monkeypatch.py with simpler and more powerfull pytest monkeypatch funcarg https://pytest.org/latest/monkeypatch.html#monkeypatching-mocking-modules-a… - Replace NamedTemporaryDir with pytest tmpdir funcarg https://pytest.org/latest/getting-started.html#going-functional-requesting-… - Simplify complex context managers with pytest fixtures https://pytest.org/latest/example/simple.html - Drop testrunner.py, most of testlib.py, some of testValidation.py, run_tests*.sh - And more Nir _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

2 1

Bridge Networking
by Eric Helms 08 Mar '16

08 Mar '16

I couldn't find anything in the documentation so asking here, is there a way to setup bridge networking for a set of VMs to get them public IPs? Thanks, Eric

3 7

Fwd: [VDSM] check_merged failures
by Yaniv Bronheim 08 Mar '16

08 Mar '16

forwarding to lago devels.. shouldn't be any issues with the tests. nothing changed since last time it succeeded ---------- Forwarded message ---------- From: Nir Soffer <nsoffer(a)redhat.com> Date: Tue, Mar 8, 2016 at 4:34 PM Subject: [VDSM] check_merged failures To: devel <devel(a)ovirt.org>, Yaniv Bronheim <ybronhei(a)redhat.com> Hi all, The new check_merged job is failing randomly. I have see this error today: 14:20:16 @ Start specified VMs: 14:20:16 # Start nets: 14:20:16 * Create network vdsm_functional_tests_lago: 14:20:16 libvirt: error : internal error: Network is already in use by interface 7f0c-bc616ef 14:20:16 * Create network vdsm_functional_tests_lago: ERROR (in 0:00:00) 14:20:16 # Start nets: ERROR (in 0:00:00) 14:20:16 @ Start specified VMs: ERROR (in 0:00:00) 14:20:16 Error occured, aborting 14:20:16 Traceback (most recent call last): 14:20:16 File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 539, in main 14:20:16 cli_plugins[args.verb].do_run(args) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in do_run 14:20:16 self._do_run(**vars(args)) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 132, in wrapper 14:20:16 return func(*args, prefix=lago_prefix.Prefix(prefix_path), **kwargs) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 141, in wrapper 14:20:16 return func(*args, prefix=prefix, **kwargs) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 191, in do_start 14:20:16 prefix.start(vm_names=vm_names) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 819, in start 14:20:16 self.virt_env.start(vm_names=vm_names) 14:20:16 File "/usr/lib/python2.7/site-packages/lago/virt.py", line 187, in start 14:20:16 net.start() 14:20:16 File "/usr/lib/python2.7/site-packages/lago/virt.py", line 354, in start 14:20:16 self._env.libvirt_con.networkCreateXML(self._libvirt_xml()) 14:20:16 File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4093, in networkCreateXML 14:20:16 if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self) 14:20:16 libvirtError: internal error: Network is already in use by interface 7f0c-bc616ef Hopefully someone is owning the infrastructure and can look into this. Nir -- *Yaniv Bronhaim.*

2 3

[2016-03-07] Lago weekly meeting minutes
by David Caro 07 Mar '16

07 Mar '16

== Discussions: * Considering taiga (https://tree.taiga.io/project/david-caro-lago/kanban) as task board (has some github integration, looking) * Check trello first * Move to vagrant/oh-my-vagrant * It turns out that oh-my-vagrant has some kind of repo crawler that can make up for the lack of an open-source vagrant boxes repo service * No layered images * The size of a vagrant box is ~2 times the size of our current images * Completely replace current virt code? Totally replace lago? * Vitali (student) proposed to write the code as project * Written in ruby (some python libs) * https://pypi.python.org/pypi/python-vagrant * Change from json to yaml - DONE * Multiple ‘prefixes’ in the same workdir workflows, now that .lago is the default prefix path (see * Should we use multiple envs per prefix? or one prefix per env? * Approved * https://github.com/lago-project/lago/pull/157 * Add default init file in the current dir * https://github.com/lago-project/lago/issues/165 * Add multiple prefixes definitions on the init config file * https://github.com/lago-project/lago/issues/164 * Add multishell (cssh/dssh/tmux/screen) and tailing * https://github.com/lago-project/lago/issues/163 == Last week action items: * Start a thread to discuss the reposync drop: * http://lists.ovirt.org/pipermail/lago-devel/Week-of-Mon-20160307/000027.html * Open github issues for all the agreed features: * https://github.com/lago-project/lago/issues/149 * https://github.com/lago-project/lago/issues/148 * https://github.com/lago-project/lago/issues/147 * Change the ovirt 4.0 publisher to publish the 3.6 sdk (until 4.0 is ready, then publish both on 4.0 repo) - DONE == Action items: * add --set-current option to init * https://github.com/lago-project/lago/issues/162 * send bugzilla about nfs issues on 3.6 ovirt system tests * https://bugzilla.redhat.com/show_bug.cgi?id=1311121 * Open discussion on change/integration with vagrant * Check trello for issue management * It requires license to use the github integration :( -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Drop reposync command
by David Caro 07 Mar '16

07 Mar '16

Hi everyone, I'd like to start a discussion about dropping (at least temporarily) the reposync command. The reposync command is used in lago for two things: * Downloads locally all the packages defined in the repo conf passed (yum config file format) * Creates an internal_repo dir in the prefix and copies over all the packages that match the distros used in the prefix The advantage is that once synced, this allows installing any packages on the vms without having connection to the internet. The downside is that right now, it downloads everything, needed or not, so it takes a lot of space and time the first run to cache all the packages. For that reason on the ovirt system tests we removed the epel/centos base repos from the reposync (it was ~20G) but as that already makes the tests require an internet connection, makes the reposync command obsolete (it is caching only some part of the packages). So the idea is to drop the reposync command until we have a better alternative (maybe a small local proxy that caches everything on the first run or something similar) -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

3 2

[04-03-2016] Weekly lago release - 0.11.0
by David Caro 04 Mar '16

04 Mar '16

Hello everyone! Mostly maintenance week this one, almost duplicated the number of tests, and removed more than 2000 lines of unused stuff. But still a couple features got in: * new --prefix-path option, that will allow you to override the automatic search for a prefix, and allow you to run a lago command for a prefix from anywhere * Added YAML support for the init configuration files, now it's a bit more easy to write the init files (you can even reuse blocks of yaml with yaml anchors! https://en.wikipedia.org/wiki/YAML#Repeated_nodes) * Added a new 'destroy' command, that will run any stop/cleanup in the prefix and finally rm -rf it For the full changelog go: http://resources.ovirt.org/repos/lago/stable/0.0/CHANGELOG_0.0.txt You can get the new rpms from: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro(a)redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605

1 0

Was the check for 'lago' group membership removed?
by Yaniv Kaul 02 Mar '16

02 Mar '16

I uninstalled and reinstalled lago, which made the lago group disappear and reapper - and my user was not part of it. Then running the tests failed, as I could not write the lock file to /var/lib/lago/... Added a bit of code to the exception to see why: File "/usr/lib/python2.7/site-packages/lockfile/linklockfile.py", line 20, in acquire raise LockFailed("Failed to create %s.\nCode: %d\nReason: %s\n" % (self.unique_name, e.errno, e.strerror)) LockFailed: Failed to create /var/lib/lago/store/ykaul-mini.tlv.redhat.com-fa517700.1377-7176078113883238881. Code: 13 Reason: Permission denied And indeed, it was because I was not part of the lago group. Adding, logging out and in again solved it - but shouldn't there be such warning ? (I remember there was in the past). TIA, Y.

2 1

'make check' fails due to yaml
by Yaniv Kaul 02 Mar '16

02 Mar '16

After today's (?) changes: tests/unit/test_config.py::test_nonexistent_throws PASSED ===================================================================================================== ERRORS ====================================================================================================== ___________________________________________________________________________________ ERROR collecting tests/unit/test_dirlock.py ___________________________________________________________________________________ tests/unit/test_dirlock.py:27: in <module> from lago import dirlock lago/dirlock.py:25: in <module> import utils lago/utils.py:36: in <module> import yaml E ImportError: No module named yaml _________________________________________________________________________________ ERROR collecting tests/unit/lago/test_prefix.py _________________________________________________________________________________ tests/unit/lago/test_prefix.py:6: in <module> from lago import prefix lago/prefix.py:36: in <module> import subnet_lease lago/subnet_lease.py:45: in <module> import utils lago/utils.py:36: in <module> import yaml E ImportError: No module named yaml _________________________________________________________________________________ ERROR collecting tests/unit/lago/test_utils.py __________________________________________________________________________________ tests/unit/lago/test_utils.py:2: in <module> import yaml E ImportError: No module named yaml ======================================================================================== 7 passed, 3 error in 0.08 seconds ======================================================================================== Makefile:52: recipe for target 'check-local' failed make: *** [check-local] Error 1 I'm on: commit 5178f92f435791a778d4fbd801baab532819e7b0 Merge: 5deacae 79bb6ca Author: Lago CI bot <dcaroest+cibot(a)redhat.com> Date: Tue Mar 1 16:38:23 2016 +0100 Merge pull request #151 from lago-project/yaml_init_definition Added yaml support for the init config file

3 3