- Devel - oVirt List Archives

[ACTION REQUIRED] oVirt 4.0.5 RC2 build starting
by Sandro Bonazzola 12 Oct '16

12 Oct '16

Fyi oVirt products maintainers, An oVirt build for an official release is going to start right now. If you're a maintainer for any of the projects included in oVirt distribution and you have changes in your package ready to be released please: - bump version and release to be GA ready - tag your release within git (implies a GitHub Release to be automatically created) - build your packages within jenkins / koji / copr / whatever - verify all bugs on MODIFIED have target release and target milestone set. - add your builds to releng-tools/releases/ovirt-4.0.5_rc2.conf within releng-tools project -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

1 0

[vdsm] exploring a possible integration between collectd and Vdsm
by Francesco Romani 12 Oct '16

12 Oct '16

Hi all, In the last 2.5 days I was exploring if and how we can integrate collectd and Vdsm. The final picture could look like: 1. collectd does all the monitoring and reporting currently Vdsm does 2. Engine consumes data from collectd 3. Vdsm consumes *notifications* from collectd - for few but important tasks like Drive high water mark monitoring Benefits (aka: why to bother?): 1. less code in Vdsm / long-awaited modularization of Vdsm 2. better integration with the system, reuse of well-known components 3. more flexibility in monitoring/reporting: collectd is special purpose existing solution 4. faster, more scalable operation because all the monitoring can be done in C At first glance, Collectd seems to have all the tools we need. 1. A plugin interface (https://collectd.org/wiki/index.php/Plugin_architecture and https://collectd.org/wiki/index.php/Table_of_Plugins) 2. Support for notifications and thresholds (https://collectd.org/wiki/index.php/Notifications_and_thresholds) 3. a libvirt plugin https://collectd.org/wiki/index.php/Plugin:virt So, the picture is like 1. we start requiring collectd as dependency of Vdsm 2. we either configure it appropriately (collectd support config drop-ins: /etc/collectd.d) or we document our requirements (or both) 3. collectd monitors the hosts and libvirt 4. Engine polls collectd 5. Vdsm listens from notifications Should libvirt deliver us the event we need (see https://bugzilla.redhat.com/show_bug.cgi?id=1181659) we can just stop using collectd notifications, everything else works as previously. Challenges: 1. Collectd does NOT consider the plugin API stable (https://collectd.org/wiki/index.php/Plugin_architecture#The_interface.27s_s…) so the plugins should be inclueded in the main tree, much like the modules of the linux kernel Worth mentioning that the plugin API itself has a good deal of rough edges. we will need to maintain this plugin ourselves, *and* we need to maintain our thin API layer, to make sure the plugin loads and works with recent versions of collectd. 2. the virt plugin is out of date, doesn't report some data we need: see https://github.com/collectd/collectd/issues/1945 3. the notification message(s) are tailored for human consumption, those messages are not easy to parse for machines. 4. the threshold support in collectd seems to match values against constants; it doesn't seem possible to match a value against another one, as we need to do for high water monitoring (capacity VS allocation). How I'm addressing, or how I plan to address those challenges (aka action items): 1. I've been experimenting with out-of-tree plugins, and I managed develop, build, install and run one out-of-tree plugin: https://github.com/mojaves/vmon/tree/master/collectd The development pace of collectd looks sustainable, so this doesn't look such a big deal. Furthermore, we can engage with upstream to merge our plugins, either as-is or to extend existing ones. 2. Write another collectd plugin based on the Vdsm python code and/or my past accelerator executable project (https://github.com/mojaves/vmon) 3. patch the collectd notification code. It is yet another plugin OR 4. send notification from the new virt module as per #2, bypassing the threshold system. This move could preclude the new virt module to be merged in the collectd tree. Current status of the action items: 1. done BUT PoC quality 2. To be done (more work than #1/possible dupe with github issue) 3. need more investigation, conflicts with #4 4. need more investigation, conflicts with #3 All the code I'm working on will be found on https://github.com/mojaves/vmon Comments are appreciated -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani

3 4

oVirt 4.0.5 RC2 merge / branch / tag / bugzilla reminder
by Sandro Bonazzola 10 Oct '16

10 Oct '16

All stable branch maintainers, please make sure to - merge all relevant open bugs until Wednesday morning 10:00 AM TLV time. For each package that need to be built (i.e oVirt product) please make sure every bug in MODIFIED has the right Target Release and Target Milestone. A Target release should state the version of the package you're building and should include the same version you used for the tag you just used for this build. (e.g. for ovirt-engine, tag: ovirt-engine-4.0.5.1, tr: 4.0.5.1) A list of bugs that require attention is here: - https://bugzilla.redhat.com/buglist.cgi?quicksearch=target_milestone%3A4.0.… -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

1 0

oVirt 4.0.5 RC2 build planned
by Sandro Bonazzola 10 Oct '16

10 Oct '16

Fyi oVirt developers, An oVirt build is planned for this Wednesday 10:00 AM TLV time (9:00 AM CEST). Taking into consideration the time it takes for Jenkins to run a full CI everything need to be backported by Tuesday 11PM. Please make sure to mark as verified and CR +2 so it will be ready for merging Wednesday morning. A list of pending blockers is available here: https://bugzilla.redhat.com/buglist.cgi?quicksearch=target_milestone%3A4.0.… -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

1 0

[VDSM] travis build updates (3520 tests pass, 2 errors)
by Nir Soffer 09 Oct '16

09 Oct '16

Hi all, Network tests issues were fixed by installing missing package in the docker image. Switching to Fedora 24 base image, python 3 tests are running now. Status: python 2: 2721 tests pass python 3: 799 tests pass, 2 errors (flakey tests) Issues: A test may fail randomly with EINTR, issue seen in local builds, for some reason we don't see this in jenkins. The root cause is not waiting for child process in tests, may fail another test. See https://travis-ci.org/nirs/vdsm/builds/166201822 Please review: https://gerrit.ovirt.org/65268 Nir

1 1

[VDSM] test_ip_info still fails randomly
by Nir Soffer 07 Oct '16

07 Oct '16

Hi all, This test still fails randomly, maybe once week. Are we planning to investigate this test, or mark it as broken_on_ci? Nir ---- 13:51:03 ====================================================================== 13:51:03 FAIL: test_ip_info (network.netinfo_test.TestNetinfo) 13:51:03 ---------------------------------------------------------------------- 13:51:03 Traceback (most recent call last): 13:51:03 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/testValidation.py", line 97, in wrapper 13:51:03 return f(*args, **kwargs) 13:51:03 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/network/netinfo_test.py", line 359, in test_ip_info 13:51:03 [IPV6_ADDR_CIDR])) 13:51:03 AssertionError: Tuples differ: ('192.0.2.2', '255.255.255.0',... != ('192.0.2.2', '255.255.255.0',... 13:51:03 13:51:03 First differing element 2: 13:51:03 ['192.0.2.2/24', '198.51.100.9/24', '198.51.100.11/32'] 13:51:03 ['192.0.2.2/24', '198.51.100.9/24', '198.51.100.11/32', '192.0.2.3/24'] 13:51:03 13:51:03 ('192.0.2.2', 13:51:03 '255.255.255.0', 13:51:03 - ['192.0.2.2/24', '198.51.100.9/24', '198.51.100.11/32'], 13:51:03 + ['192.0.2.2/24', '198.51.100.9/24', '198.51.100.11/32', '192.0.2.3/24'], 13:51:03 ? ++++++++++++++++ 13:51:03 13:51:03 ['2607:f0d0:1002:51::4/64'])

3 3

[VDSM] Reviving travis build (14 network tests failing)
by Nir Soffer 05 Oct '16

05 Oct '16

Hi all, Edi added a dokerfile and .travis.yaml few month ago for running the tests in travis. The code has rotten since then and the build would not even start. I posted these patches fixing travis build - please review: https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:d… I built a new docker image using the new updated dockerfile https://cloud.docker.com/app/nirs/repository/docker/nirs/vdsm-centos/general We really need an official image maintained by the ovirt project. The travis build is working now, but we have 14 test errors: https://travis-ci.org/nirs/vdsm/builds/165332016 Looking at the errors: - tests using ovs fail - instal ovs? - tests networking tests fail How to run your own vdsm tests in travis: - fork vdsm in github https://github.com/oVirt/vdsm - enable travis service for your fork https://github.com/<githuuser>/vdsm/settings/installations add service: Travis CI - add github remote to your vdsm repository git remote add github git(a)github.com:<githubuser>/vdsm.git - rebase your patch over https://gerrit.ovirt.org/65146 if it was not merged yet - push your topic branch to github git push github mybranch Soon you will get mail from travis about the build. Cheers, Nir

1 0

[outreachy] application task: port vmStorageTests.py to python3
by Francesco Romani 05 Oct '16

05 Oct '16

Hi Zoya, Here's the application task for the ovirt outreachy program. This task is meant to give you a good preview of how the internship with the oVirt project will look like, including for example how the process will look like, which tools will you need to use and which tasks you will need to deal with. Should you discover that the idea you previously expressed interest in doesn't fit you anymore (this is an expected possible outcome of the application task!) feel free to look for different ideas in different projects which suits you more! For additional information, besides the ovirt wiki, you are most welcome to refer to https://wiki.gnome.org/Outreachy Sections "Choose a Project" and "Make a small contribution" are especially relevant. Please note that the ideas listed on http://www.ovirt.org/community/activities/outreachy/ *all* fall into the software development projects. As usual, feel free to ask any question anytime. +++ Task: make vmStorageTests.py run under python3 Currently Vdsm is a python 2.7 application, and we want to to support python 3.y. We are porting our tests to python 3.y, fixing the application code using the compatibility packages, or switching to the equivalent more modern constructs, as we go. To run one specific test, we could do like $ cd /path/to/vdsm/source/tree $ git checkout master # make sure you are in the master branch! $ autoreconf -f -v -i && ./configure && make # you to do this just once $ cd tests/ $ ./run_tests_local.sh vmStorageTests.py The above runs the test using python 2. To use python 3, you just need to change the last step: $ PYTHON_EXE=python3 ./run_tests_local.sh vmStorageTests.py See this change as good example of how a patch like this should look like: https://gerrit.ovirt.org/#/c/65006/1 The patch has to be against the git master branch. The task is completed when the patch becomes approved (+2 score on gerrit), and there are no blockers for merge. As usual, feel free to ask any question anytime, on the devel mailing list (CC'd) or on IRC channels. Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani

1 0

[outreachy] application task: port vmSecretTests.py to python 3
by Francesco Romani 05 Oct '16

05 Oct '16

Hi Suramya, Here's the application task for the ovirt outreachy program. This task is meant to give you a good preview of how the internship with the oVirt project will look like, including for example how the process will look like, which tools will you need to use and which tasks you will need to deal with. Should you discover that the idea you previously expressed interest in doesn't fit you anymore (this is an expected possible outcome of the application task!) feel free to look for different ideas in different projects which suits you more! For additional information, besides the ovirt wiki, you are most welcome to refer to https://wiki.gnome.org/Outreachy Sections "Choose a Project" and "Make a small contribution" are especially relevant. Please note that the ideas listed on http://www.ovirt.org/community/activities/outreachy/ *all* fall into the software development projects. As usual, feel free to ask any question anytime. +++ Task: make vmSecretTest.py run under python3 Currently Vdsm is a python 2.7 application, and we want to to support python 3.y. We are porting our tests to python 3.y, fixing the application code using the compatibility packages, or switching to the equivalent more modern constructs, as we go. To run one specific test, we could do like $ cd /path/to/vdsm/source/tree $ git checkout master # make sure you are in the master branch! $ autoreconf -f -v -i && ./configure && make # you to do this just once $ cd tests/ $ ./run_tests_local.sh vmSecretTests.py The above runs the test using python 2. To use python 3, you just need to change the last step: $ PYTHON_EXE=python3 ./run_tests_local.sh vmSecretTests.py See this change as good example of how a patch like this should look like: https://gerrit.ovirt.org/#/c/65006/1 The patch has to be against the git master branch. The task is completed when the patch becomes approved (+2 score on gerrit), and there are no blockers for merge. As usual, feel free to ask any question anytime, on the devel mailing list (CC'd) or on IRC channels. Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani

1 0

backend jar dependencies that should probably be symlinks
by Sandro Bonazzola 04 Oct '16

04 Oct '16

Hi, I'm checking the packaging of ovirt-engine for 4.1 and I've some doubts: $ LC_ALL=C rpm -qlvp http://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/fc24/noarch/ovirt-… jar |grep -v ^l |grep common -rw-r--r-- 1 root root 77761 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/com/netflix/config/main/archaius-core.jar -rw-r--r-- 1 root root 16442 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/com/netflix/hystrix/contrib/main/hystrix-metrics-event-stream.jar -rw-r--r-- 1 root root 290223 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/com/netflix/hystrix/main/hystrix-core.jar -rw-r--r-- 1 root root 738300 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/io/reactivex/rxjava/main/rxjava.jar -rw-r--r-- 1 root root 6073 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/api/metamodel-server/main/metamodel-server.jar -rw-r--r-- 1 root root 8224 Oct 4 00:21 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/core/auth-plugin/main/auth-plugin.jar -rw-r--r-- 1 root root 4010 Oct 4 00:22 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/core/logger/main/logger.jar -rw-r--r-- 1 root root 370051 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-aop.jar -rw-r--r-- 1 root root 731512 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-beans.jar -rw-r--r-- 1 root root 1097552 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-context.jar -rw-r--r-- 1 root root 1078737 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-core.jar -rw-r--r-- 1 root root 262990 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-expression.jar -rw-r--r-- 1 root root 7243 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-instrument.jar -rw-r--r-- 1 root root 423369 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-jdbc.jar -rw-r--r-- 1 root root 265523 Oct 4 00:20 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-tx.jar $ dnf provides "*/archaius-core.jar" -> archaius-core-0.7.3-4.fc24.noarch $ dnf provides "*/hystrix-metrics-event-stream.jar" -> hystrix-metrics-event-stream-1.4.21-5.fc24.noarch and so on with the other jar files. Any chance we can just reuse system libs and symlink them? Please note that the question is for el7 as well: $ LC_ALL=C rpm -qlvp http://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/ovirt-e… |grep jar |grep -v ^l |grep common -rw-r--r-- 1 root root 608376 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/mchange/c3p0/main/c3p0.jar -rw-r--r-- 1 root root 77761 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/netflix/config/main/archaius-core.jar -rw-r--r-- 1 root root 16442 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/netflix/hystrix/contrib/main/hystrix-metrics-event-stream.jar -rw-r--r-- 1 root root 290223 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/netflix/hystrix/main/hystrix-core.jar -rw-r--r-- 1 root root 23234 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/cinder-client.jar -rw-r--r-- 1 root root 20755 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/cinder-model.jar -rw-r--r-- 1 root root 18277 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/glance-client.jar -rw-r--r-- 1 root root 8780 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/glance-model.jar -rw-r--r-- 1 root root 34756 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/keystone-client.jar -rw-r--r-- 1 root root 23183 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/keystone-model.jar -rw-r--r-- 1 root root 10861 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/openstack-client.jar -rw-r--r-- 1 root root 20104 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/quantum-client.jar -rw-r--r-- 1 root root 16031 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/quantum-model.jar -rw-r--r-- 1 root root 8972 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/com/woorea/openstack/sdk/main/resteasy-connector.jar -rw-r--r-- 1 root root 738300 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/io/reactivex/rxjava/main/rxjava.jar -rw-r--r-- 1 root root 621694 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/apache/sshd/main/sshd-core.jar -rw-r--r-- 1 root root 6073 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/api/metamodel-server/main/metamodel-server.jar -rw-r--r-- 1 root root 8222 Oct 3 23:15 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/core/auth-plugin/main/auth-plugin.jar -rw-r--r-- 1 root root 4010 Oct 3 23:16 /usr/share/ovirt-engine/modules/common/org/ovirt/engine/core/logger/main/logger.jar -rw-r--r-- 1 root root 554885 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/quartz/main/quartz.jar -rw-r--r-- 1 root root 450124 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/snmp4j/main/snmp4j.jar -rw-r--r-- 1 root root 370051 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-aop.jar -rw-r--r-- 1 root root 731512 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-beans.jar -rw-r--r-- 1 root root 1097552 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-context.jar -rw-r--r-- 1 root root 1078737 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-core.jar -rw-r--r-- 1 root root 262990 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-expression.jar -rw-r--r-- 1 root root 7243 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-instrument.jar -rw-r--r-- 1 root root 423369 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-jdbc.jar -rw-r--r-- 1 root root 265523 Oct 3 23:14 /usr/share/ovirt-engine/modules/common/org/springframework/main/spring-tx.jar If any dependency is missing on el7 I can try to provide it within CentOS Virt SIG, just let me know the Fedora equivalent rpm name. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

4 7