- Devel - oVirt List Archives

OST 4.1 failure: Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running', ))
by Nir Soffer 12 May '17

12 May '17

I got this failure in 4.1 build[1], which should not be relevant to the tested patch[2] - is this a known issue? *13:31:47* # add_hosts: *13:31:47* Error while running thread*13:31:47* Traceback (most recent call last):*13:31:47* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue*13:31:47* queue.put({'return': func()})*13:31:47* File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py", line 320, in _add_host_4*13:31:47* name=CLUSTER_NAME,*13:31:47* File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 8726, in add*13:31:47* return self._internal_add(host, headers, query, wait)*13:31:47* File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_add*13:31:47* context = self._connection.send(request)*13:31:47* File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 300, in send*13:31:47* sys.exc_info()[2]*13:31:47* File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 295, in send*13:31:47* return self.__send(request)*13:31:47* File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 413, in __send*13:31:47* self._multi.add_handle(curl)*13:31:47* Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running',)) [1] http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests… [2] https://gerrit.ovirt.org/76645 Nir

6 26

DBUnit upgrade and format change
by Allon Mureinik 12 May '17

12 May '17

Hi all, In an attempt to make maintaining Engine's DAO tests a bit more manageable, yesterday I've merged patches to upgrade the DBUnit version we're using to the latest available version, 2.5.3 [1], and changed the format to flat-XML [2], ending up with a fixures.xml file which is almost 10K lines shorter. What do you need to know about this change? Frankly, not much. The format itself is pretty self-descriptive. To add a row to a table in the test database, just add a row to fixtures.xml: <table_name some_column="some_value" some_other_column="123"/> Note that you can pretty much add such a row wherever you want in the file, but in the interest of maintainability, please group rows from the same tables together, and leave an empty line between tables (as the file currently is). Note also that DBUnit 2.5.3 doesn't seem to allow timezones, offsets or whitespaces at the end of timestamp values. Since our tests don't really depend on them, I just removed the handful of instances we had. If anyone wants to take a deeper look, I'd gladly review it :-) If this change causes trouble to anyone (it shouldn't - it was tested both on my machine and in the CI, several times), please let me know. You friendly neighborhood cleanup guy, Allon [1] https://gerrit.ovirt.org/#/c/76678/ [2] https://gerrit.ovirt.org/#/c/76679/

1 0

[ OST Failure Report ] [ oVirt master][12-05-2017][ 002_bootstrap.add_hosts ]
by Sandro Bonazzola 12 May '17

12 May '17

http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/1440/testRepo… ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running',)) -------------------- >> begin captured logging << -------------------- lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_4.1/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py", line 320, in _add_host_4 name=CLUSTER_NAME, File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 8726, in add return self._internal_add(host, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_add context = self._connection.send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 300, in send sys.exc_info()[2] File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 295, in send return self.__send(request) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 413, in __send self._multi.add_handle(curl) Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running',)) --------------------- >> end captured logging << --------------------- Looks like it started failing on latest ovirt-host-deploy build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.1/1431/ Sadly I can't find ovirt-host-deploy logs within the archived artifacts. Simone, Didi, can you please have a look? Ondra, can you also check? I'm not sure about the meaning of the error reported by the SDK. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 2

[VDSM] travis builds fails since "network: report dpdk devices as nics"
by Nir Soffer 11 May '17

11 May '17

Travis builds are failing because the lshw package was not added to the vdsm test docker images. The offending patch is: network: report dpdk devices as nics added link_info dictionary for dpdk dev information. Currently it contains hard coded and empty values. getLinks now reports dpdk devices as nics and providing their link_info as well. Signed-off-by: Irit Goihman <igoihman(a)redhat.com> Change-Id: I67b2083e9c6f032472fd75f6800c864083ffdef5 When adding new build dependencies, please add the required packages to docker/Dockerfile.* Thanks, Nir

2 4

ovirt-engine and vdsm repositories hooks
by Shlomo Ben David 11 May '17

11 May '17

Hi All, I updated the *ovirt-engine* and *vdsm* project hooks to the new (all) hooks. The new hooks are separated into 3 groups: - check hooks - check bug-url - check target milestone - check product - check backport - update hooks - set_post - set_modified - update_tracker - all (include both check and update hooks) I will update the other projects during next week. If you have any questions/suggestions/issues, please send an email to infra(a)ovirt.org. Best Regards, Shlomi Ben-David | Software Engineer | Red Hat ISRAEL RHCSA | RHCVA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci) OPEN SOURCE - 1 4 011 && 011 4 1

1 0

[OST/SDK4] ValueError: urandom is not a valid RngSource
by Milan Zamazal 10 May '17

10 May '17

Hi, I'm trying to run ovirt-system-tests from git master with basic-suite-master on my RHEL machine. The tests had been failing on missing ovirtsdk4 Python module, so I installed python-ovirt-engine-sdk4-4.0.4-2.el7ev.x86_64 package. Now the tests fail early and I'm getting the following error: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/pdm/ovirt/lago/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 215, in add_cluster add_cluster_4(prefix) File "/home/pdm/ovirt/lago/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 256, in add_cluster_4 ballooning_enabled=True, File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 3758, in add return readers.ClusterReader.read_one(reader) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/readers.py", line 1740, in read_one obj.required_rng_sources = [types.RngSource(s.lower()) for s in Reader.read_strings(reader)] File "/usr/lib/python2.7/site-packages/enum/__init__.py", line 330, in __call__ return cls.__new__(cls, value) File "/usr/lib/python2.7/site-packages/enum/__init__.py", line 642, in __new__ raise ValueError("%s is not a valid %s" % (value, cls.__name__)) ValueError: urandom is not a valid RngSource Does anybody know what's wrong and how to make the tests working? Thank you, Milan

2 2

[missing_subjectAltName] in engine ca certificate?
by Nir Soffer 10 May '17

10 May '17

I imported the certificate from my engine into chrome[1], but Chrome refuses to use it because: This server could not prove that it is ...; its security certificate is from [missing_subjectAltName]. Same certificate used to work 2 weeks ago, looks like new Chrome version changed the rules. Without importing engine CA, there is no way to upload images via engine. Tested on engine 4.1.1 and 4.1.2 on Centos 7.3. Is this known issue? [1] from http:// <engine_url>/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA Nir

6 8

Failing to make vdsm (master) - sanlock-python is not a build dep?
by Yaniv Kaul 10 May '17

10 May '17

I'm trying to follow http://www.ovirt.org/develop/developer-guide/vdsm/developers/ to build vdsm, and I'm failing (master, on F25). There is one issue in the middle: [ykaul@ykaul vdsm]$ sudo dnf install `cat automation/check-patch.packages.fc*` ... Package rpm-build-4.13.0.1-1.fc25.x86_64 is already installed, skipping. Package rpmlint-1.9-9.fc25.noarch is already installed, skipping. Package sudo-1.8.19p2-1.fc25.x86_64 is already installed, skipping. Package yum-3.4.3-510.fc25.noarch is already installed, skipping. Package yum-utils-1.1.31-511.fc25.noarch is already installed, skipping. Error: Unable to find a match. And then, ./autogen.sh --system succeeds, but 'make' fails: Making all in api make[3]: Entering directory '/home/ykaul/vdsm/lib/vdsm/api' Generate vdsm-api.html chmod u+w . PYTHONPATH=./../../:./../../vdsm \ /usr/bin/python schema_to_html.py vdsm-api.yml vdsm-api.html Traceback (most recent call last): File "schema_to_html.py", line 29, in <module> from vdsm.api import vdsmapi File "/home/ykaul/vdsm/lib/vdsm/api/vdsmapi.py", line 28, in <module> from vdsm import utils File "/home/ykaul/vdsm/lib/vdsm/utils.py", line 33, in <module> from .compat import pickle File "/home/ykaul/vdsm/lib/vdsm/compat.py", line 96, in <module> import sanlock ImportError: No module named sanlock which seems to be solved with manually installing sanlock-python.

3 4

[ OST Failure Report ] [ oVirt master][09-05-2017][ 002_bootstrap.add_hosts ]
by Anton Marchukov 09 May '17

09 May '17

Hello. Today around 11 am the add host for master started to consistently fail. We found the following in /var/log/messages (as I understand it might get there through journald as the host deploy log claims that vdsmd unit startup failed): May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: Traceback (most recent call last): May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: return tool_command[cmd]["command"](*args) May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 48, in upgrade_networks May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: netupgrade.upgrade() May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 55, in upgrade May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: _create_unified_configuration(rconfig, NetInfo(netinfo(vdsmnets))) May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 133, in _create_unified_configuration May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: RunningConfig.store() May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py", line 204, in store May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: _store_net_config() May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py", line 281, in _store_net_config May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: utils.rmTree(real_old_safeconf_dir) May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 125, in rmTree May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: shutil.rmtree(directoryToRemove) May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib64/python2.7/shutil.py", line 232, in rmtree May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: onerror(os.path.islink, path, sys.exc_info()) May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: File "/usr/lib64/python2.7/shutil.py", line 230, in rmtree May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: raise OSError("Cannot call rmtree on a symbolic link") May 9 07:01:25 lago-basic-suite-master-host1 vdsm-tool: OSError: Cannot call rmtree on a symbolic link May 9 07:01:25 lago-basic-suite-master-host1 systemd: vdsm-network.service: control process exited, code=exited status=1 The full logs of the run you can find in Jenkins artifacts: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6589/artif… Any vdsm patches merged that might cause this? -- Anton Marchukov Senior Software Engineer - RHEV CI - Red Hat

5 5

[ OST Failure Report ] [ oVirt master ] [ 07-05-2017 ] [ add_secondary_storage_domains ]
by Daniel Belenky 07 May '17

07 May '17

Hi, The following test failed: 002_bootstrap.add_secondary_storage_domains <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/testR…> *Error snippet from VDSM Log:* 2017-05-07 08:09:03,906-0400 ERROR (monitor/e48ddb8) [storage.Monitor] Setting up monitor for e48ddb84-77fc-41d8-98ce-a40140649c8d failed (monitor:329) Traceback (most recent call last): File "/usr/share/vdsm/storage/monitor.py", line 326, in _setupLoop self._setupMonitor() File "/usr/share/vdsm/storage/monitor.py", line 348, in _setupMonitor self._produceDomain() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 392, in wrapper value = meth(self, *a, **kw) File "/usr/share/vdsm/storage/monitor.py", line 366, in _produceDomain self.domain = sdCache.produce(self.sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 112, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 53, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 136, in _realProduce domain = self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 153, in _findDomain return findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 178, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'e48ddb84-77fc-41d8-98ce-a40140649c8d',) *Error snippet from SUPERVDSM Log:* MainProcess|jsonrpc/6::ERROR::2017-05-07 08:09:09,960::supervdsm_server::96::SuperVdsm.ServerCallback::(wrapper) Error in mount Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 94, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 136, in mount timeout=timeout, cgroup=cgroup) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 262, in _mount _runcmd(cmd, timeout) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 297, in _runcmd raise MountError(rc, ";".join((out, err))) MountError: (32, ';mount.nfs: Connection timed out\n') MainProcess|jsonrpc/1::ERROR::2017-05-07 08:09:09,960::supervdsm_server::96::SuperVdsm.ServerCallback::(wrapper) Error in mount Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 94, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 136, in mount timeout=timeout, cgroup=cgroup) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 262, in _mount _runcmd(cmd, timeout) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 297, in _runcmd raise MountError(rc, ";".join((out, err))) MountError: (32, ';mount.nfs: Connection timed out\n') *Links*: Job's artifacts <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/artif…>, All related logs <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/artif…>, Engine Log <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/artif…>, Host 1 SuperVdsm Log <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/artif…>, Host 1 Vdsm Log <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6573/artif…> Please assist to investigate the failure. Thanks, -- DANIEL BELENKY RHV DEVOPS Red Hat EMEA <https://www.redhat.com/> IRC: #rhev-integ #rhev-dev <https://red.ht/sig>

2 1