- Devel - oVirt List Archives

[VDSM] VdsmClientTests.test_event_handler - flaky test?
by Nir Soffer 04 Mar '20

04 Mar '20

Seen this unrelated failure today on: https://gerrit.ovirt.org/#/c/107206/ Build: https://jenkins.ovirt.org/job/vdsm_standard-check-patch/18831//artifact/che… This smells like the known issue with handling connections. I think the test was disabled in the past, and it started to fail since we enabled the stomp tests recently. If you see the same failures, lets mark this test as xfail until we have time to debug this. See failure bellow. =================================== FAILURES =================================== ______________________ VdsmClientTests.test_event_handler ______________________ self = <stomprpcclient_test.VdsmClientTests testMethod=test_event_handler> def test_event_handler(self): with self._create_client() as client: event_queue = queue.Queue() sub_id = client.subscribe(EVENT_TOPIC, event_queue) > client.Test.sendEvent() lib/yajsonrpc/stomprpcclient_test.py:215: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <stomprpcclient_test._MockedClient object at 0x7f87ab72d160> namespace = 'Test', method_name = 'sendEvent', kwargs = {} method = 'Test.sendEvent', timeout = 3 req = {'jsonrpc': '2.0', 'method': 'Test.sendEvent', 'params': {}, 'id': 'aa4919ae-558f-4403-aab8-f64f0d7cd2ff'} responses = None def _call(self, namespace, method_name, **kwargs): """ Client call method, executes a given command Args: namespace (string): namespace name method_name (string): method name **kwargs: Arbitrary keyword arguments Returns: method result Raises: ClientError: in case of an error in the protocol. TimeoutError: if there is no response after a pre configured time. ServerError: in case of an error while executing the command """ method = namespace + "." + method_name timeout = kwargs.pop("_timeout", self._default_timeout) req = yajsonrpc.JsonRpcRequest( method, kwargs, reqId=str(uuid.uuid4())) try: responses = self._client.call( req, timeout=timeout, flow_id=self._flow_id) except EnvironmentError as e: raise ClientError(method, kwargs, e) if not responses: > raise TimeoutError(method, kwargs, timeout) E vdsm.client.TimeoutError: Request Test.sendEvent with args {} timed out after 3 seconds ../lib/vdsm/client.py:294: TimeoutError ------------------------------ Captured log call ------------------------------- WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event ERROR vds.dispatcher:betterAsyncore.py:179 uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher ('::1', 33316, 0, 0) at 0x7f87ab72d128> (<class 'ValueError'>:'b'ept-version:1.2'' contains illegal character ':' [/usr/lib64/python3.6/asyncore.py|readwrite|108] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/protocoldetector.py|handle_read|129] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|handle_socket|413] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|54] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|createListener|379] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|StompListener|345] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|47] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|86] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|init|363] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|57] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|set_message_handler|645] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|handle_read|421] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|parse|323] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|_parse_command|245] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|decode_value|167]) WARNING vds.dispatcher:betterAsyncore.py:179 unhandled close event ERROR root:concurrent.py:267 FINISH thread <Thread(JsonRpc (StompReactor), started daemon 140220662679296)> failed Traceback (most recent call last): File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/common/concurrent.py", line 260, in run ret = func(*args, **kwargs) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py", line 393, in process_requests self._reactor.process_requests() File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", line 238, in process_requests timeout=self._get_timeout(self._map), File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", line 253, in _get_timeout interval = disp.next_check_interval() File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", line 99, in next_check_interval return getattr(self.__impl, "next_check_interval", default_func)() File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", line 486, in next_check_interval self.handle_timeout() File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", line 451, in handle_timeout self._frame_handler.handle_timeout(self) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompclient.py", line 130, in handle_timeout dispatcher._on_timeout) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", line 630, in reconnect AsyncDispatcher(self, self._async_client, count=count)) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", line 271, in reconnect dispatcher.create_socket(address, sslctx) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", line 112, in create_socket sock = sslctx.wrapSocket(sock) File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/sslutils.py", line 113, in wrapSocket ca_certs=self.ca_certs) File "/usr/lib64/python3.6/ssl.py", line 1114, in wrap_socket ciphers=ciphers) File "/usr/lib64/python3.6/ssl.py", line 704, in __init__ self._context.load_verify_locations(ca_certs) FileNotFoundError: [Errno 2] No such file or directory

1 0

RESTAPIv3 support has been removed from oVirt master branch
by Martin Perina 04 Mar '20

04 Mar '20

Hi, we have just merged https://gerrit.ovirt.org/105297 which removes support for RESTAPIv3. RESTAPIv3 was deprecated in oVirt 4.0 and it didn't provide access to any new features introduced in oVirt 4. The removal of RESTAPIv3 also means that you cannot use any longer Python/Java SDKv3 or oVirt CLI (all of them were also deprecated since 4.0) with oVirt 4.4. Everyone still using those deprecated libraries/tools need to switch to RESTAPIv4, Python/Java/Ruby/Go SDKv4 or oVirt Ansible modules, which provides access to all upcoming features available in oVirt 4.4 or features provided by previous oVirt 4.y releases. Thanks for understanding, Martin -- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.

1 0

Permission error in engine-setup in master
by Shmuel Melamud 04 Mar '20

04 Mar '20

Hi! After recent rebase, engine-setup stops on the following error: Failed to execute stage 'Misc configuration': [Errno 1] Operation not permitted: '/home/smelamud/engine-root/master/etc/pki/ovirt-engine/keys/ovn-ndb.key.nopass' The file has rw permissions for the current user, permissions for all directories are also OK. Does anybody know what is happening? Shmuel

2 2

Ovirt SDK: Any way to remove the force password reset for new users
by raymond.francis＠oracle.com 04 Mar '20

04 Mar '20

Trying to do some test automation where my new user must login and do an activity like reboot a VM However, he gets stuck as needs to reset password. Checking the SDK and CLI tools, the admin can do a rest of user password. But of course the user must still reset it themselves afterwards for security reasons. For the test, id like to find a way to bypass this - is same possible?

2 1

ovirt-engine has been tagged (ovirt-engine-4.3.9.2)
by Tal Nisan 04 Mar '20

04 Mar '20

1 0

[Bug 1808271] unboundid-ldapsdk-5.0.0 is available
by Sandro Bonazzola 28 Feb '20

28 Feb '20

Hi, unboundid-ldapsdk-5.0.0 is available upstream. Release notes are available here: https://github.com/pingidentity/ldapsdk/releases most notable change is the licensing, including ASL along with GPL and LGPL. The library is used by aaa-ldap extension. I've not yet built it for Fedora Rawhide, and I'm not prioritizing this unless the new release content / licensing change is useful for oVirt scope. Please let me know if there's interest in getting this new release available for oVirt consumption. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

2 1

net_mlx5 libibverbs.so.1 noise
by Yedidyah Bar David 27 Feb '20

27 Feb '20

Hi all, I upgraded my CentOS8 engine machine, and now whenever I run a shell (e.g. ssh, or another window in tmux), I get: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) Not sure if this is a new known issue. It happened in the past, and was fixed, but apparently got broken again. Now commented about this on the old bug: https://bugzilla.redhat.com/show_bug.cgi?id=1642945#c9 Workaround: dnf install libibverbs.so.1 libmlx5 Best regards, -- Didi

3 2

Need steps to downgrade engine from 4.4 to 4.3.
by Ritesh Chikatwar 26 Feb '20

26 Feb '20

Hello all.. Is there any steps to downgrade engine from 4.4 to 4.3 Any help will appreciate. Ritesh

2 1

[CQ 20589] engine reports storage domain does not exist
by Sandro Bonazzola 25 Feb '20

25 Feb '20

https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/ Engine fails at: 2020-02-25 02:53:53,144-05 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (default task-1) [56f93c90-3868-43f8-920f-bc1fccc72a27] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HSMGetStorageDomainInfoVDS, error = Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',), code = 358 https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… corresponding failure in VDSM is: 2020-02-25 02:53:53,128-0500 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='752b540e-fbfc-4602-8eeb-3c357fb7f5a2') Unexpected error (task:874) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 881, in _run return fn(*args, **kargs) File "<decorator-gen-129>", line 2, in getStorageDomainInfo File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2752, in getStorageDomainInfo dom = self.validateSdUUID(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 309, in validateSdUUID sdDom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 186, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) vdsm.storage.exception.StorageDomainDoesNotExist: Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) 2020-02-25 02:53:53,128-0500 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='752b540e-fbfc-4602-8eeb-3c357fb7f5a2') aborting: Task is aborted: "value=Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) abortedcode=358" (task:1184) 2020-02-25 02:53:53,128-0500 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH getStorageDomainInfo error=Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) (dispatcher:83) 2020-02-25 02:53:53,129-0500 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call StorageDomain.getInfo failed (error 358) in 0.28 seconds (__init__:312) https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… Corresponding var log messages: Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: Attached scsi generic sg8 type 0 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: alua: port group 00 state A non-preferred supports TOlUSNA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: Direct-Access LIO-ORG lun1_bdev 4.0 PQ: 0 ANSI: 5 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB) Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: alua: supports implicit and explicit TPGS Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: alua: device naa.600140541e20cc9dba94e4fa46a57322 port group 0 rel port 1 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Write Protect is off Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Write cache: enabled, read cache: enabled, supports DPO and FUA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: Attached scsi generic sg9 type 0 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: alua: port group 00 state A non-preferred supports TOlUSNA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB) Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:0: [sdf] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Write Protect is off Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:4: [sdg] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:3: [sdh] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Started Session c67 of user root. Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Starting LVM event activation on device 8:128... Feb 25 02:53:52 lago-basic-suite-master-host-0 lvm[30164]: pvscan[30164] PV /dev/sdi online, VG 8c9f3762-3bf1-48fa-9237-b6587d0268ab is complete. Feb 25 02:53:52 lago-basic-suite-master-host-0 lvm[30164]: pvscan[30164] VG 8c9f3762-3bf1-48fa-9237-b6587d0268ab skip autoactivation. Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Started LVM event activation on device 8:128. https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

pad.engineering.redhat.com not working
by Eyal Shenitzky 25 Feb '20

25 Feb '20

Hi, $SUBJECT isn't working. Does someone know what is the problem? -- Regards, Eyal Shenitzky

1 0