- Devel - oVirt List Archives

Re: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 962 - Still Failing!
by Yedidyah Bar David 17 Mar '21

17 Mar '21

On Tue, Mar 16, 2021 at 7:06 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… > Build Number: 962 > Build Status: Still Failing > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #953 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #954 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #955 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #956 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #957 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #958 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #959 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #960 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > Changes for Build #961 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > Changes for Build #962 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > > > ----------------- > Failed Tests: > ----------------- > 1 tests failed. > FAILED: basic-suite-master.test-scenarios.test_001_initialize_engine.test_set_hostnames > > Error Message: > failed on setup with "TypeError: __new__() missing 2 required positional arguments: 'version' and 'repo'" > > Stack Trace: > ansible_by_hostname = <function module_mapper_for at 0x7ffbad0acc80> > > @pytest.fixture(scope="session", autouse=True) > def check_installed_packages(ansible_by_hostname): > vms_pckgs_dict_list = [] > for hostname in backend.default_backend().hostnames(): > vm_pckgs_dict = _get_custom_repos_packages( > > ansible_by_hostname(hostname)) > > ost_utils/ost_utils/pytest/fixtures/check_repos.py:39: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ost_utils/ost_utils/pytest/fixtures/check_repos.py:55: in _get_custom_repos_packages > repo_name) > ost_utils/ost_utils/pytest/fixtures/check_repos.py:69: in _get_installed_packages > Package(*line) for line in result > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > .0 = <list_iterator object at 0x7ffba6e97860> > > > Package(*line) for line in result > ] > E TypeError: __new__() missing 2 required positional arguments: 'version' and 'repo' This failed, because 'dnf repo-pkgs' has split the output to two lines, so the first didn't include a version [1]: lago-basic-suite-master-host-1 | CHANGED | rc=0 >> Installed Packages ovirt-ansible-collection.noarch 1.3.2-0.1.master.20210315141358.el8 @extra-src-1 python3-ovirt-engine-sdk4.x86_64 4.4.10-1.20210315.gitf8b9f2a.el8 @extra-src-1 We should either give up on this, or rewrite the call 'dnf repo-pkgs' in some other language that does not require parsing of human-targeted output (perhaps python or ansible), or amend a bit the current code and hope it will survive longer... Trying last one: https://gerrit.ovirt.org/c/ovirt-system-tests/+/113895 [1] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… > > ost_utils/ost_utils/pytest/fixtures/check_repos.py:69: TypeError_______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/HBWJEEDHVVROL… -- Didi

2 4

docs: pointers to more in-depth internals?
by Greg King 16 Mar '21

16 Mar '21

I am new to vdsm and trying to understand the architecture/internals much better The ovirt documentation for architecture I have found so far seems to be relatively high level My effort to understand the architecture by walking through the vdsm code using pdb/rpdb is slow and probably not all that efficient Does anyone have pointers to documentation that might explain the vdsm modules, classes and internals a little more in depth? I'd also like to understand where I might be able to add rpdb.set_trace() so I can step through functions being called in libvirt.py [oracle-email-sig-198324-355094] Gregory King | Software Development Manager | +1.303.272.2427 Oracle Virtualization Sustaining Engineering 500 Eldorado Boulevard Build 5 | Broomfield Colorado 80021 Mobile: +1.303.968.8169 | Fax: +1.303.272.2427

1 0

Jenkins CI broken
by Sandro Bonazzola 16 Mar '21

16 Mar '21

Hi, looks like we have some issues in Jenkins: https://jenkins.ovirt.org/job/imgbased_standard-check-patch/663/console We have an issue with our local mirror which is quite outdated (2020-10-08) *08:45:53* Loading mirror speeds from cached hostfile*08:45:53* http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2020-10-08-04-48/repodata/c…: [Errno 14] HTTP Error 404 - Not Found*08:45:53* Trying other mirror.*08:45:53* To address this issue please refer to the below wiki article *08:45:53* *08:45:53* https://wiki.centos.org/yum-errors*08:45:53* *08:45:53* If above article doesn't help to resolve this issue please use https://bugs.centos.org/.*08:45:53* *08:45:54* http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2020-10-08-04-48/repodata/c…: [Errno 14] HTTP Error 404 - Not Found*08:45:54* Trying other mirror.*08:45:54* No package python36-docker available.*08:45:54* Error: Nothing to do and it's missing a python36-docker package which cause the global setup of the job to fail: *08:45:54* global_setup[docker_setup] ERROR: Failed to install Docker Python36 API.*08:45:54* + return 1*08:45:54* + failed=true There's also some weird firewalld configuration which is failing: *08:45:54* + sudo -nl firewall-cmd*08:45:54* ++ sudo -n firewall-cmd --query-service=ovirtlago*08:46:03* Error: INVALID_SERVICE: ovirtlago Can you please have a look? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Engine upgrade fails with virtio_scsi_multi_queues_enabled does not exist
by Vojtech Juranek 16 Mar '21

16 Mar '21

Hi, when I try to upgrade my engine with built artifacts (e.g. [1]), it fails with: psql:/usr/share/ovirt-engine/dbscripts/create_views.sql:1000: ERROR: column vm_templates.virtio_scsi_multi_queues_enabled does not exist LINE 86: vm_templates.virtio_scsi_multi_queues_enabled AS virtio_... ^ FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/create_views.sql Same issue happens when I tried to update my dev engine with local build. However, OST is passing [2]. Any idea what's wrong? I don't see anything wrong in [3], which is probably the culprit, but I haven't much experience with engine/db though. Or maybe some issue on my side? Thanks Vojta [1] https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/11174/artif… [2] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7880/ [3] https://github.com/oVirt/ovirt-engine/commit/52caf76e60ace2266d8283c131d467…

5 8

ovirt-engine has been tagged (ovirt-engine-4.4.5.10)
by Tal Nisan 15 Mar '21

15 Mar '21

1 0

Where to place useful Ansible recipe?
by Vojtech Juranek 11 Mar '21

11 Mar '21

Hi, the result of working on BZ #1310330 [1] is Ansible recipe, which removes multipath devices for unzoned LUN from the hosts. The recipe is attached to the BZ, but it would be good to put it somewhere in into the code base. IIUC Ansible recipes contained in engine project are used internally by the engine, but this is not the right place (can be in the future if we implement some UI in the engine which will trigger this recipe). Is ovirt-engine/contrib [2] the right place or is there a better place? Thanks Vojta [1] http://bugzilla.redhat.com/1310330 [2] https://github.com/oVirt/ovirt-engine/tree/master/contrib

3 2

corrupted jar ; jackson-mapper dependency 4.4.5?
by Prajith Kesava Prasad 11 Mar '21

11 Mar '21

Hi everyone, So while calling createBrickCommand playbook from ovirt-engine (createBrickCommand.java) a parameter called disks are constructed to parse a list of disks. Its implementation contains a class variable ArrayNode which uses JsonNodeFactory, from jackson-mapper dependency . Usually parsing of disks to playbook would contain a list of disks, but now it's parsing the absurd values of JsonNodeFactory, and parsing a weird dictionary:- Snip of the parameters parsed from ovirt-engine to ansible-runner *<snip>elements': [{'textValue': '/dev/sdc', 'binaryValue': '/dev/sdc', 'textual': True, 'valueNode': True, 'numberType': None, 'elements': [], 'array': False, 'fields': [], 'null': False, 'binary': False, 'booleanValue': False, 'numberValue': None, 'decimalValue': 0, 'bigIntegerValue': 0, 'valueAsText': '/dev/sdc', 'valueAsInt': 0, 'valueAsLong': 0, 'valueAsDouble': 0.0, 'valueAsBoolean': False, 'containerNode': False, 'missingNode': False, 'pojo': False, 'number': False, 'integralNumber': False, 'floatingPointNumber': False, 'int': False, 'long': False, 'double': False, 'bigDecimal': False, 'bigInteger': False, 'boolean': False, 'object': False, 'fieldNames': [], 'longValue': 0, 'doubleValue': 0.0, 'intValue': 0}], 'array': True, 'valueAsText': None, 'containerNode': True, 'numberType': None, 'fields': [], 'null': False, 'binary': False, 'textValue': None, 'binaryValue': None, 'booleanValue': False, 'numberValue': None, 'decimalValue': 0, 'bigIntegerValue': 0, 'valueAsInt': 0, 'valueAsLong': 0, 'valueAsDouble': 0.0, 'valueAsBoolean': False, 'valueNode': False, 'missingNode': False, 'pojo': False, 'number': False, 'integralNumber': False, 'floatingPointNumber': False, 'int': False, 'long': False, 'double': False, 'bigDecimal': False, 'bigInteger': False, 'textual': False, 'boolean': False, 'object': False, 'fieldNames': [], 'longValue': 0, 'doubleValue': 0.0, 'intValue': 0}</snip>* these values represent the Class variables of JsonNodeFactory class in the Jackson-mapper dependency instantiated from ArrayNode in CreateBrickCommand.java class. Is my understanding flawed, or did we upgrade to a maybe corrupted dependency? or does this have any other problem which I missed out on? ( tested with RHV 4.4.5 latest build - 4.4.5.7-0.1.el8ev ) Regards, Prajith Kesava Prasad.

2 1

basic-suite fails with: "... is not signed Error: GPG check FAILED"
by Yedidyah Bar David 10 Mar '21

10 Mar '21

Hi all, Now happened in [1]: STDERR b'warning: /var/cache/dnf/ovirt-master-copr:copr.fedorainfracloud.org:sbonazzo:EL8_collection-7f7c2d939ff8f7bc/packages/python3-pyyaml-5.1.2-3.el8.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY\nImporting GPG key 0x119783D1:\n Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection(a)copr.fedorahosted.org>"\n Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1\n From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pub… ovirt-engine-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-backend-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-dbscripts-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-restapi-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-base-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-cinderlib-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-imageio-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-ovirt-engine-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-ovirt-engine-common-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-websocket-proxy-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-tools-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-tools-backup-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-vmconsole-proxy-helper-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-webadmin-portal-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-websocket-proxy-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage python3-ovirt-engine-lib-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nError: GPG check FAILED\n' Known issue? I have vague memories about this being reported, not sure. Perhaps we publish to tested repo before signing? And if a job runs in the middle, it will get unsigned packages? Thanks and best regards, [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/15924/ -- Didi

1 0

Support for 5K volumes
by Sandro Bonazzola 09 Mar '21

09 Mar '21

Hi, Gluster 9 introduced support for 5k volumes ( https://docs.gluster.org/en/latest/release-notes/9.0/ ). Is there interest in getting this in oVirt too? Any thoughts about rebasing on Gluster 9 for oVirt 4.4.6? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

4 3

test_ha_recovery failed (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master - Build # 1943 - Still Failing!)
by Yedidyah Bar David 09 Mar '21

09 Mar '21

On Sun, Mar 7, 2021 at 5:12 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1943/ Failed during -basic-suite-master.test-scenarios.test_004_basic_sanity.test_ha_recovery due to timeout. Can't tell exactly why. Now rebuilt, to see if it happens again. Basic suite did pass this point with (mostly) the same code, and there shouldn't be anything HE-specific in this test. FYI. Some more details: The function is: @order_by(_TEST_LIST) def test_ha_recovery(engine_api, get_ansible_host_for_vm): engine = engine_api.system_service() with engine_utils.wait_for_event(engine, [119, 9602, 506]): # VM_DOWN_ERROR event(119) # HA_VM_FAILED event event(9602) # VDS_INITIATED_RUN_VM event(506) ansible_host = get_ansible_host_for_vm(VM2_NAME) pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() ansible_host.shell('kill -KILL {}'.format(pid)) vm_service = test_utils.get_vm_service(engine, VM2_NAME) assertions.assert_true_within_long( lambda: vm_service.get().status == types.VmStatus.UP ) with engine_utils.wait_for_event(engine, 33): # USER_STOP_VM event vm_service.stop() It failed inside the 'with' statement. Can't find any of the expected events in engine.log. In last successful I see all three of them. We do not actively log anything about running the pgrep command, but it's logged by ansible on the host, to /var/log/messages. I see it in last successful, but not in above failed run, in any of the two hosts. So most likely it didn't even get to run pgrep. IMO we probably want some more information logged in such a case, to allow further investigation. Best regards, > Build Number: 1943 > Build Status: Still Failing > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #1939 > [hbraha] ost: Check if user custom repos are used on VM > > > Changes for Build #1940 > [hbraha] network: add check_repos fixture > > > Changes for Build #1941 > [Andrej Cernek] network, pylint: Simplify pylintrc file > > > Changes for Build #1942 > [Andrej Cernek] network, pylint: Simplify pylintrc file > > > Changes for Build #1943 > [Andrej Cernek] network, pylint: Simplify pylintrc file > > > > > ----------------- > Failed Tests: > ----------------- > 1 tests failed. > FAILED: he-basic-suite-master.test-scenarios.test_004_basic_sanity.test_ha_recovery > > Error Message: > AssertionError: False != True after 600 seconds > > Stack Trace: > engine_api = <ovirtsdk4.Connection object at 0x7f408114c1d0> > get_ansible_host_for_vm = <function get_ansible_host_for_vm.<locals>.ansible_host_for at 0x7f4081171950> > > @order_by(_TEST_LIST) > def test_ha_recovery(engine_api, get_ansible_host_for_vm): > engine = engine_api.system_service() > with engine_utils.wait_for_event(engine, [119, 9602, 506]): > # VM_DOWN_ERROR event(119) > # HA_VM_FAILED event event(9602) > # VDS_INITIATED_RUN_VM event(506) > ansible_host = get_ansible_host_for_vm(VM2_NAME) > pid = ansible_host.shell('pgrep -f qemu.*guest=vm2')['stdout'].strip() > > ansible_host.shell('kill -KILL {}'.format(pid)) > > ../he-basic-suite-master/test-scenarios/test_004_basic_sanity.py:1070: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > /usr/lib64/python3.6/contextlib.py:88: in __exit__ > next(self.gen) > ../ost_utils/ost_utils/engine_utils.py:42: in wait_for_event > lambda: > ../ost_utils/ost_utils/assertions.py:96: in assert_true_within_long > assert_equals_within_long(func, True, allowed_exceptions) > ../ost_utils/ost_utils/assertions.py:81: in assert_equals_within_long > func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > func = <function wait_for_event.<locals>.<lambda> at 0x7f40810d5bf8> > value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0 > error_message = 'False != True after 600 seconds' > > def assert_equals_within( > func, value, timeout, allowed_exceptions=None, initial_wait=10, > error_message=None > ): > allowed_exceptions = allowed_exceptions or [] > res = '<no-result-obtained>' > with _EggTimer(timeout) as timer: > while not timer.elapsed(): > try: > res = func() > if res == value: > return > except Exception as exc: > if _instance_of_any(exc, allowed_exceptions): > time.sleep(3) > continue > > LOGGER.exception("Unhandled exception in %s", func) > raise > > if initial_wait == 0: > time.sleep(3) > else: > time.sleep(initial_wait) > initial_wait = 0 > try: > if error_message is None: > error_message = '%s != %s after %s seconds' % (res, value, timeout) > > raise AssertionError(error_message) > E AssertionError: False != True after 600 seconds > > ../ost_utils/ost_utils/assertions.py:59: AssertionError -- Didi

1 3