November 2021 - Devel - oVirt List Archives

CI fails to clone git repo
by Vojtech Juranek 16 Nov '21

16 Nov '21

Hi, CI fails to clone git repo, fails with: stderr: fatal: unable to access 'https://gerrit.ovirt.org/jenkins/': Could not resolve host: gerrit.ovirt.org; Unknown error See [1] for full log. Could you please take a look? Thanks Vojta [1] https://jenkins.ovirt.org/job/vdsm_standard-check-patch/30631/console

2 1

suspend resume test broken
by Michal Skrivanek 16 Nov '21

16 Nov '21

HI all, suspend/resume is currently broken on master/el8stream, can anyone please take a look and find out the reason and fix it? Thanks, michal

1 1

Bugs easy to fix but left behind for more important ones
by Sandro Bonazzola 16 Nov '21

16 Nov '21

Hi, there's a bunch of bugs which are easy to fix and can possibly be fixed also by someone with little / no programming experience. They are low priority, low severity, and likely going to be closed deferred / wontfix due to capacity by the end of the 4.5.0 development cycle. But if you have time and you want to try to contribute to oVirt for the first time, these are likely to be easy enough to let you contribute. *Bug 1581672* <https://bugzilla.redhat.com/show_bug.cgi?id=1581672> - [TEXT] add help for --otopi-environment to ovirt-engine-provisiondb tool *Bug 2000121* <https://bugzilla.redhat.com/show_bug.cgi?id=2000121> - docs/comments/etc misguide about postgresql credentials defaults *Bug 1999049* <https://bugzilla.redhat.com/show_bug.cgi?id=1999049> - hosted engine installation fails if locale is Greek If you give it a try and you have issues please let me know, I'll try to help unblock you. Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

OST: Vdsm: Occasional failures when stopping vdsmd
by Milan Zamazal 15 Nov '21

15 Nov '21

Hi, Michal has observed occasional OST failures in test_vdsm_recovery last days, which hadn't been seen before. When `systemctl stop vdsmd' is called (via Ansible) there, vdsmd (almost?) never finishes its shutdown within the 10 seconds timeout and gets then killed with SIGKILL. If this action is accompanied by "Job for vdsmd.service canceled." message then the test fails; otherwise OST continues normally. The situation is reproducible by running OST basic-suite-master and making it artificially failing after test_vdsm_recovery. Then running `systemctl stop vdsmd' manually on the given OST host (can be done repeatedly, so it provides a good opportunity to examine the problem). There are two problems there: - "Job for vdsmd.service canceled." message that sometimes occurs after `systemctl stop vdsmd' and then the test fails. I don't know what it means and I can't identify any difference in journal between when the message occurs and when it doesn't. - The fact that Vdsm doesn't stop within the timeout and must be killed. This doesn't happen in my normal oVirt installation. It apparently blocks in self.irs.prepareForShutdown() call from clientIF.py. Journal says: systemd[1]: Stopping Virtual Desktop Server Manager... systemd[1]: vdsmd.service: State 'stop-sigterm' timed out. Killing. systemd[1]: vdsmd.service: Killing process 132608 (vdsmd) with signal SIGKILL. systemd[1]: vdsmd.service: Killing process 133445 (ioprocess) with signal SIGKILL. systemd[1]: vdsmd.service: Killing process 133446 (ioprocess) with signal SIGKILL. systemd[1]: vdsmd.service: Killing process 133447 (ioprocess) with signal SIGKILL. systemd[1]: vdsmd.service: Main process exited, code=killed, status=9/KILL systemd[1]: vdsmd.service: Failed with result 'timeout'. systemd[1]: Stopped Virtual Desktop Server Manager. And vdsm.log (from a different run, sorry): 2021-11-12 07:09:30,274+0000 INFO (MainThread) [vdsm.api] START prepareForShutdown() from=internal, task_id=21b12bbd-1d61-4217-b92d-641a53d5f7bb (api:48) 2021-11-12 07:09:30,317+0000 DEBUG (vmchannels) [vds] VM channels listener thread has ended. (vmchannels:214) 2021-11-12 07:09:30,317+0000 DEBUG (vmchannels) [root] FINISH thread <Thread(vmchannels, stopped daemon 140163095193344)> (concurrent:261) 2021-11-12 07:09:30,516+0000 DEBUG (mailbox-hsm/4) [root] FINISH thread <Thread(mailbox-hsm/4, stopped daemon 140162799089408)> (concurrent:261) 2021-11-12 07:09:30,517+0000 INFO (ioprocess/143197) [IOProcess] (3a729aa1-8e14-4ea0-8794-e3d67fbde542) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,521+0000 INFO (ioprocess/143199) [IOProcess] (ost-he-basic-suite-master-storage:_exports_nfs_share1) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,535+0000 INFO (ioprocess/143193) [IOProcess] (ost-he-basic-suite-master-storage:_exports_nfs_share2) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,679+0000 INFO (ioprocess/143195) [IOProcess] (0187cf2f-2344-48de-a2a0-dd007315399f) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,719+0000 INFO (ioprocess/143192) [IOProcess] (15fa3d6c-671b-46ef-af9a-00337011fa26) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,756+0000 INFO (ioprocess/143194) [IOProcess] (ost-he-basic-suite-master-storage:_exports_nfs_exported) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,768+0000 INFO (ioprocess/143198) [IOProcess] (ost-he-basic-suite-master-storage:_exports_nfs__he) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,774+0000 INFO (ioprocess/143196) [IOProcess] (a8bab4ef-2952-4c42-ba44-dbb3e1b8c87c) Starting ioprocess (__init__:465) 2021-11-12 07:09:30,957+0000 DEBUG (mailbox-hsm/2) [root] FINISH thread <Thread(mailbox-hsm/2, stopped daemon 140162815874816)> (concurrent:261) 2021-11-12 07:09:31,629+0000 INFO (mailbox-hsm) [storage.mailbox.hsmmailmonitor] HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail (mailbox:500) 2021-11-12 07:09:31,629+0000 INFO (mailbox-hsm) [storage.mailbox.hsmmailmonitor] HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/f54c6052-437f-11ec-9094-54527d140533/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=4096', 'count=1', 'seek=2'] (mailbox:382) 2021-11-12 07:09:32,610+0000 DEBUG (mailbox-hsm/1) [root] FINISH thread <Thread(mailbox-hsm/1, stopped daemon 140162841052928)> (concurrent:261) 2021-11-12 07:09:32,792+0000 DEBUG (mailbox-hsm/3) [root] FINISH thread <Thread(mailbox-hsm/3, stopped daemon 140162807482112)> (concurrent:261) 2021-11-12 07:09:32,818+0000 DEBUG (mailbox-hsm/0) [root] FINISH thread <Thread(mailbox-hsm/0, stopped daemon 140162824267520)> (concurrent:261) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Shutting down domain monitors (monitor:243) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Stop monitoring 15fa3d6c-671b-46ef-af9a-00337011fa26 (shutdown=True) (monitor:268) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Stop monitoring a8bab4ef-2952-4c42-ba44-dbb3e1b8c87c (shutdown=True) (monitor:268) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Stop monitoring 3a729aa1-8e14-4ea0-8794-e3d67fbde542 (shutdown=True) (monitor:268) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Stop monitoring 0187cf2f-2344-48de-a2a0-dd007315399f (shutdown=True) (monitor:268) 2021-11-12 07:09:32,820+0000 INFO (MainThread) [storage.monitor] Stop monitoring 186180cb-5cc5-4aa4-868a-9e1ed7965ddf (shutdown=True) (monitor:268) 2021-11-12 07:09:32,831+0000 INFO (monitor/0187cf2) [storage.check] Stop checking '/rhev/data-center/mnt/ost-he-basic-suite-master-storage:_exports_nfs_exported/0187cf2f-2344-48de-a2a0-dd007315399f/dom_md/metadata' (check:135) 2021-11-12 07:09:32,838+0000 INFO (monitor/a8bab4e) [storage.check] Stop checking '/rhev/data-center/mnt/ost-he-basic-suite-master-storage:_exports_nfs_share1/a8bab4ef-2952-4c42-ba44-dbb3e1b8c87c/dom_md/metadata' (check:135) 2021-11-12 07:09:32,844+0000 INFO (monitor/15fa3d6) [storage.check] Stop checking '/rhev/data-center/mnt/ost-he-basic-suite-master-storage:_exports_nfs__he/15fa3d6c-671b-46ef-af9a-00337011fa26/dom_md/metadata' (check:135) 2021-11-12 07:09:32,864+0000 INFO (monitor/3a729aa) [storage.check] Stop checking '/rhev/data-center/mnt/ost-he-basic-suite-master-storage:_exports_nfs_share2/3a729aa1-8e14-4ea0-8794-e3d67fbde542/dom_md/metadata' (check:135) 2021-11-12 07:09:32,865+0000 INFO (monitor/186180c) [storage.check] Stop checking '/dev/186180cb-5cc5-4aa4-868a-9e1ed7965ddf/metadata' (check:135) 2021-11-12 07:09:32,867+0000 INFO (monitor/186180c) [storage.blocksd] Tearing down domain 186180cb-5cc5-4aa4-868a-9e1ed7965ddf (blockSD:996) 2021-11-12 07:09:32,867+0000 DEBUG (monitor/186180c) [common.commands] /usr/bin/taskset --cpu-list 0-1 /usr/bin/sudo -n /sbin/lvm vgchange --config 'devices { preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter=["a|^/dev/mapper/36001405423a0b45cef54e6e9fd0c7df4$|^/dev/mapper/36001405e9302e3f55e74aedbf352b79f$|", "r|.*|"] hints="none" obtain_device_list_from_udev=0 } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 use_lvmpolld=1 } backup { retain_min=50 retain_days=0 }' --available n 186180cb-5cc5-4aa4-868a-9e1ed7965ddf (cwd None) (commands:154) 2021-11-12 07:09:38,063+0000 DEBUG (monitor/186180c) [common.commands] FAILED: <err> = b' Logical volume 186180cb-5cc5-4aa4-868a-9e1ed7965ddf/ids in use.\n Can\'t deactivate volume group "186180cb-5cc5-4aa4-868a-9e1ed7965ddf" with 1 open logical volume(s)\n'; <rc> = 5 (commands:186) 2021-11-12 07:09:38,066+0000 WARN (monitor/186180c) [storage.lvm] Command with specific filter failed or returned no data, retrying with a wider filter: LVM command failed: 'cmd=[\'/sbin/lvm\', \'vgchange\', \'--config\', \'devices { preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter=["a|^/dev/mapper/36001405423a0b45cef54e6e9fd0c7df4$|^/dev/mapper/36001405e9302e3f55e74aedbf352b79f$|", "r|.*|"] hints="none" obtain_device_list_from_udev=0 } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 use_lvmpolld=1 } backup { retain_min=50 retain_days=0 }\', \'--available\', \'n\', \'186180cb-5cc5-4aa4-868a-9e1ed7965ddf\'] rc=5 out=[] err=[\' Logical volume 186180cb-5cc5-4aa4-868a-9e1ed7965ddf/ids in use.\', \' Can\\\'t deactivate volume group "186180cb-5cc5-4aa4-868a-9e1ed7965ddf" with 1 open logical volume(s)\']' (lvm:532) 2021-11-12 07:09:38,067+0000 DEBUG (monitor/186180c) [common.commands] /usr/bin/taskset --cpu-list 0-1 /usr/bin/sudo -n /sbin/lvm vgchange --config 'devices { preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter=["a|^/dev/mapper/36001405423a0b45cef54e6e9fd0c7df4$|^/dev/mapper/36001405d40a7e761def4ed09aac63282$|^/dev/mapper/36001405e9302e3f55e74aedbf352b79f$|^/dev/mapper/36001405ece10ae48e99470295edda1bb$|^/dev/mapper/36001405f269e8e9efb34e5f9170b349b$|^/dev/mapper/36001405f8c5fe6c239e46a0976b842df$|", "r|.*|"] hints="none" obtain_device_list_from_udev=0 } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 use_lvmpolld=1 } backup { retain_min=50 retain_days=0 }' --available n 186180cb-5cc5-4aa4-868a-9e1ed7965ddf (cwd None) (commands:154) What is interesting here is that ioprocess is started here rather than being killed as on clean vdsmd shutdowns. Any ideas? Thanks, Milan

3 5

Gerrit: ovirt-engine build is unstable
by Saif Abu Saleh 15 Nov '21

15 Nov '21

Hi, Gerrit build on ovirt-engine on master branch seems to be unstable I ran twice and it failed on two errors un-related to my changes, failures: 1- job link: https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/15074/ Errors: fatal: unable to access 'https://gerrit.ovirt.org/ovirt-engine/': Could not resolve host: gerrit.ovirt.org; Unknown error 2- job link: https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/15080/ Errors: [2021-11-14T12:32:21.224Z] No matching package to install: 'ovirt-jboss-modules-maven-plugin >= 2.0.1' [2021-11-14T12:32:21.224Z] Error: Some packages could not be found. irt.oer: Anyone is aware of / checking these failures? Thanks, Saif

2 1

ovirt-ansible-collection on el9
by Sandro Bonazzola 12 Nov '21

12 Nov '21

Hi, repository closure on ovirt-ansible-collection reports broken dependency: package: ovirt-ansible-collection-1.7.0-0.1.master.20211108095017.el9.noarch from copr:copr.fedorainfracloud.org:ovirt:ovirt-master-snapshot unresolved deps: ansible >= 2.9.21 Do we need to provide ansible 2.9.z in CentOS Virt SIG for CentOS 9 Stream? If not, can you please fix the dependency on el9? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 2

resources.ovirt.org migration
by Denis Volkov 11 Nov '21

11 Nov '21

Hello Server `resources.ovirt.org` hosting repositories for ovirt is going to be migrated to new hardware in a different datacenter within the next couple of hours. No service interruption is expected during the migration. I will follow-up when the process is complete -- Denis Volkov

1 1

oVirt SDK docs services.m.html cut in the middle
by Yedidyah Bar David 11 Nov '21

11 Nov '21

Hi all, I was searching for something in the SDK and noticed that [1] is cut in the middle. The last class that appears there is 'TemplateService'. If you scroll down on the left pane and press something below this one, such as [2], you see that the right pane does not scroll down - you are at (close to) its bottom. The last link that seems to be included is [3]. Known issue? Thanks, [1] https://ovirt.github.io/ovirt-engine-sdk/master/services.m.html [2] https://ovirt.github.io/ovirt-engine-sdk/master/services.m.html#TemplateCdr… [3] https://ovirt.github.io/ovirt-engine-sdk/master/services.m.html#TemplateSer… -- Didi

2 1

changes in dbscripts/common_sp.sql functions validation
by Eli Mesika 11 Nov '21

11 Nov '21

Hi All Patch [1] is about to be merged soon and it changes the behaviour of dbscripts/common_sp.sql functions validation. In dbscripts/common_sp.sql we have several helper functions that execute dynamic SQL using the EXECUTE command In order to validate parameters as table name , column name etc, we used to check those parameters with the information stored in the information_schema data Due to that we had some side effects: 1. fn_db_change_column_type did not accept valid types as DECIMAL since in the information_schema data in is named NUMERIC 2. fn_db_change_column_type did not accept precision in the origin column type as NUMERIC(24,4) for example 3. fn_db_change_column_type swallowed any error causing scripts not to perform the action but to consider as successful Patch [1] fixes all those issues and now the comon_sp.sql functions will trust errors thrown by using the EXECUTE command without validating the SQL syntax before. This will resolve issues 1 and 2 listed above In addition, the patch fixes issue 3 by adding the needed upgrade scripts to enforce the changes that were not performed in former upgrade scripts updates. Please feel free to contact me on any issues/questions regarding those changes Regards Eli Mesika [1] https://gerrit.ovirt.org/#/c/ovirt-engine/+/117448/

1 0

C9S/x86_64 Bug 2021809 - gcc and python3 dependencies broken
by Sandro Bonazzola 10 Nov '21

10 Nov '21

Hi, just a heads up CentOS Stream 9 repo for x86_64 is broken. Reported *Bug 2021809* <https://bugzilla.redhat.com/show_bug.cgi?id=2021809> - gcc and python3 dependencies broken Both Jenkins and COPR are failing builds on x86_64 due to that. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

ansible-runner in CentOS Virtualization SIG 4.5/el8s
by Sandro Bonazzola 10 Nov '21

10 Nov '21

Hi, in ovirt-4.5 repo we have ansible-runner-1.4.6-1.el8 <https://cbs.centos.org/koji/buildinfo?buildID=31901> which also provides python3-ansible-runner-1.4.6-1.el8. But we are also consuming openstack-yoga repos and they are shipping python-ansible-runner-1.4.6-3.el8 <https://cbs.centos.org/koji/buildinfo?buildID=30640> but there's no ansible-runner package there. This ends up with repository closure failure: package: ansible-runner-1.4.6-1.el8.noarch from ovirt-master-centos-stream-ovirt45-testing unresolved deps: python3-ansible-runner = 1.4.6-1.el8 Do we need to maintain ansible-runner packaging in CentOS Virt SIG? Should we rebuild ansible-runner with higher NVR? +Martin Necas <mnecas(a)redhat.com> can you please look into this? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 1

ovirt-engine has been tagged (ovirt-engine-4.4.9.5)
by Sandro Bonazzola 09 Nov '21

09 Nov '21

ovirt-engine has been tagged (ovirt-engine-4.4.9.5) -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Gerrit host SSH key changed
by Tomáš Golembiovský 09 Nov '21

09 Nov '21

Hi, it seems there was a change to host SSH keys on gerrit.ovirt.org. Can annyone comment on when and why that has happened? What is the new host key? Thanks, Tomas -- Tomáš Golembiovský <tgolembi(a)redhat.com>

2 1

Publishing oVirt Go modules under ovirt.org namespace
by Nir Soffer 06 Nov '21

06 Nov '21

Currently we have 3 go modules: - github.com/ovirt/go-ovirt https://github.com/oVirt/go-ovirt/ seems that this repo generated by https://github.com/oVirt/ovirt-engine-sdk-go - github.com/ovirt/go-ovirt-client https://github.com/oVirt/go-ovirt-client - github.com/ovirt/go-ovirt-client-log https://github.com/oVirt/go-ovirt-client-log These modules share the issue of depending on the hosting service and the repo the code is located. I started to work on the imageio go module here: https://gerrit.ovirt.org/c/ovirt-imageio/+/117277 And I'm trying to avoid the issues above by naming the module: ovirt.org/imageio The module name does not depend on the hosting service, or on the actual repo or the location in the repo. To make this work, the web server at ovirt.org should serve this resource: https://ovirt.org/imageio returning an HTML document that contains a magic <meta> tag in the page header <meta name="go-import" content="ovirt.org/imageio git https://github.com/ovirt/ovirt-imageio/imageio-go"/> Is this possible with our current infrastructure? Should we rename all the go modules to fit this scheme? Nir

2 4

Is https://ovirt-jira.atlassian.net/ still used?
by Sandro Bonazzola 03 Nov '21

03 Nov '21

Hi, quick question, is someone still monitoring https://ovirt-jira.atlassian.net/ ? Thanks -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 1

ost-images is not mirrored to github
by Sandro Bonazzola 02 Nov '21

02 Nov '21

Hi, looks like ost-images is not mirrored to github.com/oVirt. Infra team: can you please fix this? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

ovirt-engine-nodejs-modules is not mirrored to github
by Sandro Bonazzola 02 Nov '21

02 Nov '21

Hi, looks like ovirt-engine-nodejs-modules is not mirrored to github.com/oVirt. Infra team: can you please fix this? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0