March 2021 - Devel - oVirt List Archives

[VDSM] Started getting virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired
by Edward Haas 11 Jan '24

11 Jan '24

Hi, I have a VM which acts as an oVirt host with VDSM installed. I use it for testing without connecting it to Engine. Recently VDSM fails to come up and I see this error: Sep 26 17:58:18 localhost libvirtd: 2018-09-26 14:58:18.978+0000: 12176: error : virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired Any hint on how to generate or fetch a new certificate without the help of Engine? Thanks, Edy.

3 5

Error: Adding new Host to ovirt-engine
by Ahmad Khiet 14 Aug '23

14 Aug '23

Hi, Can't add new host to ovirt engine, because the following error: 2019-06-12 12:23:09,664 p=4134 u=engine | TASK [ovirt-host-deploy-facts : Set facts] ************************************* 2019-06-12 12:23:09,684 p=4134 u=engine | ok: [10.35.1.17] => { "ansible_facts": { "ansible_python_interpreter": "/usr/bin/python2", "host_deploy_vdsm_version": "4.40.0" }, "changed": false } 2019-06-12 12:23:09,697 p=4134 u=engine | TASK [ovirt-provider-ovn-driver : Install ovs] ********************************* 2019-06-12 12:23:09,726 p=4134 u=engine | fatal: [10.35.1.17]: FAILED! => {} MSG: The conditional check 'cluster_switch == "ovs" or (ovn_central is defined and ovn_central | ipaddr and ovn_engine_cluster_version is version_compare('4.2', '>='))' failed. The error was: The ipaddr filter requires python's netaddr be installed on the ansible controller The error appears to be in '/home/engine/apps/engine/share/ovirt-engine/playbooks/roles/ovirt-provider-ovn-driver/tasks/configure.yml': line 3, column 5, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - block: - name: Install ovs ^ here 2019-06-12 12:23:09,728 p=4134 u=engine | PLAY RECAP ********************************************************************* 2019-06-12 12:23:09,728 p=4134 u=engine | 10.35.1.17 : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 whats missing!? Thanks -- Ahmad Khiet Red Hat <https://www.redhat.com/> akhiet(a)redhat.com M: +972-54-6225629 <https://red.ht/sig>

7 10

Purging inactive maintainers from vdsm-master-maintainers
by Nir Soffer 21 Jun '21

21 Jun '21

I want to remove inactive contributors from vdsm-master-maintainers. I suggest the simple rule of 2 years of inactivity for removing from this group, based on git log. See the list below for current status: https://gerrit.ovirt.org/#/admin/groups/106,members Thoughts? Nir

6 11

Merge rights changes in the oVirt Engine project
by Tal Nisan 09 Apr '21

09 Apr '21

Hi everyone, As you probably know we are now in a mode in which we develop our next zstream version on the master branch as opposed to how we worked before where the master version was dedicated for the next major version. This makes the rapid changes in master to be delivered to customers in a much higher cadence thus affecting stability. Due to that we think it's best that from now on merges in the master branch will be done only by stable branch maintainers after inspecting those closely. What you need to do in order to get your patch merged: - Have it pass Jenkins - Have it get code review +2 - Have it mark verified +1 - It's always encourage to have it tested by OST, for bigger changes it's a must Once you have all those covered, please add me as a reviewer and I'll examine it and merge if everything seems right, if I haven't done it in a timely manner feel free to ping me.

3 4

Error Java SDK Issue??
by Geschwentner, Patrick 02 Apr '21

02 Apr '21

Dear Ladies and Gentlemen! I am currently working with the java-sdk and I encountered a problem. If I would like to retrieve the disk details, I get the following error: Disk currDisk = ovirtConnection.followLink(diskAttachment.disk()); The Error is occurring in this line: [cid:image001.png@01D44537.AF127FD0] The getResponst looks quiet ok. (I inspected: [cid:image002.png@01D44537.AF127FD0] and it looks ok). Error: wrong number of arguments The code is quiet similar to what you published on github (https://github.com/oVirt/ovirt-engine-sdk-java/blob/master/sdk/src/test/jav… ). Can you confirm the defect? Best regards Patrick

6 11

ovirt-engine has been tagged (ovirt-engine-4.4.6.2)
by Tal Nisan 31 Mar '21

31 Mar '21

1 0

ovirt-engine has been tagged (ovirt-engine-4.4.6.1)
by Tal Nisan 24 Mar '21

24 Mar '21

1 0

Jenkins queue pipeline not working
by Sandro Bonazzola 24 Mar '21

24 Mar '21

*Raising issue with pipeline:* *11:47:53* Traceback (most recent call last):*11:47:53* File "/home/cloud-user/workspace/standard-enqueue/jenkins/stdci_tools/usrc.py", line 15, in <module>*11:47:53* from xdg.BaseDirectory import xdg_cache_home*11:47:53* ImportError: No module named xdg.BaseDirectory Seen on http://ovirt-devops-jenkins.upshift.rdu2.redhat.com/job/standard-enqueue/34… -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

1 0

ovirt-engine has been tagged (ovirt-engine-4.4.5.11)
by Martin Perina 23 Mar '21

23 Mar '21

-- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.

2 1

Re: Snapshot Deletion Issue
by Vojtech Juranek 23 Mar '21

23 Mar '21

On Tuesday, 23 March 2021 04:25:25 CET Shubha Kulkarni wrote: > Hello, > > I am investigating an issue with deleting snapshot in oVirt 4.3.10. > Basically the delete snapshot operation fails and I am seeing following > error in VDSM(vdsm-4.30.46) log - > > ============================================================================ > ================================================================= 2021-03-17 > 21:38:01,346-0400 INFO (jsonrpc/1) [virt.vm] > (vmId='4b04639c-386e-463c-8a8a-dfd3bc46d306') Starting merge with > jobUUID=u'e6206d9f-6899-40b7-857a-3be3dd42d77d', original > chain=ac065664-ad6c-4ad6-aea2-9558df71d41d < > 210900dd-ff19-4a00-8706-926cb192b0db < 273fc001-1c18-4234-b2b6-f0485e5d13ef > < e1e10354-ae43-456f-9341-1bb81c5bf960 < > 7985a853-4d47-428b-bb5b-b34a38cc57e0 (top), disk='sda', base='sda[2]', > top='sda[1]', bandwidth=0, flags=8 (vm:5954) 2021-03-17 21:38:01,354-0400 > ERROR (jsonrpc/1) [virt.vm] (vmId='4b04639c-386e-463c-8a8a-dfd3bc46d306') > Live merge failed (job: e6206d9f-6899-40b7-857a-3be3dd42d77d) (vm:5960) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5958, in > merge bandwidth, flags) > File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, > in f ret = attr(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", > line 131, in wrapper ret = f(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, > in wrapper return func(inst, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 719, in > blockCommit if ret == -1: raise libvirtError ('virDomainBlockCommit() > failed', dom=self) libvirtError: internal error: qemu block name > 'json:{"backing": {"backing": {"backing": {"driver": "raw", "file": > {"driver": "file", "filename": > "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4 > 743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/ac065664-a > d6c-4ad6-aea2-9558df71d41d"}}, "driver": "qcow2", "file": {"driver": "file", > "filename": > "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4 > 743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/210900dd-f > f19-4a00-8706-926cb192b0db"}}, "driver": "qcow2", "file": {"driver": "file", > "filename": > "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4 > 743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/273fc001-1 > c18-4234-b2b6-f0485e5d13ef"}}, "driver": "qcow2", "file": {"driver": "file", > "filename": > "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4 > 743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/e1e10354-a > e43-456f-9341-1bb81c5bf960"}}' doesn 2021-03-17 21:38:01,368-0400 INFO > (jsonrpc/1) [api.virt] FINISH merge return={'status': {'message': 'Merge > failed', 'code': 52}} > =========================================================================== > ================================================================= > > I found out that there is an issue logged here > https://bugzilla.redhat.com/show_bug.cgi?id=1785939. However I am not able > to see how it was fixed see https://bugzilla.redhat.com/show_bug.cgi?id=1785939#c10 > and whether it was merged in oVirt. Where can I > find the relevant commit (in vdsm and/or libvirt)? > > Thanks

2 1

Snapshot Deletion Issue
by Shubha Kulkarni 23 Mar '21

23 Mar '21

Hello, I am investigating an issue with deleting snapshot in oVirt 4.3.10. Basically the delete snapshot operation fails and I am seeing following error in VDSM(vdsm-4.30.46) log - ============================================================================================================================================= 2021-03-17 21:38:01,346-0400 INFO (jsonrpc/1) [virt.vm] (vmId='4b04639c-386e-463c-8a8a-dfd3bc46d306') Starting merge with jobUUID=u'e6206d9f-6899-40b7-857a-3be3dd42d77d', original chain=ac065664-ad6c-4ad6-aea2-9558df71d41d < 210900dd-ff19-4a00-8706-926cb192b0db < 273fc001-1c18-4234-b2b6-f0485e5d13ef < e1e10354-ae43-456f-9341-1bb81c5bf960 < 7985a853-4d47-428b-bb5b-b34a38cc57e0 (top), disk='sda', base='sda[2]', top='sda[1]', bandwidth=0, flags=8 (vm:5954) 2021-03-17 21:38:01,354-0400 ERROR (jsonrpc/1) [virt.vm] (vmId='4b04639c-386e-463c-8a8a-dfd3bc46d306') Live merge failed (job: e6206d9f-6899-40b7-857a-3be3dd42d77d) (vm:5960) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5958, in merge bandwidth, flags) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 719, in blockCommit if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self) libvirtError: internal error: qemu block name 'json:{"backing": {"backing": {"backing": {"driver": "raw", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/ac065664-ad6c-4ad6-aea2-9558df71d41d"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/210900dd-ff19-4a00-8706-926cb192b0db"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/273fc001-1c18-4234-b2b6-f0485e5d13ef"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/nash-nfs7:_nfs_shubha__ovirt__2_data/8bce5be7-aede-4743-b3fd-1c199880892f/images/eb3cb8f2-9544-472f-946b-646eab9c621f/e1e10354-ae43-456f-9341-1bb81c5bf960"}}' doesn 2021-03-17 21:38:01,368-0400 INFO (jsonrpc/1) [api.virt] FINISH merge return={'status': {'message': 'Merge failed', 'code': 52}} ============================================================================================================================================ I found out that there is an issue logged here https://bugzilla.redhat.com/show_bug.cgi?id=1785939. However I am not able to see how it was fixed and whether it was merged in oVirt. Where can I find the relevant commit (in vdsm and/or libvirt)? Thanks

1 0

Re: docs: pointers to more in-depth internals?
by Nir Soffer 22 Mar '21

22 Mar '21

On Tue, Mar 16, 2021 at 10:47 PM Greg King <greg.king(a)oracle.com> wrote: > I am new to vdsm and trying to understand the architecture/internals much > better > Welcome to vdsm Greg! The ovirt documentation for architecture I have found so far seems to be > relatively high level > And it is mostly outdated, but we don't have anything better. > My effort to understand the architecture by walking through the vdsm code > using pdb/rpdb is slow and probably not all that efficient > > > > Does anyone have pointers to documentation that might explain the vdsm > modules, classes and internals a little more in depth? > I don't think we have more detailed documentation, but there are lot of talks and slide decks that give more info on specific topics, and are usually are more updated: https://www.ovirt.org/community/archived_conferences_presentations.html There is also lot of content on youtube, here some example that I could find easily: - [oVirt 3.6 deep dive] - live storage migration between mixed domains https://www.youtube.com/watch?v=BPy29Q__VV4 - oVirt 4.1 deep dive - VM leases https://www.youtube.com/watch?v=MVa-4fQo2V8 - Back to the future – incremental backup in oVirt https://www.youtube.com/watch?v=X-xHD9ddN6s - oVirt 4k - teaching an old dog new tricks https://www.youtube.com/watch?v=Q1VQxjYEzDY > > I’d also like to understand where I might be able to add rpdb.set_trace() > so I can step through functions being called in libvirt.py > I don't think using a debugger is very helpful with vdsm, since vdsm is not designed for stopping a thread for unlimited time. In some cases the system will log warning and traceback every 60 seconds about blocked worker. In other cases monitoring code may fail to update stats, which may cause engines to deactivate a host or migrate vms or other trouble. The best way to debug and understand vdsm is to follow the logs, and add move logs when needed. The main advantage compared with a debugger is that the time spent with the logs will pay back when you have to debug real issues in user setup, when logs are the only available resource. Having said that, being able to follow the entire flow by printing a traceback is a great way to understand how the system works. You can use vdsm.common.concurrent.format_traceback: https://github.com/oVirt/vdsm/blob/114121ab122a0cd5e529807b938b3506f247f42b… To print traceback at interesting points. For tracing function from the libvirt python binginding, you can modify libvirtconnection.py: https://github.com/oVirt/vdsm/blob/114121ab122a0cd5e529807b938b3506f247f42b… This module creates a connection, and wraps libvirt.virDomain with a wrapper that panics on fatal errors. You can modify the wrapper to log a traceback for all or some of libvirt.virDomain functions. Another option it to modify the virDomain wrapper to log a traceback: https://github.com/oVirt/vdsm/blob/114121ab122a0cd5e529807b938b3506f247f42b… For example here: https://github.com/oVirt/vdsm/blob/114121ab122a0cd5e529807b938b3506f247f42b… Good luck with your vdsm ride! Nir

2 1

ovirt-system-tests hosted-engine suites and CentOS Stream - current status
by Yedidyah Bar David 18 Mar '21

18 Mar '21

Hi all, Right now, ovirt appliance is built nightly, and publishes (to ovirt-master-snapshot) two packages: ovirt-engine-appliance (the "regular" one, based on CentOS Linux) and ovirt-engine-appliance-centos-stream (based on CentOS Stream). This can already be used by anyone that wants to try this manually. For trying in CI: 1. A pending patch to ost-images [1] can be merged once it passes the manual build I ran (link in a gerrit comment there). Should take a few more hours. This patch changes the existing image that was generated, called ost-images-el8stream-he-installed, to include a stream appliance (from current repos). 2. A patch for OST [2] can be used, as needed, for manual runs. I updated it now to expect the output of [1] once it's merged, so should probably be rebased and/or 'ci test/ci build' once [1] is merged (and image is published). TODO: 1. Decide how we want to continue, going forward. If we aim at complete move to Stream in 4.4.6, perhaps now is the time to start... An alternative is to somehow support both in parallel - will be more complex, obviously. 2. Handle ovirt-node Best regards, [1] https://gerrit.ovirt.org/c/ost-images/+/113633 [2] https://gerrit.ovirt.org/c/ovirt-system-tests/+/112977 -- Didi

1 0

ovirt-engine has been tagged (ovirt-engine-4.4.6)
by Tal Nisan 18 Mar '21

18 Mar '21

1 0

Re: basic suite fails on test_metrics_and_log_collector
by Michal Skrivanek 18 Mar '21

18 Mar '21

> On 17. 3. 2021, at 13:53, Dana Elfassy <delfassy(a)redhat.com> wrote: > > Adding +Marcin Sobczyk <mailto:msobczyk@redhat.com> > > On Mon, Mar 15, 2021 at 9:59 AM Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: > On Mon, Mar 15, 2021 at 7:55 AM Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: > > > > Hi all, > > > > This started a few days ago [1] and randomly happens since then: > > > > E DEBUG: Configuration: > > E DEBUG: command: collect > > E DEBUG: Traceback (most recent call last): > > E DEBUG: File > > "/usr/lib/python3.6/site-packages/ovirt_log_collector/__main__.py", > > line 2067, in <module> > > E DEBUG: '%s directory is not empty.' % (conf["local_tmp_dir"]) > > E DEBUG: Exception: /dev/shm/log directory is not > > empty.ERROR: /dev/shm/log directory is not empty.non-zero return code > > > > Michal tried to fix this by using a random directory but it still fails [2]: > > > > DEBUG: command: collect > > DEBUG: Traceback (most recent call last): > > DEBUG: File "/usr/lib/python3.6/site-packages/ovirt_log_collector/__main__.py", > > line 2067, in <module> > > DEBUG: '%s directory is not empty.' % (conf["local_tmp_dir"]) > > DEBUG: Exception: /dev/shm/kaN7uY directory is not empty.ERROR: > > /dev/shm/kaN7uY directory is not empty.non-zero return code > > > > Since I suppose that the randomness of mktemp is good enough, it must > > be something else. Also, the last successful run before [1] used the > > same OST git commit (same code), so I do not think it's something in > > OST's code. > > > > Any idea? > > > > I think I'll push a patch to create and use the directory right before > > calling ovirt-log-collector, which is probably better in other ways. > > My patch [1] still fails, with a somewhat different error message, but > this made me check further, and while I still do not understand, I have > this to add: > > In the failing runs, ovirt-log-collector is called *twice* in parallel. E.g. > in [2] (the check-patch of [1]): > > Mar 15 07:38:59 lago-basic-suite-master-engine platform-python[59099]: > ansible-command Invoked with _raw_params=lctmp=$(mktemp -d -p > /dev/shm); ovirt-log-collector --verbose --batch --no-hypervisors > --local-tmp="${lctmp}" --conf-file=/root/ovirt-log-collector.conf > _uses_shell=True warn=True stdin_add_newline=True > strip_empty_ends=True argv=None chdir=None executable=None > creates=None removes=None stdin=None > Mar 15 07:38:59 lago-basic-suite-master-engine platform-python[59124]: > ansible-command Invoked with _raw_params=lctmp=$(mktemp -d -p > /dev/shm); ovirt-log-collector --verbose --batch --no-hypervisors > --local-tmp="${lctmp}" --conf-file=/root/ovirt-log-collector.conf > _uses_shell=True warn=True stdin_add_newline=True > strip_empty_ends=True argv=None chdir=None executable=None > creates=None removes=None stdin=None > > It also generates two logs, which you can check/compare. > > It's the same for previous ones, e.g. latest nightly [3][4]: > > Mar 15 06:23:30 lago-basic-suite-master-engine platform-python[59343]: > ansible-command Invoked with _raw_params=ovirt-log-collector --verbose > --batch --no-hypervisors --conf-file=/root/ovirt-log-collector.conf > _uses_shell=True warn=True stdin_add_newline=True > strip_empty_ends=True argv=None chdir=None executable=None > creates=None removes=None stdin=None > Mar 15 06:23:30 lago-basic-suite-master-engine setroubleshoot[58889]: > SELinux is preventing /usr/lib/systemd/systemd from unlink access on > the sock_file ansible-ssh-lago-basic-suite-master-host-1-22-root. For > complete SELinux messages run: sealert -l > d03a8655-9430-4fcf-9892-3b4df1939899 > Mar 15 06:23:30 lago-basic-suite-master-engine setroubleshoot[58889]: > SELinux is preventing /usr/lib/systemd/systemd from unlink access on > the sock_file ansible-ssh-lago-basic-suite-master-host-1-22-root.#012#012***** > Plugin catchall (100. confidence) suggests > **************************#012#012If you believe that systemd should > be allowed unlink access on the > ansible-ssh-lago-basic-suite-master-host-1-22-root sock_file by > default.#012Then you should report this as a bug.#012You can generate > a local policy module to allow this access.#012Do#012allow this access > for now by executing:#012# ausearch -c 'systemd' --raw | audit2allow > -M my-systemd#012# semodule -X 300 -i my-systemd.pp#012 > Mar 15 06:23:30 lago-basic-suite-master-engine platform-python[59361]: > ansible-command Invoked with _raw_params=ovirt-log-collector --verbose > --batch --no-hypervisors --conf-file=/root/ovirt-log-collector.conf > _uses_shell=True warn=True stdin_add_newline=True > strip_empty_ends=True argv=None chdir=None executable=None > creates=None removes=None stdin=None > > Any idea what might have caused this to start happening? Perhaps > a bug in ansible, or ansible-runner? It reminds me of [5]. > Adding Dana and Martin. > > I think [5] is quite a serious bug, btw, should be a 4.4.5 blocker. it’s from January and tehre are no comments there. Dana, any update? It does look serious but perhaps not really hit in real world scenarios? > > Best regards, > > [1] https://gerrit.ovirt.org/c/ovirt-system-tests/+/113875 <https://gerrit.ovirt.org/c/ovirt-system-tests/+/113875> > > [2] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/15980… <https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/15980…> > > [3] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… <https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly…> > > [4] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… <https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly…> > > [5] https://bugzilla.redhat.com/show_bug.cgi?id=1917707 <https://bugzilla.redhat.com/show_bug.cgi?id=1917707> > > > > > Best regards, > > > > [1] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… <https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly…> > > > > [2] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… <https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly…> > > > > > > -- > > Didi > > > > -- > Didi > > _______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/PSO5IRX73FSH2…

3 3

Migrate VMs between DCs
by Daniel Gurgel 18 Mar '21

18 Mar '21

How to migrate virtual servers between different data centers that do not share the same SAN? There are limitations, including connection technology, depending on the Storage Model used. For example, Dell PowerStore uses only 1 iSCSI network and Dell EMC Compellent, 2 separate iSCSI networks. This connection is not possible, even for security and redundancy reasons. Exports Domains are marked as discontinued in 4.4.x. Are there plans for 4.4.x or 4.5.x Live Migrate Storage to be implemented?

3 5

Re: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 962 - Still Failing!
by Yedidyah Bar David 17 Mar '21

17 Mar '21

On Tue, Mar 16, 2021 at 7:06 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… > Build Number: 962 > Build Status: Still Failing > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #953 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #954 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #955 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #956 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #957 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #958 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #959 > [Michal Skrivanek] randomize /dev/shm logcollector tmp directory > > > Changes for Build #960 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > Changes for Build #961 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > Changes for Build #962 > [Andrej Cernek] pylint: Upgrade to 2.7 > > > > > ----------------- > Failed Tests: > ----------------- > 1 tests failed. > FAILED: basic-suite-master.test-scenarios.test_001_initialize_engine.test_set_hostnames > > Error Message: > failed on setup with "TypeError: __new__() missing 2 required positional arguments: 'version' and 'repo'" > > Stack Trace: > ansible_by_hostname = <function module_mapper_for at 0x7ffbad0acc80> > > @pytest.fixture(scope="session", autouse=True) > def check_installed_packages(ansible_by_hostname): > vms_pckgs_dict_list = [] > for hostname in backend.default_backend().hostnames(): > vm_pckgs_dict = _get_custom_repos_packages( > > ansible_by_hostname(hostname)) > > ost_utils/ost_utils/pytest/fixtures/check_repos.py:39: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ost_utils/ost_utils/pytest/fixtures/check_repos.py:55: in _get_custom_repos_packages > repo_name) > ost_utils/ost_utils/pytest/fixtures/check_repos.py:69: in _get_installed_packages > Package(*line) for line in result > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > .0 = <list_iterator object at 0x7ffba6e97860> > > > Package(*line) for line in result > ] > E TypeError: __new__() missing 2 required positional arguments: 'version' and 'repo' This failed, because 'dnf repo-pkgs' has split the output to two lines, so the first didn't include a version [1]: lago-basic-suite-master-host-1 | CHANGED | rc=0 >> Installed Packages ovirt-ansible-collection.noarch 1.3.2-0.1.master.20210315141358.el8 @extra-src-1 python3-ovirt-engine-sdk4.x86_64 4.4.10-1.20210315.gitf8b9f2a.el8 @extra-src-1 We should either give up on this, or rewrite the call 'dnf repo-pkgs' in some other language that does not require parsing of human-targeted output (perhaps python or ansible), or amend a bit the current code and hope it will survive longer... Trying last one: https://gerrit.ovirt.org/c/ovirt-system-tests/+/113895 [1] https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… > > ost_utils/ost_utils/pytest/fixtures/check_repos.py:69: TypeError_______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/HBWJEEDHVVROL… -- Didi

2 4

docs: pointers to more in-depth internals?
by Greg King 16 Mar '21

16 Mar '21

I am new to vdsm and trying to understand the architecture/internals much better The ovirt documentation for architecture I have found so far seems to be relatively high level My effort to understand the architecture by walking through the vdsm code using pdb/rpdb is slow and probably not all that efficient Does anyone have pointers to documentation that might explain the vdsm modules, classes and internals a little more in depth? I'd also like to understand where I might be able to add rpdb.set_trace() so I can step through functions being called in libvirt.py [oracle-email-sig-198324-355094] Gregory King | Software Development Manager | +1.303.272.2427 Oracle Virtualization Sustaining Engineering 500 Eldorado Boulevard Build 5 | Broomfield Colorado 80021 Mobile: +1.303.968.8169 | Fax: +1.303.272.2427

1 0

Jenkins CI broken
by Sandro Bonazzola 16 Mar '21

16 Mar '21

Hi, looks like we have some issues in Jenkins: https://jenkins.ovirt.org/job/imgbased_standard-check-patch/663/console We have an issue with our local mirror which is quite outdated (2020-10-08) *08:45:53* Loading mirror speeds from cached hostfile*08:45:53* http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2020-10-08-04-48/repodata/c…: [Errno 14] HTTP Error 404 - Not Found*08:45:53* Trying other mirror.*08:45:53* To address this issue please refer to the below wiki article *08:45:53* *08:45:53* https://wiki.centos.org/yum-errors*08:45:53* *08:45:53* If above article doesn't help to resolve this issue please use https://bugs.centos.org/.*08:45:53* *08:45:54* http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2020-10-08-04-48/repodata/c…: [Errno 14] HTTP Error 404 - Not Found*08:45:54* Trying other mirror.*08:45:54* No package python36-docker available.*08:45:54* Error: Nothing to do and it's missing a python36-docker package which cause the global setup of the job to fail: *08:45:54* global_setup[docker_setup] ERROR: Failed to install Docker Python36 API.*08:45:54* + return 1*08:45:54* + failed=true There's also some weird firewalld configuration which is failing: *08:45:54* + sudo -nl firewall-cmd*08:45:54* ++ sudo -n firewall-cmd --query-service=ovirtlago*08:46:03* Error: INVALID_SERVICE: ovirtlago Can you please have a look? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Engine upgrade fails with virtio_scsi_multi_queues_enabled does not exist
by Vojtech Juranek 16 Mar '21

16 Mar '21

Hi, when I try to upgrade my engine with built artifacts (e.g. [1]), it fails with: psql:/usr/share/ovirt-engine/dbscripts/create_views.sql:1000: ERROR: column vm_templates.virtio_scsi_multi_queues_enabled does not exist LINE 86: vm_templates.virtio_scsi_multi_queues_enabled AS virtio_... ^ FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/create_views.sql Same issue happens when I tried to update my dev engine with local build. However, OST is passing [2]. Any idea what's wrong? I don't see anything wrong in [3], which is probably the culprit, but I haven't much experience with engine/db though. Or maybe some issue on my side? Thanks Vojta [1] https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/11174/artif… [2] https://jenkins.ovirt.org/job/ovirt-system-tests_manual/7880/ [3] https://github.com/oVirt/ovirt-engine/commit/52caf76e60ace2266d8283c131d467…

5 8

ovirt-engine has been tagged (ovirt-engine-4.4.5.10)
by Tal Nisan 15 Mar '21

15 Mar '21

1 0

Where to place useful Ansible recipe?
by Vojtech Juranek 11 Mar '21

11 Mar '21

Hi, the result of working on BZ #1310330 [1] is Ansible recipe, which removes multipath devices for unzoned LUN from the hosts. The recipe is attached to the BZ, but it would be good to put it somewhere in into the code base. IIUC Ansible recipes contained in engine project are used internally by the engine, but this is not the right place (can be in the future if we implement some UI in the engine which will trigger this recipe). Is ovirt-engine/contrib [2] the right place or is there a better place? Thanks Vojta [1] http://bugzilla.redhat.com/1310330 [2] https://github.com/oVirt/ovirt-engine/tree/master/contrib

3 2

corrupted jar ; jackson-mapper dependency 4.4.5?
by Prajith Kesava Prasad 11 Mar '21

11 Mar '21

Hi everyone, So while calling createBrickCommand playbook from ovirt-engine (createBrickCommand.java) a parameter called disks are constructed to parse a list of disks. Its implementation contains a class variable ArrayNode which uses JsonNodeFactory, from jackson-mapper dependency . Usually parsing of disks to playbook would contain a list of disks, but now it's parsing the absurd values of JsonNodeFactory, and parsing a weird dictionary:- Snip of the parameters parsed from ovirt-engine to ansible-runner *<snip>elements': [{'textValue': '/dev/sdc', 'binaryValue': '/dev/sdc', 'textual': True, 'valueNode': True, 'numberType': None, 'elements': [], 'array': False, 'fields': [], 'null': False, 'binary': False, 'booleanValue': False, 'numberValue': None, 'decimalValue': 0, 'bigIntegerValue': 0, 'valueAsText': '/dev/sdc', 'valueAsInt': 0, 'valueAsLong': 0, 'valueAsDouble': 0.0, 'valueAsBoolean': False, 'containerNode': False, 'missingNode': False, 'pojo': False, 'number': False, 'integralNumber': False, 'floatingPointNumber': False, 'int': False, 'long': False, 'double': False, 'bigDecimal': False, 'bigInteger': False, 'boolean': False, 'object': False, 'fieldNames': [], 'longValue': 0, 'doubleValue': 0.0, 'intValue': 0}], 'array': True, 'valueAsText': None, 'containerNode': True, 'numberType': None, 'fields': [], 'null': False, 'binary': False, 'textValue': None, 'binaryValue': None, 'booleanValue': False, 'numberValue': None, 'decimalValue': 0, 'bigIntegerValue': 0, 'valueAsInt': 0, 'valueAsLong': 0, 'valueAsDouble': 0.0, 'valueAsBoolean': False, 'valueNode': False, 'missingNode': False, 'pojo': False, 'number': False, 'integralNumber': False, 'floatingPointNumber': False, 'int': False, 'long': False, 'double': False, 'bigDecimal': False, 'bigInteger': False, 'textual': False, 'boolean': False, 'object': False, 'fieldNames': [], 'longValue': 0, 'doubleValue': 0.0, 'intValue': 0}</snip>* these values represent the Class variables of JsonNodeFactory class in the Jackson-mapper dependency instantiated from ArrayNode in CreateBrickCommand.java class. Is my understanding flawed, or did we upgrade to a maybe corrupted dependency? or does this have any other problem which I missed out on? ( tested with RHV 4.4.5 latest build - 4.4.5.7-0.1.el8ev ) Regards, Prajith Kesava Prasad.

2 1

basic-suite fails with: "... is not signed Error: GPG check FAILED"
by Yedidyah Bar David 10 Mar '21

10 Mar '21

Hi all, Now happened in [1]: STDERR b'warning: /var/cache/dnf/ovirt-master-copr:copr.fedorainfracloud.org:sbonazzo:EL8_collection-7f7c2d939ff8f7bc/packages/python3-pyyaml-5.1.2-3.el8.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY\nImporting GPG key 0x119783D1:\n Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection(a)copr.fedorahosted.org>"\n Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1\n From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pub… ovirt-engine-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-backend-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-dbscripts-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-restapi-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-base-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-cinderlib-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-imageio-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-ovirt-engine-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-ovirt-engine-common-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-setup-plugin-websocket-proxy-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-tools-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-tools-backup-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-vmconsole-proxy-helper-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-webadmin-portal-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage ovirt-engine-websocket-proxy-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nPackage python3-ovirt-engine-lib-4.4.6-0.0.master.20210310114158.git5d97629b1af.el8.noarch.rpm is not signed\nError: GPG check FAILED\n' Known issue? I have vague memories about this being reported, not sure. Perhaps we publish to tested repo before signing? And if a job runs in the middle, it will get unsigned packages? Thanks and best regards, [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/15924/ -- Didi

1 0

Support for 5K volumes
by Sandro Bonazzola 09 Mar '21

09 Mar '21

Hi, Gluster 9 introduced support for 5k volumes ( https://docs.gluster.org/en/latest/release-notes/9.0/ ). Is there interest in getting this in oVirt too? Any thoughts about rebasing on Gluster 9 for oVirt 4.4.6? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

4 3