- Devel - oVirt List Archives

Vdsm: ovirt-4.4.z branch created
by Milan Zamazal 19 Aug '21

19 Aug '21

Hi, similarly to Engine, as announced here today, Vdsm branch ovirt-4.4.z for further 4.4 development was created. Regular development continues on master, with 4.5 as the upcoming release. Bug fixes that should go to 4.4 must be backported from master to ovirt-4.4.z. Regards, Milan

1 0

[IMPORTANT] ovirt-engine-4.4 branch was created, master branch will be used for upcoming oVirt Engine 4.5 version
by Martin Perina 19 Aug '21

19 Aug '21

Hi, Earlier today the ovirt-engine-4.4 branch was created. This branch should be used for fixes for upcoming oVirt Engine 4.4.z release: https://gerrit.ovirt.org/q/project:ovirt-engine+branch:ovirt-engine-4.4 Master branch is going to be used for the upcoming oVirt Engine 4.5 release. This means that any fix for oVirt Engine 4.4.z needs to be merged to master first and only then backported to ovirt-engine-4.4 branch. Also please be aware that database upgrade scripts on master should start with 04_05 prefix, so all patches which are not yet merged on master and contain database upgrade scripts should be updated to 4.5 numbering, for example: https://gerrit.ovirt.org/116267 Thanks, Martin -- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.

1 0

Re: RHV and oVirt CBT issue
by luwen.zhang 18 Aug '21

18 Aug '21

Thank you so much for the update Pavel! Ok, we will try to check the DB leftovers and will share with you if we find something new. But so far we haven’t seen the same problem again. Best regards! Luwen Zhang ------------------ Original ------------------ From: Pavel Bar <pbar(a)redhat.com> Date: Tue,Aug 17,2021 11:30 PM To: luwen.zhang <luwen.zhang(a)vinchin.com>, huangchuanbo(a)vinchin.com <huangchuanbo(a)vinchin.com> Cc: devel(a)ovirt.org <devel(a)ovirt.org> Subject: Re: [ovirt-devel] Re: RHV and oVirt CBT issue Hi,After investigating the logs I believe that there were few issues that might have caused this unstable behavior. 1) Looks like the DB still contained some leftovers from the already fixed bug which was causing duplicate entries. Even after the date of engine upgrade I still saw some "Duplicate key" errors. I suggest cleaning the DB (if that is possible). 2) Also saw a few other known (and already reported) issues that also might have caused the problem. The issues are going to be fixed in the coming 4.4.9 version, resulting in more stability. I asked our QE team to inform me if they encounter this 'disks stuck in FINALIZING' issue, so I'll be able to try and investigate it locally. But again, this issue might be caused by either some DB leftovers or issues that we are already WIP fixing them. If you opened a bug, please send the link to it here. Thank you in advance. Pavel On Wed, 11 Aug 2021 at 15:24, Pavel Bar <pbar(a)redhat.com> wrote: Hi,Just a short status update FYI. The issue is not forgotten, I'm looking at it. Found in logs a few known issues that are gonna be fixed in the next version, but still not sure that the issue you described will be fixed by them. WIP, will continue updating. Thank you! Pavel On Sun, 1 Aug 2021 at 10:38, Eyal Shenitzky <eshenitz(a)redhat.com> wrote: +Pavel Bar +Benny Zlotnik  On Sun, 1 Aug 2021 at 08:53, luwen.zhang <luwen.zhang(a)vinchin.com> wrote: Sorry I was trying to open a new thread for this issue, but it seems I failed to submit. Here let me explain how the issue is reproduced. It’s a regular backup by using CBT+imageip API, after a series of successful backup, at one of the backup session beginning, when we try to obtain the VM config and the snapshot list (obtain snapshot list can determine the VM virtual disk format is RAW or QCOW2) by using `GET vms/<vm-id>/snapshots`, but get the following error. <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <fault>     <detail>duplicate key acf1edaa-e950-4c4f-94df-1bd6b3da49c1 (attempted merging values org.ovirt.engine.core.common.businessentities.storage.diskimage@5103046c and org.ovirt.engine.core.common.businessentities.storage.diskimage@d973046c)</detail>     <reason>Operation Failed</reason> </fault>  After this, on oVirt engine web console, the VM show 2 disks (actually it only has 1) , and the disk status always showing “Finalizing”, it’s been more than 30 hours now, and during this, cannot modify VM disk or take snapshots. Before upgrading oVirt engine to 4.4.7.7-1.el8 this problem happened frequently, after upgrading, the frequency is reduced. Here I’m adding the engine logs and vdsm logs. Engine logs: https://drive.google.com/file/d/1T3-EOxYYl3oFZOA9VMMBte5WyBoUO48… VDSM logs: https://drive.google.com/file/d/1x0B8lGqnKEDrgn666CuN3hqUGwD7fcY… Thanks & regards! On 07/29/2021 19:20，Nir Soffer<nsoffer(a)redhat.com> wrote： On Thu, Jul 29, 2021 at 10:08 AM luwen.zhang <luwen.zhang(a)vinchin.com> wrote: The problem occurred yesterday, but we waited for more than 20 hours, still 2 disks and in Finalizing state. If the image transfer is "finalizing" it means the image transfer is trying to finalize, but the operation could not complete. In this phase the disk remains locked, and it should not be possible to start a new image transfer (e.g perform another backup). Engine and vdsm logs should explain why the image transfer is stuck in the finalizing phase. Can you add detailed instructions on how to reproduce this issue? _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/V2AEZX2V6JICY… -- Regards, Eyal Shenitzky

1 0

Re: RHV and oVirt CBT issue
by Eyal Shenitzky 17 Aug '21

17 Aug '21

+Pavel Bar <pbar(a)redhat.com> +Benny Zlotnik <bzlotnik(a)redhat.com> On Sun, 1 Aug 2021 at 08:53, luwen.zhang <luwen.zhang(a)vinchin.com> wrote: > > Sorry I was trying to open a new thread for this issue, but it seems I > failed to submit. Here let me explain how the issue is reproduced. > > It’s a regular backup by using CBT+imageip API, after a series of > successful backup, at one of the backup session beginning, when we try to > obtain the VM config and the snapshot list (obtain snapshot list can > determine the VM virtual disk format is RAW or QCOW2) by using `GET > vms/<vm-id>/snapshots`, but get the following error. > > <?xml version="1.0" encoding="UTF-8" standalone="yes"?> > > <fault> > > <detail>duplicate key acf1edaa-e950-4c4f-94df-1bd6b3da49c1 (attempted > merging values > org.ovirt.engine.core.common.businessentities.storage.diskimage@5103046c > and org.ovirt.engine.core.common.businessentities.storage.diskimage@d973046c > )</detail> > > <reason>Operation Failed</reason> > </fault> > > After this, on oVirt engine web console, the VM show 2 disks (actually it > only has 1) , and the disk status always showing “Finalizing”, it’s been > more than 30 hours now, and during this, cannot modify VM disk or take > snapshots. > > Before upgrading oVirt engine to 4.4.7.7-1.el8 this problem happened > frequently, after upgrading, the frequency is reduced. > > Here I’m adding the engine logs and vdsm logs. > Engine logs: > https://drive.google.com/file/d/1T3-EOxYYl3oFZOA9VMMBte5WyBoUO48U/view?usp=… > VDSM logs: > https://drive.google.com/file/d/1x0B8lGqnKEDrgn666CuN3hqUGwD7fcYv/view?usp=… > > Thanks & regards! > On 07/29/2021 19:20，Nir Soffer<nsoffer(a)redhat.com> <nsoffer(a)redhat.com> > wrote： > > On Thu, Jul 29, 2021 at 10:08 AM luwen.zhang <luwen.zhang(a)vinchin.com> > wrote: > > The problem occurred yesterday, but we waited for more than 20 hours, > still 2 disks and in Finalizing state. > > > If the image transfer is "finalizing" it means the image transfer is > trying to finalize, but the operation could not complete. > > In this phase the disk remains locked, and it should not be possible > to start a new image transfer > (e.g perform another backup). > > Engine and vdsm logs should explain why the image transfer is stuck in > the finalizing phase. > > Can you add detailed instructions on how to reproduce this issue? > > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/V2AEZX2V6JICY… > -- Regards, Eyal Shenitzky

2 2

Vdsm: ovirt-4.4.8 branch created
by Milan Zamazal 16 Aug '21

16 Aug '21

Hi, ovirt-4.4.8 branch has been created for 4.4.8 backports. You can merge 4.4.9 stuff to master now. I'm aware about the current CI problems and that we may need fixes for the new branch. Once the CI problems are resolved we can decide what contingent fixes to apply in ovirt-4.4.8 branch. Regards, Milan

1 0

Vdsm 4.4.8 branch?
by Milan Zamazal 16 Aug '21

16 Aug '21

Hi, we are after code freeze for 4.4.8 now. Can we create a 4.4.8 branch in Vdsm now and start 4.4.9 development on master or does anybody need to postpone the branch? Thanks, Milan

3 2

Inconsistent Jenkins node time zones - use UTC?
by Nir Soffer 13 Aug '21

13 Aug '21

We found a broken test that fail when running on EDT timezone: https://gerrit.ovirt.org/c/vdsm/+/116187 However this test fails randomly. Looking at the run logs, we see that some nodes are using UTC: [2021-08-13T15:04:26.855Z] ## Fri Aug 13 15:04:26 UTC 2021 Running env: el8stream:centos-stream-8-x86_64 But others use EDT: [2021-08-13T16:32:26.866Z] ## Fri Aug 13 12:32:26 PM EDT 2021 Running env: el8stream:centos-stream-8-x86_64 Run logs: - https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/vdsm_st… - https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/vdsm_st… Having different time zones on the same system wastes time for everyone. Please use the same timezone on all systems. I think using UTC will make everyone life easier since developers and admins are located on multiple timezones and everyone know what is UTC. Nir

1 0

[VDSM] CI fails in QemuGuestAgentTests.test_pci_devices
by Nir Soffer 12 Aug '21

12 Aug '21

VDSM CI is failing now consistently. Last related change is: https://gerrit.ovirt.org/c/vdsm/+/116090 Build: https://jenkins.ovirt.org/job/vdsm_standard-check-patch/29212//artifact/che… _____________________ QemuGuestAgentTests.test_pci_devices _____________________ self = <virt.qemuguestagent_test.QemuGuestAgentTests testMethod=test_pci_devices> def test_pci_devices(self): devices = self.qga_poller._qga_call_get_devices(self.vm)['pci_devices'] # Ethernet is returned twice by the agent but should appear only # once in the list assert len(devices) == 2 eth = [d for d in devices if d['device_id'] == 4096][0] assert eth == { 'device_id': 4096, 'driver_date': '2019-08-12', 'driver_name': 'Red Hat VirtIO Ethernet Adapter', 'driver_version': '100.80.104.17300', 'vendor_id': 6900, } balloon = [d for d in devices if d['device_id'] == 4098][0] > assert balloon == { 'device_id': 4098, 'driver_date': '2019-08-12', 'driver_name': 'VirtIO Balloon Driver', 'driver_version': '100.80.104.17300', 'vendor_id': 6900, } E AssertionError: assert {'device_id':...4.17300', ...} == {'device_id':...4.17300', ...} E Omitting 4 identical items, use -vv to show E Differing items: E {'driver_date': '2019-08-11'} != {'driver_date': '2019-08-12'} Are we using real time in this test? E Use -v to get the full diff

2 5

Proposing Eyal Shenitsky as ovirt-engine backend maintener
by Benny Zlotnik 11 Aug '21

11 Aug '21

Hi, Eyal has been working on ovirt-engine for a while now and is currently the most prolific active contributor from the storage team. As such, I would like to propose Eyal as an ovirt-engine maintainer

4 3

ovirt-engine has been tagged (ovirt-engine-4.4.8.4)
by Tal Nisan 11 Aug '21

11 Aug '21

2 1