- Devel - oVirt List Archives

ovirt-provider-ovn git repo got broken
by Sandro Bonazzola 30 Jul '21

30 Jul '21

Hi, not sure what happened but ovirt-provider-ovn repo in gerrit (and its backup on github) now contains a bunch of ovirt-engine tags shipping ovirt-engine content. Can you please have a look? $ git fetch --tags $ git tag --list |grep ovirt-engine ovirt-engine-3.0.0_0001 ovirt-engine-3.1.0 ovirt-engine-3.2.0 ovirt-engine-3.2.0-4 ovirt-engine-3.2.1 ovirt-engine-3.2.1-1 ovirt-engine-3.2.2 ovirt-engine-3.3.0 ovirt-engine-3.3.0.1 ovirt-engine-3.3.1 ovirt-engine-3.3.1_beta ovirt-engine-3.3.1_rc1 ovirt-engine-3.3.2 ovirt-engine-3.3.2_beta1 ovirt-engine-3.3.2_rc1 ovirt-engine-3.3.3_beta1 ovirt-engine-3.3_beta1 ovirt-engine-3.3_rc ovirt-engine-3.3_rc2 ovirt-engine-3.5.2 ovirt-engine-3.5.2.1 ovirt-engine-3.5.4.1 ovirt-engine-3.6.0_alpha1 ovirt-engine-3.6.2.1 ovirt-engine-4.0.2.3 ovirt-engine-4.1.5 ovirt-engine-4.1.7 ovirt-engine-4.1.7.1 ovirt-engine-4.1.7.3 ovirt-engine-4.1.9.2 ovirt-engine-4.2.0 ovirt-engine-4.2.1.5 -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Re: RHV and oVirt CBT issue
by Nir Soffer 30 Jul '21

30 Jul '21

On Thu, Jul 29, 2021 at 10:08 AM luwen.zhang <luwen.zhang(a)vinchin.com> wrote: > The problem occurred yesterday, but we waited for more than 20 hours, still 2 disks and in Finalizing state. If the image transfer is "finalizing" it means the image transfer is trying to finalize, but the operation could not complete. In this phase the disk remains locked, and it should not be possible to start a new image transfer (e.g perform another backup). Engine and vdsm logs should explain why the image transfer is stuck in the finalizing phase. Can you add detailed instructions on how to reproduce this issue?

2 1

after a regular backup, VM got duplicated disks and disks' status always in Finalizing
by luwen.zhang＠vinchin.com 30 Jul '21

30 Jul '21

Version: oVirt engine: 4.4.7.7-1.el8 oVirt node: 4.4.7 Steps of reproducing the problem: After a series of regular/successful backup tests by using CBT+imageio API, at one of the backup session beginning, when trying to obtain the VM config and snapshot list (obtain snapshot list can determine the VM disk is in RAW or QCOW2 format) by using `GET vms/<vm-id>/snapshots`, the below error is received. <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <fault> <detail>duplicate key acf1edaa-e950-4c4f-94df-1bd6b3da49c1 (attempted merging values org.ovirt.engine.core.common.businessentities.storage.diskimage@5103046c and org.ovirt.engine.core.common.businessentities.storage.diskimage@d973046c)</detail> <reason>Operation Failed</reason> </fault> After this on the oVirt engine web console, we saw the VM which should only have 1 disk is now showing 2 disks, the disk status always in Finalizing state. It's been more than 30 hours but the VM still got duplicated disks and still showing Finalizing, there's no way to modify the virtual disk or take snapshots for the VM.

1 0

Re: RHV and oVirt CBT issue
by Eyal Shenitzky 29 Jul '21

29 Jul '21

+Benny Zlotnik <bzlotnik(a)redhat.com> +Pavel Bar <pbar(a)redhat.com> On Thu, 29 Jul 2021 at 07:08, luwen.zhang <luwen.zhang(a)vinchin.com> wrote: > > > Dear all, > > FYI, after upgrading oVirt engine to 4.4.7.7-1.el8 the chance of > reproducing this issue is reduced, but it still persists. > Can you please open a bug for it and mention the steps you did to reproduce the problem? Also, please include all the relevant logs in the bug. Benny/Pavel, is there a WA for getting out of this issue besides removing the entry from the DB? > > The oVirt node version is also 4.4.7, the virtual disks are stored in > iSCSI LUN. > > The problem occurred yesterday, but we waited for more than 20 hours, > still 2 disks and in Finalizing state. > > > At this moment, we are unable to check/delete snapshots, or perform > backups. > > Please let us know if we can do anything to help with the debug. > > Thanks & regards! > > Luwen Zhang > On 07/27/2021 18:30，luwen.zhang<luwen.zhang(a)vinchin.com> > <luwen.zhang(a)vinchin.com> wrote： > > > Thank you all for your help, after upgrading oVirt engine the issue is now > fixed. > > Best regards! > Luwen Zhang > On 07/27/2021 18:03，Lev Veyde<lveyde(a)redhat.com> <lveyde(a)redhat.com> > wrote： > > Hi Sandro, > > Yes, it was released as part of the 0-day fix on July 20, including a new > appliance build. > > Thanks in advance, > > On Tue, Jul 27, 2021 at 12:20 PM Sandro Bonazzola <sbonazzo(a)redhat.com> > wrote: > >> >> >> Il giorno mar 27 lug 2021 alle ore 10:53 Eyal Shenitzky < >> eshenitz(a)redhat.com> ha scritto: >> >>> As +Michal Skrivanek <mskrivan(a)redhat.com> already replied, this issue >>> cause by - https://bugzilla.redhat.com/show_bug.cgi?id=1980428. >>> >>> It is already fixed in RHV 4.4.7 and on the master branch. >>> +Sandro Bonazzola <sbonazzo(a)redhat.com>, did we include it on oVirt >>> 4.4.7 also? >>> >> >> $ git tag --contains 6cd8bc532b0a4a6839ed667c67489256ae9ff5b9 >> ovirt-engine-4.4.7.7 >> >> It was shipped on July 20th but I don't see an announcement for it. +Lev >> Veyde <lveyde(a)redhat.com> ? >> >> >> >> >> >>> >>> On Tue, 27 Jul 2021 at 09:44, luwen.zhang <luwen.zhang(a)vinchin.com> >>> wrote: >>> >>>> >>>> Dear team could you please help on the below issue? >>>> >>>> Thanks Sandro for the advice! >>>> On 07/27/2021 14:22，Sandro Bonazzola<sbonazzo(a)redhat.com> >>>> <sbonazzo(a)redhat.com> wrote： >>>> >>>> I would recommend to ask this on devel(a)ovirt.org. >>>> Looping in +Eyal Shenitzky <eshenitz(a)redhat.com> >>>> >>>> Il giorno mar 27 lug 2021 alle ore 08:14 luwen.zhang < >>>> luwen.zhang(a)vinchin.com> ha scritto: >>>> >>>>> >>>>> Dear team, >>>>> >>>>> We had implemented CBT support for RHV and oVirt in our new version, >>>>> but when testing we encountered problems of obtain VM increments and also >>>>> had abnormal with the VM virtual disks. >>>>> >>>>> Our testing environment as below: >>>>> OS Version: RHEL - 8.4.2105.0 - 3.el8 >>>>> OS Description: oVirt Node 4.4.7 >>>>> Kernel Version: 4.18.0 - 315.el8.x86_64 >>>>> KVM Version: 6.0.0 - 19.el8s >>>>> LIBVIRT Version: libvirt-7.4.0-1.el8s >>>>> VDSM Version: vdsm-4.40.70.6-1.el8 >>>>> >>>>> One of the problem is after a successful backup, the VM which gets >>>>> only one virtual disk will get 2 identical virtual disks marked as Complete >>>>> on oVirt engine web console, like shown below. >>>>> >>>>> The additional disk might disappear after a while, but before it’s >>>>> gone there could be a lot of problems with the VM. >>>>> >>>>> 1. We cannot perform a new backup, the backup will fail directly. >>>>> >>>>> 2. The VM itself will run into exception, we cannot even power it off, >>>>> and it will also probably get crashed. >>>>> >>>>> 3. If the VM was initially powered off, after backing up, we cannot >>>>> power on the VM. >>>>> >>>>> And there could be some other problems related with this, is this >>>>> normal or there’s something we did wrong? >>>>> >>>>> Thank you in advance for your help! >>>>> >>>>> >>>>> *Luwen Zhang* | Product Manager >>>>> >>>>> *Mobile*: +8613880424687 | *Skype*: luwen.zhang_cn >>>>> >>>>> *WhatsApp*: +8613880424687 | *WeChat*: +8613880424687 >>>>> >>>>> *Website*: www.vinchin.com >>>>> >>>>> *Address*: F5, Block 8, National Information Security Industry Park, >>>>> >>>>> No.333 Yunhua Road, Hi-tech Zone, Chengdu, China | *Zip*: 610015 >>>>> >>>>> *INNOVATIVE VM BACKUP EXPERT* >>>>> >>>>> VMware Backup >>>>> <https://www.vinchin.com/en/hypervisor/vmware-backup.html> | XenServer/Citrix >>>>> Hypervisor/XCP-ng Backup >>>>> <https://www.vinchin.com/en/hypervisor/xcp-ng-xenserver-backup.html> >>>>> | Hyper-V Backup >>>>> <https://www.vinchin.com/en/hypervisor/hyper-v-backup.html> >>>>> >>>>> RHEV/oVirt Backup >>>>> <https://www.vinchin.com/en/hypervisor/red-hat-virtualization-ovirt-backup.h…> >>>>> | OLVM Backup >>>>> <https://www.vinchin.com/en/hypervisor/oracle-linux-virtualization-manager-b…> >>>>> | Sangfor HCI Backup >>>>> <https://www.vinchin.com/en/hypervisor/sangfor-hci-backup.html> | OpenStack >>>>> Backup <https://www.vinchin.com/en/hypervisor/openstack-backup.html> >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> Sandro Bonazzola >>>> >>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>> >>>> Red Hat EMEA <https://www.redhat.com/> >>>> >>>> sbonazzo(a)redhat.com >>>> <https://www.redhat.com/> >>>> >>>> *Red Hat respects your work life balance. Therefore there is no need to >>>> answer this email out of your office hours.* >>>> >>>> >>>> >>> >>> -- >>> Regards, >>> Eyal Shenitzky >>> >> >> >> -- >> >> Sandro Bonazzola >> >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> >> Red Hat EMEA <https://www.redhat.com/> >> >> sbonazzo(a)redhat.com >> <https://www.redhat.com/> >> >> *Red Hat respects your work life balance. Therefore there is no need to >> answer this email out of your office hours. >> <https://mojo.redhat.com/docs/DOC-1199578>* >> >> >> > > -- > > Lev Veyde > > Senior Software Engineer, RHCE | RHCVA | MCITP > > Red Hat Israel > > <https://www.redhat.com> > > lev(a)redhat.com | lveyde(a)redhat.com > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > > -- Regards, Eyal Shenitzky

3 3

ovirt-engine has been tagged (ovirt-engine-4.4.8.2)
by Tal Nisan 29 Jul '21

29 Jul '21

1 0

Re: RHV and oVirt CBT issue
by luwen.zhang 29 Jul '21

29 Jul '21

1 0

Re: RHV and oVirt CBT issue
by Eyal Shenitzky 27 Jul '21

27 Jul '21

As +Michal Skrivanek <mskrivan(a)redhat.com> already replied, this issue cause by - https://bugzilla.redhat.com/show_bug.cgi?id=1980428. It is already fixed in RHV 4.4.7 and on the master branch. +Sandro Bonazzola <sbonazzo(a)redhat.com>, did we include it on oVirt 4.4.7 also? On Tue, 27 Jul 2021 at 09:44, luwen.zhang <luwen.zhang(a)vinchin.com> wrote: > > Dear team could you please help on the below issue? > > Thanks Sandro for the advice! > On 07/27/2021 14:22，Sandro Bonazzola<sbonazzo(a)redhat.com> > <sbonazzo(a)redhat.com> wrote： > > I would recommend to ask this on devel(a)ovirt.org. > Looping in +Eyal Shenitzky <eshenitz(a)redhat.com> > > Il giorno mar 27 lug 2021 alle ore 08:14 luwen.zhang < > luwen.zhang(a)vinchin.com> ha scritto: > >> >> Dear team, >> >> We had implemented CBT support for RHV and oVirt in our new version, but >> when testing we encountered problems of obtain VM increments and also had >> abnormal with the VM virtual disks. >> >> Our testing environment as below: >> OS Version: RHEL - 8.4.2105.0 - 3.el8 >> OS Description: oVirt Node 4.4.7 >> Kernel Version: 4.18.0 - 315.el8.x86_64 >> KVM Version: 6.0.0 - 19.el8s >> LIBVIRT Version: libvirt-7.4.0-1.el8s >> VDSM Version: vdsm-4.40.70.6-1.el8 >> >> One of the problem is after a successful backup, the VM which gets only >> one virtual disk will get 2 identical virtual disks marked as Complete on >> oVirt engine web console, like shown below. >> >> The additional disk might disappear after a while, but before it’s gone >> there could be a lot of problems with the VM. >> >> 1. We cannot perform a new backup, the backup will fail directly. >> >> 2. The VM itself will run into exception, we cannot even power it off, >> and it will also probably get crashed. >> >> 3. If the VM was initially powered off, after backing up, we cannot power >> on the VM. >> >> And there could be some other problems related with this, is this normal >> or there’s something we did wrong? >> >> Thank you in advance for your help! >> >> >> *Luwen Zhang* | Product Manager >> >> *Mobile*: +8613880424687 | *Skype*: luwen.zhang_cn >> >> *WhatsApp*: +8613880424687 | *WeChat*: +8613880424687 >> >> *Website*: www.vinchin.com >> >> *Address*: F5, Block 8, National Information Security Industry Park, >> >> No.333 Yunhua Road, Hi-tech Zone, Chengdu, China | *Zip*: 610015 >> >> *INNOVATIVE VM BACKUP EXPERT* >> >> VMware Backup <https://www.vinchin.com/en/hypervisor/vmware-backup.html> >> | XenServer/Citrix Hypervisor/XCP-ng Backup >> <https://www.vinchin.com/en/hypervisor/xcp-ng-xenserver-backup.html> | Hyper-V >> Backup <https://www.vinchin.com/en/hypervisor/hyper-v-backup.html> >> >> RHEV/oVirt Backup >> <https://www.vinchin.com/en/hypervisor/red-hat-virtualization-ovirt-backup.h…> >> | OLVM Backup >> <https://www.vinchin.com/en/hypervisor/oracle-linux-virtualization-manager-b…> >> | Sangfor HCI Backup >> <https://www.vinchin.com/en/hypervisor/sangfor-hci-backup.html> | OpenStack >> Backup <https://www.vinchin.com/en/hypervisor/openstack-backup.html> >> >> >> > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA <https://www.redhat.com/> > > sbonazzo(a)redhat.com > <https://www.redhat.com/> > > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours.* > > > -- Regards, Eyal Shenitzky

4 4

Re: RHV and oVirt CBT issue
by Nir Soffer 27 Jul '21

27 Jul '21

On Tue, Jul 27, 2021 at 9:47 AM luwen.zhang <luwen.zhang(a)vinchin.com> wrote: > > Dear team could you please help on the below issue? > > Thanks Sandro for the advice! > On 07/27/2021 14:22，Sandro Bonazzola<sbonazzo(a)redhat.com> > <sbonazzo(a)redhat.com> wrote： > > I would recommend to ask this on devel(a)ovirt.org. > Looping in +Eyal Shenitzky <eshenitz(a)redhat.com> > > Il giorno mar 27 lug 2021 alle ore 08:14 luwen.zhang < > luwen.zhang(a)vinchin.com> ha scritto: > >> >> Dear team, >> >> We had implemented CBT support for RHV and oVirt in our new version, but >> when testing we encountered problems of obtain VM increments and also had >> abnormal with the VM virtual disks. >> >> Our testing environment as below: >> OS Version: RHEL - 8.4.2105.0 - 3.el8 >> OS Description: oVirt Node 4.4.7 >> Kernel Version: 4.18.0 - 315.el8.x86_64 >> KVM Version: 6.0.0 - 19.el8s >> LIBVIRT Version: libvirt-7.4.0-1.el8s >> VDSM Version: vdsm-4.40.70.6-1.el8 >> >> One of the problem is after a successful backup, the VM which gets only >> one virtual disk will get 2 identical virtual disks marked as Complete on >> oVirt engine web console, like shown below. >> >> The additional disk might disappear after a while, but before it’s gone >> there could be a lot of problems with the VM. >> >> 1. We cannot perform a new backup, the backup will fail directly. >> >> 2. The VM itself will run into exception, we cannot even power it off, >> and it will also probably get crashed. >> >> 3. If the VM was initially powered off, after backing up, we cannot power >> on the VM. >> >> And there could be some other problems related with this, is this normal >> or there’s something we did wrong? >> >> Thank you in advance for your help! >> > Unfortunately you are hitting this bug: https://bugzilla.redhat.com/1980428 The bug is fixed in 4.4.7 async release, which should be available now according to the bug. Please upgrade to the latest 4.4.7 version. Nir

2 1

Re: RHV and oVirt CBT issue
by luwen.zhang 27 Jul '21

27 Jul '21

1 0

ovirt-engine has been tagged (ovirt-engine-4.4.8.1)
by Tal Nisan 21 Jul '21

21 Jul '21

2 1