On 23. 8. 2022, at 10:53, Andrei Verovski <andreil1(a)starlett.lv&gt; wrote: Hi, I had same problem with VM of our accounting / stock control system, which is a top priority which can’t be down no matter what. Solution: Make a new snapshot on problematic VM, then make a new VM from that snapshot. This is only a workaround I found so far.

> On 23 Aug 2022, at 10:50, Jirka Simon <jirka(a)vesim.cz <mailto:jirka@vesim.cz>> wrote: > > Hello there. > > we have troubles with one VM ad it's snapshot > > here is short log from engine.log > > 2022-08-23 09:02:38,735+02 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1265784) [4d79272b-ffac-4b2b-a987-8c19577cd8c4] CommandAsyncTask:: > HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '5842cafc-60d4-4131-8da1-8a909123f08c' > 2022-08-23 09:02:41,435+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-68) [4d79272b > -ffac-4b2b-a987-8c19577cd8c4] Command id: 'ead3fbba-7439-4337-8aaf-1d6cf65bbb15 failed child command status for step 'REDUCE_IMAGE' > 2022-08-23 09:02:41,435+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-68) [ > 4d79272b-ffac-4b2b-a987-8c19577cd8c4] Command 'RemoveSnapshotSingleDiskLive' id: 'ead3fbba-7439-4337-8aaf-1d6cf65bbb15' child commands '[5b33d970-6ea0-4b8b-bc31-607afd8af1ca, 185d3fec-8c76- > 44d3-a0a1-fe8fff331354, 308baf9d-1fc5-43a1-a624-b5f08f7bec1b, f11f3550-4356-417e-af10-a3afb2051dec, 5842cafc-60d4-4131-8da1-8a909123f08c]' executions were completed, status 'FAILED' > 2022-08-23 09:02:42,445+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) [4d79272b > -ffac-4b2b-a987-8c19577cd8c4] Merging of snapshot '6634db4e-23d8-4849-a008-864f15d28c3d' images 'd24a9199-741b-49cb-96d2-0d76fcd21f48'..'48af3301-0cbb-4a6e-97d1-7299e7de883f' failed. Images > have been marked illegal and can no longer be previewed or reverted to. Please retry Live Merge on the snapshot to complete the operation. > 2022-08-23 09:02:42,451+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) [4d79272b > -ffac-4b2b-a987-8c19577cd8c4] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand' with failure. > 2022-08-23 09:02:42,457+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) [4d79272b-ffac > -4b2b-a987-8c19577cd8c4] Command 'RemoveSnapshot' id: '73340ac8-07d7-4de2-bfeb-8062fa1e8cfd' child commands '[ead3fbba-7439-4337-8aaf-1d6cf65bbb15]' executions were completed, status 'FAILE > D' > 2022-08-23 09:02:43,470+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [4d79272b-ffac-4b2b-a98 > 7-8c19577cd8c4] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. > 2022-08-23 09:02:43,490+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [4d79272b-ff > ac-4b2b-a987-8c19577cd8c4] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'vProtect 2022-08-05 22:33:44.088143' for VM 'web1.wiki.prod.hq.sldev.cz <http://wiki.prod.hq.sldev.cz/>-2';. > > > > Snapshot can't be deleted, and VM is not possible to clone. > > > > thank you for any help. > > > > Jirka > > _______________________________________________ > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org <mailto:users-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZOVL22OGZKE... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZOVL22OGZKE... _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AWITGPKJKLT...

> On 23. 8. 2022, at 10:53, Andrei Verovski <andreil1(a)starlett.lv&gt; wrote: > > Hi, > > I had same problem with VM of our accounting / stock control system, > which is a top priority which can’t be down no matter what. > > Solution: Make a new snapshot on problematic VM, then make a new VM > from that snapshot. This is only a workaround I found so far. Make sure you run latest version, there were plenty of live merge bugs solved in the last months. Also if you're using vProtect you can likely use the new incremental backup(CBT) feature, that avoids all the snapshotting in the first place. > > > >> On 23 Aug 2022, at 10:50, Jirka Simon <jirka(a)vesim.cz&gt; wrote: >> >> Hello there. >> >> we have troubles with one VM ad it's snapshot >> >> here is  short log  from engine.log >> >> 2022-08-23 09:02:38,735+02 INFO >>  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] >> (EE-ManagedThreadFactory-engine-Thread-1265784) >> [4d79272b-ffac-4b2b-a987-8c19577cd8c4] CommandAsyncTask:: >> HandleEndActionResult [within thread]: Removing >> CommandMultiAsyncTasks object for entity >> '5842cafc-60d4-4131-8da1-8a909123f08c' >> 2022-08-23 09:02:41,435+02 ERROR >> [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-68) >> [4d79272b >> -ffac-4b2b-a987-8c19577cd8c4] Command id: >> 'ead3fbba-7439-4337-8aaf-1d6cf65bbb15 failed child command status >> for step 'REDUCE_IMAGE' >> 2022-08-23 09:02:41,435+02 INFO >>  [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-68) >> [ >> 4d79272b-ffac-4b2b-a987-8c19577cd8c4] Command >> 'RemoveSnapshotSingleDiskLive' id: >> 'ead3fbba-7439-4337-8aaf-1d6cf65bbb15' child commands >> '[5b33d970-6ea0-4b8b-bc31-607afd8af1ca, 185d3fec-8c76- >> 44d3-a0a1-fe8fff331354, 308baf9d-1fc5-43a1-a624-b5f08f7bec1b, >> f11f3550-4356-417e-af10-a3afb2051dec, >> 5842cafc-60d4-4131-8da1-8a909123f08c]' executions were completed, >> status 'FAILED' >> 2022-08-23 09:02:42,445+02 ERROR >> [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) >> [4d79272b >> -ffac-4b2b-a987-8c19577cd8c4] Merging of snapshot >> '6634db4e-23d8-4849-a008-864f15d28c3d' images >> 'd24a9199-741b-49cb-96d2-0d76fcd21f48'..'48af3301-0cbb-4a6e-97d1-7299e7de883f' >> failed. Images >> have been marked illegal and can no longer be previewed or reverted >> to. Please retry Live Merge on the snapshot to complete the operation. >> 2022-08-23 09:02:42,451+02 ERROR >> [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) >> [4d79272b >> -ffac-4b2b-a987-8c19577cd8c4] Ending command >> 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand' >> with failure. >> 2022-08-23 09:02:42,457+02 INFO >>  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) >> [4d79272b-ffac >> -4b2b-a987-8c19577cd8c4] Command 'RemoveSnapshot' id: >> '73340ac8-07d7-4de2-bfeb-8062fa1e8cfd' child commands >> '[ead3fbba-7439-4337-8aaf-1d6cf65bbb15]' executions were completed, >> status 'FAILE >> D' >> 2022-08-23 09:02:43,470+02 ERROR >> [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) >> [4d79272b-ffac-4b2b-a98 >> 7-8c19577cd8c4] Ending command >> 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with >> failure. >> 2022-08-23 09:02:43,490+02 ERROR >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) >> [4d79272b-ff >> ac-4b2b-a987-8c19577cd8c4] EVENT_ID: >> USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete >> snapshot 'vProtect 2022-08-05 22:33:44.088143' for VM >> 'web1.wiki.prod.hq.sldev.cz <http://wiki.prod.hq.sldev.cz/>-2';. >> >> >> Snapshot can't be deleted, and VM is not possible to clone. >> >> >> thank you for any help. >> >> >> Jirka >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZOVL22OGZKE... > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/AWITGPKJKLT... _______________________________________________ Users mailing list --users(a)ovirt.org To unsubscribe send an email tousers-leave(a)ovirt.org Privacy Statement:https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct:https://www.ovirt.org/community/about/community-guidelines/ List Archives:https://lists.ovirt.org/archives/list/users@ovirt.org/message/GL...

We are currently sitting on a not really working backup solution. oVirt 4.5.x has constant issues removing snapshots, same like OP. vProtect with Snapshots has issues with oVirt deleting snapshots. vProtect 5.1 has broken CBT with oVirt 4.5.x. vProtect support tells us that the snapshot issues is in oVirt (true) and CBT is a preview feature not really supported. So currently vProtect 5.1 and oVirt 4.5 is a no go solution. We are thinking going to xcp-ng due to constant issues with oVirt and or vProtect. _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/V7VSQLOGJDT...

Hello Sven, Please do you have any details about it? Like bug report od knowledge base or article ?   We have ovit and vprotect as well and our troubles started after last update of ovirt 4.4 ( frozen vm with snapshot crating job, then after unlock some snapshots is not possible to delete.)  And now I'm preparing upgrade to 4.5.2 Yesterday we upgraded vProtect to the latest version. Thank you for any detail. Jirka On 8/24/22 15:20, Sven Jansen via Users wrote: > We are currently sitting on a not really working backup solution. > oVirt 4.5.x has constant issues removing snapshots, same like OP. > vProtect with Snapshots has issues with oVirt deleting snapshots. > vProtect 5.1 has broken CBT with oVirt 4.5.x. > > vProtect support tells us that the snapshot issues is in oVirt (true) > and CBT is a preview feature not really supported. So currently > vProtect 5.1 and oVirt 4.5 is a no go solution. We are thinking going > to xcp-ng due to constant issues with oVirt and or vProtect. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/V7VSQLOGJDT... > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FMK2Z5B5K7Y...

The bug you mentioned was fixed in 4.5.2.2. But to clarify the issue that's fixed: The fix is for a situation where a snapshot removal failed for whatever reason, but the volume was removed in vdsm and remains present in the engine's database. Without the fix, attempting to remove the snapshot again would fail. Is this the scenario you are facing now? There is another question of why the snapshot removal failed initially, and I did not get logs with this information (although it could be due to network issues) On Fri, Aug 26, 2022 at 7:19 PM Giorgio Biacchi <giorgio(a)di.unimi.it&gt; wrote: > > Hi, > I filed a bug about this last year... > > https://bugzilla.redhat.com/show_bug.cgi?id=1948599 > > We've dome some tests with vProtect after the leatest Ovirt release > 4.5.2.4-1.el8 but we still have problems with snapshots removal and > disks left in locked or illegal state. > > To bring back a disk from illegal state we live migrate the disk(s) to > another data domain. After this the snapshots are back to legal and can > be removed from the UI. This is pratical only for VMs with small disks. > Sadly one of the VMs we have problems with has a 2.5Tb disk attached :( > > Hope this get solved soon. > > Regards > > Il 25/08/2022 10:26, Jirka Simon ha scritto: >> Hello Sven, >> >> Please do you have any details about it? Like bug report od knowledge >> base or article ? We have ovit and vprotect as well and our troubles >> started after last update of ovirt 4.4 ( frozen vm with snapshot crating >> job, then after unlock some snapshots is not possible to delete.) And >> now I'm preparing upgrade to 4.5.2 Yesterday we upgraded vProtect to the >> latest version. >> >> Thank you for any detail. >> >> >> Jirka >> >> On 8/24/22 15:20, Sven Jansen via Users wrote: >>> We are currently sitting on a not really working backup solution. >>> oVirt 4.5.x has constant issues removing snapshots, same like OP. >>> vProtect with Snapshots has issues with oVirt deleting snapshots. >>> vProtect 5.1 has broken CBT with oVirt 4.5.x. >>> >>> vProtect support tells us that the snapshot issues is in oVirt (true) >>> and CBT is a preview feature not really supported. So currently >>> vProtect 5.1 and oVirt 4.5 is a no go solution. We are thinking going >>> to xcp-ng due to constant issues with oVirt and or vProtect. >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/V7VSQLOGJDT... >>> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/FMK2Z5B5K7Y... >> > > -- > gb > > PGP Key: http://pgp.mit.edu/ > Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34 > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/K5P7G25WSOU...

Hi Benny, we filed a new bug: https://bugzilla.redhat.com/show_bug.cgi?id=2122525 On 30/08/2022 16:00, Giorgio Biacchi wrote: > So it seems there's another bug. > > Since the bug was opened for an issue caused by the removal of a > snapshot by vprotect (backup application) I thought that the fix > should also fix the issue we have with vprotect, but this didn't > happened. > > I think we'll file a new bug as soon as possible. > > Regards > > Il 26/08/2022 20:25, Benny Zlotnik ha scritto: >> The bug you mentioned was fixed in 4.5.2.2. But to clarify the issue >> that's fixed: The fix is for a situation where a snapshot removal >> failed for whatever reason, but the volume was removed in vdsm and >> remains present in the engine's database. Without the fix, attempting >> to remove the snapshot again would fail. >> >> Is this the scenario you are facing now? >> >> There is another question of why the snapshot removal failed >> initially, and I did not get logs with this information (although it >> could be due to network issues) >> >> On Fri, Aug 26, 2022 at 7:19 PM Giorgio Biacchi <giorgio(a)di.unimi.it&gt; >> wrote: >>> >>> Hi, >>> I filed a bug about this last year... >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1948599 >>> >>> We've dome some tests with vProtect after the leatest Ovirt release >>> 4.5.2.4-1.el8  but we still have problems with snapshots removal and >>> disks left in locked or illegal state. >>> >>> To bring back a disk from illegal state we live migrate the disk(s) to >>> another data domain. After this the snapshots are back to legal and can >>> be removed from the UI. This is pratical only for VMs with small disks. >>> Sadly one of the VMs we have problems with has a 2.5Tb disk attached :( >>> >>> Hope this get solved soon. >>> >>> Regards >>> >>> Il 25/08/2022 10:26, Jirka Simon ha scritto: >>>> Hello Sven, >>>> >>>> Please do you have any details about it? Like bug report od knowledge >>>> base or article ?   We have ovit and vprotect as well and our troubles >>>> started after last update of ovirt 4.4 ( frozen vm with snapshot >>>> crating >>>> job, then after unlock some snapshots is not possible to delete.)  And >>>> now I'm preparing upgrade to 4.5.2 Yesterday we upgraded vProtect >>>> to the >>>> latest version. >>>> >>>> Thank you for any detail. >>>> >>>> >>>> Jirka >>>> >>>> On 8/24/22 15:20, Sven Jansen via Users wrote: >>>>> We are currently sitting on a not really working backup solution. >>>>> oVirt 4.5.x has constant issues removing snapshots, same like OP. >>>>> vProtect with Snapshots has issues with oVirt deleting snapshots. >>>>> vProtect 5.1 has broken CBT with oVirt 4.5.x. >>>>> >>>>> vProtect support tells us that the snapshot issues is in oVirt (true) >>>>> and CBT is a preview feature not really supported. So currently >>>>> vProtect 5.1 and oVirt 4.5 is a no go solution. We are thinking going >>>>> to xcp-ng due to constant issues with oVirt and or vProtect. >>>>> _______________________________________________ >>>>> Users mailing list -- users(a)ovirt.org >>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/V7VSQLOGJDT... >>>>> >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/FMK2Z5B5K7Y... >>>> >>> >>> -- >>> gb >>> >>> PGP Key: http://pgp.mit.edu/ >>> Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 >>> B9CB 0F34 >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/K5P7G25WSOU... >> > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/E5CWFX34ZVF...