
Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599) we discovered that my issue is not depending on that bug :-( So they pointed me here again. Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks. In my logs I can find: 2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b] My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8). Thank you in advance. Regards, gc

Hi Giulio, To get to the root cause of this problem, I think it is best to look at the engine and vdsm logs from the initial failures with vprotect. You can open a new bug in BZ and place the relevant logs there. Thanks, Mark On Tue, Aug 30, 2022 at 10:25 AM Giulio Casella <giulio@di.unimi.it> wrote:
Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599) we discovered that my issue is not depending on that bug :-(
So they pointed me here again.
Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks.
In my logs I can find:
2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b]
My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8).
Thank you in advance.
Regards, gc
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUP...

Hi Mark, bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=2122525 Regards, gc On 30/08/2022 10:22, Mark Kemel wrote:
Hi Giulio,
To get to the root cause of this problem, I think it is best to look at the engine and vdsm logs from the initial failures with vprotect. You can open a new bug in BZ and place the relevant logs there.
Thanks, Mark
On Tue, Aug 30, 2022 at 10:25 AM Giulio Casella <giulio@di.unimi.it <mailto:giulio@di.unimi.it>> wrote:
Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599 <https://bugzilla.redhat.com/show_bug.cgi?id=1948599>) we discovered that my issue is not depending on that bug :-(
So they pointed me here again.
Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks.
In my logs I can find:
2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b]
My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8).
Thank you in advance.
Regards, gc
_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUP... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUPSQSOBIWAZMSIG57TGP/>

Seems like this is a libvirt/qemu issue: 2022-08-30 08:18:31,121+0000 ERROR (libvirt/events) [virt.vm] (vmId='53adff44-8506-41e7-86d1-5a6ca760721e') Block job 89eab626-9c32-48fb-b006-dbc09cb0026a type COMMIT for drive sdb has failed (vm:5972) Can you also share the qemu/libvirt logs? On 30/08/2022 10:42, Giulio Casella wrote:
Hi Mark, bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=2122525
Regards, gc
On 30/08/2022 10:22, Mark Kemel wrote:
Hi Giulio,
To get to the root cause of this problem, I think it is best to look at the engine and vdsm logs from the initial failures with vprotect. You can open a new bug in BZ and place the relevant logs there.
Thanks, Mark
On Tue, Aug 30, 2022 at 10:25 AM Giulio Casella <giulio@di.unimi.it <mailto:giulio@di.unimi.it>> wrote:
Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599 <https://bugzilla.redhat.com/show_bug.cgi?id=1948599>) we discovered that my issue is not depending on that bug :-(
So they pointed me here again.
Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks.
In my logs I can find:
2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b]
My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8).
Thank you in advance.
Regards, gc
_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUP... <https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUPSQSOBIWAZMSIG57TGP/>
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/R2DC5YRVNTVKIS...

Log file regarding that vm (/var/log/libvirt/qemu/<vm-name>.log on hypervisor) has been modified on august, 19. Nothing about today's failure. On 30/08/2022 11:03, Jean-Louis Dupond wrote:
Seems like this is a libvirt/qemu issue: 2022-08-30 08:18:31,121+0000 ERROR (libvirt/events) [virt.vm] (vmId='53adff44-8506-41e7-86d1-5a6ca760721e') Block job 89eab626-9c32-48fb-b006-dbc09cb0026a type COMMIT for drive sdb has failed (vm:5972)
Can you also share the qemu/libvirt logs?
On 30/08/2022 10:42, Giulio Casella wrote:
Hi Mark, bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=2122525
Regards, gc
On 30/08/2022 10:22, Mark Kemel wrote:
Hi Giulio,
To get to the root cause of this problem, I think it is best to look at the engine and vdsm logs from the initial failures with vprotect. You can open a new bug in BZ and place the relevant logs there.
Thanks, Mark
On Tue, Aug 30, 2022 at 10:25 AM Giulio Casella <giulio@di.unimi.it <mailto:giulio@di.unimi.it>> wrote:
Hi folks, since some month I have an issue with snapshot removals (I have storware vprotect backup system, heavily using snapshots). After some time spent on a bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1948599 <https://bugzilla.redhat.com/show_bug.cgi?id=1948599>) we discovered that my issue is not depending on that bug :-(
So they pointed me here again.
Briefly: sometime snapshot removal fails, leaving snapshot in illegal state. Trying to remove again (via ovirt UI) keeps failing and doesn't solve. The only way to rebuild a consistent situation is live migrating affected disk to another storage domain; after moving the disk, snapshot is no more marked illegal and then I can remove it. You can imagine this is a bit tricky, specially for large disks.
In my logs I can find:
2022-08-29 09:17:11,890+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-1) [0eced56f-689d-422b-b15c-20b824377b08] Failed to live merge. Top volume f8f84b1c-53ab-4c99-a01d-743ed3d7859b is still in qemu chain [0ea89fbc-d39a-48ff-aa2b-0381d79d7714, 55bb387f-01a6-41b6-b585-4bcaf2ea5e32, f8f84b1c-53ab-4c99-a01d-743ed3d7859b]
My setup is ovirt-engine-4.5.2.4-1.el8.noarch, with hypervisors based on oVirt Node 4.5.2 (vdsm-4.50.2.2-1.el8).
Thank you in advance.
Regards, gc
_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://www.ovirt.org/privacy-policy.html> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUP...
<https://lists.ovirt.org/archives/list/users@ovirt.org/message/45YQ3UFQSUCDUPSQSOBIWAZMSIG57TGP/>
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/R2DC5YRVNTVKIS...
participants (3)
-
Giulio Casella
-
Jean-Louis Dupond
-
Mark Kemel