4:47 a.m.
after some investigation we have foud that:
via virsh our VM have 71 checkpoint
on engine's db, in the table vm_checkpoints there are ZERO checkpoint.
Is ther a way to sync the checkpoints?!?
Regards,
Tommaso
Il 26/05/2021 10:27, Tommaso - Shellrent via Users ha scritto:
Hi to all. We have almost the same problem.
After a snapshot restore in a checkpoint taken previous last
incremental backup, we have alwes the error "Parent checkpoint ID does
not match the actual leaf checkpoint".
we have also the " parent_checkpoint_id: None"
there is a wey to fix it on a production env?
our engine version is 4.4.5.11 and vdsm 4.40.50
Il 19/07/2020 23:27, Nir Soffer ha scritto:
> On Sun, Jul 19, 2020 at 5:38 PM Łukasz Kołaciński
> <l.kolacinski(a)storware.eu <mailto:l.kolacinski@storware.eu>> wrote:
>
> Hello,
> Thanks to previous answers, I was able to make backups.
> Unfortunately, we had some infrastructure issues and after the
> host reboots new problems appeared. I am not able to do any
> backup using the commands that worked yesterday. I looked through
> the logs and there is something like this:
>
> 2020-07-17 15:06:30,644+02 ERROR
> [org.ovirt.engine.core.bll.StartVmBackupCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-54)
> [944a1447-4ea5-4a1c-b971-0bc612b6e45e] Failed to execute VM
> backup operation 'StartVmBackup': {}:
> org.ovirt.engine.core.common.errors.EngineException:
> EngineException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to
> StartVmBackupVDS, error = Checkpoint Error:
> {'parent_checkpoint_id': None, 'leaf_checkpoint_id':
> 'cd078706-84c0-4370-a6ec-654ccd6a21aa', 'vm_id':
> '116aa6eb-31a1-43db-9b1e-ad6e32fb9260', 'reason':
'*Parent
> checkpoint ID does not match the actual leaf checkpoint*'},
> code = 1610 (Failed with error unexpected and code 16)
>
>
> It looks like engine sent:
>
> parent_checkpoint_id: None
>
> This issue was fix in engine few weeks ago.
>
> Which engine and vdsm versions are you testing?
>
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2114)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.performVmBackupOperation(StartVmBackupCommand.java:368)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.runVmBackup(StartVmBackupCommand.java:225)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.performNextOperation(StartVmBackupCommand.java:199)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
> at
>
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
> at
>
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
>
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
>
>
> And the last error is:
>
> 2020-07-17 15:13:45,835+02 ERROR
> [org.ovirt.engine.core.bll.StartVmBackupCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14)
> [f553c1f2-1c99-4118-9365-ba6b862da936] Failed to execute VM
> backup operation 'GetVmBackupInfo': {}:
> org.ovirt.engine.core.common.errors.EngineException:
> EngineException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to
> GetVmBackupInfoVDS, error = No such backup Error: {'vm_id':
> '116aa6eb-31a1-43db-9b1e-ad6e32fb9260', 'backup_id':
> 'bf1c26f7-c3e5-437c-bb5a-255b8c1b3b73', 'reason': '*VM
backup
> not exists: Domain backup job id not found: no domain backup
> job present'*}, code = 1601 (Failed with error unexpected and
> code 16)
>
>
> This is likely a result of the first error. If starting backup failed
> the backup entity
> is deleted.
>
> (these errors are from full backup)
>
> Like I said this is very strange because everything was working
> correctly.
>
>
> Regards
>
> Łukasz Kołaciński
>
> Junior Java Developer
>
> e-mail: l.kolacinski(a)storware.eu <mailto:l.kolacinski@storware.eu>
> <mailto:m.helbert@storware.eu>
>
>
> *STORWARE* <http://www.storware.eu/>**
>
>
>
> *ul. Leszno 8/44
> 01-192 Warszawa
> www.storware.eu <https://www.storware.eu/>*
>
>
>
> *facebook* <https://www.facebook.com/storware>**
>
>
>
> *twitter* <https://twitter.com/storware>**
>
>
>
> *linkedin* <https://www.linkedin.com/company/storware>**
>
>
>
> *Storware_Stopka_09*
> <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw>**
>
> *//*
>
> */Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St.
> Warszawa 000510131/*//*/, NIP 5213672602./*/ Wiadomość ta jest
> przeznaczona jedynie dla osoby lub podmiotu, który jest jej
> adresatem i może zawierać poufne i/lub uprzywilejowane
> informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie,
> rozpowszechnianie lub inne wykorzystanie tych informacji lub
> podjęcie jakichkolwiek działań odnośnie tych informacji przez
> osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo
> otrzymali przez pomyłkę tę informację prosimy o poinformowanie o
> tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów.
> //This message is intended only for the person or entity to which
> it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use
> of, or taking of any action in reliance upon, this information by
> persons or entities other than the intended recipient is
> prohibited. If you have received this message in error, please
> contact the sender and remove the material from all of your
> computer systems./
>
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org>
> To unsubscribe send an email to users-leave(a)ovirt.org
> <mailto:users-leave@ovirt.org>
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> <https://www.ovirt.org/privacy-policy.html>
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> <https://www.ovirt.org/community/about/community-guidelines/>
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/S3PLYPOZGT6...
>
<https://lists.ovirt.org/archives/list/users@ovirt.org/message/S3PLYPOZGT6...
>
>
> _______________________________________________
> Users mailing list --users(a)ovirt.org
> To unsubscribe send an email tousers-leave(a)ovirt.org
> Privacy Statement:https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:https://www.ovirt.org/community/about/community-guidelines/
> List
Archives:https://lists.ovirt.org/archives/list/users@ovirt.org/message/7X...
--
--
Shellrent - Il primo hosting italiano Security First
*Tommaso De Marchi*
/COO - Chief Operating Officer/
Shellrent Srl
Via dell'Edilizia, 19 - 36100 Vicenza
Tel. 0444321155 <tel:+390444321155> | Fax 04441492177
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DD2R2PWCIQD...
--
--
Shellrent - Il primo hosting italiano Security First
*Tommaso De Marchi*
/COO - Chief Operating Officer/
Shellrent Srl
Via dell'Edilizia, 19 - 36100 Vicenza
Tel. 0444321155 <tel:+390444321155> | Fax 04441492177
10:25 a.m.
New subject: [ovirt-users] Re: Parent checkpoint ID does not match the actual leaf checkpoint
On Wed, May 26, 2021 at 12:48 PM Tommaso - Shellrent <tommaso(a)shellrent.com wrote:
after some investigation we have foud that:
via virsh our VM have 71 checkpoint
on engine's db, in the table vm_checkpoints there are ZERO checkpoint.
Is ther a way to sync the checkpoints?!?
Engine deletes all checkpoints from engine if it cannot redefine
the checkpoints in libvirt. After this you must start again with full
backup.
The stale checkpoint in libvirt should not happen, please file a bug for
this.
We need to understand why engine deleted the checkpoints without deleting
the checkpoints in libvirt.
To checkpoints in libvirt are deleted when you stop the vm.
You can also delete them safely using virsh.
Nir
12:38 a.m.
New subject: [ovirt-users] Re: Parent checkpoint ID does not match the actual leaf checkpoint
On Wed, 26 May 2021 at 12:51, Tommaso - Shellrent via Users <users(a)ovirt.org>
wrote:
after some investigation we have foud that:
via virsh our VM have 71 checkpoint
on engine's db, in the table vm_checkpoints there are ZERO checkpoint.
Is ther a way to sync the checkpoints?!?
Regards,
Tommaso
Il 26/05/2021 10:27, Tommaso - Shellrent via Users ha scritto:
Hi to all. We have almost the same problem.
After a snapshot restore in a checkpoint taken previous last incremental
backup, we have alwes the error "Parent checkpoint ID does not match the
actual leaf checkpoint".
we have also the " parent_checkpoint_id: None"
If you restored a snapshot then all the checkpoints that were taken after
that snapshot are now gone -
https://bugzilla.redhat.com/show_bug.cgi?id=1952577.
A full backup is needed in this case.
In order to get over "Parent checkpoint ID does not match the actual leaf
checkpoint" you should upgrade to a newer version, fixed in 4.4.6.
there is a wey to fix it on a production env?
our engine version is 4.4.5.11 and vdsm 4.40.50
Il 19/07/2020 23:27, Nir Soffer ha scritto:
On Sun, Jul 19, 2020 at 5:38 PM Łukasz Kołaciński <
l.kolacinski(a)storware.eu> wrote:
> Hello,
> Thanks to previous answers, I was able to make backups. Unfortunately,
> we had some infrastructure issues and after the host reboots new problems
> appeared. I am not able to do any backup using the commands that worked
> yesterday. I looked through the logs and there is something like this:
>
> 2020-07-17 15:06:30,644+02 ERROR
> [org.ovirt.engine.core.bll.StartVmBackupCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-54)
> [944a1447-4ea5-4a1c-b971-0bc612b6e45e] Failed to execute VM backup
> operation 'StartVmBackup': {}:
> org.ovirt.engine.core.common.errors.EngineException: EngineException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to StartVmBackupVDS, error =
> Checkpoint Error: {'parent_checkpoint_id': None,
'leaf_checkpoint_id':
> 'cd078706-84c0-4370-a6ec-654ccd6a21aa', 'vm_id':
> '116aa6eb-31a1-43db-9b1e-ad6e32fb9260', 'reason': '*Parent
checkpoint ID
> does not match the actual leaf checkpoint*'}, code = 1610 (Failed with
> error unexpected and code 16)
>
>
It looks like engine sent:
parent_checkpoint_id: None
This issue was fix in engine few weeks ago.
Which engine and vdsm versions are you testing?
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2114)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.performVmBackupOperation(StartVmBackupCommand.java:368)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.runVmBackup(StartVmBackupCommand.java:225)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.performNextOperation(StartVmBackupCommand.java:199)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
> at
>
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
> at
>
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
>
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> at
>
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
>
>
> And the last error is:
>
> 2020-07-17 15:13:45,835+02 ERROR
> [org.ovirt.engine.core.bll.StartVmBackupCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14)
> [f553c1f2-1c99-4118-9365-ba6b862da936] Failed to execute VM backup
> operation 'GetVmBackupInfo': {}:
> org.ovirt.engine.core.common.errors.EngineException: EngineException:
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to GetVmBackupInfoVDS, error
> = No such backup Error: {'vm_id':
'116aa6eb-31a1-43db-9b1e-ad6e32fb9260',
> 'backup_id': 'bf1c26f7-c3e5-437c-bb5a-255b8c1b3b73',
'reason': '*VM
> backup not exists: Domain backup job id not found: no domain backup job
> present'*}, code = 1601 (Failed with error unexpected and code 16)
>
>
This is likely a result of the first error. If starting backup failed the
backup entity
is deleted.
> (these errors are from full backup)
>
> Like I said this is very strange because everything was working correctly.
>
>
> Regards
>
> Łukasz Kołaciński
>
> Junior Java Developer
>
> e-mail: l.kolacinski(a)storware.eu
> <m.helbert(a)storware.eu>
>
>
>
>
> *[image: STORWARE]* <http://www.storware.eu/>
>
>
>
> *ul. Leszno 8/44 01-192 Warszawa www.storware.eu
> <https://www.storware.eu/>*
>
> *[image: facebook]* <https://www.facebook.com/storware>
>
> *[image: twitter]* <https://twitter.com/storware>
>
> *[image: linkedin]* <https://www.linkedin.com/company/storware>
>
> *[image: Storware_Stopka_09]*
> <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw>
>
>
>
> *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa
> 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie
> dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne
> i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie,
> przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub
> podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub
> podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez
> pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie
> tej wiadomości z wszelkich komputerów. **This message is intended only
> for the person or entity to which it is addressed and may contain
> confidential and/or privileged material. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient
> is prohibited. If you have received this message in error, please contact
> the sender and remove the material from all of your computer systems.*
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/S3PLYPOZGT6...
>
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/7XWQUEIS7X5...
--
--
[image: Shellrent - Il primo hosting italiano Security First]
*Tommaso De Marchi*
*COO - Chief Operating Officer*
Shellrent Srl
Via dell'Edilizia, 19 - 36100 Vicenza
Tel. 0444321155 <+390444321155> | Fax 04441492177
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DD2R2PWCIQD...
--
--
[image: Shellrent - Il primo hosting italiano Security First]
*Tommaso De Marchi*
*COO - Chief Operating Officer*
Shellrent Srl
Via dell'Edilizia, 19 - 36100 Vicenza
Tel. 0444321155 <+390444321155> | Fax 04441492177
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4PRP6322NHK...
--
Regards,
Eyal Shenitzky
1064
days inactive
1065
days old
2 comments
3 participants
participants (3)
-
Eyal Shenitzky -
Nir Soffer -
Tommaso - Shellrent