Hi Lukasz,

Please see my comments below

On Fri, 4 Dec 2020 at 17:55, Łukasz Kołaciński <l.kolacinski@storware.eu> wrote:
Hello,
Could you tell me more about this Error: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',).  It happened on creating incremental backup using ovirt sdk.

This error means that you have a checkpoint that was created during one of your previous backups that missing the checkpoint XML in the DB, this XML is needed in order to redefine the checkpoint on the host after the VM powered-off.

This situation shouldn't happen if the the backup that created checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e' was successful.

Did you enable the incremental backup feature before you took the first backup for that VM?

# engine-config -s "IsIncrementalBackupSupported=true"
# systemctl restart ovirt-engine 

Do you have the logs for the backup that created checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e'?

Can you please provide the content of the checkpoint from the DB?

select * from vm_checkpoints where checkpoint_id='6a569f20-f1e4-469a-8982-7f8cbfbbfc9e'



How can I fix it? I know that is not because of wrong checkpointId because I tried to use wrong Id on other environment and logs are different.

In case of a problem when trying to redefine one of the checkpoints, a full backup should be taken for the VM.
 

Logs from this event:

2020-12-03 08:07:27,020Z INFO  [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: StartVmBackupCommand internal: false. Entities affected :  ID: fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with role type ADMIN,  ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction group BACKUP_DISK with role type ADMIN,  ID: 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK with role type ADMIN
2020-12-03 08:07:27,020Z INFO  [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Creating VmBackup entity for VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc'
2020-12-03 08:07:27,034Z INFO  [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Created VmBackup entity 'abb3161b-13c5-444c-b656-11e79a474342'
2020-12-03 08:07:27,034Z INFO  [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine previous VM checkpoints for VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc'
2020-12-03 08:07:27,045Z INFO  [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: RedefineVmCheckpointCommand internal: true. Entities affected :  ID: fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with role type ADMIN,  ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction group BACKUP_DISK with role type ADMIN,  ID: 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK with role type ADMIN
2020-12-03 08:07:27,048Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, ListVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VdsAndVmIDVDSParametersBase:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 55174e1c
2020-12-03 08:07:27,052Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] FINISH, ListVmCheckpointsVDSCommand, return: [870b213b-4190-48fe-9306-5f6cb759b05a], log id: 55174e1c
2020-12-03 08:07:27,052Z INFO  [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e'
2020-12-03 08:07:27,060Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, RedefineVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 76a75d56
2020-12-03 08:07:27,069Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Failed in 'RedefineVmCheckpointsVDS' method
2020-12-03 08:07:27,077Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM prd-ovirt-host-07.live.rot01.kwebbl.cloud command RedefineVmCheckpointsVDS failed: General Exception: ('Property xml is required',)
2020-12-03 08:07:27,077Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command 'RedefineVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'})' execution failed: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100

[...]

2020-12-03 08:07:27,080Z ERROR [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100 (Failed with error GeneralException and code 100)
2020-12-03 08:07:27,089Z ERROR [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Transaction rolled-back for command 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand'.
2020-12-03 08:07:27,090Z INFO  [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Lock freed to object 'EngineLock:{exclusiveLocks='[64fd8dd9-d16f-4359-a4b9-8435a5abd8ca=DISK, 794ffad6-1d1a-406a-a65e-c0600abb44f1=DISK]', sharedLocks='[fafb9627-a20b-47a0-aece-3b4a09ce70dc=VM]'}'
2020-12-03 08:07:27,101Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VM_INCREMENTAL_BACKUP_FAILED_FULL_VM_BACKUP_NEEDED(10,791), Incremental backup abb3161b-13c5-444c-b656-11e79a474342 for VM prd-event-syslog-test2 failed. Full backup is now needed (User: admin@internal-authz).

Here you can see that the engine reports the checkpoint redefinition failed and a full backup should be taken.

 
2020-12-03 08:07:27,120Z INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VM_BACKUP_STARTED(10,790), Backup abb3161b-13c5-444c-b656-11e79a474342 for VM prd-event-syslog-test2 started (User: admin@internal-authz).


Pozdrawiam,

Łukasz Kołaciński

Junior Java Developer

e-mail: l.kolacinski@storware.eu


 

STORWARE

ul. Leszno 8/44
01-192 Warszawa
www.storware.eu

facebook

twitter

linkedin

Storware_Stopka_09

 

Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa 000510131 , NIP 5213672602. Wiadomość ta jest przeznaczona jedynie dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów. This message is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you have received this message in error, please contact the sender and remove the material from all of your computer systems.




--
Regards,
Eyal Shenitzky