May snapshot status be `OK' before snapshot creation finishes?
Hi, a failure of previewing a snapshot with memory has been experienced in OST master suite and I'm not sure whether REST API responses about snapshot status are correct or not. When I'm creating a snapshot with memory, <snapshot_status> reported from Engine REST API in /api/vms/…/snapshots is initially `locked' and later switches to `ok'. The problem is that `ok' starts being reported before snapshot creation completes, resulting in errors if I try to stop the VM or to preview the snapshot at the moment. Does <snapshot_status>ok</snapshot_status> guarantee that the snapshot is completed or not? I can see the following example in ovirt-engine-sdk (for snapshot without memory): # 'Waiting for Snapshot creation to finish' snapshot_service = snapshots_service.snapshot_service(snapshot.id) while True: time.sleep(5) snapshot = snapshot_service.get() if snapshot.snapshot_status == types.SnapshotStatus.OK: break So I suppose snapshot status shouldn't switch to OK before snapshot creation finishes. But it's not true in Engine master. Is it a bug or a feature? Thanks, Milan
Yes, we had this race in another test, that's because of the engine memory lock which is released after the database one I can't provide links since I'm on my phone and away from the laptop but you can look at the cold merge test, we fixed it there by polling the status of the job On Fri, 20 Jul 2018, 14:11 Milan Zamazal, <mzamazal@redhat.com> wrote:
Hi, a failure of previewing a snapshot with memory has been experienced in OST master suite and I'm not sure whether REST API responses about snapshot status are correct or not.
When I'm creating a snapshot with memory, <snapshot_status> reported from Engine REST API in /api/vms/…/snapshots is initially `locked' and later switches to `ok'. The problem is that `ok' starts being reported before snapshot creation completes, resulting in errors if I try to stop the VM or to preview the snapshot at the moment.
Does <snapshot_status>ok</snapshot_status> guarantee that the snapshot is completed or not? I can see the following example in ovirt-engine-sdk (for snapshot without memory):
# 'Waiting for Snapshot creation to finish' snapshot_service = snapshots_service.snapshot_service(snapshot.id) while True: time.sleep(5) snapshot = snapshot_service.get() if snapshot.snapshot_status == types.SnapshotStatus.OK: break
So I suppose snapshot status shouldn't switch to OK before snapshot creation finishes. But it's not true in Engine master. Is it a bug or a feature?
Thanks, Milan _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/JLHBHQGNVWPDNA...
Benny Zlotnik <bzlotnik@redhat.com> writes:
Yes, we had this race in another test, that's because of the engine memory lock which is released after the database one
I believe that in this case it's not just a tight race since from what I've seen in Vdsm logs it seems the snapshot is really not completed when its status is switched to OK. Anyway, thank you for confirming that snapshot status check is unreliable, I'll replace it with Engine event check.
I can't provide links since I'm on my phone and away from the laptop but you can look at the cold merge test, we fixed it there by polling the status of the job
On Fri, 20 Jul 2018, 14:11 Milan Zamazal, <mzamazal@redhat.com> wrote:
Hi, a failure of previewing a snapshot with memory has been experienced in OST master suite and I'm not sure whether REST API responses about snapshot status are correct or not.
When I'm creating a snapshot with memory, <snapshot_status> reported from Engine REST API in /api/vms/…/snapshots is initially `locked' and later switches to `ok'. The problem is that `ok' starts being reported before snapshot creation completes, resulting in errors if I try to stop the VM or to preview the snapshot at the moment.
Does <snapshot_status>ok</snapshot_status> guarantee that the snapshot is completed or not? I can see the following example in ovirt-engine-sdk (for snapshot without memory):
# 'Waiting for Snapshot creation to finish' snapshot_service = snapshots_service.snapshot_service(snapshot.id) while True: time.sleep(5) snapshot = snapshot_service.get() if snapshot.snapshot_status == types.SnapshotStatus.OK: break
So I suppose snapshot status shouldn't switch to OK before snapshot creation finishes. But it's not true in Engine master. Is it a bug or a feature?
Thanks, Milan _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/JLHBHQGNVWPDNA...
Milan Zamazal <mzamazal@redhat.com> writes:
Benny Zlotnik <bzlotnik@redhat.com> writes:
Yes, we had this race in another test, that's because of the engine memory lock which is released after the database one
I believe that in this case it's not just a tight race since from what I've seen in Vdsm logs it seems the snapshot is really not completed when its status is switched to OK.
While I have been able to solve the OST issue by examining Engine events, which is a better way to make the check anyway, there still remains the question whether the behavior I observed is a bug that should be reported. (According to the SDK example cited below there is either bug in snapshot status reporting or in the example.) The behavior is demonstrated at https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8725/artifact... As seen in https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8725/artifact...: 2018-07-18 15:54:49,686-04 -- Engine is asked to create the snapshot 2018-07-18 15:55:09,131-04 -- Engine calls SnapshotVDSCommand on lago-basic-suite-master-host-1 2018-07-18 15:55:09,585-04 -- Engine gets Destroy call from OST that got Status.OK on the snapshot I think the behavior should be reproducible in OST basic-suite-4.2, when you disable add_vm_pool test in 004_basic_sanity.py.
Anyway, thank you for confirming that snapshot status check is unreliable, I'll replace it with Engine event check.
I can't provide links since I'm on my phone and away from the laptop but you can look at the cold merge test, we fixed it there by polling the status of the job
On Fri, 20 Jul 2018, 14:11 Milan Zamazal, <mzamazal@redhat.com> wrote:
Hi, a failure of previewing a snapshot with memory has been experienced in OST master suite and I'm not sure whether REST API responses about snapshot status are correct or not.
When I'm creating a snapshot with memory, <snapshot_status> reported from Engine REST API in /api/vms/…/snapshots is initially `locked' and later switches to `ok'. The problem is that `ok' starts being reported before snapshot creation completes, resulting in errors if I try to stop the VM or to preview the snapshot at the moment.
Does <snapshot_status>ok</snapshot_status> guarantee that the snapshot is completed or not? I can see the following example in ovirt-engine-sdk (for snapshot without memory):
# 'Waiting for Snapshot creation to finish' snapshot_service = snapshots_service.snapshot_service(snapshot.id) while True: time.sleep(5) snapshot = snapshot_service.get() if snapshot.snapshot_status == types.SnapshotStatus.OK: break
So I suppose snapshot status shouldn't switch to OK before snapshot creation finishes. But it's not true in Engine master. Is it a bug or a feature?
Thanks, Milan _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/JLHBHQGNVWPDNA...
participants (2)
-
Benny Zlotnik -
Milan Zamazal