On Tue, Jul 7, 2020 at 5:05 PM Łukasz Kołaciński <l.kolacinski(a)storware.eu&gt; wrote: > Dear ovirt community, > Hi Łukasz, Adding devel(a)ovit.org since this topic is more appropriate for the devel list. > I am trying to use ovirt imageio api to receive changed blocks (dirty > bitmap) on ovirt 4.4. Could anyone tell me how to get them step by step? On > the documentation I saw endpoint "GET /images/ticket-uuid/map". I don't > know what ticket-uuid is and how to generate it. I also need to know how to > use this api because I can't reach it via /ovirt-engine/api/ > > I am asking about this endpoint: > > https://www.ovirt.org/documentation/incremental-backup-guide/incremental-... > This guide is outdated and should not be used now.

The most up to date information is here: https://www.ovirt.org/develop/release-management/features/storage/increme... However the extents API is also outdated in the feature page. We are working on updating it. So here is example: First you must start backup with from_checkpoint_id argument: backup = backups_service.add( types.Backup( disks=disks, from_checkpoint_id="checkpoint-id", ) ) > "checkpoint-id" is the checkpoint created in the last backup. This starts a backup in in incremental mode. Dirty extents are available only in this mode. Then you start a transfer for download, using the backup id: transfer = imagetransfer.create_transfer( connection, disk, types.ImageTransferDirection.DOWNLOAD, backup=types.Backup(id=backup_uuid)) The transfer.transfer_url is the URL to download from, for example: https://host:54322/images/53787351-3f72-44a1-8a26-1323524fac4a Connect to host:54322 and send this request: GET /images/53787351-3f72-44a1-8a26-1323524fac4a/extents?context=dirty And parse the return json list, containing objects like: [ {"start": 0, "length": 65536, "dirty": true}, {"start": 65536, "length": 1048576, "dirty": false}, ... ] For example code of using the imageio API, see imageio http backend: https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71... https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71... We are adding a ImageioClient API that makes it easier to consume without writing any HTTP code: https://gerrit.ovirt.org/c/110068 With this you can use: with ImageioClient(transfer.transfer_url, cafile=args.cafile) as client: for extent in client.extent("dirty"): if extent.dirty: print("##dirty start={} length={}".format(extent.start, extent.length)) client.write_to(sys.stdout.buffer, extent.start, extent.length) print() This will stream the dirty extents to stdout. Not very useful as is, but illustrates how you can consume the data. Here is an example writing extents to a sparse stream format: https://gerrit.ovirt.org/c/110069 For complete backup example code see: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/backup... Note the new imagetransfer helper module: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/helper... Nir e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu&gt; > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/AIJMMJOR354... > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/S6OR7YRZCJM...

On Mon, 13 Jul 2020 16:58:17 +0200 Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > Il giorno mar 7 lug 2020 alle ore 18:07 Nir Soffer <nsoffer(a)redhat.com&gt; ha > scritto: > > > > > > > On Tue, Jul 7, 2020 at 5:05 PM Łukasz Kołaciński < l.kolacinski(a)storware.eu&gt; > > wrote: > > > >> Dear ovirt community, > >> > > > > Hi Łukasz, > > > > Adding devel(a)ovit.org since this topic is more appropriate for the devel > > list. > > > > > >> I am trying to use ovirt imageio api to receive changed blocks (dirty > >> bitmap) on ovirt 4.4. Could anyone tell me how to get them step by step? On > >> the documentation I saw endpoint "GET /images/ticket-uuid/map". I don't > >> know what ticket-uuid is and how to generate it. I also need to know how to > >> use this api because I can't reach it via /ovirt-engine/api/ > >> > >> I am asking about this endpoint: > >> > >> https://www.ovirt.org/documentation/incremental-backup-guide/incremental-... > >> > > > > This guide is outdated and should not be used now. > > > > Let's ensure the official documentation will be aligned ASAP to what has > been released. > +Petr Kovar <pkovar(a)redhat.com&gt; can you please handle this? Is this about updating oVirt docs based on what we have documented downstream? Downstream we had to target 4.4.2 with incremental backup updates due to a large number of 4.4.1 priorities.

https://www.ovirt.org/documentation/incremental-backup-guide/ doesn't even seem to be linked from https://www.ovirt.org/documentation -- maybe we should just unpublish it?

Thanks, pk > > The most up to date information is here: > > > > https://www.ovirt.org/develop/release-management/features/storage/increme... > > > > However the extents API is also outdated in the feature page. We are > > working on updating it. > > > > So here is example: > > > > First you must start backup with from_checkpoint_id argument: > > > > backup = backups_service.add( > > types.Backup( > > disks=disks, > > from_checkpoint_id="checkpoint-id", > > ) > > ) > > > >> > > "checkpoint-id" is the checkpoint created in the last backup. > > > > This starts a backup in in incremental mode. Dirty extents are available > > only > > in this mode. > > > > Then you start a transfer for download, using the backup id: > > > > transfer = imagetransfer.create_transfer( > > connection, > > disk, > > types.ImageTransferDirection.DOWNLOAD, > > backup=types.Backup(id=backup_uuid)) > > > > The transfer.transfer_url is the URL to download from, for example: > > > > https://host:54322/images/53787351-3f72-44a1-8a26-1323524fac4a > > > > Connect to host:54322 and send this request: > > > > GET /images/53787351-3f72-44a1-8a26-1323524fac4a/extents?context=dirty > > > > And parse the return json list, containing objects like: > > > > [ > > {"start": 0, "length": 65536, "dirty": true}, > > {"start": 65536, "length": 1048576, "dirty": false}, > > ... > > ] > > > > For example code of using the imageio API, see imageio http backend: > > > > https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71... > > > > https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71... > > > > We are adding a ImageioClient API that makes it easier to consume without > > writing any HTTP code: > > https://gerrit.ovirt.org/c/110068 > > > > With this you can use: > > > > with ImageioClient(transfer.transfer_url, cafile=args.cafile) as > > client: > > for extent in client.extent("dirty"): > > if extent.dirty: > > print("##dirty start={} length={}".format(extent.start, > > extent.length)) > > client.write_to(sys.stdout.buffer, extent.start, > > extent.length) > > print() > > > > This will stream the dirty extents to stdout. Not very useful as is, but > > illustrates how > > you can consume the data. > > > > Here is an example writing extents to a sparse stream format: > > https://gerrit.ovirt.org/c/110069 > > > > For complete backup example code see: > > > > https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/backup... > > > > Note the new imagetransfer helper module: > > > > https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/helper... > > > > Nir > > > > e-mail: l.kolacinski(a)storware.eu > >> <m.helbert(a)storware.eu&gt; > >> > >> > >> > >> > >> *[image: STORWARE]* <http://www.storware.eu/> > >> > >> > >> > >> *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > >> <https://www.storware.eu/>* > >> > >> *[image: facebook]* <https://www.facebook.com/storware> > >> > >> *[image: twitter]* <https://twitter.com/storware> > >> > >> *[image: linkedin]* <https://www.linkedin.com/company/storware> > >> > >> *[image: Storware_Stopka_09]* > >> <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > >> > >> > >> > >> *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > >> 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > >> dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > >> i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > >> przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > >> podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > >> podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > >> pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > >> tej wiadomości z wszelkich komputerów. **This message is intended only > >> for the person or entity to which it is addressed and may contain > >> confidential and/or privileged material. Any review, retransmission, > >> dissemination or other use of, or taking of any action in reliance upon, > >> this information by persons or entities other than the intended recipient > >> is prohibited. If you have received this message in error, please contact > >> the sender and remove the material from all of your computer systems.* > >> > >> _______________________________________________ > >> Users mailing list -- users(a)ovirt.org > >> To unsubscribe send an email to users-leave(a)ovirt.org > >> Privacy Statement: https://www.ovirt.org/privacy-policy.html > >> oVirt Code of Conduct: > >> https://www.ovirt.org/community/about/community-guidelines/ > >> List Archives: > >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/AIJMMJOR354... > >> > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/privacy-policy.html > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/S6OR7YRZCJM... > > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA <https://www.redhat.com/> > > sbonazzo(a)redhat.com > <https://www.redhat.com/> > > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours. > <https://mojo.redhat.com/docs/DOC-1199578>* -- Petr Kovar Documentation Program Manager | Red Hat Virtualization & Developer Tools Customer Content Services | Red Hat Czech s.r.o.