- Devel - oVirt List Archives

Re: [ovirt-users] Using ovirt imageio
by Nir Soffer 13 Jul '20

13 Jul '20

On Tue, Jul 7, 2020 at 5:05 PM Łukasz Kołaciński <l.kolacinski(a)storware.eu> wrote: > Dear ovirt community, > Hi Łukasz, Adding devel(a)ovit.org since this topic is more appropriate for the devel list. > I am trying to use ovirt imageio api to receive changed blocks (dirty > bitmap) on ovirt 4.4. Could anyone tell me how to get them step by step? On > the documentation I saw endpoint "GET /images/ticket-uuid/map". I don't > know what ticket-uuid is and how to generate it. I also need to know how to > use this api because I can't reach it via /ovirt-engine/api/ > > I am asking about this endpoint: > > https://www.ovirt.org/documentation/incremental-backup-guide/incremental-ba… > This guide is outdated and should not be used now. The most up to date information is here: https://www.ovirt.org/develop/release-management/features/storage/increment… However the extents API is also outdated in the feature page. We are working on updating it. So here is example: First you must start backup with from_checkpoint_id argument: backup = backups_service.add( types.Backup( disks=disks, from_checkpoint_id="checkpoint-id", ) ) > "checkpoint-id" is the checkpoint created in the last backup. This starts a backup in in incremental mode. Dirty extents are available only in this mode. Then you start a transfer for download, using the backup id: transfer = imagetransfer.create_transfer( connection, disk, types.ImageTransferDirection.DOWNLOAD, backup=types.Backup(id=backup_uuid)) The transfer.transfer_url is the URL to download from, for example: https://host:54322/images/53787351-3f72-44a1-8a26-1323524fac4a Connect to host:54322 and send this request: GET /images/53787351-3f72-44a1-8a26-1323524fac4a/extents?context=dirty And parse the return json list, containing objects like: [ {"start": 0, "length": 65536, "dirty": true}, {"start": 65536, "length": 1048576, "dirty": false}, ... ] For example code of using the imageio API, see imageio http backend: https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71e0… https://github.com/oVirt/ovirt-imageio/blob/d5aa0e1fe659f1bf1247516f83c71e0… We are adding a ImageioClient API that makes it easier to consume without writing any HTTP code: https://gerrit.ovirt.org/c/110068 With this you can use: with ImageioClient(transfer.transfer_url, cafile=args.cafile) as client: for extent in client.extent("dirty"): if extent.dirty: print("##dirty start={} length={}".format(extent.start, extent.length)) client.write_to(sys.stdout.buffer, extent.start, extent.length) print() This will stream the dirty extents to stdout. Not very useful as is, but illustrates how you can consume the data. Here is an example writing extents to a sparse stream format: https://gerrit.ovirt.org/c/110069 For complete backup example code see: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/backup_v… Note the new imagetransfer helper module: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/helpers/… Nir e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu> > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/AIJMMJOR354VK… >

3 4

VM export: task stucked
by francesco＠shellrent.com 10 Jul '20

10 Jul '20

Hi all, everyday at 01:00 AM we perform a daily backup on many VMs hosted in multiple hosts (all with oVirt 4.3) using a custom sciprt written in python3 (using SDK) and everything works "almost" fine. There is one single VM (Windows Server 2016) with a disk of 600 GB (the real disk usage is about 150 GB) hosted on a single node that has a strange behaviour. 1) The export start after a few seconds of the execution of the script, we use "vm_service.export_to_path_on_host" class for exporting the VM as ova file; 2) After few minutes i see in the engine the command "START, DumpXmlsVDSCommand" that is completely istantly both on the host side ad the engine side, and it's fine: 2020-07-02 01:05:45,428+0200 INFO (jsonrpc/7) [api.host] START dumpxmls(vmList=[u'10e88cab-ec4f-4491-b51f-94e3d2e81a0a']) from=::ffff:$ENGINE_IP,39308 (api:48) 2020-07-02 01:05:45,428+0200 INFO (jsonrpc/7) [api.host] FINISH dumpxmls return={... LONG XML ...} 3) after 3 hours i see the following logs about the export task: 2020-07-02 04:11:39,201+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [895d6936-f45c-4766-afd4-408a4e4e9a41] START, GetVolumeInfoVDSCommand(HostName = $OVIRT_HOST, GetVolumeInfoVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', storagePoolId='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', storageDomainId='6775c41c-7d67-451b-8beb-4fd086eade2e', imageGroupId='a084fa36-0f93-45c2-a323-ea9ca2d16677', imageId='55b3eac5-05b2-4bae-be50-37cde7050697'}), log id: 3cbf2c7c 2020-07-02 04:11:39,299+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetQemuImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [895d6936-f45c-4766-afd4-408a4e4e9a41] START, GetQemuImageInfoVDSCommand(HostName = $OVIRT_HOST, GetVolumeInfoVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', storagePoolId='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', storageDomainId='6775c41c-7d67-451b-8beb-4fd086eade2e', imageGroupId='a084fa36-0f93-45c2-a323-ea9ca2d16677', imageId='55b3eac5-05b2-4bae-be50-37cde7050697'}), log id: 47d5122e 2020-07-02 04:43:21,339+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [312261bb] START, PrepareImageVDSCommand(HostName = $OVIRT_HOST, PrepareImageVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 28c880d12020-07-02 04:43:21,650+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [312261bb] Executing Ansible command: ANSIBLE_STDOUT_CALLBACK=imagemeasureplugin /usr/bin/ansible-playbook --ssh-common-args=-F /var/lib/ovirt-engine/.ssh/config --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/ansible-inventory1121551177697847734 --extra-vars=image_path="/rhev/data-center/mnt/_data/6775c41c-7d67-451b-8beb-4fd086eade2e/images/a084fa36-0f93-45c2-a323-ea9ca2d16677/5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04" /usr/share/ovirt-engine/playbooks/ovirt-image-measure.yml [Logfile: /var/log/ovirt-engine/ova /ovirt-image-measure-ansible-20200702044321-$OVIRT_HOST-312261bb.log] 2020-07-02 05:05:20,428+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [64c95c4d] START, UploadStreamVDSCommand(HostName = $OVIRT_HOST, UploadStreamVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 666809cb 2020-07-02 05:05:22,104+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [64c95c4d] START, UploadStreamVDSCommand(HostName = $OVIRT_HOST, UploadStreamVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 33ff46bf 2020-07-02 05:05:29,602+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-287) [64c95c4d] START, HSMClearTaskVDSCommand(HostName = $OVIRT_HOST, HSMTaskGuidBaseVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', taskId='e40a2740-37f6-455d-bed7-554efef761ff'}), log id: f01ff26 2020-07-02 05:05:29,619+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-269) [64c95c4d] START, HSMClearTaskVDSCommand(HostName = $OVIRT_HOST, HSMTaskGuidBaseVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', taskId='1c94df38-88c1-4bf8-89a2-8c322513b21b'}), log id: 6150e13b 4) And then, after several hours, the export fail with the following logs: engine logs. 2020-07-02 12:43:21,653+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [312261bb] Failed to measure image: null. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-image-measure-ansible-20200702044321-$OVIRT_HOST-312261bb.log 2020-07-02 12:43:23,741+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] EVENT_ID: IMPORTEXPORT_EXPORT_VM_TO_OVA_FAILED(1,225), Failed to export Vm $OVIRT_VM_TO_EXPORT as a Virtual Appliance to path /backup/$OVIRT_VM_TO_EXPORT_daily_2020Jul02.ova on Host $OVIRT_HOST 2020-07-02 12:43:26,162+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [54b569e8] START, MergeVDSCommand(HostName = $OVIRT_HOST, MergeVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', vmId='10e88cab-ec4f-4491-b51f-94e3d2e81a0a', storagePoolId='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', storageDomainId='6775c41c-7d67-451b-8beb-4fd086eade2e', imageGroupId='a084fa36-0f93-45c2-a323-ea9ca2d16677', imageId='55b3eac5-05b2-4bae-be50-37cde7050697', baseImageId='5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', topImageId='55b3eac5-05b2-4bae-be50-37cde7050697', bandwidth='0'}), log id: a0b417d 2020-07-02 12:43:26,328+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [54b569e8] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM $OVIRT_HOST command MergeVDS failed: Merge failed 2020-07-02 12:43:26,328+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [54b569e8] HostName = $OVIRT_HOST 2020-07-02 12:43:26,328+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [54b569e8] Command 'MergeVDSCommand(HostName = $OVIRT_HOST, MergeVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', vmId='10e88cab-ec4f-4491-b51f-94e3d2e81a0a', storagePoolId='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', storageDomainId='6775c41c-7d67-451b-8beb-4fd086eade2e', imageGroupId='a084fa36-0f93-45c2-a323-ea9ca2d16677', imageId='55b3eac5-05b2-4bae-be50-37cde7050697', baseImageId='5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', topImageId='55b3eac5-05b2-4bae-be50-37cde7050697', bandwidth='0'})' execution failed: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 2020-07-02 12:43:28,996+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [54b569e8] START, DumpXmlsVDSCommand(HostName = $OVIRT_HOST, Params:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', vmIds='[10e88cab-ec4f-4491-b51f-94e3d2e81a0a]'}), log id: 65709a96 2020-07-02 13:05:38,206+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [5179fcb] START, UploadStreamVDSCommand(HostName = $OVIRT_HOST, UploadStreamVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 44e9ec23 2020-07-02 13:05:39,851+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [5179fcb] START, UploadStreamVDSCommand(HostName = $OVIRT_HOST, UploadStreamVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 68231c70 2020-07-02 13:05:40,356+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-240) [5179fcb] START, HSMClearTaskVDSCommand(HostName = $OVIRT_HOST, HSMTaskGuidBaseVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', taskId='5d61682b-cf47-4637-9014-c647eb5265ee'}), log id: 718885f3 2020-07-02 13:05:50,357+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-290) [5179fcb] START, HSMClearTaskVDSCommand(HostName = $OVIRT_HOST, HSMTaskGuidBaseVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5', taskId='5709eb6c-a656-4a4d-b67f-72dd0bebf3e4'}), log id: 40c18d57 And here the vdsm.log of the host: 2020-07-02 12:43:26,169+0200 INFO (jsonrpc/6) [api.virt] START merge(drive={u'imageID': u'a084fa36-0f93-45c2-a323-ea9ca2d16677', u'volumeID': u'55b3eac5-05b2-4bae-be50-37cde7050697', u'domainID': u'6775c41c-7d67-451b-8beb-4fd086eade2e', u'poolID': u'79d774b7-ca5b-49c2-baf8-9275ba3f1a84'}, baseVolUUID=u'5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', topVolUUID=u'55b3eac5-05b2-4bae-be50-37cde7050697', bandwidth=u'0', jobUUID=u'b49fa67b-4a5a-4f73-823f-9e33a8e20cd2') from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, vmId=10e88cab-ec4f-4491-b51f-94e3d2e81a0a (api:48) 2020-07-02 12:43:26,196+0200 INFO (jsonrpc/6) [vdsm.api] START getVolumeInfo(sdUUID=u'6775c41c-7d67-451b-8beb-4fd086eade2e', spUUID='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', imgUUID=u'a084fa36-0f93-45c2-a323-ea9ca2d16677', volUUID=u'5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', options=None) from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, task_id=bb1d1fd0-9746-4949-9c6c-b069947f5875 (api:48) 2020-07-02 12:43:26,199+0200 INFO (jsonrpc/6) [storage.VolumeManifest] Info request: sdUUID=6775c41c-7d67-451b-8beb-4fd086eade2e imgUUID=a084fa36-0f93-45c2-a323-ea9ca2d16677 volUUID = 5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04 (volume:240) 2020-07-02 12:43:26,205+0200 INFO (jsonrpc/6) [storage.VolumeManifest] 6775c41c-7d67-451b-8beb-4fd086eade2e/a084fa36-0f93-45c2-a323-ea9ca2d16677/5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04 info is {'status': 'OK', 'domain': '6775c41c-7d67-451b-8beb-4fd086eade2e', 'voltype': 'INTERNAL', 'description': '', 'parent': '7e802d0d-d6cd-4979-8f8d-5f4c5d5c3013', 'format': 'COW', 'generation': 0, 'image': 'a084fa36-0f93-45c2-a323-ea9ca2d16677', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '6635716608', 'children': [], 'pool': '', 'ctime': '1593558283', 'capacity': '644245094400', 'uuid': u'5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', 'truesize': '6637277184', 'type': 'SPARSE'} (volume:279) 2020-07-02 12:43:26,205+0200 INFO (jsonrpc/6) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': '6775c41c-7d67-451b-8beb-4fd086eade2e', 'voltype': 'INTERNAL', 'description': '', 'parent': '7e802d0d-d6cd-4979-8f8d-5f4c5d5c3013', 'format': 'COW', 'generation': 0, 'image': 'a084fa36-0f93-45c2-a323-ea9ca2d16677', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '6635716608','children': [], 'pool': '', 'ctime': '1593558283', 'capacity': '644245094400', 'uuid': u'5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', 'truesize': '6637277184', 'type': 'SPARSE'}} from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, task_id=bb1d1fd0-9746-4949-9c6c-b069947f5875 (api:54) 2020-07-02 12:43:26,206+0200 INFO (jsonrpc/6) [vdsm.api] START getVolumeInfo(sdUUID=u'6775c41c-7d67-451b-8beb-4fd086eade2e', spUUID='79d774b7-ca5b-49c2-baf8-9275ba3f1a84', imgUUID=u'a084fa36-0f93-45c2-a323-ea9ca2d16677', volUUID=u'55b3eac5-05b2-4bae-be50-37cde7050697', options=None) from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, task_id=32ce9b21-b9f3-4319-a229-b32df1fb9283 (api:48) 2020-07-02 12:43:26,208+0200 INFO (jsonrpc/6) [storage.VolumeManifest] Info request: sdUUID=6775c41c-7d67-451b-8beb-4fd086eade2e imgUUID=a084fa36-0f93-45c2-a323-ea9ca2d16677 volUUID = 55b3eac5-05b2-4bae-be50-37cde7050697 (volume:240) 2020-07-02 12:43:26,219+0200 INFO (jsonrpc/6) [storage.VolumeManifest] 6775c41c-7d67-451b-8beb-4fd086eade2e/a084fa36-0f93-45c2-a323-ea9ca2d16677/55b3eac5-05b2-4bae-be50-37cde7050697 info is {'status': 'OK', 'domain': '6775c41c-7d67-451b-8beb-4fd086eade2e', 'voltype': 'LEAF', 'description': '', 'parent': '5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', 'format': 'COW', 'generation': 0, 'image': 'a084fa36-0f93-45c2-a323-ea9ca2d16677', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '4077387776', 'children': [], 'pool': '', 'ctime': '1593644682', 'capacity': '644245094400', 'uuid': u'55b3eac5-05b2-4bae-be50-37cde7050697', 'truesize': '4077940736', 'type': 'SPARSE'} (volume:279) 2020-07-02 12:43:26,219+0200 INFO (jsonrpc/6) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': '6775c41c-7d67-451b-8beb-4fd086eade2e', 'voltype': 'LEAF', 'description': '', 'parent': '5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04', 'format': 'COW', 'generation': 0, 'image': 'a084fa36-0f93-45c2-a323-ea9ca2d16677', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '4077387776', 'children': [], 'pool': '', 'ctime': '1593644682', 'capacity': '644245094400', 'uuid': u'55b3eac5-05b2-4bae-be50-37cde7050697', 'truesize': '4077940736', 'type': 'SPARSE'}} from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, task_id=32ce9b21-b9f3-4319-a229-b32df1fb9283 (api:54) 2020-07-02 12:43:26,235+0200 INFO (jsonrpc/6) [virt.vm] (vmId='10e88cab-ec4f-4491-b51f-94e3d2e81a0a') Starting merge with jobUUID=u'b49fa67b-4a5a-4f73-823f-9e33a8e20cd2', original chain=e2bad960-6ad4-44b6-8ff0-69dcca6bf722 < 7e802d0d-d6cd-4979-8f8d-5f4c5d5c3013 < 5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04 < 55b3eac5-05b2-4bae-be50-37cde7050697 (top), disk='sda', base='sda[1]', top=None, bandwidth=0, flags=12 (vm:5951) 2020-07-02 12:43:26,280+0200 ERROR (jsonrpc/6) [virt.vm] (vmId='10e88cab-ec4f-4491-b51f-94e3d2e81a0a') Live merge failed (job: b49fa67b-4a5a-4f73-823f-9e33a8e20cd2) (vm:5957) 2020-07-02 12:43:26,296+0200 INFO (jsonrpc/6) [api.virt] FINISH merge return={'status': {'message': 'Merge failed', 'code': 52}} from=::ffff:$ENGINE_IP,39308, flow_id=54b569e8, vmId=10e88cab-ec4f-4491-b51f-94e3d2e81a0a (api:54) 2020-07-02 12:43:26,296+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.merge failed (error 52) in 0.13 seconds (__init__:312) 2020-07-02 12:43:26,627+0200 INFO (jsonrpc/1) [vdsm.api] START getSpmStatus(spUUID=u'79d774b7-ca5b-49c2-baf8-9275ba3f1a84', options=None) from=::ffff:$ENGINE_IP,39308, flow_id=4968c8cb, task_id=ee82b2d6-39f5-43ad-83a3-803f1af8e234 (api:48) 2020-07-02 12:43:26,628+0200 INFO (jsonrpc/1) [vdsm.api] FINISH getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': -1}} from=::ffff:$ENGINE_IP,39308, flow_id=4968c8cb, task_id=ee82b2d6-39f5-43ad-83a3-803f1af8e234 (api:54) 2020-07-02 12:43:26,628+0200 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00 seconds (__init__:312) 2020-07-02 12:43:26,645+0200 INFO (jsonrpc/3) [vdsm.api] START getStoragePoolInfo(spUUID=u'79d774b7-ca5b-49c2-baf8-9275ba3f1a84', options=None) from=::ffff:$ENGINE_IP,39304, flow_id=4968c8cb, task_id=ab1124bc-651d-4ade-aae4-ff9fea23f6a0 (api:48) 2020-07-02 12:43:26,646+0200 INFO (jsonrpc/3) [vdsm.api] FINISH getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': -1, 'domains': u'6775c41c-7d67-451b-8beb-4fd086eade2e:Active', 'master_uuid': u'6775c41c-7d67-451b-8beb-4fd086eade2e', 'version': '5', 'spm_id': 1, 'type': 'LOCALFS', 'master_ver': 1}, 'dominfo': {u'6775c41c-7d67-451b-8beb-4fd086eade2e': {'status': u'Active', 'diskfree': '3563234983936', 'isoprefix': '', 'alerts': [], 'disktotal': '3923683311616', 'version': 5}}} from=::ffff:$ENGINE_IP,39304, flow_id=4968c8cb, task_id=ab1124bc-651d-4ade-aae4-ff9fea23f6a0 (api:54) 2020-07-02 12:43:26,647+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01 seconds (__init__:312) 5) His task seems block any other export tasks, leave them in pending. There were about 5 export tasks and 1 import temaplte task and as soon we killed the following process, the other 4 tasks and the import one completed after few minutes. 2020-07-02 04:43:21,339+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [312261bb] START, PrepareImageVDSCommand(HostName = $OVIRT_HOST, PrepareImageVDSCommandParameters:{hostId='e8c07142-fcd8-4f78-9158-ffb2caa06dc5'}), log id: 28c880d12020-07-02 04:43:21,650+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [312261bb] Executing Ansible command: ANSIBLE_STDOUT_CALLBACK=imagemeasureplugin /usr/bin/ansible-playbook --ssh-common-args=-F /var/lib/ovirt-engine/.ssh/config --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/ansible-inventory1121551177697847734 --extra-vars=image_path="/rhev/data-center/mnt/_data/6775c41c-7d67-451b-8beb-4fd086eade2e/images/a084fa36-0f93-45c2-a323-ea9ca2d16677/5b76cc6c-1a6b-4e02-8ce4-dc80faf9ed04" /usr/share/ovirt-engine/playbooks/ovirt-image-measure.yml [Logfile: /var/log/ovirt-engine/ova /ovirt-image-measure-ansible-20200702044321-$OVIRT_HOST-312261bb.log] The process that we killed was: vdsm 25338 25332 99 04:14 pts/0 07:40:09 qemu-img measure -O qcow2 /rhev/data-center/mnt/_data/6775c41c-7d67-451b-8beb-4fd086eade2e/images/a084fa36-0f93-45c2-a323-ea9ca2d16677/55b3eac5-05b2-4bae-be50-37cde705069 A strace -p of the PID 25338 showed following lines: lseek(11, 3056795648, SEEK_DATA) = 3056795648 lseek(11, 3056795648, SEEK_HOLE) = 13407092736 lseek(14, 128637468672, SEEK_DATA) = 128637468672 lseek(14, 128637468672, SEEK_HOLE) = 317708828672 lseek(14, 128646250496, SEEK_DATA) = 128646250496 lseek(14, 128646250496, SEEK_HOLE) = 317708828672 lseek(14, 128637730816, SEEK_DATA) = 128637730816 lseek(14, 128637730816, SEEK_HOLE) = 317708828672 lseek(14, 128646774784, SEEK_DATA) = 128646774784 lseek(14, 128646774784, SEEK_HOLE) = 317708828672 lseek(14, 128646709248, SEEK_DATA) = 128646709248 Can yu help us figuring out what is going on? If any other informations is needed I'll provide 'em ASAP. Thank you for your patience and your time, Regards Francesco

3 2

oVirt vdsm Bulk-API deep dive
by Nir Soffer 09 Jul '20

09 Jul '20

I want to share an interesting talk on the new StorageDomain dump API. When: Thursday, July 9⋅16:00 – 16:45 (Israel time) Where: https://redhat.bluejeans.com/212781836 Who: Amit Bawer StorageDomain dump API is a faster way to query oVirt storage domain contents, optimized for bulk operations such as collecting sos reports or importing storage domains during disaster recovery. In recent tests we found that it is 295 times faster compared with vdsm-tool dump-volume-chains: https://bugzilla.redhat.com/show_bug.cgi?id=1557147#c26 What can you do with this API? Query volumes metadata by disk id: # vdsm-client StorageDomain dump sd_id=56ecc03c-4bb5-4792-8971-3c51ea924d2e | jq '.volumes | .[] | select(.image=="d7ead22a-0fbf-475c-a62f-6f7bc8473acb")' { "apparentsize": 6442450944, "capacity": 6442450944, "ctime": 1593128884, "description": "{\"DiskAlias\":\"fedora-32.raw\",\"DiskDescription\":\"Uploaded disk\"}", "disktype": "DATA", "format": "RAW", "generation": 0, "image": "d7ead22a-0fbf-475c-a62f-6f7bc8473acb", "legality": "LEGAL", "parent": "00000000-0000-0000-0000-000000000000", "status": "OK", "truesize": 6442455040, "type": "PREALLOCATED", "voltype": "LEAF" } Find all templates: # vdsm-client StorageDomain dump sd_id=56ecc03c-4bb5-4792-8971-3c51ea924d2e | jq '.volumes | .[] | select(.voltype=="SHARED")' { "apparentsize": 6442450944, "capacity": 6442450944, "ctime": 1593116563, "description": "{\"DiskAlias\":\"fedora-32.raw\",\"DiskDescription\":\"Uploaded disk\"}", "disktype": "DATA", "format": "RAW", "generation": 0, "image": "9b62b5fa-920e-4d0c-baf6-40406106e48e", "legality": "LEGAL", "parent": "00000000-0000-0000-0000-000000000000", "status": "OK", "truesize": 6442516480, "type": "PREALLOCATED", "voltype": "SHARED" } Please join the talk if you want to learn more. Cheers, Nir

2 2

ovirt-engine has been tagged (ovirt-engine-4.3.11.2)
by Tal Nisan 09 Jul '20

09 Jul '20

1 0

Re: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master - Build # 1655 - Still Failing!
by Yedidyah Bar David 07 Jul '20

07 Jul '20

On Sun, Jun 28, 2020 at 6:23 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1655/ This fails for some time now. Checked last one ^^, and it failed in: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1655… 2020-06-27 23:05:02,253-0400 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_task': 'ovirt.hosted_engine_setup : Check OVF_STORE volume status'} 2020-06-27 23:05:02,253-0400 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Check OVF_STORE volume status kwargs is_conditional:False 2020-06-27 23:05:02,254-0400 DEBUG ansible on_any args localhostTASK: ovirt.hosted_engine_setup : Check OVF_STORE volume status kwargs 2020-06-27 23:05:03,816-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f454e3d1eb8> kwargs ... 2020-06-27 23:09:39,166-0400 DEBUG var changed: host "localhost" var "ovf_store_status" type "<class 'dict'>" value: "{ "changed": true, "failed": true, "msg": "All items completed", "results": [ { "ansible_loop_var": "item", "attempts": 12, Meaning, it ran 12 times the command: vdsm-client Volume getInfo storagepoolID=41c9fdea-b8e9-11ea-ae2a-5452c0a8c863 storagedomainID=10a69775-8fb6-437d-9e78-2ecfd77c0a45 imageID=c2ad2065-1c8b-4ec1-afdd-f7cefc708cf9 volumeID=6b835f55-a512-4f83-9d25-f6837d8b5cb1 and never got a result with output including "Updated". Any idea? Perhaps it's just a bug in the test, and we should test something else other than "Updated"? Or perhaps there was some other problem but I failed to find it? Thanks, > Build Number: 1655 > Build Status: Still Failing > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #1643 > [Galit] Add missing repo which has collectd-write_syslog > > [Sandro Bonazzola] Revert "ovirt-release: run only on fc29 nodes" > > > Changes for Build #1644 > [Michal Skrivanek] make GLANCE failures fatal > > > Changes for Build #1645 > [Michal Skrivanek] use // in glance URL > > > Changes for Build #1646 > [Galit] Fix: No module named ost_utils.memoized > > > Changes for Build #1647 > [Martin Necas] ansible: fix deploy scripts to el8 > > [arachmani] Add arachman(a)redhat.com to jenkins recipient lists > > > Changes for Build #1648 > [Martin Necas] ansible: fix deploy scripts to el8 > > [Evgheni Dereveanchin] Do not run jenkins check-patch job on fc29 > > > Changes for Build #1649 > [Martin Necas] ansible: fix deploy scripts to el8 > > > Changes for Build #1650 > [Martin Necas] ansible: fix deploy scripts to el8 > > > Changes for Build #1651 > [Galit] Add centos8.2 official to templates > > > Changes for Build #1652 > [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run > > [Gal Ben Haim] Add gcc to fcraw > > [Sandro Bonazzola] pipelines: add ovirt-dependencies > > > Changes for Build #1653 > [Evgeny Slutsky] he-basic-suite-master: Add Domain_name to the host when vm_run > > > Changes for Build #1654 > [Galit] Add centos 8.2 image: el8.2-base > > [Evgheni Dereveanchin] do not expand values in run_oc_playbook > > > Changes for Build #1655 > [Galit] Add centos 8.2 image: el8.2-base > > > > > ----------------- > Failed Tests: > ----------------- > No tests ran. -- Didi

3 5

sic transit gloria mundi
by Dan Kenigsberg 06 Jul '20

06 Jul '20

Seven years ago, in June 2013, Giuseppe Valarelli introduced tests/functional/networkTests.py in https://gerrit.ovirt.org/#/c/14840/. It held high aspirations: to provide a comprehensive functional test for Vdsm's network API, in order to ease development of new code and reduce chances of reintroducing old bugs. 390 commits later, Andrej Cernak removed the last lines of code from that file in https://gerrit.ovirt.org/#/c/109182/. In between, many many other developers contributed to the set of tests, and later dissipated its content to multiple pytest-driven test modules. tests/functional/networkTests.py was never an emblem of great software design; it makes me happy to see it gone. But it was very useful and its spirit lives on: API tests are now the rule of Vdsm development, not an exception. I'd like to thank all the people who made this possible. $ git log 1c5f15c9b~ -- tests/functional/networkTests.py|grep Author|sort|uniq -c|sort -nr 72 Author: Edward Haas <edwardh(a)redhat.com> 58 Author: Ido Barkan <ibarkan(a)redhat.com> 46 Author: Ondřej Svoboda <osvoboda(a)redhat.com> 40 Author: Petr Horáček <phoracek(a)redhat.com> 35 Author: Dan Kenigsberg <danken(a)redhat.com> 34 Author: Antoni S. Puimedon <asegurap(a)redhat.com> 24 Author: Bell Levin <blevin(a)redhat.com> 23 Author: Leon Goldberg <lgoldber(a)redhat.com> 18 Author: Giuseppe Vallarelli <gvallare(a)redhat.com> 8 Author: Assaf Muller <amuller(a)redhat.com> 5 Author: Sveta Haas <sveta.haas(a)gmail.com> 4 Author: Mark Wu <wudxw(a)linux.vnet.ibm.com> 3 Author: Nir Soffer <nsoffer(a)redhat.com> 2 Author: Yaniv Bronhaim <ybronhei(a)redhat.com> 2 Author: Sagi Shnaidman <sshnaidm(a)redhat.com> 2 Author: Petr Sebek <psebek(a)redhat.com> 2 Author: Miguel Angel Ajo <miguelangel(a)ajo.es> 2 Author: Andrej Cernek <acernek(a)redhat.com> 1 Author: Viliam Serecun <v.serecun(a)gmail.com> 1 Author: Vered Volansky <vvolansk(a)redhat.com> 1 Author: Timothy Asir Jeyasingh <tjeyasin(a)redhat.com> 1 Author: shlad <a.shlado(a)gmail.com> 1 Author: pkliczewski <piotr.kliczewski(a)gmail.com> 1 Author: Petr Benas <pbenas(a)redhat.com> 1 Author: Marcin Sobczyk <msobczyk(a)redhat.com> 1 Author: Marcin Mirecki <mmirecki(a)redhat.com> 1 Author: Ilia Meerovich <iliam(a)redhat.com> 1 Author: Genadi Chereshnya <gcheresh(a)redhat.com> 1 Author: Dominik Holler <dholler(a)redhat.com>

2 1

ovirt-engine has been tagged (ovirt-engine-4.4.1.7)
by Sandro Bonazzola 03 Jul '20

03 Jul '20

1 0

Backup: how to download only used extents from imageio backend
by Michael Ablassmeier 02 Jul '20

02 Jul '20

hi, im currently looking at the new incremental backup api that has been part of the 4.4 and RHV 4.4-beta release. So far i was able to create full/incremental backups and restore without any problem. Now, using the backup_vm.py example from the ovirt-engine-sdk i get the following is happening during a full backup: 1) imageio client api requests transfer 2) starts qemu-img to create a local qemu image with same size 3) starts qemu-nbd to serve this image 4) reads used extents from provided imageio source, passes data to qemu-nbd process 5) resulting file is a thin provisioned qcow image with the actual data of the VM's used space. while this works great, it has one downside: if i backup a virtual machine with lots of used extents, or multiple virtual machines at the same time, i may run out of space, if my primary backup target is not a regular disk. Imagine i want to stream the FULL backup to tape directly like backup_vm.py full [..] <vm_uuid> /dev/nst0 thats currently not possible, because qemu-img is not able to open a tape device directly, given its nature of the qcow2 format. So what iam basically looking for, is a way to download only the extents from the imageio server that are really in use, not depending on qemu-* tools, to be able to pipe the data somehwere else. Standard tools, like for example curl, will allways download the full provisioned image from the imageio backend (of course). I noticed is that it is possible to query the extents via: https://tranfer_node:54322/images/d471c659-889f-4e7f-b55a-a475649c48a6/exte… As i failed to find them, are there any existing functions/api calls that could be used to download only the used extents to a file/fifo pipe? So far, i played around with the _internal.io.copy function, beeing able to at least read the data into a in memory BytesIO stream, but thats not the solution to my "problem" :) bye, - michael

3 7

ovirt-engine has been tagged (ovirt-engine-4.4.1.6)
by Tal Nisan 02 Jul '20

02 Jul '20

1 0

Issue while installing ovirt-imageio-daemon
by Sandro Bonazzola 02 Jul '20

02 Jul '20

Hi, just noticed in appliance build at https://jenkins.ovirt.org/job/ovirt-appliance_standard-check-patch/286/cons… 15:18:22,373 INFO anaconda:program: Installing : ovirt-imageio-daemon-2.0.9-1.el8.x86_64 90/223 <https://jenkins.ovirt.org/job/ovirt-appliance_standard-check-patch/286/cons…> 15:18:22,373 INFO anaconda:program: warning: group Development/Debug does not exist - using root <https://jenkins.ovirt.org/job/ovirt-appliance_standard-check-patch/286/cons…> 15:18:22,373 INFO anaconda:program: <https://jenkins.ovirt.org/job/ovirt-appliance_standard-check-patch/286/cons…> 15:18:22,374 INFO anaconda:program: Running scriptlet: ovirt-imageio-daemon-2.0.9-1.el8.x86_64 90/223 Sounds like a bug in the spec file. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

3 3