how (inefficient) does vdsm import sparse disks in local storage(file based) data domain?
Hi, I might be totally mistaken, so please tell me first if I understand the process: when vdsm imports a template from an export domain to local storage, according to my "ps" output, vdsm copies sparse templates using "dd": vdsm 14378 0.3 0.0 59032 4424 ? S<l 14:43 0:05 \_ /usr/bin/qemu-img convert -t none -f raw /rhev/data-center/d6ec1bb0-43ce-4e09-9a7a-e54f9779af64/4ed22092-e961-4e04-b77b-2648ad21f83a/image vdsm 17291 0.0 0.0 8344 608 ? D< 15:14 0:00 \_ /bin/dd iflag=direct if=/rhev/data-center/mnt/$REDACTED:_home_iso/e7b8f2e2-b9f7-493c-96b5-33bfb091d0ba/dom_md/metadata bs=4096 count=1 If this is correct I have some questions regarding this behaviour: I understand that "dd" is "rock solid" works everywhere and for maybe every storage/format combination and thus was maybe chosen to do the work. But if I'm right and I have a 100 GB sparse disk which actually uses just 1 GB "dd" will copy the whole 100 GB over network, which is really not efficient at all. So here's my proposal (if I do not have any mistakes in the above): at least make a: tar -cpSvf disk.tar /path/to/disk and than "dd" this over the network, because tar _can_ handle sparse files. Of course any other more efficient mechanism would be okay too. If my observation is right, and the project think it's worth to implement such a thing I would create an RFE for this and I also would like to help code this (in my leisure time). PS: I did observe this behaviour on ancient ovirt 3.3 with vdsm version: vdsm-4.13.4-0.el6.x86_64 but I didn't see anything in release notes which changed this behaviour. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
----- Original Message -----
From: "Sven Kieske" <s.kieske@mittwald.de> To: devel@ovirt.org Sent: Thursday, October 23, 2014 4:31:02 PM Subject: [ovirt-devel] how (inefficient) does vdsm import sparse disks in local storage(file based) data domain?
Hi,
I might be totally mistaken, so please tell me first if I understand the process:
when vdsm imports a template from an export domain to local storage, according to my "ps" output, vdsm copies sparse templates using "dd":
vdsm 14378 0.3 0.0 59032 4424 ? S<l 14:43 0:05 \_ /usr/bin/qemu-img convert -t none -f raw /rhev/data-center/d6ec1bb0-43ce-4e09-9a7a-e54f9779af64/4ed22092-e961-4e04-b77b-2648ad21f83a/image vdsm 17291 0.0 0.0 8344 608 ? D< 15:14 0:00 \_ /bin/dd iflag=direct if=/rhev/data-center/mnt/$REDACTED:_home_iso/e7b8f2e2-b9f7-493c-96b5-33bfb091d0ba/dom_md/metadata bs=4096 count=1
If this is correct I have some questions regarding this behaviour:
I understand that "dd" is "rock solid" works everywhere and for maybe every storage/format combination and thus was maybe chosen to do the work.
But if I'm right and I have a 100 GB sparse disk which actually uses just 1 GB "dd" will copy the whole 100 GB over network, which is really not efficient at all.
We have this patch for next version: http://gerrit.ovirt.org/33355
So here's my proposal (if I do not have any mistakes in the above):
at least make a: tar -cpSvf disk.tar /path/to/disk and than "dd" this over the network, because tar _can_ handle sparse files.
Of course any other more efficient mechanism would be okay too.
If my observation is right, and the project think it's worth to implement such a thing I would create an RFE for this
Please do
and I also would like to help code this (in my leisure time).
PS: I did observe this behaviour on ancient ovirt 3.3 with vdsm version: vdsm-4.13.4-0.el6.x86_64 but I didn't see anything in release notes which changed this behaviour.
-- Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 23/10/14 15:51, Nir Soffer wrote:
We have this patch for next version: http://gerrit.ovirt.org/33355
Awesome!
So here's my proposal (if I do not have any mistakes in the above):
at least make a: tar -cpSvf disk.tar /path/to/disk and than "dd" this over the network, because tar _can_ handle sparse files.
Of course any other more efficient mechanism would be okay too.
If my observation is right, and the project think it's worth to implement such a thing I would create an RFE for this
Please do
Here it is: https://bugzilla.redhat.com/show_bug.cgi?id=1156115 -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
participants (2)
-
Nir Soffer -
Sven Kieske