[ovirt-users] VM incremental backup
Nathanaël Blanchet
blanchet at abes.fr
Wed Oct 12 09:08:25 UTC 2016
Hi,
After testing many ways to transfer sparse file between servers (in
local cp is enough), here is the only way wich is time efficient:
Source side
# tar cSf - /var/lib/libvirt/images/acore_thin.img | nc -l 5000
Destination side
# nc 10.34.100.128 5000 | pv -btr | tar xSf -
3,54GB 0:02:19 [ 26MB/s]
# ls -lsh /var/lib/libvirt/images/acore_thin.img
3,8G -rw-r--r--. 1 root root 40G 6 oct. 12:43
/var/lib/libvirt/images/acore_thin.img
Le 07/10/2016 à 08:50, Arman Khalatyan a écrit :
>
> The sparse files are special files(please checkout the wiki pages) to
> conserve some diskspace and copy time.
> The rsync --sparce does not take an advantage of it. It is trying to
> checksum whole virtual diskspace which is obviously zero filled.
> If you "cp" file then "cp" will try to autodetect it otherwise your
> file willbe converted to regular file. You can also copy files with
> "tar -S ".
> To check the size of actual vs virtual(sparse) you can use:
> df -bsh filename vs df -sh
>
>
> Am 06.10.2016 11:34 vorm. schrieb "Nathanaël Blanchet"
> <blanchet at abes.fr <mailto:blanchet at abes.fr>>:
>
> Hi all,
>
> I didn't know this tool, after testing the original 40GB image
> size is reduced as expected to 2,9GB with du -sh, but transfer
> between servers continue to act as a 40GB image.
>
> So if I understand virt-sparsify reduces the effective space on
> the local disk, but the disk is seen with its original size by
> other systems, and there is no advantage in transferring the image
> from one server to an other.
>
> Please tell me if I'm wrong.
>
> Le 29/09/2016 à 15:10, Kai Wagner a écrit :
>>
>> Hey,
>>
>> I don't wont to break into your discussion, but I also never
>> heard about virt-sparsfiy and the sparse option for dd.
>>
>> I tested it and it works like a charm. I converted a block device
>> 65GB with dd sparse to 40GB raw image and afterwards I used
>> virsh-sparsify to reduce the size down to 6.8GB into a qcow2
>> image file.
>>
>> Thanks a lot for that hint.
>>
>> Kai
>>
>> Am 27.09.2016 um 15:17 schrieb Sven Achtelik:
>>>
>>> No, I never came across this approach. I didn’t know about
>>> virt-sparsify.
>>>
>>> I’ll look into that and give it a try.
>>>
>>> Thank you
>>>
>>> it-novum GmbH
>>> i. A. Kai Wagner ● Team Lead Support & Presales openATTIC
>>> _____________________________________________________________
>>> Unser Newsletter Service: Jetzt Business Open Source News
>>> abonnieren! <http://www.it-novum.com/newsletter?wm=mail>
>>> <http://www.it-novum.com/newsletter?wm=mail>
>>> Folgen Sie uns:
>>> <http://www.it-novum.com/blog/?wm=mail><http://www.it-novum.com/blog/feed?wm=mail><https://twitter.com/itnovum><http://www.xing.com/companies/itnovumgmbh/about><http://www.linkedin.com/company/it-novum-gmbh><https://plus.google.com/105835582713486693501><http://www.youtube.com/user/itnovumGmbH?sub_confirmation=1%20><https://www.facebook.com/itnovum>
>>>
>>>
>>>
>>> <http://www.it-novum.com/?wm=mail> Tel: +49 661 103-762
>>> <tel:%2B49%20661%20103-762>
>>> Fax: +49 661 10317762 <tel:%2B49%20661%2010317762>
>>> Mail: Kai.Wagner at it-novum.com <mailto:Kai.Wagner at it-novum.com>
>>>
>>>
>>> it-novum GmbH • Edelzeller Straße 44 • 36043 Fulda •
>>> http://www.it-novum.com/
>>> Handelsregister Amtsgericht Fulda, HRB 1934 • Geschäftsführer:
>>> Michael Kienle • Sitz der Gesellschaft: Fulda
>>>
>>> Der Inhalt dieser E-Mail ist vertraulich. Wenn Sie nicht der
>>> eigentliche Empfänger sein sollten, informieren Sie bitte sofort
>>> den Absender oder vernichten umgehend diese Mail. Jegliche
>>> unerlaubte Vervielfältigung oder Weiterleitung dieser Mail ist
>>> strengstens verboten.
>>> This e-mail may contain confidential and/or priviledged
>>> information. If you are not the intended recepient (or have
>>> received this e-mail in error) please notify the sender
>>> immediately and destroy this e-mail. Any unauthorised copying,
>>> disclosure or distribution of material in this e-mail is
>>> strictly forbidden.
>>>
>>> *Von:*Yaniv Dary [mailto:ydary at redhat.com]
>>> *Gesendet:* Dienstag, 27. September 2016 15:10
>>> *An:* Sven Achtelik <Sven.Achtelik at mailpool.us>
>>> <mailto:Sven.Achtelik at mailpool.us>
>>> *Cc:* Maton, Brett <matonb at ltresources.co.uk>
>>> <mailto:matonb at ltresources.co.uk>; vasily.lamykin at MegaFon.ru
>>> <mailto:vasily.lamykin at MegaFon.ru>; Ovirt Users
>>> <users at ovirt.org> <mailto:users at ovirt.org>
>>> *Betreff:* Re: [ovirt-users] VM incremental backup
>>>
>>> As I see it you have two options:
>>> - In backup use 'dd' with 'conv=sparse' (or similar tool that
>>> allows sparse).
>>>
>>> - After backup use virt-sparsify [1] to reduce the size to the
>>> real used size prior to restore.
>>>
>>> To make this extra efficient you can use virt-sparsify anyways
>>> after backup to make the file even smaller.
>>>
>>> Have you considered this approach?
>>>
>>> [1] http://libguestfs.org/virt-sparsify.1.html
>>> <http://libguestfs.org/virt-sparsify.1.html>
>>>
>>>
>>> Yaniv Dary
>>> Technical Product Manager
>>> Red Hat Israel Ltd.
>>> 34 Jerusalem Road
>>> Building A, 4th floor
>>> Ra'anana, Israel 4350109
>>> Tel : +972 (9) 7692306 <tel:%2B972%20%289%29%207692306>
>>> 8272306
>>> Email: ydary at redhat.com <mailto:ydary at redhat.com>
>>> IRC : ydary
>>>
>>> On Tue, Sep 27, 2016 at 4:01 PM, Sven Achtelik
>>> <Sven.Achtelik at mailpool.us <mailto:Sven.Achtelik at mailpool.us>>
>>> wrote:
>>>
>>> Hi Yaniv,
>>>
>>> how can this be done with DD ? Since it doesn’t know if the
>>> block is free space ? I’ve been looking for such a solution
>>> for a long time now. Everything I could find out was that I
>>> have to use a utility that understands the FS and therefore
>>> knows where the free space is.
>>>
>>> Thank you,
>>>
>>> Sven
>>>
>>> *Von:*users-bounces at ovirt.org
>>> <mailto:users-bounces at ovirt.org>
>>> [mailto:users-bounces at ovirt.org
>>> <mailto:users-bounces at ovirt.org>] *Im Auftrag von *Yaniv Dary
>>> *Gesendet:* Dienstag, 27. September 2016 14:38
>>> *An:* Maton, Brett <matonb at ltresources.co.uk
>>> <mailto:matonb at ltresources.co.uk>>
>>> *Cc:* vasily.lamykin at MegaFon.ru
>>> <mailto:vasily.lamykin at MegaFon.ru>; Ovirt Users
>>> <users at ovirt.org <mailto:users at ovirt.org>>
>>> *Betreff:* Re: [ovirt-users] VM incremental backup
>>>
>>> Full VM disk backup. If you use dd you can drop the 0 parts
>>> of the disk.
>>>
>>>
>>> Yaniv Dary
>>>
>>> Technical Product Manager
>>>
>>> Red Hat Israel Ltd.
>>>
>>> 34 Jerusalem Road
>>>
>>> Building A, 4th floor
>>>
>>> Ra'anana, Israel 4350109
>>>
>>> Tel : +972 (9) 7692306 <tel:%2B972%20%289%29%207692306>
>>>
>>> 8272306
>>>
>>> Email: ydary at redhat.com <mailto:ydary at redhat.com>
>>>
>>> IRC : ydary
>>>
>>> 2016-09-27 15:27 GMT+03:00 Maton, Brett
>>> <matonb at ltresources.co.uk <mailto:matonb at ltresources.co.uk>>:
>>>
>>> Is there a preferred or recommended backup method in the
>>> meantime ?
>>>
>>> 2016-09-22 10:30 GMT+01:00 vasily.lamykin at MegaFon.ru
>>> <mailto:vasily.lamykin at MegaFon.ru>
>>> <vasily.lamykin at megafon.ru
>>> <mailto:vasily.lamykin at megafon.ru>>:
>>>
>>> Hello!
>>>
>>> I would like to understand the existence of the
>>> possibility of incremental VM backups.
>>>
>>> And if it is, the means by which it is achieved.
>>>
>>> While I was only able to achieve a complete backup
>>> of the virtual machine (but it is a very large
>>> amount of information and time) by cloning a
>>> snapshot of the VM and export to data domain.
>>>
>>> *Ламыкин Василий*
>>>
>>> Старший инженер по эксплуатации сервисных платформ
>>>
>>> Столичный ф-ал ПАО "МегаФон
>>>
>>> +7 (926) 500-3308 <tel:%2B7%20%28926%29%20500-3308>
>>>
>>> МегаФон лого+знак РУС B2C
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> Информация в этом сообщении предназначена
>>> исключительно для конкретных лиц, которым она
>>> адресована. В сообщении может содержаться
>>> конфиденциальная информация, которая не может быть
>>> раскрыта или использована кем-либо, кроме адресатов.
>>> Если вы не адресат этого сообщения, то
>>> использование, переадресация, копирование или
>>> распространение содержания сообщения или его части
>>> незаконно и запрещено. Если Вы получили это
>>> сообщение ошибочно, пожалуйста, незамедлительно
>>> сообщите отправителю об этом и удалите со всем
>>> содержимым само сообщение и любые возможные его
>>> копии и приложения.
>>>
>>> The information contained in this communication is
>>> intended solely for the use of the individual or
>>> entity to whom it is addressed and others authorized
>>> to receive it. It may contain confidential or
>>> legally privileged information. The contents may not
>>> be disclosed or used by anyone other than the
>>> addressee. If you are not the intended recipient(s),
>>> any use, disclosure, copying, distribution or any
>>> action taken or omitted to be taken in reliance on
>>> it is prohibited and may be unlawful. If you have
>>> received this communication in error please notify
>>> us immediately by responding to this email and then
>>> delete the e-mail and all attachments and any copies
>>> thereof.
>>>
>>> (c)20mf50
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org <mailto:Users at ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/users
>> <http://lists.ovirt.org/mailman/listinfo/users>
>
> --
> Nathanaël Blanchet
>
> Supervision réseau
> Pôle Infrastrutures Informatiques
> 227 avenue Professeur-Jean-Louis-Viala
> 34193 MONTPELLIER CEDEX 5
> Tél. 33 (0)4 67 54 84 55
> Fax 33 (0)4 67 54 84 14
> blanchet at abes.fr <mailto:blanchet at abes.fr>
>
> _______________________________________________ Users mailing list
> Users at ovirt.org <mailto:Users at ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet at abes.fr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1317 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0010.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 24630 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 4050 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0011.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1422 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0012.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1318 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0013.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1359 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0014.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1276 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0015.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1378 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0016.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1399 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0017.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 1206 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0018.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/x-png
Size: 6808 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0019.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 18112 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161012/29b67d3c/attachment-0001.png>
More information about the Users
mailing list