7:35 a.m.
Dear all,
I have an Ovirt cluster with 3 Dell R7525 nodes and about 70 virtual
machines. I have been using ovirtbackup (python) for a long time to save
the VMs.
Unfortunately it has always been a very slow process that with time and
the ever-increasing size of the VMs has become unusable.
The oVirt cluster has two storage domains. "DATA" is a the fibre channel
type (8Gbs) with 18TB and an EXPORT domain NFS managed by a
dual-processor server HPE Proliant with 4x1Gbs.
I'll give you an example. I have a vm with four vdisk for a 1.8TB and
these are the steps performed to backup this VM.
As you will see it takes many hours to get the clone from the snapshot
and then many hours to copy the clone to the NFS storage.
How can I reduce the time needed to save ?
Thank you !
Best Regards
Enrico
-----------------------------------------------------------------
Feb 2, 2025, 6:26:47 AM
Snapshot 'Snapshot for backup script' creation for VM 'GRAYLOG-9' was
initiated by admin@internal-authz.
45
admin@internal-authz
infn-vm10.management
GRAYLOG-9
Blank
INFNPG
DELL
70bbb81b-ed44-4700-839a-96c1df8daeea
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 6:27:22 AM
Snapshot 'Snapshot for backup script' creation for VM 'GRAYLOG-9' has
been completed.
68
admin@internal-authz
infn-vm10.management
GRAYLOG-9
Blank
INFNPG
DELL
70bbb81b-ed44-4700-839a-96c1df8daeea
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 6:27:37 AM
VM GRAYLOG-9_BCK_020244 creation was initiated by admin@internal-authz.
37
admin@internal-authz
GRAYLOG-9_BCK_020244
Blank
INFNPG
DATA_FC_P2050
DELL
edde40d9-2a26-4d88-8a02-bd928c1ded4a
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 1:59:30 PM
VM GRAYLOG-9_BCK_020244 creation has been completed.
53
admin@internal-authz
GRAYLOG-9_BCK_020244
Blank
INFNPG
DATA_FC_P2050
DELL
edde40d9-2a26-4d88-8a02-bd928c1ded4a
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 1:59:34 PM
Snapshot 'Snapshot for backup script' deletion for VM 'GRAYLOG-9' was
initiated by admin@internal-authz.
342
admin@internal-authz
infn-vm10.management
GRAYLOG-9
Blank
INFNPG
DELL
fcfc6291-a55f-431a-8b4c-87cb8e3615ff
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 2:04:00 PM
Snapshot 'Snapshot for backup script' deletion for VM 'GRAYLOG-9' has
been completed.
356
admin@internal-authz
infn-vm10.management
GRAYLOG-9
Blank
INFNPG
DELL
fcfc6291-a55f-431a-8b4c-87cb8e3615ff
oVirt
-----------------------------------------------------------------
Feb 2, 2025, 2:04:08 PM
Starting export Vm GRAYLOG-9_BCK_020244 to VMS_EXPORT
1162
admin@internal-authz
GRAYLOG-9_BCK_020244
INFNPG
VMS_EXPORT
DELL
b964e3b3-df13-4b82-8128-00984eea380b
oVirt
-----------------------------------------------------------------
Feb 4, 2025, 5:27:50 AM
Vm GRAYLOG-9_BCK_020244 was exported successfully to VMS_EXPORT
1150
admin@internal-authz
GRAYLOG-9_BCK_020244
INFNPG
VMS_EXPORT
DELL
b964e3b3-df13-4b82-8128-00984eea380b
oVirt
-----------------------------------------------------------------
Feb 4, 2025, 5:27:53 AM
VM GRAYLOG-9_BCK_020244 configuration was updated by admin@internal-authz.
35
admin@internal-authz
GRAYLOG-9_BCK_020244
Blank
INFNPG
DELL
44480c68-10ad-4660-bd9b-297043210503
oVirt
-----------------------------------------------------------------
Feb 4, 2025, 5:28:00 AM
VM GRAYLOG-9_BCK_020244 was successfully removed by admin@internal-authz.
113
admin@internal-authz
GRAYLOG-9_BCK_020244
Blank
INFNPG
DELL
d141ad60-6cf8-4e77-b438-26b5481f4256
oVirt
-----------------------------------------------------------------
8:24 a.m.
Same here.
We happily use storware vprotect, performing full backups during the
weekend and a couple of incremental backups every other day (for some VM
4 backups a day), for every VM living in our cluster.
Full backup is quite slow (obviously proportional to VM size), but
incremental backup is a matter of (few) minutes.
An export domain is not the right choice for backups, I think you should
look at it as a storage to move around VMs (between datacenters) when
you need.
HTH,
gc
On 04/02/2025 15:11, Jayme wrote:
9 a.m.
Dear all
What do you think about the VM clone creation which happens very slowly
even though the domain data is fibre channel ?
with vproect it would be faster because it does not use this mechanism
to make backups?
Enrico
Il 04/02/25 15:24, Giulio Casella ha scritto:
--
__________________________________________________________________________
Enrico Becchetti Servizio di Calcolo e Reti
Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
Via Pascoli,c/o Dipartimento di Fisica 06123 Perugia (ITALY)
Phone:+39 075 5852777 Mobile: +39 075 9696225
FAX: +39 075 5847296 Microsoft Teams: becchett(a)infn.it
Mail: Enrico.Becchetti<at>pg.infn.it Skype:enrico_becchetti
Pagina web personale: https://www.pg.infn.it/home/enrico-becchetti
_________________________________________________________________________
10:33 a.m.
Correct, usually backups are performed through the snapshots mechanism,
not cloning the VM. Ovirt implements also other export mechanism for
backups (for example CBT, changed block tracking) that minimize the
amount of data to be transfered.
On 04/02/2025 16:00, Enrico Becchetti via Users wrote:
--
Giulio Casella giulio at di.unimi.it
System and network architect
Computer Science Dept. - University of Milano
6:43 a.m.
Dear Sandro dear all
if I understand correctly, the sequence for backing up a VM in oVirt
environment is as follows:
1 A snapshot is created (data domain Fibre Channel).
2 A clone based on the snapshot is created (data domain Fibre
Channel).
3 The clone is exported from the "data domain Fibre Channel" to
the EXPORT Domain (NFS).
Both step 2 and step 3 are extremely slow, considering that the data
domain storage is Fibre Channel with
an 8Gbps link and that the NFS server has a 4x1Gbps Ethernet network.
For example, looking at what VDSM server during step two, I see the
following:
/usr/bin/qemu-img convert -p -t none -T none -f qcow2
/rhev/data-center/mnt/blockSD/——id—— -O raw -W
/rhev/data-center/mnt/blockSD/——id——
creating a clone of a VM with 100GB of disk space takes approximately 55
minutes.
Checking the processes on the same node during step three, I see the
following:
/usr/bin/qemu-img convert -p -t none -T none -f raw
/rhev/data-center/mnt/blockSD/——id—— -O raw -o preallocation=falloc
/rhev/data-center/mnt/infn-vm05.management:_ovirt_vms/—id——
it takes about 25 minutes.
Is it possible to optimize the qemu-img convert process by modifying
some parameters using the oVirt setup ?
I am currently using oVirt 4.3.10. If I'll make an update 4.5 are there
any improvements ?
Thanks
Enrico
Il 04/02/25 17:33, Giulio Casella ha scritto:
--
__________________________________________________________________________
Enrico Becchetti Servizio di Calcolo e Reti
Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
Via Pascoli,c/o Dipartimento di Fisica 06123 Perugia (ITALY)
Phone:+39 075 5852777 Mobile: +39 075 9696225
FAX: +39 075 5847296 Microsoft Teams: becchett(a)infn.it
Mail: Enrico.Becchetti<at>pg.infn.it Skype:enrico_becchetti
Pagina web personale: https://www.pg.infn.it/home/enrico-becchetti
_________________________________________________________________________
127
days inactive
129
days old
5 comments
3 participants
participants (3)
-
Enrico Becchetti -
Giulio Casella -
Jayme