Convert local storage domain to shared
--=_345563be-5719-4ca3-a875-90e8c94299c9 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Dear Users, We have an old ovirt3.5 install with a local and a shared clusters. Meanwhile we created a new data center, that based on 4.1 and it use only shared infrastructure. I would like to migrate an big VM from the old local datacenter to our new, but I don't have enough downtime. Is it possible to convert the old local storage to shared (by share via NFS) and attach that as new storage domain to the new cluster? I just want to import VM and copy (while running) with live storage migration function. I know, the official way for move vms between ovirt clusters is the export domain, but it has very big disks. What can I do? Thanks Tibor --=_345563be-5719-4ca3-a875-90e8c94299c9 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><body><div id=3D"zimbraEditorContainer" style=3D"font-family: arial, = helvetica, sans-serif; font-size: 12pt; color: #000000" class=3D"403"><div>= <br></div><div>Dear Users,</div><div><br data-mce-bogus=3D"1"></div><div>We= have an old ovirt3.5 install with a local and a shared clusters. Meanwhile= we created a new data center, that based on 4.1 and it use only shared inf= rastructure.</div><div>I would like to migrate an big VM from the old local= datacenter to our new, but I don't have enough downtime. </div><div><= br data-mce-bogus=3D"1"></div><div>Is it possible to convert the old local = storage to shared (by share via NFS) and attach that as new storage domain = to the new cluster?</div><div></div><div>I just want to import VM and copy = (while running) with live storage migration function.</div><div><br data-mc= e-bogus=3D"1"></div><div>I know, the official way for move vms between ovir= t clusters is the export domain, but it has very big disks.</div><div><br d= ata-mce-bogus=3D"1"></div><div>What can I do?</div><div><br data-mce-bogus= =3D"1"></div><div>Thanks </div><div><br data-mce-bogus=3D"1"></div><di= v>Tibor</div><div data-marker=3D"__SIG_PRE__"><p style=3D"font-family: 'Tim= es New Roman'; font-size: medium; margin: 0px;" data-mce-style=3D"font-fami= ly: 'Times New Roman'; font-size: medium; margin: 0px;"><strong><span style= =3D"font-size: medium;" data-mce-style=3D"font-size: medium;"><span style= =3D"color: #2d67b0;" data-mce-style=3D"color: #2d67b0;"></span></span></str= ong></p><p></p></div></div></body></html> --=_345563be-5719-4ca3-a875-90e8c94299c9--
On 11/29/2017 09:39 AM, Demeter Tibor wrote:
Dear Users,
We have an old ovirt3.5 install with a local and a shared clusters. Meanwhile we created a new data center, that based on 4.1 and it use only shared infrastructure. I would like to migrate an big VM from the old local datacenter to our new, but I don't have enough downtime.
Is it possible to convert the old local storage to shared (by share via NFS) and attach that as new storage domain to the new cluster? I just want to import VM and copy (while running) with live storage migration function.
I know, the official way for move vms between ovirt clusters is the export domain, but it has very big disks.
What can I do?
Just my opinion, but if you don't figure out a way to have occasional downtime, you'll probably pay the price with unplanned downtime eventually (and it could be painful). Define "large disks"? Terabytes? I know for a fact that if you don't have good network segmentation that live migrations of large disks can be very problematic. And I'm not talking about what you're wanting to do. I'm just talking about storage migration. We successfully migrated hundreds of VMs from a 3.4 to a 3.6 (on new blades and storage) last year over time using the NFS export domain method. If storage is the same across DC's, you might be able to shortcut this with minimal downtime, but I'm pretty sure there will be some downtime. I've seen large storage migrations render entire nodes offline (not nice) due to non-isolated paths or QoS.
Hi, Yes, I understand what do you talk about. It isn't too safe..:( We have terrabytes under that VM. I could make a downtime at most for eight hours (maybe), but meanwhile I have to copy 3 TB of vdisks. Firstly I need export (with a gigabit nic) to export domain, and back under 10gbe nic. I don't know how is enough this. Thanks Tibor ----- 2017. nov.. 29., 18:26, Christopher Cox ccox@endlessnow.com írta:
On 11/29/2017 09:39 AM, Demeter Tibor wrote:
Dear Users,
We have an old ovirt3.5 install with a local and a shared clusters. Meanwhile we created a new data center, that based on 4.1 and it use only shared infrastructure. I would like to migrate an big VM from the old local datacenter to our new, but I don't have enough downtime.
Is it possible to convert the old local storage to shared (by share via NFS) and attach that as new storage domain to the new cluster? I just want to import VM and copy (while running) with live storage migration function.
I know, the official way for move vms between ovirt clusters is the export domain, but it has very big disks.
What can I do?
Just my opinion, but if you don't figure out a way to have occasional downtime, you'll probably pay the price with unplanned downtime eventually (and it could be painful).
Define "large disks"? Terabytes?
I know for a fact that if you don't have good network segmentation that live migrations of large disks can be very problematic. And I'm not talking about what you're wanting to do. I'm just talking about storage migration.
We successfully migrated hundreds of VMs from a 3.4 to a 3.6 (on new blades and storage) last year over time using the NFS export domain method.
If storage is the same across DC's, you might be able to shortcut this with minimal downtime, but I'm pretty sure there will be some downtime.
I've seen large storage migrations render entire nodes offline (not nice) due to non-isolated paths or QoS.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Wed, Nov 29, 2017 at 6:49 PM, Demeter Tibor <tdemeter@itsmart.hu> wrote:
Hi,
Yes, I understand what do you talk about. It isn't too safe..:( We have terrabytes under that VM. I could make a downtime at most for eight hours (maybe), but meanwhile I have to copy 3 TB of vdisks. Firstly I need export (with a gigabit nic) to export domain, and back under 10gbe nic. I don't know how is enough this.
Thanks
Tibor
Hi Tibor, I'm in shortage of time these days, but I have to admit your problem was so intriguing that I couldn't resist and I decided to try and reproduce it. All happened on my laptop with Fedora 26 (time to upgrade? not enough time... ;-) So this is the test environment of all vms inside virt-manager: 1) Create 3.5.6 environment - CentOS 6.6 VM (this was the iso I had at hand...) with hostname c6engine35 where I installed oVirt 3.5.6 as engine - CentOS 6.6 VM with hostname c6rhv35 (sorry for the rhv in the name but these weeks I'm also working on it so it came out quite naturally...) were I installed the Hypervisor of 3.5.6 repo I created a local DC on top of a directory of the hypervisor (/ldomain) I created a CentOS 6.6 VM in this storage domain with a 4Gb disk 2) Detach the local domain from DC HERE YOUR THEORETICAL DOWNTIME BEGINS To do so I powered off the test VM and created a fake further local domain based on another directory of c6rhv35 Then put into maintenance the local domain to be imported in 4.1 The fake local domain becomes the master. Detach the local domain. 3) Create 4.1.7 environment (in your case it is already there..) - CentOS 7.4 VM with hostname c7engine41 where I installed oVirt 4.1.7 as engine - CentOS 7.4 VM with hostname c7rhv41 were I installed the Hypervisor of 4.1.7 repo I created a shared DC NFSDC with a cluster NFSCL To speed things I exported a directory from the engine and used it to create an NFS storage domain (DATANFS) for the 4.1 host and activated it 4) Shutdown 3.5 environment and start/configure the 3.5 hypervisor to export its previously local storage domain directory Start c6rhv35 in single user mode chkconfig service_name off for this service_name: ebtables ip6tables iptables libvirt-guests libvirtd momd numad sanlock supervdsmd vdsmd wdmd reboot create an entry in /etc/exports /ldomain c7rhv41.localdomain.local(rw) service nfs start set up accordingly the /etc/hosts of the servers involved so that all know all... 5) import domain in 4.1 Select Storage -> Import domain and put c6rhv35.localdomain.local:/ldomain You will get a warning about it being already part of another DC: https://drive.google.com/file/d/1HjFZhW6fCkasPak0jQH5k49Bdsg1NLSN/view?usp=s... Approve operation and you arrive here: https://drive.google.com/file/d/10d1ea0TbPCZhoaAf7br5IVqnvZx0LzSu/view?usp=s... Activate the domain and you arrive here: https://drive.google.com/file/d/1-4sMfVVj5WyaglPI8zhWUsdJqkVkxzAT/view?usp=s... Now you can proceed importing your VMs; in my case only the testvm Select he imported storage domain and then the "VM Import" tab; select the VM and "Import": https://drive.google.com/file/d/18yjPvoHjTw6mOhUrlHJ2RpsdPph4qBxL/view?usp=s... https://drive.google.com/file/d/1CrCzVUYC3vI4aQ2ly83b3uAQ3QQh1xhm/view?usp=s... Note that it is an immediate operation, and not depending on the size of the disks of the VM itself At the end you get your VM imported; here details: https://drive.google.com/file/d/1W00TpIKAQ7cWUit_tLIQkm30wj5j56AN/view?usp=s... https://drive.google.com/file/d/1qq7sZV2vwapRRdjbi21Z43OOBM0m2NuY/view?usp=s... While you import, you can then gradually start your VMs, so that your downtime becomes partial and not total https://drive.google.com/file/d/1kwrzSJBXISC0wBTtZIpdh3yJdA0g3G0A/view?usp=s... When you have started all your imported VMs, YOUR THEORETICAL DOWNTIME ENDS Your VMS are now running on your old local storage, exported from your old 3.5 host to your new 4.1 hosts via NFS You can now execute live storage migration of your disks one by one to the desired 4.1 storage domain: https://drive.google.com/file/d/1p6OOgDBbOFFGgy3uuWT-V8VnCMaxk4iP/view?usp=s... and at the end of the move https://drive.google.com/file/d/1dUuKQQxI0r4Bhz-N0TmRcsnjwQl1bwU6/view?usp=s... Obviously there are many caveats in a real environment such as: - actual oVirt origin and target version could differ from mine and behavior be different - network visibility between the two oVirt environments - layout of the logical networks of the two oVirt environments: when you import you could need to change logical network and have conflicting MACS: in my test scenario it was all on ovirtmgmt with the same macs range - live storage migration of TB of disks.. not tested yet (by me at least).... - other things that don't come to mind right now.... HIH, Gianluca
Hi Gianluca, Thanks for another great post! Any chance you'd like to convert it to a blog/article on ovirt.org? On Fri, Dec 1, 2017 at 2:25 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Wed, Nov 29, 2017 at 6:49 PM, Demeter Tibor <tdemeter@itsmart.hu> wrote:
Hi,
Yes, I understand what do you talk about. It isn't too safe..:( We have terrabytes under that VM. I could make a downtime at most for eight hours (maybe), but meanwhile I have to copy 3 TB of vdisks. Firstly I need export (with a gigabit nic) to export domain, and back under 10gbe nic. I don't know how is enough this.
Thanks
Tibor
Hi Tibor, I'm in shortage of time these days, but I have to admit your problem was so intriguing that I couldn't resist and I decided to try and reproduce it. All happened on my laptop with Fedora 26 (time to upgrade? not enough time... ;-)
So this is the test environment of all vms inside virt-manager:
1) Create 3.5.6 environment
- CentOS 6.6 VM (this was the iso I had at hand...) with hostname c6engine35 where I installed oVirt 3.5.6 as engine - CentOS 6.6 VM with hostname c6rhv35 (sorry for the rhv in the name but these weeks I'm also working on it so it came out quite naturally...) were I installed the Hypervisor of 3.5.6 repo
I created a local DC on top of a directory of the hypervisor (/ldomain) I created a CentOS 6.6 VM in this storage domain with a 4Gb disk
2) Detach the local domain from DC
HERE YOUR THEORETICAL DOWNTIME BEGINS
To do so I powered off the test VM and created a fake further local domain based on another directory of c6rhv35 Then put into maintenance the local domain to be imported in 4.1 The fake local domain becomes the master. Detach the local domain.
3) Create 4.1.7 environment (in your case it is already there..) - CentOS 7.4 VM with hostname c7engine41 where I installed oVirt 4.1.7 as engine - CentOS 7.4 VM with hostname c7rhv41 were I installed the Hypervisor of 4.1.7 repo
I created a shared DC NFSDC with a cluster NFSCL To speed things I exported a directory from the engine and used it to create an NFS storage domain (DATANFS) for the 4.1 host and activated it
4) Shutdown 3.5 environment and start/configure the 3.5 hypervisor to export its previously local storage domain directory
Start c6rhv35 in single user mode chkconfig service_name off
for this service_name: ebtables ip6tables iptables libvirt-guests libvirtd momd numad sanlock supervdsmd vdsmd wdmd
reboot create an entry in /etc/exports
/ldomain c7rhv41.localdomain.local(rw)
service nfs start
set up accordingly the /etc/hosts of the servers involved so that all know all...
5) import domain in 4.1 Select Storage -> Import domain and put
c6rhv35.localdomain.local:/ldomain
You will get a warning about it being already part of another DC: https://drive.google.com/file/d/1HjFZhW6fCkasPak0jQH5k49Bdsg1NLSN/view?usp=s...
Approve operation and you arrive here: https://drive.google.com/file/d/10d1ea0TbPCZhoaAf7br5IVqnvZx0LzSu/view?usp=s...
Activate the domain and you arrive here: https://drive.google.com/file/d/1-4sMfVVj5WyaglPI8zhWUsdJqkVkxzAT/view?usp=s...
Now you can proceed importing your VMs; in my case only the testvm Select he imported storage domain and then the "VM Import" tab; select the VM and "Import":
https://drive.google.com/file/d/18yjPvoHjTw6mOhUrlHJ2RpsdPph4qBxL/view?usp=s...
https://drive.google.com/file/d/1CrCzVUYC3vI4aQ2ly83b3uAQ3QQh1xhm/view?usp=s...
Note that it is an immediate operation, and not depending on the size of the disks of the VM itself At the end you get your VM imported; here details:
https://drive.google.com/file/d/1W00TpIKAQ7cWUit_tLIQkm30wj5j56AN/view?usp=s... https://drive.google.com/file/d/1qq7sZV2vwapRRdjbi21Z43OOBM0m2NuY/view?usp=s...
While you import, you can then gradually start your VMs, so that your downtime becomes partial and not total https://drive.google.com/file/d/1kwrzSJBXISC0wBTtZIpdh3yJdA0g3G0A/view?usp=s...
When you have started all your imported VMs, YOUR THEORETICAL DOWNTIME ENDS
Your VMS are now running on your old local storage, exported from your old 3.5 host to your new 4.1 hosts via NFS
You can now execute live storage migration of your disks one by one to the desired 4.1 storage domain: https://drive.google.com/file/d/1p6OOgDBbOFFGgy3uuWT-V8VnCMaxk4iP/view?usp=s...
and at the end of the move https://drive.google.com/file/d/1dUuKQQxI0r4Bhz-N0TmRcsnjwQl1bwU6/view?usp=s...
Obviously there are many caveats in a real environment such as:
- actual oVirt origin and target version could differ from mine and behavior be different - network visibility between the two oVirt environments - layout of the logical networks of the two oVirt environments: when you import you could need to change logical network and have conflicting MACS: in my test scenario it was all on ovirtmgmt with the same macs range - live storage migration of TB of disks.. not tested yet (by me at least).... - other things that don't come to mind right now....
HIH, Gianluca
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
Il 03 Dic 2017 07:42, "Yedidyah Bar David" <didi@redhat.com> ha scritto: Hi Gianluca, Thanks for another great post! Any chance you'd like to convert it to a blog/article on ovirt.org? Thanks for your compliments Didi, much appreciated. I can try to do something at the end of the week
participants (4)
-
Christopher Cox -
Demeter Tibor -
Gianluca Cecchi -
Yedidyah Bar David