Manually starting VMs via vdsClient (HE offline)
Hi Guys, My Hosted-Engine has failed & it looks like the easiest solution will be to install a new one. Now before I try to re-add the old hosts (still running the guest VMs) & import the storage domain into the new engine, in case things don't go to plan, I want to make sure I'm able to bring up the guests on the hosts manually. The problem is vdsClient is giving me an "Unexpected exception", without much more info as to why it's failing. Any idea? [root@v0 ~]# vdsClient -s 0 list table | grep georep 9d1c3fef-498e-4c20-b124-01364d4d45a8 30455 georep-proxy Down [root@v0 ~]# vdsClient -s 0 continue 9d1c3fef-498e-4c20-b124-01364d4d45a8 Unexpected exception /var/log/vdsm/vdsm.log periodic/1063::WARNING::2017-02-08 17:57:52,532::periodic::276::virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['65c9807c-7216-40b3-927c-5fd93bbd42ba', u'9d1c3fef-498e-4c20-b124-01364d4d45a8'] Cheers, -- Doug
On Wed, Feb 8, 2017 at 4:07 PM, Doug Ingham <dougti@gmail.com> wrote:
Hi Guys, My Hosted-Engine has failed & it looks like the easiest solution will be to install a new one. Now before I try to re-add the old hosts (still running the guest VMs) & import the storage domain into the new engine, in case things don't go to plan, I want to make sure I'm able to bring up the guests on the hosts manually.
The problem is vdsClient is giving me an "Unexpected exception", without much more info as to why it's failing.
Any idea?
[root@v0 ~]# vdsClient -s 0 list table | grep georep 9d1c3fef-498e-4c20-b124-01364d4d45a8 30455 georep-proxy Down
[root@v0 ~]# vdsClient -s 0 continue 9d1c3fef-498e-4c20-b124-01364d4d45a8 Unexpected exception
/var/log/vdsm/vdsm.log periodic/1063::WARNING::2017-02-08 17:57:52,532::periodic::276:: virt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['65c9807c-7216-40b3-927c-5fd93bbd42ba', u'9d1c3fef-498e-4c20-b124-01364d4d45a8']
continue meane un-pause, not "start from a stopped state" now having said that, if you expect the VMs not to be able to start after you rebuild the engine and the VMs exist on the hosts, I'd collect a virsh -r dumpxml VMNAME for each - that way you have the disks in use, and all the VM configuration in a file, and with some minor LVM manipulation you'll be able to start the VM via virsh
Cheers, -- Doug
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Dan, On 8 February 2017 at 18:10, Dan Yasny <dyasny@gmail.com> wrote:
On Wed, Feb 8, 2017 at 4:07 PM, Doug Ingham <dougti@gmail.com> wrote:
Hi Guys, My Hosted-Engine has failed & it looks like the easiest solution will be to install a new one. Now before I try to re-add the old hosts (still running the guest VMs) & import the storage domain into the new engine, in case things don't go to plan, I want to make sure I'm able to bring up the guests on the hosts manually.
The problem is vdsClient is giving me an "Unexpected exception", without much more info as to why it's failing.
Any idea?
[root@v0 ~]# vdsClient -s 0 list table | grep georep 9d1c3fef-498e-4c20-b124-01364d4d45a8 30455 georep-proxy Down
[root@v0 ~]# vdsClient -s 0 continue 9d1c3fef-498e-4c20-b124-01364d4d45a8 Unexpected exception
/var/log/vdsm/vdsm.log periodic/1063::WARNING::2017-02-08 17:57:52,532::periodic::276::v irt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['65c9807c-7216-40b3-927c-5fd93bbd42ba', u'9d1c3fef-498e-4c20-b124-0136 4d4d45a8']
continue meane un-pause, not "start from a stopped state"
I search the manual for start/init/resume syntax, and "continue" was the closest thing I found.
now having said that, if you expect the VMs not to be able to start after you rebuild the engine and the VMs exist on the hosts, I'd collect a virsh -r dumpxml VMNAME for each - that way you have the disks in use, and all the VM configuration in a file, and with some minor LVM manipulation you'll be able to start the VM via virsh
My main concern is that I might have to halt the VMs or VDSM services for some reason when trying to migrate to the new engine. I just want to make sure that no matter what happens, I can still get the VMs back online. I'm still getting myself acquainted with virsh/vdsClient. Could you provide any insight into what I'd have to do to restart the guests manually? Thanks, -- Doug
On Wed, Feb 8, 2017 at 4:18 PM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:10, Dan Yasny <dyasny@gmail.com> wrote:
On Wed, Feb 8, 2017 at 4:07 PM, Doug Ingham <dougti@gmail.com> wrote:
Hi Guys, My Hosted-Engine has failed & it looks like the easiest solution will be to install a new one. Now before I try to re-add the old hosts (still running the guest VMs) & import the storage domain into the new engine, in case things don't go to plan, I want to make sure I'm able to bring up the guests on the hosts manually.
The problem is vdsClient is giving me an "Unexpected exception", without much more info as to why it's failing.
Any idea?
[root@v0 ~]# vdsClient -s 0 list table | grep georep 9d1c3fef-498e-4c20-b124-01364d4d45a8 30455 georep-proxy Down
[root@v0 ~]# vdsClient -s 0 continue 9d1c3fef-498e-4c20-b124-01364d 4d45a8 Unexpected exception
/var/log/vdsm/vdsm.log periodic/1063::WARNING::2017-02-08 17:57:52,532::periodic::276::v irt.periodic.VmDispatcher::(__call__) could not run <class 'vdsm.virt.periodic.DriveWatermarkMonitor'> on ['65c9807c-7216-40b3-927c-5fd93bbd42ba', u'9d1c3fef-498e-4c20-b124-0136 4d4d45a8']
continue meane un-pause, not "start from a stopped state"
I search the manual for start/init/resume syntax, and "continue" was the closest thing I found.
I don't have a 4.x vdsm handy, but on 3.6 the verb is "create". With a LOT of params of course.
now having said that, if you expect the VMs not to be able to start after you rebuild the engine and the VMs exist on the hosts, I'd collect a virsh -r dumpxml VMNAME for each - that way you have the disks in use, and all the VM configuration in a file, and with some minor LVM manipulation you'll be able to start the VM via virsh
My main concern is that I might have to halt the VMs or VDSM services for some reason when trying to migrate to the new engine. I just want to make sure that no matter what happens, I can still get the VMs back online.
The way oVirt works is very much tied into the engine DB. When you click "start" on a VM, the engine will query the DB, pull out the VM details (CPU config, disks, RAM etc), pick a suitable host, enable the hosts' access to the disks on the storage domain, generate the libvirt domxml for the VM (the file you'd get from virsh dumpxml) and start the VM according to the generated XML. With vdsClient and without the engine DB you'll be missing all those details the database provides, while my way, with the XML, they are all already in place, populated by the engine when it was still alive.
I'm still getting myself acquainted with virsh/vdsClient. Could you provide any insight into what I'd have to do to restart the guests manually?
see above. But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
Thanks, -- Doug
Hi Dan, On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host... -- Doug
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all. So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
-- Doug
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts. See the top section of this page: http://www.ovirt.org/documentation/self-hosted/chap-Backing_up_and_Restoring... It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain. -- Doug
On Thu, Feb 9, 2017 at 11:00 AM, Doug Ingham <dougti@gmail.com> wrote:
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.
See the top section of this page: http://www.ovirt.org/documentation/self-hosted/ chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment
It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.
Yeah, that's another downside to using HE I suppose. Though maybe you don't need to be as paranoid as I am and do backups out of hours once per day, which will be much less disruptive. This is pretty easy to script with the SDK - just connect to the API with a python script and deactivate a host. Once it's in maintenance, run engine-backup and pull the backup out, then activate the host again.
-- Doug
On Thu, Feb 9, 2017 at 6:00 PM, Doug Ingham <dougti@gmail.com> wrote:
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.
See the top section of this page: http://www.ovirt.org/documentation/self-hosted/ chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment
It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.
I don't see the point in an hourly backup. Of what? The DB? The VM? What storage will it be based on? I suggest revising the strategy.
-- Doug
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Thu, Feb 9, 2017 at 1:48 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Thu, Feb 9, 2017 at 6:00 PM, Doug Ingham <dougti@gmail.com> wrote:
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.
See the top section of this page: http://www.ovirt.org/documentation/self-hosted/chap-Backing_ up_and_Restoring_an_EL-Based_Self-Hosted_Environment
It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.
I don't see the point in an hourly backup. Of what? The DB? The VM? What storage will it be based on? I suggest revising the strategy.
That's pretty much what I said :)
-- Doug
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 9 February 2017 at 15:48, Yaniv Kaul <ykaul@redhat.com> wrote:
On Thu, Feb 9, 2017 at 6:00 PM, Doug Ingham <dougti@gmail.com> wrote:
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.
See the top section of this page: http://www.ovirt.org/documentation/self-hosted/chap-Backing_ up_and_Restoring_an_EL-Based_Self-Hosted_Environment
It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.
I don't see the point in an hourly backup. Of what? The DB? The VM? What storage will it be based on? I suggest revising the strategy.
Um, that wasn't my suggestion. To be honest, a fortnight's worth of daily engine-backups & snapshots of the engine volume should suffice for us. It's a "Hyperconverged" setup using gluster storage on the compute nodes. -- Doug
This is a multi-part message in MIME format. --------------D69F31E133DB61D4D95110A9 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable Le 09/02/2017 =E0 19:48, Yaniv Kaul a =E9crit :
On Thu, Feb 9, 2017 at 6:00 PM, Doug Ingham <dougti@gmail.com=20 <mailto:dougti@gmail.com>> wrote:
On 9 February 2017 at 12:03, Dan Yasny <dyasny@gmail.com <mailto:dyasny@gmail.com>> wrote:
On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <dougti@gmail.com <mailto:dougti@gmail.com>> wrote:
Hi Dan,
On 8 February 2017 at 18:26, Dan Yasny <dyasny@gmail.com <mailto:dyasny@gmail.com>> wrote:
But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.
If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM? I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
I'd be interested to know what type of scale needs a real hardware for=20 engine, rather 100 vms or 1000 vms? it may be about the hosts number?
So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night
This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC. I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.
See the top section of this page: http://www.ovirt.org/documentation/self-hosted/chap-Backing_up_and_=
Restoring_an_EL-Based_Self-Hosted_Environment
<http://www.ovirt.org/documentation/self-hosted/chap-Backing_up_and=
_Restoring_an_EL-Based_Self-Hosted_Environment>
It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.
I don't see the point in an hourly backup. Of what? The DB? The VM?=20 What storage will it be based on? I suggest revising the strategy.
--=20 Doug
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--=20 Nathana=EBl Blanchet Supervision r=E9seau P=F4le Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 =09 T=E9l. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet@abes.fr --------------D69F31E133DB61D4D95110A9 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable <html> <head> <meta content=3D"text/html; charset=3Dwindows-1252" http-equiv=3D"Content-Type"> </head> <body bgcolor=3D"#FFFFFF" text=3D"#000000"> <p><br> </p> <br> <div class=3D"moz-cite-prefix">Le 09/02/2017 =E0 19:48, Yaniv Kaul a =E9crit=A0:<br> </div> <blockquote cite=3D"mid:CAJgorsZxmr=3Dargj2kVkONNE4RFKP0z35r2AEANOKf-+40ZfM8w@mail.gm= ail.com" type=3D"cite"> <div dir=3D"ltr"><br> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote">On Thu, Feb 9, 2017 at 6:00 PM, Doug Ingham <span dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:dougti@gmail.com" target=3D"_blank">dougti= @gmail.com</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div dir=3D"ltr"><br> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote"><span class=3D"">On 9 Februa= ry 2017 at 12:03, Dan Yasny <span dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:dyasny@gmail.com" target=3D"_bla= nk">dyasny@gmail.com</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote"><span class=3D"m_5385279511923747685gmail-">On Thu, Feb 9, 2017 at 9:55 AM, Doug Ingham <span dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:dougti@gmail.com" target=3D"_blank">dougti@gmail.com</a=
></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr">Hi Dan,<br> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote"><span>On 8 February 2017 at 18:26, Dan Yasny <span dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:dyasny@gmail= .com" target=3D"_blank">dyasny@gm= ail.com</a>></span> wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"><span></span=
<div class=3D"gmail_extra"> <div class=3D"gmail_quote= "><span> <div><br> </div> </span> <div>But seriously, above all, I'd recommend you backup the engine (it comes with a utility) often and well. I do it via cron every hour in production, keeping a rotation of hourly and daily backups, just in case. It doesn't take much space or resources, but it's more than just best practice - that database is the summary of the entire setup.</div> <br clear=3D"all"> </div> </div> </div> </blockquote> <div><br> </div> </span> <div>If you don't mind, may I ask what process you use for backing up your engine? If you use HE, do you keep one server dedicated to just that VM?<br> </div> <div>I've not had that particular issue in the restore process yet, however I read that it's recommended the HE host is free of virtual load before the backup takes place. And as they need to be done frequently, I'm reading that as a dedicated host...<spa= n class=3D"m_5385279511923747685gmail-m_-816755705011915139HOEnZb"><font color=3D"#888888"><br> </font></span></div> </div> <span class=3D"m_5385279511923747685gma= il-m_-816755705011915139HOEnZb"><font color=3D"#888888"><br> </font></span></div> </div> </blockquote> <div><br> </div> </span> <div>If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all. <br> </div> </div> </div> </div> </blockquote> </span></div> </div> </div> </blockquote> </div> </div> </div> </blockquote> I'd be interested to know what type of scale needs a real hardware for engine, rather 100 vms or 1000 vms? it may be about the hosts number?<br> <blockquote cite=3D"mid:CAJgorsZxmr=3Dargj2kVkONNE4RFKP0z35r2AEANOKf-+40ZfM8w@mail.gm= ail.com" type=3D"cite"> <div dir=3D"ltr"> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"><span class=3D""> <blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <div><br> </div> <div>So as I said, all I do is add the engine-backup command to cron on the engine, and then my backup server comes in and pulls out the files via scp, also through cron. Nothing fancy really, but it lets me sleep at night</div> </div> </div> </div> </blockquote> <div><br> </div> </span> <div>This particular project has 10 new maxed out servers to back it, and I don't see it outgrowing that for at least a year or so. It's hardly a full DC.<br> I presume the DB will become the heaviest part of the load, and I'm already planning a separate high I/O environment for dedicated HA DB hosts.<br> <br> </div> <div>See the top section of this page:</div> </div> <a moz-do-not-send=3D"true" href=3D"http://www.ovirt.org/documentation/self-hosted/chap-Backing_up_an= d_Restoring_an_EL-Based_Self-Hosted_Environment" target=3D"_blank">http://www.ovirt.org/<wbr>documenta= tion/self-hosted/<wbr>chap-Backing_up_and_Restoring_<wbr>an_EL-Based_Self= -Hosted_<wbr>Environment</a><br> <br> </div> <div class=3D"gmail_extra">It seems that I'll always have to keep at least one host free to be able to avoid restore problems. If not, and I were to keep hourly backups, then migrating VMs off the host every hour would just be a pain.</div> </div> </blockquote> <div><br> </div> <div>I don't see the point in an hourly backup. Of what? The DB? The VM? What storage will it be based on?</div> <div>I suggest revising the strategy.</div> <div>=A0</div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div dir=3D"ltr"> <div class=3D"gmail_extra"><span class=3D"HOEnZb"><font color=3D"#888888"><br> </font></span></div> <span class=3D"HOEnZb"><font color=3D"#888888"> <div class=3D"gmail_extra"><br> -- <br> <div class=3D"m_5385279511923747685gmail_signature"=
Doug</div> </div> </font></span></div> <br> ______________________________<wbr>_________________<br> Users mailing list<br> <a moz-do-not-send=3D"true" href=3D"mailto:Users@ovirt.org"= Users@ovirt.org</a><br> <a moz-do-not-send=3D"true" href=3D"http://lists.ovirt.org/mailman/listinfo/users" rel=3D"noreferrer" target=3D"_blank">http://lists.ovirt.o= rg/<wbr>mailman/listinfo/users</a><br> <br> </blockquote> </div> <br> </div> </div> <br> <fieldset class=3D"mimeAttachmentHeader"></fieldset> <br> <pre wrap=3D"">_______________________________________________ Users mailing list <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:Users@ovirt.org">Use= rs@ovirt.org</a> <a class=3D"moz-txt-link-freetext" href=3D"http://lists.ovirt.org/mailman= /listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> <pre class=3D"moz-signature" cols=3D"72">--=20 Nathana=EBl Blanchet
Supervision r=E9seau P=F4le Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 =09 T=E9l. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:blanchet@abes.fr">bl= anchet@abes.fr</a> </pre> </body> </html> --------------D69F31E133DB61D4D95110A9--
On Fri, Feb 10, 2017 at 8:20 AM, Nathanaël Blanchet <blanchet@abes.fr> wrote:
If you use a dedicated host, you might as well abandon self hosted. HE is nice for small setups with the HA built in for extra fun, but once you scale, it might not be able to cope and you'll need real hardware. You're running a heavy-ish java engine plus two databases after all.
I'd be interested to know what type of scale needs a real hardware for engine, rather 100 vms or 1000 vms? it may be about the hosts number?
YMMV of course. But it only stands to reason, that a certain amount of load is going to overwhelm a single VM. VMWare also have a scale limitation on the VM based vcenter btw
participants (4)
-
Dan Yasny -
Doug Ingham -
Nathanaël Blanchet -
Yaniv Kaul