[ovirt-users] Autostart VMS

Fri Apr 8 22:06:25 UTC 2016

On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholcomb at l1049h.com> wrote:
> On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb at l1049h.com>
> wrote:
>
> On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb at l1049h.com>
> wrote:
>
> On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
>
> Hi,
>
>
> I set highly available on, did not pin to any host, and also set the
> watchdog which should reset if they go down but I'm not sure that will start
> them if the host comes up and the VMs are not running.  I'll look at the CLI
> first.
>
>
> The engine will try to keep the VM running. So if one host goes down,
> it will restart the VM on some other host automatically. We will also
> migrate the VM (or some other to free resources) when the current host
> gets too loaded. We do not require any migration addons, it just
> works. But of course we have usually more hosts in a cluster to make
> this possible.
>
> I do not really remember what happens when all hosts are restarted
> (power outage) though as that is quite special case.
>
> Regards
>
> --
> Martin Sivak
> SLA / oVirt
>
>
>
> Thanks.  I only have one host so who knows what will happen.  I'm working on
> a script that will basically emulate what VMware does - start VMS in a given
> order at startup of the host/engine.  I'll also file a feature request.
>
>
> Why do you care about the order?
>
> Isn't it enough to restart all the vms after a host was restarted?
>
> Nir
>
>
> Does the Engine start all VMs after the host is powered up and the engine
> running if I set a watchdog on a VM?  I know it will try to restart the VM
> if it goes down once oVirt is up and running.   If it will start the VMs at
> reboot time that helps.
>
>
> I think only HA vms are restarted. Adding Michal to add more info on this.
>
>
> The reason for the order is that you need some servers such as DNS, file
> servers, up first before other systems can resolve addresses or mount
> shares.  For Windows you need domain controllers running before the other
> windows systems that are part of the domain.  For applications such as Lotus
> Notes the servers had to come up in the correct order.
>
>
> Lets say you have a way to order the vms when some vms are down.
>
> What will happen to when dns, file servers or domain controller will crash?
> Do you have to restart all the vms depending on them or they can
> recover and detect that vm they depend on were restarted?
>
> Seems that what you need is a systemd for your data center - every
> host define the host it depends on, and the system generate the
> correct order to start the vms, starting vms in the same time when
> possible.
>
> Please reply also to the list, this thread may be useful to others.
>
> Nir
>
>
> Hopefully this will go to the list as i"ve been making sure users at ovirt.org
> is in the To field.
>
> I think we need to make sure we distinguish between the start up phase and
> the running phase which is what happens after everyone is up and running
> happily.  Based on my experience the crash of a server is considered
> separately from startup.
>
> As mentioned in my response to the list to someone else's comment the
> startup sequence is the same as people flipping switches on physical servers
> following a documented procedure to start or shutdown a datacenter as we did
> in the "old days".  With physical boxes we had to shutdown/startup manually
> and we did it in a sequence that we had written down.  With virtualization
> and since we had VMware we could automate that process so as we spun up the
> hosts VMware started spinning up the guests in the order specified so we got
> our DNS boxes up first, then others.  For Active Directory we started the
> domain controllers first, then other servers such as file servers,
> application servers in the sequence needed for the applications to run.
>
> Crashing of DNS, File Server, Domain Controllers crashing after the
> datacenter is up and running is handled (or should be) by redundancy of
> servers or process the service provides.  You have multiple DNS servers and
> the resolver will try the secondary/tertiary/whatever if the primary is
> down.  File servers are the same.  For Gluster, CephFS, MS DFS you have (or
> should have setup) the ability to keep running if one of the servers goes
> down.  A redundant file server setup will handle a server crash.  For Domain
> Controllers you should have at least two (we had six in our environment) and
> when one goes down the others keep the domain running by shifting the
> services to others and continuing to provide authentication, etc.  Generally
> what we did when a domain controller crashed is fix it if possible and if it
> was not fixable pull it's pieces out of the domain and spin up a new one.
> Same for DNS or file servers.  When they go down find out why, fix it or
> replace the server, and get the service redundant again.  Also, oVirt has
> the watchdog function so if a VM goes down it will try and restart it.  If
> it can't restart then we're dealing with a crashed server which we should
> have provided for in our data center design.
>
> My wish is to have oVirt allow us to do what VMware does and allow us to say
> start/don't start these servers up in the order I specify so that when the
> Engine is ready it looks at the list and begins turning on VMs in the order
> specified.  Shutdown is done in the reverse order).  For large datacenters
> with many VMS manual startup is a pain.  Once it's running rely on good
> practices of having redundant servers (i.e. more than one DNS), file servers
> that can handle the failure of a server, multiple domain controllers which
> is not something we need to burden oVirt with.  Handling of failures needs
> to be done by the people in charge who are supposed to design the data
> center based on their risk/cost analysis.
>
> If I  understand what you said - I'm not sure we have to define a dependancy
> list where we define dependancies like we would with systemd or a package
> manger.  It doesn't need to be complex since just a simple list of priority:
> VM id/name pairs will work. All we need is to be able to say "start these
> VMs when the Engine is ready".  The order or dependency is set by where I
> put the VM in the list.  If it's at the top start it first, then move to the
> second one, and so on.  There can be settings for delay between starting VMs
> and how long to wait for a VM to come up before assuming it's dead and
> moving on to the next.  Tasking oVirt with the job of figuring VM
> dependancies would be a nightmare for oVirt and whoever had to program it
> <G>.  We as data center administrators should be handling that task.  Yes,
> we manually set the list but trying to automate a dependancy chain would be
> pretty difficult.
>
> I envision a web admin portal GUI where we define a simple list of VMs that
> we want the Engine to autostart. We need to be able to move them up/down the
> list.  The list is stored somewhere for the Engine (database?, whatever else
> the Engine has as storage areas - not really familiar with the Engine
> internals) and when the Engine is up and ready to start VMs it retrieves the
> list and starts at the first VM in the list and starts it, waits some time
> (0-?? seconds), and then moves on to the next one.  If the VM hasn't started
> in the set time the Engine moves on to the next one.
>
> Note that Microsoft's Hyper-V also provides automatic VM startup but it is
> done on a VM level where you just tell the VM to start whenthe Hypr-V
> starts.  If you want sequencing you have to set time delays.  Auto  start up
> is better than nothing but Hyper-V is a nightmare trying to sequence VMs.
>
> I think VMware did it right in allowing both autostart and being able to
> sequence the startup of VMS so it's host independent.  As information VMware
> also allowed delays between VM startups as does Hyper-V.

Thanks for the detailed description.

Did you file a bug for this feature?

Adding Moran and Doron.

Nir

> On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholcomb at l1049h.com>
> wrote:
>
> On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
>
> On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
>
> In VMware we could setup guests to autostart when the host started and
> define the order.  Is that doable in oVirt?  The only thing I've seen
> is the watchdog and tell it to reset but nothing that allows me to
> define who starts up when and if they autostart.  I assume it's there
> but I must be missing it or haven't found it in the web portal.
>
>
> In oVirt guests aren't tied to a host by default (although you can set
> them to run only on a specific host if you want).  The closest thing I
> can think of would be the High Availability features (VM->Edit).
> oVirt will try to restart highly available VMs if they go down.  You
> can also set the priority for migration and restart in that pane.
> Hopefully a combination of host pinning and the  high availability
> settings will get you close enough to where you want to be.
>
> Otherwise, you could always do some scripting with the ovirt REST API
> using the SDK or CLI.
>
>
> If you had the VMware migration extra add-on you could have hosts move as
> needed so they were not tied to any host either but we could set a startup
> order and specify auto, manual so that once the host started the VMs were
> brought up as specified no matter what host they were running on.
>
> I am running hosted-engine deployment with the Engine VM on the host.
>
> I set highly available on, did not pin to any host, and also set the
> watchdog which should reset if they go down but I'm not sure that will start
> them if the host comes up and the VMs are not running.  I'll look at the CLI
> first.
>
> It would be nice if oVirt added this feature as it's really required for
> large installations and is a help for any size installation, even small
> ones.
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>