<html><head></head><body bgcolor="#ffffff" text="#2e3436" link="#2a76c6" vlink="#215d9c" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div>On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:</div><blockquote type="cite"><pre>On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <<a href="mailto:biholcomb@l1049h.com">biholcomb@l1049h.com</a>> wrote:
<blockquote type="cite">
On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <<a href="mailto:biholcomb@l1049h.com">biholcomb@l1049h.com</a>>
wrote:
On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
Hi,
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
The engine will try to keep the VM running. So if one host goes down,
it will restart the VM on some other host automatically. We will also
migrate the VM (or some other to free resources) when the current host
gets too loaded. We do not require any migration addons, it just
works. But of course we have usually more hosts in a cluster to make
this possible.
I do not really remember what happens when all hosts are restarted
(power outage) though as that is quite special case.
Regards
--
Martin Sivak
SLA / oVirt
Thanks. I only have one host so who knows what will happen. I'm working on
a script that will basically emulate what VMware does - start VMS in a given
order at startup of the host/engine. I'll also file a feature request.
Why do you care about the order?
Isn't it enough to restart all the vms after a host was restarted?
Nir
Does the Engine start all VMs after the host is powered up and the engine
running if I set a watchdog on a VM? I know it will try to restart the VM
if it goes down once oVirt is up and running. If it will start the VMs at
reboot time that helps.
</blockquote>
I think only HA vms are restarted. Adding Michal to add more info on this.
<blockquote type="cite">
The reason for the order is that you need some servers such as DNS, file
servers, up first before other systems can resolve addresses or mount
shares. For Windows you need domain controllers running before the other
windows systems that are part of the domain. For applications such as Lotus
Notes the servers had to come up in the correct order.
</blockquote>
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
</pre></blockquote><div><br></div><div>Hopefully this will go to the list as i"ve been making sure <a href="mailto:users@ovirt.org">users@ovirt.org</a> is in the To field.</div><div><br></div><div>I think we need to make sure we distinguish between the start up phase and the running phase which is what happens after everyone is up and running happily. Based on my experience the crash of a server is considered separately from startup. </div><div><br></div><div>As mentioned in my response to the list to someone else's comment the startup sequence is the same as people flipping switches on physical servers following a documented procedure to start or shutdown a datacenter as we did in the "old days". With physical boxes we had to shutdown/startup manually and we did it in a sequence that we had written down. With virtualization and since we had VMware we could automate that process so as we spun up the hosts VMware started spinning up the guests in the order specified so we got our DNS boxes up first, then others. For Active Directory we started the domain controllers first, then other servers such as file servers, application servers in the sequence needed for the applications to run.</div><div><br></div><div>Crashing of DNS, File Server, Domain Controllers crashing after the datacenter is up and running is handled (or should be) by redundancy of servers or process the service provides. You have multiple DNS servers and the resolver will try the secondary/tertiary/whatever if the primary is down. File servers are the same. For Gluster, CephFS, MS DFS you have (or should have setup) the ability to keep running if one of the servers goes down. A redundant file server setup will handle a server crash. For Domain Controllers you should have at least two (we had six in our environment) and when one goes down the others keep the domain running by shifting the services to others and continuing to provide authentication, etc. Generally what we did when a domain controller crashed is fix it if possible and if it was not fixable pull it's pieces out of the domain and spin up a new one. Same for DNS or file servers. When they go down find out why, fix it or replace the server, and get the service redundant again. Also, oVirt has the watchdog function so if a VM goes down it will try and restart it. If it can't restart then we're dealing with a crashed server which we should have provided for in our data center design.</div><div><br></div><div>My wish is to have oVirt allow us to do what VMware does and allow us to say start/don't start these servers up in the order I specify so that when the Engine is ready it looks at the list and begins turning on VMs in the order specified. Shutdown is done in the reverse order). For large datacenters with many VMS manual startup is a pain. Once it's running rely on good practices of having redundant servers (i.e. more than one DNS), file servers that can handle the failure of a server, multiple domain controllers which is not something we need to burden oVirt with. Handling of failures needs to be done by the people in charge who are supposed to design the data center based on their risk/cost analysis. </div><div><br></div><div>If I understand what you said - I'm not sure we have to define a dependancy list where we define dependancies like we would with systemd or a package manger. It doesn't need to be complex since just a simple list of priority: VM id/name pairs will work. All we need is to be able to say "start these VMs when the Engine is ready". The order or dependency is set by where I put the VM in the list. If it's at the top start it first, then move to the second one, and so on. There can be settings for delay between starting VMs and how long to wait for a VM to come up before assuming it's dead and moving on to the next. Tasking oVirt with the job of figuring VM dependancies would be a nightmare for oVirt and whoever had to program it <G>. We as data center administrators should be handling that task. Yes, we manually set the list but trying to automate a dependancy chain would be pretty difficult.</div><div><br></div><div>I envision a web admin portal GUI where we define a simple list of VMs that we want the Engine to autostart. We need to be able to move them up/down the list. The list is stored somewhere for the Engine (database?, whatever else the Engine has as storage areas - not really familiar with the Engine internals) and when the Engine is up and ready to start VMs it retrieves the list and starts at the first VM in the list and starts it, waits some time (0-?? seconds), and then moves on to the next one. If the VM hasn't started in the set time the Engine moves on to the next one.</div><div><br></div><div>Note that Microsoft's Hyper-V also provides automatic VM startup but it is done on a VM level where you just tell the VM to start whenthe Hypr-V starts. If you want sequencing you have to set time delays. Auto start up is better than nothing but Hyper-V is a nightmare trying to sequence VMs.</div><div><br></div><div>I think VMware did it right in allowing both autostart and being able to sequence the startup of VMS so it's host independent. As information VMware also allowed delays between VM startups as does Hyper-V.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><blockquote type="cite"><pre>
<blockquote type="cite">
On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <<a href="mailto:biholcomb@l1049h.com">biholcomb@l1049h.com</a>>
wrote:
On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
In VMware we could setup guests to autostart when the host started and
define the order. Is that doable in oVirt? The only thing I've seen
is the watchdog and tell it to reset but nothing that allows me to
define who starts up when and if they autostart. I assume it's there
but I must be missing it or haven't found it in the web portal.
In oVirt guests aren't tied to a host by default (although you can set
them to run only on a specific host if you want). The closest thing I
can think of would be the High Availability features (VM->Edit).
oVirt will try to restart highly available VMs if they go down. You
can also set the priority for migration and restart in that pane.
Hopefully a combination of host pinning and the high availability
settings will get you close enough to where you want to be.
Otherwise, you could always do some scripting with the ovirt REST API
using the SDK or CLI.
If you had the VMware migration extra add-on you could have hosts move as
needed so they were not tied to any host either but we could set a startup
order and specify auto, manual so that once the host started the VMs were
brought up as specified no matter what host they were running on.
I am running hosted-engine deployment with the Engine VM on the host.
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
It would be nice if oVirt added this feature as it's really required for
large installations and is a help for any size installation, even small
ones.
_______________________________________________
Users mailing list
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
_______________________________________________
Users mailing list
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</blockquote></pre></blockquote></body></html>