It is done. I would be willing to do what I help with it within my skill set.
On Sat, 2016-04-09 at 01:06 +0300, Nir Soffer wrote:
On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholcomb@l1049h.com>
wrote:
On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb@l1049h.com>
wrote:
On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb@l1049h.com>
wrote:
On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
Hi,
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
The engine will try to keep the VM running. So if one host goes down,
it will restart the VM on some other host automatically. We will also
migrate the VM (or some other to free resources) when the current host
gets too loaded. We do not require any migration addons, it just
works. But of course we have usually more hosts in a cluster to make
this possible.
I do not really remember what happens when all hosts are restarted
(power outage) though as that is quite special case.
Regards
--
Martin Sivak
SLA / oVirt
Thanks. I only have one host so who knows what will happen. I'm working on
a script that will basically emulate what VMware does - start VMS in a given
order at startup of the host/engine. I'll also file a feature request.
Why do you care about the order?
Isn't it enough to restart all the vms after a host was restarted?
Nir
Does the Engine start all VMs after the host is powered up and the engine
running if I set a watchdog on a VM? I know it will try to restart the VM
if it goes down once oVirt is up and running. If it will start the VMs at
reboot time that helps.
I think only HA vms are restarted. Adding Michal to add more info on this.
The reason for the order is that you need some servers such as DNS, file
servers, up first before other systems can resolve addresses or mount
shares. For Windows you need domain controllers running before the other
windows systems that are part of the domain. For applications such as Lotus
Notes the servers had to come up in the correct order.
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
Hopefully this will go to the list as i"ve been making sure users@ovirt.org
is in the To field.
I think we need to make sure we distinguish between the start up phase and
the running phase which is what happens after everyone is up and running
happily. Based on my experience the crash of a server is considered
separately from startup.
As mentioned in my response to the list to someone else's comment the
startup sequence is the same as people flipping switches on physical servers
following a documented procedure to start or shutdown a datacenter as we did
in the "old days". With physical boxes we had to shutdown/startup manually
and we did it in a sequence that we had written down. With virtualization
and since we had VMware we could automate that process so as we spun up the
hosts VMware started spinning up the guests in the order specified so we got
our DNS boxes up first, then others. For Active Directory we started the
domain controllers first, then other servers such as file servers,
application servers in the sequence needed for the applications to run.
Crashing of DNS, File Server, Domain Controllers crashing after the
datacenter is up and running is handled (or should be) by redundancy of
servers or process the service provides. You have multiple DNS servers and
the resolver will try the secondary/tertiary/whatever if the primary is
down. File servers are the same. For Gluster, CephFS, MS DFS you have (or
should have setup) the ability to keep running if one of the servers goes
down. A redundant file server setup will handle a server crash. For Domain
Controllers you should have at least two (we had six in our environment) and
when one goes down the others keep the domain running by shifting the
services to others and continuing to provide authentication, etc. Generally
what we did when a domain controller crashed is fix it if possible and if it
was not fixable pull it's pieces out of the domain and spin up a new one.
Same for DNS or file servers. When they go down find out why, fix it or
replace the server, and get the service redundant again. Also, oVirt has
the watchdog function so if a VM goes down it will try and restart it. If
it can't restart then we're dealing with a crashed server which we should
have provided for in our data center design.
My wish is to have oVirt allow us to do what VMware does and allow us to say
start/don't start these servers up in the order I specify so that when the
Engine is ready it looks at the list and begins turning on VMs in the order
specified. Shutdown is done in the reverse order). For large datacenters
with many VMS manual startup is a pain. Once it's running rely on good
practices of having redundant servers (i.e. more than one DNS), file servers
that can handle the failure of a server, multiple domain controllers which
is not something we need to burden oVirt with. Handling of failures needs
to be done by the people in charge who are supposed to design the data
center based on their risk/cost analysis.
If I understand what you said - I'm not sure we have to define a dependancy
list where we define dependancies like we would with systemd or a package
manger. It doesn't need to be complex since just a simple list of priority:
VM id/name pairs will work. All we need is to be able to say "start these
VMs when the Engine is ready". The order or dependency is set by where I
put the VM in the list. If it's at the top start it first, then move to the
second one, and so on. There can be settings for delay between starting VMs
and how long to wait for a VM to come up before assuming it's dead and
moving on to the next. Tasking oVirt with the job of figuring VM
dependancies would be a nightmare for oVirt and whoever had to program it
<G>. We as data center administrators should be handling that task. Yes,
we manually set the list but trying to automate a dependancy chain would be
pretty difficult.
I envision a web admin portal GUI where we define a simple list of VMs that
we want the Engine to autostart. We need to be able to move them up/down the
list. The list is stored somewhere for the Engine (database?, whatever else
the Engine has as storage areas - not really familiar with the Engine
internals) and when the Engine is up and ready to start VMs it retrieves the
list and starts at the first VM in the list and starts it, waits some time
(0-?? seconds), and then moves on to the next one. If the VM hasn't started
in the set time the Engine moves on to the next one.
Note that Microsoft's Hyper-V also provides automatic VM startup but it is
done on a VM level where you just tell the VM to start whenthe Hypr-V
starts. If you want sequencing you have to set time delays. Auto start up
is better than nothing but Hyper-V is a nightmare trying to sequence VMs.
I think VMware did it right in allowing both autostart and being able to
sequence the startup of VMS so it's host independent. As information VMware
also allowed delays between VM startups as does Hyper-V.
Thanks for the detailed description.
Did you file a bug for this feature?
Adding Moran and Doron.
Nir
On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholcomb@l1049h.com>
wrote:
On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
In VMware we could setup guests to autostart when the host started and
define the order. Is that doable in oVirt? The only thing I've seen
is the watchdog and tell it to reset but nothing that allows me to
define who starts up when and if they autostart. I assume it's there
but I must be missing it or haven't found it in the web portal.
In oVirt guests aren't tied to a host by default (although you can set
them to run only on a specific host if you want). The closest thing I
can think of would be the High Availability features (VM->Edit).
oVirt will try to restart highly available VMs if they go down. You
can also set the priority for migration and restart in that pane.
Hopefully a combination of host pinning and the high availability
settings will get you close enough to where you want to be.
Otherwise, you could always do some scripting with the ovirt REST API
using the SDK or CLI.
If you had the VMware migration extra add-on you could have hosts move as
needed so they were not tied to any host either but we could set a startup
order and specify auto, manual so that once the host started the VMs were
brought up as specified no matter what host they were running on.
I am running hosted-engine deployment with the Engine VM on the host.
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
It would be nice if oVirt added this feature as it's really required for
large installations and is a help for any size installation, even small
ones.
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users