2:50 p.m.
On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb(a)l1049h.com> wrote:
On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
Hi,
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
The engine will try to keep the VM running. So if one host goes down,
it will restart the VM on some other host automatically. We will also
migrate the VM (or some other to free resources) when the current host
gets too loaded. We do not require any migration addons, it just
works. But of course we have usually more hosts in a cluster to make
this possible.
I do not really remember what happens when all hosts are restarted
(power outage) though as that is quite special case.
Regards
--
Martin Sivak
SLA / oVirt
Thanks. I only have one host so who knows what will happen. I'm working on
a script that will basically emulate what VMware does - start VMS in a given
order at startup of the host/engine. I'll also file a feature request.
Why do you care about the order?
Isn't it enough to restart all the vms after a host was restarted?
Nir
Does the Engine start all VMs after the host is powered up and the engine
running if I set a watchdog on a VM? I know it will try to restart the VM
if it goes down once oVirt is up and running. If it will start the VMs at
reboot time that helps.
I think only HA vms are restarted. Adding Michal to add more info on this.
The reason for the order is that you need some servers such as DNS,
file
servers, up first before other systems can resolve addresses or mount
shares. For Windows you need domain controllers running before the other
windows systems that are part of the domain. For applications such as Lotus
Notes the servers had to come up in the correct order.
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb
<biholcomb(a)l1049h.com>
wrote:
On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
In VMware we could setup guests to autostart when the host started and
define the order. Is that doable in oVirt? The only thing I've seen
is the watchdog and tell it to reset but nothing that allows me to
define who starts up when and if they autostart. I assume it's there
but I must be missing it or haven't found it in the web portal.
In oVirt guests aren't tied to a host by default (although you can set
them to run only on a specific host if you want). The closest thing I
can think of would be the High Availability features (VM->Edit).
oVirt will try to restart highly available VMs if they go down. You
can also set the priority for migration and restart in that pane.
Hopefully a combination of host pinning and the high availability
settings will get you close enough to where you want to be.
Otherwise, you could always do some scripting with the ovirt REST API
using the SDK or CLI.
If you had the VMware migration extra add-on you could have hosts move as
needed so they were not tied to any host either but we could set a startup
order and specify auto, manual so that once the host started the VMs were
brought up as specified no matter what host they were running on.
I am running hosted-engine deployment with the Engine VM on the host.
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
It would be nice if oVirt added this feature as it's really required for
large installations and is a help for any size installation, even small
ones.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
5:07 p.m.
New subject: Autostart VMS
On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb
m> wrote:
>
> On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb
> com>
> wrote:
>
> On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
>
> Hi,
>
>
> I set highly available on, did not pin to any host, and also set
> the
> watchdog which should reset if they go down but I'm not sure that
> will start
> them if the host comes up and the VMs are not running. I'll look
> at the CLI
> first.
>
>
> The engine will try to keep the VM running. So if one host goes
> down,
> it will restart the VM on some other host automatically. We will
> also
> migrate the VM (or some other to free resources) when the current
> host
> gets too loaded. We do not require any migration addons, it just
> works. But of course we have usually more hosts in a cluster to
> make
> this possible.
>
> I do not really remember what happens when all hosts are restarted
> (power outage) though as that is quite special case.
>
> Regards
>
> --
> Martin Sivak
> SLA / oVirt
>
>
>
> Thanks. I only have one host so who knows what will happen. I'm
> working on
> a script that will basically emulate what VMware does - start VMS
> in a given
> order at startup of the host/engine. I'll also file a feature
> request.
>
>
> Why do you care about the order?
>
> Isn't it enough to restart all the vms after a host was restarted?
>
> Nir
>
>
> Does the Engine start all VMs after the host is powered up and the
> engine
> running if I set a watchdog on a VM? I know it will try to restart
> the VM
> if it goes down once oVirt is up and running. If it will start
> the VMs at
> reboot time that helps.
I think only HA vms are restarted. Adding Michal to add more info on
this.
>
> The reason for the order is that you need some servers such as DNS,
> file
> servers, up first before other systems can resolve addresses or
> mount
> shares. For Windows you need domain controllers running before the
> other
> windows systems that are part of the domain. For applications such
> as Lotus
> Notes the servers had to come up in the correct order.
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will
crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
Hopefully this will go to the list as i"ve been making sure users@ovirt
.org is in the To field.
I think we need to make sure we distinguish between the start up phase
and the running phase which is what happens after everyone is up and
running happily. Based on my experience the crash of a server is
considered separately from startup.
As mentioned in my response to the list to someone else's comment the
startup sequence is the same as people flipping switches on physical
servers following a documented procedure to start or shutdown a
datacenter as we did in the "old days". With physical boxes we had to
shutdown/startup manually and we did it in a sequence that we had
written down. With virtualization and since we had VMware we could
automate that process so as we spun up the hosts VMware started
spinning up the guests in the order specified so we got our DNS boxes
up first, then others. For Active Directory we started the domain
controllers first, then other servers such as file servers, application
servers in the sequence needed for the applications to run.
Crashing of DNS, File Server, Domain Controllers crashing after the
datacenter is up and running is handled (or should be) by redundancy of
servers or process the service provides. You have multiple DNS servers
and the resolver will try the secondary/tertiary/whatever if the
primary is down. File servers are the same. For Gluster, CephFS, MS
DFS you have (or should have setup) the ability to keep running if one
of the servers goes down. A redundant file server setup will handle a
server crash. For Domain Controllers you should have at least two (we
had six in our environment) and when one goes down the others keep the
domain running by shifting the services to others and continuing to
provide authentication, etc. Generally what we did when a domain
controller crashed is fix it if possible and if it was not fixable pull
it's pieces out of the domain and spin up a new one. Same for DNS or
file servers. When they go down find out why, fix it or replace the
server, and get the service redundant again. Also, oVirt has the
watchdog function so if a VM goes down it will try and restart it. If
it can't restart then we're dealing with a crashed server which we
should have provided for in our data center design.
My wish is to have oVirt allow us to do what VMware does and allow us
to say start/don't start these servers up in the order I specify so
that when the Engine is ready it looks at the list and begins turning
on VMs in the order specified. Shutdown is done in the reverse order).
For large datacenters with many VMS manual startup is a pain. Once
it's running rely on good practices of having redundant servers (i.e.
more than one DNS), file servers that can handle the failure of a
server, multiple domain controllers which is not something we need to
burden oVirt with. Handling of failures needs to be done by the people
in charge who are supposed to design the data center based on their
risk/cost analysis.
If I understand what you said - I'm not sure we have to define a
dependancy list where we define dependancies like we would with systemd
or a package manger. It doesn't need to be complex since just a simple
list of priority: VM id/name pairs will work. All we need is to be able
to say "start these VMs when the Engine is ready". The order or
dependency is set by where I put the VM in the list. If it's at the
top start it first, then move to the second one, and so on. There can
be settings for delay between starting VMs and how long to wait for a
VM to come up before assuming it's dead and moving on to the next.
Tasking oVirt with the job of figuring VM dependancies would be a
nightmare for oVirt and whoever had to program it <G>. We as data
center administrators should be handling that task. Yes, we manually
set the list but trying to automate a dependancy chain would be pretty
difficult.
I envision a web admin portal GUI where we define a simple list of VMs
that we want the Engine to autostart. We need to be able to move them
up/down the list. The list is stored somewhere for the Engine
(database?, whatever else the Engine has as storage areas - not really
familiar with the Engine internals) and when the Engine is up and ready
to start VMs it retrieves the list and starts at the first VM in the
list and starts it, waits some time (0-?? seconds), and then moves on
to the next one. If the VM hasn't started in the set time the Engine
moves on to the next one.
Note that Microsoft's Hyper-V also provides automatic VM startup but it
is done on a VM level where you just tell the VM to start whenthe Hypr-
V starts. If you want sequencing you have to set time delays. Auto
start up is better than nothing but Hyper-V is a nightmare trying to
sequence VMs.
I think VMware did it right in allowing both autostart and being able
to sequence the startup of VMS so it's host independent. As
information VMware also allowed delays between VM startups as does
Hyper-V.
>
> On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
> >
> wrote:
>
> On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
>
> On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
>
> In VMware we could setup guests to autostart when the host started and
> define the order. Is that doable in oVirt? The only thing I've seen
> is the watchdog and tell it to reset but nothing that allows me to
> define who starts up when and if they autostart. I assume it's there
> but I must be missing it or haven't found it in the web portal.
>
>
> In oVirt guests aren't tied to a host by default (although you can set
> them to run only on a specific host if you want). The closest thing I
> can think of would be the High Availability features (VM->Edit).
> oVirt will try to restart highly available VMs if they go down. You
> can also set the priority for migration and restart in that pane.
> Hopefully a combination of host pinning and the high availability
> settings will get you close enough to where you want to be.
>
> Otherwise, you could always do some scripting with the ovirt REST API
> using the SDK or CLI.
>
>
> If you had the VMware migration extra add-on you could have hosts move as
> needed so they were not tied to any host either but we could set a startup
> order and specify auto, manual so that once the host started the VMs were
> brought up as specified no matter what host they were running on.
>
> I am running hosted-engine deployment with the Engine VM on the host.
>
> I set highly available on, did not pin to any host, and also set the
> watchdog which should reset if they go down but I'm not sure that will start
> them if the host comes up and the VMs are not running. I'll look at the CLI
> first.
>
> It would be nice if oVirt added this feature as it's really required for
> large installations and is a help for any size installation, even small
> ones.
>
>
>
>
> _______________________________________________
> Users mailing list
>
Users(a)ovirt.org> >
http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
>
Users(a)ovirt.org> >
http://lists.ovirt.org/mailman/listinfo/users
>
>
>
6:06 p.m.
New subject: Autostart VMS
On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholcomb(a)l1049h.com> wrote:
On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
Hi,
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
The engine will try to keep the VM running. So if one host goes down,
it will restart the VM on some other host automatically. We will also
migrate the VM (or some other to free resources) when the current host
gets too loaded. We do not require any migration addons, it just
works. But of course we have usually more hosts in a cluster to make
this possible.
I do not really remember what happens when all hosts are restarted
(power outage) though as that is quite special case.
Regards
--
Martin Sivak
SLA / oVirt
Thanks. I only have one host so who knows what will happen. I'm working on
a script that will basically emulate what VMware does - start VMS in a given
order at startup of the host/engine. I'll also file a feature request.
Why do you care about the order?
Isn't it enough to restart all the vms after a host was restarted?
Nir
Does the Engine start all VMs after the host is powered up and the engine
running if I set a watchdog on a VM? I know it will try to restart the VM
if it goes down once oVirt is up and running. If it will start the VMs at
reboot time that helps.
I think only HA vms are restarted. Adding Michal to add more info on this.
The reason for the order is that you need some servers such as DNS, file
servers, up first before other systems can resolve addresses or mount
shares. For Windows you need domain controllers running before the other
windows systems that are part of the domain. For applications such as Lotus
Notes the servers had to come up in the correct order.
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
Hopefully this will go to the list as i"ve been making sure users(a)ovirt.org
is in the To field.
I think we need to make sure we distinguish between the start up phase and
the running phase which is what happens after everyone is up and running
happily. Based on my experience the crash of a server is considered
separately from startup.
As mentioned in my response to the list to someone else's comment the
startup sequence is the same as people flipping switches on physical servers
following a documented procedure to start or shutdown a datacenter as we did
in the "old days". With physical boxes we had to shutdown/startup manually
and we did it in a sequence that we had written down. With virtualization
and since we had VMware we could automate that process so as we spun up the
hosts VMware started spinning up the guests in the order specified so we got
our DNS boxes up first, then others. For Active Directory we started the
domain controllers first, then other servers such as file servers,
application servers in the sequence needed for the applications to run.
Crashing of DNS, File Server, Domain Controllers crashing after the
datacenter is up and running is handled (or should be) by redundancy of
servers or process the service provides. You have multiple DNS servers and
the resolver will try the secondary/tertiary/whatever if the primary is
down. File servers are the same. For Gluster, CephFS, MS DFS you have (or
should have setup) the ability to keep running if one of the servers goes
down. A redundant file server setup will handle a server crash. For Domain
Controllers you should have at least two (we had six in our environment) and
when one goes down the others keep the domain running by shifting the
services to others and continuing to provide authentication, etc. Generally
what we did when a domain controller crashed is fix it if possible and if it
was not fixable pull it's pieces out of the domain and spin up a new one.
Same for DNS or file servers. When they go down find out why, fix it or
replace the server, and get the service redundant again. Also, oVirt has
the watchdog function so if a VM goes down it will try and restart it. If
it can't restart then we're dealing with a crashed server which we should
have provided for in our data center design.
My wish is to have oVirt allow us to do what VMware does and allow us to say
start/don't start these servers up in the order I specify so that when the
Engine is ready it looks at the list and begins turning on VMs in the order
specified. Shutdown is done in the reverse order). For large datacenters
with many VMS manual startup is a pain. Once it's running rely on good
practices of having redundant servers (i.e. more than one DNS), file servers
that can handle the failure of a server, multiple domain controllers which
is not something we need to burden oVirt with. Handling of failures needs
to be done by the people in charge who are supposed to design the data
center based on their risk/cost analysis.
If I understand what you said - I'm not sure we have to define a dependancy
list where we define dependancies like we would with systemd or a package
manger. It doesn't need to be complex since just a simple list of priority:
VM id/name pairs will work. All we need is to be able to say "start these
VMs when the Engine is ready". The order or dependency is set by where I
put the VM in the list. If it's at the top start it first, then move to the
second one, and so on. There can be settings for delay between starting VMs
and how long to wait for a VM to come up before assuming it's dead and
moving on to the next. Tasking oVirt with the job of figuring VM
dependancies would be a nightmare for oVirt and whoever had to program it
<G>. We as data center administrators should be handling that task. Yes,
we manually set the list but trying to automate a dependancy chain would be
pretty difficult.
I envision a web admin portal GUI where we define a simple list of VMs that
we want the Engine to autostart. We need to be able to move them up/down the
list. The list is stored somewhere for the Engine (database?, whatever else
the Engine has as storage areas - not really familiar with the Engine
internals) and when the Engine is up and ready to start VMs it retrieves the
list and starts at the first VM in the list and starts it, waits some time
(0-?? seconds), and then moves on to the next one. If the VM hasn't started
in the set time the Engine moves on to the next one.
Note that Microsoft's Hyper-V also provides automatic VM startup but it is
done on a VM level where you just tell the VM to start whenthe Hypr-V
starts. If you want sequencing you have to set time delays. Auto start up
is better than nothing but Hyper-V is a nightmare trying to sequence VMs.
I think VMware did it right in allowing both autostart and being able to
sequence the startup of VMS so it's host independent. As information VMware
also allowed delays between VM startups as does Hyper-V.
Thanks for the detailed description.
Did you file a bug for this feature?
Adding Moran and Doron.
Nir
On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb
<biholcomb(a)l1049h.com>
wrote:
On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
In VMware we could setup guests to autostart when the host started and
define the order. Is that doable in oVirt? The only thing I've seen
is the watchdog and tell it to reset but nothing that allows me to
define who starts up when and if they autostart. I assume it's there
but I must be missing it or haven't found it in the web portal.
In oVirt guests aren't tied to a host by default (although you can set
them to run only on a specific host if you want). The closest thing I
can think of would be the High Availability features (VM->Edit).
oVirt will try to restart highly available VMs if they go down. You
can also set the priority for migration and restart in that pane.
Hopefully a combination of host pinning and the high availability
settings will get you close enough to where you want to be.
Otherwise, you could always do some scripting with the ovirt REST API
using the SDK or CLI.
If you had the VMware migration extra add-on you could have hosts move as
needed so they were not tied to any host either but we could set a startup
order and specify auto, manual so that once the host started the VMs were
brought up as specified no matter what host they were running on.
I am running hosted-engine deployment with the Engine VM on the host.
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
It would be nice if oVirt added this feature as it's really required for
large installations and is a help for any size installation, even small
ones.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:09 p.m.
New subject: Autostart VMS
You are welcome. I haven't filed a bug for it yet but will be glad to
do so or would it be a Feature request? I assume I go to the oVirt bug
tracker site.
On Sat, 2016-04-09 at 01:06 +0300, Nir Soffer wrote:
On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb
om> wrote:
>
> On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb
> com>
> wrote:
>
> On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb
> com>
> wrote:
>
> On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
>
> Hi,
>
>
> I set highly available on, did not pin to any host, and also set
> the
> watchdog which should reset if they go down but I'm not sure that
> will start
> them if the host comes up and the VMs are not running. I'll look
> at the CLI
> first.
>
>
> The engine will try to keep the VM running. So if one host goes
> down,
> it will restart the VM on some other host automatically. We will
> also
> migrate the VM (or some other to free resources) when the current
> host
> gets too loaded. We do not require any migration addons, it just
> works. But of course we have usually more hosts in a cluster to
> make
> this possible.
>
> I do not really remember what happens when all hosts are restarted
> (power outage) though as that is quite special case.
>
> Regards
>
> --
> Martin Sivak
> SLA / oVirt
>
>
>
> Thanks. I only have one host so who knows what will happen. I'm
> working on
> a script that will basically emulate what VMware does - start VMS
> in a given
> order at startup of the host/engine. I'll also file a feature
> request.
>
>
> Why do you care about the order?
>
> Isn't it enough to restart all the vms after a host was restarted?
>
> Nir
>
>
> Does the Engine start all VMs after the host is powered up and the
> engine
> running if I set a watchdog on a VM? I know it will try to restart
> the VM
> if it goes down once oVirt is up and running. If it will start
> the VMs at
> reboot time that helps.
>
>
> I think only HA vms are restarted. Adding Michal to add more info
> on this.
>
>
> The reason for the order is that you need some servers such as DNS,
> file
> servers, up first before other systems can resolve addresses or
> mount
> shares. For Windows you need domain controllers running before the
> other
> windows systems that are part of the domain. For applications such
> as Lotus
> Notes the servers had to come up in the correct order.
>
>
> Lets say you have a way to order the vms when some vms are down.
>
> What will happen to when dns, file servers or domain controller
> will crash?
> Do you have to restart all the vms depending on them or they can
> recover and detect that vm they depend on were restarted?
>
> Seems that what you need is a systemd for your data center - every
> host define the host it depends on, and the system generate the
> correct order to start the vms, starting vms in the same time when
> possible.
>
> Please reply also to the list, this thread may be useful to others.
>
> Nir
>
>
> Hopefully this will go to the list as i"ve been making sure users@o
> virt.org
> is in the To field.
>
> I think we need to make sure we distinguish between the start up
> phase and
> the running phase which is what happens after everyone is up and
> running
> happily. Based on my experience the crash of a server is
> considered
> separately from startup.
>
> As mentioned in my response to the list to someone else's comment
> the
> startup sequence is the same as people flipping switches on
> physical servers
> following a documented procedure to start or shutdown a datacenter
> as we did
> in the "old days". With physical boxes we had to shutdown/startup
> manually
> and we did it in a sequence that we had written down. With
> virtualization
> and since we had VMware we could automate that process so as we
> spun up the
> hosts VMware started spinning up the guests in the order specified
> so we got
> our DNS boxes up first, then others. For Active Directory we
> started the
> domain controllers first, then other servers such as file servers,
> application servers in the sequence needed for the applications to
> run.
>
> Crashing of DNS, File Server, Domain Controllers crashing after the
> datacenter is up and running is handled (or should be) by
> redundancy of
> servers or process the service provides. You have multiple DNS
> servers and
> the resolver will try the secondary/tertiary/whatever if the
> primary is
> down. File servers are the same. For Gluster, CephFS, MS DFS you
> have (or
> should have setup) the ability to keep running if one of the
> servers goes
> down. A redundant file server setup will handle a server
> crash. For Domain
> Controllers you should have at least two (we had six in our
> environment) and
> when one goes down the others keep the domain running by shifting
> the
> services to others and continuing to provide authentication,
> etc. Generally
> what we did when a domain controller crashed is fix it if possible
> and if it
> was not fixable pull it's pieces out of the domain and spin up a
> new one.
> Same for DNS or file servers. When they go down find out why, fix
> it or
> replace the server, and get the service redundant again. Also,
> oVirt has
> the watchdog function so if a VM goes down it will try and restart
> it. If
> it can't restart then we're dealing with a crashed server which we
> should
> have provided for in our data center design.
>
> My wish is to have oVirt allow us to do what VMware does and allow
> us to say
> start/don't start these servers up in the order I specify so that
> when the
> Engine is ready it looks at the list and begins turning on VMs in
> the order
> specified. Shutdown is done in the reverse order). For large
> datacenters
> with many VMS manual startup is a pain. Once it's running rely on
> good
> practices of having redundant servers (i.e. more than one DNS),
> file servers
> that can handle the failure of a server, multiple domain
> controllers which
> is not something we need to burden oVirt with. Handling of
> failures needs
> to be done by the people in charge who are supposed to design the
> data
> center based on their risk/cost analysis.
>
> If I understand what you said - I'm not sure we have to define a
> dependancy
> list where we define dependancies like we would with systemd or a
> package
> manger. It doesn't need to be complex since just a simple list of
> priority:
> VM id/name pairs will work. All we need is to be able to say "start
> these
> VMs when the Engine is ready". The order or dependency is set by
> where I
> put the VM in the list. If it's at the top start it first, then
> move to the
> second one, and so on. There can be settings for delay between
> starting VMs
> and how long to wait for a VM to come up before assuming it's dead
> and
> moving on to the next. Tasking oVirt with the job of figuring VM
> dependancies would be a nightmare for oVirt and whoever had to
> program it
> . We as data center administrators should be handling that
> task. Yes,
> we manually set the list but trying to automate a dependancy chain
> would be
> pretty difficult.
>
> I envision a web admin portal GUI where we define a simple list of
> VMs that
> we want the Engine to autostart. We need to be able to move them
> up/down the
> list. The list is stored somewhere for the Engine (database?,
> whatever else
> the Engine has as storage areas - not really familiar with the
> Engine
> internals) and when the Engine is up and ready to start VMs it
> retrieves the
> list and starts at the first VM in the list and starts it, waits
> some time
> (0-?? seconds), and then moves on to the next one. If the VM
> hasn't started
> in the set time the Engine moves on to the next one.
>
> Note that Microsoft's Hyper-V also provides automatic VM startup
> but it is
> done on a VM level where you just tell the VM to start whenthe
> Hypr-V
> starts. If you want sequencing you have to set time
> delays. Auto start up
> is better than nothing but Hyper-V is a nightmare trying to
> sequence VMs.
>
> I think VMware did it right in allowing both autostart and being
> able to
> sequence the startup of VMS so it's host independent. As
> information VMware
> also allowed delays between VM startups as does Hyper-V.
Thanks for the detailed description.
Did you file a bug for this feature?
Adding Moran and Doron.
Nir
>
> On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb
> com>
> wrote:
>
> On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
>
> On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
>
> In VMware we could setup guests to autostart when the host started
> and
> define the order. Is that doable in oVirt? The only thing I've
> seen
> is the watchdog and tell it to reset but nothing that allows me to
> define who starts up when and if they autostart. I assume it's
> there
> but I must be missing it or haven't found it in the web portal.
>
>
> In oVirt guests aren't tied to a host by default (although you can
> set
> them to run only on a specific host if you want). The closest
> thing I
> can think of would be the High Availability features (VM->Edit).
> oVirt will try to restart highly available VMs if they go
> down. You
> can also set the priority for migration and restart in that pane.
> Hopefully a combination of host pinning and the high availability
> settings will get you close enough to where you want to be.
>
> Otherwise, you could always do some scripting with the ovirt REST
> API
> using the SDK or CLI.
>
>
> If you had the VMware migration extra add-on you could have hosts
> move as
> needed so they were not tied to any host either but we could set a
> startup
> order and specify auto, manual so that once the host started the
> VMs were
> brought up as specified no matter what host they were running on.
>
> I am running hosted-engine deployment with the Engine VM on the
> host.
>
> I set highly available on, did not pin to any host, and also set
> the
> watchdog which should reset if they go down but I'm not sure that
> will start
> them if the host comes up and the VMs are not running. I'll look
> at the CLI
> first.
>
> It would be nice if oVirt added this feature as it's really
> required for
> large installations and is a help for any size installation, even
> small
> ones.
>
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
6:17 p.m.
New subject: Autostart VMS
On Sat, Apr 9, 2016 at 1:09 AM, Brett I. Holcomb <biholcomb(a)l1049h.com> wrote:
You are welcome. I haven't filed a bug for it yet but will be
glad to do so
or would it be a Feature request? I assume I go to the oVirt bug tracker
site.
Yes, here:
https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-engine
Component should be RFEs
oVirt team is probably SLA
We have a voting feature now, so users interested in this feature
can vote on the bug.
Thanks,
Nir
On Sat, 2016-04-09 at 01:06 +0300, Nir Soffer wrote:
On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
Hi,
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
The engine will try to keep the VM running. So if one host goes down,
it will restart the VM on some other host automatically. We will also
migrate the VM (or some other to free resources) when the current host
gets too loaded. We do not require any migration addons, it just
works. But of course we have usually more hosts in a cluster to make
this possible.
I do not really remember what happens when all hosts are restarted
(power outage) though as that is quite special case.
Regards
--
Martin Sivak
SLA / oVirt
Thanks. I only have one host so who knows what will happen. I'm working on
a script that will basically emulate what VMware does - start VMS in a given
order at startup of the host/engine. I'll also file a feature request.
Why do you care about the order?
Isn't it enough to restart all the vms after a host was restarted?
Nir
Does the Engine start all VMs after the host is powered up and the engine
running if I set a watchdog on a VM? I know it will try to restart the VM
if it goes down once oVirt is up and running. If it will start the VMs at
reboot time that helps.
I think only HA vms are restarted. Adding Michal to add more info on this.
The reason for the order is that you need some servers such as DNS, file
servers, up first before other systems can resolve addresses or mount
shares. For Windows you need domain controllers running before the other
windows systems that are part of the domain. For applications such as Lotus
Notes the servers had to come up in the correct order.
Lets say you have a way to order the vms when some vms are down.
What will happen to when dns, file servers or domain controller will crash?
Do you have to restart all the vms depending on them or they can
recover and detect that vm they depend on were restarted?
Seems that what you need is a systemd for your data center - every
host define the host it depends on, and the system generate the
correct order to start the vms, starting vms in the same time when
possible.
Please reply also to the list, this thread may be useful to others.
Nir
Hopefully this will go to the list as i"ve been making sure users(a)ovirt.org
is in the To field.
I think we need to make sure we distinguish between the start up phase and
the running phase which is what happens after everyone is up and running
happily. Based on my experience the crash of a server is considered
separately from startup.
As mentioned in my response to the list to someone else's comment the
startup sequence is the same as people flipping switches on physical servers
following a documented procedure to start or shutdown a datacenter as we did
in the "old days". With physical boxes we had to shutdown/startup manually
and we did it in a sequence that we had written down. With virtualization
and since we had VMware we could automate that process so as we spun up the
hosts VMware started spinning up the guests in the order specified so we got
our DNS boxes up first, then others. For Active Directory we started the
domain controllers first, then other servers such as file servers,
application servers in the sequence needed for the applications to run.
Crashing of DNS, File Server, Domain Controllers crashing after the
datacenter is up and running is handled (or should be) by redundancy of
servers or process the service provides. You have multiple DNS servers and
the resolver will try the secondary/tertiary/whatever if the primary is
down. File servers are the same. For Gluster, CephFS, MS DFS you have (or
should have setup) the ability to keep running if one of the servers goes
down. A redundant file server setup will handle a server crash. For Domain
Controllers you should have at least two (we had six in our environment) and
when one goes down the others keep the domain running by shifting the
services to others and continuing to provide authentication, etc. Generally
what we did when a domain controller crashed is fix it if possible and if it
was not fixable pull it's pieces out of the domain and spin up a new one.
Same for DNS or file servers. When they go down find out why, fix it or
replace the server, and get the service redundant again. Also, oVirt has
the watchdog function so if a VM goes down it will try and restart it. If
it can't restart then we're dealing with a crashed server which we should
have provided for in our data center design.
My wish is to have oVirt allow us to do what VMware does and allow us to say
start/don't start these servers up in the order I specify so that when the
Engine is ready it looks at the list and begins turning on VMs in the order
specified. Shutdown is done in the reverse order). For large datacenters
with many VMS manual startup is a pain. Once it's running rely on good
practices of having redundant servers (i.e. more than one DNS), file servers
that can handle the failure of a server, multiple domain controllers which
is not something we need to burden oVirt with. Handling of failures needs
to be done by the people in charge who are supposed to design the data
center based on their risk/cost analysis.
If I understand what you said - I'm not sure we have to define a dependancy
list where we define dependancies like we would with systemd or a package
manger. It doesn't need to be complex since just a simple list of priority:
VM id/name pairs will work. All we need is to be able to say "start these
VMs when the Engine is ready". The order or dependency is set by where I
put the VM in the list. If it's at the top start it first, then move to the
second one, and so on. There can be settings for delay between starting VMs
and how long to wait for a VM to come up before assuming it's dead and
moving on to the next. Tasking oVirt with the job of figuring VM
dependancies would be a nightmare for oVirt and whoever had to program it
<G>. We as data center administrators should be handling that task. Yes,
we manually set the list but trying to automate a dependancy chain would be
pretty difficult.
I envision a web admin portal GUI where we define a simple list of VMs that
we want the Engine to autostart. We need to be able to move them up/down the
list. The list is stored somewhere for the Engine (database?, whatever else
the Engine has as storage areas - not really familiar with the Engine
internals) and when the Engine is up and ready to start VMs it retrieves the
list and starts at the first VM in the list and starts it, waits some time
(0-?? seconds), and then moves on to the next one. If the VM hasn't started
in the set time the Engine moves on to the next one.
Note that Microsoft's Hyper-V also provides automatic VM startup but it is
done on a VM level where you just tell the VM to start whenthe Hypr-V
starts. If you want sequencing you have to set time delays. Auto start up
is better than nothing but Hyper-V is a nightmare trying to sequence VMs.
I think VMware did it right in allowing both autostart and being able to
sequence the startup of VMS so it's host independent. As information VMware
also allowed delays between VM startups as does Hyper-V.
Thanks for the detailed description.
Did you file a bug for this feature?
Adding Moran and Doron.
Nir
On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
wrote:
On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
In VMware we could setup guests to autostart when the host started and
define the order. Is that doable in oVirt? The only thing I've seen
is the watchdog and tell it to reset but nothing that allows me to
define who starts up when and if they autostart. I assume it's there
but I must be missing it or haven't found it in the web portal.
In oVirt guests aren't tied to a host by default (although you can set
them to run only on a specific host if you want). The closest thing I
can think of would be the High Availability features (VM->Edit).
oVirt will try to restart highly available VMs if they go down. You
can also set the priority for migration and restart in that pane.
Hopefully a combination of host pinning and the high availability
settings will get you close enough to where you want to be.
Otherwise, you could always do some scripting with the ovirt REST API
using the SDK or CLI.
If you had the VMware migration extra add-on you could have hosts move as
needed so they were not tied to any host either but we could set a startup
order and specify auto, manual so that once the host started the VMs were
brought up as specified no matter what host they were running on.
I am running hosted-engine deployment with the Engine VM on the host.
I set highly available on, did not pin to any host, and also set the
watchdog which should reset if they go down but I'm not sure that will start
them if the host comes up and the VMs are not running. I'll look at the CLI
first.
It would be nice if oVirt added this feature as it's really required for
large installations and is a help for any size installation, even small
ones.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
8:46 p.m.
New subject: Autostart VMS
On Sat, 2016-04-09 at 01:17 +0300, Nir Soffer wrote:
On Sat, Apr 9, 2016 at 1:09 AM, Brett I. Holcomb
m> wrote:
>
> You are welcome. I haven't filed a bug for it yet but will be glad
> to do so
> or would it be a Feature request? I assume I go to the oVirt bug
> tracker
> site.
Yes, here:
https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-engine
Component should be RFEs
oVirt team is probably SLA
We have a voting feature now, so users interested in this feature
can vote on the bug.
Thanks,
Nir
It is done. I would be willing to do what I help with it within my
skill set.
Bug 1325468 has been added to the database
Email sent to:
s.kieske(a)mittwald.de,
levi.harper(a)dart.biz,
vanoppen.koen(a)gmail.com,
sherold(a)redhat.com,
didi(a)redhat.com,
gklein(a)redhat.com,
bugs(a)ovirt.org
Excluding:
biholcomb(a)l1049h.com
>
>
>
>
>
> On Sat, 2016-04-09 at 01:06 +0300, Nir Soffer wrote:
>
> On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholcomb(a)l1049h.com>
> >
> wrote:
>
> On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
> >
> wrote:
>
> On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote:
>
> On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
> >
> wrote:
>
> On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote:
>
> Hi,
>
>
> I set highly available on, did not pin to any host, and also set the
> watchdog which should reset if they go down but I'm not sure that will start
> them if the host comes up and the VMs are not running. I'll look at the CLI
> first.
>
>
> The engine will try to keep the VM running. So if one host goes down,
> it will restart the VM on some other host automatically. We will also
> migrate the VM (or some other to free resources) when the current host
> gets too loaded. We do not require any migration addons, it just
> works. But of course we have usually more hosts in a cluster to make
> this possible.
>
> I do not really remember what happens when all hosts are restarted
> (power outage) though as that is quite special case.
>
> Regards
>
> --
> Martin Sivak
> SLA / oVirt
>
>
>
> Thanks. I only have one host so who knows what will happen. I'm working on
> a script that will basically emulate what VMware does - start VMS in a given
> order at startup of the host/engine. I'll also file a feature request.
>
>
> Why do you care about the order?
>
> Isn't it enough to restart all the vms after a host was restarted?
>
> Nir
>
>
> Does the Engine start all VMs after the host is powered up and the engine
> running if I set a watchdog on a VM? I know it will try to restart the VM
> if it goes down once oVirt is up and running. If it will start the VMs at
> reboot time that helps.
>
>
> I think only HA vms are restarted. Adding Michal to add more info on this.
>
>
> The reason for the order is that you need some servers such as DNS, file
> servers, up first before other systems can resolve addresses or mount
> shares. For Windows you need domain controllers running before the other
> windows systems that are part of the domain. For applications such as Lotus
> Notes the servers had to come up in the correct order.
>
>
> Lets say you have a way to order the vms when some vms are down.
>
> What will happen to when dns, file servers or domain controller will crash?
> Do you have to restart all the vms depending on them or they can
> recover and detect that vm they depend on were restarted?
>
> Seems that what you need is a systemd for your data center - every
> host define the host it depends on, and the system generate the
> correct order to start the vms, starting vms in the same time when
> possible.
>
> Please reply also to the list, this thread may be useful to others.
>
> Nir
>
>
> Hopefully this will go to the list as i"ve been making sure users(a)ovirt.org
>
> is in the To field.
>
> I think we need to make sure we distinguish between the start up phase and
> the running phase which is what happens after everyone is up and running
> happily. Based on my experience the crash of a server is considered
> separately from startup.
>
> As mentioned in my response to the list to someone else's comment the
> startup sequence is the same as people flipping switches on physical servers
> following a documented procedure to start or shutdown a datacenter as we did
> in the "old days". With physical boxes we had to shutdown/startup
manually
> and we did it in a sequence that we had written down. With virtualization
> and since we had VMware we could automate that process so as we spun up the
> hosts VMware started spinning up the guests in the order specified so we got
> our DNS boxes up first, then others. For Active Directory we started the
> domain controllers first, then other servers such as file servers,
> application servers in the sequence needed for the applications to run.
>
> Crashing of DNS, File Server, Domain Controllers crashing after the
> datacenter is up and running is handled (or should be) by redundancy of
> servers or process the service provides. You have multiple DNS servers and
> the resolver will try the secondary/tertiary/whatever if the primary is
> down. File servers are the same. For Gluster, CephFS, MS DFS you have (or
> should have setup) the ability to keep running if one of the servers goes
> down. A redundant file server setup will handle a server crash. For Domain
> Controllers you should have at least two (we had six in our environment) and
> when one goes down the others keep the domain running by shifting the
> services to others and continuing to provide authentication, etc. Generally
> what we did when a domain controller crashed is fix it if possible and if it
> was not fixable pull it's pieces out of the domain and spin up a new one.
> Same for DNS or file servers. When they go down find out why, fix it or
> replace the server, and get the service redundant again. Also, oVirt has
> the watchdog function so if a VM goes down it will try and restart it. If
> it can't restart then we're dealing with a crashed server which we should
> have provided for in our data center design.
>
> My wish is to have oVirt allow us to do what VMware does and allow us to say
> start/don't start these servers up in the order I specify so that when the
> Engine is ready it looks at the list and begins turning on VMs in the order
> specified. Shutdown is done in the reverse order). For large datacenters
> with many VMS manual startup is a pain. Once it's running rely on good
> practices of having redundant servers (i.e. more than one DNS), file servers
> that can handle the failure of a server, multiple domain controllers which
> is not something we need to burden oVirt with. Handling of failures needs
> to be done by the people in charge who are supposed to design the data
> center based on their risk/cost analysis.
>
> If I understand what you said - I'm not sure we have to define a dependancy
> list where we define dependancies like we would with systemd or a package
> manger. It doesn't need to be complex since just a simple list of priority:
> VM id/name pairs will work. All we need is to be able to say "start these
> VMs when the Engine is ready". The order or dependency is set by where I
> put the VM in the list. If it's at the top start it first, then move to the
> second one, and so on. There can be settings for delay between starting VMs
> and how long to wait for a VM to come up before assuming it's dead and
> moving on to the next. Tasking oVirt with the job of figuring VM
> dependancies would be a nightmare for oVirt and whoever had to program it
> . We as data center administrators should be handling that task. Yes,
> we manually set the list but trying to automate a dependancy chain would be
> pretty difficult.
>
> I envision a web admin portal GUI where we define a simple list of VMs that
> we want the Engine to autostart. We need to be able to move them up/down the
> list. The list is stored somewhere for the Engine (database?, whatever else
> the Engine has as storage areas - not really familiar with the Engine
> internals) and when the Engine is up and ready to start VMs it retrieves the
> list and starts at the first VM in the list and starts it, waits some time
> (0-?? seconds), and then moves on to the next one. If the VM hasn't started
> in the set time the Engine moves on to the next one.
>
> Note that Microsoft's Hyper-V also provides automatic VM startup but it is
> done on a VM level where you just tell the VM to start whenthe Hypr-V
> starts. If you want sequencing you have to set time delays. Auto start up
> is better than nothing but Hyper-V is a nightmare trying to sequence VMs.
>
> I think VMware did it right in allowing both autostart and being able to
> sequence the startup of VMS so it's host independent. As information VMware
> also allowed delays between VM startups as does Hyper-V.
>
>
> Thanks for the detailed description.
>
> Did you file a bug for this feature?
>
> Adding Moran and Doron.
>
> Nir
>
>
> On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholcomb(a)l1049h.com>
> >
> wrote:
>
> On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote:
>
> On 06/04/16 01:46 -0400, Brett I. Holcomb wrote:
>
> In VMware we could setup guests to autostart when the host started and
> define the order. Is that doable in oVirt? The only thing I've seen
> is the watchdog and tell it to reset but nothing that allows me to
> define who starts up when and if they autostart. I assume it's there
> but I must be missing it or haven't found it in the web portal.
>
>
> In oVirt guests aren't tied to a host by default (although you can set
> them to run only on a specific host if you want). The closest thing I
> can think of would be the High Availability features (VM->Edit).
> oVirt will try to restart highly available VMs if they go down. You
> can also set the priority for migration and restart in that pane.
> Hopefully a combination of host pinning and the high availability
> settings will get you close enough to where you want to be.
>
> Otherwise, you could always do some scripting with the ovirt REST API
> using the SDK or CLI.
>
>
> If you had the VMware migration extra add-on you could have hosts move as
> needed so they were not tied to any host either but we could set a startup
> order and specify auto, manual so that once the host started the VMs were
> brought up as specified no matter what host they were running on.
>
> I am running hosted-engine deployment with the Engine VM on the host.
>
> I set highly available on, did not pin to any host, and also set the
> watchdog which should reset if they go down but I'm not sure that will start
> them if the host comes up and the VMs are not running. I'll look at the CLI
> first.
>
> It would be nice if oVirt added this feature as it's really required for
> large installations and is a help for any size installation, even small
> ones.
>
>
>
>
> _______________________________________________
> Users mailing list
>
Users(a)ovirt.org> >
http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
>
Users(a)ovirt.org> >
http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
>
Users(a)ovirt.org> >
http://lists.ovirt.org/mailman/listinfo/users
>
>
>
2957
days inactive
2958
days old
5 comments
2 participants
participants (2)
-
Brett I. Holcomb -
Nir Soffer