Unfortunately, I haven't found anything like this.
At least this type of event didn't happen again, but I just can't find
anything in the logs that justifies these reboots, I've read everything
line by line, if anything was recorded there it passed my eyes.
Em qua., 30 de nov. de 2022 às 14:20, Volenbovskyi, Konstantin <
Konstantin.Volenbovskyi(a)haufe.com> escreveu:
Hi,
“I'm assuming it used ssh as the hosts in question have a configuration
problem with their power management and cannot be reset currently by the PDU
”
It is interesting to find the events in logs on ovengine really doing
reboot of ovirt hosts via SSH in this case/any cases.
I guess that /var/log/messages and probably some ovengine logs should
provide evidence if it is the case (and ovengine logs can contain
additional information, what is functionality behind
‘restart host that lost a connection to storage domain’
BR,
Konstantin
*From: *Patrick Hibbs <hibbsncc1701(a)gmail.com>
*Date: *Wednesday, 30 November 2022 at 16:31
*To: *"users(a)ovirt.org" <users(a)ovirt.org>
*Subject: *[ovirt-users] Re: Forced restart when losing communication
with the Storages
Hello,
I've seen something similar to this too. Although, for me it occurred when
a standalone engine attempted to allocate a new disk on a Gluster storage.
While the cluster's VMs were experiencing high virtual disk I/O. (Found out
later they were doing updates at an odd time...)
The result was random VMs being forced off until it had cleared enough of
the bottleneck, and one host was rebooted. After around 3 minutes of wait
time. I'm assuming it used ssh as the hosts in question have a
configuration problem with their power management and cannot be reset
currently by the PDU. But it was still an odd occurrence given that the
engine host itself was the cause of the storage "outage."
Is this the correct behavior of oVirt?
-Patrick Hibbs
On 11/30/22 07:45, Murilo Morais wrote:
Konstantin, thank you very much for the explanation, it was very
enlightening.
I believe I left something open in the previous message.
I'm using Hosted Engine, all VMs have HA enabled and Power Management is
disabled on all hosts. No IPMI configured (at least I didn't configure
anything about iLO/IPMI in oVirt).
There was a loss of communication with the Storage for approximately 3
minutes and this caused all Hosts to reboot.
Em qua., 30 de nov. de 2022 às 08:50, Volenbovskyi, Konstantin <
Konstantin.Volenbovskyi(a)haufe.com> escreveu:
Hi,
I would say that you observed ‘fencing’ and not SSH soft fencing, but
actual reboot via IPMI.
https://www.ovirt.org/develop/developer-guide/engine/automatic-fencing.html
You can disable Power management for hosts.
Before doing that you need to understand following:
-what is impact on VMs when this happens?
-the working assumption is that your VMs
work just fine, but you need to think about other cases where VMs lose
their storage and/or network.
For me it seems that this was storage domain that is not a VM storage
domain, so VMs’ disks were just fine.
Maybe it was hosted_storage domain in your case…
-any of those VMs are High-availability VMs? Once you
disable Power Management you will not have automatic restart on different
hosts of those.
You need to understand that idea of fencing is either to recover host
automatically and possibly to restart VMs
and make sure that there are no duplicated VMs.
There are 100% cases where fencing is used and there is subset of those,
X% number of cases where you would consider that behavior is suboptimal.
The drawback of disabling fencing is that you might get suboptimal
behavior in Y% cases (100% minus X%)
BR,
Konstantin
*From: *Murilo Morais <murilo(a)evocorp.com.br>
*Date: *Wednesday, 30 November 2022 at 12:13
*To: *users <users(a)ovirt.org>
*Subject: *[ovirt-users] Forced restart when losing communication with
the Storages
Good morning everyone!
Is there a way to disable the forced reboot of the machines? This morning
there was an event in our infrastructure where the hosts lost communication
with the Storage but this caused all the hosts to restart abruptly.
Would this be the correct behavior of oVirt? Is there any way to disable
this?
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement:
https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JVNIMYBAXJE...
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement:
https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/YDEO4442DAN...