7:07 p.m.
I have a dual host, self hosted engine, cluster/datacenter. I call these Node1 and Node2.
They are running CentOS 7 and I have been running the updates regular by putting each host
into maintenance mode and installing the update via the web gui.
Node1 is restarting at odd times, causing VMs on that node to start up on Node2 (which is
by design). However the restarts are becoming more frequent and I am having a hard time
figuring out what is causing it. Below is a snippet from the
/var/log/ovirt-hosted-engine-ha/agent.log on Node1. My current plan is to simply rebuild
Node1 by re-installing CentOS 7, and using the cluster tools to import it.
MainThread::INFO::2018-09-10
10:04:25,907::hosted_engine::244::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
Found certificate common name: node1.***.com
MainThread::INFO::2018-09-10
10:04:34,200::hosted_engine::522::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
Initializing ha-broker connection
MainThread::INFO::2018-09-10
10:04:34,202::brokerlink::77::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
Starting monitor ping, options {'addr': '192.168.4.1'}
MainThread::ERROR::2018-09-10
10:04:34,203::hosted_engine::538::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
Failed to start necessary monitors
MainThread::ERROR::2018-09-10
10:04:34,203::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback
(most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 131, in _run_agent
return action(he)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 55, in action_proper
return he.start_monitoring()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 413, in start_monitoring
self._initialize_broker()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 535, in _initialize_broker
m.get('options', {}))
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 83, in start_monitor
.format(type, options, e))
RequestError: Failed to start monitor ping, options {'addr':
'192.168.4.1'}: [Errno 2] No such file or directory
MainThread::ERROR::2018-09-10
10:04:34,204::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to
restart agent
MainThread::INFO::2018-09-10
10:04:34,204::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting
down
MainThread::INFO::2018-09-10
10:04:44,545::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
ovirt-hosted-engine-ha agent 2.2.16 started
MainThread::INFO::2018-09-10
10:04:44,583::hosted_engine::244::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
Found certificate common name: node1.***.com
MainThread::INFO::2018-09-10
10:04:44,744::hosted_engine::522::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
Initializing ha-broker connection
MainThread::INFO::2018-09-10
10:04:44,745::brokerlink::77::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
Starting monitor ping, options {'addr': '192.168.4.1'}
MainThread::ERROR::2018-09-10
10:04:44,746::hosted_engine::538::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
Failed to start necessary monitors
MainThread::ERROR::2018-09-10
10:04:44,746::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback
(most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 131, in _run_agent
return action(he)
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
line 55, in action_proper
return he.start_monitoring()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 413, in start_monitoring
self._initialize_broker()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 535, in _initialize_broker
m.get('options', {}))
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 83, in start_monitor
.format(type, options, e))
RequestError: Failed to start monitor ping, options {'addr':
'192.168.4.1'}: [Errno 2] No such file or directory
1:58 p.m.
We originally noticed an issue when the host would restart at midnight on Saturdays EST
time. The other day it restarted at 10AM EST which was a surprise.
The system is under low load and no hardware issues were found in iDRAC nor in kernel
messages.
5:33 p.m.
Can you please attach also /var/log/sanlock.log and /var/log/messages for
the relevant time frame?
On Wed, Sep 12, 2018 at 1:59 PM <turboaaa(a)gmail.com> wrote:
We originally noticed an issue when the host would restart at
midnight on
Saturdays EST time. The other day it restarted at 10AM EST which was a
surprise.
The system is under low load and no hardware issues were found in iDRAC
nor in kernel messages.
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/37CJPQ6G3V5...
12:06 a.m.
This is a stupid question, but how do I attach a file to this post? I am using the web
form, as soon as I figure out this mailing list thing I will attach full logs
(NOOOOB!!!!)
Regardless, there are no logs before the reboot of Node1 in sanlock.log on that day. Node1
started coming back online at 10:04AM.
2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
Messages has more information, and when I figure out how to attach files I will post it.
Here are the entries around the reboot time. The node rebooted between 09:58:52 and
10:00:01
Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513: error :
qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest agent is not
connected
Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514: error :
qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest agent is not
connected
Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to revision 0x42d, date =
2018-04-25
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64
(builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) )
#1 SMP Tue Aug 14 21:49:04 UTC 2018
Sep 10 10:03:51 node1 kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64
root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root
rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8
Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
10:56 a.m.
On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com> wrote:
This is a stupid question, but how do I attach a file to this post? I
am
using the web form, as soon as I figure out this mailing list thing I will
attach full logs (NOOOOB!!!!)
Sorry, which web form?
This is a mailing list: users(a)ovirt.org
Regardless, there are no logs before the reboot of Node1 in sanlock.log on
that day. Node1 started coming back online at 10:04AM.
2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
Messages has more information, and when I figure out how to attach files I
will post it. Here are the entries around the reboot time. The node
rebooted between 09:58:52 and 10:00:01
Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513: error
: qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest
agent is not connected
Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514: error
: qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest
agent is not connected
Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to
revision 0x42d, date = 2018-04-25
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64 (
builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
Sep 10 10:03:51 node1 kernel: Command line:
BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos-root
ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb
quiet LANG=en_US.UTF-8
Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7I26QE6OC4...
1:41 p.m.
I have never participated in a mailing list. In this case I have been using
the following web form.
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/AXONQYNLZHO7...
But for this reply I am simply replying to the email in my inbox, will see
if it shows up. I am used to forums where I get an email notification, but
need to log in to add content.
On Thu, Sep 13, 2018 at 4:57 AM Simone Tiraboschi <stirabos(a)redhat.com>
wrote:
On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com> wrote:
> This is a stupid question, but how do I attach a file to this post? I am
> using the web form, as soon as I figure out this mailing list thing I will
> attach full logs (NOOOOB!!!!)
>
Sorry, which web form?
This is a mailing list: users(a)ovirt.org
>
> Regardless, there are no logs before the reboot of Node1 in sanlock.log
> on that day. Node1 started coming back online at 10:04AM.
>
> 2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
> f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
> 2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
> edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
>
> Messages has more information, and when I figure out how to attach files
> I will post it. Here are the entries around the reboot time. The node
> rebooted between 09:58:52 and 10:00:01
>
> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513: error
> : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest
> agent is not connected
> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514: error
> : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU guest
> agent is not connected
> Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
> Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
> Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to
> revision 0x42d, date = 2018-04-25
> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
> Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64 (
> builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
> 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
> Sep 10 10:03:51 node1 kernel: Command line:
> BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos-root
> ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb
> quiet LANG=en_US.UTF-8
> Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
>
List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7I26QE6OC4...
>
--
-
Michael Mast
2:16 p.m.
My last post worked, good to know. As an FYI, on the ovirt.org website
there is a "Forum" link at the top right hand side that takes me to the
mailing list. With the web form I figured it was more or less just a
standard forum. But now I know better lol
Please see attached for log files.
On Thu, Sep 13, 2018 at 7:41 AM Michael Mast <turboaaa(a)gmail.com> wrote:
I have never participated in a mailing list. In this case I have
been
using the following web form.
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/AXONQYNLZHO7...
But for this reply I am simply replying to the email in my inbox, will see
if it shows up. I am used to forums where I get an email notification, but
need to log in to add content.
On Thu, Sep 13, 2018 at 4:57 AM Simone Tiraboschi <stirabos(a)redhat.com>
wrote:
>
>
> On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com> wrote:
>
>> This is a stupid question, but how do I attach a file to this post? I am
>> using the web form, as soon as I figure out this mailing list thing I will
>> attach full logs (NOOOOB!!!!)
>>
>
> Sorry, which web form?
> This is a mailing list: users(a)ovirt.org
>
>
>>
>> Regardless, there are no logs before the reboot of Node1 in sanlock.log
>> on that day. Node1 started coming back online at 10:04AM.
>>
>> 2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
>> f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
>> 2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
>> edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
>>
>> Messages has more information, and when I figure out how to attach files
>> I will post it. Here are the entries around the reboot time. The node
>> rebooted between 09:58:52 and 10:00:01
>>
>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513:
>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU
>> guest agent is not connected
>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514:
>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU
>> guest agent is not connected
>> Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
>> Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
>> Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to
>> revision 0x42d, date = 2018-04-25
>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
>> Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64 (
>> builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
>> 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
>> Sep 10 10:03:51 node1 kernel: Command line:
>> BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos-root
>> ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb
>> quiet LANG=en_US.UTF-8
>> Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
>
>
>> _______________________________________________
>> Users mailing list -- users(a)ovirt.org
>> To unsubscribe send an email to users-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>>
> List Archives:
>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7I26QE6OC4...
>>
> --
-
Michael Mast
--
-
Michael Mast
4:14 p.m.
Anything strange in sanlock logs before the reboot?
On Thu, Sep 13, 2018 at 2:16 PM Michael Mast <turboaaa(a)gmail.com> wrote:
My last post worked, good to know. As an FYI, on the ovirt.org
website
there is a "Forum" link at the top right hand side that takes me to the
mailing list. With the web form I figured it was more or less just a
standard forum. But now I know better lol
Please see attached for log files.
On Thu, Sep 13, 2018 at 7:41 AM Michael Mast <turboaaa(a)gmail.com> wrote:
> I have never participated in a mailing list. In this case I have been
> using the following web form.
>
>
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/AXONQYNLZHO7...
> But for this reply I am simply replying to the email in my inbox, will
> see if it shows up. I am used to forums where I get an email notification,
> but need to log in to add content.
>
> On Thu, Sep 13, 2018 at 4:57 AM Simone Tiraboschi <stirabos(a)redhat.com>
> wrote:
>
>>
>>
>> On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com> wrote:
>>
>>> This is a stupid question, but how do I attach a file to this post? I
>>> am using the web form, as soon as I figure out this mailing list thing I
>>> will attach full logs (NOOOOB!!!!)
>>>
>>
>> Sorry, which web form?
>> This is a mailing list: users(a)ovirt.org
>>
>>
>>>
>>> Regardless, there are no logs before the reboot of Node1 in sanlock.log
>>> on that day. Node1 started coming back online at 10:04AM.
>>>
>>> 2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
>>> f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
>>> 2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
>>> edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
>>>
>>> Messages has more information, and when I figure out how to attach
>>> files I will post it. Here are the entries around the reboot time. The node
>>> rebooted between 09:58:52 and 10:00:01
>>>
>>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513:
>>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU
>>> guest agent is not connected
>>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514:
>>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding: QEMU
>>> guest agent is not connected
>>> Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
>>> Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
>>> Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to
>>> revision 0x42d, date = 2018-04-25
>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
>>> Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64 (
>>> builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat
>>> 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
>>> Sep 10 10:03:51 node1 kernel: Command line:
>>> BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos-root
>>> ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb
>>> quiet LANG=en_US.UTF-8
>>> Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
>>
>>
>>> _______________________________________________
>>> Users mailing list -- users(a)ovirt.org
>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>>
>> List Archives:
>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7I26QE6OC4...
>>>
>> --
> -
> Michael Mast
>
--
-
Michael Mast
4:18 p.m.
No. The log file literally jumps from the 8th to the 10th. The first entry
on the 10th was when the host booted back up.
On Thu, Sep 13, 2018, 10:14 Simone Tiraboschi <stirabos(a)redhat.com> wrote:
Anything strange in sanlock logs before the reboot?
On Thu, Sep 13, 2018 at 2:16 PM Michael Mast <turboaaa(a)gmail.com> wrote:
> My last post worked, good to know. As an FYI, on the ovirt.org website
> there is a "Forum" link at the top right hand side that takes me to the
> mailing list. With the web form I figured it was more or less just a
> standard forum. But now I know better lol
>
> Please see attached for log files.
>
> On Thu, Sep 13, 2018 at 7:41 AM Michael Mast <turboaaa(a)gmail.com> wrote:
>
>> I have never participated in a mailing list. In this case I have been
>> using the following web form.
>>
>>
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/AXONQYNLZHO7...
>> But for this reply I am simply replying to the email in my inbox, will
>> see if it shows up. I am used to forums where I get an email notification,
>> but need to log in to add content.
>>
>> On Thu, Sep 13, 2018 at 4:57 AM Simone Tiraboschi <stirabos(a)redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com> wrote:
>>>
>>>> This is a stupid question, but how do I attach a file to this post? I
>>>> am using the web form, as soon as I figure out this mailing list thing I
>>>> will attach full logs (NOOOOB!!!!)
>>>>
>>>
>>> Sorry, which web form?
>>> This is a mailing list: users(a)ovirt.org
>>>
>>>
>>>>
>>>> Regardless, there are no logs before the reboot of Node1 in
>>>> sanlock.log on that day. Node1 started coming back online at 10:04AM.
>>>>
>>>> 2018-09-08 09:19:38 3081 [1127]: s1 host 2 16 249
>>>> f9704a0b-9a09-42f4-819c-35055c1177af.node2.<host>
>>>> 2018-09-10 10:03:54 5 [1154]: sanlock daemon started 3.6.0 host
>>>> edfe4d27-bdbe-45e3-9e4c-a4e02b2101bd.node1.<host>
>>>>
>>>> Messages has more information, and when I figure out how to attach
>>>> files I will post it. Here are the entries around the reboot time. The
node
>>>> rebooted between 09:58:52 and 10:00:01
>>>>
>>>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2513:
>>>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding:
QEMU
>>>> guest agent is not connected
>>>> Sep 10 09:58:52 node1 libvirtd: 2018-09-10 13:58:52.405+0000: 2514:
>>>> error : qemuDomainAgentAvailable:6946 : Guest agent is not responding:
QEMU
>>>> guest agent is not connected
>>>> Sep 10 10:00:01 node1 systemd: Started Session 353 of user root.
>>>> Sep 10 10:00:01 node1 systemd: Starting Session 353 of user root.
>>>> Sep 10 10:03:51 node1 kernel: microcode: microcode updated early to
>>>> revision 0x42d, date = 2018-04-25
>>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuset
>>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpu
>>>> Sep 10 10:03:51 node1 kernel: Initializing cgroup subsys cpuacct
>>>> Sep 10 10:03:51 node1 kernel: Linux version 3.10.0-862.11.6.el7.x86_64
>>>> (builder(a)kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red
>>>> Hat 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
>>>> Sep 10 10:03:51 node1 kernel: Command line:
>>>> BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64
root=/dev/mapper/centos-root
>>>> ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb
>>>> quiet LANG=en_US.UTF-8
>>>> Sep 10 10:03:51 node1 kernel: e820: BIOS-provided physical RAM map:
>>>
>>>
>>>> _______________________________________________
>>>> Users mailing list -- users(a)ovirt.org
>>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>
>>> List Archives:
>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7I26QE6OC4...
>>>>
>>> --
>> -
>> Michael Mast
>>
> --
> -
> Michael Mast
>
--
-
Michael Mast
10:08 p.m.
We had it happen again, but this time on the other host. This started
happening not long after I tried to update the engine, and I think I
screwed that up. The instructions I was working off of was outdated and a
lot of the links on the ovirt site are broken. My theory is that the
engines were not "synced" since a copy runs on each host. This would cause
the engine on one host to come online and think the other is not
responding, at this point it forces the other host offline using iDRAC
(This is in the logs how it reboots the other hsot) and forces all VMs over.
So what I did was remove one host from the cluster, re-install CentOS7, and
re-add it to the cluster. This way the engine config would be copied from
the primary host and keep them synced. If this does not work the next step
is to convert the hosted engine into a dedicated engine. Are there any good
guides for making this conversion?
--
-
Michael Mast
2:28 a.m.
Use the backup and restore procedure to move your engine maybe
On Thu, Sep 27, 2018 at 4:08 PM Michael Mast <turboaaa(a)gmail.com> wrote:
We had it happen again, but this time on the other host. This
started
happening not long after I tried to update the engine, and I think I
screwed that up. The instructions I was working off of was outdated and a
lot of the links on the ovirt site are broken. My theory is that the
engines were not "synced" since a copy runs on each host. This would cause
the engine on one host to come online and think the other is not
responding, at this point it forces the other host offline using iDRAC
(This is in the logs how it reboots the other hsot) and forces all VMs over.
So what I did was remove one host from the cluster, re-install CentOS7,
and re-add it to the cluster. This way the engine config would be copied
from the primary host and keep them synced. If this does not work the next
step is to convert the hosted engine into a dedicated engine. Are there any
good guides for making this conversion?
--
-
Michael Mast
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/2EDXBKVNRMF...
2:29 a.m.
I figured as much, but wasn't sure if there would be any changes needed to
the database so the engine will know it is no longer self hosted.
On Thu, Sep 27, 2018, 20:28 Donny Davis <donny(a)fortnebula.com> wrote:
Use the backup and restore procedure to move your engine maybe
On Thu, Sep 27, 2018 at 4:08 PM Michael Mast <turboaaa(a)gmail.com> wrote:
> We had it happen again, but this time on the other host. This started
> happening not long after I tried to update the engine, and I think I
> screwed that up. The instructions I was working off of was outdated and a
> lot of the links on the ovirt site are broken. My theory is that the
> engines were not "synced" since a copy runs on each host. This would cause
> the engine on one host to come online and think the other is not
> responding, at this point it forces the other host offline using iDRAC
> (This is in the logs how it reboots the other hsot) and forces all VMs over.
>
> So what I did was remove one host from the cluster, re-install CentOS7,
> and re-add it to the cluster. This way the engine config would be copied
> from the primary host and keep them synced. If this does not work the next
> step is to convert the hosted engine into a dedicated engine. Are there any
> good guides for making this conversion?
> --
> -
> Michael Mast
>
_______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/2EDXBKVNRMF...
>
--
-
Michael Mast
2:42 a.m.
I am sure there would be some challenges there because you cannot delete
that vm, its protected... However the rest of your stuff should run just
fine. I have never done it myself, most people head the opposite direction.
On Thu, Sep 27, 2018 at 8:31 PM Michael Mast <turboaaa(a)gmail.com> wrote:
I figured as much, but wasn't sure if there would be any changes
needed to
the database so the engine will know it is no longer self hosted.
On Thu, Sep 27, 2018, 20:28 Donny Davis <donny(a)fortnebula.com> wrote:
> Use the backup and restore procedure to move your engine maybe
>
> On Thu, Sep 27, 2018 at 4:08 PM Michael Mast <turboaaa(a)gmail.com> wrote:
>
>> We had it happen again, but this time on the other host. This started
>> happening not long after I tried to update the engine, and I think I
>> screwed that up. The instructions I was working off of was outdated and a
>> lot of the links on the ovirt site are broken. My theory is that the
>> engines were not "synced" since a copy runs on each host. This would
cause
>> the engine on one host to come online and think the other is not
>> responding, at this point it forces the other host offline using iDRAC
>> (This is in the logs how it reboots the other hsot) and forces all VMs over.
>>
>> So what I did was remove one host from the cluster, re-install CentOS7,
>> and re-add it to the cluster. This way the engine config would be copied
>> from the primary host and keep them synced. If this does not work the next
>> step is to convert the hosted engine into a dedicated engine. Are there any
>> good guides for making this conversion?
>> --
>> -
>> Michael Mast
>>
> _______________________________________________
>> Users mailing list -- users(a)ovirt.org
>> To unsubscribe send an email to users-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct:
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/2EDXBKVNRMF...
>>
> --
-
Michael Mast
3:14 a.m.
Thanks. I will create a new thread for that, I'm getting off topic.
What are the top reasons one host would think the other is offline?
On Thu, Sep 27, 2018, 20:42 Donny Davis <donny(a)fortnebula.com> wrote:
I am sure there would be some challenges there because you cannot
delete
that vm, its protected... However the rest of your stuff should run just
fine. I have never done it myself, most people head the opposite direction.
On Thu, Sep 27, 2018 at 8:31 PM Michael Mast <turboaaa(a)gmail.com> wrote:
> I figured as much, but wasn't sure if there would be any changes needed
> to the database so the engine will know it is no longer self hosted.
>
> On Thu, Sep 27, 2018, 20:28 Donny Davis <donny(a)fortnebula.com> wrote:
>
>> Use the backup and restore procedure to move your engine maybe
>>
>> On Thu, Sep 27, 2018 at 4:08 PM Michael Mast <turboaaa(a)gmail.com> wrote:
>>
>>> We had it happen again, but this time on the other host. This started
>>> happening not long after I tried to update the engine, and I think I
>>> screwed that up. The instructions I was working off of was outdated and a
>>> lot of the links on the ovirt site are broken. My theory is that the
>>> engines were not "synced" since a copy runs on each host. This
would cause
>>> the engine on one host to come online and think the other is not
>>> responding, at this point it forces the other host offline using iDRAC
>>> (This is in the logs how it reboots the other hsot) and forces all VMs over.
>>>
>>> So what I did was remove one host from the cluster, re-install CentOS7,
>>> and re-add it to the cluster. This way the engine config would be copied
>>> from the primary host and keep them synced. If this does not work the next
>>> step is to convert the hosted engine into a dedicated engine. Are there any
>>> good guides for making this conversion?
>>> --
>>> -
>>> Michael Mast
>>>
>> _______________________________________________
>>> Users mailing list -- users(a)ovirt.org
>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/2EDXBKVNRMF...
>>>
>> --
> -
> Michael Mast
>
--
-
Michael Mast
3:38 p.m.
On Thu, Sep 13, 2018 at 12:07 AM <turboaaa(a)gmail.com>
wrote:
Sorry, which web form?
This is a mailing list: users(a)ovirt.org
I think it's referring to the
Hyperkitty web interface.
Regards,
Paul S.
2057
days inactive
2075
days old
15 comments
5 participants
participants (5)
-
Donny Davis -
Michael Mast -
p.staniforth＠leedsbeckett.ac.uk -
Simone Tiraboschi -
turboaaa＠gmail.com