6:11 p.m.
This is a multi-part message in MIME format.
--------------7A7A94A488D7D26C7D1B2EB5
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Hello.
Yesterday I had a pretty strange problem in one of our architectures. My
oVirt which runs in one Datacenter and controls Nodes locally and also
remotelly lost communication with the remote Nodes in another Datacenter.
To this point nothing wrong as the Nodes can continue working as
expected and running their Virtual Machines each without dependency of
the oVirt Engine.
What happened at some point is that when the communication between
Engine and Hosts came back Hosts got confused and initiated a Live
Migration of ALL VMs from one of the other. I had also to restart vdsmd
agent on all Hosts in order to get sanity my environment.
What adds up even more strangeness to this scenario is that one of the
Hosts affected doesn't belong to the same Cluster as the others and had
to have the vdsmd restarted.
I understand the Hosts can survive without the Engine online with
reduced possibilities but can communicated between them, but without
affecting the VMs or even needing to do what happened in this scenario.
Am I wrong on any of the assumptions ?
Fernando
--------------7A7A94A488D7D26C7D1B2EB5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font face="arial, helvetica, sans-serif">Hello.<br>
<br>
Yesterday I had a pretty strange problem in one of our
architectures. My oVirt which runs in one Datacenter and controls
Nodes locally and also remotelly lost communication with the
remote Nodes in another Datacenter.<br>
To this point nothing wrong as the Nodes can continue working as
expected and running their Virtual Machines each without
dependency of the oVirt Engine.<br>
<br>
What happened at some point is that when the communication between
Engine and Hosts came back Hosts got confused and initiated a Live
Migration of ALL VMs from one of the other. I had also to restart
vdsmd agent on all Hosts in order to get sanity my environment.<br>
What adds up even more strangeness to this scenario is that one of
the Hosts affected doesn't belong to the same Cluster as the
others and had to have the vdsmd restarted.<br>
<br>
I understand the Hosts can survive without the Engine online with
reduced possibilities but can communicated between them, but
without affecting the VMs or even needing to do what happened in
this scenario.<br>
<br>
Am I wrong on any of the assumptions ?<br>
<br>
Fernando<br>
</font>
</body>
</html>
--------------7A7A94A488D7D26C7D1B2EB5--
1 p.m.
Fernando,
Which ovirt version are you running? Please share the logs so I could
check what caused the hosts to become unresponsive.
Thanks,
Piotr
On Wed, Aug 2, 2017 at 5:11 PM, FERNANDO FREDIANI
<fernando.frediani(a)upx.com> wrote:
Hello.
Yesterday I had a pretty strange problem in one of our architectures. My
oVirt which runs in one Datacenter and controls Nodes locally and also
remotelly lost communication with the remote Nodes in another Datacenter.
To this point nothing wrong as the Nodes can continue working as expected
and running their Virtual Machines each without dependency of the oVirt
Engine.
What happened at some point is that when the communication between Engine
and Hosts came back Hosts got confused and initiated a Live Migration of ALL
VMs from one of the other. I had also to restart vdsmd agent on all Hosts in
order to get sanity my environment.
What adds up even more strangeness to this scenario is that one of the Hosts
affected doesn't belong to the same Cluster as the others and had to have
the vdsmd restarted.
I understand the Hosts can survive without the Engine online with reduced
possibilities but can communicated between them, but without affecting the
VMs or even needing to do what happened in this scenario.
Am I wrong on any of the assumptions ?
Fernando
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
4:20 p.m.
Hello Piotr. Thanks for your reply
I was running version 4.1.1, but since that day I have upgraded to 4.1.5
(the Engine because the hosts remain on 4.1.1). I am not sure the logs
still exists (how long they are kept normally).
Just to clarify the hosts didn't become unresponsive, but the
communication between the Engine and the Hosts in question (each in a
different Datacenter was interrupted - but locally the hosts were fine
and accessible). What was strange was that since the Hosts could not
talk to the Engine they seem to have got 'confused' and started several
VM live migrations which was not expected. As a note I don't have any
Fencing policy enabled.
Regards
Fernando
On 16/08/2017 07:00, Piotr Kliczewski wrote:
Fernando,
Which ovirt version are you running? Please share the logs so I could
check what caused the hosts to become unresponsive.
Thanks,
Piotr
On Wed, Aug 2, 2017 at 5:11 PM, FERNANDO FREDIANI
<fernando.frediani(a)upx.com> wrote:
> Hello.
>
> Yesterday I had a pretty strange problem in one of our architectures. My
> oVirt which runs in one Datacenter and controls Nodes locally and also
> remotelly lost communication with the remote Nodes in another Datacenter.
> To this point nothing wrong as the Nodes can continue working as expected
> and running their Virtual Machines each without dependency of the oVirt
> Engine.
>
> What happened at some point is that when the communication between Engine
> and Hosts came back Hosts got confused and initiated a Live Migration of ALL
> VMs from one of the other. I had also to restart vdsmd agent on all Hosts in
> order to get sanity my environment.
> What adds up even more strangeness to this scenario is that one of the Hosts
> affected doesn't belong to the same Cluster as the others and had to have
> the vdsmd restarted.
>
> I understand the Hosts can survive without the Engine online with reduced
> possibilities but can communicated between them, but without affecting the
> VMs or even needing to do what happened in this scenario.
>
> Am I wrong on any of the assumptions ?
>
> Fernando
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
5:01 p.m.
Fernando,
Do you know how log it took when you had connection issues between
data centers? Please collect the logs when it will happen again.
Thanks,
Piotr
On Wed, Aug 16, 2017 at 3:20 PM, FERNANDO FREDIANI
<fernando.frediani(a)upx.com> wrote:
Hello Piotr. Thanks for your reply
I was running version 4.1.1, but since that day I have upgraded to 4.1.5
(the Engine because the hosts remain on 4.1.1). I am not sure the logs still
exists (how long they are kept normally).
Just to clarify the hosts didn't become unresponsive, but the communication
between the Engine and the Hosts in question (each in a different Datacenter
was interrupted - but locally the hosts were fine and accessible). What was
strange was that since the Hosts could not talk to the Engine they seem to
have got 'confused' and started several VM live migrations which was not
expected. As a note I don't have any Fencing policy enabled.
Regards
Fernando
On 16/08/2017 07:00, Piotr Kliczewski wrote:
>
> Fernando,
>
> Which ovirt version are you running? Please share the logs so I could
> check what caused the hosts to become unresponsive.
>
> Thanks,
> Piotr
>
> On Wed, Aug 2, 2017 at 5:11 PM, FERNANDO FREDIANI
> <fernando.frediani(a)upx.com> wrote:
>>
>> Hello.
>>
>> Yesterday I had a pretty strange problem in one of our architectures. My
>> oVirt which runs in one Datacenter and controls Nodes locally and also
>> remotelly lost communication with the remote Nodes in another Datacenter.
>> To this point nothing wrong as the Nodes can continue working as expected
>> and running their Virtual Machines each without dependency of the oVirt
>> Engine.
>>
>> What happened at some point is that when the communication between Engine
>> and Hosts came back Hosts got confused and initiated a Live Migration of
>> ALL
>> VMs from one of the other. I had also to restart vdsmd agent on all Hosts
>> in
>> order to get sanity my environment.
>> What adds up even more strangeness to this scenario is that one of the
>> Hosts
>> affected doesn't belong to the same Cluster as the others and had to have
>> the vdsmd restarted.
>>
>> I understand the Hosts can survive without the Engine online with reduced
>> possibilities but can communicated between them, but without affecting
>> the
>> VMs or even needing to do what happened in this scenario.
>>
>> Am I wrong on any of the assumptions ?
>>
>> Fernando
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
2656
days inactive
2670
days old
3 comments
2 participants
participants (2)
-
FERNANDO FREDIANI -
Piotr Kliczewski