Do you have power management configured? Was the "failed" host fenced/rebooted? =20 =20 On Fri, Apr 4, 2014 at 2:21 PM, Koen Vanoppen = <vanoppen.koen@gmail.com> wrote: So... It is possible for a fully automatic migration of the VM to = another hypervisor in case Storage connection fails? How can we make this happen? Because for the moment, when we tested =
--Apple-Mail=_288231F1-4E67-445C-BD5C-619AAD0235E3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 4 Apr 2014, at 15:14, Sander Grendelman wrote: the situation they stayed in pause state.
(Test situation: Unplug the 2 fibre cables from the hypervisor VM's go in pause state VM's stayed in pause state until the failure was solved
as said before, it's not safe hence we (try to) not migrate them. They only get paused when they actually access the storage which may not = be always the case. I.e. the storage connection is severed, host deemed = NonOperational and VMs are getting migrated from it, then some of them = will succeed if they didn't access that "bad" storage =85 the paused VMs = will remain (mostly, it can still happen that they appear paused = migrated on other host when the disk access occurs only at the last = stage of migration) so in other words, if you want to migrate the VMs without interruption = it's not sometimes possible if you are fine with the VMs restarted in short time on other host then = power management/fencing will help here Thanks, michal
) =20 =20 =20 They only returned when we restored the fiber connection to the = Hypervisor=85
=20 Kind Regards, =20 Koen =20 =20 =20 2014-04-04 13:52 GMT+02:00 Koen Vanoppen <vanoppen.koen@gmail.com>: So... It is possible for a fully automatic migration of the VM to = another hypervisor in case Storage connection fails? How can we make this happen? Because for the moment, when we tested =
(Test situation: Unplug the 2 fibre cables from the hypervisor VM's go in pause state VM's stayed in pause state until the failure was solved ) =20 =20 =20 They only returned when we restored the fiber connection to the = Hypervisor... =20 Kind Regards, =20 Koen =20 =20 2014-04-03 16:53 GMT+02:00 Koen Vanoppen <vanoppen.koen@gmail.com>: =20 ---------- Forwarded message ---------- From: "Doron Fediuck" <dfediuck@redhat.com> Date: Apr 3, 2014 4:51 PM Subject: Re: [Users] HA To: "Koen Vanoppen" <vanoppen.koen@gmail.com> Cc: "Omer Frenkel" <ofrenkel@redhat.com>, <users@ovirt.org>, "Federico = Simoncelli" <fsimonce@redhat.com>, "Allon Mureinik" = <amureini@redhat.com> =20 =20 =20 ----- Original Message -----
From: "Koen Vanoppen" <vanoppen.koen@gmail.com> To: "Omer Frenkel" <ofrenkel@redhat.com>, users@ovirt.org Sent: Wednesday, April 2, 2014 4:17:36 PM Subject: Re: [Users] HA
Yes, indeed. I meant not-operational. Sorry. So, if I understand this correctly. When we ever come in a situation =
yes, since 3.3 we have the autoresume feature Thanks, michal the situation they stayed in pause state. that we
loose both storage connections on our hypervisor, we will have to = manually restore the connections first?
And thanx for the tip for speeding up thins :-).
Kind regards,
Koen
2014-04-02 15:14 GMT+02:00 Omer Frenkel < ofrenkel@redhat.com > :
----- Original Message -----
From: "Koen Vanoppen" < vanoppen.koen@gmail.com > To: users@ovirt.org Sent: Wednesday, April 2, 2014 4:07:19 PM Subject: [Users] HA
Dear All,
Due our acceptance testing, we discovered something. (Document = will follow). When we disable one fiber path, no problem multipath finds it way = no pings are lost. BUT when we disabled both the fiber paths (so one of the storage = domain is gone on this host, but still available on the other host), vms go = in paused mode... He chooses a new SPM (can we speed this up?), put's the = host in non-responsive (can we speed this up, more important) and the VM's = stay on Paused mode... I would expect that they would be migrated (yes, HA = is
i guess you mean the host moves to not-operational (in contrast to non-responsive)? if so, the engine will not migrate vms that are paused to do io = error, because of data corruption risk.
to speed up you can look at the storage domain monitoring timeout: engine-config --get StorageDomainFalureTimeoutInMinutes
enabled) to the other host and reboot there... Any solution? We = are still using oVirt 3.3.1 , but we are planning a upgrade to 3.4 after the = easter holiday.
Kind Regards,
Koen
=20 Hi Koen, Resuming from paused due to io issues is supported (adding relevant = folks). Regardless, if you did not define power management, you should = manually approve source host was rebooted in order for migration to proceed. Otherwise = we risk split-brain scenario. =20 Doron =20 =20 =20
Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users =20 =20 _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_288231F1-4E67-445C-BD5C-619AAD0235E3 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 <html><head></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = "><br><div><div>On 4 Apr 2014, at 15:14, Sander Grendelman = wrote:</div><br class=3D"Apple-interchange-newline"><blockquote = type=3D"cite"><div dir=3D"ltr"><div>Do you have power management = configured?<br></div>Was the "failed" host = fenced/rebooted?<br></div><div class=3D"gmail_extra"><br><br><div = class=3D"gmail_quote">On Fri, Apr 4, 2014 at 2:21 PM, Koen Vanoppen = <span dir=3D"ltr"><<a href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div = dir=3D"ltr"><div><div><div>So... It is possible for a fully automatic = migration of the VM to another hypervisor in case Storage connection = fails?<br> </div>How can we make this happen? Because for the moment, when we = tested the situation they stayed in pause state.<br> </div><div>(Test situation:<br><ul><li>Unplug the 2 fibre cables from = the hypervisor</li><li>VM's go in pause state</li><li>VM's stayed in = pause state until the failure was = solved</li></ul></div></div></div></div></blockquote></div></div></blockqu= ote><div><br></div>as said before, it's not safe hence we (try to) not = migrate them.</div><div>They only get paused when they actually access = the storage which may not be always the case. I.e. the storage = connection is severed, host deemed NonOperational and VMs are getting = migrated from it, then some of them will succeed if they didn't access = that "bad" storage =85 the paused VMs will remain (mostly, it can still = happen that they appear paused migrated on other host when the disk = access occurs only at the last stage of = migration)</div><div><br></div><div><br></div><div>so in other words, if = you want to migrate the VMs without interruption it's not sometimes = possible</div><div>if you are fine with the VMs restarted in short time = on other host then power management/fencing will help = here</div><div><br></div><div>Thanks,</div><div>michal<br><blockquote = type=3D"cite"><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><blockquote class=3D"gmail_quote" = style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; = margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, = 204, 204); border-left-style: solid; padding-left: 1ex; position: = static; z-index: auto; "><div dir=3D"ltr"><div = dir=3D"ltr"><div><div><p>)</p><p><br></p></div> They only returned when we restored the fiber connection to the = Hypervisor=85<br></div></div></div></blockquote></div></div></blockquote><= div><br></div>yes, since 3.3 we have the autoresume = feature</div><div><br></div><div>Thanks,</div><div>michal</div><div><br></= div><div><br><blockquote type=3D"cite"><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><blockquote class=3D"gmail_quote" = style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; = margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, = 204, 204); border-left-style: solid; padding-left: 1ex; position: = static; z-index: auto; "><div dir=3D"ltr"><div = dir=3D"ltr"><div><br></div>Kind = Regards,<br><br>Koen<br></div><br></div><div = class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">2014-04-04 = 13:52 GMT+02:00 Koen Vanoppen <span dir=3D"ltr"><<a = href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a>></span>:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div = dir=3D"ltr"><div><div><div>So... It is possible for a fully automatic = migration of the VM to another hypervisor in case Storage connection = fails?<br> </div>How can we make this happen? Because for the moment, when we = tested the situation they stayed in pause state.<br> </div><div>(Test situation:<br><ul><li>Unplug the 2 fibre cables from = the hypervisor</li><li>VM's go in pause state</li><li>VM's stayed in = pause state until the failure was = solved</li></ul><p>)</p><p><br></p></div> They only returned when we restored the fiber connection to the = Hypervisor...<br><br></div>Kind Regards,<br><br>Koen<br></div><div = class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">2014-04-03 = 16:53 GMT+02:00 Koen Vanoppen <span dir=3D"ltr"><<a = href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a>></span>:<div> <div><br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div = class=3D"gmail_quote"><div class=3D""><div>---------- Forwarded message = ----------<br>From: "Doron Fediuck" <<a = href=3D"mailto:dfediuck@redhat.com" = target=3D"_blank">dfediuck@redhat.com</a>><br> Date: Apr 3, 2014 4:51 PM<br>Subject: Re: [Users] = HA<br></div></div><div><div class=3D"h5"><div><div> To: "Koen Vanoppen" <<a href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a>><br>Cc: "Omer Frenkel" = <<a href=3D"mailto:ofrenkel@redhat.com" = target=3D"_blank">ofrenkel@redhat.com</a>>, <<a = href=3D"mailto:users@ovirt.org" target=3D"_blank">users@ovirt.org</a>>,= "Federico Simoncelli" <<a href=3D"mailto:fsimonce@redhat.com" = target=3D"_blank">fsimonce@redhat.com</a>>, "Allon Mureinik" <<a = href=3D"mailto:amureini@redhat.com" = target=3D"_blank">amureini@redhat.com</a>><br> <br type=3D"attribution"><br> <br> ----- Original Message -----<br> > From: "Koen Vanoppen" <<a href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a>><br> > To: "Omer Frenkel" <<a href=3D"mailto:ofrenkel@redhat.com" = target=3D"_blank">ofrenkel@redhat.com</a>>, <a = href=3D"mailto:users@ovirt.org" target=3D"_blank">users@ovirt.org</a><br> > Sent: Wednesday, April 2, 2014 4:17:36 PM<br> > Subject: Re: [Users] HA<br> ><br> > Yes, indeed. I meant not-operational. Sorry.<br> > So, if I understand this correctly. When we ever come in a = situation that we<br> > loose both storage connections on our hypervisor, we will have to = manually<br> > restore the connections first?<br> ><br> > And thanx for the tip for speeding up thins :-).<br> ><br> > Kind regards,<br> ><br> > Koen<br> ><br> ><br> > 2014-04-02 15:14 GMT+02:00 Omer Frenkel < <a = href=3D"mailto:ofrenkel@redhat.com" = target=3D"_blank">ofrenkel@redhat.com</a> > :<br> ><br> ><br> ><br> ><br> ><br> > ----- Original Message -----<br> > > From: "Koen Vanoppen" < <a = href=3D"mailto:vanoppen.koen@gmail.com" = target=3D"_blank">vanoppen.koen@gmail.com</a> ><br> > > To: <a href=3D"mailto:users@ovirt.org" = target=3D"_blank">users@ovirt.org</a><br> > > Sent: Wednesday, April 2, 2014 4:07:19 PM<br> > > Subject: [Users] HA<br> > ><br> > > Dear All,<br> > ><br> > > Due our acceptance testing, we discovered something. (Document = will<br> > > follow).<br> > > When we disable one fiber path, no problem multipath finds it = way no pings<br> > > are lost.<br> > > BUT when we disabled both the fiber paths (so one of the = storage domain is<br> > > gone on this host, but still available on the other host), vms = go in paused<br> > > mode... He chooses a new SPM (can we speed this up?), put's = the host in<br> > > non-responsive (can we speed this up, more important) and the = VM's stay on<br> > > Paused mode... I would expect that they would be migrated = (yes, HA is<br> ><br> > i guess you mean the host moves to not-operational (in contrast = to<br> > non-responsive)?<br> > if so, the engine will not migrate vms that are paused to do io = error,<br> > because of data corruption risk.<br> ><br> > to speed up you can look at the storage domain monitoring = timeout:<br> > engine-config --get StorageDomainFalureTimeoutInMinutes<br> ><br> ><br> > > enabled) to the other host and reboot there... Any solution? = We are still<br> > > using oVirt 3.3.1 , but we are planning a upgrade to 3.4 after = the easter<br> > > holiday.<br> > ><br> > > Kind Regards,<br> > ><br> > > Koen<br> > ><br> <br> Hi Koen,<br> Resuming from paused due to io issues is supported (adding relevant = folks).<br> Regardless, if you did not define power management, you should manually = approve<br> source host was rebooted in order for migration to proceed. Otherwise we = risk<br> split-brain scenario.<br> <br> Doron<br> </div></div></div></div></div> </blockquote></div></div></div><br></div> </blockquote></div><br></div> <br>_______________________________________________<br> Users mailing list<br> <a href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><br> <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" = target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br> <br></blockquote></div><br></div> _______________________________________________<br>Users mailing = list<br><a = href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><br>http://lists.ovirt.= org/mailman/listinfo/users<br></blockquote></div><br></body></html>= --Apple-Mail=_288231F1-4E67-445C-BD5C-619AAD0235E3--