Re: [ovirt-users] Fake power management?
Von:Barak Azulay <bazulay@redhat.com <mailto:bazulay@redhat.com> <= mailto:bazulay@redhat.com <mailto:bazulay@redhat.com> > >=0D=0A> > > Gese= ndet: Mon 17 November 2014 11:35=0D=0A> > > An: Patrick Lottenbach <pl@a-= bot.ch <mailto:pl@a-bot.ch> <mailto:pl@a-bot.ch <mailto:pl@a-bot.ch> > >= =0D=0A> > > CC: users@ovirt.org <mailto:users@ovirt.org> <mailto:users@o= virt.org <mailto:users@ovirt.org> >=0D=0A> > > Betreff: Re: [ovirt-users]= Fake power management=3F=0D=0A> > >=20=0D=0A> > >=20=0D=0A> > >=20=0D=0A= ----- Original Message -----=0D=0A> > > > From: "mots" <mots@nepu.m= oe <mailto:mots@nepu.moe> <mailto:mots@nepu.moe <mailto:mots@nepu.moe> >= =0D=0A> > > > To: users@ovirt.org <mailto:users@ovirt.org> <mailto:use= rs@ovirt.org <mailto:users@ovirt.org> >=0D=0A> > > > Sent: Friday, Novemb= er 14, 2014 4:54:08 PM=0D=0A> > > > Subject: [ovirt-users] Fake power man= agement=3F=0D=0A> > > >=20=0D=0A> > > > Fake power management=3F Hello,=0D= =0A> > > >=20=0D=0A> > > > I'm building a small demonstration system for = our sales team to take to a=0D=0A> > > > customer so that they can show t= hem our solutions.=0D=0A> > > > Hardware: Two Intel NUC's, a 4 port switc= h and a laptop.=0D=0A> > > > Engine: Runs as a VM on one of the NUCs, whi= ch one it runs on is=0D=0A> > > > determined=0D=0A> > > > by pacemaker.=0D= =0A> > > > Storage: Also managed by pacemaker, it's drbd backed and acces= sed with=0D=0A> > > > iscsi.=0D=0A> > > > oVirt version: 3.5=0D=0A> > > >= OS: CentOS 6.6=0D=0A> > > >=20=0D=0A> > > > The idea is to have our sale= s representative (or the potential customer=0D=0A> > > > himself) randoml= y pull the plug on one of the NUCs to show that the=0D=0A> > > > system=0D= =0A> > > > stays operational when part of the hardware fails.=0D=0A> > >=20= =0D=0A> > > I assume you are aware that the engine might fence the node i= t is running=0D=0A> > > on ...=0D=0A> > > Or do you use pacemaker to run =
------=_Part_39_1687046635.1416266377877 Content-Type: multipart/alternative; boundary="=_t8EqHT3xdZg+XwUZZBd4cSFdl9lEQF49Im0BRY50j2FqCbqe" This is a multi-part message in MIME format. Your mail reader does not understand MIME message format. --=_t8EqHT3xdZg+XwUZZBd4cSFdl9lEQF49Im0BRY50j2FqCbqe Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable -----Urspr=C3=BCngliche Nachricht-----=0D=0A=0D=0A> Von:Barak Azulay <baz= ulay@redhat.com <mailto:bazulay@redhat.com> >=0D=0A> Gesendet: Mon 17 Nov= ember 2014 23:30=0D=0A> An: Patrick Lottenbach <pl@a-bot.ch <mailto:pl@a-= bot.ch> >=0D=0A> CC: users@ovirt.org <mailto:users@ovirt.org>=20=0D=0A> B= etreff: Re: AW: [ovirt-users] Fake power management=3F=0D=0A>=20=0D=0A> W= ell you can hack the solution in the form of replacing the fencing master= script to always return success (Eli can help you with that),=0D=0A> and= define an imaginary fencing device on each host ... meaning that the fen= cing command will always succeeds.=0D=0A>=20=0D=0A=0D=0AThis sounds inter= esting. It's exactly what I need.=0D=0A=0D=0A> But this may be risky ... = as you might end up with the same VM running on 2 hosts.=20=0D=0A=0D=0AAs= I see it, this would only happen if someone unplugs the network interfac= e. I know this is a way to break the cluster. If someone unplugs the inte= rface, then everything gets started twice anyways thanks to pacemaker bei= ng configured to ignore the lack of quorum and it would look silly in fro= nt of the customer.=0D=0A=0D=0A> And one last note ... when you disconnec= t one of the hosts in the demo you mentioned, I think you'll be better to= disconnect the host that does not run the engine ...=0D=0A=20=0D=0AIt ju= st gets restarted on the remaining node and resumes operation. It even re= members which guests ran on which host.=0D=0AThat part is really safe. Th= e storage is configured to only report data as written when the write ope= ration has finished on all (currently online) nodes, disk write caches ar= e turned off in lvm.conf. PostreSQL is resilient enough to survive a cras= h like this.=0D=0A=0D=0AOr am I missing something that might break=3F=0D=0A= =0D=0A> Barak=20=0D=0A=0D=0Amots=0D=0A=0D=0A>=20=0D=0A> ----- Original Me= ssage -----=0D=0A> > From: "mots" <mots@nepu.moe <mailto:mots@nepu.moe> >= =0D=0A> > To: "Barak Azulay" <bazulay@redhat.com <mailto:bazulay@redhat.c= om> >=0D=0A> > Cc: users@ovirt.org <mailto:users@ovirt.org>=20=0D=0A> > S= ent: Monday, November 17, 2014 12:58:20 PM=0D=0A> > Subject: AW: [ovirt-u= sers] Fake power management=3F=0D=0A> >=20=0D=0A> > Yes, pacemaker manage= s the engine. That part is working fine, the engine=0D=0A> > restarts on = the remaining node without problems.=0D=0A> > It's just that the guests d= on't come back up until the powered down node has=0D=0A> > been fenced ma= nually.=0D=0A> >=20=0D=0A> > -----Urspr=C3=BCngliche Nachricht-----=0D=0A= the engine as well =3F=0D=0A> > >=20=0D=0A> > > > My problem is that I do= n't have any way to implement power management, so=0D=0A> > > > the=0D=0A=
Engine can't fence nodes and won't restart guests that were runni= ng on=0D=0A> > > > the=0D=0A> > > > node which lost power. In pacemaker I= can just configure fencing over SSH=0D=0A> > > > or=0D=0A> > > > even di= sable the requirement to do so completely. Is there something=0D=0A> > > = similar=0D=0A> > > > for oVirt, so that the Engine will consider a node= which it can't connect=0D=0A> > > > to=0D=0A> > > > to be powered down=3F= =0D=0A> > > >=20=0D=0A> > > > Regards,=0D=0A> > > >=20=0D=0A> > > > mots=0D= =0A> > > >=20=0D=0A> > > > ______________________________________________= _=0D=0A> > > > Users mailing list=0D=0A> > > > Users@ovirt.org <mailto:Us= ers@ovirt.org> <mailto:Users@ovirt.org <mailto:Users@ovirt.org> >=0D=0A>= http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.= org/mailman/listinfo/users>=20=0D=0A> > > > <" target=3D"_blank">http://l= ists.ovirt.org/mailman/listinfo/users> <http://lists.ovirt.org/mailman/li= stinfo/users> ;=0D=0A> > > >=20=0D=0A> > >=20=0D=0A> >=20=0D=0A> >=0D=0A>= =20=0D=0A=0D=0A --=_t8EqHT3xdZg+XwUZZBd4cSFdl9lEQF49Im0BRY50j2FqCbqe Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
> But this may be risky ... as you might end up with the same VM runn= ing on 2 hosts. <br><br>As I see it, this would only happen if someone un=
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://ww= w.w3.org/TR/html4/loose.dtd"><html>=0A<head>=0A <meta name=3D"Generator"= content=3D"Zarafa WebApp v7.1.10-44973">=0A <meta http-equiv=3D"Content= -Type" content=3D"text/html; charset=3Dutf-8">=0A <title>AW: [ovirt-user= s] Fake power management=3F</title>=0A</head>=0A<body>=0A-----Urspr=C3=BC= ngliche Nachricht-----<br><pre style=3D"white-space: -moz-pre-wrap; white= -space: -pre-wrap; white-space: -o-pre-wrap; white-space: pre-wrap; word-= wrap: break-word;" wrap=3D"">> Von:Barak Azulay <<a href=3D"mailto:= bazulay@redhat.com">bazulay@redhat.com</a>><br>> Gesendet: Mon 17 N= ovember 2014 23:30<br>> An: Patrick Lottenbach <<a href=3D"mailto:p= l@a-bot.ch">pl@a-bot.ch</a>><br>> CC: <a href=3D"mailto:users@ovirt= =2Eorg">users@ovirt.org</a><br>> Betreff: Re: AW: [ovirt-users] Fake p= ower management=3F<br>> <br>> Well you can hack the solution in the= form of replacing the fencing master script to always return success (El= i can help you with that),<br>> and define an imaginary fencing device= on each host ... meaning that the fencing command will always succeeds.<= br>> <br><br>This sounds interesting. It's exactly what I need.<br><br= plugs the network interface. I know this is a way to break the cluster. I= f someone unplugs the interface, then everything gets started twice anywa= ys thanks to pacemaker being configured to ignore the lack of quorum and = it would look silly in front of the customer.<br><br>> And one last no= te ... when you disconnect one of the hosts in the demo you mentioned, I = think you'll be better to disconnect the host that does not run the engin= e ...<br> <br>It just gets restarted on the remaining node and resumes op= eration. It even remembers which guests ran on which host.<br>That part i= s really safe. The storage is configured to only report data as written w= hen the write operation has finished on all (currently online) nodes, dis= k write caches are turned off in lvm.conf. PostreSQL is resilient enough = to survive a crash like this.<br><br>Or am I missing something that might= break=3F<br><br>> Barak <br><br>mots<br><br>> <br>> ----- Origi= nal Message -----<br>> > From: "mots" <<a href=3D"mailto:mots@ne= pu.moe">mots@nepu.moe</a>><br>> > To: "Barak Azulay" <<a href= =3D"mailto:bazulay@redhat.com">bazulay@redhat.com</a>><br>> > Cc= : <a href=3D"mailto:users@ovirt.org">users@ovirt.org</a><br>> > Sen= t: Monday, November 17, 2014 12:58:20 PM<br>> > Subject: AW: [ovirt= -users] Fake power management=3F<br>> > <br>> > Yes, pacemake= r manages the engine. That part is working fine, the engine<br>> > = restarts on the remaining node without problems.<br>> > It's just t= hat the guests don't come back up until the powered down node has<br>>= > been fenced manually.<br>> > <br>> > -----Urspr=C3=BCng= liche Nachricht-----<br>> > > Von:Barak Azulay <<a href=3D"ma= ilto:bazulay@redhat.com">bazulay@redhat.com</a> <mailto:<a href=3D"mai= lto:bazulay@redhat.com">bazulay@redhat.com</a>> ><br>> > >= Gesendet: Mon 17 November 2014 11:35<br>> > > An: Patrick Lotte= nbach <<a href=3D"mailto:pl@a-bot.ch">pl@a-bot.ch</a> <mailto:<a hr= ef=3D"mailto:pl@a-bot.ch">pl@a-bot.ch</a>> ><br>> > > CC: = <a href=3D"mailto:users@ovirt.org">users@ovirt.org</a> <mailto:<a href= =3D"mailto:users@ovirt.org">users@ovirt.org</a>><br>> > > Bet= reff: Re: [ovirt-users] Fake power management=3F<br>> > > <br>&g= t; > > <br>> > > <br>> > > ----- Original Message= -----<br>> > > > From: "mots" <<a href=3D"mailto:mots@nep= u.moe">mots@nepu.moe</a> <mailto:<a href=3D"mailto:mots@nepu.moe">mots= @nepu.moe</a>> ><br>> > > > To: <a href=3D"mailto:users= @ovirt.org">users@ovirt.org</a> <mailto:<a href=3D"mailto:users@ovirt.= org">users@ovirt.org</a>><br>> > > > Sent: Friday, Novembe= r 14, 2014 4:54:08 PM<br>> > > > Subject: [ovirt-users] Fake = power management=3F<br>> > > > <br>> > > > Fake p= ower management=3F Hello,<br>> > > > <br>> > > > = I'm building a small demonstration system for our sales team to take to a= <br>> > > > customer so that they can show them our solutions= =2E<br>> > > > Hardware: Two Intel NUC's, a 4 port switch and= a laptop.<br>> > > > Engine: Runs as a VM on one of the NUCs= , which one it runs on is<br>> > > > determined<br>> > = > > by pacemaker.<br>> > > > Storage: Also managed by p= acemaker, it's drbd backed and accessed with<br>> > > > iscsi= =2E<br>> > > > oVirt version: 3.5<br>> > > > OS: = CentOS 6.6<br>> > > > <br>> > > > The idea is to = have our sales representative (or the potential customer<br>> > >= ; > himself) randomly pull the plug on one of the NUCs to show that th= e<br>> > > > system<br>> > > > stays operational = when part of the hardware fails.<br>> > > <br>> > > I a= ssume you are aware that the engine might fence the node it is running<br=
> > > on ...<br>> > > Or do you use pacemaker to run t= he engine as well =3F<br>> > > <br>> > > > My proble= m is that I don't have any way to implement power management, so<br>> = > > > the<br>> > > > Engine can't fence nodes and wo= n't restart guests that were running on<br>> > > > the<br>>= ; > > > node which lost power. In pacemaker I can just configure= fencing over SSH<br>> > > > or<br>> > > > even d= isable the requirement to do so completely. Is there something<br>> &g= t; > > similar<br>> > > > for oVirt, so that the Engine= will consider a node which it can't connect<br>> > > > to<br= > > > > to be powered down=3F<br>> > > > <br>>= ; > > > Regards,<br>> > > > <br>> > > > = mots<br>> > > > <br>> > > > _____________________= __________________________<br>> > > > Users mailing list<br>&= gt; > > > <a href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a>= <mailto:<a href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a>><br= > > > > <a href=3D"http://lists.ovirt.org/mailman/listinfo/u= sers" target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/users</a>= <br>> > > > <<a href=3D"http://lists.ovirt.org/mailman/lis= tinfo/users>" target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/u= sers></a>;<br>> > > > <br>> > > <br>> > <br= > ><br>> </pre>=0A</body>=0A</html> --=_t8EqHT3xdZg+XwUZZBd4cSFdl9lEQF49Im0BRY50j2FqCbqe--
------=_Part_39_1687046635.1416266377877 Content-Type: application/pgp-signature; name=signature.asc Content-Transfer-Encoding: 7bit Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: CIPHERMAIL (2.8.6-4) iQEcBAEBCAAGBQJUaoKJAAoJELfzdzVzTtoKUzQH/3pNM2Xw4MjY7OkJRj2c7VbR Lk454UhQ3kBTz/OiFLI5burVnapyzNcrIOAkwwWdseZkFulgqJhh+wiwS4EN/3aV cc3FrESB5sbmObO87tqLiQMkVvc8nUASZhIOcQJEiUhRmDMTGerR58YHR3cKBTiN hygbEkcmPw9pSYrJeQ/jXMlapju4Xet5FHbd4EYcgW8Gh7QmQszdMcNc9qwZ3dyM rzsY8OpaHogzQv1DbnXJAjdGR8LMm/XDP2ZBazuafnjQZli3kEl2WwNFrda+kZhj HTpdJFwt/yC05hkz3hXzkRwfqXDsYmCLsetn4Ym8w/pcA80iLiUp/oDqi7BheTE= =V9lP -----END PGP SIGNATURE----- ------=_Part_39_1687046635.1416266377877--
----- Original Message -----
From: "mots" <mots@nepu.moe> To: "Barak Azulay" <bazulay@redhat.com> Cc: users@ovirt.org Sent: Tuesday, November 18, 2014 1:17:49 AM Subject: AW: [ovirt-users] Fake power management?
-----Ursprüngliche Nachricht-----
Von:Barak Azulay <bazulay@redhat.com <mailto:bazulay@redhat.com> > Gesendet: Mon 17 November 2014 23:30 An: Patrick Lottenbach <pl@a-bot.ch <mailto:pl@a-bot.ch> > CC: users@ovirt.org <mailto:users@ovirt.org> Betreff: Re: AW: [ovirt-users] Fake power management?
Well you can hack the solution in the form of replacing the fencing master script to always return success (Eli can help you with that), and define an imaginary fencing device on each host ... meaning that the fencing command will always succeeds.
This sounds interesting. It's exactly what I need.
But this may be risky ... as you might end up with the same VM running on 2 hosts.
As I see it, this would only happen if someone unplugs the network interface. I know this is a way to break the cluster. If someone unplugs the interface, then everything gets started twice anyways thanks to pacemaker being configured to ignore the lack of quorum and it would look silly in front of the customer.
And one last note ... when you disconnect one of the hosts in the demo you mentioned, I think you'll be better to disconnect the host that does not run the engine ...
It just gets restarted on the remaining node and resumes operation. It even remembers which guests ran on which host. That part is really safe. The storage is configured to only report data as written when the write operation has finished on all (currently online) nodes, disk write caches are turned off in lvm.conf. PostreSQL is resilient enough to survive a crash like this.
Or am I missing something that might break?
Let me know if we missed something This is an interesting demo ;-)
Barak
mots
----- Original Message -----
From: "mots" <mots@nepu.moe <mailto:mots@nepu.moe> > To: "Barak Azulay" <bazulay@redhat.com <mailto:bazulay@redhat.com> > Cc: users@ovirt.org <mailto:users@ovirt.org> Sent: Monday, November 17, 2014 12:58:20 PM Subject: AW: [ovirt-users] Fake power management?
Yes, pacemaker manages the engine. That part is working fine, the engine restarts on the remaining node without problems. It's just that the guests don't come back up until the powered down node has been fenced manually.
-----Ursprüngliche Nachricht-----
Von:Barak Azulay <bazulay@redhat.com <mailto:bazulay@redhat.com> <mailto:bazulay@redhat.com <mailto:bazulay@redhat.com> > > Gesendet: Mon 17 November 2014 11:35 An: Patrick Lottenbach <pl@a-bot.ch <mailto:pl@a-bot.ch> <mailto:pl@a-bot.ch <mailto:pl@a-bot.ch> > > CC: users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org> > Betreff: Re: [ovirt-users] Fake power management?
----- Original Message -----
From: "mots" <mots@nepu.moe <mailto:mots@nepu.moe> <mailto:mots@nepu.moe <mailto:mots@nepu.moe> > > To: users@ovirt.org <mailto:users@ovirt.org> <mailto:users@ovirt.org <mailto:users@ovirt.org> > Sent: Friday, November 14, 2014 4:54:08 PM Subject: [ovirt-users] Fake power management?
Fake power management? Hello,
I'm building a small demonstration system for our sales team to take to a customer so that they can show them our solutions. Hardware: Two Intel NUC's, a 4 port switch and a laptop. Engine: Runs as a VM on one of the NUCs, which one it runs on is determined by pacemaker. Storage: Also managed by pacemaker, it's drbd backed and accessed with iscsi. oVirt version: 3.5 OS: CentOS 6.6
The idea is to have our sales representative (or the potential customer himself) randomly pull the plug on one of the NUCs to show that the system stays operational when part of the hardware fails.
I assume you are aware that the engine might fence the node it is running on ... Or do you use pacemaker to run the engine as well ?
My problem is that I don't have any way to implement power management, so the Engine can't fence nodes and won't restart guests that were running on the node which lost power. In pacemaker I can just configure fencing over SSH or even disable the requirement to do so completely. Is there something similar for oVirt, so that the Engine will consider a node which it can't connect to to be powered down?
Regards,
mots
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> <mailto:Users@ovirt.org <mailto:Users@ovirt.org> > http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users> <" target="_blank">http://lists.ovirt.org/mailman/listinfo/users> <http://lists.ovirt.org/mailman/listinfo/users> ;
participants (2)
-
Barak Azulay -
mots