This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DtNOHPq0pvAdvxKNIida56NVOQMQS9co0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi oVirt group, I'm currently trying to set up an oVirt 3.6 self hosed engine on a replica 2 gluster volume. As you can imagine it fails since it expects a replica 3 volume. Since this part is hard coded during the installation this could be easily remedied but on the other hand I'm not familiar with the oVirt setup so I'm not sure if I would be disturbing something far a head that again is puzzled by the lack of a replica 3 volume. Is there a reason why it has to be exactly replica 3? Shouldn't the redundancy of the storage back end be the responsibility of the admin and therefore the option to choose (at least replicated and distributed replicated) open? I'm asking because the storage back and I'm using is kind of a 'big' thing and it's hardware is put together to facilitate the use of replica 2. Sure I could create multiple bricks on those two machines but other than a negative performance impact I would gain nothing. Since the price tag of one of those servers is quite high the likelihood of getting another one is exactly 0. So I'm sure it makes sens to you that I would rather choose my own replication level than having one predetermined. All the best Richard --DtNOHPq0pvAdvxKNIida56NVOQMQS9co0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iEYEARECAAYFAlXpQIMACgkQnGohgOrO9GEOmwCglzYqsJ+N2jCwtp2dJY8wKdUg jS0AoK9Vy/ysvsHJCjzvwTmYuIXFd28V =obax -----END PGP SIGNATURE----- --DtNOHPq0pvAdvxKNIida56NVOQMQS9co0--
On Fri, Sep 4, 2015 at 8:56 AM, Richard Neuboeck <hawk@tbi.univie.ac.at> wrote:
Hi oVirt group,
I'm currently trying to set up an oVirt 3.6 self hosed engine on a replica 2 gluster volume. As you can imagine it fails since it expects a replica 3 volume.
Since this part is hard coded during the installation this could be easily remedied but on the other hand I'm not familiar with the oVirt setup so I'm not sure if I would be disturbing something far a head that again is puzzled by the lack of a replica 3 volume.
Is there a reason why it has to be exactly replica 3?
To have a valid quorum having the system being able to decide witch is the right and safe copy avoiding an issue called split brain. Under certain circumstances/issues (network issue, hosts down or whatever could happen) the data on different replica could diverge: if you have two and just two different hosts that claim each other that its copy is the right one there is no way to automatically take the right decision. Having three hosts and setting the quorum according to that solves/mitigates the issue.
Shouldn't the redundancy of the storage back end be the responsibility of the admin and therefore the option to choose (at least replicated and distributed replicated) open?
I'm asking because the storage back and I'm using is kind of a 'big' thing and it's hardware is put together to facilitate the use of replica 2. Sure I could create multiple bricks on those two machines but other than a negative performance impact I would gain nothing. Since the price tag of one of those servers is quite high the likelihood of getting another one is exactly 0. So I'm sure it makes sens to you that I would rather choose my own replication level than having one predetermined.
All the best Richard
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --rkOk7SUlOX6I5tCrr6nXdCjW1e49G1hP9 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 04.09.15 10:02, Simone Tiraboschi wrote:
Is there a reason why it has to be exactly replica 3?
To have a valid quorum having the system being able to decide witch is the right and safe copy avoiding an issue called split brain. Under certain circumstances/issues (network issue, hosts down or whatever could happen) the data on different replica could diverge: if you have two and just two different hosts that claim each other
that its
copy is the right one there is no way to automatically take the right decision. Having three hosts and setting the quorum according to that solves/mitigates the issue.
Thanks for the explanation. I do understand the problem but since I'm somewhat limited in my hardware options is there a way to override this requirement? Meaning if I change the checks for replica 3 in the installation scripts does something else fail on the way? In my case coherence checks would come from outside the storage and vm host setup and fencing would be applied appropriately. I would very much appreciate it if the particulars of the storage setup could be either selected from a list of possibilities or be ignored and just a warning be issued that this setup is not recommended. Thanks! Richard --=20 /dev/null --rkOk7SUlOX6I5tCrr6nXdCjW1e49G1hP9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlXv6nYACgkQnGohgOrO9GHXogCgwshYj4KLmuVucmc+UG5404au 2A8AnRfQFKGVJOcm9arkrz6snEttd0DN =C0cn -----END PGP SIGNATURE----- --rkOk7SUlOX6I5tCrr6nXdCjW1e49G1hP9--
This is a multi-part message in MIME format. ------------MIME-2974521450-724884092-delim Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable On 09/09/2015 10=3A14 AM=2C Richard Neuboeck wrote=3A =3E On 04=2E09=2E15 10=3A02=2C Simone Tiraboschi wrote=3A =3E=3E Is there a reason why it has to be exactly replica 3=3F =3E=3E =3E=3E =3E=3E To have a valid quorum having the system being able to decide witch= is =3E=3E the right and safe copy avoiding an issue called split brain=2E =3E=3E Under certain circumstances/issues =28network issue=2C hosts down or= =3E=3E whatever could happen=29 the data on different replica could diverge= =3A if =3E=3E you have two and just two different hosts that claim each other =3E that its =3E=3E copy is the right one there is no way to automatically take the righ= t =3E=3E decision=2E Having three hosts and setting the quorum according to t= hat =3E=3E solves/mitigates the issue=2E =3E =3E Thanks for the explanation=2E I do understand the problem but since =3E I=27m somewhat limited in my hardware options is there a way to =3E override this requirement=3F Meaning if I change the checks for =3E replica 3 in the installation scripts does something else fail on =3E the way=3F =3E =3E In my case coherence checks would come from outside the storage and =3E vm host setup and fencing would be applied appropriately=2E =3E =3E I would very much appreciate it if the particulars of the storage =3E setup could be either selected from a list of possibilities or be =3E ignored and just a warning be issued that this setup is not recommended= =2E =3E =3E Thanks! =3E Richard =3E =3E As a side question=2C in Glusterfs 3=2E7 there is an =22AFR arbiter volume= =22 =28http=3A//gluster=2Ereadthedocs=2Eorg/en/release-3=2E7=2E0-1/Features/afr= -arbiter-volumes/=29 that will only contain the metadata=2E Will ovirt support this in 3=2E6=3F Kind regards=2C Jorick Astrego Met vriendelijke groet=2C With kind regards=2C Jorick Astrego Netbulae Virtualization Experts=20 ---------------- =09Tel=3A 053 20 30 270 =09info=40netbulae=2Eeu =09Staalsteden 4-3A =09KvK= 08198180 =09Fax=3A 053 20 30 271 =09www=2Enetbulae=2Eeu =097547 TA Enschede =09BTW= NL821234584B01 ---------------- ------------MIME-2974521450-724884092-delim Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable =3Chtml=3E =3Cbody=3E <br> <br> On 09/09/2015 10:14 AM, Richard Neuboeck wrote: <br> <font color=3D"#000000">> On 04.09.15 10:02, Simone Tiraboschi wrote:= 3;</font><br> <font color=3D"#000000">>> Is there a reason = why it has to be exactly replica 3? </font><br> <font color=3D"#000000">>> </font><br> <font color=3D"#000000">>> </font><br> <font color=3D"#000000">>> To have a valid quorum having the system b= eing able to decide witch is </font><br> <font color=3D"#000000">>> the right and safe copy avoiding an issue = called split brain. </font><br> <font color=3D"#000000">>> Under certain circumstances/issues (networ= k issue, hosts down or </font><br> <font color=3D"#000000">>> whatever could happen) the data on differe= nt replica could diverge: if </font><br> <font color=3D"#000000">>> you have two and just two different hosts = that claim each other </font><br> <font color=3D"#000000">> that its </font><br> <font color=3D"#000000">>> copy is the right one there is no way to a= utomatically take the right </font><br> <font color=3D"#000000">>> decision. Having three hosts and setting t= he quorum according to that </font><br> <font color=3D"#000000">>> solves/mitigates the issue. </font><br=
<font color=3D"#000000">> </font><br> <font color=3D"#000000">> Thanks for the explanation. I do understand th= e problem but since </font><br> <font color=3D"#000000">> I'm somewhat limited in my hardware options is= there a way to </font><br> <font color=3D"#000000">> override this requirement? Meaning if I change= the checks for </font><br> <font color=3D"#000000">> replica 3 in the installation scripts does som= ething else fail on </font><br> <font color=3D"#000000">> the way? </font><br> <font color=3D"#000000">> </font><br> <font color=3D"#000000">> In my case coherence checks would come from ou= tside the storage and </font><br> <font color=3D"#000000">> vm host setup and fencing would be applied app= ropriately. </font><br> <font color=3D"#000000">> </font><br> <font color=3D"#000000">> I would very much appreciate it if the particu= lars of the storage </font><br> <font color=3D"#000000">> setup could be either selected from a list of = possibilities or be </font><br> <font color=3D"#000000">> ignored and just a warning be issued that this= setup is not recommended. </font><br> <font color=3D"#000000">> </font><br> <font color=3D"#000000">> Thanks! </font><br> <font color=3D"#000000">> Richard </font><br> <font color=3D"#000000">> </font><br> <font color=3D"#000000">> </font><br> As a side question, in Glusterfs 3.7 there is an "AFR arbiter volume&q= uot; <br> (<a href=3D"http://gluster.readthedocs.org/en/release-3.7.0-1/Features/afr-= arbiter-volumes/">http://gluster.readthedocs.org/en/release-3.7.0-1/Feature= s/afr-arbiter-volumes/</a>) <br> that will only contain the metadata. <br> <br> Will ovirt support this in 3.6? <br> <br> Kind regards, <br> <br> Jorick Astrego <br> = =3CBR /=3E =3CBR /=3E =3Cb style=3D=22color=3A=23604c78=22=3E=3C/b=3E=3Cbr=3E=3Cbr=3E=3Cspan styl= e=3D=22color=3A=23604c78=3B=22=3E=3Cfont color=3D=22000000=22=3E=3Cspan sty= le=3D=22mso-fareast-language=3Aen-gb=3B=22 lang=3D=22NL=22=3EMet vriendelij= ke groet=2C With kind regards=2C=3Cbr=3E=3Cbr=3EJorick Astrego=3Cbr=3E=3C/s= pan=3E=3C/font=3E=3C/span=3E=3Cb style=3D=22color=3A=23604c78=22=3E=3Cbr=3E= Netbulae Virtualization Experts =3C/b=3E=3Cbr=3E=3Chr style=3D=22border=3An= one=3Bborder-top=3A1px solid =23ccc=3B=22=3E=3Ctable style=3D=22width=3A 52= 2px=22=3E=3Ctbody=3E=3Ctr=3E=3Ctd style=3D=22width=3A 130px=3Bfont-size=3A= 10px=22=3ETel=3A 053 20 30 270=3C/td=3E =3Ctd style=3D=22width=3A 130p= x=3Bfont-size=3A 10px=22=3Einfo=40netbulae=2Eeu=3C/td=3E =3Ctd style=3D= =22width=3A 130px=3Bfont-size=3A 10px=22=3EStaalsteden 4-3A=3C/td=3E =20= =3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EKvK 08198180=3C/td= =3E=3C/tr=3E=3Ctr=3E =3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px= =22=3EFax=3A 053 20 30 271=3C/td=3E =3Ctd style=3D=22width=3A 130px=3Bfo= nt-size=3A 10px=22=3Ewww=2Enetbulae=2Eeu=3C/td=3E =3Ctd style=3D=22width= =3A 130px=3Bfont-size=3A 10px=22=3E7547 TA Enschede=3C/td=3E =3Ctd style= =3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EBTW NL821234584B01=3C/td=3E= =3C/tr=3E=3C/tbody=3E=3C/table=3E=3Cbr=3E=3Chr style=3D=22border=3Anone=3Bb= order-top=3A1px solid =23ccc=3B=22=3E=3CBR /=3E =3C/body=3E =3C/html=3E ------------MIME-2974521450-724884092-delim--
On Wed, Sep 9, 2015 at 10:49 AM, Jorick Astrego <j.astrego@netbulae.eu> wrote:
On 09/09/2015 10:14 AM, Richard Neuboeck wrote:
On 04.09.15 10:02, Simone Tiraboschi wrote:
Is there a reason why it has to be exactly replica 3?
To have a valid quorum having the system being able to decide witch is the right and safe copy avoiding an issue called split brain. Under certain circumstances/issues (network issue, hosts down or whatever could happen) the data on different replica could diverge: if you have two and just two different hosts that claim each other
that its
copy is the right one there is no way to automatically take the right decision. Having three hosts and setting the quorum according to that solves/mitigates the issue.
Thanks for the explanation. I do understand the problem but since I'm somewhat limited in my hardware options is there a way to override this requirement? Meaning if I change the checks for replica 3 in the installation scripts does something else fail on the way?
In my case coherence checks would come from outside the storage and vm host setup and fencing would be applied appropriately.
I would very much appreciate it if the particulars of the storage setup could be either selected from a list of possibilities or be ignored and just a warning be issued that this setup is not recommended.
Thanks! Richard
As a side question, in Glusterfs 3.7 there is an "AFR arbiter volume" ( http://gluster.readthedocs.org/en/release-3.7.0-1/Features/afr-arbiter-volum...)
that will only contain the metadata.
Will ovirt support this in 3.6?
Not explicitly and I never tried it with hosted-engine but I suspect it works as far it will report replica level = 3. By the way you can save disk space on the third host but you still need three hosts.
Kind regards,
Jorick Astrego
Met vriendelijke groet, With kind regards,
Jorick Astrego
*Netbulae Virtualization Experts * ------------------------------ Tel: 053 20 30 270 info@netbulae.eu Staalsteden 4-3A KvK 08198180 Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01 ------------------------------
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Wed, Sep 9, 2015 at 10:14 AM, Richard Neuboeck <hawk@tbi.univie.ac.at> wrote:
On 04.09.15 10:02, Simone Tiraboschi wrote:
Is there a reason why it has to be exactly replica 3?
To have a valid quorum having the system being able to decide witch is the right and safe copy avoiding an issue called split brain. Under certain circumstances/issues (network issue, hosts down or whatever could happen) the data on different replica could diverge: if you have two and just two different hosts that claim each other
that its
copy is the right one there is no way to automatically take the right decision. Having three hosts and setting the quorum according to that solves/mitigates the issue.
Thanks for the explanation. I do understand the problem but since I'm somewhat limited in my hardware options is there a way to override this requirement? Meaning if I change the checks for replica 3 in the installation scripts does something else fail on the way?
I'm advising that it's not a safe configuration so it's not recommended for a production environment. Having said that, as far as I know it's enforced only in the setup script so tweaking it should be enough. Otherwise, if you have enough disk space, you can also have a different trick: you could create a replica 3 volume with 2 bricks from a single host. It's not a safe procedure at all cause you still have only 2 hosts, so it's basically just replica 2, and in case of split brain the host with two copies will win by configuration which is not always the right decision.
In my case coherence checks would come from outside the storage and vm host setup and fencing would be applied appropriately.
Can I ask how?
I would very much appreciate it if the particulars of the storage setup could be either selected from a list of possibilities or be ignored and just a warning be issued that this setup is not recommended.
Thanks! Richard
-- /dev/null
=20 =20 On Wed, Sep 9, 2015 at 10:14 AM, Richard Neuboeck <hawk@tbi.univie.ac.at <mailto:hawk@tbi.univie.ac.at>> wrote: =20 On 04.09.15 10:02, Simone Tiraboschi wrote: > Is there a reason why it has to be exactly replica 3? > > > To have a valid quorum having the system being able to decide wit= ch is > the right and safe copy avoiding an issue called split brain. > Under certain circumstances/issues (network issue, hosts down or > whatever could happen) the data on different replica could diverg= e: if > you have two and just two different hosts that claim each other that its > copy is the right one there is no way to automatically take the r= ight > decision. Having three hosts and setting the quorum according to =
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --deVVEXUdPHOeuenfhO5VGUjVsuc3DdN5d Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 09/09/2015 10:54 AM, Simone Tiraboschi wrote: that
> solves/mitigates the issue. =20 =20 Thanks for the explanation. I do understand the problem but since I'm somewhat limited in my hardware options is there a way to override this requirement? Meaning if I change the checks for replica 3 in the installation scripts does something else fail on the way? =20 =20 I'm advising that it's not a safe configuration so it's not recommended for a production environment. Having said that, as far as I know it's enforced only in the setup script so tweaking it should be enough. Otherwise, if you have enough disk space, you can also have a different trick: you could create a replica 3 volume with 2 bricks from a single host.
I've thought about that but since that would obviously only help to fool the installation script there is nothing else in this setup that would improve the situation. Worse the read/write overhead on the second machine would be a performance downgrade.
It's not a safe procedure at all cause you still have only 2 hosts, so it's basically just replica 2, and in case of split brain the host with two copies will win by configuration which is not always the right decision.
Right. I'm thinking of trying to add a dummy node as mentioned in the RHEL documentation. This would (in theory) prevent the read only state in the split brain scenario and make it possible to access the storage. But still the installation requirement of replica 3 would not be satisfied.
=20 In my case coherence checks would come from outside the storage and=
vm host setup and fencing would be applied appropriately. =20 =20 Can I ask how?
Multiple machines separated from the storage and virtualization machines that will check communication (in general and of several services) and try to intervene if there is something going awry first by accessing the machines directly (if possible) and then by deactivating those machines by remote management. Cheers Richard
=20 I would very much appreciate it if the particulars of the storage setup could be either selected from a list of possibilities or be ignored and just a warning be issued that this setup is not recommended. =20 Thanks! Richard =20 =20 -- /dev/null =20 =20
--=20 /dev/null --deVVEXUdPHOeuenfhO5VGUjVsuc3DdN5d Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlXwFVQACgkQnGohgOrO9GEd0QCgk58o8mOE/RFp6zvXuzOUvhRm YOoAoK56/9X5or7oIa5K7QA3PTUdyZei =v/NJ -----END PGP SIGNATURE----- --deVVEXUdPHOeuenfhO5VGUjVsuc3DdN5d--
participants (3)
-
Jorick Astrego -
Richard Neuboeck -
Simone Tiraboschi