Hi, I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :- 1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ??? Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server. Thanks, Punit
--Apple-Mail=_D6A50034-D6B0-41DD-A3C8-84BF3B838A86 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 You need to setup a virtual IP to use as the mount point, most people = use keepalived to provide a virtual ip via vrrp for this. Setup = something like 10.10.10.10 and use that for your mounts. Right now, if 10.10.10.2 goes away, all your gluster mounts go away and = your VMs get paused because the hypervisors can=92t access the storage. = Your gluster storage is still fine, but ovirt can=92t talk to it because = 10.10.10.2 isn=92t there. If the SPM goes down, it the other hypervisor hosts will elect a new one = (under control of the ovirt engine). Same scenarios if storage & compute are on the same server, you still = need a vip address for the storage portion to serve as the mount point = so it=92s not dependent on any one server. -Darrell On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi, =20 I have some HA related concern about glusterfs with Ovirt...let say i = have 4 storage node with gluster bricks as below :-=20 =20 1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed = replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following = mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) = SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host = can still access the storage ?? 5. What happen if SPM goes down ??? =20 Note :- What happen for point 4 &5 ,If storage and Compute both = working on the same server. =20 Thanks, Punit=20 _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_D6A50034-D6B0-41DD-A3C8-84BF3B838A86 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">You = need to setup a virtual IP to use as the mount point, most people use = keepalived to provide a virtual ip via vrrp for this. Setup something = like 10.10.10.10 and use that for your mounts.<div><br></div><div>Right = now, if 10.10.10.2 goes away, all your gluster mounts go away and your = VMs get paused because the hypervisors can=92t access the storage. Your = gluster storage is still fine, but ovirt can=92t talk to it because = 10.10.10.2 isn=92t there.</div><div><br></div><div>If the SPM goes down, = it the other hypervisor hosts will elect a new one (under control of the = ovirt engine).</div><div><br></div><div>Same scenarios if storage & = compute are on the same server, you still need a vip address for the = storage portion to serve as the mount point so it=92s not dependent on = any one server.<br><br></div><div>-Darrell<br><br><div><div>On Jul 3, = 2014, at 1:14 AM, Punit Dambiwal <<a = href=3D"mailto:hypunit@gmail.com">hypunit@gmail.com</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><blockquote = type=3D"cite"><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"><div = dir=3D"ltr"><div><div><div><div><div><div><div><div>Hi,<br><br></div>I = have some HA related concern about glusterfs with Ovirt...let say i have = 4 storage node with gluster bricks as below :- <br><br></div>1. = 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed = replicated architecture...<br> </div>2. Now attached this gluster storge to ovrit-engine with the = following mount point <a = href=3D"http://10.10.10.2/vol1">10.10.10.2/vol1</a><br></div>3. In my = cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on = 10.10.10.5...<br> </div>4. What happen if 10.10.10.2 will goes down.....can hypervisior = host can still access the storage ??<br></div>5. What happen if SPM goes = down ???<br><br></div>Note :- What happen for point 4 &5 ,If storage = and Compute both working on the same server.<br> <br></div>Thanks,<br>Punit <br></div> _______________________________________________<br>Users mailing = list<br><a = href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><br>http://lists.ovirt.= org/mailman/listinfo/users<br></blockquote></div><br></div></body></html>= --Apple-Mail=_D6A50034-D6B0-41DD-A3C8-84BF3B838A86--
Hi, Thanks...can you suggest me any good how to/article for the glusterfs with ovirt... One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen.... --------------------- Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. --------------------- Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition). On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote:
You need to setup a virtual IP to use as the mount point, most people use keepalived to provide a virtual ip via vrrp for this. Setup something like 10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Don't forget to take into consideration quroum, that's something people often forget The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state. For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/ I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300 HTH On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote:
You need to setup a virtual IP to use as the mount point, most people use keepalived to provide a virtual ip via vrrp for this. Setup something like 10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Andrew, Thanks for the update....that means HA can not work without VIP in the gluster,so better to use the glusterfs with the VIP to take over the ip...in case of any storage node failure... On Fri, Jul 4, 2014 at 12:35 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Don't forget to take into consideration quroum, that's something people often forget
The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state.
For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300
HTH
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com
wrote:
You need to setup a virtual IP to use as the mount point, most people
use
keepalived to provide a virtual ip via vrrp for this. Setup something
On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com> wrote: like
10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Or just localhost as your computer and storage are on the same box. On Fri, Jul 4, 2014 at 2:48 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Thanks for the update....that means HA can not work without VIP in the gluster,so better to use the glusterfs with the VIP to take over the ip...in case of any storage node failure...
On Fri, Jul 4, 2014 at 12:35 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Don't forget to take into consideration quroum, that's something people often forget
The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state.
For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300
HTH
On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote:
You need to setup a virtual IP to use as the mount point, most people use keepalived to provide a virtual ip via vrrp for this. Setup something like 10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Andrew, Yes..both on the same node...but i have 4 nodes of this type in the same cluster....So it should work or not ?? 1. 4 physical nodes with 12 bricks each(distributed replicated)... 2. The same all 4 nodes use for the compute purpose also... Do i still require the VIP...or not ?? because i tested even the mount point node goes down...the VM will not pause and not affect... On Fri, Jul 4, 2014 at 1:18 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Or just localhost as your computer and storage are on the same box.
On Fri, Jul 4, 2014 at 2:48 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Thanks for the update....that means HA can not work without VIP in the gluster,so better to use the glusterfs with the VIP to take over the ip...in case of any storage node failure...
On Fri, Jul 4, 2014 at 12:35 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Don't forget to take into consideration quroum, that's something people often forget
The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state.
For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300
HTH
On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com>
wrote:
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote:
You need to setup a virtual IP to use as the mount point, most people use keepalived to provide a virtual ip via vrrp for this. Setup something like 10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away
and
your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
As long as all your compute nodes are part of the gluster peer, localhost will work. Just remember, gluster will connect to any server, so even if you mount as localhost:/ it could be accessing the storage from another host in the gluster peer group. On Fri, Jul 4, 2014 at 3:26 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Yes..both on the same node...but i have 4 nodes of this type in the same cluster....So it should work or not ??
1. 4 physical nodes with 12 bricks each(distributed replicated)... 2. The same all 4 nodes use for the compute purpose also...
Do i still require the VIP...or not ?? because i tested even the mount point node goes down...the VM will not pause and not affect...
On Fri, Jul 4, 2014 at 1:18 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Or just localhost as your computer and storage are on the same box.
On Fri, Jul 4, 2014 at 2:48 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Thanks for the update....that means HA can not work without VIP in the gluster,so better to use the glusterfs with the VIP to take over the ip...in case of any storage node failure...
On Fri, Jul 4, 2014 at 12:35 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Don't forget to take into consideration quroum, that's something people often forget
The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state.
For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300
HTH
On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote:
You need to setup a virtual IP to use as the mount point, most people use keepalived to provide a virtual ip via vrrp for this. Setup something like 10.10.10.10 and use that for your mounts.
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there.
If the SPM goes down, it the other hypervisor hosts will elect a new one (under control of the ovirt engine).
Same scenarios if storage & compute are on the same server, you still need a vip address for the storage portion to serve as the mount point so it’s not dependent on any one server.
-Darrell
On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
I have some HA related concern about glusterfs with Ovirt...let say i have 4 storage node with gluster bricks as below :-
1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have distributed replicated architecture... 2. Now attached this gluster storge to ovrit-engine with the following mount point 10.10.10.2/vol1 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to 10.10.10.7) SPM is on 10.10.10.5... 4. What happen if 10.10.10.2 will goes down.....can hypervisior host can still access the storage ?? 5. What happen if SPM goes down ???
Note :- What happen for point 4 &5 ,If storage and Compute both working on the same server.
Thanks, Punit _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
I was never able to achieve a stable system that could survive the loss of a single node with glusterfs. I attempted to use replica 2 across 3 nodes (which required 2 bricks per node as the number of bricks must be a multiple of the replica, and you have to order them so the brick pairs span servers). I enabled server-side quorum, but found out later that client side quorum is based on 'sub volumes', which means that with a single node failure on replica 2, even though there were 3 nodes, it would go into a readonly state. After disabling client-side quorum (but keeping server side quorum) I thought the issue was fixed, but every once in a while, rebooting one of the nodes (after ensuring gluster was healed) would lead to i/o errors on the VM guest and essentially make it so it needed to be rebooted (which was successful and everything worked after even before bringing the downed node back up). My nodes were all combined glusterfs and ovirt nodes. I tried using both 'localhost' on the nodes as well as using a keepalived VIP. Its possible my issues were all due to client-side quorum not being enabled, but that would require replica 3 to be able to survive a single node failure, but I never pursued testing that theory. Also, heal times seemed a bit long for a single idle VM, it would consume 2 full cores of CPU for about 5 minutes for healing a single idle VM (granted, I was testing on a 1Gbps network, but that doesn't explain the CPU usage). -Brad On 7/4/14 1:29 AM, Andrew Lau wrote:
As long as all your compute nodes are part of the gluster peer, localhost will work. Just remember, gluster will connect to any server, so even if you mount as localhost:/ it could be accessing the storage from another host in the gluster peer group.
On Fri, Jul 4, 2014 at 3:26 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Yes..both on the same node...but i have 4 nodes of this type in the same cluster....So it should work or not ??
1. 4 physical nodes with 12 bricks each(distributed replicated)... 2. The same all 4 nodes use for the compute purpose also...
Do i still require the VIP...or not ?? because i tested even the mount point node goes down...the VM will not pause and not affect...
On Fri, Jul 4, 2014 at 1:18 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Or just localhost as your computer and storage are on the same box.
On Fri, Jul 4, 2014 at 2:48 PM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi Andrew,
Thanks for the update....that means HA can not work without VIP in the gluster,so better to use the glusterfs with the VIP to take over the ip...in case of any storage node failure...
On Fri, Jul 4, 2014 at 12:35 PM, Andrew Lau <andrew@andrewklau.com> wrote:
Don't forget to take into consideration quroum, that's something people often forget
The reason you're having the current happen, is gluster only uses the initial IP address to get the volume details. After that it'll connect directly to ONE of the servers, so with your 2 storage server case, 50% chance it won't go to paused state.
For the VIP, you could consider CTDB or keepelived, or even just using localhost (as your storage and compute are all on the same machine). For CTDB, checkout http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
I have a BZ open regarding gluster VMs going into paused state and not being resumable, so it's something you should also consider. My case, switch dies, gluster volume goes away, VMs go into paused state but can't be resumed. If you lose one server out of a cluster is a different story though. https://bugzilla.redhat.com/show_bug.cgi?id=1058300
HTH
On Fri, Jul 4, 2014 at 11:48 AM, Punit Dambiwal <hypunit@gmail.com> wrote:
Hi,
Thanks...can you suggest me any good how to/article for the glusterfs with ovirt...
One strange thing is if i will try both (compute & storage) on the same node...the below quote not happen....
---------------------
Right now, if 10.10.10.2 goes away, all your gluster mounts go away and your VMs get paused because the hypervisors can’t access the storage. Your gluster storage is still fine, but ovirt can’t talk to it because 10.10.10.2 isn’t there. ---------------------
Even the 10.10.10.2 goes down...i can still access the gluster mounts and no VM pause....i can access the VM via ssh...no connection failure.....the connection drop only in case of SPM goes down and the another node will elect as SPM(All the running VM's pause in this condition).
On Fri, Jul 4, 2014 at 4:12 AM, Darrell Budic <darrell.budic@zenfire.com> wrote: > > You need to setup a virtual IP to use as the mount point, most > people > use > keepalived to provide a virtual ip via vrrp for this. Setup > something > like > 10.10.10.10 and use that for your mounts. > > Right now, if 10.10.10.2 goes away, all your gluster mounts go away > and > your VMs get paused because the hypervisors can’t access the > storage. > Your > gluster storage is still fine, but ovirt can’t talk to it because > 10.10.10.2 > isn’t there. > > If the SPM goes down, it the other hypervisor hosts will elect a new > one > (under control of the ovirt engine). > > Same scenarios if storage & compute are on the same server, you > still > need > a vip address for the storage portion to serve as the mount point so > it’s > not dependent on any one server. > > -Darrell > > On Jul 3, 2014, at 1:14 AM, Punit Dambiwal <hypunit@gmail.com> > wrote: > > Hi, > > I have some HA related concern about glusterfs with Ovirt...let say > i > have > 4 storage node with gluster bricks as below :- > > 1. 10.10.10.1 to 10.10.10.4 with 2 bricks each and i have > distributed > replicated architecture... > 2. Now attached this gluster storge to ovrit-engine with the > following > mount point 10.10.10.2/vol1 > 3. In my cluster i have 3 hypervisior hosts (10.10.10.5 to > 10.10.10.7) > SPM > is on 10.10.10.5... > 4. What happen if 10.10.10.2 will goes down.....can hypervisior host > can > still access the storage ?? > 5. What happen if SPM goes down ??? > > Note :- What happen for point 4 &5 ,If storage and Compute both > working > on > the same server. > > Thanks, > Punit > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > >
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 21-7-2014 19:34, Dan Kenigsberg wrote:
On Mon, Jul 21, 2014 at 06:05:45PM +0100, Dan Kenigsberg wrote:
On Mon, Jul 21, 2014 at 09:03:58AM -0400, Maurice James wrote:
I submitted a bug report https://bugzilla.redhat.com/show_bug.cgi?id=1121643
----- Original Message -----
From: "Joop" <jvdwege at xs4all.nl To: users at ovirt.org Sent: Monday, July 21, 2014 8:46:18 AM Subject: Re: [ovirt-users] 3.4.3 Network problem
On 21-7-2014 14:38, Maurice James wrote:
I just upgraded to 3.4.3, not its complaining that em1 and em2 are down. They are not down not sure why it thinks the interfaces are down. Its doing this for all 4 of my hosts It is a horrible bug, due to my http://gerrit.ovirt.org/29689, I'll try to send a quick fix asap.
Please help me verify that a removal of two lines http://gerrit.ovirt.org/#/c/30547/ fixes the issue.
I commented out the indicated 2 lines and could activate my host and itstayed activated (1h) while before this patch it would turn unresponsive quite quickly (minutes).Joop
I hit this too, it was complaining about both base interfaces in my bonded interface being down. Commenting out the two lines resolved the problem for me as well.
On 26-7-2014 20:20, Darrell Budic wrote:
I hit this too, it was complaining about both base interfaces in my bonded interface being down. Commenting out the two lines resolved the problem for me as well. _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users Version 4.14.xx-2 is out which fixes this so no need to patch if you update vdsm
Joop
participants (6)
-
Andrew Lau -
Brad House -
Darrell Budic -
Darrell Budic -
Joop -
Punit Dambiwal