[Users] Why choose glusterfs over iscsi for oVirt storage?
</span></span><![en= dif]>performance? (multiple layers of filesystems everywhere – fs in = vm + image on gluster + gluster + block filesystem)<o:p></o:p></p><p class= =3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><= ![if !supportLists]><span style=3D'mso-list:Ignore'>-<span style=3D'font:7.= 0pt "Times New Roman"'> &nbs=
--_000_C79B906B4ECB8E46A3281AC932C805502F49039Aexchangeredfish_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, I'm just setting up some storage for use with oVirt and am wondering why I = might choose glusterfs rather than just exporting a raid array as iscsi. T= he oVirt team seem to be pushing gluster (though that could just be because= it's a Red Hat product). Can anyone answer this one? What I have come up with is as follows. For: - easy to expand - redundancy across storage devices/easy replication - high availablility - performance - it's kind of cool :) - maintenance? Against (these are guesses): - performance? (multiple layers of filesystems everywhere - fs in = vm + image on gluster + gluster + block filesystem) - complexity - maintenance? Any help here is appreciated. Also, does the underlying block level filesy= stem matter here? VMs running under ovirt would be typical business applic= ations - file serving (samba4), email, databases, web servers, etc. Cheers, Justin. --_000_C79B906B4ECB8E46A3281AC932C805502F49039Aexchangeredfish_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content= =3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros= oft Word 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {mso-style-priority:34; margin-top:0cm; margin-right:0cm; margin-bottom:0cm; margin-left:36.0pt; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-fareast-language:EN-US;} span.EmailStyle17 {mso-style-type:personal-compose; font-family:"Calibri","sans-serif"; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri","sans-serif"; mso-fareast-language:EN-US;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} /* List Definitions */ @list l0 {mso-list-id:913467833; mso-list-type:hybrid; mso-list-template-ids:-1938410588 947050786 201916419 201916421 201916417 = 201916419 201916421 201916417 201916419 201916421;} @list l0:level1 {mso-level-start-at:0; mso-level-number-format:bullet; mso-level-text:-; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Calibri","sans-serif"; mso-fareast-font-family:Calibri; mso-bidi-font-family:"Times New Roman";} @list l0:level2 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} @list l0:level3 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} @list l0:level4 {mso-level-number-format:bullet; mso-level-text:\F0B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Symbol;} @list l0:level5 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} @list l0:level6 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} @list l0:level7 {mso-level-number-format:bullet; mso-level-text:\F0B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Symbol;} @list l0:level8 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} @list l0:level9 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} ol {margin-bottom:0cm;} ul {margin-bottom:0cm;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DEN-AU link=3D"#0563C1= " vlink=3D"#954F72"><div class=3DWordSection1><p class=3DMsoNormal>Hi,<o:p>= </o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>I&= #8217;m just setting up some storage for use with oVirt and am wondering wh= y I might choose glusterfs rather than just exporting a raid array as iscsi= . The oVirt team seem to be pushing gluster (though that could just b= e because it’s a Red Hat product). Can anyone answer this one?<= o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNorma= l>What I have come up with is as follows.<o:p></o:p></p><p class=3DMsoNorma= l><o:p> </o:p></p><p class=3DMsoNormal>For:<o:p></o:p></p><p class=3DM= soListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if= !supportLists]><span style=3D'mso-list:Ignore'>-<span style=3D'font:7.0pt = "Times New Roman"'> <= /span></span><![endif]>easy to expand<o:p></o:p></p><p class=3DMsoListParag= raph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLi= sts]><span style=3D'mso-list:Ignore'>-<span style=3D'font:7.0pt "Times New = Roman"'> </span></spa= n><![endif]>redundancy across storage devices/easy replication<o:p></o:p></= p><p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 leve= l1 lfo1'><![if !supportLists]><span style=3D'mso-list:Ignore'>-<span style= =3D'font:7.0pt "Times New Roman"'> = ; </span></span><![endif]>high availablility<o:p></o:p></p><p c= lass=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo= 1'><![if !supportLists]><span style=3D'mso-list:Ignore'>-<span style=3D'fon= t:7.0pt "Times New Roman"'> = </span></span><![endif]>performance<o:p></o:p></p><p class=3DMsoList= Paragraph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supp= ortLists]><span style=3D'mso-list:Ignore'>-<span style=3D'font:7.0pt "Times= New Roman"'> </span>= </span><![endif]>it’s kind of cool <span style=3D'font-family:Wingdin= gs'>J</span><o:p></o:p></p><p class=3DMsoListParagraph style=3D'text-indent= :-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style=3D'mso-l= ist:Ignore'>-<span style=3D'font:7.0pt "Times New Roman"'> = ; </span></span><![endif]>maintenance?<= o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNorma= l>Against (these are guesses):<o:p></o:p></p><p class=3DMsoListParagraph st= yle=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><s= pan style=3D'mso-list:Ignore'>-<span style=3D'font:7.0pt "Times New Roman"'= p; </span></span><![endif]>complexity<o:p></o:p></p><p class=3DMsoListParag= raph style=3D'text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLi= sts]><span style=3D'mso-list:Ignore'>-<span style=3D'font:7.0pt "Times New = Roman"'> </span></spa= n><![endif]>maintenance?<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:= p></p><p class=3DMsoNormal>Any help here is appreciated. Also, does t= he underlying block level filesystem matter here? VMs running under o= virt would be typical business applications – file serving (samba4), = email, databases, web servers, etc.<o:p></o:p></p><p class=3DMsoNormal><o:p=
</o:p></p><p class=3DMsoNormal>Cheers,<o:p></o:p></p><p class=3DMsoN= ormal>Justin.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p cl= ass=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal><o:p> </o:p>= </p></div></body></html>=
--_000_C79B906B4ECB8E46A3281AC932C805502F49039Aexchangeredfish_--
On Mon, Feb 17, 2014 at 3:50 AM, Justin Clacherty <justin@redfish.com.au>wrote:
Hi,
I'm just setting up some storage for use with oVirt and am wondering why I might choose glusterfs rather than just exporting a raid array as iscsi. The oVirt team seem to be pushing gluster (though that could just be because it's a Red Hat product). Can anyone answer this one?
If you're contemplating software iscsi then you probably aren't working on a production setup that requires high availability or data consistency. If you are, you should take a deep dive on what your single point of failures are. If you're actually talking about a SAN lun, then I would imagine you aren't the storage admin, and can leave the particulars in their hands. Its funny, in RH's current supported version of RHEV, native gluster storage isn't even an option. This is a pretty new project, and if you follow the bread crumbs on distributed file systems like glusterfs or ceph you'll start to see the future advantages. You can also figure that seeing as both projects are spearheaded by RH dev's that there is likely a lot of cross-talk and feature requests making both projects better, and who wouldn't promote a better solution?
What I have come up with is as follows.
For:
There is too much to cover in any of these advantages, you're going to need to do a lot of research on each of these 'features' if you want to use them successfully.
- easy to expand
- redundancy across storage devices/easy replication
- high availablility
- performance
- it's kind of cool J
- maintenance?
Against (these are guesses):
- performance? (multiple layers of filesystems everywhere - fs in vm + image on gluster + gluster + block filesystem)
Its worse than just a ton of software layers. NAS protocols seem to be focused on data consistency, which means they don't write async to storage. iscsi is typically async and has much better throughput, but also a greater chance for data loss or corruption. Technically you can achieve the same level of performance as iscsi using NFS (backed by glusterfs if you like) but you need to set options on the volume to allow async writes.
- complexity
If you're doing storage properly, the underpinnings are always complex unless you're paying someone else to do it (read: SAN / managed service provider). Research multipath and HA on software iscsi and you'll see what I mean.
- maintenance?
Any help here is appreciated. Also, does the underlying block level filesystem matter here? VMs running under ovirt would be typical business applications - file serving (samba4), email, databases, web servers, etc.
There is a lot to answer here and I don't have all the answers. Take a look at the gluster docs for underlying file system requirements. Any block device should work. Specifically I'll mention that the glusterfs team doesn't suggest hosting db's on glusterfs - many small reads/writes are not one of glusters strong points.
Cheers,
Justin.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Justin Clacherty -
Steve Dainard