[Users] Problem with creating a glusterfs volume
So I have a problem creating glusterfs volumes. Here is the install: 1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt 5. created a directory on each node this path /data 6. chmod 36.36 -R /data all nodes 7. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data I received this error: Creation of Gluster Volume maingfs1 failed. I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean? -- Dominic Kaiser Greater Boston Vineyard Director of Operations cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org
This is a multi-part message in MIME format. --------------030101030201050005020508 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 09/05/2012 02:20 PM, Dominic Kaiser wrote:
So I have a problem creating glusterfs volumes. Here is the install:
1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt
Did you confirm the peers were working after adding them in as adding a host updates the firewall.
1. created a directory on each node this path /data 2. chmod 36.36 -R /data all nodes 3. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data
Want to confirm you are using gluster 3.3 not 3.2.
I received this error:
Creation of Gluster Volume maingfs1 failed.
I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?
You can review the steps I did to get a working gluster setup at http://www.middleswarth.net/content/installing-ovirt-31-and-glusterfs-using-...
-- Dominic Kaiser Greater Boston Vineyard Director of Operations
cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org <mailto:dominic@bostonvineyard.org>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Thanks Robert Middleswarth @rmiddle (twitter/Freenode IRC) @RobertM (OFTC IRC) --------------030101030201050005020508 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">On 09/05/2012 02:20 PM, Dominic Kaiser wrote:<br> </div> <blockquote cite="mid:CAOZfw6uiCCeww1y7Zi=DugKhJFKRemN+P5a=fPN-fhUNqARHSA@mail.gmail.com" type="cite">So I have a problem creating glusterfs volumes. Here is the install: <div><br> </div> <div> <ol> <li>Ovirt 3.1</li> <li>4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64</li> <li>4 nodes peer joined and running</li> <li>4 nodes added as hosts to ovirt</li> </ol> </div> </blockquote> Did you confirm the peers were working after adding them in as adding a host updates the firewall.<br> <blockquote cite="mid:CAOZfw6uiCCeww1y7Zi=DugKhJFKRemN+P5a=fPN-fhUNqARHSA@mail.gmail.com" type="cite"> <div> <ol> <li>created a directory on each node this path /data</li> <li>chmod 36.36 -R /data all nodes</li> <li>went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data</li> </ol> </div> </blockquote> Want to confirm you are using gluster 3.3 not 3.2.<br> <blockquote cite="mid:CAOZfw6uiCCeww1y7Zi=DugKhJFKRemN+P5a=fPN-fhUNqARHSA@mail.gmail.com" type="cite"> <div>I received this error:</div> <div><br> </div> <div><span style="background-color:rgb(255,255,255)"><font color="#ff0000">Creation of Gluster Volume maingfs1 failed.</font></span></div> <div><span style="background-color:rgb(255,255,255)"><font color="#ff0000"><br> </font></span></div> <div>I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?</div> <div><br> </div> </blockquote> You can review the steps I did to get a working gluster setup at <a class="moz-txt-link-freetext" href="http://www.middleswarth.net/content/installing-ovirt-31-and-glusterfs-using-either-nfs-or-posix-native-file-system">http://www.middleswarth.net/content/installing-ovirt-31-and-glusterfs-using-either-nfs-or-posix-native-file-system</a><br> <blockquote cite="mid:CAOZfw6uiCCeww1y7Zi=DugKhJFKRemN+P5a=fPN-fhUNqARHSA@mail.gmail.com" type="cite"> <div> <div><br> </div> -- <br> Dominic Kaiser<br> Greater Boston Vineyard<br> Director of Operations<br> <br> cell: 617-230-1412<br> fax: 617-252-0238<br> email: <a moz-do-not-send="true" href="mailto:dominic@bostonvineyard.org">dominic@bostonvineyard.org</a><br> <br> <br> </div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> <pre class="moz-signature" cols="72">-- Thanks Robert Middleswarth @rmiddle (twitter/Freenode IRC) @RobertM (OFTC IRC) </pre> </body> </html> --------------030101030201050005020508--
Did you confirm the peers were working after adding them in as adding a host updates the firewall. Yes here it is. [root@gfs1 ~]# gluster gluster> peer status Number of Peers: 3 Hostname: gfs2 Uuid: dce3ed1d-d38b-4eaa-81e5-12577a2f1a39 State: Peer in Cluster (Connected) Hostname: gfs3 Uuid: 807b83f9-bff2-4cf1-a002-f1f1c5f694ad State: Peer in Cluster (Connected) Hostname: gfs4 Uuid: 7699d992-ee42-45c2-afca-d51dcdd9fdf6 State: Peer in Cluster (Connected) Want to confirm you are using gluster 3.3 not 3.2. Yes using 3.3 Installed Packages glusterfs.x86_64 3.3.0-5.fc17 @fedora-glusterfs glusterfs-fuse.x86_64 3.3.0-5.fc17 @fedora-glusterfs glusterfs-geo-replication.x86_64 3.3.0-5.fc17 @fedora-glusterfs glusterfs-rdma.x86_64 3.3.0-5.fc17 @fedora-glusterfs glusterfs-server.x86_64 3.3.0-5.fc17 I reviewed your steps. The only thing I can think is let us say you create a /data directory. When I am adding bricks should I add for instance 10.4.0.97 /data 10.4.0.98 /data 10.4.0.99 /data 10.4.0.100 /data? Those are the directories I have created and owned on each server 36.36 -R /data. On Wed, Sep 5, 2012 at 2:25 PM, Robert Middleswarth <robert@middleswarth.net
wrote:
On 09/05/2012 02:20 PM, Dominic Kaiser wrote:
So I have a problem creating glusterfs volumes. Here is the install:
1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt
Did you confirm the peers were working after adding them in as adding a host updates the firewall.
1. created a directory on each node this path /data 2. chmod 36.36 -R /data all nodes 3. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data
Want to confirm you are using gluster 3.3 not 3.2.
I received this error:
Creation of Gluster Volume maingfs1 failed.
I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?
You can review the steps I did to get a working gluster setup at http://www.middleswarth.net/content/installing-ovirt-31-and-glusterfs-using-...
-- Dominic Kaiser Greater Boston Vineyard Director of Operations
cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
-- Thanks Robert Middleswarth @rmiddle (twitter/Freenode IRC) @RobertM (OFTC IRC)
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Dominic Kaiser Greater Boston Vineyard Director of Operations cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org
----- Original Message -----
From: "Dominic Kaiser" <dominic@bostonvineyard.org> To: users@ovirt.org Sent: Wednesday, September 5, 2012 11:20:31 AM Subject: [Users] Problem with creating a glusterfs volume
So I have a problem creating glusterfs volumes. Here is the install:
1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt 5. created a directory on each node this path /data 6. chmod 36.36 -R /data all nodes 7. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data
Is this an NFS domain? Is your volume named "data" -- the path should match the gluster volume name, not the path on the filesystem (unless they're the same). If your volume is "maingfs1" then the nfs path should be hostname:/maingfs1. Also, I've found that I've had to chown 36.36 -R /data after creating the volumes, because the subdirs that get created when you create a volume start out as root-owned by default. Jason
I received this error:
Creation of Gluster Volume maingfs1 failed.
I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?
-- Dominic Kaiser Greater Boston Vineyard Director of Operations
cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
This is a multi-part message in MIME format. --------------030704030404030303080902 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi Dominic, Looking at the engine log immediately after trying to create the volume should tell us on which node the gluster volume creation was attempted. Then looking at the vdsm log on that node should help us identifying the exact reason for failure. In case this doesn't help you, can you please simulate the issue again and send back all the 5 log files? (engine.log from engine server and vdsm.log from the 4 nodes) Regards, Shireesh On Wednesday 05 September 2012 11:50 PM, Dominic Kaiser wrote:
So I have a problem creating glusterfs volumes. Here is the install:
1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt 5. created a directory on each node this path /data 6. chmod 36.36 -R /data all nodes 7. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data
I received this error:
Creation of Gluster Volume maingfs1 failed.
I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?
-- Dominic Kaiser Greater Boston Vineyard Director of Operations
cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org <mailto:dominic@bostonvineyard.org>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--------------030704030404030303080902 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">Hi Dominic,<br> <br> Looking at the engine log immediately after trying to create the volume should tell us on which node the gluster volume creation was attempted. Then looking at the vdsm log on that node should help us identifying the exact reason for failure.<br> <br> In case this doesn't help you, can you please simulate the issue again and send back all the 5 log files? (engine.log from engine server and vdsm.log from the 4 nodes)<br> <br> Regards,<br> Shireesh<br> <br> On Wednesday 05 September 2012 11:50 PM, Dominic Kaiser wrote:<br> </div> <blockquote cite="mid:CAOZfw6uiCCeww1y7Zi=DugKhJFKRemN+P5a=fPN-fhUNqARHSA@mail.gmail.com" type="cite">So I have a problem creating glusterfs volumes. Here is the install: <div><br> </div> <div> <ol> <li>Ovirt 3.1</li> <li>4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64</li> <li>4 nodes peer joined and running</li> <li>4 nodes added as hosts to ovirt</li> <li>created a directory on each node this path /data</li> <li>chmod 36.36 -R /data all nodes</li> <li>went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data</li> </ol> I received this error:</div> <div><br> </div> <div><span style="background-color:rgb(255,255,255)"><font color="#ff0000">Creation of Gluster Volume maingfs1 failed.</font></span></div> <div><span style="background-color:rgb(255,255,255)"><font color="#ff0000"><br> </font></span></div> <div>I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?</div> <div><br> </div> <div> <div><br> </div> -- <br> Dominic Kaiser<br> Greater Boston Vineyard<br> Director of Operations<br> <br> cell: 617-230-1412<br> fax: 617-252-0238<br> email: <a moz-do-not-send="true" href="mailto:dominic@bostonvineyard.org">dominic@bostonvineyard.org</a><br> <br> <br> </div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> </body> </html> --------------030704030404030303080902--
I just ran into this as well, and it seems that you have to either reformat previously used gluster bricks or manually tweak some extended attributes. Maybe this helps you in setting up your gluster volume, Dominic? More info here: https://bugzilla.redhat.com/show_bug.cgi?id=812214 Maxim Burgerhout maxim@wzzrd.com ---------------- EB11 5E56 E648 9D99 E8EF 05FB C513 6FD4 1302 B48A On Thu, Sep 6, 2012 at 7:50 AM, Shireesh Anjal <sanjal@redhat.com> wrote:
Hi Dominic,
Looking at the engine log immediately after trying to create the volume should tell us on which node the gluster volume creation was attempted. Then looking at the vdsm log on that node should help us identifying the exact reason for failure.
In case this doesn't help you, can you please simulate the issue again and send back all the 5 log files? (engine.log from engine server and vdsm.log from the 4 nodes)
Regards, Shireesh
On Wednesday 05 September 2012 11:50 PM, Dominic Kaiser wrote:
So I have a problem creating glusterfs volumes. Here is the install:
1. Ovirt 3.1 2. 4 Nodes are Fedora 17 with kernel 3.3.4 - 5.fc17.x86_64 3. 4 nodes peer joined and running 4. 4 nodes added as hosts to ovirt 5. created a directory on each node this path /data 6. chmod 36.36 -R /data all nodes 7. went back to ovirt and created a distributed/replicated volume and added the 4 nodes with brick path of /data
I received this error:
Creation of Gluster Volume maingfs1 failed.
I went and looked at the vdsm logs on the nodes and the ovirt server which did not say much. Where else should I look? Also this error is vague what does it mean?
-- Dominic Kaiser Greater Boston Vineyard Director of Operations
cell: 617-230-1412 fax: 617-252-0238 email: dominic@bostonvineyard.org
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (5)
-
Dominic Kaiser -
Jason Brooks -
Maxim Burgerhout -
Robert Middleswarth -
Shireesh Anjal