Hi Jorick, which version of GlusterFS are you using? Try to restart glusterd on each of the bricks, then try to follow this link: http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-pa... Please tell me if this helps Regards, Maor ----- Original Message ----- From: "Jorick Astrego" <j.astrego(a)netbulae.eu&gt; To: users(a)ovirt.org Sent: Tuesday, August 12, 2014 3:16:06 PM Subject: [ovirt-users] method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) Hi, I'm trying to test glusterfs on a couple of Centos 7 ovirt nodes with ovirt 3.5rc1. I've enabled glusterfs service for the cluster, created a xfs data partition with mount point, installed "vdsm-gluster" rpm and started glusterfsd. I also cleared the firewall rules. When I try to add the storage domain, I get the following error 2014-08-12 14:07:07,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a), log id: 5d2ee913 2014-08-12 14:07:07,350 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a) execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterVolumesList" is not supported 2014-08-12 14:07:07,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, log id: 5d2ee913 2014-08-12 14:07:07,351 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-65) Error while refreshing Gluster lightweight data of cluster Default!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:65) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:406) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:392) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:363) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:108) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Anything else I need to configure? Kind regards, Jorick Astrego _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

This is a multi-part message in MIME format. --------------050201050003010402050804 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Could you provide the vdsm and supervdsm log from the node? On 08/12/2014 07:08 PM, Jorick Astrego wrote: --------------050201050003010402050804 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> Could you provide the vdsm and supervdsm log from the node?<br> <br> <div class="moz-cite-prefix">On 08/12/2014 07:08 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA18EE.6030103@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> Hi,<br> <br> No restarting glusterfsd doesn't help. The steps in the link you gave me don't help either. The node is clean installed and has never been a part of a glusterfs.... Running gluster version 3.5.2 <br> <blockquote>glusterfs-server-3.5.2-1.el7.x86_64<br> glusterfs-api-3.5.2-1.el7.x86_64<br> glusterfs-fuse-3.5.2-1.el7.x86_64<br> glusterfs-cli-3.5.2-1.el7.x86_64<br> vdsm-gluster-4.16.1-4.gitb2bf270.el7.noarch<br> glusterfs-libs-3.5.2-1.el7.x86_64<br> glusterfs-3.5.2-1.el7.x86_64<br> glusterfs-rdma-3.5.2-1.el7.x86_64<br> <br> </blockquote> From the log:<br> <blockquote>[2014-08-12 11:57:30.006092] W [glusterfsd.c:1095:cleanup_and_exit] (--&gt;/lib64/libc.so.6(clone+0x6d) [0x7fac29bc23dd] (--&gt;/lib64/libpthread.so.0(+0x7df3) [0x7fac2a279df3] (--&gt;/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x7fac2b8471b5]))) 0-: received signum (15), shutting down<br> [2014-08-12 11:57:30.014919] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.2 (/usr/sbin/glusterd -p /run/glusterd.pid)<br> [2014-08-12 11:57:30.018089] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory<br> [2014-08-12 11:57:30.020315] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.020331] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.021533] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device)<br> [2014-08-12 11:57:30.021548] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device<br> [2014-08-12 11:57:30.021555] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed<br> [2014-08-12 11:57:30.021610] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed<br> [2014-08-12 11:57:30.021669] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.021679] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.022603] I [glusterd.c:367:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system<br> <b>[2014-08-12 11:57:30.023046] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><b><br> </b><b>[2014-08-12 11:57:30.023075] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><br> [2014-08-12 11:57:30.023090] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30501<br> Final graph:<br> +------------------------------------------------------------------------------+<br> &nbsp; 1: volume management<br> &nbsp; 2:&nbsp;&nbsp;&nbsp;&nbsp; type mgmt/glusterd<br> &nbsp; 3:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-glusterfs on<br> &nbsp; 4:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-unix on<br> &nbsp; 5:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-null on<br> &nbsp; 6:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.listen-backlog 128<br> &nbsp; 7:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.read-fail-log off<br> &nbsp; 8:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-interval 2<br> &nbsp; 9:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-time 10<br> &nbsp;10:&nbsp;&nbsp;&nbsp;&nbsp; option transport-type rdma<br> &nbsp;11:&nbsp;&nbsp;&nbsp;&nbsp; option working-directory /var/lib/glusterd<br> &nbsp;12: end-volume<br> &nbsp;13: <br> +------------------------------------------------------------------------------+<br> <br> </blockquote> Jorick<br> <br> <div class="moz-cite-prefix">On 08/12/2014 02:56 PM, Maor Lipchuk wrote:<br> </div> <blockquote cite="mid:1418364179.36891472.1407848205422.JavaMail.zimbra@redhat.com" type="cite"> <pre wrap="">Hi Jorick, which version of GlusterFS are you using? Try to restart glusterd on each of the bricks, then try to follow this link: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/">http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/</a> Please tell me if this helps Regards, Maor ----- Original Message ----- From: "Jorick Astrego" <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:j.astrego@netbulae.eu">&lt;j.astrego@netbulae.eu&gt;</a> To: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:users@ovirt.org">users@ovirt.org</a> Sent: Tuesday, August 12, 2014 3:16:06 PM Subject: [ovirt-users] method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) Hi, I'm trying to test glusterfs on a couple of Centos 7 ovirt nodes with ovirt 3.5rc1. I've enabled glusterfs service for the cluster, created a xfs data partition with mount point, installed "vdsm-gluster" rpm and started glusterfsd. I also cleared the firewall rules. When I try to add the storage domain, I get the following error 2014-08-12 14:07:07,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a), log id: 5d2ee913 2014-08-12 14:07:07,350 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a) execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported 2014-08-12 14:07:07,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, log id: 5d2ee913 2014-08-12 14:07:07,351 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-65) Error while refreshing Gluster lightweight data of cluster Default!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:65) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:406) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:392) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:363) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:108) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Anything else I need to configure? Kind regards, Jorick Astrego _______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </body> </html> --------------050201050003010402050804--

This is a multi-part message in MIME format. --------------080305020007030506080900 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 08/12/2014 08:10 PM, Jorick Astrego wrote: I don't see any errors here corresponding to the unsupported method "glusterVolumesList" Are these logs from the node "node3.test.nu"? However, this error is unrelated to adding the storage domain. Typically, this unsupported method error is returned from vdsm when vdsm-gluster is not installed on the node. You could try running the following on node3.test.nu #vdsClient -s localhost glusterVolumesList --------------080305020007030506080900 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <br> <div class="moz-cite-prefix">On 08/12/2014 08:10 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA2745.7040705@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <br> <div class="moz-cite-prefix">On 08/12/2014 03:44 PM, Sahina Bose wrote:<br> </div> <blockquote cite="mid:53EA1A29.5080806@redhat.com" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> Could you provide the vdsm and supervdsm log from the node?<br> <br> </blockquote> vdsm:<br> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/edZrvnkr">http://pastebin.com/edZr... (cut some repetitive loglines out to fit on pastebin)<br> <br> Supervdsm:<br> <br> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/V8xTTMKk">http://pastebin.com/V8xT... </blockquote> <br> <br> I don't see any errors here corresponding to the unsupported method "glusterVolumesList"<br> <br> Are these logs from the node "node3.test.nu"? <br> <br> However, this error is unrelated to adding the storage domain. Typically, this unsupported method error is returned from vdsm when vdsm-gluster is not installed on the node.<br> You could try running the following on node3.test.nu<br> #vdsClient -s localhost glusterVolumesList<br> <pre wrap=""> </pre> <blockquote cite="mid:53EA2745.7040705@netbulae.eu" type="cite"> <blockquote cite="mid:53EA1A29.5080806@redhat.com" type="cite"> <div class="moz-cite-prefix">On 08/12/2014 07:08 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA18EE.6030103@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> Hi,<br> <br> No restarting glusterfsd doesn't help. The steps in the link you gave me don't help either. The node is clean installed and has never been a part of a glusterfs.... Running gluster version 3.5.2 <br> <blockquote>glusterfs-server-3.5.2-1.el7.x86_64<br> glusterfs-api-3.5.2-1.el7.x86_64<br> glusterfs-fuse-3.5.2-1.el7.x86_64<br> glusterfs-cli-3.5.2-1.el7.x86_64<br> vdsm-gluster-4.16.1-4.gitb2bf270.el7.noarch<br> glusterfs-libs-3.5.2-1.el7.x86_64<br> glusterfs-3.5.2-1.el7.x86_64<br> glusterfs-rdma-3.5.2-1.el7.x86_64<br> <br> </blockquote> From the log:<br> <blockquote>[2014-08-12 11:57:30.006092] W [glusterfsd.c:1095:cleanup_and_exit] (--&gt;/lib64/libc.so.6(clone+0x6d) [0x7fac29bc23dd] (--&gt;/lib64/libpthread.so.0(+0x7df3) [0x7fac2a279df3] (--&gt;/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x7fac2b8471b5]))) 0-: received signum (15), shutting down<br> [2014-08-12 11:57:30.014919] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.2 (/usr/sbin/glusterd -p /run/glusterd.pid)<br> [2014-08-12 11:57:30.018089] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory<br> [2014-08-12 11:57:30.020315] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.020331] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.021533] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device)<br> [2014-08-12 11:57:30.021548] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device<br> [2014-08-12 11:57:30.021555] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed<br> [2014-08-12 11:57:30.021610] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed<br> [2014-08-12 11:57:30.021669] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.021679] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.022603] I [glusterd.c:367:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system<br> <b>[2014-08-12 11:57:30.023046] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><b><br> </b><b>[2014-08-12 11:57:30.023075] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><br> [2014-08-12 11:57:30.023090] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30501<br> Final graph:<br> +------------------------------------------------------------------------------+<br> &nbsp; 1: volume management<br> &nbsp; 2:&nbsp;&nbsp;&nbsp;&nbsp; type mgmt/glusterd<br> &nbsp; 3:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-glusterfs on<br> &nbsp; 4:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-unix on<br> &nbsp; 5:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-null on<br> &nbsp; 6:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.listen-backlog 128<br> &nbsp; 7:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.read-fail-log off<br> &nbsp; 8:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-interval 2<br> &nbsp; 9:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-time 10<br> &nbsp;10:&nbsp;&nbsp;&nbsp;&nbsp; option transport-type rdma<br> &nbsp;11:&nbsp;&nbsp;&nbsp;&nbsp; option working-directory /var/lib/glusterd<br> &nbsp;12: end-volume<br> &nbsp;13: <br> +------------------------------------------------------------------------------+<br> <br> </blockquote> Jorick<br> <br> <div class="moz-cite-prefix">On 08/12/2014 02:56 PM, Maor Lipchuk wrote:<br> </div> <blockquote cite="mid:1418364179.36891472.1407848205422.JavaMail.zimbra@redhat.com" type="cite"> <pre wrap="">Hi Jorick, which version of GlusterFS are you using? Try to restart glusterd on each of the bricks, then try to follow this link: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/">http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/</a> Please tell me if this helps Regards, Maor ----- Original Message ----- From: "Jorick Astrego" <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:j.astrego@netbulae.eu">&lt;j.astrego@netbulae.eu&gt;</a> To: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:users@ovirt.org">users@ovirt.org</a> Sent: Tuesday, August 12, 2014 3:16:06 PM Subject: [ovirt-users] method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) Hi, I'm trying to test glusterfs on a couple of Centos 7 ovirt nodes with ovirt 3.5rc1. I've enabled glusterfs service for the cluster, created a xfs data partition with mount point, installed "vdsm-gluster" rpm and started glusterfsd. I also cleared the firewall rules. When I try to add the storage domain, I get the following error 2014-08-12 14:07:07,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a), log id: 5d2ee913 2014-08-12 14:07:07,350 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a) execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported 2014-08-12 14:07:07,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, log id: 5d2ee913 2014-08-12 14:07:07,351 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-65) Error while refreshing Gluster lightweight data of cluster Default!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:65) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:406) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:392) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:363) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:108) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Anything else I need to configure? Kind regards, Jorick Astrego _______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </body> </html> --------------080305020007030506080900--

This is a multi-part message in MIME format. --------------020000030606060406020109 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 08/12/2014 10:59 PM, Jorick Astrego wrote: Gluster needs selinux to be in permissive mode. Could you change this and try? --------------020000030606060406020109 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <br> <div class="moz-cite-prefix">On 08/12/2014 10:59 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA4EFE.6030603@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <br> <div class="moz-cite-prefix">On 08/12/2014 05:09 PM, Sahina Bose wrote:<br> </div> <blockquote cite="mid:53EA2E1F.7080304@redhat.com" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <br> <div class="moz-cite-prefix">On 08/12/2014 08:10 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA2745.7040705@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <br> <div class="moz-cite-prefix">On 08/12/2014 03:44 PM, Sahina Bose wrote:<br> </div> <blockquote cite="mid:53EA1A29.5080806@redhat.com" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> Could you provide the vdsm and supervdsm log from the node?<br> <br> </blockquote> vdsm:<br> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/edZrvnkr">http://pastebin.com/edZr... (cut some repetitive loglines out to fit on pastebin)<br> <br> Supervdsm:<br> <br> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/V8xTTMKk">http://pastebin.com/V8xT... </blockquote> <br> <br> I don't see any errors here corresponding to the unsupported method "glusterVolumesList"<br> <br> Are these logs from the node "node3.test.nu"? <br> <br> However, this error is unrelated to adding the storage domain. Typically, this unsupported method error is returned from vdsm when vdsm-gluster is not installed on the node.<br> You could try running the following on node3.test.nu<br> #rge<br> </blockquote> <br> Found some selinux warnings:<br> <br> <blockquote>[78080.160692] type=1400 audit(1407844503.916:35): avc:&nbsp; denied&nbsp; { write } for&nbsp; pid=2286 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> [78219.465352] type=1404 audit(1407844643.107:36): enforcing=0 old_enforcing=1 auid=0 ses=106<br> [78222.942816] type=1107 audit(1407844646.582:37): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc:&nbsp; received setenforce notice (enforcing=0)<br> &nbsp;exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'<br> [78226.383697] type=1400 audit(1407844650.020:38): avc:&nbsp; denied&nbsp; { write } for&nbsp; pid=2348 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> [78226.383712] type=1400 audit(1407844650.020:39): avc:&nbsp; denied&nbsp; { unlink } for&nbsp; pid=2348 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> </blockquote> </blockquote> <br> <br> Gluster needs selinux to be in permissive mode. Could you change this and try?<br> <br> <br> <blockquote cite="mid:53EA4EFE.6030603@netbulae.eu" type="cite"> <blockquote> </blockquote> <br> <blockquote cite="mid:53EA2E1F.7080304@redhat.com" type="cite"> <blockquote cite="mid:53EA2745.7040705@netbulae.eu" type="cite"> <blockquote cite="mid:53EA1A29.5080806@redhat.com" type="cite"> <div class="moz-cite-prefix">On 08/12/2014 07:08 PM, Jorick Astrego wrote:<br> </div> <blockquote cite="mid:53EA18EE.6030103@netbulae.eu" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> Hi,<br> <br> No restarting glusterfsd doesn't help. The steps in the link you gave me don't help either. The node is clean installed and has never been a part of a glusterfs.... Running gluster version 3.5.2 <br> <blockquote>glusterfs-server-3.5.2-1.el7.x86_64<br> glusterfs-api-3.5.2-1.el7.x86_64<br> glusterfs-fuse-3.5.2-1.el7.x86_64<br> glusterfs-cli-3.5.2-1.el7.x86_64<br> vdsm-gluster-4.16.1-4.gitb2bf270.el7.noarch<br> glusterfs-libs-3.5.2-1.el7.x86_64<br> glusterfs-3.5.2-1.el7.x86_64<br> glusterfs-rdma-3.5.2-1.el7.x86_64<br> <br> </blockquote> From the log:<br> <blockquote>[2014-08-12 11:57:30.006092] W [glusterfsd.c:1095:cleanup_and_exit] (--&gt;/lib64/libc.so.6(clone+0x6d) [0x7fac29bc23dd] (--&gt;/lib64/libpthread.so.0(+0x7df3) [0x7fac2a279df3] (--&gt;/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x7fac2b8471b5]))) 0-: received signum (15), shutting down<br> [2014-08-12 11:57:30.014919] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.2 (/usr/sbin/glusterd -p /run/glusterd.pid)<br> [2014-08-12 11:57:30.018089] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory<br> [2014-08-12 11:57:30.020315] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.020331] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.021533] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device)<br> [2014-08-12 11:57:30.021548] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device<br> [2014-08-12 11:57:30.021555] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed<br> [2014-08-12 11:57:30.021610] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed<br> [2014-08-12 11:57:30.021669] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.021679] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.022603] I [glusterd.c:367:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system<br> <b>[2014-08-12 11:57:30.023046] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><b><br> </b><b>[2014-08-12 11:57:30.023075] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info, returned error: (No such file or directory)</b><br> [2014-08-12 11:57:30.023090] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30501<br> Final graph:<br> +------------------------------------------------------------------------------+<br> &nbsp; 1: volume management<br> &nbsp; 2:&nbsp;&nbsp;&nbsp;&nbsp; type mgmt/glusterd<br> &nbsp; 3:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-glusterfs on<br> &nbsp; 4:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-unix on<br> &nbsp; 5:&nbsp;&nbsp;&nbsp;&nbsp; option rpc-auth.auth-null on<br> &nbsp; 6:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.listen-backlog 128<br> &nbsp; 7:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.read-fail-log off<br> &nbsp; 8:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-interval 2<br> &nbsp; 9:&nbsp;&nbsp;&nbsp;&nbsp; option transport.socket.keepalive-time 10<br> &nbsp;10:&nbsp;&nbsp;&nbsp;&nbsp; option transport-type rdma<br> &nbsp;11:&nbsp;&nbsp;&nbsp;&nbsp; option working-directory /var/lib/glusterd<br> &nbsp;12: end-volume<br> &nbsp;13: <br> +------------------------------------------------------------------------------+<br> <br> </blockquote> Jorick<br> <br> <div class="moz-cite-prefix">On 08/12/2014 02:56 PM, Maor Lipchuk wrote:<br> </div> <blockquote cite="mid:1418364179.36891472.1407848205422.JavaMail.zimbra@redhat.com" type="cite"> <pre wrap="">Hi Jorick, which version of GlusterFS are you using? Try to restart glusterd on each of the bricks, then try to follow this link: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/">http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/</a> Please tell me if this helps Regards, Maor ----- Original Message ----- From: "Jorick Astrego" <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:j.astrego@netbulae.eu">&lt;j.astrego@netbulae.eu&gt;</a> To: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:users@ovirt.org">users@ovirt.org</a> Sent: Tuesday, August 12, 2014 3:16:06 PM Subject: [ovirt-users] method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) Hi, I'm trying to test glusterfs on a couple of Centos 7 ovirt nodes with ovirt 3.5rc1. I've enabled glusterfs service for the cluster, created a xfs data partition with mount point, installed "vdsm-gluster" rpm and started glusterfsd. I also cleared the firewall rules. When I try to add the storage domain, I get the following error 2014-08-12 14:07:07,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a), log id: 5d2ee913 2014-08-12 14:07:07,350 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GlusterVolumesListVDSCommand(HostName = node3.test.nu, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a) execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported 2014-08-12 14:07:07,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, log id: 5d2ee913 2014-08-12 14:07:07,351 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-65) Error while refreshing Gluster lightweight data of cluster Default!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:65) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:406) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:392) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:363) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:108) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Anything else I need to configure? Kind regards, Jorick Astrego _______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </blockquote> <br> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://... </pre> </blockquote> <br> </body> </html> --------------020000030606060406020109--

This is a multi-part message in MIME format. --------------050601000505020707080204 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 08/13/2014 09:47 AM, Humble Devassy Chirammal wrote: I had to reinstall the host because another bug broke the network settings. So after an installation with selinux --permissive it works. I did switch it while testing before but something must have broken during gluster initialization or something like that. Now it works... Thanks! vdsClient -s localhost glusterVolumesList {'status': {'code': 0, 'message': 'Done'}, 'volumes': {}} Done --------------050601000505020707080204 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> <br> <div class="moz-cite-prefix">On 08/13/2014 09:47 AM, Humble Devassy Chirammal wrote:<br> </div> <blockquote cite="mid:CAEAGfOVcXM9f9-fZeqcnBGL8o6k2S2QsNzynSCxCeQZTFq-bmA@mail.gmail.com" type="cite"> <div dir="ltr"> <div>I think its worth to check the result of, some other verb to isolate this issue . Can u try below in your host ?<br> <br> #vdsClient -s localhost glusterHostsList<br> <br> </div> <div><br> <br> </div> <div><br> </div> <br> </div> <div class="gmail_extra"><br> <br> <div class="gmail_quote">On Wed, Aug 13, 2014 at 11:45 AM, Sahina Bose <span dir="ltr">&lt;<a moz-do-not-send="true" href="mailto:sabose@redhat.com" target=&quot;_blank&quot;&gt;sabose(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <div class=""> <br> <div>On 08/12/2014 10:59 PM, Jorick Astrego wrote:<br> </div> <blockquote type="cite"> <br> <div>On 08/12/2014 05:09 PM, Sahina Bose wrote:<br> </div> <blockquote type="cite"> <br> <div>On 08/12/2014 08:10 PM, Jorick Astrego wrote:<br> </div> <blockquote type="cite"> <br> <div>On 08/12/2014 03:44 PM, Sahina Bose wrote:<br> </div> <blockquote type="cite"> Could you provide the vdsm and supervdsm log from the node?<br> <br> </blockquote> vdsm:<br> <a moz-do-not-send="true" href="http://pastebin.com/edZrvnkr" target="_blank">http://pastebin.com/edZrvnkr</a><br... (cut some repetitive loglines out to fit on pastebin)<br> <br> Supervdsm:<br> <br> <a moz-do-not-send="true" href="http://pastebin.com/V8xTTMKk" target="_blank">http://pastebin.com/V8xTTMKk</a><br... </blockquote> <br> <br> I don't see any errors here corresponding to the unsupported method "glusterVolumesList"<br> <br> Are these logs from the node "<a moz-do-not-send="true" href="http://node3.test.nu" target="_blank">node3.test.nu</a>"? <br> <br> However, this error is unrelated to adding the storage domain. Typically, this unsupported method error is returned from vdsm when vdsm-gluster is not installed on the node.<br> You could try running the following on <a moz-do-not-send="true" href="http://node3.test.nu" target="_blank">node3.test.nu</a><br> #rge<br> </blockquote> <br> Found some selinux warnings:<br> <br> <blockquote>[78080.160692] type=1400 audit(1407844503.916:35): avc:  denied  { write } for  pid=2286 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> [78219.465352] type=1404 audit(1407844643.107:36): enforcing=0 old_enforcing=1 auid=0 ses=106<br> [78222.942816] type=1107 audit(1407844646.582:37): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc:  received setenforce notice (enforcing=0)<br>  exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'<br> [78226.383697] type=1400 audit(1407844650.020:38): avc:  denied  { write } for  pid=2348 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> [78226.383712] type=1400 audit(1407844650.020:39): avc:  denied  { unlink } for  pid=2348 comm="glusterd" name="glusterd.socket" dev="tmpfs" ino=857069 scontext=system_u:system_r:glusterd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file<br> </blockquote> </blockquote> <br> <br> </div> Gluster needs selinux to be in permissive mode. Could you change this and try? <div> <div class="h5"><br> </div> </div> </div> </blockquote> </div> </div> </blockquote> <br> I had to reinstall the host because another bug broke the network settings.<br> <br> So after an installation with selinux --permissive it works. I did switch it while testing before but something must have broken during gluster initialization or something like that.<br> <br> Now it works... Thanks!<br> <br> <blockquote>vdsClient -s localhost glusterVolumesList <br> {'status': {'code': 0, 'message': 'Done'}, 'volumes': {}}<br> Done<br> <br> </blockquote> <blockquote cite="mid:CAEAGfOVcXM9f9-fZeqcnBGL8o6k2S2QsNzynSCxCeQZTFq-bmA@mail.gmail.com" type="cite"> <div class="gmail_extra"> <div class="gmail_quote"> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <div> <div class="h5"> <br> <blockquote type="cite"> <blockquote> </blockquote> <br> <blockquote type="cite"> <blockquote type="cite"> <blockquote type="cite"> <div>On 08/12/2014 07:08 PM, Jorick Astrego wrote:<br> </div> <blockquote type="cite"> Hi,<br> <br> No restarting glusterfsd doesn't help. The steps in the link you gave me don't help either. The node is clean installed and has never been a part of a glusterfs.... Running gluster version 3.5.2 <br> <blockquote>glusterfs-server-3.5.2-1.el7.x86_64<br> glusterfs-api-3.5.2-1.el7.x86_64<br> glusterfs-fuse-3.5.2-1.el7.x86_64<br> glusterfs-cli-3.5.2-1.el7.x86_64<br> vdsm-gluster-4.16.1-4.gitb2bf270.el7.noarch<br> glusterfs-libs-3.5.2-1.el7.x86_64<br> glusterfs-3.5.2-1.el7.x86_64<br> glusterfs-rdma-3.5.2-1.el7.x86_64<br> <br> </blockquote> From the log:<br> <blockquote>[2014-08-12 11:57:30.006092] W [glusterfsd.c:1095:cleanup_and_exit] (--&gt;/lib64/libc.so.6(clone+0x6d) [0x7fac29bc23dd] (--&gt;/lib64/libpthread.so.0(+0x7df3) [0x7fac2a279df3] (--&gt;/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x7fac2b8471b5]))) 0-: received signum (15), shutting down<br> [2014-08-12 11:57:30.014919] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.5.2 (/usr/sbin/glusterd -p /run/glusterd.pid)<br> [2014-08-12 11:57:30.018089] I [glusterd.c:1122:init] 0-management: Using /var/lib/glusterd as working directory<br> [2014-08-12 11:57:30.020315] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.020331] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.021533] W [rdma.c:4194:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device)<br> [2014-08-12 11:57:30.021548] E [rdma.c:4482:init] 0-rdma.management: Failed to initialize IB Device<br> [2014-08-12 11:57:30.021555] E [rpc-transport.c:333:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed<br> [2014-08-12 11:57:30.021610] W [rpcsvc.c:1535:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed<br> [2014-08-12 11:57:30.021669] I [socket.c:3561:socket_init] 0-socket.management: SSL support is NOT enabled<br> [2014-08-12 11:57:30.021679] I [socket.c:3576:socket_init] 0-socket.management: using system polling thread<br> [2014-08-12 11:57:30.022603] I [glusterd.c:367:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system<br> <b>[2014-08-12 11:57:30.023046] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/<a moz-do-not-send="true" href="http://glusterd.info" target="_blank">glusterd.info</a>, returned error: (No such file or directory)</b><b><br> </b><b>[2014-08-12 11:57:30.023075] E [store.c:408:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/<a moz-do-not-send="true" href="http://glusterd.info" target="_blank">glusterd.info</a>, returned error: (No such file or directory)</b><br> [2014-08-12 11:57:30.023090] I [glusterd-store.c:1441:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 30501<br> Final graph:<br> +------------------------------------------------------------------------------+<br>   1: volume management<br>   2:     type mgmt/glusterd<br>   3:     option rpc-auth.auth-glusterfs on<br>   4:     option rpc-auth.auth-unix on<br>   5:     option rpc-auth.auth-null on<br>   6:     option transport.socket.listen-backlog 128<br>   7:     option transport.socket.read-fail-log off<br>   8:     option transport.socket.keepalive-interval 2<br>   9:     option transport.socket.keepalive-time 10<br>  10:     option transport-type rdma<br>  11:     option working-directory /var/lib/glusterd<br>  12: end-volume<br>  13: <br> +------------------------------------------------------------------------------+<br> <br> </blockquote> Jorick<br> <br> <div>On 08/12/2014 02:56 PM, Maor Lipchuk wrote:<br> </div> <blockquote type="cite"> <pre>Hi Jorick, which version of GlusterFS are you using? Try to restart glusterd on each of the bricks, then try to follow this link: <a moz-do-not-send="true" href="http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/" target="_blank">http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/</a> Please tell me if this helps Regards, Maor ----- Original Message ----- From: "Jorick Astrego" <a moz-do-not-send="true" href="mailto:j.astrego@netbulae.eu" target=&quot;_blank&quot;&gt;&amp;lt;j.astrego(a)netbulae.eu&amp;gt;&lt;/a&gt; To: <a moz-do-not-send="true" href="mailto:users@ovirt.org" target=&quot;_blank&quot;&gt;users(a)ovirt.org&lt;/a&gt; Sent: Tuesday, August 12, 2014 3:16:06 PM Subject: [ovirt-users] method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) Hi, I'm trying to test glusterfs on a couple of Centos 7 ovirt nodes with ovirt 3.5rc1. I've enabled glusterfs service for the cluster, created a xfs data partition with mount point, installed "vdsm-gluster" rpm and started glusterfsd. I also cleared the firewall rules. When I try to add the storage domain, I get the following error 2014-08-12 14:07:07,346 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GlusterVolumesListVDSCommand(HostName = <a moz-do-not-send="true" href="http://node3.test.nu" target="_blank">node3.test.nu</a>, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a), log id: 5d2ee913 2014-08-12 14:07:07,350 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GlusterVolumesListVDSCommand(HostName = <a moz-do-not-send="true" href="http://node3.test.nu" target="_blank">node3.test.nu</a>, HostId = 5bff5a65-6d3c-46b4-aa7c-d87ab25ccb3a) execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported 2014-08-12 14:07:07,350 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GlusterVolumesListVDSCommand, log id: 5d2ee913 2014-08-12 14:07:07,351 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-65) Error while refreshing Gluster lightweight data of cluster Default!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: &lt;type 'exceptions.Exception'&gt;:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:65) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:406) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:392) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:363) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:108) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:87) [bll.jar:] at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) [:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Anything else I need to configure? Kind regards, Jorick Astrego _______________________________________________ Users mailing list <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt; <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/user... </pre> </blockquote> <br> <br> <fieldset></fieldset> <br> <pre>_______________________________________________ Users mailing list <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt; <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/user... </pre> </blockquote> <br> </blockquote> <br> <br> <fieldset></fieldset> <br> <pre>_______________________________________________ Users mailing list <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt; <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/user... </pre> </blockquote> <br> </blockquote> <br> <br> <fieldset></fieldset> <br> <pre>_______________________________________________ Users mailing list <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt; <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/user... </pre> </blockquote> <br> </div> </div> </div> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send="true" href="mailto:Users@ovirt.org">Users@ovirt.org</a><br> <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/user... <br> </blockquote> </div> <br> </div> </blockquote> <br> </body> </html> --------------050601000505020707080204--

On 08/12/2014 08:31 PM, Jorick Astrego wrote: do you have vdsm-gluster deployed?