[Users] Using storage network for gluster
H, I'm experimenting with gluster and as my DC is a iscsi dc, I've just created a new cluster with two hosts and now I'm trying to create a volume. I've followed all kind of guides in the web without success. Now I'm having ovirt to complaint about a network error in the volume creation and I'd like to ask two things: - Why I must use ovirtmgmt network for gluster (can't choose other from the add brick drop down list) instead of the network I have set up for iscsi? - What may be the problem with network? I get success on both hosts when I proob each other # gluster peer probe ovirt3.montecarlotv.com.uy peer probe: success: host ovirt3.montecarlotv.com.uy port 24007 already in peer list # gluster peer probe ovirt4.montecarlotv.com.uy peer probe: success: host ovirt4.montecarlotv.com.uy port 24007 already in peer list I get a lot of this ones in engine log: 2013-12-05 03:40:01,882 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-50) Error while refreshing Gluster lightweight data of cluster Penryn!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:64) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:389) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:375) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:346) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:104) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:83) [bll.jar:] at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source) [:1.7.0_45] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] So I guess there's something with gluster version maybe: glusterfs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-api.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-cli.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-fuse.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-libs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-rdma.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-server.x86_64 3.4.0-8.el6 @glusterfs-epel vdsm-gluster.noarch 4.13.0-11.el6 @ovirt-stable this are the main links I used to configure: http://www.andrewklau.com/returning-to-glusterized-ovirt-3-3/ http://www.gluster.org/2013/09/ovirt-3-3-glusterized/ Regards
Well, It seems that I rushed into the list as I finally managed to get the volume up. It was a mix of things, had to restart glusterd in both bircks, then following http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-... that pointed a solution for a possible bug in gluster 3.4 I managed to get things working (I had tried before but didn't work, seems like restarting glusterd made it possible) Thanks and sorry for so much noise. Regards,
On 12/05/2013 10:34 PM, Juan Pablo Lorier wrote:
H,
I'm experimenting with gluster and as my DC is a iscsi dc, I've just created a new cluster with two hosts and now I'm trying to create a volume. I've followed all kind of guides in the web without success. Now I'm having ovirt to complaint about a network error in the volume creation and I'd like to ask two things:
- Why I must use ovirtmgmt network for gluster (can't choose other from the add brick drop down list) instead of the network I have set up for iscsi? - What may be the problem with network?
The issue is not with the network, but some dependency package on the host. It looks like you're missing vdsm-gluster package. Could you share details of your setup? ovirt engine version? result of rpm -qa | grep vdsm on the hosts?
I get success on both hosts when I proob each other
# gluster peer probe ovirt3.montecarlotv.com.uy peer probe: success: host ovirt3.montecarlotv.com.uy port 24007 already in peer list
# gluster peer probe ovirt4.montecarlotv.com.uy peer probe: success: host ovirt4.montecarlotv.com.uy port 24007 already in peer list
I get a lot of this ones in engine log:
2013-12-05 03:40:01,882 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-50) Error while refreshing Gluster lightweight data of cluster Penryn!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:64) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:389) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:375) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:346) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:104) [bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:83) [bll.jar:] at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source) [:1.7.0_45] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
So I guess there's something with gluster version maybe:
glusterfs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-api.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-cli.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-fuse.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-libs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-rdma.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-server.x86_64 3.4.0-8.el6 @glusterfs-epel vdsm-gluster.noarch 4.13.0-11.el6 @ovirt-stable
this are the main links I used to configure:
http://www.andrewklau.com/returning-to-glusterized-ovirt-3-3/
http://www.gluster.org/2013/09/ovirt-3-3-glusterized/
Regards
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 12/06/2013 11:49 AM, Sahina Bose wrote:
On 12/05/2013 10:34 PM, Juan Pablo Lorier wrote:
H,
I'm experimenting with gluster and as my DC is a iscsi dc, I've just created a new cluster with two hosts and now I'm trying to create a volume. I've followed all kind of guides in the web without success. Now I'm having ovirt to complaint about a network error in the volume creation and I'd like to ask two things:
- Why I must use ovirtmgmt network for gluster (can't choose other from the add brick drop down list) instead of the network I have set up for iscsi? - What may be the problem with network?
The issue is not with the network, but some dependency package on the host. It looks like you're missing vdsm-gluster package.
Could you share details of your setup?
ovirt engine version?
result of rpm -qa | grep vdsm on the hosts?
Sorry, I jumped the gun. Din't see you had already sorted this out.
I get success on both hosts when I proob each other
# gluster peer probe ovirt3.montecarlotv.com.uy peer probe: success: host ovirt3.montecarlotv.com.uy port 24007 already in peer list
# gluster peer probe ovirt4.montecarlotv.com.uy peer probe: success: host ovirt4.montecarlotv.com.uy port 24007 already in peer list
I get a lot of this ones in engine log:
2013-12-05 03:40:01,882 ERROR [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-50) Error while refreshing Gluster lightweight data of cluster Penryn!: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterVolumesList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterJob.runVdsCommand(GlusterJob.java:64)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:389)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.fetchVolumes(GlusterSyncJob.java:375)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshVolumeData(GlusterSyncJob.java:346)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshClusterData(GlusterSyncJob.java:104)
[bll.jar:] at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:83)
[bll.jar:] at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source) [:1.7.0_45] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.7.0_45] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
[quartz.jar:]
So I guess there's something with gluster version maybe:
glusterfs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-api.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-cli.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-fuse.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-libs.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-rdma.x86_64 3.4.0-8.el6 @glusterfs-epel glusterfs-server.x86_64 3.4.0-8.el6 @glusterfs-epel vdsm-gluster.noarch 4.13.0-11.el6 @ovirt-stable
this are the main links I used to configure:
http://www.andrewklau.com/returning-to-glusterized-ovirt-3-3/
http://www.gluster.org/2013/09/ovirt-3-3-glusterized/
Regards
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Thu, Dec 5, 2013 at 6:04 PM, Juan Pablo Lorier wrote:
H,
I'm experimenting with gluster and as my DC is a iscsi dc, I've just created a new cluster with two hosts and now I'm trying to create a volume. I've followed all kind of guides in the web without success. Now I'm having ovirt to complaint about a network error in the volume creation and I'd like to ask two things:
- Why I must use ovirtmgmt network for gluster (can't choose other from the add brick drop down list) instead of the network I have set up for iscsi? - What may be the problem with network?
this worked for me after default install that, as you already wrote, doesn't allow to have anything but the ovirtmgmt. Of course it needs downtime... suboptimal... see this thread of mine: http://lists.ovirt.org/pipermail/users/2013-October/016891.html Using ovirtmgmt I had problems with cluster due to ovirtmgmt saturated if I had heavy I/O on VMs (so network bandwith from gluster point of view)... with hosts going into non operational status. No problem after changing network. Gianluca
Thanks Gianluca, I'll try that. I still am under testing for this gluster cluster, so I have no problem with downtime. :-) I'll let you know what happened. Regards, On 06/12/13 08:11, Gianluca Cecchi wrote:
H,
I'm experimenting with gluster and as my DC is a iscsi dc, I've just created a new cluster with two hosts and now I'm trying to create a volume. I've followed all kind of guides in the web without success. Now I'm having ovirt to complaint about a network error in the volume creation and I'd like to ask two things:
- Why I must use ovirtmgmt network for gluster (can't choose other from the add brick drop down list) instead of the network I have set up for iscsi? - What may be the problem with network?
On Thu, Dec 5, 2013 at 6:04 PM, Juan Pablo Lorier wrote: this worked for me after default install that, as you already wrote, doesn't allow to have anything but the ovirtmgmt. Of course it needs downtime... suboptimal... see this thread of mine: http://lists.ovirt.org/pipermail/users/2013-October/016891.html
Using ovirtmgmt I had problems with cluster due to ovirtmgmt saturated if I had heavy I/O on VMs (so network bandwith from gluster point of view)... with hosts going into non operational status. No problem after changing network.
Gianluca
participants (3)
-
Gianluca Cecchi -
Juan Pablo Lorier -
Sahina Bose