10:33 a.m.
Hello,
I remember in the past that could be a problem to have high usage of
ovirtmgmt network because engine sometimes detects hosts as
unresponsive.
And it should this the reason about bandwith limitation on vm
migration, until dedicated network for it has been released.
SO the question is : what about ovirtmgmt network for gluster
replication when gluster domain is provided by ovirt nodes?
I suppose it could be a problem too, couldn't it?
In case I have a dedicated network for gluster for the nodes, how can
i configure it?
Can I configure in this case Gluster storage domain from ovirt engine
or does it require to be on this network too?
Thanks in advance for any suggestion
Gianluca
10:52 a.m.
On 09/28/2013 08:33 PM, Gianluca Cecchi wrote:
Hello,
I remember in the past that could be a problem to have high usage of
ovirtmgmt network because engine sometimes detects hosts as
unresponsive.
And it should this the reason about bandwith limitation on vm
migration, until dedicated network for it has been released.
SO the question is : what about ovirtmgmt network for gluster
replication when gluster domain is provided by ovirt nodes?
I suppose it could be a problem too, couldn't it?
In case I have a dedicated network for gluster for the nodes, how can
i configure it?
Can I configure in this case Gluster storage domain from ovirt engine
or does it require to be on this network too?
Thanks in advance for any suggestion
Gianluca
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
livnat - would network QoS cover this?
vijay/sahina - does it make sense to define a 'gluster replication
network' or something like that?
2:44 a.m.
Hi Gianluca,
<snip>
On 10/02/2013 08:52 PM, Itamar Heim wrote:
On 09/28/2013 08:33 PM, Gianluca Cecchi wrote:
> Hello,
> I remember in the past that could be a problem to have high usage of
> ovirtmgmt network because engine sometimes detects hosts as
> unresponsive.
> And it should this the reason about bandwith limitation on vm
> migration, until dedicated network for it has been released.
We have a dedicated migration network in oVirt 3.3, which means all the
migration network is using a dedicated network and not the management
network.
The caveat is that we still did not add support for capping it's bandwidth.
The problem you detailed above can be bypassed by configuring the
migration network on a dedicated interface.
> SO the question is : what about ovirtmgmt network for gluster
> replication when gluster domain is provided by ovirt nodes?
> I suppose it could be a problem too, couldn't it?
yes that is correct, using ovirtmgmt for non management traffic is not a
good idea.
> In case I have a dedicated network for gluster for the nodes, how
can
> i configure it?
Generally there two ways to go about this:
1. add a new network role 'gluster replication' and then adjust the
engine code to pass the network with this role as parameter to the
replication verb in VDSM.
2. in the replication verb in the UI let the user choose which network
to use for the replication.
I am not familiar with the replication verb so I might be missing
something but otherwise I think that solution 2 is more simple and less
invasive then requiring a new network role.
> Can I configure in this case Gluster storage domain from ovirt
engine
> or does it require to be on this network too?
>
> Thanks in advance for any suggestion
>
> Gianluca
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
livnat - would network QoS cover this?
Network Qos is needed in addition to what I have specified above. and it
would be available in oVirt 3.4 hopefully, but if you use a dedicated
network as suggested above you can bypass the traffic shaping issue by
using a dedicated NIC.
Livnat
vijay/sahina - does it make sense to define a 'gluster
replication
network' or something like that?
5:08 a.m.
On Thu, Oct 03, 2013 at 12:44:14PM +0300, Livnat Peer wrote:
Hi Gianluca,
<snip>
On 10/02/2013 08:52 PM, Itamar Heim wrote:
> On 09/28/2013 08:33 PM, Gianluca Cecchi wrote:
>> Hello,
>> I remember in the past that could be a problem to have high usage of
>> ovirtmgmt network because engine sometimes detects hosts as
>> unresponsive.
>> And it should this the reason about bandwith limitation on vm
>> migration, until dedicated network for it has been released.
We have a dedicated migration network in oVirt 3.3, which means all the
migration network is using a dedicated network and not the management
network.
The caveat is that we still did not add support for capping it's bandwidth.
The problem you detailed above can be bypassed by configuring the
migration network on a dedicated interface.
>> SO the question is : what about ovirtmgmt network for gluster
>> replication when gluster domain is provided by ovirt nodes?
>> I suppose it could be a problem too, couldn't it?
yes that is correct, using ovirtmgmt for non management traffic is not a
good idea.
>> In case I have a dedicated network for gluster for the nodes, how can
>> i configure it?
Generally there two ways to go about this:
1. add a new network role 'gluster replication' and then adjust the
engine code to pass the network with this role as parameter to the
replication verb in VDSM.
2. in the replication verb in the UI let the user choose which network
to use for the replication.
I am not familiar with the replication verb so I might be missing
something but otherwise I think that solution 2 is more simple and less
invasive then requiring a new network role.
Bala, is there a means to select, for each node, which IP address is to
be used for replication? AFAICT we use the host fqdn, which most
probably resolves to the ovirtmgmt device.
>> Can I configure in this case Gluster storage domain from ovirt engine
>> or does it require to be on this network too?
>>
>> Thanks in advance for any suggestion
>>
>> Gianluca
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
> livnat - would network QoS cover this?
Network Qos is needed in addition to what I have specified above. and it
would be available in oVirt 3.4 hopefully, but if you use a dedicated
network as suggested above you can bypass the traffic shaping issue by
using a dedicated NIC.
Livnat
> vijay/sahina - does it make sense to define a 'gluster replication
> network' or something like that?
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
5:45 a.m.
On Thu, Oct 3, 2013 at 2:08 PM, Dan Kenigsberg wrote:
>
> >> SO the question is : what about ovirtmgmt network for gluster
> >> replication when gluster domain is provided by ovirt nodes?
> >> I suppose it could be a problem too, couldn't it?
>
> yes that is correct, using ovirtmgmt for non management traffic is not a
> good idea.
>
> >> In case I have a dedicated network for gluster for the nodes, how can
> >> i configure it?
>
> Generally there two ways to go about this:
>
> 1. add a new network role 'gluster replication' and then adjust the
> engine code to pass the network with this role as parameter to the
> replication verb in VDSM.
>
> 2. in the replication verb in the UI let the user choose which network
> to use for the replication.
>
> I am not familiar with the replication verb so I might be missing
> something but otherwise I think that solution 2 is more simple and less
> invasive then requiring a new network role.
Bala, is there a means to select, for each node, which IP address is to
be used for replication? AFAICT we use the host fqdn, which most
probably resolves to the ovirtmgmt device.
what is "replication verb in the UI"? Inside UI when I click "create
volume" and then "add Bricks" buttons I can only select "Server"
that
in my case is a drop down menu containing only ip addresses of my 2
hypervisors....
Or simply do you mean development steps to integrate this missing
functionality in oVirt with steps 1. and 2.?
Thanks,
Gianluca
8:57 a.m.
On 10/03/2013 03:45 PM, Gianluca Cecchi wrote:
On Thu, Oct 3, 2013 at 2:08 PM, Dan Kenigsberg wrote:
>>
>>>> SO the question is : what about ovirtmgmt network for gluster
>>>> replication when gluster domain is provided by ovirt nodes?
>>>> I suppose it could be a problem too, couldn't it?
>>
>> yes that is correct, using ovirtmgmt for non management traffic is not a
>> good idea.
>>
>>>> In case I have a dedicated network for gluster for the nodes, how can
>>>> i configure it?
>>
>> Generally there two ways to go about this:
>>
>> 1. add a new network role 'gluster replication' and then adjust the
>> engine code to pass the network with this role as parameter to the
>> replication verb in VDSM.
>>
>> 2. in the replication verb in the UI let the user choose which network
>> to use for the replication.
>>
>> I am not familiar with the replication verb so I might be missing
>> something but otherwise I think that solution 2 is more simple and less
>> invasive then requiring a new network role.
>
> Bala, is there a means to select, for each node, which IP address is to
> be used for replication? AFAICT we use the host fqdn, which most
> probably resolves to the ovirtmgmt device.
what is "replication verb in the UI"? Inside UI when I click "create
volume" and then "add Bricks" buttons I can only select "Server"
that
in my case is a drop down menu containing only ip addresses of my 2
hypervisors....
Or simply do you mean development steps to integrate this missing
functionality in oVirt with steps 1. and 2.?
I am probably missing some background around Gluster, could you share
what triggers the gluster replication?
Thanks,
Gianluca
3:01 a.m.
On 10/03/2013 09:27 PM, Livnat Peer wrote:
On 10/03/2013 03:45 PM, Gianluca Cecchi wrote:
> On Thu, Oct 3, 2013 at 2:08 PM, Dan Kenigsberg wrote:
>
>>>>> SO the question is : what about ovirtmgmt network for gluster
>>>>> replication when gluster domain is provided by ovirt nodes?
>>>>> I suppose it could be a problem too, couldn't it?
>>> yes that is correct, using ovirtmgmt for non management traffic is not a
>>> good idea.
>>>
>>>>> In case I have a dedicated network for gluster for the nodes, how
can
>>>>> i configure it?
>>> Generally there two ways to go about this:
>>>
>>> 1. add a new network role 'gluster replication' and then adjust the
>>> engine code to pass the network with this role as parameter to the
>>> replication verb in VDSM.
>>>
>>> 2. in the replication verb in the UI let the user choose which network
>>> to use for the replication.
>>>
>>> I am not familiar with the replication verb so I might be missing
>>> something but otherwise I think that solution 2 is more simple and less
>>> invasive then requiring a new network role.
>> Bala, is there a means to select, for each node, which IP address is to
>> be used for replication? AFAICT we use the host fqdn, which most
>> probably resolves to the ovirtmgmt device.
> what is "replication verb in the UI"? Inside UI when I click
"create
> volume" and then "add Bricks" buttons I can only select
"Server" that
> in my case is a drop down menu containing only ip addresses of my 2
> hypervisors....
>
> Or simply do you mean development steps to integrate this missing
> functionality in oVirt with steps 1. and 2.?
I am probably missing some background around Gluster, could you share
what triggers the gluster replication?
[Adding Amar from gluster team]
There's no user interface way to trigger replication.
Gianluca, I assume you do not mean geo-replication but replication of
files when volume is of replicated type.
Currently, we do not have a way to specify the network to be used when
gluster communicates with peers.
Like Joop mentioned in a separate mail, if you use FQDN to add host to
the engine, and FQDN resolves differently for internal and external
communication, gluster would then use the internal network for its
communication
thanks
sahina
> Thanks,
> Gianluca
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
3:10 a.m.
On Fri, Oct 4, 2013 at 12:01 PM, Sahina Bose wrote:
There's no user interface way to trigger replication.
Gianluca, I assume you do not mean geo-replication but replication of files
when volume is of replicated type.
YesSahina, I mean replication of files when volume is of replicated type
(in my case distributed replicated)
Currently, we do not have a way to specify the network to be used when
gluster communicates with peers.
Like Joop mentioned in a separate mail, if you use FQDN to add host to the
engine, and FQDN resolves differently for internal and external
communication, gluster would then use the internal network for its
communication
OK, eventuallt I'm going to try this way.
Do you think it would work if without dns at all for a test
environment I set up this way
mgmt lan 192.x
gluster network 10.x
server hostnames involved
engine
node01
node02
In /etc/hosts of engine
192.xxx engine
192.yyy node01
192.zzz node02
In /etc/hosts of nodes
192.xxx engine
10.yyy node01
10.zzz node02
and accordingly in sysconfig files of nodes for their hostnames/ip bindng
And when I preconfigure gluster on nodes I use on node01 the command
gluster peer probe node02
Gianluca
3:15 a.m.
On 10/04/2013 03:40 PM, Gianluca Cecchi wrote:
On Fri, Oct 4, 2013 at 12:01 PM, Sahina Bose wrote:
> There's no user interface way to trigger replication.
> Gianluca, I assume you do not mean geo-replication but replication of files
> when volume is of replicated type.
YesSahina, I mean replication of files when volume is of replicated type
(in my case distributed replicated)
> Currently, we do not have a way to specify the network to be used when
> gluster communicates with peers.
> Like Joop mentioned in a separate mail, if you use FQDN to add host to the
> engine, and FQDN resolves differently for internal and external
> communication, gluster would then use the internal network for its
> communication
OK, eventuallt I'm going to try this way.
Do you think it would work if without dns at all for a test
environment I set up this way
mgmt lan 192.x
gluster network 10.x
server hostnames involved
engine
node01
node02
In /etc/hosts of engine
192.xxx engine
192.yyy node01
192.zzz node02
In /etc/hosts of nodes
192.xxx engine
10.yyy node01
10.zzz node02
and accordingly in sysconfig files of nodes for their hostnames/ip bindng
And when I preconfigure gluster on nodes I use on node01 the command
gluster peer probe node02
I believe it should. When you add host to engine you would use node01.
Please let us know how it goes,
thanks again
sahina
Gianluca
12:03 p.m.
On Fri, Oct 4, 2013 at 12:15 PM, Sahina Bose wrote:
I believe it should. When you add host to engine you would use node01.
Please let us know how it goes,
thanks again
sahina
I tried and it works.
I was able to maintain my data too.
I stopped the only VM created and put both hosts in maintenance.
Then from f18ovn01 (with 10.4.4.58 on ovirtmgmt and 192.168.3.1 on new
replication network):
please note f18 in the name but is fedora 19....
gluster volume stop gvdata
gluster volume delete gvdata
gluster peer detach 10.4.4.59
change /etc/hosts on both hosts and activate another interface with
192.168.3.1 and 192.168.3.3
to clear data about previous gluster volume and reuse it with same values:
on both hosts
setfattr -x trusted.glusterfs.volume-id /gluster/DATA_GLUSTER/brick1
setfattr -x trusted.gfid /gluster/DATA_GLUSTER/brick1
rm -rf rm -rf /gluster/DATA_GLUSTER/brick1/.glusterfs
gluster peer probe f18ovn03.mydomain
remake the manual steps to create gvdata volume and settings as before
from oVirt webadmin activated both hosts and all came up transparently.
able to start c6s VM (from run once only due to bug...)
wget a netinst.iso for fedora 19 from inside VM and during transfer
(VM is running on this f18ovn01 node):
[root@f18ovn01 ]# bwm-ng v0.6 (probing every 1.000s), press 'h' for help
input: libstatnet type: rate
| iface Rx Tx Total
==============================================================================
ovirtmgmt: 0.19 KB/s 0.24 KB/s 0.43 KB/s
enp3s0: 214.34 KB/s 28794.59 KB/s 29008.93 KB/s
------------------------------------------------------------------------------
total: 214.53 KB/s 28794.82 KB/s 29009.35 KB/s
Also restarted ovirt-engine service on engine.
Note that from engine point of view when I go in volumes --> gvdata
--> bricks, still I see 10.4.4.58 and 10.4.4.59, probably because for
the engine those are the ip addresses of f18ovn01 and f18ovn03
nodes....
Or I don't know if it wrote anywhere in the db the gluster data or
only gets them dinamically
Gianluca
11:45 a.m.
Gianluca Cecchi wrote:
Hello,
I remember in the past that could be a problem to have high usage of
ovirtmgmt network because engine sometimes detects hosts as
unresponsive.
And it should this the reason about bandwith limitation on vm
migration, until dedicated network for it has been released.
SO the question is : what about ovirtmgmt network for gluster
replication when gluster domain is provided by ovirt nodes?
I suppose it could be a problem too, couldn't it?
In case I have a dedicated network for gluster for the nodes, how can
i configure it?
Can I configure in this case Gluster storage domain from ovirt engine
or does it require to be on this network too?
No, it doesn't. I have a setup where the two gluster servers are using
10G and ovirtmgmt is 1G. We use split-dns but the poor mens variety.
The engine server (mgmt01) sees both gluster servers on its own network
(192.168.x.x) but the gluster servers itself have host entries for the
same hostnames in the 172.19.x.x range, same holds for the vmhosts. What
happens is that you add the storage servers by fqdn and the gluster peer
probe commands resolve to the 10G network and all storage traffic is
confined to this network. So managment commands arrive on the 192
network and the actual work is done on the storage network.
Hope that I haven't confused you beyond hope:-)
Joop
3855
days inactive
3863
days old
10 comments
6 participants
participants (6)
-
Dan Kenigsberg -
Gianluca Cecchi -
Itamar Heim -
Joop -
Livnat Peer -
Sahina Bose