[oVirt HC] Gluster traffic still flows on mgmt even after choosing a different Gluster nw
Hi, I've deployed an oVirt HC starting with latest oVirt Node 4.3.6; this is my simple network plan (FQDNs only resolves the front-end addresses): front-end back-end engine.ovirt 192.168.110.10 node1.ovirt 192.168.110.11 192.168.210.11 node2.ovirt 192.168.110.12 192.168.210.12 node3.ovirt 192.168.110.13 192.168.210.13 at the end I followed the RHHI-V 1.6 Deployment Guide where, at chapter 9 [1], it suggests to create a logical network for Gluster traffic. Now I can see, indeed, back-end addresses added in the address pool: [root@node1 ~]# gluster peer status Number of Peers: 2 Hostname: node3.ovirt Uuid: 3fe33e8b-d073-4d7a-8bda-441c42317c92 State: Peer in Cluster (Connected) Other names: 192.168.210.13 Hostname: node2.ovirt Uuid: a95a9233-203d-4280-92b9-04217fa338d8 State: Peer in Cluster (Connected) Other names: 192.168.210.12 The problem is that the Gluster traffic seems still to flow on the management interfaces: [root@node1 ~]# tcpdump -i ovirtmgmt portrange 49152-49664 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ovirtmgmt, link-type EN10MB (Ethernet), capture size 262144 bytes 14:04:58.746574 IP node2.ovirt.49129 > node1.ovirt.49153: Flags [.], ack 484303246, win 18338, options [nop,nop,TS val 6760049 ecr 6760932], length 0 14:04:58.753050 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 2507489191:2507489347, ack 2889633200, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753131 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 156:312, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753142 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 312:468, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753148 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 468:624, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753203 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 624:780, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753216 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 780:936, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753231 IP node1.ovirt.49152 > node2.ovirt.49131: Flags [.], ack 936, win 15566, options [nop,nop,TS val 6760978 ecr 6760055], length 0 ... and no yet to the eth1 I dedicated to gluster: [root@node1 ~]# tcpdump -i eth1 portrange 49152-49664 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes What am I missing here? What can I do to force the Gluster traffic to really flow on dedicated Gluster network? Thank you, Stefano. [1] https://red.ht/2MiZ4Ge
On Wed, Oct 16, 2019 at 2:16 PM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Hi,
I've deployed an oVirt HC starting with latest oVirt Node 4.3.6; this is my simple network plan (FQDNs only resolves the front-end addresses):
front-end back-end engine.ovirt 192.168.110.10 node1.ovirt 192.168.110.11 192.168.210.11 node2.ovirt 192.168.110.12 192.168.210.12 node3.ovirt 192.168.110.13 192.168.210.13
The storage traffic allocation over multiple subnets is implicitly set by name resolution and routing rules. Please use two distinct hostnames for each host: the first one should resolve only as an address on the management network and the second one as an address on the storage network. In the cockpit wizard for the hyperconverged deployment you will be prompted twice about the name of the three hosts: on the first tab (named 'Hosts') use the three host-names that resolves on the storage network. On the second tab ('Additional Hosts') please use the hostnames that are going to be resolved over the management network. at the end I followed the RHHI-V 1.6 Deployment Guide where, at chapter 9
[1], it suggests to create a logical network for Gluster traffic. Now I can see, indeed, back-end addresses added in the address pool:
[root@node1 ~]# gluster peer status Number of Peers: 2
Hostname: node3.ovirt Uuid: 3fe33e8b-d073-4d7a-8bda-441c42317c92 State: Peer in Cluster (Connected) Other names: 192.168.210.13
Hostname: node2.ovirt Uuid: a95a9233-203d-4280-92b9-04217fa338d8 State: Peer in Cluster (Connected) Other names: 192.168.210.12
The problem is that the Gluster traffic seems still to flow on the management interfaces:
[root@node1 ~]# tcpdump -i ovirtmgmt portrange 49152-49664
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ovirtmgmt, link-type EN10MB (Ethernet), capture size 262144 bytes 14:04:58.746574 IP node2.ovirt.49129 > node1.ovirt.49153: Flags [.], ack 484303246, win 18338, options [nop,nop,TS val 6760049 ecr 6760932], length 0 14:04:58.753050 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 2507489191:2507489347, ack 2889633200, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753131 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 156:312, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753142 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 312:468, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753148 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 468:624, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753203 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 624:780, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753216 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 780:936, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753231 IP node1.ovirt.49152 > node2.ovirt.49131: Flags [.], ack 936, win 15566, options [nop,nop,TS val 6760978 ecr 6760055], length 0 ...
and no yet to the eth1 I dedicated to gluster:
[root@node1 ~]# tcpdump -i eth1 portrange 49152-49664 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
What am I missing here? What can I do to force the Gluster traffic to really flow on dedicated Gluster network?
Thank you, Stefano.
[1] https://red.ht/2MiZ4Ge _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U3ZAM3DGE3EBGC...
This is an interesting topic, I've had a 3 node HCI cluster configured the same way and now I'm questioning whether or not my gluster traffic has been using my gig network instead of my 10gig network on separate subnet. On Wed, Oct 16, 2019 at 10:08 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Wed, Oct 16, 2019 at 2:16 PM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Hi,
I've deployed an oVirt HC starting with latest oVirt Node 4.3.6; this is my simple network plan (FQDNs only resolves the front-end addresses):
front-end back-end engine.ovirt 192.168.110.10 node1.ovirt 192.168.110.11 192.168.210.11 node2.ovirt 192.168.110.12 192.168.210.12 node3.ovirt 192.168.110.13 192.168.210.13
The storage traffic allocation over multiple subnets is implicitly set by name resolution and routing rules.
Please use two distinct hostnames for each host: the first one should resolve only as an address on the management network and the second one as an address on the storage network.
In the cockpit wizard for the hyperconverged deployment you will be prompted twice about the name of the three hosts: on the first tab (named 'Hosts') use the three host-names that resolves on the storage network. On the second tab ('Additional Hosts') please use the hostnames that are going to be resolved over the management network.
at the end I followed the RHHI-V 1.6 Deployment Guide where, at chapter 9
[1], it suggests to create a logical network for Gluster traffic. Now I can see, indeed, back-end addresses added in the address pool:
[root@node1 ~]# gluster peer status Number of Peers: 2
Hostname: node3.ovirt Uuid: 3fe33e8b-d073-4d7a-8bda-441c42317c92 State: Peer in Cluster (Connected) Other names: 192.168.210.13
Hostname: node2.ovirt Uuid: a95a9233-203d-4280-92b9-04217fa338d8 State: Peer in Cluster (Connected) Other names: 192.168.210.12
The problem is that the Gluster traffic seems still to flow on the management interfaces:
[root@node1 ~]# tcpdump -i ovirtmgmt portrange 49152-49664
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ovirtmgmt, link-type EN10MB (Ethernet), capture size 262144 bytes 14:04:58.746574 IP node2.ovirt.49129 > node1.ovirt.49153: Flags [.], ack 484303246, win 18338, options [nop,nop,TS val 6760049 ecr 6760932], length 0 14:04:58.753050 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 2507489191:2507489347, ack 2889633200, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753131 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 156:312, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753142 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 312:468, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753148 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 468:624, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753203 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 624:780, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753216 IP node2.ovirt.49131 > node1.ovirt.49152: Flags [P.], seq 780:936, ack 1, win 20874, options [nop,nop,TS val 6760055 ecr 6757892], length 156 14:04:58.753231 IP node1.ovirt.49152 > node2.ovirt.49131: Flags [.], ack 936, win 15566, options [nop,nop,TS val 6760978 ecr 6760055], length 0 ...
and no yet to the eth1 I dedicated to gluster:
[root@node1 ~]# tcpdump -i eth1 portrange 49152-49664 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
What am I missing here? What can I do to force the Gluster traffic to really flow on dedicated Gluster network?
Thank you, Stefano.
[1] https://red.ht/2MiZ4Ge _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U3ZAM3DGE3EBGC...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NGQG6SIWR7R4B2...
Thank you Simone for the clarifications. I've redeployed with both management and storage FQDNs; now everything seems to be in its place. I only have a couple of questions: 1) In the Gluster deployment Wizard, section 1 (Hosts) and 2 (Additional Hosts) are misleading; should be renamed in something like "Host Configuration: Storage side" / "Host Configuration: Management side". 2) what is the real function of the "Gluster Network" cluster traffic type? What it actually does? Thanks, Stefano.
Is there a way to fix this on a hci deployment which is already in operation? I do have a separate gluster network which is chosen for migration and gluster network but when I originally deployed I used just one set of host names which resolve to management network subnet. I appear to have a situation where gluster traffic may be going through both networks in seeing what looks like gluster traffic on both the gluster interface and ovirt management. On Wed, Oct 16, 2019 at 11:34 AM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Thank you Simone for the clarifications.
I've redeployed with both management and storage FQDNs; now everything seems to be in its place.
I only have a couple of questions:
1) In the Gluster deployment Wizard, section 1 (Hosts) and 2 (Additional Hosts) are misleading; should be renamed in something like "Host Configuration: Storage side" / "Host Configuration: Management side".
2) what is the real function of the "Gluster Network" cluster traffic type? What it actually does?
Thanks, Stefano. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NTNXPJMOZEYVHI...
I had originally setup my cluster following this older guide at https://blogs.ovirt.org/2018/02/up-and-running-with-ovirt-4-2-and-gluster-st... and the fqdn steps did not exist at that time. After deployment I then followed steps to add gluster network and set that network as storage and migration. What is the difference between what I did and the new method gdeploy uses? If my gluster config is using hostnames on the management network subnet but the gluster network is set for gluster and migration traffic what happens? On Wed, Oct 16, 2019 at 12:05 PM Jayme <jaymef@gmail.com> wrote:
Is there a way to fix this on a hci deployment which is already in operation? I do have a separate gluster network which is chosen for migration and gluster network but when I originally deployed I used just one set of host names which resolve to management network subnet.
I appear to have a situation where gluster traffic may be going through both networks in seeing what looks like gluster traffic on both the gluster interface and ovirt management.
On Wed, Oct 16, 2019 at 11:34 AM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Thank you Simone for the clarifications.
I've redeployed with both management and storage FQDNs; now everything seems to be in its place.
I only have a couple of questions:
1) In the Gluster deployment Wizard, section 1 (Hosts) and 2 (Additional Hosts) are misleading; should be renamed in something like "Host Configuration: Storage side" / "Host Configuration: Management side".
2) what is the real function of the "Gluster Network" cluster traffic type? What it actually does?
Thanks, Stefano. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NTNXPJMOZEYVHI...
On Wed, Oct 16, 2019 at 8:38 PM Jayme <jaymef@gmail.com> wrote:
Is there a way to fix this on a hci deployment which is already in operation? I do have a separate gluster network which is chosen for migration and gluster network but when I originally deployed I used just one set of host names which resolve to management network subnet.
You will need to change the interface that's used by the bricks. You can do this by using the "Reset brick" option, once the gluster management network is set correctly on the storage interface from ovirt-engine
I appear to have a situation where gluster traffic may be going through both networks in seeing what looks like gluster traffic on both the gluster interface and ovirt management.
On Wed, Oct 16, 2019 at 11:34 AM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Thank you Simone for the clarifications.
I've redeployed with both management and storage FQDNs; now everything seems to be in its place.
I only have a couple of questions:
1) In the Gluster deployment Wizard, section 1 (Hosts) and 2 (Additional Hosts) are misleading; should be renamed in something like "Host Configuration: Storage side" / "Host Configuration: Management side".
2) what is the real function of the "Gluster Network" cluster traffic type? What it actually does?
Thanks, Stefano. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NTNXPJMOZEYVHI...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KO5CLJNLRZOGLN...
What does the reset brick option do and is it safe to do this on a live system or do all VMs need to be brought down first? How does resetting the brick fix the issue with gluster peers using the server hostnames which are attached to IPs on the ovirtmanagement network? On Thu, Oct 17, 2019 at 4:16 AM Sahina Bose <sabose@redhat.com> wrote:
On Wed, Oct 16, 2019 at 8:38 PM Jayme <jaymef@gmail.com> wrote:
Is there a way to fix this on a hci deployment which is already in operation? I do have a separate gluster network which is chosen for migration and gluster network but when I originally deployed I used just one set of host names which resolve to management network subnet.
You will need to change the interface that's used by the bricks. You can do this by using the "Reset brick" option, once the gluster management network is set correctly on the storage interface from ovirt-engine
I appear to have a situation where gluster traffic may be going through both networks in seeing what looks like gluster traffic on both the gluster interface and ovirt management.
On Wed, Oct 16, 2019 at 11:34 AM Stefano Stagnaro < stefanos@prismatelecomtesting.com> wrote:
Thank you Simone for the clarifications.
I've redeployed with both management and storage FQDNs; now everything seems to be in its place.
I only have a couple of questions:
1) In the Gluster deployment Wizard, section 1 (Hosts) and 2 (Additional Hosts) are misleading; should be renamed in something like "Host Configuration: Storage side" / "Host Configuration: Management side".
2) what is the real function of the "Gluster Network" cluster traffic type? What it actually does?
Thanks, Stefano. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NTNXPJMOZEYVHI...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KO5CLJNLRZOGLN...
participants (4)
-
Jayme -
Sahina Bose -
Simone Tiraboschi -
Stefano Stagnaro