Re: [ovirt-users] jumbo frames inside vm best practices?

Hello, I'm testing an Oracle RAC with 2 Oracle Linux VMs inside a 4.0.6 environment. They run on two different hosts I would like to configure RAC intracluster communication with jumbo frames. At VM level network adapter is eth1 (mapped to a vlan 95 at oVirt hosts side) At oVirt side I configured a vm enabled vlan with mtu=9000 I verified that at hosts side I have vlan95: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 ether 00:1c:c4:ab:be:ba txqueuelen 1000 (Ethernet) RX packets 61706 bytes 3631426 (3.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3 bytes 258 (258.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 And able to do a ping -M do -s 8972 ip from each host to the other one In VMs I configure the same MTU=9000 in ifcfg-eth1 But actually inside VMs it works erratically: the same ping test is ok between the VMs but Oracle checks sometimes work and sometimes give error on communication. At initial cluster config, the second node fails to start the cluster. I tried 5-6 times and also tried then to set mtu=8000 inside the VMs, supposing some sort of inner overhead to consider (such as 2 times 28 bytes) but nothing. As soon as I set MTU=1500 at VM side, the cluster is able to form without any problem. I can survive without jumbo frames in this particular case, because this is only a test, but the question remains about eventual best practices to put in place if I want to use jumbo frames. One thing I see is that at VM side I see many drops when interface mtu was 9000, such as eth1 Link encap:Ethernet HWaddr 00:1A:4A:17:01:57 inet addr:192.168.10.32 Bcast:192.168.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:93046 errors:0 dropped:54964 overruns:0 frame:0 TX packets:26258 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:25726242 (24.5 MiB) TX bytes:33573207 (32.0 MiB) at host side I see drops at bond0 level only: [root@ovmsrv05 ~]# brctl show bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000.000000000000 no vlan100 8000.001cc446ef73 no bond1.100 vlan65 8000.001cc446ef73 no bond1.65 vnet0 vnet1 vlan95 8000.001cc4abbeba no bond0.95 vnet2 bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 9000 ether 00:1c:c4:ab:be:ba txqueuelen 1000 (Ethernet) RX packets 2855175 bytes 3126868334 <(312)%20686-8334> (2.9 GiB) RX errors 0 dropped 11686 overruns 0 frame 0 TX packets 1012849 bytes 478702140 (456.5 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 bond0.95: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 ether 00:1c:c4:ab:be:ba txqueuelen 1000 (Ethernet) RX packets 100272 bytes 27125992 (25.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 42355 bytes 40833904 (38.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vlan95: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 ether 00:1c:c4:ab:be:ba txqueuelen 1000 (Ethernet) RX packets 62576 bytes 3719175 (3.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3 bytes 258 (258.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 inet6 fe80::fc1a:4aff:fe17:157 prefixlen 64 scopeid 0x20<link> ether fe:1a:4a:17:01:57 txqueuelen 1000 (Ethernet) RX packets 21014 bytes 24139492 (23.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 85777 bytes 21089777 (20.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@ovmsrv05 ~]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp3s0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp3s0 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:1c:c4:ab:be:ba Slave queue ID: 0 Slave Interface: enp5s0 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:1c:c4:ab:be:bc Slave queue ID: 0 Any hint? Thanks in advance, Gianluca _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users