[Users] oVirt Solaris support
Hi, I want to start the discussion about Solaris support on oVirt again, as there was no solution for it yet. On my oVirt 3.2.2 environment I installed Solaris 11 U1 with the following specs: * Operating System: Other * nic1: rtl8139 * Disk1: IDE (Thin Provision) * Host: CentOS 6.4 with qemu-kvm-0.12.1.2-2.355.0.1.el6.centos.7.x86_64 These are the same settings as on my RHEL 6.4 KVM host (except I can choose Solaris 10 as OS in virt-manager), which has KVM version: qemu-kvm-rhev-0.12.1.2-2.295.el6_3.2.x86_64 (I wanted to use this host as a RHEV host, so the qemu-kvm-rhev package is installed in case you wounder)... What's working: * OS installation on IDE disk * Bringing up network interface What's not working on oVirt: * Network connections - on RHEL 6.4 with plain libvirt/kvm this is working... I can see the mac address on my CentOS host, but can't ping the Solaris vm: # brctl showmacs ovirtmgmt | egrep '00:99:4a:00:64:83|port' port no mac addr is local? ageing timer 2 00:99:4a:00:64:83 no 10.72 # arp -an | grep '00:99:4a:00:64:83' ? (10.0.100.123) at 00:99:4a:00:64:83 [ether] on ovirtmgmt When using tcpdump on the vnet interface which belongs to the Solaris vm (ip 10.0.100.123) I can see ARP requests from the vm for ip address of my CentOS host (10.0.100.42) but no response to it. Same when pinging other ips in this network: # tcpdump -n -i vnet2 tcpdump: WARNING: vnet2: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vnet2, link-type EN10MB (Ethernet), capture size 65535 bytes 18:15:35.987868 ARP, Request who-has 10.0.100.42 (Broadcast) tell 10.0.100.123, length 46 18:15:36.487399 ARP, Request who-has 10.0.100.42 (Broadcast) tell 10.0.100.123, length 46 18:15:36.987536 ARP, Request who-has 10.0.100.42 (Broadcast) tell 10.0.100.123, length 46 I also compared the qemu-kvm process list on the KVM with the oVirt machine and can't see much differences except that oVirt has more information like smbios.... oVirt host: /usr/libexec/qemu-kvm <snip> -netdev tap,fd=27,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:99:4a:00:64:83,bus=pci.0,addr=0x3 RHEL KVM host: /usr/libexec/qemu-kvm <snip> -netdev tap,fd=32,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:51:c2:97,bus=pci.0,addr=0x3 Any suggestions on how to troubleshoot / get Solaris networking running is welcome. Changing the interface to e1000 doesn't work either. Thanks, René
On Fri, Sep 06, 2013 at 06:23:42PM +0200, René Koch (ovido) wrote:
- Would you share the output of your `brctl show`? - Particulalry, does the host device with 10.0.100.42 sit on the same bridge as your Solaris VM? - Do you see the who-has packet when sniffing the bridge device? - Have yout tried guest-to-guest communication (over the same bridge)? Dan.
On Fri, 2013-09-06 at 22:26 +0100, Dan Kenigsberg wrote:
Thanks a lot for your answer and sorry for my late response - I was sick leaving last week.
- Would you share the output of your `brctl show`?
# brctl show bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000.000000000000 no dmz 8000.0025901ad140 no eth0.200 ovirtmgmt 8000.0025901ad140 no eth0.100 vnet1 vnet2 setup 8000.0025901ad140 no eth0.500 test2 8000.0025901ad140 no eth0.602 vnet0 vnet1 is a Debian 7 vm which has a perfectly working network. vnet2 is the Solaris 11 vm which can't communicate over the network.
- Particulalry, does the host device with 10.0.100.42 sit on the same bridge as your Solaris VM?
10.0.100.42 is the ip of the oVirt host and it's ip is configured on this bridge (and reachable from the Debian 7 vm): # ip a | grep ovirt 5: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN inet 10.0.100.42/24 brd 10.0.100.255 scope global ovirtmgmt
- Do you see the who-has packet when sniffing the bridge device?
Yes, I can see them: 12:24:16.311218 ARP, Request who-has 10.0.100.42 (Broadcast) tell 10.0.100.123, length 46 12:24:16.311246 ARP, Reply 10.0.100.42 is-at 00:25:90:1a:d1:40, length 28 I can also see the MAC of the Solaris vm in arp cache of oVirt host: # arp -an ? (10.0.100.123) at 00:99:4a:00:64:83 [ether] on ovirtmgmt But I can't see the MAC address of oVirt host (10.0.100.42) in arp table of the Solaris 11 vm. To be more clear: I can't see any MAC address in ARP table of Solaris vm except the MAC of the vm itself.
- Have yout tried guest-to-guest communication (over the same bridge)?
I can communicate from the Debian 7 vm to all hosts in my network, but I can't reach the Solaris 11 vm. The network of the oVirt setup is working fine for Windows and various Linux guests, but not for Solaris. Btw, excatly same behavior on RHEV 3.2 with Solaris 10/11 - Windows and Linux is working as expected, but no networking for Solaris guests...
Dan.
On Mon, Sep 16, 2013 at 12:33:34PM +0200, René Koch (ovido) wrote:
I'm afraid I'm out of ideas. Is the kernel the same on both hosts? Would you reply with the complete qemu commandline? Maybe Michael could sport the difference between the working plain-KVM host and the non-working oVirt host. Regrads, Dan.
On Mon, 2013-09-16 at 15:14 +0100, Dan Kenigsberg wrote:
The kernel is not the same: CentOS 6 (oVirt): 2.6.32-358.18.1.el6.x86_64 RHEL 6 (KVM): 2.6.32-358.11.1.el6.x86_64 But I think it's not kernel related as I successfully ran Solaris on the KVM server 1 year ago (so kernel was an old RHEL 6.2 or 6.3 kernel) and didn't worked this time on oVirt 3.1, too (hypervisor was CentOS 6.2 or 6.3 using dreyou's-repository, now I'm running CentOS 6.4 with official oVirt repository). So many kernel upgrades later Solaris still has a working network on plain KVM but not on oVirt 3.2 or RHEV 3.2. Here's the complete qemu cmd: RHEL 6.4 (plain KVM - Solaris networking is fine): /usr/libexec/qemu-kvm -name solaris11 -S -M rhel6.3.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid e7d3b9a4-8605-dc4f-9092-41d68dd83735 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/solaris11.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/solaris11.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=40,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:f0:27:d3,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:13 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 CentOS 6.4 (oVirt - Solaris can't reach other nodes): /usr/libexec/qemu-kvm -name solaris11 -S -M rhel6.3.0 -cpu Nehalem -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5 -smbios type=1,manufacturer=oVirt,product=oVirt Node,version=6-4.el6.centos.10,serial=54443858-4E54-2500-901A-0025901AD140,uuid=cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5,sku=solaris11 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/solaris11.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2013-09-06T15:08:22,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/rhev/data-center/484e62d7-7a01-4b5e-aec8-59d366100281/2055b80d-d59d-4fe6-80ec-5499674feed5/images/0786a273-780a-4f58-acc4-9b123953bde8/0c07f1f9-f9c0-4ac0-bfec-bee7a1c35a0b,if=none,id=drive-ide0-0-0,format=raw,serial=0786a273-780a-4f58-acc4-9b123953bde8,cache=none,werror=stop,rerror=stop,aio=threads -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/var/run/vdsm/payload/cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5.f3b4e68b93e9d35d3f17328fae2e178e.img,if=none,media=cdrom,id=drive-ide0-1-1,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -netdev tap,fd=27,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:99:4a:00:64:83,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/solaris11.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/solaris11.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -spice port=5902,tls-port=5903,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -k en-us -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Thanks, René
Regrads, Dan.
participants (2)
-
Dan Kenigsberg -
René Koch (ovido)