<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/4.6.6">
</HEAD>
<BODY>
<BR>
On Mon, 2013-09-16 at 15:14 +0100, Dan Kenigsberg wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
On Mon, Sep 16, 2013 at 12:33:34PM +0200, René Koch (ovido) wrote:
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> On Fri, 2013-09-06 at 22:26 +0100, Dan Kenigsberg wrote:</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> > On Fri, Sep 06, 2013 at 06:23:42PM +0200, René Koch (ovido) wrote:</FONT>
<FONT COLOR="#737373">> > > Hi,</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > I want to start the discussion about Solaris support on oVirt again, as</FONT>
<FONT COLOR="#737373">> > > there was no solution for it yet.</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > On my oVirt 3.2.2 environment I installed Solaris 11 U1 with the</FONT>
<FONT COLOR="#737373">> > > following specs:</FONT>
<FONT COLOR="#737373">> > > * Operating System: Other</FONT>
<FONT COLOR="#737373">> > > * nic1: rtl8139</FONT>
<FONT COLOR="#737373">> > > * Disk1: IDE (Thin Provision)</FONT>
<FONT COLOR="#737373">> > > * Host: CentOS 6.4 with qemu-kvm-0.12.1.2-2.355.0.1.el6.centos.7.x86_64</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > These are the same settings as on my RHEL 6.4 KVM host (except I can</FONT>
<FONT COLOR="#737373">> > > choose Solaris 10 as OS in virt-manager), which has KVM version:</FONT>
<FONT COLOR="#737373">> > > qemu-kvm-rhev-0.12.1.2-2.295.el6_3.2.x86_64 (I wanted to use this host</FONT>
<FONT COLOR="#737373">> > > as a RHEV host, so the qemu-kvm-rhev package is installed in case you</FONT>
<FONT COLOR="#737373">> > > wounder)...</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > What's working:</FONT>
<FONT COLOR="#737373">> > > * OS installation on IDE disk</FONT>
<FONT COLOR="#737373">> > > * Bringing up network interface</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > What's not working on oVirt:</FONT>
<FONT COLOR="#737373">> > > * Network connections - on RHEL 6.4 with plain libvirt/kvm this is</FONT>
<FONT COLOR="#737373">> > > working...</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > I can see the mac address on my CentOS host, but can't ping the Solaris</FONT>
<FONT COLOR="#737373">> > > vm:</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > # brctl showmacs ovirtmgmt | egrep '00:99:4a:00:64:83|port'</FONT>
<FONT COLOR="#737373">> > > port no mac addr is local? ageing timer</FONT>
<FONT COLOR="#737373">> > > 2 00:99:4a:00:64:83 no 10.72</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > # arp -an | grep '00:99:4a:00:64:83'</FONT>
<FONT COLOR="#737373">> > > ? (10.0.100.123) at 00:99:4a:00:64:83 [ether] on ovirtmgmt</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > When using tcpdump on the vnet interface which belongs to the Solaris vm</FONT>
<FONT COLOR="#737373">> > > (ip 10.0.100.123) I can see ARP requests from the vm for ip address of</FONT>
<FONT COLOR="#737373">> > > my CentOS host (10.0.100.42) but no response to it. Same when pinging</FONT>
<FONT COLOR="#737373">> > > other ips in this network:</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > # tcpdump -n -i vnet2</FONT>
<FONT COLOR="#737373">> > > tcpdump: WARNING: vnet2: no IPv4 address assigned</FONT>
<FONT COLOR="#737373">> > > tcpdump: verbose output suppressed, use -v or -vv for full protocol</FONT>
<FONT COLOR="#737373">> > > decode</FONT>
<FONT COLOR="#737373">> > > listening on vnet2, link-type EN10MB (Ethernet), capture size 65535</FONT>
<FONT COLOR="#737373">> > > bytes</FONT>
<FONT COLOR="#737373">> > > 18:15:35.987868 ARP, Request who-has 10.0.100.42 (Broadcast) tell</FONT>
<FONT COLOR="#737373">> > > 10.0.100.123, length 46</FONT>
<FONT COLOR="#737373">> > > 18:15:36.487399 ARP, Request who-has 10.0.100.42 (Broadcast) tell</FONT>
<FONT COLOR="#737373">> > > 10.0.100.123, length 46</FONT>
<FONT COLOR="#737373">> > > 18:15:36.987536 ARP, Request who-has 10.0.100.42 (Broadcast) tell</FONT>
<FONT COLOR="#737373">> > > 10.0.100.123, length 46</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > I also compared the qemu-kvm process list on the KVM with the oVirt</FONT>
<FONT COLOR="#737373">> > > machine and can't see much differences except that oVirt has more</FONT>
<FONT COLOR="#737373">> > > information like smbios....</FONT>
<FONT COLOR="#737373">> > > oVirt host:</FONT>
<FONT COLOR="#737373">> > > /usr/libexec/qemu-kvm</FONT>
<FONT COLOR="#737373">> > > <snip></FONT>
<FONT COLOR="#737373">> > > -netdev tap,fd=27,id=hostnet0</FONT>
<FONT COLOR="#737373">> > > -device</FONT>
<FONT COLOR="#737373">> > > rtl8139,netdev=hostnet0,id=net0,mac=00:99:4a:00:64:83,bus=pci.0,addr=0x3</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > RHEL KVM host:</FONT>
<FONT COLOR="#737373">> > > /usr/libexec/qemu-kvm</FONT>
<FONT COLOR="#737373">> > > <snip></FONT>
<FONT COLOR="#737373">> > > -netdev tap,fd=32,id=hostnet0</FONT>
<FONT COLOR="#737373">> > > -device</FONT>
<FONT COLOR="#737373">> > > rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:51:c2:97,bus=pci.0,addr=0x3</FONT>
<FONT COLOR="#737373">> > > </FONT>
<FONT COLOR="#737373">> > > Any suggestions on how to troubleshoot / get Solaris networking running</FONT>
<FONT COLOR="#737373">> > > is welcome.</FONT>
<FONT COLOR="#737373">> > > Changing the interface to e1000 doesn't work either.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Thanks a lot for your answer and sorry for my late response - I was sick</FONT>
<FONT COLOR="#737373">> leaving last week.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> > </FONT>
<FONT COLOR="#737373">> > - Would you share the output of your `brctl show`?</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> # brctl show</FONT>
<FONT COLOR="#737373">> bridge name bridge id STP enabled interfaces</FONT>
<FONT COLOR="#737373">> ;vdsmdummy; 8000.000000000000 no </FONT>
<FONT COLOR="#737373">> dmz 8000.0025901ad140 no eth0.200</FONT>
<FONT COLOR="#737373">> ovirtmgmt 8000.0025901ad140 no eth0.100</FONT>
<FONT COLOR="#737373">> vnet1</FONT>
<FONT COLOR="#737373">> vnet2</FONT>
<FONT COLOR="#737373">> setup 8000.0025901ad140 no eth0.500</FONT>
<FONT COLOR="#737373">> test2 8000.0025901ad140 no eth0.602</FONT>
<FONT COLOR="#737373">> vnet0</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> vnet1 is a Debian 7 vm which has a perfectly working network.</FONT>
<FONT COLOR="#737373">> vnet2 is the Solaris 11 vm which can't communicate over the network.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> > - Particulalry, does the host device with 10.0.100.42 sit on the same</FONT>
<FONT COLOR="#737373">> > bridge as your Solaris VM?</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> 10.0.100.42 is the ip of the oVirt host and it's ip is configured on</FONT>
<FONT COLOR="#737373">> this bridge (and reachable from the Debian 7 vm):</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> # ip a | grep ovirt</FONT>
<FONT COLOR="#737373">> 5: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue</FONT>
<FONT COLOR="#737373">> state UNKNOWN </FONT>
<FONT COLOR="#737373">> inet 10.0.100.42/24 brd 10.0.100.255 scope global ovirtmgmt</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> > - Do you see the who-has packet when sniffing the bridge device?</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Yes, I can see them:</FONT>
<FONT COLOR="#737373">> 12:24:16.311218 ARP, Request who-has 10.0.100.42 (Broadcast) tell</FONT>
<FONT COLOR="#737373">> 10.0.100.123, length 46</FONT>
<FONT COLOR="#737373">> 12:24:16.311246 ARP, Reply 10.0.100.42 is-at 00:25:90:1a:d1:40, length</FONT>
<FONT COLOR="#737373">> 28</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> I can also see the MAC of the Solaris vm in arp cache of oVirt host:</FONT>
<FONT COLOR="#737373">> # arp -an</FONT>
<FONT COLOR="#737373">> ? (10.0.100.123) at 00:99:4a:00:64:83 [ether] on ovirtmgmt</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> But I can't see the MAC address of oVirt host (10.0.100.42) in arp table</FONT>
<FONT COLOR="#737373">> of the Solaris 11 vm. To be more clear: I can't see any MAC address in</FONT>
<FONT COLOR="#737373">> ARP table of Solaris vm except the MAC of the vm itself.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> > - Have yout tried guest-to-guest communication (over the same bridge)?</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> I can communicate from the Debian 7 vm to all hosts in my network, but I</FONT>
<FONT COLOR="#737373">> can't reach the Solaris 11 vm.</FONT>
<FONT COLOR="#737373">> The network of the oVirt setup is working fine for Windows and various</FONT>
<FONT COLOR="#737373">> Linux guests, but not for Solaris.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Btw, excatly same behavior on RHEV 3.2 with Solaris 10/11 - Windows and</FONT>
<FONT COLOR="#737373">> Linux is working as expected, but no networking for Solaris guests...</FONT>
I'm afraid I'm out of ideas. Is the kernel the same on both hosts?
Would you reply with the complete qemu commandline? Maybe Michael could
sport the difference between the working plain-KVM host and the
non-working oVirt host.
</PRE>
</BLOCKQUOTE>
<BR>
<BR>
The kernel is not the same:<BR>
CentOS 6 (oVirt): 2.6.32-358.18.1.el6.x86_64<BR>
RHEL 6 (KVM): 2.6.32-358.11.1.el6.x86_64<BR>
But I think it's not kernel related as I successfully ran Solaris on the KVM server 1 year ago (so kernel was an old RHEL 6.2 or 6.3 kernel) and didn't worked this time on oVirt 3.1, too (hypervisor was CentOS 6.2 or 6.3 using dreyou's-repository, now I'm running CentOS 6.4 with official oVirt repository). So many kernel upgrades later Solaris still has a working network on plain KVM but not on oVirt 3.2 or RHEV 3.2.<BR>
<BR>
Here's the complete qemu cmd:<BR>
<BR>
RHEL 6.4 (plain KVM - Solaris networking is fine):<BR>
/usr/libexec/qemu-kvm -name solaris11 -S -M rhel6.3.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid e7d3b9a4-8605-dc4f-9092-41d68dd83735 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/solaris11.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/solaris11.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=40,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:f0:27:d3,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:13 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5<BR>
<BR>
CentOS 6.4 (oVirt - Solaris can't reach other nodes):<BR>
/usr/libexec/qemu-kvm -name solaris11 -S -M rhel6.3.0 -cpu Nehalem -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5 -smbios type=1,manufacturer=oVirt,product=oVirt Node,version=6-4.el6.centos.10,serial=54443858-4E54-2500-901A-0025901AD140,uuid=cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5,sku=solaris11 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/solaris11.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2013-09-06T15:08:22,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/rhev/data-center/484e62d7-7a01-4b5e-aec8-59d366100281/2055b80d-d59d-4fe6-80ec-5499674feed5/images/0786a273-780a-4f58-acc4-9b123953bde8/0c07f1f9-f9c0-4ac0-bfec-bee7a1c35a0b,if=none,id=drive-ide0-0-0,format=raw,serial=0786a273-780a-4f58-acc4-9b123953bde8,cache=none,werror=stop,rerror=stop,aio=threads -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/var/run/vdsm/payload/cbaa0cf5-d9e6-40e4-a269-9dc495f4c0d5.f3b4e68b93e9d35d3f17328fae2e178e.img,if=none,media=cdrom,id=drive-ide0-1-1,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -netdev tap,fd=27,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:99:4a:00:64:83,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/solaris11.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/solaris11.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -spice port=5902,tls-port=5903,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -k en-us -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5<BR>
<BR>
<BR>
Thanks,<BR>
René<BR>
<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<PRE>
Regrads,
Dan.
</PRE>
</BLOCKQUOTE>
</BODY>
</HTML>