[ovirt-users] oVirt 3.2 - iSCSI offload (broadcom - bnx2i)
Ricardo Esteves
ricardo.m.esteves at gmail.com
Wed Oct 8 11:29:22 UTC 2014
Hi,
lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 5500 I/O Hub to ESI Port
[8086:3403] (rev 13)
00:01.0 PCI bridge [0604]: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 1 [8086:3408] (rev 13)
00:02.0 PCI bridge [0604]: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 2 [8086:3409] (rev 13)
00:03.0 PCI bridge [0604]: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 3 [8086:340a] (rev 13)
00:07.0 PCI bridge [0604]: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 7 [8086:340e] (rev 13)
00:08.0 PCI bridge [0604]: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 8 [8086:340f] (rev 13)
00:09.0 PCI bridge [0604]: Intel Corporation 7500/5520/5500/X58 I/O Hub
PCI Express Root Port 9 [8086:3410] (rev 13)
00:0a.0 PCI bridge [0604]: Intel Corporation 7500/5520/5500/X58 I/O Hub
PCI Express Root Port 10 [8086:3411] (rev 13)
00:0d.0 Host bridge [0600]: Intel Corporation Device [8086:343a] (rev 13)
00:0d.1 Host bridge [0600]: Intel Corporation Device [8086:343b] (rev 13)
00:0d.2 Host bridge [0600]: Intel Corporation Device [8086:343c] (rev 13)
00:0d.3 Host bridge [0600]: Intel Corporation Device [8086:343d] (rev 13)
00:0d.4 Host bridge [0600]: Intel Corporation 7500/5520/5500/X58
Physical Layer Port 0 [8086:3418] (rev 13)
00:0d.5 Host bridge [0600]: Intel Corporation 7500/5520/5500 Physical
Layer Port 1 [8086:3419] (rev 13)
00:0d.6 Host bridge [0600]: Intel Corporation Device [8086:341a] (rev 13)
00:0e.0 Host bridge [0600]: Intel Corporation Device [8086:341c] (rev 13)
00:0e.1 Host bridge [0600]: Intel Corporation Device [8086:341d] (rev 13)
00:0e.2 Host bridge [0600]: Intel Corporation Device [8086:341e] (rev 13)
00:0e.3 Host bridge [0600]: Intel Corporation Device [8086:341f] (rev 13)
00:0e.4 Host bridge [0600]: Intel Corporation Device [8086:3439] (rev 13)
00:14.0 PIC [0800]: Intel Corporation 7500/5520/5500/X58 I/O Hub System
Management Registers [8086:342e] (rev 13)
00:14.1 PIC [0800]: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO
and Scratch Pad Registers [8086:3422] (rev 13)
00:14.2 PIC [0800]: Intel Corporation 7500/5520/5500/X58 I/O Hub Control
Status and RAS Registers [8086:3423] (rev 13)
00:1c.0 PCI bridge [0604]: Intel Corporation 82801JI (ICH10 Family) PCI
Express Root Port 1 [8086:3a40]
00:1d.0 USB controller [0c03]: Intel Corporation 82801JI (ICH10 Family)
USB UHCI Controller #1 [8086:3a34]
00:1d.1 USB controller [0c03]: Intel Corporation 82801JI (ICH10 Family)
USB UHCI Controller #2 [8086:3a35]
00:1d.2 USB controller [0c03]: Intel Corporation 82801JI (ICH10 Family)
USB UHCI Controller #3 [8086:3a36]
00:1d.3 USB controller [0c03]: Intel Corporation 82801JI (ICH10 Family)
USB UHCI Controller #6 [8086:3a39]
00:1d.7 USB controller [0c03]: Intel Corporation 82801JI (ICH10 Family)
USB2 EHCI Controller #1 [8086:3a3a]
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge
[8086:244e] (rev 90)
00:1f.0 ISA bridge [0601]: Intel Corporation 82801JIB (ICH10) LPC
Interface Controller [8086:3a18]
01:03.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc.
[AMD/ATI] ES1000 [1002:515e] (rev 02)
01:04.0 System peripheral [0880]: Compaq Computer Corporation Integrated
Lights Out Controller [0e11:b203] (rev 03)
01:04.2 System peripheral [0880]: Compaq Computer Corporation Integrated
Lights Out Processor [0e11:b204] (rev 03)
01:04.4 USB controller [0c03]: Hewlett-Packard Company Integrated
Lights-Out Standard Virtual USB Controller [103c:3300]
01:04.6 IPMI SMIC interface [0c07]: Hewlett-Packard Company Integrated
Lights-Out Standard KCS Interface [103c:3302]
02:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II
BCM57711E 10-Gigabit PCIe [14e4:1650]
02:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II
BCM57711E 10-Gigabit PCIe [14e4:1650]
06:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II
BCM5709S Gigabit Ethernet [14e4:163a] (rev 20)
06:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II
BCM5709S Gigabit Ethernet [14e4:163a] (rev 20)
0c:00.0 RAID bus controller [0104]: Hewlett-Packard Company Smart Array
G6 controllers [103c:323a] (rev 01)
-----------
lsmod | grep bnx2
bnx2i 52167 0
libiscsi 49836 8
be2iscsi,cxgb4i,cxgb3i,libcxgbi,ib_iser,iscsi_tcp,libiscsi_tcp,bnx2i
scsi_transport_iscsi 84241 8
be2iscsi,libcxgbi,ib_iser,iscsi_tcp,bnx2i,libiscsi
cnic 72983 1 bnx2i
bnx2 199384 0
bnx2x 1445902 0
libcrc32c 1246 1 bnx2x
mdio 4769 2 cxgb3,bnx2x
--------
dmesg | grep bnx2
bnx2x: Broadcom NetXtreme II 5771x/578xx 10/20-Gigabit Ethernet Driver
bnx2x 1.710.39 ($DateTime: 2014/06/02 04:31:55 $)
bnx2x 0000:02:00.0: msix capability found
bnx2x 0000:02:00.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
bnx2x 0000:02:00.0: setting latency timer to 64
bnx2x 0000:02:00.0: part number 412F4E-0-0-0
bnx2x 0000:02:00.0: irq 53 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 54 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 55 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 56 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 57 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 58 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 59 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 60 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 61 for MSI/MSI-X
bnx2x 0000:02:00.0: irq 62 for MSI/MSI-X
bnx2x 0000:02:00.1: msix capability found
bnx2x 0000:02:00.1: PCI INT B -> GSI 40 (level, low) -> IRQ 40
bnx2x 0000:02:00.1: setting latency timer to 64
bnx2x 0000:02:00.1: part number 412F4E-0-0-0
bnx2x 0000:02:00.1: irq 63 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 64 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 65 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 66 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 67 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 68 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 69 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 70 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 71 for MSI/MSI-X
bnx2x 0000:02:00.1: irq 72 for MSI/MSI-X
bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.2.4g (Nov
07, 2013)
bnx2 0000:06:00.0: PCI INT A -> GSI 30 (level, low) -> IRQ 30
bnx2 0000:06:00.0: setting latency timer to 64
bnx2 0000:06:00.0: eth2: Broadcom NetXtreme II BCM5709 1000Base-SX (C0)
PCI Express found at mem f8000000, IRQ 30, node addr 1c:c1:de:7b:71:30
bnx2 0000:06:00.1: PCI INT B -> GSI 37 (level, low) -> IRQ 37
bnx2 0000:06:00.1: setting latency timer to 64
bnx2 0000:06:00.1: eth3: Broadcom NetXtreme II BCM5709 1000Base-SX (C0)
PCI Express found at mem f6000000, IRQ 37, node addr 1c:c1:de:7b:71:32
bnx2x 0000:02:00.0: eth0: using MSI-X IRQs: sp 53 fp[0] 55 ... fp[7] 62
bnx2 0000:06:00.0: irq 73 for MSI/MSI-X
bnx2 0000:06:00.0: irq 74 for MSI/MSI-X
bnx2 0000:06:00.0: irq 75 for MSI/MSI-X
bnx2 0000:06:00.0: irq 76 for MSI/MSI-X
bnx2 0000:06:00.0: irq 77 for MSI/MSI-X
bnx2 0000:06:00.0: irq 78 for MSI/MSI-X
bnx2 0000:06:00.0: irq 79 for MSI/MSI-X
bnx2 0000:06:00.0: irq 80 for MSI/MSI-X
bnx2 0000:06:00.0: irq 81 for MSI/MSI-X
bnx2 0000:06:00.0: eth2: using MSIX
bnx2 0000:06:00.0: eth2: NIC SerDes Link is Up, 1000 Mbps full duplex,
receive & transmit flow control ON
bnx2x 0000:02:00.0: eth0: NIC Link is Up, 1000 Mbps full duplex, Flow
control: ON - receive & transmit
bnx2x 0000:02:00.1: eth1: using MSI-X IRQs: sp 63 fp[0] 65 ... fp[7] 72
bnx2 0000:06:00.0: eth2: NIC SerDes Link is Down
bnx2 0000:06:00.0: eth2: NIC SerDes Link is Up, 1000 Mbps full duplex,
receive & transmit flow control ON
bnx2x 0000:02:00.1: eth1: NIC Link is Up, 1000 Mbps full duplex, Flow
control: ON - receive & transmit
bnx2 0000:06:00.1: irq 82 for MSI/MSI-X
bnx2 0000:06:00.1: irq 83 for MSI/MSI-X
bnx2 0000:06:00.1: irq 84 for MSI/MSI-X
bnx2 0000:06:00.1: irq 85 for MSI/MSI-X
bnx2 0000:06:00.1: irq 86 for MSI/MSI-X
bnx2 0000:06:00.1: irq 87 for MSI/MSI-X
bnx2 0000:06:00.1: irq 88 for MSI/MSI-X
bnx2 0000:06:00.1: irq 89 for MSI/MSI-X
bnx2 0000:06:00.1: irq 90 for MSI/MSI-X
bnx2 0000:06:00.1: eth3: using MSIX
bnx2 0000:06:00.1: eth3: NIC SerDes Link is Up, 1000 Mbps full duplex,
receive & transmit flow control ON
bnx2x 0000:02:00.0: eth0: Added CNIC device
bnx2x 0000:02:00.1: eth1: Added CNIC device
bnx2 0000:06:00.0: eth2: Added CNIC device
bnx2 0000:06:00.1: eth3: Added CNIC device
Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.10.31b (Mar 06, 2014)
iscsi: registered transport (bnx2i)
bnx2i [06:00.01]: ISCSI_INIT passed
bnx2i [06:00.00]: ISCSI_INIT passed
bnx2i: dev eth1 does not support iSCSI
bnx2i init: (eth1) hba ffff880493155640 init failed
bnx2i: eth1 free_hba done after 0 retries
bnx2i: dev eth0 does not support iSCSI
bnx2i init: (eth0) hba ffff880493155640 init failed
bnx2i: eth0 free_hba done after 0 retries
Loading kernel module for a network device with CAP_SYS_MODULE
(deprecated). Use CAP_NET_ADMIN and alias netdev-bnx2x instead
-------
There is no option on bios to activate offloading, and i had it working
on vmware esxi.
Anyway, i think i found the solution, the mac address of my eth3 nic is
1c:c1:de:7b:71:32, so from what i read it seems that that corresponding
iscsi device mac address is 1c:c1:de:7b:71:33, so adding the mac addres
and ip to /var/lib/iscsi/ifaces/eth3 and change from tcp to bnx2i and then
remove and re add it in "iSCSI Mutipathing" tab works
iface.hwaddress = 1c:c1:de:7b:71:33
iface.ipaddress = 192.168.12.125
iface.transport_name = bnx2i
I see that already exists a
/var/lib/iscsi/ifaces/bnx2i.1c:c1:de:7b:71:33, so if i create
/etc/sysconfig/network-script/ifcfg-bnx2i.1c:c1:de:7b:71:33 i should be
able to add it in Ovirt GUI to a logical network and then use it in
"iSCSI Mutipathing" tab? I'm going to try this...
Best regards,
Ricardo Esteves.
On 08-10-2014 09:24, Federico Simoncelli wrote:
> ----- Original Message -----
>> From: "Ricardo Esteves" <ricardo.m.esteves at gmail.com>
>> To: "Federico Simoncelli" <fsimonce at redhat.com>
>> Cc: users at ovirt.org
>> Sent: Wednesday, October 8, 2014 1:32:51 AM
>> Subject: Re: [ovirt-users] oVirt 3.2 - iSCSI offload (broadcom - bnx2i)
>>
>> Hi, here it goes:
>>
>> ethtool -i eth3
>>
>> driver: bnx2
>> version: 2.2.4g
>> firmware-version: bc 5.2.3
>> bus-info: 0000:06:00.1
>> supports-statistics: yes
>> supports-test: yes
>> supports-eeprom-access: yes
>> supports-register-dump: yes
>> supports-priv-flags: no
> Thanks, can you also add the output of:
>
> # lspci -nn
>
> I'd have expected the driver to be bnx2i (bnx2 is a regular ethernet driver,
> no offloading).
>
> Can you also check if you have the bnx2i driver loaded?
>
> # lsmod | grep bnx2
>
> and eventually if there's any bnx2 related message in /var/log/messages
>
> Check also the adapter bios (at boot time) if by any chance you have to enable
> the offloading there first (check the specific manual too).
>
More information about the Users
mailing list