Emulex OCe11102

2-PORT IBM/EMULEX 10GB VIRTUAL FABRIC ADAPTER 49Y7952 OCE11102 HIGH

1. Low Profile
2. Pci Express 2.0 X8
3. IBM 49Y7952
4. ~50% faster in IOPS comparing to Intel X520
5. Converged Network Adapter (FCoE offloading, iSCSI offload, VIRT offloading)
6. Broadcom Emulex OneConnect OCe11102 ( Chipset: BladeEngine 3 be3)

04:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)

On AMD Ryzen with X570 chipset:

iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=04:00.0 address=0x1001e0b00]
AMD-Vi: Completion-Wait loop timed out
AMD-Vi: Completion-Wait loop timed out
AMD-Vi: Completion-Wait loop timed out

Workaround:

1. add to kernel cmdline: iommu=soft
2. PCI Passthrough: AMD-Vi: Completion-Wait loop timed out

There is issue where Emulex is a bond0 slave and bond0 is part of a bridge.

More here: No DCHPOFFERS received

Solution: upgrade to latest firmware.

Latest published (from 2011): Support CD Release for Emulex-based 10GbE v5.1.42 - IBM System x

Please read user guide:

1. Emulex® Drivers for Linux for OneConnect® Adapters - User Guide v 12.6
2. https://docs.broadcom.com/doc/12378876|Emulex® Drivers for Linux for OneConnect®

Adapters - User Guide v 11.2]]

Download firmware upgrade from:

1. Broadcom page. Look for OneConnect Flash ISO Image

https://www.dell.com/support/home/pl-pl/drivers/driversdetails?driverid=1dhk6

Issues

Frame error with driver be2net & Emulex OneConnect 10Gb

1. 4.1.455.10 - on old i7 machine - bond and bridge works OK
2. 4.6.281.21 - on old i5 machine - bond and bridge works OK
3. 4.6.281.8 - on AMD Ryzne - bond and bridge broadcast issue present
4. 11.4.1186.4 - latest firmware - works OK

ethtool -i enp37s0f0
 
driver: be2net
version: 5.15.30-2-pve
firmware-version: 4.6.281.8
expansion-rom-version: 
bus-info: 0000:25:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: yes

Flashin firmware with ethtool. Note - process takes long time~

cp oc11-11.4.1186.4.ufi /lib/firmware/
ethtool --flash enp37s0f0 oc11-11.4.1186.4.ufi
ethtool --flash enp37s0f1 oc11-11.4.1186.4.ufi

# ethtool -i enp37s0f0 
driver: be2net
version: 5.15.30-2-pve
firmware-version: 4.6.281.8 [11.4.1186.4]
expansion-rom-version: 
bus-info: 0000:25:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: yes

Apply new firmware - reboot the system.