====== Emulex OCe11102 ======

2-PORT IBM/EMULEX 10GB VIRTUAL FABRIC ADAPTER 49Y7952 OCE11102 HIGH 
  - Low Profile
  - Pci Express 2.0 X8
  - IBM 49Y7952
  - ~50% faster in IOPS comparing to Intel X520
  - Converged Network Adapter (FCoE offloading, iSCSI offload, VIRT offloading)
  - Broadcom Emulex OneConnect OCe11102 ( Chipset: BladeEngine 3 be3)



<code>
04:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
</code>

===== Issues =====

==== AMD-Vi ====

On AMD Ryzen with X570 chipset:
<code bash>
iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=04:00.0 address=0x1001e0b00]
AMD-Vi: Completion-Wait loop timed out
AMD-Vi: Completion-Wait loop timed out
AMD-Vi: Completion-Wait loop timed out
</code>

**Workaround:**
  - add to kernel cmdline: ''iommu=soft''
  - [[https://forum.proxmox.com/threads/pci-passthrough-amd-vi-completion-wait-loop-timed-out.75832/|PCI Passthrough: AMD-Vi: Completion-Wait loop timed out]]

==== bond and bridge ====

There is issue where Emulex is a bond0 slave and bond0 is part of a bridge.

More here: [[vm:proxmox:issues:no_dhcp_offer]]

**Solution:** upgrade to latest firmware.


===== Firmware =====

==== IBM ====

Latest published (from 2011): [[https://www.ibm.com/support/pages/support-cd-release-emulex-based-10gbe-v5142-ibm-system-x|
Support CD Release for Emulex-based 10GbE v5.1.42 - IBM System x
]]


==== Broadcom ====

Please read user guide: 
  - [[https://docs.broadcom.com/doc/12398083|Emulex® Drivers for Linux for OneConnect®
Adapters - User Guide v 12.6]]
  - https://docs.broadcom.com/doc/12378876|Emulex® Drivers for Linux for OneConnect®
Adapters - User Guide v 11.2]]


Download firmware upgrade from:
  - [[https://www.broadcom.com/|Broadcom]] page. Look for ''OneConnect Flash ISO Image''


https://www.dell.com/support/home/pl-pl/drivers/driversdetails?driverid=1dhk6

Issues

[[https://community.hpe.com/t5/networking/frame-error-with-driver-be2net-amp-emulex-oneconnect-10gb/td-p/4792712|Frame error with driver be2net & Emulex OneConnect 10Gb]]

=== fw versions ===
  - 4.1.455.10 - on old i7 machine - bond and bridge works OK
  - 4.6.281.21 - on old i5 machine - bond and bridge works OK
  - 4.6.281.8 - on AMD Ryzne - bond and bridge broadcast issue present
  - 11.4.1186.4 - latest firmware - works OK

=== fw upgrade ===

<code bash>
ethtool -i enp37s0f0

driver: be2net
version: 5.15.30-2-pve
firmware-version: 4.6.281.8
expansion-rom-version: 
bus-info: 0000:25:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: yes

</code>

Flashin firmware with ethtool. Note - process takes long time~

<code bash>
cp oc11-11.4.1186.4.ufi /lib/firmware/
ethtool --flash enp37s0f0 oc11-11.4.1186.4.ufi
ethtool --flash enp37s0f1 oc11-11.4.1186.4.ufi
</code>

<code bash>
# ethtool -i enp37s0f0 
driver: be2net
version: 5.15.30-2-pve
firmware-version: 4.6.281.8 [11.4.1186.4]
expansion-rom-version: 
bus-info: 0000:25:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: yes
</code>

Apply new firmware - reboot the system.