meta data for this page
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
hw:server:10gbe [2022/10/26 21:31] niziak |
hw:server:10gbe [2022/10/27 21:12] (current) niziak |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Emulex ====== | + | ====== Emulex OCe11102 ====== |
| + | |||
| + | 2-PORT IBM/EMULEX 10GB VIRTUAL FABRIC ADAPTER 49Y7952 OCE11102 HIGH | ||
| + | - Low Profile | ||
| + | - Pci Express 2.0 X8 | ||
| + | - IBM 49Y7952 | ||
| + | - ~50% faster in IOPS comparing to Intel X520 | ||
| + | - Converged Network Adapter (FCoE offloading, iSCSI offload, VIRT offloading) | ||
| + | - Broadcom Emulex OneConnect OCe11102 ( Chipset: BladeEngine 3 be3) | ||
| - | Emulex OneConnect OCe11102-N 10Gb Ethernet (10GbE) | ||
| <code> | <code> | ||
| Line 7: | Line 15: | ||
| 04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) | 04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) | ||
| </code> | </code> | ||
| + | |||
| + | ===== Issues ===== | ||
| + | |||
| + | ==== AMD-Vi ==== | ||
| On AMD Ryzen with X570 chipset: | On AMD Ryzen with X570 chipset: | ||
| Line 16: | Line 28: | ||
| </code> | </code> | ||
| - | Workaround: | + | **Workaround:** |
| - | add to kernel cmdline: ''iommu=soft'' | + | - add to kernel cmdline: ''iommu=soft'' |
| + | - [[https://forum.proxmox.com/threads/pci-passthrough-amd-vi-completion-wait-loop-timed-out.75832/|PCI Passthrough: AMD-Vi: Completion-Wait loop timed out]] | ||
| - | [[https://forum.proxmox.com/threads/pci-passthrough-amd-vi-completion-wait-loop-timed-out.75832/]] | + | ==== bond and bridge ==== |
| + | |||
| + | There is issue where Emulex is a bond0 slave and bond0 is part of a bridge. | ||
| + | |||
| + | More here: [[vm:proxmox:issues:no_dhcp_offer]] | ||
| + | |||
| + | **Solution:** upgrade to latest firmware. | ||
| + | |||
| + | |||
| + | ===== Firmware ===== | ||
| + | |||
| + | ==== IBM ==== | ||
| + | |||
| + | Latest published (from 2011): [[https://www.ibm.com/support/pages/support-cd-release-emulex-based-10gbe-v5142-ibm-system-x| | ||
| + | Support CD Release for Emulex-based 10GbE v5.1.42 - IBM System x | ||
| + | ]] | ||
| + | |||
| + | |||
| + | ==== Broadcom ==== | ||
| + | |||
| + | Please read user guide: | ||
| + | - [[https://docs.broadcom.com/doc/12398083|Emulex® Drivers for Linux for OneConnect® | ||
| + | Adapters - User Guide v 12.6]] | ||
| + | - https://docs.broadcom.com/doc/12378876|Emulex® Drivers for Linux for OneConnect® | ||
| + | Adapters - User Guide v 11.2]] | ||
| + | |||
| + | |||
| + | Download firmware upgrade from: | ||
| + | - [[https://www.broadcom.com/|Broadcom]] page. Look for ''OneConnect Flash ISO Image'' | ||
| + | |||
| + | |||
| + | https://www.dell.com/support/home/pl-pl/drivers/driversdetails?driverid=1dhk6 | ||
| + | |||
| + | Issues | ||
| + | |||
| + | [[https://community.hpe.com/t5/networking/frame-error-with-driver-be2net-amp-emulex-oneconnect-10gb/td-p/4792712|Frame error with driver be2net & Emulex OneConnect 10Gb]] | ||
| + | |||
| + | === fw versions === | ||
| + | - 4.1.455.10 - on old i7 machine - bond and bridge works OK | ||
| + | - 4.6.281.21 - on old i5 machine - bond and bridge works OK | ||
| + | - 4.6.281.8 - on AMD Ryzne - bond and bridge broadcast issue present | ||
| + | - 11.4.1186.4 - latest firmware - works OK | ||
| + | |||
| + | === fw upgrade === | ||
| + | |||
| + | <code bash> | ||
| + | ethtool -i enp37s0f0 | ||
| + | |||
| + | driver: be2net | ||
| + | version: 5.15.30-2-pve | ||
| + | firmware-version: 4.6.281.8 | ||
| + | expansion-rom-version: | ||
| + | bus-info: 0000:25:00.0 | ||
| + | supports-statistics: yes | ||
| + | supports-test: yes | ||
| + | supports-eeprom-access: yes | ||
| + | supports-register-dump: no | ||
| + | supports-priv-flags: yes | ||
| + | |||
| + | </code> | ||
| + | |||
| + | Flashin firmware with ethtool. Note - process takes long time~ | ||
| + | |||
| + | <code bash> | ||
| + | cp oc11-11.4.1186.4.ufi /lib/firmware/ | ||
| + | ethtool --flash enp37s0f0 oc11-11.4.1186.4.ufi | ||
| + | ethtool --flash enp37s0f1 oc11-11.4.1186.4.ufi | ||
| + | </code> | ||
| + | |||
| + | <code bash> | ||
| + | # ethtool -i enp37s0f0 | ||
| + | driver: be2net | ||
| + | version: 5.15.30-2-pve | ||
| + | firmware-version: 4.6.281.8 [11.4.1186.4] | ||
| + | expansion-rom-version: | ||
| + | bus-info: 0000:25:00.0 | ||
| + | supports-statistics: yes | ||
| + | supports-test: yes | ||
| + | supports-eeprom-access: yes | ||
| + | supports-register-dump: no | ||
| + | supports-priv-flags: yes | ||
| + | </code> | ||
| + | Apply new firmware - reboot the system. | ||
| - | Firmware upgrade on [[https://www.broadcom.com/|Broadcom]] page. Look for ''OneConnect Flash ISO Image'' | ||
| - | [[https://docs.broadcom.com/doc/12378876|Emulex® Drivers for Linux for OneConnect® | ||
| - | Adapters - User Guide]] | ||