Flashes (Alerts)
Abstract
When a Linux system is booted with the disable_ddw kernel command-line parameter, the amount of memory available for DMA-capable I/O adapters is limited to 2 GB (32-bit). This limitation can be an issue for high-speed I/O devices that need larger amounts of DMA-capable memory to run at maximum throughput such as the IBM Quad-Port 10Gbit NIC (P/N 03JP016). When two or more such NICs are installed in an LPAR, the i40e device driver might fail to initialize some of the instances on the device correctly and report a memory allocation error.
Content
Linux Releases Affected
Red Hat Enterprise Linux (RHEL) 8.X (all versions)
Red Hat Enterprise Linux (RHEL) 9.X (all versions)
SuSE SLES 12 (all versions)
SuSE SLES 15 (all versions)
Any other Linux distribution with the i40e device driver
IBM Systems Affected
All IBM POWER systems running Linux with the quad-port, 10Gb NIC (IBM P/N 03JP012, 03JP014,
03JP015, or 03JP016)
Symptoms
When Linux boots, the following messages are observed in the system message log:
i40e XXXX:XX:XX.X: Unable to allocate memory for the Tx descriptor ring, size=XXXXX
Or
i40e XXXX:XX:XX.X: Unable to allocate memory for the Rx descriptor ring, size=XXXXX
Workaround
Based on the concept of increasing the amount of available DMA-capable memory in the system or reducing the amount of memory required by the NIC, following are few workarounds for this issue:
- Remove the disable_ddw Linux kernel command-line option and any older PCI devices that require the disable_ddw option to be set.
- Limit the number of NICs connected to a Linux LPAR as a Physical I/O adapter to one.
- Reduce the number of RX/TX descriptors used by the NIC by using the following command:
ethtool -G <INTERFACE> rx <NUM> tx <NUM>
Fix Outlook
As there are no plans to resolve this issue currently, use the workarounds.
I/O device impacted IBM P/Ns: 03JP012, 03JP014, 03JP015 (Add-In), 03JP016 (FRU)
[{"Type":"MASTER","Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SGMV157","label":"IBM Support for Red Hat Enterprise Linux Server"},"ARM Category":[{"code":"a8m0z000000Gnl7AAC","label":"Red Hat Enterprise Linux"},{"code":"a8m0z000000GnlCAAS","label":"SUSE Linux Enterprise Server"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
10 October 2022
UID
ibm16827533