OFED & DPDK installation

Resources (OFED and DPDK) are under /opt on 021 and 022.:

OFED: /opt/MLNX_OFED/MLNX_OFED_LINUX-5.6-2.0.9.0-rhel8.5-x86_64

Installation arguments (will need some system packages installed: kernel-rpm-macros libtool createrepo_c python3-pyelftools):

./mlnxofedinstall --add-kernel-support --dpdk --umad-dev-rw

DPDK: /opt/dpdk/dpdk-22.03

Build steps (no special parameters here):

meson build
cd build
ninja
ninja install

After reboots we do the following hugepage setup for DPDK:

[root@np04-srv-021 opt]# pwd
/opt
[root@np04-srv-021 opt]# cat setup-100g.sh 
#!/bin/bash

ifconfig ens801f0np0 mtu 9000

/opt/dpdk/dpdk-22.03/usertools/dpdk-hugepages.py -c

echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

/opt/dpdk/dpdk-22.03/usertools/dpdk-hugepages.py -m

Give godlike persmissin for hugepages for every user:

chmod -R 777 /dev/hugepages

When we are running applications, we usually preload LD with the DPDK installation location:

export LD_LIBRARY_PATH=/usr/local/lib64/:$LD_LIBRARY_PATH
export DPDK_LIB=/usr/local/lib64

Then.... finally the goal is.... To run this app as np04daq or any user:

/opt/dpdk/dpdk-22.03/build/app/dpdk-testpmd

Tarballs for OFED and DPDK are under /opt/MLNX_OFED and /opt/dpdk/:

[root@np04-srv-021 MLNX_OFED]# ls
MLNX_OFED_LINUX-5.6-2.0.9.0-rhel8.5-x86_64
MLNX_OFED_LINUX-5.6-2.0.9.0-rhel8.5-x86_64.tgz

[root@np04-srv-021 dpdk]# ls
dpdk-22.03  dpdk-22.03.tar.gz

The error log with non root user is the following:

[rsipos@np04-srv-022 app]$ ./dpdk-testpmd
EAL: Detected CPU lcores: 48
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /run/user/57871/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:5e:00.0 (socket 0)
mlx5_common: Failed to create TIS using DevX
mlx5_net: Failed to TIS 0 for bonding device mlx5_0.
mlx5_net: TIS allocation failure
mlx5_net: probe of PCI device 0000:5e:00.0 aborted after encountering an error: Cannot allocate memory
mlx5_common: Failed to load driver mlx5_eth
EAL: Requested device 0000:5e:00.0 cannot be used