<img src="https://ezycourse.b-cdn.net/2587/cmi2mkc9713nu939ngcp203e7.png" alt="image" style="position: relative; width: 779px; height: auto; cursor: pointer;"><h1 id="c5640627-cac9-4481-ae84-5223f7f9a894" data-toc-id="c5640627-cac9-4481-ae84-5223f7f9a894" style="text-align: left; font-size: 32px; line-height: 1.2;">Goal</h1>This Tutorial is for setting up DPDK development environment on your Linux Machine ( Native or as a VM ) following which user can focus on DPDK-powered network application development locally on his machine.<hr><h1 id="b8429e97-f699-4d2d-960f-ab78f83a8549" data-toc-id="b8429e97-f699-4d2d-960f-ab78f83a8549" style="text-align: left; font-size: 32px; line-height: 1.2;">Pre-Requisites</h1>OS : Assume ubuntu 26.04 OR similar , Running as VM or Native.My Host : MAC ( ARM 64 ) (steps for x86_64 host should be similar )Ensure you can access the internet.Before we begin, install some basic day-to-day pkgs on your Ubuntu VM.<blockquote><code>1. screen. (sudo apt install screen )</code><code>2. ifconfig (sudo apt install -y net-tools)</code><code>3. sudo apt-get install make</code><code>4. sudo apt-get install openssh-server</code><code>5. sudo apt install udhcpc</code></blockquote><hr><h1 id="df864e54-31d2-47c9-804a-bdca8bae0137" data-toc-id="df864e54-31d2-47c9-804a-bdca8bae0137" style="text-align: left; font-size: 32px; line-height: 1.2;">Chapter 1 : Installaing DPDK</h1>1. Check Machine Architecturevm-dpdk@vm-dpdk:~$ uname -m<blockquote><code>aarch64 (or x86_64 for Non-MAC Users )</code></blockquote>2. Check Numa layoutvm-dpdk@vm-dpdk:~$ lscpu | grep -i numa<blockquote><code>NUMA node(s): 1</code><code>NUMA node0 CPU(s): 0-3</code></blockquote>It means, you have 1 Numa node, and 4 CPUs fabricated on Numa Node. We will writedpdk code considering how many NUMA nodes our machine has, and how many CPUs eachnuma node has. We will discuss during course about NUMA nodes.3. Detailed information about NUMA nodes.<blockquote><code>vm-dpdk@vm-dpdk:~$ numactl --hardware 2&gt;/dev/null</code><code>available: 1 nodes (0)</code><code>node 0 cpus: 0 1 2 3</code><code>node 0 size: 7396 MB</code><code>node 0 free: 6689 MB</code><code>node distances:</code><code>node 0</code><code>0: 10</code></blockquote>4. Ensure your Ubuntu machine ( or VM) has atleast 8GB of RAM.5. Install all dependencies :<blockquote><code>sudo apt update</code><code>sudo apt install -y \</code><code>build-essential \</code><code>gcc \</code><code>g++ \</code><code>git \</code><code>meson \</code><code>ninja-build \</code><code>pkg-config \</code><code>python3 \</code><code>python3-pyelftools \</code><code>python3-setuptools \</code><code>python3-wheel \</code><code>libnuma-dev \</code><code>libpcap-dev \</code><code>pciutils \</code><code>numactl \</code><code>iproute2</code></blockquote>6. Check version installed , mine is :<blockquote>v<code>m-dpdk@vm-dpdk:~$ ninja --version</code><code>1.13.2</code><code>vm-dpdk@vm-dpdk:~$ pkg-config --version</code><code>2.5.1</code><code>vm-dpdk@vm-dpdk:~$ meson --version</code><code>1.10.1</code></blockquote>7. Download DPDK<blockquote>c<code>d ~</code><code>git clone https://github.com/DPDK/dpdk.git</code><code>cd dpdk</code></blockquote>8. Configure and compile the build<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ meson setup build -Dexamples=all -Dbuildtype=debug -Dplatform=generic</code><code>vm-dpdk@vm-dpdk:~/dpdk$ ninja -C build</code></blockquote>9. Install<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ sudo meson install -C build</code><code>vm-dpdk@vm-dpdk:~/dpdk$ sudo ldconfig</code></blockquote>10. Verify<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ pkg-config --modversion libdpdk</code><code>26.11.0-rc1</code></blockquote><blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ pkg-config --cflags libdpdk</code><code>-I/usr/local/include -include rte_config.h -march=armv8-a+crc -moutline-atomics -I/usr/include/dbus-1.0 -I/usr/lib/aarch64-linux-gnu/dbus-1.0/include -I/usr/include/libnl3</code><code>vm-dpdk@vm-dpdk:~/dpdk$ pkg-config --libs libdpdk</code><code>-Wl,--as-needed -L/usr/local/lib/aarch64-linux-gnu -lrte_node -lrte_graph -lrte_pipeline -lrte_table -lrte_pdump -lrte_port -lrte_fib -lrte_pdcp -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_mldev -lrte_regexdev -lrte_rawdev -lrte_power -lrte_pcapng -lrte_member -lrte_lpm -lrte_latencystats -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_gpudev -lrte_dispatcher -lrte_eventdev -lrte_efd -lrte_dmadev -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bpf -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_pmu -lrte_telemetry -lrte_argparse -lrte_kvargs -lrte_log</code></blockquote>11. Set the PKG_CONFIG_PATH<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ find /usr/local -name libdpdk.pc 2&gt;/dev/null</code><code>/usr/local/lib/aarch64-linux-gnu/pkgconfig/libdpdk.pc</code></blockquote>Add below line in ~/.bashrc<blockquote><code>export PKG_CONFIG_PATH=/usr/local/lib/aarch64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/lib64/pkgconfig:$PKG_CONFIG_PATH</code></blockquote>12. Configure Huge Pages nowWe'll reserve 256 pages, which equals 512 MiB.12.1 First, inspect the kernel's huge-page support:<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ ls /sys/kernel/mm/hugepages/</code><code>hugepages-1048576kB hugepages-2048kB hugepages-32768kB hugepages-64kB</code></blockquote>12.2 Reserve the pages<blockquote><code>echo 256 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages</code></blockquote>verify :vm-dpdk@vm-dpdk:~/dpdk$ grep -E 'HugePages|Hugepagesize' /proc/meminfo<blockquote><code>AnonHugePages: 90112 kB</code><code>ShmemHugePages: 0 kB</code><code>FileHugePages: 24576 kB</code><code>HugePages_Total: 256 &lt;&lt;&lt;&lt;&lt;&lt;</code><code>HugePages_Free: 256 &lt;&lt;&lt;&lt;&lt;&lt;</code><code>HugePages_Rsvd: 0</code><code>HugePages_Surp: 0</code><code>Hugepagesize: 2048 kB &lt;&lt;&lt;&lt;&lt;&lt;</code></blockquote>12.3 Mount the huge page file system<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ sudo mkdir -p /dev/hugepages</code><code>vm-dpdk@vm-dpdk:~/dpdk$ mountpoint -q /dev/hugepages || sudo mount -t hugetlbfs nodev /dev/hugepages</code></blockquote>12.4 confirmvm-dpdk@vm-dpdk:~/dpdk$ findmnt /dev/hugepages<blockquote><code>TARGET SOURCE FSTYPE OPTIONS</code><code>/dev/hugepages hugetlbfs hugetlbfs rw,nosuid,nodev,relatime,pagesize=2M</code></blockquote>12.5 Make Huge page Configuration persistent12.5.1 Open file : vm-dpdk@vm-dpdk:~/dpdk$ sudo vim /etc/default/grubchange line :from : GRUB_CMDLINE_LINUX_DEFAULT="console=tty0"to : GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 hugepagesz=2M hugepages=256"If your existing line contains other parameters, don't remove them; just append hugepagesz=2M hugepages=256 inside the quotes.Now, update grub :vm-dpdk@vm-dpdk:~/dpdk$ sudo update-grub<blockquote><code>Sourcing file `/etc/default/grub'</code><code>Sourcing file `/etc/default/grub.d/kdump-tools.cfg'</code><code>Generating grub configuration file ...</code><code>Found linux image: /boot/vmlinuz-7.0.0-38-generic</code><code>Found initrd image: /boot/initrd.img-7.0.0-38-generic</code><code>Found linux image: /boot/vmlinuz-7.0.0-30-generic</code><code>Found initrd image: /boot/initrd.img-7.0.0-30-generic</code><code>Warning: os-prober will not be executed to detect other bootable partitions.</code><code>Systems on them will not be added to the GRUB boot configuration.</code><code>Check GRUB_DISABLE_OS_PROBER documentation entry.</code><code>Adding boot menu entry for UEFI Firmware Settings ...</code><code>done</code></blockquote>12.5.2 Configure persistent hugetlbfs mountingvm-dpdk@vm-dpdk:~/dpdk$ sudo mkdir -p /dev/hugepagesedit file :vm-dpdk@vm-dpdk:~/dpdk$ sudo vim /etc/fstabAdd below line at the bottom and save the file :<blockquote><code>nodev /dev/hugepages hugetlbfs defaults,pagesize=2M 0 0</code></blockquote>Mount it without rebooting as below :vm-dpdk@vm-dpdk:~/dpdk$ sudo mount /dev/hugepages<blockquote><code>mount: (hint) your fstab has been modified, but systemd still uses</code><code>the old version; use 'systemctl daemon-reload' to reload.</code></blockquote>vm-dpdk@vm-dpdk:~/dpdk$ systemctl daemon-reload<blockquote><code>==== AUTHENTICATING FOR org.freedesktop.systemd1.reload-daemon ====</code><code>Authentication is required to reload the systemd state.</code><code>Authenticating as: vm-dpdk</code><code>Password:</code><code>==== AUTHENTICATION COMPLETE ====</code></blockquote>vm-dpdk@vm-dpdk:~/dpdk$Finally, in ~/.bashrc file, add below line at the bottom.<blockquote><code>export HUGEPAGES_DIR=/dev/hugepages</code></blockquote>verify :vm-dpdk@vm-dpdk:~/dpdk$ findmnt /dev/hugepages<blockquote><code>TARGET SOURCE FSTYPE OPTIONS</code><code>/dev/hugepages hugetlbfs hugetlbfs rw,nosuid,nodev,relatime,pagesize=2M</code><code>/dev/hugepages nodev hugetlbfs rw,relatime,pagesize=2M</code></blockquote>13. Install uio_pci_generic driver<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk$ echo "uio_pci_generic" | sudo tee -a /etc/modules</code><code>vm-dpdk@vm-dpdk:~/dpdk$ sudo modprobe uio</code><code>vm-dpdk@vm-dpdk:~/dpdk$ sudo modprobe uio_pci_generic</code></blockquote>verify:vm-dpdk@vm-dpdk:~/dpdk$ lsmod | grep uio<blockquote><code>uio_pci_generic 12288 0</code><code>uio 32768 1 uio_pci_generic</code></blockquote>14. reboot your Ubuntu Machine15. Verify after rebootvm-dpdk@vm-dpdk:~$ grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo<blockquote><code>HugePages_Total: 256</code><code>HugePages_Free: 256</code><code>Hugepagesize: 2048 kB</code></blockquote>vm-dpdk@vm-dpdk:~$ findmnt /dev/hugepages<blockquote><code>TARGET SOURCE FSTYPE OPTIONS</code><code>/dev/hugepages nodev hugetlbfs rw,relatime,pagesize=2M</code></blockquote><hr><h1 id="93c23099-0ee0-4044-9437-d52168f082cc" data-toc-id="93c23099-0ee0-4044-9437-d52168f082cc" style="text-align: left; font-size: 32px; line-height: 1.2;">Chapter 2 : Network Interface Cards Binding with DPDK</h1>Adding Network Interface cards to your Linux MachineAs of now, my Ubuntu VM has two interfaces :vm-dpdk@vm-dpdk:~/dpdk$ ifconfig -a<code>ens160: flags=4163&lt;UP,BROADCAST,RUNNING,MULTICAST&gt; mtu 1500</code><code>inet 172.16.250.153 netmask 255.255.255.0 broadcast 172.16.250.255</code><code>. . .</code><code>lo: flags=73&lt;UP,LOOPBACK,RUNNING&gt; mtu 65536</code><code>inet 127.0.0.1 netmask 255.0.0.0</code><code>inet6 ::1 prefixlen 128 scopeid 0x10&lt;host&gt;</code><code>. . .</code>Where is eth1,eth2,eth3 ... etc which will be used for receiving and transmitting pkts? Lets create them ...1. shutdown the VM2. Add interface adapters to it, 3-4 is more than enough3. In VMWARE Fusion ( Mac Users but similar settings for VMWARE WS users ) -<ul><li>3.a one adapter as "Share with my MAC" for internet access</li><li>3.b One adapter as "Wifi"</li><li>3.c one adapter as "Private to my mac" - put your VM in same Pvt Network as your host machine</li><li>3.d Then create any other 3-4 custom network adapters in vmnetx where x=4,5,6 for example.</li></ul>4. Start your VM5. Check ifconfig -a, now it will show more NICs.6. Check which interface has IP address 192.168.1.x , use these IP address to ssh from host into VM7. check if your VM is connected to internet : ping 8.8.8.8 must succeed.So, now your VM network settings are setup.Now let's bind the NIC of your ubuntu VM with DPDK Drivers.8. Binding NICs to DPDK drivers.Run the command. <a target="_blank" href="http://dpdk-devbind.py">dpdk-devbind.py</a> is the Python tool provided by dpdk to list all NICs on the VM machine.8.1 Check status before Bindingvm-dpdk@vm-dpdk:~/dpdk/usertools$ <a target="_blank" href="http://dpdk-devbind.py">dpdk-devbind.py</a> --status<blockquote><code>Network devices using kernel driver</code><code>===================================</code><code>0000:02:00.0 '82574L Gigabit Network Connection 10d3' if=ens160 drv=e1000e unused=uio_pci_generic Active</code><code>0000:03:00.0 '82574L Gigabit Network Connection 10d3' if=ens161 drv=e1000e unused=uio_pci_generic</code><code>0000:0b:00.0 '82574L Gigabit Network Connection 10d3' if=ens193 drv=e1000e unused=uio_pci_generic</code><code>0000:13:00.0 '82574L Gigabit Network Connection 10d3' if=ens225 drv=e1000e unused=uio_pci_generic Active</code><code>0000:1a:00.0 '82574L Gigabit Network Connection 10d3' if=ens256 drv=e1000e unused=uio_pci_generic</code><code>0000:1b:00.0 '82574L Gigabit Network Connection 10d3' if=ens257 drv=e1000e unused=uio_pci_generic Active</code></blockquote>Here, we are interested in NICs which do not have any ip address configured on it ( confirm using ifconfig -a).Those NICs are ens161, ens193, ens256 in my case. We will bind these NICs with DPDK drivers.Command to bind these NICs with dpdk drivers is below. After execution of below commands, "ifconfig -a" would not list those NICs which are bind below.8.2 Binding with DPDK<blockquote><code>vm-dpdk@vm-dpdk:~/dpdk/usertools$ sudo ~/dpdk/usertools/dpdk-devbind.py --bind=uio_pci_generic 0000:03:00.0</code><code>vm-dpdk@vm-dpdk:~/dpdk/usertools$ sudo ~/dpdk/usertools/dpdk-devbind.py --bind=uio_pci_generic 0000:0b:00.0</code><code>vm-dpdk@vm-dpdk:~/dpdk/usertools$ sudo ~/dpdk/usertools/dpdk-devbind.py --bind=uio_pci_generic 0000:1a:00.0</code></blockquote>Run the same status command again as below.8.3 Check status After Bindingvm-dpdk@vm-dpdk:~/dpdk/usertools$ <a target="_blank" href="http://dpdk-devbind.py">dpdk-devbind.py</a> --status<blockquote><code>Network devices using DPDK-compatible driver</code><code>============================================</code><code>0000:03:00.0 '82574L Gigabit Network Connection 10d3' drv=uio_pci_generic unused=e1000e</code><code>0000:0b:00.0 '82574L Gigabit Network Connection 10d3' drv=uio_pci_generic unused=e1000e</code><code>0000:1a:00.0 '82574L Gigabit Network Connection 10d3' drv=uio_pci_generic unused=e1000e</code><code>Network devices using kernel driver</code><code>===================================</code><code>0000:02:00.0 '82574L Gigabit Network Connection 10d3' if=ens160 drv=e1000e unused=uio_pci_generic Active</code><code>0000:13:00.0 '82574L Gigabit Network Connection 10d3' if=ens225 drv=e1000e unused=uio_pci_generic Active</code><code>0000:1b:00.0 '82574L Gigabit Network Connection 10d3' if=ens257 drv=e1000e unused=uio_pci_generic Active</code></blockquote>Note : This command (<a target="_blank" href="http://dpdk-devbind.py">dpdk-devbind.py</a>) will not persist across reboots, so if you restart the VM, you need to run it again.<hr><h1 id="d8c01609-a93e-45ec-9455-1cf945912361" data-toc-id="d8c01609-a93e-45ec-9455-1cf945912361" style="text-align: left; font-size: 32px; line-height: 1.2;">Chapter 3. Building Test Topology</h1>Lets Create a linear Topology, in whic R2 is out DPDK enabled Node, where R1 and R3 are normal Linux machines to begin with.R1---------------------------R2--------------------------------R3Create clones of our Ub-26.4-Demo as R1 and R3 in VMWare fusion. Create them as LinkedClone.So, now we have three VMs, lets Connect them as per the topology.

This is quick tutorial about Setting up DPDK Development environment.

Setup DPDK Development Environment

Chapter 2 : Network Interface Cards Binding with DPDK

Learn how to grow and scale your online course business using our online course platform. EzyCourse blog is designed to teach you how to create and sell online courses.

Setup DPDK Development Environment

Setup DPDK Development Environment

Related Posts

Setting up DPDK-Dev Environment

GNS3 LAB on L3 VPN

Understanding BGP ( Border Gateway Protocol ) Configuration

Network Address Translation

BGP-EVPN VxLAN Lab - Part 3 - Inter VNI Routing