Using XDP/AF_XDP sockets
AF_XDP sockets introduce a fast-path for network packets from the device driver directly to user-space memory, bypassing the rest of the Linux network stack and facilitating high packet-rate and bandwidth processing of network packets—DNS queries (only via UDP) in our case.
Compiling NSD with XDP
As this feature is experimental and introduces new dependencies to NSD, it needs
to be compiled in with --enable-xdp.
The additional dependencies are: libxdp libbpf libcap clang llvm (and
gcc-multilib to fix a missing asm/types.h file, if you’re on Ubuntu).
For Debian/Ubuntu based systems, you would install the dependencies as follows:
sudo apt install -y libxdp-dev libbpf-dev libcap-dev clang llvm gcc-multilib
When using the git source repository, make sure to also initialize the submodules and auxilary files:
git clone https://github.com/NLnetLabs/nsd --branch features/af-xdp
cd nsd
git submodule update --init
autoreconf -fi
After installing the dependencies, you can build NSD:
./configure --enable-xdp
make -j4
sudo make install
Configuring XDP
The configuration options are described in nsd.conf(5).
By default, you can enable XDP for a single interface that supports it, with
the xdp-interface option:
server:
xdp-interface: enp1s0
In this configuration, NSD will load and (after stopping NSD) unload its bundled XDP program that redirects UDP traffic to port 53 directly to NSD user-space. You usually don’t have to do anything else. If it doesn’t work, check out the It’s not working section.
Note
Even though NSD uses libxdp, NSD skips the xdp-dispatcher. This is done so that NSD can unload the XDP program itself when finished without requiring the SYS_ADMIN capability (see xdp-project/xdp-tools#432 and xdp-project/xdp-tools#434 on GitHub). If you use multiple XDP programs on your system, please refer to Loading the bundled XDP program yourself, until we turn this into a config option.
Configuring XDP in special cases
If you have custom requirements for the use of XDP, e.g. because you want to integrate NSD into you existing XDP setup, you have two options:
Loading the bundled XDP program yourself
NSD includes two XDP programs (in /usr/share/nsd/, by default). You’ll need
the file named xdp-dns-redirect_kern_pinned.o. The two programs are
functionally identical, the _pinned variant just defines the pinning option
for the xsks_map.
When loading the program, make sure to instruct your xdp program loader of
choice to pin the map and adjust the file permissions so that NSD can modify the
pinned map. For example with the xdp-loader from xdp-tools:
sudo xdp-loader load -p /sys/fs/bpf <iface> /usr/share/nsd/xdp-dns-redirect_kern_pinned.o
sudo chown nsd /sys/fs/bpf/xsks_map
sudo chmod o+x /sys/fs/bpf
You’ll need to instruct NSD to not load any XDP programs, and inform NSD about
which XDP program you loaded and the bpffs path used, if that differs from the
default of /sys/fs/bpf. For example:
server:
xdp-interface: enp1s0
xdp-program-load: no
xdp-program-path: "/usr/share/nsd/xdp-dns-redirect_kern_pinned.o"
xdp-bpffs-path: "/sys/fs/bpf"
Writing/Extending your own custom XDP program
If you want to write or extend your own XDP program, you’ll need to define
a BPF_MAP_TYPE_XSKMAP BPF MAP with the name xsks_map:
struct {
__uint(type, BPF_MAP_TYPE_XSKMAP);
__type(key, __u32);
__type(value, __u32);
__uint(max_entries, 64); // max_entries must be >= number of network queues
__uint(pinning, LIBBPF_PIN_BY_NAME);
} xsks_map SEC(".maps");
Like with Loading the bundled XDP program yourself (see above), you’ll need to pin the map to a bpffs and configure NSD appropriately.
NSD (the XDP code path) internally checks whether incoming traffic is destined
for port 53. If you want to redirect UDP traffic incoming at a port other than
53, you’ll currently have to adjust DNS_PORT in xdp-server.c
accordingly.
It’s not working
Some drivers don’t support AF_XDP sockets fully. For those you can try out the
xdp-force-copy option:
server:
xdp-force-copy: yes