nvbrent / nv_opof

Nvidia DPU Open Offload Daemon

Introduction

Application to communicate with firewall via grpc, and do HW offload on Nvidia BlueField DPU

Prerequisite

1. Create 2 VFs for each PF on x86 side
2. An option configure file can be added to /opt/mellanox/nv_opof/nv_opof.conf with json format. For example:

{
  "grpc_addr": "169.254.33.51",
  "grpc_port": 3443
}

Building

Dependency: libev (e.g, yum install -y libev libev-devel)

To build and install this daemon with all deps, run:

$ meson build
$ ninja -C build

Setup hugepages

$ mkdir -p /dev/hugepages
$ mountpoint -q /dev/hugepages || mount -t hugetlbfs nodev /dev/hugepages
$ echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

Usage

The controller has a systemd service running and a user interface tool to communicate with the service.

• Service: opof.service
• User Interface: opof
• Log: journalctl -u opof -f

Setup

User can run opof_setup to setup opof.

1. As default, the script does configures below:

• Configure OVS fallback bridges. So that, when daemon is failed or killed, all traffic will be forward to Host VM for processing.
• Configure gRPC interface(default enp3s0f0s0) IP address.
• Reserve hugepages, default 2048 * 2M.

2. User can specify the number of HugePages

$ opof_setup -p 2048

3. User can specify the interface used by grpc

$ opof_setup -g enp3s0f0s0

SystemD Service

If daemon is running on a DPU, most likely the service already started automatically. Run command below to check the status.

$ systemctl status opof.service

If daemon is not running, start controller by running command below. Make sure to check the status after command start.

$ systemctl start opof.service

To restart the daemon, run

$ systemctl restart opof.service

User Interface

Each command has its own help manual, e.g, opof query -h

1. Query a session

$ opof query -i <session_id>

2. Query daemon offload stats

$ opof stats