demikernel / dpdk-rs

Rust Bindings for DPDK

Demikernel

Demikernel is a library operating system (LibOS) architecture designed for use with kernel-bypass I/O devices. This architecture offers a uniform system call API across kernel-bypass technologies (e.g., RDMA, DPDK) and OS functionality (e.g., a user-level networking stack for DPDK).

To read more about the motivation behind the Demikernel, check out this blog post.

To get details about the system, read our paper in SOSP '21.

To read more about Demikernel check out https://aka.ms/demikernel.

Codename for LibOSes

• catloop - TCP Socket Loopback LibOS
• catmem - Shared Memory LibOS
• catnap - Linux Sockets LibOS
• catnip - DPDK LibOS
• catpowder - Linux Raw Sockets

Documentation

• For instructions on development environment setup, see doc/setup.md.
• For instructions on building, see doc/building.md.
• For instructions on testing and running, doc/testing.md.
• For instructions for running on CloudLab, see doc/cloudlab.md.
• For documentation on the API, see documents in man.
• For instructions on how to contribute to this project, see CONTRIBUTING.

Usage Statement

This project is a prototype. As such, we provide no guarantees that it will work and you are assuming any risks with using the code. We welcome comments and feedback. Please send any questions or comments to one of the following maintainers of the project:

• Irene Zhang - [email protected]
• Pedro Henrique Penna - [email protected]

By sending feedback, you are consenting that it may be used in the further development of this project.

Trademark Notice

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.

dpdk-rs's Issues

[build] Do no unwrap build.rs's `library_location` if none is avaliable

If dpdk is installed globally, there will be no -L location when pkg-config queries the linker flags:

> pkg-config --libs libdpdk
-lrte_telemetry -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_rcu -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_ring -lrte_eal -lrte_kvargs -lbsd

So building dpdk-rs will fail:

error: failed to run custom build command for `dpdk-rs v0.2.0 (https://github.com/demikernel/dpdk-rs?rev=5526fb9751cb973b3a3b3f69c57c8f1d0c4df7c1#5526fb97)`

Caused by:
  process didn't exit successfully: `/home/omarsa/echo/src/rust/target/debug/build/dpdk-rs-82cf4d94875a8c66/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH

  --- stderr
  thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/omarsa/.cargo/git/checkouts/dpdk-rs-ff8d275db0957e26/5526fb9/build.rs:62:26
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

This happens because build.rs assumes there will always be at least one -L flag:

    // Step 1: Now that we've compiled and installed DPDK, point cargo to the libraries.
    println!(
        "cargo:rustc-link-search=native={}",
        library_location.unwrap()
    );

So we unwrap a None.

The solution is to only set cargo:rustc-link-search=native if library_location is not None. Since dpdk is installed globally, the linker will still find the library even without specifying this flag. Like so:

    if let Some(location) = library_location {
       println!(
        "cargo:rustc-link-search=native={}",
	location
       );
    }

I can make a PR for this issue.

[libdpdk] Cannot Use with Non-Mellanox NIC

I try to use this wrapper with non-Mellanox NIC but it doesn't work.

I ran the following command (EAL args are for my machine):

./target/debug/init -c3 -n4

and got the following output:

EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: No legacy callbacks, legacy socket not created
thread 'main' panicked at 'assertion failed: nb_ports > 0', src/bin/init.rs:24:9

Apparently, it does not detect any NIC device. On my machine, I have properly configured the NIC for DPDK (tested using my old/working DPDK application).

I'm using the default feature (mlx4) because load_mlx_driver does not do anything for mlx4.

I also tried linking statically (identical link targets to my old/working DPDK application written in C), but it does not help.

Any advice on how to port this to non-Mellanox NIC would be really helpful.

[build] Auto-Generated Tests Dereference Null Pointer

Description

Unit tests that are automatically generated by bindgen dereference a null pointer, thus causing compilation warnings to be emitted.

Dereferencing null pointers has undefined behavior (deref_nullptr)

How to Reproduce

1. Build
2. See warnings

Expected Behavior

• No dependency on undefined behavior.
• No warnings emitted.

[build] Auto-Generated Bindings Use Improper C Types

Description

Auto-generated bindings use improper_ctypes.

How to Reproduce

1. Build
2. See warnings

Expected Behavior

• No dependency on undefined behavior.
• No warnings emitted.

[build] Auto-Generated Tests References Packed Fields

Description

Unit tests that are automatically generated by bindgen access packed fields, thus causing compilation warnings to be emitted.

Referencing packed fields has undefined behavior (unaligned_references)

How to Reproduce

1. Drop the #![allow(unaligned_references) annotation
2. Build
3. Run test
4. See warnings

Expected Behavior

• No dependency on undefined behavior.
• No warnings emitted.

[build] Compilation Error: "packed type cannot transitively contain a `#[repr(align)]` type"

I am trying to build dpdk-rs. It currently fails with the following error:

error[E0588]: packed type cannot transitively contain a `#[repr(align)]` type
     --> /home/gatowololo/Research/Capybara/dpdk-rs/target/debug/build/dpdk-rs-94b0d4307bb6697e/out/bindings.rs:33985:1
      |
33985 | / pub struct rte_l2tpv2_combined_msg_hdr {
33986 | |     #[doc = "< common header"]
33987 | |     pub common: rte_l2tpv2_common_hdr,
33988 | |     pub __bindgen_anon_1: rte_l2tpv2_combined_msg_hdr__bindgen_ty_1,
33989 | | }
      | |_^
      |
note: `rte_l2tpv2_common_hdr__bindgen_ty_1__bindgen_ty_1` has a `#[repr(align)]` attribute
     --> /home/gatowololo/Research/Capybara/dpdk-rs/target/debug/build/dpdk-rs-94b0d4307bb6697e/out/bindings.rs:32913:1
      |
32913 | / pub struct rte_l2tpv2_common_hdr__bindgen_ty_1__bindgen_ty_1 {
32914 | |     pub _bitfield_align_1: [u8; 0],
32915 | |     pub _bitfield_1: __BindgenBitfieldUnit<[u8; 2usize]>,
32916 | | }
      | |_^
note: `rte_l2tpv2_combined_msg_hdr` contains a field of type `rte_l2tpv2_common_hdr`
     --> /home/gatowololo/Research/Capybara/dpdk-rs/target/debug/build/dpdk-rs-94b0d4307bb6697e/out/bindings.rs:33987:9
      |
33987 |     pub common: rte_l2tpv2_common_hdr,
      |         ^^^^^^
note: ...which contains a field of type `rte_l2tpv2_common_hdr__bindgen_ty_1`
     --> /home/gatowololo/Research/Capybara/dpdk-rs/target/debug/build/dpdk-rs-94b0d4307bb6697e/out/bindings.rs:32901:9
      |
32901 |     pub __bindgen_anon_1: rte_l2tpv2_common_hdr__bindgen_ty_1,
      |         ^^^^^^^^^^^^^^^^
note: ...which contains a field of type `rte_l2tpv2_common_hdr__bindgen_ty_1__bindgen_ty_1`
     --> /home/gatowololo/Research/Capybara/dpdk-rs/target/debug/build/dpdk-rs-94b0d4307bb6697e/out/bindings.rs:32908:9
      |
32908 |     pub __bindgen_anon_1: rte_l2tpv2_common_hdr__bindgen_ty_1__bindgen_ty_1,

Any thoughts on how to fix this?

This error happens with the latest version of cargo and also the rust-toolchain version of cargo:

> cargo --version
cargo 1.63.0-nightly (39ad1039d 2022-05-25)

This error happens both on the master and dev branches. And building with either mlx4 or mlx5.

is there a verion match required? which version of DPDK is used?

I use DPDK v22.07, and current master version of dpdk-rs.
I compiled this repo as the guide. but there is some errors about constant and structure member, which seems changed between DPDK versions.
So, there question is: is there a verion match required? which version of DPDK is used?

Thanks.

Part of the cargo build output as following:
error[E0425]: cannot find value rte_eth_rx_mq_mode_ETH_MQ_RX_RSS in this scope
--> src/bin/init.rs:81:32
|
81 | port_conf.rxmode.mq_mode = rte_eth_rx_mq_mode_ETH_MQ_RX_RSS;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: rte_eth_rx_mq_mode_RTE_ETH_MQ_RX_RSS
|
::: /home/can/workspace/dpdk-rs/target/debug/build/dpdk-rs-b510e6cbc4fa5b0a/out/bindings.rs:11075:1
|
11075 | pub const rte_eth_rx_mq_mode_RTE_ETH_MQ_RX_RSS: rte_eth_rx_mq_mode = 1;
| ----------------------------------------------------------------------- similarly named constant rte_eth_rx_mq_mode_RTE_ETH_MQ_RX_RSS defined here

error[E0425]: cannot find value ETH_RSS_IP in this scope
--> src/bin/init.rs:82:45
|
82 | port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP as u64 | dev_info.flow_type_rss_offloads;
| ^^^^^^^^^^ not found in this scope

error[E0425]: cannot find value rte_eth_tx_mq_mode_ETH_MQ_TX_NONE in this scope
--> src/bin/init.rs:83:32
|
83 | port_conf.txmode.mq_mode = rte_eth_tx_mq_mode_ETH_MQ_TX_NONE;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: rte_eth_tx_mq_mode_RTE_ETH_MQ_TX_NONE
|
::: /home/can/workspace/dpdk-rs/target/debug/build/dpdk-rs-b510e6cbc4fa5b0a/out/bindings.rs:11083:1
|
11083 | pub const rte_eth_tx_mq_mode_RTE_ETH_MQ_TX_NONE: rte_eth_tx_mq_mode = 0;
| ------------------------------------------------------------------------ similarly named constant rte_eth_tx_mq_mode_RTE_ETH_MQ_TX_NONE defined here

error[E0425]: cannot find value ETH_LINK_UP in this scope
--> src/bin/init.rs:129:45
|
129 | if link.link_status() as u32 == ETH_LINK_UP {
| ^^^^^^^^^^^ not found in this scope

error[E0425]: cannot find value ETH_LINK_FULL_DUPLEX in this scope
--> src/bin/init.rs:130:62
|
130 | let duplex = if link.link_duplex() as u32 == ETH_LINK_FULL_DUPLEX {
| ^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: RTE_ETH_LINK_FULL_DUPLEX
|
::: /home/can/workspace/dpdk-rs/target/debug/build/dpdk-rs-b510e6cbc4fa5b0a/out/bindings.rs:1105:1
|
1105 | pub const RTE_ETH_LINK_FULL_DUPLEX: u32 = 1;
| -------------------------------------------- similarly named constant RTE_ETH_LINK_FULL_DUPLEX defined here

error[E0609]: no field max_rx_pkt_len on type rte_eth_rxmode
--> src/bin/init.rs:80:22
|
80 | port_conf.rxmode.max_rx_pkt_len = RTE_ETHER_MAX_LEN;
| ^^^^^^^^^^^^^^ unknown field
|
= note: available fields are: mq_mode, mtu, max_lro_pkt_size, split_hdr_size, offloads ... and 2 others