ecaslab / cynq

PYNQ bindings for C and C++ to avoid requiring Python or Vitis to execute hardware acceleration.

CYNQ

Framework to develop FPGA applications in C++ with the easiness of PYNQ

Introduction

CYNQ is a C++ framework to implement FPGA-based accelerated applications with the same ease of use as PYNQ framework for Python. This allows users to implement their own applications with better performance than in Python and avoids the long processing times of coding applications with Vitis.

Dependencies

1. Meson >= 1.x
2. Python >= 3.8
3. GCC >= 9.x
4. XRT >= 2.13
5. Linux FPGA Manager

Index

• Foundations
  • Class Diagram
• Installation
• Getting Started
• About

How does CYNQ look like?

CYNQ is pretty similar to PYNQ, let's have a look.

PYNQ:

from pynq import allocate, Overlay

# Configure the FPGA
design = Overlay("design.bit")

# Extract the accelerator (IP Core) and DMA
dma = design.axi_dma_0
accel = design.multiplication_accel_0

# Allocate buffers
inbuf = allocate(shape=(input_elements,), dtype=np.uint16)
outbuf = allocate(shape=(output_elements,), dtype=np.uint16)

# Run
dma.sendchannel.transfer(inbuf)
accel.write(accel.register_map.CTRL.address, 0x81)
accel.write(accel.register_map.n_elements.address, input_elements)
dma.recvchannel.transfer(outbuf)
dma.recvchannel.wait()

# Dispose the buffers
del input_hw
del output_hw

With CYNQ for Xilinx Ultrascale+:

#include <cynq/cynq.hpp>

using namespace cynq;

// Configure the FPGA
auto kArch = HardwareArchitecture::UltraScale;
auto platform = IHardware::Create(kArch, "design.bit");

// Extract the accelerator (IP Core) and DMA
// Addresses are given by the design
const uint64_t accel_addr = 0xa000000;
const uint64_t dma_addr = 0xa0010000;
auto accel = platform->GetAccelerator(accel_addr);
auto dma = platform->GetDataMover(dma_addr);

// Allocate buffers and get the pointers
auto inbuf = mover->GetBuffer(input_size, accel->GetMemoryBank(0));
auto outbuf = mover->GetBuffer(output_size, accel->GetMemoryBank(1));
uint16_t* input_ptr = inbuf->HostAddress<uint16_t>().get();
uint16_t* output_ptr = outbuf->HostAddress<uint16_t>().get();

// Configure data - Bus: AXI4 Stream is handled by DMA
uint32_t num_elements = 4096;
const uint64_t addr_num_elements = 0x20;
accel->Attach(addr_num_elements, &num_elements, RegisterAccess::RO);

// Run
mover->Upload(in_mem, infbuf->Size(), 0, ExecutionType::Async);
accel->Start(StartMode::Continuous);
mover->Download(out_mem, outbuf->Size(), 0, ExecutionType::Sync);
accel->Stop();

// Dispose? We use RAII

With CYNQ for Alveo

#include <cynq/cynq.hpp>

using namespace cynq;

// Configure the FPGA
auto kArch = HardwareArchitecture::Alveo;
auto platform = IHardware::Create(kArch, "design.xclbin");

// Extract the accelerator (IP Core) and DMA
// Addresses are given by the design
auto accel = platform->GetAccelerator("vadd");
auto dma = platform->GetDataMover(0);

// Allocate buffers and get the pointers
auto inbuf = mover->GetBuffer(input_size, accel->GetMemoryBank(0));
auto outbuf = mover->GetBuffer(output_size, accel->GetMemoryBank(1));
uint16_t* input_ptr = inbuf->HostAddress<uint16_t>().get();
uint16_t* output_ptr = outbuf->HostAddress<uint16_t>().get();

// Configure the accel - memory mapped
const uint32_t num_elements = 4096;
accel->Attach(0, bo_0);
accel->Attach(1, bo_1);
accel->Attach(2, &num_elements, RegisterAccess::RO);

// Run
mover->Upload(in_mem, infbuf->Size(), 0, ExecutionType::Async);
accel->Start(StartMode::Once);
mover->Download(out_mem, outbuf->Size(), 0, ExecutionType::Sync);

// Dispose? We use RAII

Currently tested

So far, we have tested CYNQ on:

1. Xilinx KV26-based with Ubuntu 2022.04
2. Xilinx Alveo U250 (it should be compatible with other similar Alveo cards) - Shell: xilinx_u250_gen3x16_xdma_4_1_202210_1

Links & References:

• Docs: https://ecaslab.github.io/cynq
• Github: https://github.com/ECASLab/cynq

Cite Us:

@misc{cynq,
  author = {{León-Vega, Luis G.
              AND Ávila-Torres, Diego
              AND Castro-Godínez, Jorge
            }},
  title = {{CYNQ (v0.2)}},
  year  = {2024},
  url   = {https://github.com/ECASLab/cynq},
} 

cynq's Issues

Add attach support for MMIO accels

This intends to universalise the API between Alveo and MMIO

Improve the Hardware Creation and Accelerator API

Improve the Hardware Factory to require only the parameters needed.

The Accelerator API must guarantee the Read/Write/Attach to be uniform across platforms.

How to Compile A Program ?

Hi,
I'm trying to compile a program using your interface by running the GCC compiler on it and I keep getting this list of errors that i'm not really sure how to solve. What instructions and options do I need to configure with my compiler in order to get software properly with this frame work ?

test_cynq.cpp:(.text+0x28): undefined reference to `std::allocator<char>::allocator()'
/usr/bin/ld: test_cynq.cpp:(.text+0x5c): undefined reference to `cynq::IHardware::Create(cynq::HardwareArchitecture, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: test_cynq.cpp:(.text+0x64): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()'
/usr/bin/ld: test_cynq.cpp:(.text+0x6c): undefined reference to `std::allocator<char>::~allocator()'
/usr/bin/ld: test_cynq.cpp:(.text+0x110): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()'
/usr/bin/ld: test_cynq.cpp:(.text+0x120): undefined reference to `std::allocator<char>::~allocator()'
/usr/bin/ld: /tmp/cc3ZiyKX.o: in function `__static_initialization_and_destruction_0(int, int)':
test_cynq.cpp:(.text+0x198): undefined reference to `std::ios_base::Init::Init()'
/usr/bin/ld: test_cynq.cpp:(.text+0x1ac): undefined reference to `std::ios_base::Init::~Init()'
/usr/bin/ld: test_cynq.cpp:(.text+0x1b0): undefined reference to `std::ios_base::Init::~Init()'
/usr/bin/ld: /tmp/cc3ZiyKX.o: in function `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Alloc_hider::~_Alloc_hider()':
test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_Alloc_hiderD2Ev[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_Alloc_hiderD5Ev]+0x10): undefined reference to `std::allocator<char>::~allocator()'
/usr/bin/ld: /tmp/cc3ZiyKX.o: in function `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&)':
test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC2IS3_EEPKcRKS3_[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC5IS3_EEPKcRKS3_]+0x34): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_local_data()'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC2IS3_EEPKcRKS3_[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC5IS3_EEPKcRKS3_]+0x44): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Alloc_hider::_Alloc_hider(char*, std::allocator<char> const&)'
/usr/bin/ld: /tmp/cc3ZiyKX.o: in function `void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag)':
test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0x70): undefined reference to `std::__throw_logic_error(char const*)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xa0): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xac): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data(char*)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xbc): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_capacity(unsigned long)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xc4): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data() const'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xd0): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy_chars(char*, char const*, char const*)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0xe0): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_set_length(unsigned long)'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0x108): undefined reference to `__cxa_begin_catch'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0x110): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose()'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0x114): undefined reference to `__cxa_rethrow'
/usr/bin/ld: test_cynq.cpp:(.text._ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag[_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPKcEEvT_S8_St20forward_iterator_tag]+0x11c): undefined reference to `__cxa_end_catch'
/usr/bin/ld: /tmp/cc3ZiyKX.o:(.data.rel.local.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0'
collect2: error: ld returned 1 exit status

Memory synchronisation in XRT

There is no way in Ubuntu 22.04 to synchronise memory in a ordered fashion. The idea is to create a mechanism based on std::async when using eager mode.

Rename XRT Accelerator and XRT Data Mover to be MMIO and DMA respectively

This implies:

• Renaming the XRTAccelerator to be MMIOAccelerator
• Renaming the XRTDataMover to be DMADataMover
• Use the proper enumerators
• Return not implemented in case of trying another one.
• Handle everything at the HW Level
• Detect if the bitstream is provided or not in UltraScale hardware class

Personalize installation of XRT

In ETH: /opt/xilinx/xrt/include

In Kria: /usr/include/xrt

Add support for multiple devices

Add the Hardware to work with a device ID.

Alveo already supports it.

Queue support

Using queues allows the creation of execution graphs. This support requires implementation