lgarbarini / cores-swervolf

FuseSoC-based SoC for SweRV EH1

SweRVolf

SweRVolf is a FuseSoC-based SoC for the SweRV RISC-V core.

This can be used to run the RISC-V compliance tests, Zephyr OS or other software in simulators or on FPGA boards. Focus is on portability, extendability and ease of use; to allow SweRV users to quickly get software running, modify the SoC to their needs or port it to new target devices.

Structure

To ease portability, the SoC consists of a portable technology-agnostic core with target-specific wrappers. This chapter describes the functionality of the core and the technology-specific targets.

SweRVolf Core

The core of SweRVolf consists of the SweRV CPU with a boot ROM, AXI4 interconnect, UART, SPI, RISC-V timer and GPIO. The core doesn't include any RAM but instead exposes a memory bus that the target-specific wrapper will connect to an appropriate memory controller. Other external connections are clock, reset, UART, GPIO, SPI and DMI (Debug Module Interface).

SwerVolf Core

Memory map

Core	Address
RAM	0x00000000-0x07FFFFFF
Boot ROM	0x80000000-0x80000FFF
syscon	0x80001000-0x80001FFF
UART	0x80002000-0x80002FFF

RAM

The SweRVolf core does not contain a memory controller but allocates the first 128MiB of the address for RAM that can be used by a target application and exposes an AXI bus to the wrapper.

Boot ROM

The boot ROM contains a first-stage bootloader. After system reset, SweRV will start fetching its first instructions from this area.

To select a bootloader, set the bootrom_file parameter. See the Booting chapter for more information about available bootloaders.

System controller

The system controller contains common system functionality such as keeping register with the SoC version information, RAM initialization status and the RISC-V machine timer. Below is the memory map of the system controller

Address	Register	Description
0x00	version_rev	SweRVolf revision
0x01	version_minor	SweRVolf minor version
0x02	version_major	SweRVolf major version
0x03	version_misc	Bit 7 is set when SweRVolf was built from modified sources
0x04-0x07	version_sha	SHA hash of the build
0x08	sim_print	Outputs a character in simulation. No effect on hardware
0x09	sim_exit	Exits a simulation. No effect on hardware
0x0A	init_status	Bit 0 = RAM initialization complete. Bit 1 = RAM initialization reported errors
0x10	gpio	Writing to bit 0 clears or sets GPIO bit
0x20-0x27	mtime	mtime from RISC-V privilege spec
0x28-0x2f	mtimecmp	mtimecmp from RISC-V privilege spec
0x40	SPI_SPCR	Simple SPI Control register
0x48	SPI_SPSR	Simple SPI status register
0x50	SPI_SPDR	Simple SPI data register
0x58	SPI_SPER	Simple SPI extended register
0x60	SPI_SPSS	Simple SPI slave select register

UART

SweRVolf contains a ns16550-compatible UART

SweRVolf sim

SweRVolf sim is a simulation target that wraps the SweRVolf core in a testbench to be used by verilator or event-driven simulators such as QuestaSim. It can be used for full-system simulations that executes programs running on SweRV. It also supports connecting a debugger through OpenOCD and JTAG VPI. The Debugging chapter contains more information on how to connect a debugger.

SwerVolf Simulation target

The simulation target exposes a number of parameters for compile-time and run-time configuration. These parameters are all exposed as FuseSoC parameters. The most relevant parameters are:

• --jtag_vpi_enable : Enables the JTAG server which OpenOCD can connect to
• --ram_init_file : Loads a Verilog hex file to use as initial on-chip RAM contents
• --vcd : Enable VCD dumping

Memory files suitable for loading with --ram_init_file can be created from binary files with the sw/makehex.py script

SweRVolf Nexys

SweRVolf Nexys is a version of the SweRVolf SoC created for the Digilent Nexys A7 board. It uses the on-board 128MB DDR2 for RAM, has GPIO connected to LED, supports booting from SPI Flash and uses the microUSB port for UART and JTAG communication. The default bootloader for the SweRVolf Nexys target will attempt to load a program stored in SPI Flash by default.

SwerVolf Nexys A7 target

I/O

The active on-board I/O consists of a LED, a switch and the microUSB connector for UART, JTAG and power.

LED 0

LED 0 is controlled by the memory-mapped GPIO at address 0x80001010

Switch 0

Switch 0 selects whether to output serial communication from the SoC or from the embedded self-test program in the DDR2 controller.

micro USB

UART and JTAG communication is tunneled through the microUSB port on the board and will appear as /dev/ttyUSB0, /dev/ttyUSB1 or similar depending on OS configuration. A terminal emulator can be used to connect to the UART (e.g. by running screen /dev/ttyUSB0 115200) and OpenOCD can connect to the JTAG port to program the FPGA or connect the debug proxy. The debugging chapter goes into more detail on how to connect a debugger.

SPI Flash

An SPI controller is connected to the on-board SPI Flash. This can be used for storing data such as program to be loaded into memory during boot. The SPI uImage loader chapter goes into more detail on how to prepare, write and boot a program stored in SPI Flash

How to use

Prerequisites

Install verilator

Create a directory structure consisting of a workspace directory (from now on called $WORKSPACE) and a root directory for the SweRV SoC (from now on called $CORES_ROOT). All further commands will be run from $WORKSPACE unless otherwise stated. The structure will look like this

├──cores
└──workspace

1. Make sure you have FuseSoC installed or install it with pip install fusesoc
2. Initialize the FuseSoC base library with fusesoc init
3. From $CORES_ROOT, clone the SweRVolf repository git clone https://github.com/chipsalliance/Cores-SweRVolf
4. Add the cores directory as a FuseSoC core library fusesoc library add swervolf ../cores
5. Make sure you have verilator installed to run the simulation. Note This requires at least version 3.918. The version that is shipped with Ubuntu 18.04 will NOT work

Running the SoC

The SweRVolf SoC can be run in simulation or on hardware (Digilent Nexys A7 currently supported). In either case FuseSoC is used to launch the simulation or build and run the FPGA build. To select what to run, use the fusesoc run command with the --target parameter. To run in simulation use

fusesoc run --target=sim swervolf

This will load a small example program that prints a string and exits. If you want to rerun the program without rebuilding the simulation model, you can add the --run parameter

fusesoc run --target=sim --run swervolf

To build (and optionally program) an image for a Nexys A7 board, run

fusesoc run --target=nexys_a7 swervolf

All targets support different compile- and run-time options. To see all options for a target run

fusesoc run --target=$TARGET swervolf --help

To list all available targets, run

fusesoc core show swervolf

Run a precompiled example in simulation

In simulation, SweRVolf supports preloading an application to memory with the --ram_init_file parameter. SweRVolf comes bundled with some example applications in the sw directory.

To build the simulation model and run the bundled Zephyr Hello world example in a simulator. fusesoc run --target=sim swervolf --ram_init_file=../cores/Cores-SweRVolf/sw/zephyr_hello.vh.

After running the above command, the simulation model should be built and run. At the end it will output

Releasing reset
***** Booting Zephyr OS zephyr-v1.14.0 *****
Hello World! swervolf_nexys

At this point the simulation can be aborted with Ctrl-C.

Another example to run is the Zephyr philosophers demo.

fusesoc run --run --target=sim swervolf --ram_init_file=../cores/Cores-SweRVolf/sw/zephyr_philosophers.vh

• Note the --run option which will prevent rebuilding the simulator model

Run RISC-V compliance tests

1. Build the simulation model, if that hasn't already been done, with fusesoc run --target=sim --setup --build swervolf

2. Download the RISC-V compliance tests somewhere. git clone https://github.com/riscv/riscv-compliance in a sibling directory to the workspace and work root. Your directory structure should now look like this ├──cores ├──riscv-compliance └──workspace

3. Enter the riscv-compliance directory and run make TARGETDIR=$CORES_ROOT/Cores-SweRVolf/riscv-target/swerv riscv-target $RISCV_TARGET=swerv RISCV_DEVICE=rv32i RISCV_ISA=rv32imc TARGET_SIM=$WORKSPACE/build/swervolf_0/sim-verilator/Vswervolf_core_tb

Note: Other test suites can be run by replacing RISCV_ISA=rv32imc with rv32im or rv32i

Run on hardware

The SweRVolf SoC can be built for a Digilent Nexys A7 board with

fusesoc run --target=nexys_a7 swervolf

If the board is connected, it will automatically be programmed when the FPGA image has been built. It can also be programmed manually afterwards by running fusesoc run --target=nexys_a7 --run swervolf or running OpenOCD as described in the debugging chapter.

The default bootloader will just blink the LED and other programs are uploaded through the debug interface. The default bootloader can be replaced with the --bootrom_file parameter. Note that the boot ROM is not connected to the data port, so it can only execute instructions. Data can not be read or written to this segment. The below example will compile the memtest application and use that as boot ROM instead.

make -C ../cores/Cores-SweRVolf/sw memtest.vh
fusesoc run --target=nexys_a7 swervolf --bootrom_file=../cores/Cores-SweRVolf/sw/memtest.vh

Build Zephyr applications

1. Download and install Zephyr according to the official guidelines at https://www.zephyrproject.org/

2. Enter the directory of the application to build in the samples directory (e.g. basic/blinky for the Zephyr blinky example). From now on, the program to build and run will be called $APP

3. Build the code with mkdir build cd build cmake -GNinja -DBOARD=swervolf_nexys -DBOARD_ROOT=$CORES_ROOT/swervolf/zephyr -DSOC_ROOT=$CORES_ROOT/swervolf/zephyr -DDTS_ROOT=$CORES_ROOT/swervolf/zephyr .. ninja

4. There will now be a binary file in zephyr/zephyr.bin

5. Enter the FuseSoC workspace directory and convert the binary file into a suitable verilog hex file with python $CORES_ROOT/swervolf/sw/makehex.py $ZEPHYR_BASE/samples/$APP/build/zephyr/zephyr.bin > $APP.hex

6. The new hex file can now be embedded as a bootloader for a new FPGA build with

   fusesoc run --target=nexys_a7 swervolf --bootrom_file=$APP.hex

or in a simulation with

fusesoc run --target=sim swervolf --ram_init_file=$APP.hex

The SweRVolf demo application in sw/swervolf_zephyr_demo is also a Zephyr program and can be built in the same way

Debugging

SweRVolf supports debugging both on hardware and in simulation. There are different procedures on how to connect the debugger, but once connected, the same commands can be used (although it's a lot slower in simulations).

Prerequisites

Install the RISC-V-specific version of OpenOCD

git clone https://github.com/riscv/riscv-openocd
cd riscv-openocd
./bootstrap
./configure --enable-jtag_vpi --enable-ftdi
make
sudo make install

Connecting debugger to simulation

When a SweRVolf simulation is launched with the --jtag_vpi_enable, it will start a JTAG server waiting for a client to connect and send JTAG commands.

fusesoc run --target=sim swervolf --jtag_vpi_enable

After compilation, the simulation should now say

Listening on port 5555

This means that it's ready to accept a JTAG client.

Open a new terminal, navigate to the workspace directory and run openocd -f $CORES_ROOT/Cores-SweRVolf/data/swervolf_sim.cfg to connect OpenOCD to the simulation instance. If successful, OpenOCD should output

Info : only one transport option; autoselect 'jtag'
Info : Set server port to 5555
Info : Set server address to 127.0.0.1
Info : Connection to 127.0.0.1 : 5555 succeed
Info : This adapter doesn't support configurable speed
Info : JTAG tap: riscv.cpu tap/device found: 0x00000001 (mfg: 0x000 (<invalid>), part: 0x0000, ver: 0x0)
Info : datacount=2 progbufsize=0
Warn : We won't be able to execute fence instructions on this target. Memory may not always appear consistent. (progbufsize=0, impebreak=0)
Info : Examined RISC-V core; found 1 harts
Info :  hart 0: XLEN=32, misa=0x40001104
Info : Listening on port 3333 for gdb connections
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections

and the simulation should report

Waiting for client connection...ok
Preloading TOP.swervolf_core_tb.swervolf.bootrom.ram from jumptoram.vh
Releasing reset

Open a third terminal and connect to the debug session through OpenOCD with telnet localhost 4444. From this terminal, it is now possible to view and control the state of of the CPU and memory. Try this by running mwb 0x80001010 1. This will write to the GPIO register. To verify that it worked, there should now be a message from the simulation instance saying gpio0 is on. By writing 0 to the same register (mwb 0x80001010 0), the gpio will be turned off.

Connecting debugger to Nexys A7

SweRVolf can be debugged using the same USB cable that is used for programming the FPGA, communicating over UART and powering the board. There is however one restriction. If the Vivado programmer has been used, it will have exclusive access to the JTAG channel. For that reason it is recommended to avoid using the Vivado programming tool and instead use OpenOCD for programming the FPGA as well. Unplugging and plugging the USB cable back will make Vivado lose the grip on the JTAG port.

Programming the board with OpenOCD can be performed by running (from $WORKSPACE)

openocd -f ../cores/Cores-SweRVolf/data/swervolf_nexys_program.cfg

To change the default FPGA image to load, add -c "set BITFILE /path/to/bitfile" as the first argument to openocd.

If everything goes as expected, this should output

Info : ftdi: if you experience problems at higher adapter clocks, try the command "ftdi_tdo_sample_edge falling"
Info : clock speed 10000 kHz
Info : JTAG tap: xc7.tap tap/device found: 0x13631093 (mfg: 0x049 (Xilinx), part: 0x3631, ver: 0x1)
Warn : gdb services need one or more targets defined
loaded file build/swervolf_0/nexys_a7-vivado/swervolf_0.bit to pld device 0 in 3s 201521us
shutdown command invoked

OpenOCD can now be connected to SweRVolf by running

openocd -f ../cores/Cores-SweRVolf/data/swervolf_nexys_debug.cfg

This should output

Info : ftdi: if you experience problems at higher adapter clocks, try the command "ftdi_tdo_sample_edge falling"
Info : clock speed 10000 kHz
Info : JTAG tap: riscv.cpu tap/device found: 0x13631093 (mfg: 0x049 (Xilinx), part: 0x3631, ver: 0x1)
Info : datacount=2 progbufsize=0
Warn : We won't be able to execute fence instructions on this target. Memory may not always appear consistent. (progbufsize=0, impebreak=0)
Info : Examined RISC-V core; found 1 harts
Info :  hart 0: XLEN=32, misa=0x40001104
Info : Listening on port 3333 for gdb connections
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections

Open a third terminal and connect to the debug session through OpenOCD with telnet localhost 4444. From this terminal, it is now possible to view and control the state of of the CPU and memory. Try this by running mwb 0x80001010 1. This will write to the GPIO register. To verify that it worked, LED0 should light up. By writing 0 to the same register (mwb 0x80001010 0), the LED will be turned off.

Loading programs with OpenOCD

OpenOCD support loading ELF program files by running load_image /path/to/file.elf. Remember that the path is relative to the directory from where OpenOCD was launched.

After the program has been loaded, set the program counter to address zero with reg pc 0 and run resume to start the program.

Booting

SweRVolf is set up by default to read its initial instructions from address 0x80000000 which point to the on-chip boot ROM. Several first-stage bootloaders are provided for different applications

Jump to RAM

For simulations, the most common option is to load a program into the on-chip RAM and start executing from there. The default bootloader in such cases is a single instruction that jumps to address 0x0 and continues execution from there.

SPI uImage loader

For most applications on real hardware it is preferred to store them in an on-board SPI Flash memory. The SPI uImage loader can read an image in the u-boot uImage format, copy it to RAM and start executing. This process requires creating a suitable image, writing it to Flash and set up the SPI uImage loader to read from the correct address in Flash.

Create a flash image

The mkimage tool available from u-boot is used to prepare an image to be written to Flash. mkimage expects a .bin file, which has been created with iscv64-unknown-elf-objcopy -O binary. Given a $IMAGE.bin we can now create $IMAGE.ub with the following command:

mkimage -A riscv -C none -T standalone -a 0x0 -e 0x0 -n '' -d $IMAGE.bin $IMAGE.ub

Refer to the uimage manual for a description of each parameter. There arae also Makefile targets in sw/Makefile that can be used as reference.

Writing SPI Flash

Simulation

In order to test the SPI image loading mechanism in simulation, a specific FuseSoC target, spi_tb is available. If no run-time parameters are supplied it will load a prebuilt image containing the hello program (source available in sw/hello.S) from Flash, execute it and exit. This testbench will not work in Verilator as it uses a non synthesizable model of the SPI Flash. The default simulator is instead ModelSim. Other simulators can be used by adding the --tool=$TOOL argument to the command-line.

fusesoc run --target=spi_tb swervolf

The simulated Flash contents can be changed at compile-time with the --flash_init_file parameter. The model expects a uImage in verilog hex format. Such files can be created by running

objcopy -I binary -O verilog $IMAGE.ub $IMAGE.hex

Nexys A7

For Nexys A7, OpenOCD is used to write to Flash. As the connection to the SPI Flash goes through the FPGA, this consists of a two-stage process where a proxy FPGA image is first written, which will handle communication between OpenOCD and the SPI Flash

1. Obtain the proxy FPGA image from here and place it in $WORKSPACE
2. Run openocd -c "set BINFILE $IMAGE" -f ../cores/Cores-SweRVolf/data/swervolf_nexys_write_flash.cfg, where $IMAGE is the path to the uImage file that should be written to Flash

Set up SPI uImage loader

The final step is to prepare the bootloader for SweRVolf which will be responsible for reading the image from Flash, copy it to RAM and execute it. This process is the same for both simulation and hardware targets. Note that both the spi_tb target and nexys_a7 target will have this as the default boot loader so in most cases nothing else needs to be done. There are however a couple of defines in sw/spi_uimage_loader.S that might need to be adjusted if the SPI controller is mapped to another base address or if the image is not stored at address 0 in the Flash.