metta-systems / vesper

Single-address-space capability-based nanokernel

Vesper

About kernel

Vesper is a capability-based single-address-space nanokernel, it tries to remain small and secure. To achieve this, kernel functionality is extremely limited - it provides only address space isolation and IPC, after bootup kernel does not allocate any memory itself.

Exokernel's distinctive trait is that it provides mechanisms but not policies. Vesper tries to move as many policy decisions as possible to the library OS.

• Single-address-space is a mechanism for providing pointer transparency between processes. Sharing a buffer is nearly as simple as passing out its address. Even though single-address-space is not a basic requirement and may be seen as an imposed policy decision, it does provide mechanism for efficient data passing between protection domains. This is important for modern media-rich communications.

• IPC is a mechanism providing secure interaction between processes.

• Capabilities are a mechanism providing access rights control and universal authority delegation for OS objects.

• Interrupts come from hardware, usually in privileged mode and kernel is responsible for translating them into invocations of the device drivers' handlers.

Scheduling

Scheduling can be viewed as the process of multiplexing the CPU resource between computational tasks. The schedulable entity of an operating system often places constraints both on the scheduling algorithms which may be employed and the functionality provided to the application. The recent gain in popularity of multi-threaded programming due to languages such as Modula-3 [Nelson 91] has led many operating system designers to provide kernel-level thread support mechanisms [Accetta 86, Rozier 90]. The kernel therefore schedules threads rather than processes. Whilst this reduces the functionality required in applications and usually results in more efficient processor context-switches, the necessary thread scheduling policy decisions must also be migrated into the kernel. As pointed out in [Barham 96], this is highly undesirable.

The desire to move such decisions out of the kernel make interesting variants where actual scheduling is performed by the user-level domain scheduler upon an upcall from the kernel. TBD

Real Time

At the moment this is not a real-time kernel. It has a small number of potentially long-running kernel operations that are not preemptable (e.g., endpoint deletion and recycling, scheduling, frame and CNode initialisation). This may change in future versions.

Credits

Vesper has been influenced by the kernels in L4 family, notably seL4. Fawn and Nemesis provided inspiration for single-address-space and vertical integration of the applications.

Build instructions

MSRV: 1.61.0

We require cargo build --build-std feature (since 2020-07-15), compiler_builtins memory operations (since 2020-09-30) and const_fn_fn_ptr_basics feature (stable since Rust 1.61.0).

• Install tools: cargo install just cargo-make.
• Install qemu (at least version 4.1.1): brew install qemu.
• Optionally install OpenOCD with RTT patches.
• Install aarch64 gdb.

You can override invoked qemu, openocd and gdb by specifying full paths to them as env variables QEMU, OPENOCD and GDB, respectively.

You can override the name of mounted sdcard volume by specifying env variable VOLUME (it defaults to /Volumes/BOOT).

To build kernel and run it in QEMU emulator

just qemu

To build kernel for Raspberry Pi and copy it to the mounted SDCard

just device

On the device boot SD card you'll need a configuration file instructing RasPi to launch in 64-bit mode.

# config.txt on RPi3
arm_64bit=1

To run tests (tests require QEMU)

just test

To launch JTAG connected JLink probe

just ocd

To launch GDB and load kernel binary into it

just gdb

If you launch OpenOCD or QEMU before (for example, via just qemu-gdb), then gdb shall connect to it and allow you to load the kernel binary directly into memory. Type load in gdb to do that.

To see kernel symbols and their values

just nm

To see kernel disassembly

You need to have Hopper and hopperv4 cli helper installed.

just disasm

To see other available commands

just

It will list all just commands with their short descriptions.

Development flow

mainline, develop and released branches:

• feature branches are fluid development lines which may be discarded or merged into develop. Feature branches must be either merged or fast-forward merged ("landed") into develop. Squashing history during merge is not permitted - commits must be sorted and squashed as necessary before merge.
• develop is currenly developed changes. History is recommended to be immutable, however mutations are possible in some cases. Feature branches are merged into develop for stabilisation, then develop is merged into the mainline. Develop must be either merged or fast-forward merged ("landed") into mainline. Squashing history during merge is not permitted - commits must be sorted as necessary before merge. Avoid direct commits to develop. It is recommended to perform stabilisation fixes in a separate branch and then landing it into develop.
• mainline is for generally accepted changes. History is immutable, to record reversals make a revert commit with explanations why. Changes from develop are merged or landed into the mainline after stabilisation.
• released branch records points from mainline which were officially released. Mutations are not possible. Only non-fast-forward merges from mainline are acceptable. Releases are marked as annotated tags on this branch.

OSdev help

Based on Raspi3 tutorials by Andre Richter, which are in turn based on Raspi3 tutorials by bzt. Various references from OSDev Wiki and RaspberryPi.org manuals.

Debug with JTAG.

Badges

License scan

Individual files contain the following tag instead of the full license text.

SPDX-License-Identifier: BlueOak-1.0.0

This enables machine processing of license information based on the SPDX License Identifiers that are here available: http://spdx.org/licenses/

---

For more information please re-read.

vesper's Issues

Explore: Add interrupt controller (ARM GICv3)

See https://github.com/metta-systems/vesper/tree/explore/interrupt-control

• Add support for generic interrupts support on RPi3 and ARM GIC support on RPi4.

Dependabot couldn't fetch the branch/reference for cortex-a

Your dependency file specified a branch or reference for cortex-a, but Dependabot couldn't find it at the project's source. Has it been removed?

View the update logs.

Build `chainofcommand` on windows

Currently CI fails because termion cannot be build on windows.

This dependency can be made optional, I suspect.

Explore: Advanced MMU management

See https://github.com/metta-systems/vesper/tree/explore/mmu-and-allocation

• Implement full abstractions and API to manipulate MMU pages

Transfer documentation to appropriate places

Collect docs from doc/ folder in the old tree and spread it into blog, docs and notes.

Explore: Add DTB parsing support

See https://github.com/metta-systems/vesper/tree/explore/parse-dtb

• Parse DTB structures and properly detect necessary hardware
  • Interrupt controller
  • Memory size
  • Serial ports locations and types (??)

Evaluate Profile-Guided Optimization (PGO)

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here. The most interesting to Vesper results probably are PGO results on operating systems (Linux kernel and Windows): https://github.com/zamazan4ik/awesome-pgo#operating-systems

We need to perform PGO benchmarks on Vesper. And if it shows improvements - add a note about possible improvements in Vesper performance with PGO. Providing an easier way (e.g. a build option) to build scripts with PGO can be useful for the end-users too. I think this kind of optimization can be useful for the project since if PGO will be able to optimize somehow CPU workload of the operating system, we can potentially reduce requirements for the hardware and possibly decrease power consumption.

For the Rust projects, I suggest trying to start with cargo-pgo.

Here you can find different materials about PGO: benchmarks in different software, examples of how PGO is already integrated with different projects, PGO support in multiple compilers, and some PGO-related pieces of advice.

Map kernel and init_thread space and enable MMU

• map kernel through TTBR1 and full phys-mem access
• map init_thread through minimal page directory mappings and set of boot capabilities to the rest of physmem
• enable MMU

Explore: Use of GATs

See here and discussion here.

Enable higher-half kernel with TTBR1

Remap kernel immediately from EL2/EL3 before going to EL1, this is simplest.
Pre-fill mapping tables using ttt tool.

Reach parity with old unsorted branch

Steps:

• cargo build --build-std
• enable testing infrastructure
  • run qemu semihosting tests on gh actions
• init BSS
• switch from EL3/EL2 to EL1
• bump some obsolete dependencies
• add license check
• init MMU
• init memory types and map correctly
• init exception handling, handle traps
• add mailbox interface
  • add BumpAllocator
  • add DMA_ALLOCATOR
• add FrameBuffer interface
• add gpio
• run serial test with probe-rs and merge JTAG configuration before merging UART code - probe not functioning with cortex-a/bcm
• add UART serial
  • add miniUart
  • add PL011 Uart
• find out why core::fmt::num::imp::fmt_u64 crashes somewhere in the stack (only before MMU is enabled)
  • this is due to use of ldrh/strh on an unaligned storage (perf optimization in DEC_DIGITS_LUT use)
• add JTAG debug helpers (see if anything additional for probe-rs support is needed)
  • add SEGGER RTT interface using rtt-target crate -- postponed in feature/add-rtt branch, current situation with RTT support on the host side is pretty bad. - see #35 for follow up.
• add graphical Display
• add console with kernel command interface
• add Power management
• add working GitPod workspace (at least Edit, Build and Run tests in QEMU)
• transfer documentation to appropriate places -- see #36 for follow up.

Specify kernel API surface

• Low-level calls
• Kernel object types
• Capability-based calls

Add kernel syscall API and ABI

1. Add user-space thread which would call a syscall,
2. Parse syscall ABI arguments and call kernel API function,
3. Parse API function result and return appropriate response via ABI to user space.

Rename asm! to llvm_asm!

Recent rust-lang change rust-lang/rust#68404 deprecated asm! - rename it to llvm_asm! to avoid breakage.

MMU: enable aarch64 mmu with identity paging

• Must have working framebuffer (properly cached)
• Must clean up TLB etc
• Must use type-system limits in addressing different page levels (similar to phil-opp tutorial)

Add SEGGER RTT interface using `rtt-target` crate

See https://github.com/metta-systems/vesper/tree/explore/jlink-rtt

• postponed in feature/add-rtt branch, current situation with RTT support on the host side is pretty bad.

Add serial boot

• Add just boot target to build and boot through tftp or serial interface (see example in rust-raspberrypi-OS-tutorials' uart-chainloader)