skyzh / core-os-riscv

🖥️ An xv6-like operating system on RISC-V with multi-core support. Documentation available online.

Hi, I'm Chi, currently working @neondatabase. I spent most of my life in Shanghai, and now I'm living in Pittsburgh.

• I'm maintaining RisingLight, an educational OLAP database system.
• I worked on BusTub as a teaching assistant for @cmu-db's intro to database course when I was in Carnegie Mellon University.
• I'm interested in exploring how to productively use the Rust programming language in data-intensive applications.
  • In mini-lsm, you will learn how to build a LSM-tree storage engine in Rust.
  • In type-exercise-in-rust, you will learn how to build a vectorized expression evaluation framework in database systems with Rust.
  • In write-you-a-vector-db, you will add vector extensions to a relational database system.

---

• FAQ
• About Me
• Curriculum Vitae
• Blog

core-os-riscv's Issues

Compile Error

warning: 1 warning emitted

Finished dev [unoptimized + debuginfo] target(s) in 0.01s

cd kernel && cargo xbuild --target=riscv64gc-unknown-none-elf
WARNING: There is no root package to read the cargo-xbuild config from.
Compiling riscv v0.5.6
Compiling bare-metal v0.2.4
Compiling core-os-riscv v0.1.0 (/home/mahdi/xv6_riscv_rust/core-os-riscv/kernel)
error[E0557]: feature has been removed
--> kernel/src/lib.rs:13:12
|
13 | #![feature(const_in_array_repeat_expressions)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ feature has been removed
|
= note: removed due to causing promotable bugs

warning: the feature const_generics is incomplete and may not be safe to use and/or cause compiler crashes
--> kernel/src/lib.rs:12:12
|
12 | #![feature(const_generics)]
| ^^^^^^^^^^^^^^
|
= note: #[warn(incomplete_features)] on by default
= note: see issue #44580 rust-lang/rust#44580 for more information

error: aborting due to previous error; 1 warning emitted

For more information about this error, try rustc --explain E0557.
error: could not compile core-os-riscv

To learn more, run the command again with --verbose.
make: *** [Makefile:43: target/riscv64gc-unknown-none-elf/debug/libkernel.a] Error 101

Security Issues

In this project, I neglect some of the security issues. They should be fixed after this project is close to completion.

• syscall: copy from and to user space
• syscall: overflow check

Re-implement process scheduling system

After investigating #8 , I found the issue is mainly due to my re-design of xv6 process scheduling system.

xv6's scheduling system

xv6 stores processes in a global array. Each element of this array holds a mutex of itself. The process just stays at its place, it won't be moved back and forth.

My implementation

There's also a process array called PROCS_POOL in core-os. However, in order to meet the ownership requirements in Rust, I made some changes to the scheduling system.

If a process is to be scheduled on hart, scheduler will swap process object in PROCS_POOL into CPU object. In this way, CPU has full ownership of the process object. That's how I address that ownership issue in early stage. Therefore, there's no need to add a mutex to every process as only current hart takes ownership of the process object.

However, I always challenges myself of this implementation. That's because that:

• Traps: exceptions and interrupts will always change control flow. After implementing timer-based scheduler (aka. preemptive scheduling), the kernel might be interrupted at any time. For example, when context is being switched. If a trap happens at this time, there'll be problems when checking whether there's a running process on CPU.
• Sleep Locks (its equivalent is conditional variable in pthread): As is mentioned in #2 , I proposed a global lock to indicate if there're any kernel thread in the process of being put back into pool.
• Frequent kernel panic in #8 : After 5 hours of debugging, I figured out that this issue is caused by traps happening during scheduling. If the hart is running scheduler thread and a timer interrupt happens after proc is swapped into CPU but scheduler hasn't called swtch yet, the kernel will assume that there's a process running on CPU, and tries calling scheduler thread. After a few failed attempts, I gave up on current scheduling system.

After all, I'll implement new scheduling system as described in xv6, and meanwhile adapt async-style kernel thread in #1 . This issue will be resolved in next big refactor and milestone of core-os. Now I just take a break and focus on my course projects.

Now I just check if scheduler thread is running with context being zero. As there's no locking mechanisms, kernel panic may occur in very low probability. It just works.

Screenshots

Reference Explained

• https://github.com/rust-embedded/rust-raspi3-OS-tutorials
• https://github.com/bztsrc/raspi3-tutorial/
• https://os.phil-opp.com/
• http://osblog.stephenmarz.com/
• https://github.com/mit-pdos/xv6-riscv/
• https://pdos.csail.mit.edu/6.828/2012/labs
• https://gist.github.com/cb372/5f6bf16ca0682541260ae52fc11ea3bb
• https://github.com/rcore-os/rCore

Proposal: Simple File System and Syscalls

I'll implement the following xv6 syscalls: open, close, read, write, dup.

A new trait File will be introduced. A file is one of a pipe, an on-disk file or a device. And each of them will have its own struct. File trait has two functions: read and write. I plan to implement the same prototype as read_exact in Rust std Read trait.

First of all Process will be redesigned. Process will own Arc of File objects. This eliminates the need of rc in xv6.

Also the filesystem will get a small modification by adding console file.

The init process now may have the same functionalities as it is in xv6. /init will open /console file and dup it as stdin, stdout and stderr. This completes the I/O and file system design.

Investigate LLVM Codegen: Atomic Variables and Atomic Instructions

From my previous experience of RISC-V programming with Rust, I found that Rust won't generate amoswap instruction for spin lock. That's why I wrote __sync_lock_release and __sync_lock_test_and_set in arch.rs. On that, spin crate will generate non-atomic instruction for implementing the lock, which may cause significant slowdown. This affects all Atomic type in Rust.

Proposal: Async-like way in handling user process

In xv6, code path for trapping into user-space and back is really confusing. Typically this is done with:

forkret() -> usertrap() -> making syscalls, etc. -> usertrapret()

I've encountered the issue of RAII. As we call usertrap in forkret, any object created in forkret won't be dropped as usertrap won't return.

Previously I thought this could been done by introducing a return_to function, in which RISC-V return address register ra is rewritten with the address of that function. For example,

fn forkret() -> NeverReturn {
    return_to(usertrap)
}

Therefore, object can be correctly dropped before jumping to usertrap. However, there're many issues with that.

After that, I propose to use a k_thread() function to represent full code path for trapping into and returning from user-space.

func k_thread() {
    // forkret contents
    loop {
        // usertrap contents
        await_into_userspace(); // A function calls into user space and returns on trap
        if scause == timer { yield(); }
        
        syscall();

        // usertrapret contents
    }
}

And the scheduler just schedules those k_threads, which solves the RAII issue.

Furthermore, I would like to have this issue solved with async-std crate, which supports no-std environment.

Investigate locking and slow fs issue

core-os may run in release mode, but will be stuck in debug mode. Even in release mode, I found that reading elf file from disk is very slow. I will investigate into this situation.

Proposal: Sleep Lock implementation

I found implementation of sleep lock in Rust is much harder than it is in C. The reasons are:

• Acquiring sleep lock requires unlocking another spinlock and enabling interrupt.
• Lost wake-up issue: if a process is not put back into process pool before waking up, this wake-up is lost. This is because I didn't implement the scheduler in a xv6 way.

Therefore, I would add some constraints in current implementation and introduce new structures.

• intr_on and intr_off will only be called in IntrLocker. The only way to turn on or off interrupt is to acquire an interrupt lock.
• New ProcPool design: There'll be four types of objects in process pool. None, Scheduled, Some, BeingSlept. Wake-ups must wait until BeingSlept objects in pool are replaced with Some by put_back_proc in scheduler.

In this way, we can solve the lost-wake up issue and implement a sleep lock in a Rust flavor.