heroickatora / static-alloc

Data structures like Rust `alloc` which reuse user allocated memory

About this repository

This repository contains a few interconnected Rust libraries that enable generic, dynamic data structures in environments that do not have a global allocator or where its usage should be avoided.

• alloc-traits: This base crate contains a trait which defines the contract of a local allocator, substituting or supplementing the global allocator.
• static-alloc: Defines a limited allocator that requires no runtime operating system interaction by allocating from a memory region embedded in the program binary, and allocated by the program loader.
• unsize: A utility trait that can replace the unstable internal trait combination Unsize/CoerceUnsize so that custom smart pointers such as boxes can be unsized.
• without-alloc: Defines data structures similar to those in alloc which do not require a global allocator. They instead work on a once-allocated memory region with interoperability with allocators that impl alloc-traits traits.

static-alloc's Issues

`static-alloc`: Implement a speedy version of `alloc_zeroed`

In a special case, when the original constructor was zeroed, then the function does not need to perform its own zeroing of memory. This is helped by the fact that there is no reset function.

This could be implemented by a simple bool attribute containing the information on whether all of its remaining memory is zeroed.

Naming of the allocator

Slab usually describes an allocator for fixed size chunks. The more fitting name is BumpPtr or something.

Eccentric `FixedVec` interfaces–A wish list

Not managing the allocation within FixedVec makes some rather outlandish interfaces necessary but also possible. Each should get forked into a separate issue when it gets a better draft or is being worked on. This is a quick sketch/wish list.

• impl<T> DoubleEndedIterator for Drain<'_, T> { } (closed in #19)

• fn FixedVec::extend_from_within(&mut self, idx: impl SliceIndex<[T]>)
fn FixedVec::fill_from_within(&mut self, f: impl FnOnce(&[T]) -> impl Iterator<Item=T>)
  The elements can be borrowed when extend the FixedVec as we never have to relocate them. That's quite awesome.

• Drain::skip(&mut self). Move an element to the start, then advance over it as if it were not part of the original drain(..) invocation. This would be a useful precursor to DrainFilter as well. An symmetric operation may put it in-front of the tail but needs some naming discussion (skip_back is confusing when double ended iterator should provide the same for the other side).

• struct SpliceVec<'a, T> {
      // A vector view on the contents but forgotten instead of dropped.
      vec: ManuallyDrop<FixedVec<'a, T>>,
      // Unused tail and capacity of the underlying vec.
      tail_len: usize,
      tail_capacity: usize, 
  }
  
  impl<T> FixedVec<'_, T> {
      splice_vec(&mut self) -> SpliceVec<'_, T>;
  }
  
  impl<'a, T> Deref for SpliceVec<'a, T> {
      type Target = FixedVec<'a, T>;
      // ..
  }

  A splice that works strictly in-place. The elements in the tail of the original FixedVec may be manually shifted backwards to provide more capacity to the splice. No more work is involved exactly if the vector within the Splice is empty when it is dropped.

Running out of memory

I have the following code that's using Bump.
As you can see there's only one single allocation, that allocation should be cleared right after that scope closes, but my alloc_error_handler is being called after running for a short period of time. Any idea why this could be happening? Am I missing something here?

#![no_std]
#![feature(alloc_error_handler)]

use alloc::string::ToString;
use static_alloc::Bump;

// initialize allocator
extern crate alloc;
#[global_allocator]
static A: Bump<[u8; 1 << 16]> = Bump::uninit();

// import function for printing
#[link(wasm_import_module = "arduino")]
extern {
    fn log_char(c: u8);
}
pub fn println(message: &str) {
    message.as_bytes().iter().for_each(|c| {
        unsafe { log_char(*c) };
    });

    unsafe { log_char('\n' as u8) };
}

// main function
#[no_mangle]
pub extern fn _start() {
    let mut i = u32::MAX;
    loop {
        i -= 1;
        // only allocation, should be cleaned up at end of loop
        let i_string = i.to_string();
        println(&i_string);
    }
}

// error handlers
#[panic_handler]
pub fn panic(_: &::core::panic::PanicInfo) -> ! {
    println("panic");
    core::arch::wasm32::unreachable();
}
#[alloc_error_handler]
pub fn oom(_: ::core::alloc::Layout) -> ! {
    println("out of memory");
    core::arch::wasm32::unreachable();
}

Full example here

Building slices incrementally

The design could support building slices without requiring all elements be prepared immediately. The only requirement is that no other allocation happens between the individual allocations of the elements. A specialized alloc which errors instead of retrying when the number of used bytes is different from the expected one would enable this.

Once all individual parts are allocated join them together into a single slice. And, since we control the underlying allocation, we avoid the remaining unsafety of usual slice concatenation, e.g. this one.

`static-alloc`: Implement an `unsync` bump allocator

The allocator is currently Send + Sync to be usable as the global allocator. For a local allocator however this is unecessary and wastes a few CPU cycles. Idea: provide an alternative, unsync version that uses a simple Cell<usize> instead of an AtomicUsize to keep track of allocation state.

`Extend` should panic when container is full

As decided by judgement of t-libs. Let's push for an alternative fill trait that captures the partial semantics instead.

FixedVec drain can implement DoubleEndedIterator

Unless FixedVec or its Drain iterator uses some weird tricks I am not aware of, it should be possible to iterate a FixedVec drain from the back as well as the front, much like Vec's Drain iterator, and with a similar strategy of reading an element from the end, then moving the end of the range of memory under the Drain's purview inwards.

Evaluate: A struct version of allocator

https://internals.rust-lang.org/t/why-bring-your-own-allocator-apis-are-not-more-popular-in-rust/14494/3?u=heroickatora

It seemed interesting to me because this has the distinct advantage of not necessitating a type parameter, which was also a 'revelation' when dealing with forward compatible ways to design markers for a type being transmutable from bytes (compare my blog post on the topic). This comes at the cost of extra layout size of all containers containing such an allocator.

Implement Debug for FixedVec?