lpghatguy / thunderdome

Arena type inspired by generational-arena

Thunderdome

Thunderdome is a gladitorial generational arena inspired by generational-arena, slotmap, and slab. It provides constant time insertion, lookup, and removal via small (8 byte) keys returned from Arena.

Thunderdome's key type, Index, is still 8 bytes when put inside of an Option<T> thanks to Rust's NonZero* types.

Basic Examples

let mut arena = Arena::new();

let foo = arena.insert("Foo");
let bar = arena.insert("Bar");

assert_eq!(arena[foo], "Foo");
assert_eq!(arena[bar], "Bar");

arena[bar] = "Replaced";
assert_eq!(arena[bar], "Replaced");

let foo_value = arena.remove(foo);
assert_eq!(foo_value, Some("Foo"));

// The slot previously used by foo will be reused for baz
let baz = arena.insert("Baz");
assert_eq!(arena[baz], "Baz");

// foo is no longer a valid key
assert_eq!(arena.get(foo), None);

Comparison With Similar Crates

Feature	Thunderdome	generational-arena	slotmap	slab
Generational Indices¹	Yes	Yes	Yes	No
size_of::<Index>()	8	16	8	8
size_of::<Option<Index>>()	8	24	8	16
Max Elements	2³²	2⁶⁴	2³²	2⁶⁴
Non-Copy Values	Yes	Yes	Yes	Yes
no_std Support	Yes	Yes	Yes	No
Serde Support	No	Yes	Yes	No

• Sizes calculated on rustc 1.44.0-x86_64-pc-windows-msvc
• See the Thunderdome comparison Cargo.toml for versions of each library tested.

1. Generational indices help solve the ABA Problem, which can cause dangling keys to mistakenly access newly-inserted data.

Minimum Supported Rust Version (MSRV)

Thunderdome supports Rust 1.47.0 and newer. Until Thunderdome reaches 1.0, changes to the MSRV will require major version bumps. After 1.0, MSRV changes will only require minor version bumps, but will need significant justification.

Crate Features

• std (default): Use the standard library. Disable to make this crate no-std compatible.

License

Licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

thunderdome's Issues

`Index::from_bits` should return `Option<Index>`

Currently, when given an invalid bit pattern, Index::from_bits will panic. This can happen with just Index::from_bits(0), for example.

In order to allow the Index type to be constructed from untrusted inputs safely, we should update Index::from_bits to return an Option<Index> and never panic.

`miri` test complains about `Arena::get2_mut`

miri test result:

$ cargo +nightly miri test
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests (target/miri/x86_64-apple-darwin/debug/deps/thunderdome-fba4034cefd759f7)
error: Undefined Behavior: trying to reborrow for Unique at alloc84249+0x8, but parent tag <217107> does not have an appropriate item in the borrow stack
   --> src/arena.rs:254:48
    |
254 |         let item1 = unsafe { item1_ptr.map(|x| &mut *x) };
    |                                                ^^^^^^^ trying to reborrow for Unique at alloc84249+0x8, but parent tag <217107> does not have an appropriate item in the borrow stack
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
            
    = note: inside closure at src/arena.rs:254:48
    = note: inside `std::option::Option::<*mut i32>::map::<&mut i32, [closure@src/arena.rs:254:44: 254:55]>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/option.rs:836:29
note: inside `arena::Arena::<i32>::get2_mut` at src/arena.rs:254:30
   --> src/arena.rs:254:30
    |
254 |         let item1 = unsafe { item1_ptr.map(|x| &mut *x) };
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arena::test::get2_mut` at src/arena.rs:600:40
   --> src/arena.rs:600:40
    |
600 |         let (foo_handle, bar_handle) = arena.get2_mut(foo, bar);
    |                                        ^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at src/arena.rs:595:5
   --> src/arena.rs:595:5
    |
594 |       #[test]
    |       ------- in this procedural macro expansion
595 | /     fn get2_mut() {
596 | |         let mut arena = Arena::new();
597 | |         let foo = arena.insert(100);
598 | |         let bar = arena.insert(500);
...   |
607 | |         assert_eq!(arena.get(bar), Some(&505));
608 | |     }
    | |_____^
    = note: inside `<[closure@src/arena.rs:595:5: 608:6] as std::ops::FnOnce<()>>::call_once - shim` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:578:5
    = note: inside closure at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:569:30
    = note: inside `<[closure@test::run_test::{closure#2}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1572:9
    = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
    = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:401:40
    = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:365:19
    = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:129:14
    = note: inside `test::run_test_in_process` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:601:18
    = note: inside closure at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:493:39
    = note: inside `test::run_test::run_test_inner` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:531:13
    = note: inside `test::run_test` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:565:28
    = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:306:17
    = note: inside `test::run_tests_console` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/console.rs:290:5
    = note: inside `test::test_main` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:123:15
    = note: inside `test::test_main_static` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:142:5
    = note: inside `main`
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:125:18
    = note: inside closure at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:63:18
    = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
    = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:401:40
    = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:365:19
    = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:129:14
    = note: inside closure at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:45:48
    = note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:401:40
    = note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:365:19
    = note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:129:14
    = note: inside `std::rt::lang_start_internal` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:45:20
    = note: inside `std::rt::lang_start::<()>` at /Users/tbm/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:62:5
    = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to previous error

error: test failed, to rerun pass '--lib'

running 25 tests
test arena::test::get2_mut ... 

shrink_to_fit

Currently, the struct cannot return the allocated back to OS. Would be nice to have this method

Arena::insert(impl Into<T>)

impl Into<T> is good for usability, but not good for compilation time.

---

I'm opening issues these days, but they are actually questions.
If thuderdome become the best arena crate (TM), that's both nice and good for my learning :)

Consider exposing generation

I have a few thunderdome::Arenas so I'm wrapping the Index in my own newtypes to keep things manageable. That means that my Debug output looks like:

ElementIndex(Index { slot: 1, generation: Generation(2) })

I'd prefer to use something a bit shorter:

ElementIndex(2, 1)

However, it's not possible to get the generation out of an Index for use in my own Debug implementation. I assume there's good reasons for that, so this suggestion may go nowhere, but it's worth asking!

Exposing iterators via `iter` module

Current documentation:

Suggestion:

Code:

pub mod iters {
    //! Iterator types

    pub use crate::drain::Drain;
    pub use crate::drain_filter::DrainFilter;
    pub use crate::into_iter::IntoIter;
    pub use crate::iter::Iter;
    pub use crate::iter_mut::IterMut;
    pub use crate::values::ValuesIter;
    pub use crate::values_mut::ValuesIterMut;
}

Feel free to skip this idea. Thank you :)

Add Arena::get2_mut(Index, Index) or similar methods

As one coming from generational-arena, I found thunderdome is lacking a method similar to Arena::get2_mut(&mut self, Index, Index) -> (Option<&mut Value>, Option<&mut Value>).

This method allows library users to access two different values at once (as long as their indices are different), allowing one to perform complex operations efficiently. An example usage would be connecting doubly-linked lists inside this arena, where having mutable references of both lists' ends simplifies the code.

Taking this idea one step further, adding another method like unsafe Arena::get_many_mut(&mut self, impl Iterator<Item=Index>) -> impl Iterator<Option<&mut Value>> (where the user ensures no two indices are the same) also seems reasonable. I'm just not sure if it's really needed.

Fuzzing

Even though this code was written with care and has a test suite, there's still potential for unexpected issues. Setting up fuzzing would help vet this code more thoroughly.

I don't have any experience in this area, so it might be a good learning exercise too.

Add iter() and iter_mut() methods

Iterating over an arena is an important operation.

Benchmarks

It would be good to know how Thunderdome ranks against other arena/slab crates to at least verify that it isn't catastrophically slower.

The bencher crate seems to be the way to go currently.

Update slotmap features

Hey, I'm the author of slotmap. I see you have a comparison table in your README.

Since slotmap 1.0 it supports no_std as well as non-Copy values for all data structures. I would appreciate it if you could update the table to reflect this in your next version.

Serde Support

This one is tricky! What serialization makes sense that preserves as much information as possible about the arena without letting incorrect serialized data break up our variants?

Release a new version

It seems IterMut was fixed in #10 but the version on crates.io is still broken. Without it, this crate is almost unusable.

Iterator API design

thunderdome::Iter returns (Index, &T) pair. Iterator of &T would also be useful.

Arena API design choices would be:

1. iter only
2. iter + values
3. iter + indexed_values (swapped version of 2.)

Index<T>?

Can I hear your thoughts on adding type parameter T to Index?

For example, current Index in thunderdome would look like this:

pub struct AttackCommand {
    pub actor: Index,
    pub dir: Dir,
}

If Index is typed with T, it would look like this:

pub struct AttackCommand {
    pub actor: IndexT<Actor>,
    pub dir: Dir,
}

It depends on preferences, but I thought the latter would make more sense.

What would you think? Thank you.

API for tree

I'm using thunderdome to store UI Node s. Nodes can have children, so API like indextree would be nice; it would be like ChildAwareNode and ParentAwareNode trait. If it's a common pattern, any chance thunderdome would get it?

Thank you.

Miri reports Undefined Behavior in tests

Hi! Evaluating some crates and decided to check out thunderdome. It looked pretty good until I tried running the test suite under miri, which reports Undefined Behavior:

$ cargo miri test --lib -- get2_mut
Preparing a sysroot for Miri (target: x86_64-unknown-linux-gnu)... done
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests src/lib.rs (target/miri/x86_64-unknown-linux-gnu/debug/deps/thunderdome-1f5b4f350601acba)

running 5 tests
test arena::test::get2_mut ... error: Undefined Behavior: trying to retag from <233349> for Unique permission at alloc93875[0x4], but that tag does not exist in the borrow stack for this location
   --> src/arena.rs:409:48
    |
409 |         let item1 = unsafe { item1_ptr.map(|x| &mut *x) };
    |                                                ^^^^^^^
    |                                                |
    |                                                trying to retag from <233349> for Unique permission at alloc93875[0x4], but that tag does not exist in the borrow stack for this location
    |                                                this error occurs as part of retag at alloc93875[0x4..0x8]
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <233349> was created by a SharedReadWrite retag at offsets [0x4..0x8]
   --> src/arena.rs:406:54
    |
406 |         let item1_ptr = self.get_mut(index1).map(|x| x as *mut T);
    |                                                      ^
help: <233349> was later invalidated at offsets [0x0..0x18] by a Unique retag
   --> src/arena.rs:372:15
    |
372 |         match self.storage.get_mut(index.slot as usize) {
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: BACKTRACE (of the first span):
    = note: inside closure at src/arena.rs:409:48: 409:55
    = note: inside `std::option::Option::<*mut i32>::map::<&mut i32, [closure@src/arena.rs:409:44: 409:47]>` at /home/jay/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:972:29: 972:33
note: inside `arena::Arena::<i32>::get2_mut`
   --> src/arena.rs:409:30
    |
409 |         let item1 = unsafe { item1_ptr.map(|x| &mut *x) };
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `arena::test::get2_mut`
   --> src/arena.rs:810:40
    |
810 |         let (foo_handle, bar_handle) = arena.get2_mut(foo, bar);
    |                                        ^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure
   --> src/arena.rs:805:19
    |
804 |     #[test]
    |     ------- in this procedural macro expansion
805 |     fn get2_mut() {
    |                   ^
    = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

FWIW, asan does not report any issues when running the test suite.

drain_filter

Vec::drain_filter is an unstable Rust feature. Alternative to it in Arena would be helpful in some cases.

Links:

• Tracking issue for drain_filter
• library/alloc/src/vec/drain_filter.rs

no_std support

We should be able to add an std feature, enabled by default. When disabled, we'll use core and alloc instead of std.

Implement IntoIterator

This can probably be very similar to with Drain for simplicity.