Currently, the traits exposed via the public API are incomplete. Traits are being rearranged, with trait bounds being relaxed in some cases, while in others they are more restrictive.

This crate is intended to provide a more ergonomic interface and API for working with raw byte buffers. It is absolutely imperative that the types and trait implementations in [abio] are sound. At the cost of a small runtime penalty, bounds checks, alignment checks on pointers (that cannot be optimized away due to not being evaluated within const context), etc., This crate makes working in extremely hazardous environments simpler, easier and more intuitive.

Release v0.3.0

This marks an absolute major release for this project. The entire code base has been refactored, traits have been modified to work together and produce an API that is both easy to use and leverages zero-cost abstractions as much as possible to push as much evaluation as possible to oonst contexts.

For additional information about const contexts and their relevance, please refer to reference section on const evaluation

Affected Types

[Chunk](https://docs.rs/abio/0.3.0/abio/struct.Chunk.html)
    A newtype wrapper for fixed-size byte arrays (generic over the constant N parameter).
[Slice](https://docs.rs/abio/0.3.0/abio/struct.Slice.html)
    A newtype wrapper for byte slices.
    Note: The Slice type will be renamed to Bytes in the upcoming 0.4.0 release.

Problem

The Bytes (formerly Slice) and Chunk<N> types serve as core primitives for handling byte slices and arrays. Their importance is emphasized by their extensive use throughout the codebase. Moreover, there are plans to further extend their APIs and functionalities. In other words, these types are not going anywhere anytime soon.

Deref Polymorphism

Introducing these types into the public API demands caution. Their primary purpose is to encapsulate unsafe code, offering safe interfaces in return. While "Deref polymorphism" might be tempting due to its convenience, it appears to be actively harmful in the context of this library. Its inherently risky nature presents numerous pitfalls. As a result, retaining the status quo for these types feels inappropriate.

Proposed Solutions

The first solution that came to mind was to simply remove these types from the public API entirely. This change would eliminate the potential misuse and these types would remain, but serve as mere implementation details.

Instead, I think that removing the current Deref implementations is both less invasive and more useful. As such, the code will be refactored to function without relying on "Deref polymorphism." This approach requires minimal adjustments and negates the necessity of removing types from the public API.

Benefits of Publicly Exposing Bytes / Chunk<N>

By keeping the types public while reducing their API scope, we can safely expose these types and allow to benefit from pending niche optimizations. Moreover, it paves the way for enhancing the utility and applicability of these types. My motivation for originating this library stemmed from my dissatisfaction with prevailing solutions; I yearned for a more flexible alternative.

Type Extensions

It would be great to introduce some additional utility types for FFI, which is a major use-case for this library. Introducing variants such as Utf8Str for working with Rust's &str types, or NullStr to handle C-style null-terminated strings, would be welcome additions.

Conclusion

One of the core "values" of this library is that it provides safety guarantees to users by imposing restrictions. This is how Rust works as a whole, so it makes sense to me to use what the language offers.

Overall, this should result in safer code and allow more aggressive optimizations for these types. The more restricted they are, the more we can play fast and loose elsewhere.

This library is currently still quite unstable. With the release of 0.4.0, the API will be significantly more stable and the project will be heading in the right direction.

This is largely an enhancement and not a priority at the moment. If someone feels like picking up this issue, please feel free. I could use the help and this could help make the library more performant.

The main issue that I see right now is that the library still lacks a proper benchmarking suite. This will be implemented as part of version 0.4.0, so once the 0.4.0 milestone is reached, this issue can be addressed.

There is, as of right now, an insufficient amount of testing to definitively say that the following implementations are sound:

unsafe impl<T: Abi> Abi for *const T {}
unsafe impl<T: Abi> Abi for *mut T {}

It may be prudent to impose additional restrictions via trait bounds for these blanket implementations. However, I think it makes a bit more sense right now, with this library being in its absolute infancy, to allow this until it can be proven to be sound or unsound.

This crate is meant to be flexible, yet safe. Flexibility and ergonomics are a very high priority for abio, so I will leave this issue open until the question is resolved.

This project will soon be made public and published to crates.io when v0.3.0 is stabilized.

I have decided that abio will be using "trunk-based development". Within the context of this library, that means that there will be no long-lived feature branches. The foundation of the library has been implemented and I am happy with the shape of the API.

With that said, the most important thing to take away from this is that this library will use a trunk-based approach to development such that any pushes to the main branch must pass all existing tests and all workflows must succeed. Broken code should never be pushed to main.

This crate needs additional unit testing and, more importantly, integration testing.

The proposed test suite will involve miri to help detect UB, and the addition of a few more dev-dependencies to make it easier to write high-quality, meaningful tests.

In addition to this, I would like to add a basic benchmarking suite to start, and then it can be expanded upon from there. The most important functions to benchmark are those that involve reading data into slices vs. arrays (Bytes / Chunk<N> types).

Ideally, this will make it into v0.3.0 to give end-users a better picture of what the crate's capabilities are. There are not many crates in this particular area (zero copy direct reading/writing from byte slices aka "safe transmute"), so it is crucial that abio is able to compete with these crates.

Current Design

The decision was made to make the error module largely private, or pub(crate). Very little of it is actually exposed. As of this moment, the public API contains Error and a Result type, but there is no way to use the constructor functions local to the crate to instantiate abio::Error types.

Unanswered Questions

• Should we expose some of, or all of, the constructor methods?
• Is keeping the API as-is better, since it forces users to use their own error-handling facilities?

Update on 0.4.0

This project is an experiment concerning "Compile Time Function Evaluation" (CTFE) and how it may be used to create components such as lexers, parsers, memory mappers, and anything else that relies on the use of zero-copy, endian-aware operations against raw bytes/memory.

I want to push CTFE to its limits, because I believe it can be used to improve our collective tooling as developers and it can do so safely (by evaluating as much as possible at compile time) with minimal runtime cost.

State of Const Evaluation in Rust

Unfortunately, Rust simply lacks the required features to realize this project. As powerful as it is, many of the needed features are still unstable and may change. The initial release of "const generics" was a step in the right direction, but things like "generic const expressions" are not even close to being stable.

Procedural Macros

Procedural macros are absolutely one solution to this problem. However, I am not willing to take on that amount of technical debt when it is not necessary. Rust meta-programming is quite complex and is not at all something I am interested in delving into for the sake of this project.

Having a lot of procedural macros may generate a significant amount of code, which can cause bloating of end-user binary/library sizes, very long compilation times (relatively speaking) and they are frankly just not easy to understand for most people.

This poses then yet another problem: if this library relies on complex procedural macros, it will be inherently more difficult to find contributors to help out. As powerful as they are, procedural macros are arguably arcane and esoteric in nature. If I had to guess, most Rust developers are probably not experts with procedural macros.

Migration to Zig

For the reasons stated above, along with several others, I have decided to begin porting this project from Rust to Zig. A new repository will be created, separate from this one, but some things will be copied over verbatim.

Development Status

This project is still under heavy development. The new repository will contain any information needed to contribute, if you are interested.