multiformats / rust-multibase

Multibase in rust

rust-multibase

multibase implementation in Rust.

Table of Contents

• Install
• Usage
• Maintainers
• Contribute
• License

Install

First add this to your Cargo.toml

[dependencies]
multibase = "0.9"

For no_std

[dependencies]
multibase = { version ="0.9", default-features = false }

note: This crate relies on the currently unstable host_dep feature to compile proc macros with the proper dependencies, thus requiring nightly rustc to use.

Then run cargo build.

Usage

use multibase::Base;

let base64 = multibase::encode(Base::Base64, b"hello world");
let (base, data) = multibase::decode(base64);

Note: base32 and base64 are orders of magnitude faster due to byte alignment. Don't be surprised if using a different base turns into a performance bottleneck. You were warned!

Maintainers

Captain: @dignifiedquire.

Contribute

Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS Code of Conduct.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT © Friedel Ziegelmayer

rust-multibase's Issues

most bases implemented are not what the label says!

base_x uses a different algorithm than base32 and base64. While it can be used as a base32 or base64 encoding it is not what most people expect and it is definitely not rfc4648 compliant.

Any plans to support no_std?

i started using this lib together with rust-cid and rust-multihash on a Substrate project and i realised that you don't support no_std any plans to support that?

Thank you

Performance

So I implemented the ipld multiblock vector and did some benchmarks. Here are the results from my dell xps where > means faster.

cbor > json
blake2s > sha2-256 > blake2b

Interestingly most of time is spent on base encoding (by a margin).

Here is what I flamegraphed for reference:

use criterion::black_box;
use ipld_vec::{mock::Store, Vector};
use libipld::{DefaultPrefix, Ipld};

type IpldVec = Vector<DefaultPrefix, Store>;

fn main() {
    // push: 1024xi128; n: 4; width: 256; size: 4096
    let mut vec = IpldVec::new(256).unwrap();
    for i in 0..1024 {
        vec.push(Ipld::Integer(i as i128)).unwrap();
    }
    black_box(vec);
}

It only uses a simple hashmap as block storage, it might not matter much once I implement a file system block storage.
NOTE: the bs58 crate is much slower.

Implement lowercase base32

Not anything urgent but would be nice to have. Happy to make this change when I have time. Sure it's self explanatory but having a code of 'b' (and 'c' for padded but I don't personally care about that variant) and lower case alphabet variant of base32

lookup table / simd implementation of base32/64

While we probably can't do better for base58 since it's not byte aligned. We should add a performant base32/64 implementation and mention in the readme that other base encodings are expected to be slow.

Requesting to create PR code supporting WordBase-2048 or get feedback on the idea

I would like to discuss and solicit me working on a PR to add another base see: multiformats/multibase#89

Bump version to 0.7.0 and upload to crates.io

The current version on crates.io is 0.6.0 and it lacks some of the improvements on the current master branch.

It would be nice to get new features like padded encodings on the crates.io.

A complete implementation of encodings in the multibase table

Please see implementation for details.

@dignifiedquire If you think there is no problem with the implementation, I will open a PR.

Make it no-alloc compatible

The data-encoding library is no_alloc compatible, you can use e.g. encode_mut.

We could provide a similar trait, which writes into a mutable slice.

Things to keep in mind:

• base-x currently needs alloc. It's used for bases which are not power of two.
• It increases the API surface, currently this library is pretty small, should we change that?
• Can we make it work without making this library hard to use when you don't need no_std and no-alloc compatibility?
• How would it relate to rust-cid? Could we follow the idea of rust-multihash and use GenericArray when converting into a certain Multibase? Given that we know the maximum digest size of the CID, we also know the possible maximum base encoded size at compile time. Can this be wrapped in a nice to use API?

Encoding ambiguity due to underlying design

The truncation of leading zeros in the base-x package this library is built with creates an ambiguous encoding. This is especially noticeable in base-2 where the example test case "yes mani !" encodes to "01111001011001010111001100100000011011010110000101101110011010010010000000100001", while a strict encoding would be "001111001011001010111001100100000011011010110000101101110011010010010000000100001". This means that decoding "\x00yes mani !" results in "yes mani !", an erroneous interpretation in many contexts.

More info: multiformats/multibase#34

Use macros, slim down fat

impls.rs could be cleaned up with macros, along with some other minor changes here and there around the repo.

It could also get the same treatment as https://github.com/multiformats/rust-multihash where each base code is a feature, which may be useful for those looking to use no_std to slim down crate size although that might be a bit overkill.

I'd be interested in making these changes but figured opening an issue first before making a large PR would be better than not, especially if people are interested in helping out.