Comments (8)
Some options: https://godbolt.org/z/C34XMV (looks like the biggest code-size/branch-count diff is the as_bytes() change, though the branch predictor may optimize most of that away given the input is all ASCII, the Pairable iterator, if you make it non-pub, doesn't seem to add much to the code size).
That said, I don't see why we need to keep around a pub Pairable iterator in rust-bitcoin, seems way out of scope. I wouldn't really be surprised if the biggest CPU user was realloc and the with_capacity change were the biggest winner, but may be worth benchmarking some of those options.
from rust-bitcoin.
I did some benchmarking using @TheBlueMatt's implementations, the current one and two I wrote myself. You can find the code here and the results on my T480s/i7-8650U CPU @ 1.90GHz here.
On my machine my completely non-functional-style implementation is currently the fastest one (465ns with the current code -> 76ns with mine for 32bytes, @TheBlueMatt's second solution takes 95ns). I even think it's quite readable, but I might be a bit biased π
The reason why we can't use the Chars
iterator is quite sad: to me it seems that handling unicode codepoints isn't much more complicated than ascii characters, but what makes it inefficient in the end id that the size_hint
is way off (lower bound is len()/4
because of the max. UTF8 character size). That means we get many more allocations than necessary. At first I had an implementation using characters but this turned out to be a complete show stopper. I could have built a wrapper iterator struct that lets me add size hints after the fact, but imo it would still be slower then the currently fastest solutions or completely unreadable if written in a functional style.
You can benchmark it using cargo bench
. If it's well received I'll PR my solution.
from rust-bitcoin.
Now we know what are the ballparks here for original, and possibly optimal solution. I agree that lack of unsafe
is more important and I'm personally satisfied with current speedup. Thank you for working on it! I'll patiently await when this lands and gets released.
from rust-bitcoin.
@sgeisler I don't think it's going to add anything, but just for double-checking: can you bench with a 4MB worth of hex-encoded data? That corresponds to a biggest hex string that could potentially be hex-decoded in Bitcoin, if I'm not mistaken.
from rust-bitcoin.
I did the benchmark with 4M again and the result shows a much smaller difference between the current and the better implementations (which can be expected due to memory latency/cache misses).
In terms of throughput the best implementation achieves 260MByte/s input or 130MByte/s output. That somehow doesn't sound really good to me, but also not horrible. One could try out a lookup table approach, but I won't right now.
from rust-bitcoin.
https://github.com/nervosnetwork/faster-hex/ could be used as a datapoint too
from rust-bitcoin.
That's a really nice crate π but avoiding unsafe is probably the bigger concern than speed in this case imo (except someone wants to do a really thorough review on it, but I doubt it), it also seems to require the 2018 edition of rust, which we won't be able to use for some time (#206).
The stats for 256bytes:
- current:
1773 ns
- optimized:
464 ns
- faster-hex:
98 ns
The effect is even noticeable for 64 bytes but not that strongly.
Fun fact: switching to the current stable (because of faster-hex
) made the current code 45% faster π
from rust-bitcoin.
from rust-bitcoin.
Related Issues (20)
- Fuzz failure in hashes/cbor HOT 2
- The `sha256t_hash_newtype!` call in the `taproot` module is unclear HOT 3
- Make `bitcoin::p2p::Magic::from_bytes` const function HOT 1
- Unused manifest key warning HOT 2
- What's the point of `Denomination::MilliSatoshi`? HOT 23
- `SegwitV0Sighash` is missing an `into_secp256k1_message` HOT 3
- Should be able to directly sign with sighashes HOT 7
- `Amount` should support milisats HOT 1
- Links to crate code inside `hash_newtype!` don't work HOT 3
- Mutation testing improvements (get rid of mutagen) HOT 9
- Fix bug in `pow.rs` HOT 3
- Consider Enabling explicitly formatting a derivation path with the `m/` prefix HOT 6
- Rustdoc: private doc tests lint HOT 2
- Add a script for common tasks updating lock files HOT 1
- Pin `cargo public-api` and other tools HOT 2
- Change `PushBytes::read_scriptint(x)` to `x.read_scriptint()`
- Displaying hex HOT 5
- No way to access `UnprefixedHexError` HOT 3
- Remove usage of wildcard imports
- Cannot get DerivationPath serialised from root anymore HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust-bitcoin.