indexmap-rs / indexmap

A hash table with consistent order and fast iteration; access items by key or sequence index

indexmap

A pure-Rust hash table which preserves (in a limited sense) insertion order.

This crate implements compact map and set data-structures, where the iteration order of the keys is independent from their hash or value. It preserves insertion order (except after removals), and it allows lookup of entries by either hash table key or numerical index.

Note: this crate was originally released under the name ordermap, but it was renamed to indexmap to better reflect its features.

Background

This was inspired by Python 3.6's new dict implementation (which remembers the insertion order and is fast to iterate, and is compact in memory).

Some of those features were translated to Rust, and some were not. The result was indexmap, a hash table that has following properties:

• Order is independent of hash function and hash values of keys.
• Fast to iterate.
• Indexed in compact space.
• Preserves insertion order as long as you don't call .remove(), .swap_remove(), or other methods that explicitly change order. The alternate .shift_remove() does preserve relative order.
• Uses hashbrown for the inner table, just like Rust's libstd HashMap does.

Performance

IndexMap derives a couple of performance facts directly from how it is constructed, which is roughly:

A raw hash table of key-value indices, and a vector of key-value pairs.

• Iteration is very fast since it is on the dense key-values.

• Removal is fast since it moves memory areas only in the table, and uses a single swap in the vector.

• Lookup is fast-ish because the initial 7-bit hash lookup uses SIMD, and indices are densely stored. Lookup also is slow-ish since the actual key-value pairs are stored separately. (Visible when cpu caches size is limiting.)

• In practice, IndexMap has been tested out as the hashmap in rustc in PR45282 and the performance was roughly on par across the whole workload.

• If you want the properties of IndexMap, or its strongest performance points fits your workload, it might be the best hash table implementation.

Recent Changes

See RELEASES.md.

indexmap's Issues

An insert_sorted() method

I would like to use IndexMap as a BTreeMap with faster reads and slower writes.
But calling sort_keys() after every insert() is pretty slow.
Would it be possible to have a insert_sorted() method that would just insert the element before the next bigger one?

Implement Hash on IndexMap and IndexSet

I'm currently looking into embedding an IndexMap into a larger recursive enum (which looks pretty similar to this). However I hit a blocker on my journey, as the type-checker complains that IndexMap does not implement Hash (duh).

While I think in theory it is quite unlikely to have a usecase where an IndexMap is used as an hashable key, the type system is technically correct on this. Would a PR to derive Hash on all the relevant types up to IndexMap+IndexSet be welcome here? For reference, within stdlib BTreeMap implements it but HashMap doesn't.

Copyright?

Please, write a clear copyright notice on your source-code and insert the full text of the license in the repository.

serde feature is incompatible with no_std

The serde-1 feature is still always compiled with its std feature, meaning that indexmap's serde feature is not no_std compatible.

Feature Request: OrderSet?

Like HashSet, but with this crate's hash table under the hood

Change name?

original-insertion-order-preserving removal would want to be implemented with tombstones, but I've decided this is going to be a separately named hash map. So: fork time.

Just an idle thought -- OrderMap was a good name for something that does guarantee insertion-order-preserved for all operations, so have a new name for this new thing which does not make that guarantee.

Is there a FastMap ? :-)

Deriving `Default` on type containing `IndexSet<T>` requires `T: Default`

The following code fails to compile:

pub struct Data;

#[derive(Default)]
pub struct Newtype<T>(indexmap::IndexSet<T>);

fn test() {
    let _: Newtype<Data> = Default::default();
}

The error I observe is the following (this is in a module called 'types.rs'):

error[E0277]: the trait bound `types::Data: std::default::Default` is not satisfied
 --> src/types.rs:8:28
  |
8 |     let _: Newtype<Data> = Default::default();
  |                            ^^^^^^^^^^^^^^^^ the trait `std::default::Default` is not implemented for `types::Data`
  |
  = note: required because of the requirements on the impl of `std::default::Default` for `types::Newtype<types::Data>`
  = note: required by `std::default::Default::default`

However, adding the #[derive(Default)] to Data fixes the issue.

I am surprised that the automatic derive of Default requires that the data type stored in the IndexSet also be Default! It doesn't seem to make sense in this case.

1.5.0 breaks h2 build

After re-installing my environment I'm getting an error on same code base:

error[E0107]: wrong number of type arguments: expected 3, found 2
  --> /home/pah/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.2.6/src/proto/streams/store.rs:15:10
   |
15 |     ids: IndexMap<StreamId, SlabIndex>,
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 3 type arguments

error: aborting due to previous error

For more information about this error, try `rustc --explain E0107`.
error: could not compile `h2`.

Is there a way to somehow set has_std for indexmap crate so deep in dependency tree?

Edit: It works on windows.

Differences from linked-hash-map?

What are the reasons someone would prefer one or the other?

They both provide the same basic guarantees: iteration order matches insertion order and "constant time" for various operations.

From the readme it sounds like OrderMap lookups are faster than LinkedHashMap (OrderMap is faster than HashMap, while LinkedHashMap uses HashMap internally). Also linked-hash-map uses gobs of unsafe code.

Would you recommend OrderMap in all cases over LinkedHashMap or are there some disadvantages?

OrderMap 1.0 Roadmap

Must have

• Full consistency check list should be complete and remaining discrepancies should be fixable backwards compatibly or marked as not important (#19)
• Mutable keys #7 #39
• Settle crate name #14 Settle removal semantics #15 (and split crate for full insertion order #12)
• Move map stuff to map:: submodule (like set::) #60

Nice to have

(can be added backwards compatibly)

• Full consistency (#19). Being drop in is good, but some of the details don't matter much, you can be effectively drop in without being 100% type by type compatible.
• OrderSet #8
• Order preserving retain #56
• Every public item has documentation

At Release

• Blog post about the name change and explain the properties of the data structure

Question about ordered remove

I wanted to remove an element without perturbing the order of the remaining elements, and used something like the code below. Is there a better way than this? (In my case it is not really that important, this works just fine for me.)

type Map = IndexMap<K, V>;
fn remove_ordered(m: &mut Map, key: &K) -> Option<V> {
    let (index, _key, val) = m.swap_remove_full(key)?;
    if index < m.len() {
        let tail = m.len() - index - 1;
        let mut stack = Vec::with_capacity(tail);
        for _ in 0..tail {
            stack.push(m.pop().expect("disappearing elements"));
        }
        let (last_key, last_val) = m.pop().expect("disappearing elements");
        m.extend(stack.into_iter().rev());
        m.insert(last_key, last_val);
    }
    Some(val)
}

Improve .entry() performance

In some of the benchmarks, if I replace .insert() with code based on entry, the benchmarks run slower. If we could get these to performance parity, then we can delete the .insert() code, since it is duplicated by the functionality in .entry().

To test this, you can update insert to be like this (note insert_full uses entry right now):

    pub fn insert(&mut self, key: K, value: V) -> Option<V> {
        self.insert_full(key, value).1
    }

The following is the benchmark difference I see when using entry in insert.

The hashmap benchmarks can be ignored - ordermap is the interesting part. (The hashmap benchmarks are controls.)

Change from before to after.

 name                                   63 ns/iter  62 ns/iter  diff ns/iter  diff % 
 insert_hashmap_100_000                 2,488,863   2,504,113         15,250   0.61% 
 insert_hashmap_10_000                  214,543     221,693            7,150   3.33% 
 insert_hashmap_150                     3,210       3,339                129   4.02% 
 insert_hashmap_int_bigvalue_10_000     262,701     264,518            1,817   0.69% 
 insert_hashmap_str_10_000              236,377     247,805           11,428   4.83% 
 insert_hashmap_string_10_000           1,117,104   1,148,170         31,066   2.78% 
 insert_hashmap_string_oneshot_10_000   1,078,875   1,085,733          6,858   0.64% 
 insert_orderedmap_100_000              2,727,119   3,874,446      1,147,327  42.07% 
 insert_orderedmap_10_000               267,361     363,741           96,380  36.05% 
 insert_orderedmap_150                  3,956       5,664              1,708  43.17% 
 insert_orderedmap_int_bigvalue_10_000  359,582     413,168           53,586  14.90% 
 insert_orderedmap_str_10_000           305,463     312,993            7,530   2.47% 
 insert_orderedmap_string_10_000        1,066,189   1,091,526         25,337   2.38%

DOC: Comparison with HashMap

See the list here: rust-lang/rust#45273 (comment)

A design for single-pass borrowed insert_if_not_there

I have an IndexSet<String> that I want to be able to insert or get index from &str without allocating needlessly if the value is already in the set.

This is actually possible with one small library addition: entry_index. It would return an entry for the value at the index up to and including .len(), thus allowing insertion of the value there. It wouldn't even need to be unsafe: having two keys that compare equal in the map is only very bad for domain correctness, but not for memory correctness if I understand correctly.

It could then be used for inserting if not present with something like the following:

set.get(&key)
    .unwrap_or_else(|| set.entry_index(set.len)
        .insert(key.to_owned())

Make IndexSet::intersection() work with stdlib's hashset

There's no reason it shouldn't work with the stdlib hashset. Perhaps we can add a second method for this?

published indexmap 1.3.1 has broken Cargo.toml

1.3.1 seems to not be shipped with a Cargo.toml generated by cargo package, which is what is expected from running cargo publish

Here's the diff from 1.3.0 to 1.3.1:

diff 1.3.0/Cargo.toml 1.3.1/Cargo.toml

diff --git a/Cargo.toml b/Cargo.toml
index 7b4e98d..166a7f8 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,66 +1,65 @@
-# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
-#
-# When uploading crates to the registry Cargo will automatically
-# "normalize" Cargo.toml files for maximal compatibility
-# with all versions of Cargo and also rewrite `path` dependencies
-# to registry (e.g., crates.io) dependencies
-#
-# If you believe there's an error in this file please file an
-# issue against the rust-lang/cargo repository. If you're
-# editing this file be aware that the upstream Cargo.toml
-# will likely look very different (and much more reasonable)
-
 [package]
 name = "indexmap"
-version = "1.3.0"
-authors = ["bluss", "Josh Stone <[email protected]>"]
-build = "build.rs"
-description = "A hash table with consistent order and fast iteration.\n\nThe indexmap is a hash table where the iteration order of the key-value\npairs is independent of the hash values of the keys. It has the usual\nhash table functionality, it preserves insertion order except after\nremovals, and it allows lookup of its elements by either hash table key\nor numerical index. A corresponding hash set type is also provided.\n\nThis crate was initially published under the name ordermap, but it was renamed to\nindexmap.\n"
+version = "1.3.1"
+authors = [
+"bluss",
+"Josh Stone <[email protected]>"
+]
 documentation = "https://docs.rs/indexmap/"
+repository = "https://github.com/bluss/indexmap"
+license = "Apache-2.0/MIT"
+description = """
+A hash table with consistent order and fast iteration.
+
+The indexmap is a hash table where the iteration order of the key-value
+pairs is independent of the hash values of the keys. It has the usual
+hash table functionality, it preserves insertion order except after
+removals, and it allows lookup of its elements by either hash table key
+or numerical index. A corresponding hash set type is also provided.
+
+This crate was initially published under the name ordermap, but it was renamed to
+indexmap.
+"""
+
 keywords = ["hashmap", "no_std"]
 categories = ["data-structures", "no-std"]
-license = "Apache-2.0/MIT"
-repository = "https://github.com/bluss/indexmap"
-[package.metadata.docs.rs]
-features = ["serde-1", "rayon"]
 
-[package.metadata.release]
-no-dev-version = true
-tag-name = "{{version}}"
-[profile.bench]
-debug = true
+build = "build.rs"
 
 [lib]
 bench = false
-[dependencies.rayon]
-version = "1.0"
-optional = true
 
-[dependencies.serde]
-version = "1.0"
-optional = true
-[dev-dependencies.fnv]
-version = "1.0"
+[build-dependencies]
+autocfg = "1"
+[dependencies]
+serde = { version = "1.0", optional = true }
+rayon = { version = "1.0", optional = true }
 
-[dev-dependencies.itertools]
-version = "0.8"
+[dev-dependencies]
+itertools = "0.8"
+rand = "0.6"
+quickcheck = { version = "0.8", default-features = false }
+fnv = "1.0"
+lazy_static = "1.3"
+serde_test = "1.0.99"
 
-[dev-dependencies.lazy_static]
-version = "1.3"
+[features]
+# Serialization with serde 1.0
+serde-1 = ["serde"]
+
+# for testing only, of course
+test_low_transition_point = []
+test_debug = []
 
-[dev-dependencies.quickcheck]
-version = "0.8"
-default-features = false
+[profile.bench]
+debug = true
 
-[dev-dependencies.rand]
-version = "0.6"
+[package.metadata.release]
+no-dev-version = true
+tag-name = "{{version}}"
 
-[dev-dependencies.serde_test]
-version = "1.0.99"
-[build-dependencies.autocfg]
-version = "0.1.6"
+[package.metadata.docs.rs]
+features = ["serde-1", "rayon"]
 
-[features]
-serde-1 = ["serde"]
-test_debug = []
-test_low_transition_point = []
+[workspace]
+members = ["test-nostd"]

Additionally, unlike 1.3.0, which works fine with the cargo-generated Cargo.toml

1.3.0 cargo test success

cargo test
    Updating crates.io index
  Downloaded autocfg v0.1.7
  Downloaded serde_test v1.0.104
  Downloaded rand v0.6.5
  Downloaded quickcheck v0.8.5
  Downloaded itertools v0.8.2
  Downloaded serde v1.0.104
  Downloaded rand_jitter v0.1.4
  Downloaded rand_pcg v0.1.2
  Downloaded rand_xorshift v0.1.1
  Downloaded rand_hc v0.1.0
  Downloaded rand_os v0.1.3
  Downloaded rand_chacha v0.1.1
  Downloaded rand_isaac v0.1.1
  Downloaded either v1.5.3
   Compiling autocfg v0.1.7
   Compiling rand_core v0.4.2
   Compiling libc v0.2.66
   Compiling serde v1.0.104
   Compiling either v1.5.3
   Compiling lazy_static v1.4.0
   Compiling fnv v1.0.6
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling itertools v0.8.2
   Compiling rand_chacha v0.1.1
   Compiling rand_pcg v0.1.2
   Compiling rand v0.6.5
   Compiling indexmap v1.3.0 (/home/kent/.cpanm/work/1580894972.28404/indexmap-1.3.1)
   Compiling rand_xorshift v0.1.1
   Compiling rand_hc v0.1.0
   Compiling rand_isaac v0.1.1
   Compiling rand_os v0.1.3
   Compiling quickcheck v0.8.5
   Compiling serde_test v1.0.104
    Finished test [unoptimized + debuginfo] target(s) in 47.28s
     Running target/debug/deps/indexmap-af6a4872f48a66f7

running 33 tests
test map::tests::entry_or_default ... ok
test map::tests::entry ... ok
test map::tests::entry_and_modify ... ok
test map::tests::extend ... ok
test map::tests::insert ... ok
test map::tests::insert_full ... ok
test map::tests::grow ... ok
test map::tests::it_works ... ok
test map::tests::insert_order ... ok
test map::tests::keys ... ok
test map::tests::new ... ok
test map::tests::partial_eq_and_eq ... ok
test map::tests::remove_to_empty ... ok
test map::tests::remove ... ok
test map::tests::values ... ok
test map::tests::values_mut ... ok
test map::tests::swap_remove_index ... ok
test set::tests::comparisons ... ok
test set::tests::extend ... ok
test set::tests::insert ... ok
test set::tests::grow ... ok
test set::tests::insert_dup ... ok
test set::tests::insert_full ... ok
test set::tests::insert_order ... ok
test set::tests::it_works ... ok
test set::tests::iter_comparisons ... ok
test set::tests::new ... ok
test set::tests::partial_eq_and_eq ... ok
test set::tests::remove ... ok
test set::tests::ops ... ok
test set::tests::swap_remove_index ... ok
test map::tests::insert_2 ... ok
test set::tests::insert_2 ... ok

test result: ok. 33 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/equivalent_trait-1769a2528452c354

running 2 tests
test test_lookup ... ok
test test_string_str ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/macros_full_path-e6fdf8b9bff872b5

running 2 tests
test test_create_map ... ok
test test_create_set ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/quick-711278843a6d10b5

running 16 tests
test drain ... ok
test contains ... ok
test equality ... ok
test contains_not ... ok
test insertion_order ... ok
test insert_remove ... ok
test operations_string ... ok
test pop ... ok
test keys_values ... ok
test operations_i8 ... ok
test keys_values_mut ... ok
test shift_remove ... ok
test with_cap ... ok
test sort_2 ... ok
test sort_1 ... ok
test retain_ordered ... ok

test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/serde-406ea1513b9cfc01

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/tests-f4d162c97b6224f6

running 2 tests
test test_sort ... ok
test test_sort_set ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests indexmap

running 4 tests
test src/set.rs - set::IndexSet (line 55) ... ok
test src/macros.rs - indexset (line 47) ... ok
test src/map.rs - map::IndexMap (line 259) ... ok
test src/macros.rs - indexmap (line 8) ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

1.3.1 is broken for all cargo commands:

1.3.1 breakage

> cargo build
error: failed to read `/home/kent/.cpanm/work/1580894972.28404/indexmap-1.3.1/test-nostd/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

> cargo test
error: failed to read `/home/kent/.cpanm/work/1580894972.28404/indexmap-1.3.1/test-nostd/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

 > cargo package
error: failed to read `/home/kent/.cpanm/work/1580894972.28404/indexmap-1.3.1/test-nostd/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

> cargo update
error: failed to read `/home/kent/.cpanm/work/1580894972.28404/indexmap-1.3.1/test-nostd/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

This seems like a pretty serious regression, and is likely to be a substantial impediment to all linux vendors packaging rust.

Minimum Rust Version

Petgraph uses Ordermap 0.3.x and uses a minimum Rust version of 1.12 (and it compiles), in this repo we test from Rust 1.18 in travis. So I need to look at docs & travis test for that in ordermap. More relevant for 1.0-ing maybe. At the point of an 1.x release series, it's no longer reasonable to freeze the minimum Rust version, so we'd have to consider letting it upgrade (slowly).

any idea how to return an iterator of a OrderMap value?

Hey,

Thank you for your work on this great library.

I have been banging my head trying to figure out how to return an Iterator with impl Trait if the collection is in an OrderMap. Do you know how to solve this? (There many not be any way.)

Example:

#![feature(conservative_impl_trait)]

extern crate ordermap;
extern crate uuid;

use ordermap::OrderMap;
use uuid::Uuid;

struct Map<T> {
    items: OrderMap<Uuid, Vec<T>>
}

impl<T> Map<T> {
    fn iter_key<'a>(&'a self, k: &'a Uuid) -> impl Iterator<Item = &'a T> + 'a {
        self.items.get(k)
            .iter()
            .flat_map(|xs| {
                xs.iter()
            })
    }
}

Note: I used the iter method on the Option<&V> returned from get but have also tried several other approaches. Also, the + 'a may or may not be necessary - I have no idea what it does, but it solved some of my earlier iterator problems after I found it suggested on a github issue.

The approach above results in the error:

error[E0597]: borrowed value does not live long enough
  --> src/main.rs:15:9
   |
15 |         self.items.get(k)
   |         ^^^^^^^^^^^^^^^^^ does not live long enough
...
20 |     }
   |     - temporary value only lives until here
   |
note: borrowed value must be valid for the lifetime 'a as defined on the method body at 14:5...
  --> src/main.rs:14:5
   |
14 | /     fn iter_key<'a>(&'a self, k: &'a Uuid) -> impl Iterator<Item = &'a T> + 'a {
15 | |         self.items.get(k)
16 | |             .iter()
17 | |             .flat_map(|xs| {
18 | |                 xs.iter()
19 | |             })
20 | |     }
   | |_____^

playground

Returning an Iterator on OrderMap values would be a hugely useful tool for me to have, so figured I would see if perhaps some more experienced people here could show the right way. Thanks for any help you can provide.

impl Hash for IndexSet, IndexMap

I think it would be useful for IndexSet, like Vec, to implement Hash. Technically there's no reason IndexMap couldn't either, though I'm not sure how useful it would be practically.

It should be a pretty simple addition, are there downsides?

IndexMap compilation fails "expected 3 type arguments"

Hi, this is a weird issue that I stumbled upon that is roughly similar to #144 but I have some new information that may or may not be relevant.

   Compiling h2 v0.2.6
   Compiling futures v0.3.5
error[E0107]: wrong number of type arguments: expected 3, found 2
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/h2-0.2.6/src/proto/streams/store.rs:15:10
   |
15 |     ids: IndexMap<StreamId, SlabIndex>,
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 3 type arguments

error: aborting due to previous error

For more information about this error, try `rustc --explain E0107`.
error: could not compile `h2`.

What I found about this error, I found in an unrelated software tool's repository, issue here.

The root issue I am having is that I am building from an nfs mount that does not support extended file attributes, and so as a result the build of IndexMap is failing with the following:

rror: could not copy "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe0.probe0.3a1fbbbh-cgu.0.rcgu.ll" to "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe0.ll": Operation not supported (os error 95)

error: aborting due to previous error

error: could not copy "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe1.probe1.3a1fbbbh-cgu.0.rcgu.ll" to "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe1.ll": Operation not supported (os error 95)

error: aborting due to previous error

warning: autocfg could not probe for `std`
error: could not copy "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe2.probe2.3a1fbbbh-cgu.0.rcgu.ll" to "/workspace/herald/target/debug/build/indexmap-62ff17e61db67c74/out/probe2.ll": Operation not supported (os error 95)

error: aborting due to previous error

From the issue #144 it sounds like that problem originated in autocfg but I'm not certain if this is the same situation here, as the other 260 includes I have for my package compiled without incident. I'm curious if there might be some part of IndexMap's compilation that might be prompting this attempt to read extended attributes and if possible, is there a way to skip that?

New release

Hi.

Is a new release planned soon? I could really use HashSet::insert_full() in my code.

Full Consistency With HashMap

Missing methods (stable):

• with_hasher #35
• hasher #35
• shrink_to_fit
• is_empty #22
• drain #27
• values, values_mut #17

New methods:

As a rule, we don't implement unstable methods; they are implemented when they are stable in Rust beta

• get_key_value rust-lang/rust/issues/49347
• Entry::and_modify (unstable)
• Entry::or_default (unstable)
• try_reserve rust-lang/rust/issues/48043
• remove_entry rust-lang/rust/issues/46344

Trait impls

• PartialEq + Eq #23
• Extend<(&K, &V)> #34
• Ser/De #28
• ESI for iterators #26
• Debug for Entry
• Clone for some iterators (Keys, Values, Iter, probably all of them)

Type Conflicts:

• Entry<K, V> vs Entry<K, V, S> (not important) #59
• retain's &K vs &mut K #39
• drain's signature (not important)
• OccupiedEntry::insert's signature #68

Rayon integration

Ordermap's simple data structure again makes it easy to implement awesome stuff for?

Feature request: Entry::get_index

In the Entry API, OccupiedEntry should expose a function to get the index of the entry.

BLOCKER: Clarify the role of entry_index, maybe rename

Before we cut any new release, we should resolve the questions on the new entry_index.

@bluss #115 (comment) :

Wait - this method seems underspecified and surprising? Both the PR and the doc comment needs to explain in detail what it does.

The method should not be called entry_* when it is more similar to get_full than to entry. I can guess that the motivation for adding this is performance, just having a simpler return type than get_full, is not enough to warrant inclusion.

@cuviper #115 (comment) :

Yes, the main advantage I see is to avoid the indexing operation in get_full, which seems reasonable to me. The author originally called it get_index, which might be a better name, but that method already exists for using an index to get the key-value.

cc @Thermatix

Implement serde::IntoDeserializer for IndexMap/IndexSet

It must be trivial because serde already has generic MapDeserializer: https://docs.serde.rs/src/serde/de/value.rs.html#1254-1266

Feature request: `.values_mut_range(idx..)`

I was hacking on the compiler and I wrote:

                    // We've rejected overlap in each product in the sum.
                    // Now we must account for the possibility that or-pattern is a factor
                    // in a product. A basic case to reject here is `(V1(a) | V2(a), a)`.
                    let last_id = *prod_ids.last().unwrap();
                    bindings.values_mut().skip(len_before).for_each(|val| *val = last_id);

which is nice.

Maybe bindings.values_mut_range(len_before..).for_each(..) would be nicer?

Feature Request: `iter_full` and `iter_full_mut`

I've recently run into a situation where I needed access to the key, mutable value, and position all at the same time. I found myself fighting the borrow-checker trying to put something together with some combination iter, iter_mut, get_full and get_full_mut, and in the end I had to give up and manually track positions outside of the iter loop.

Given all that, it would be super handy to have iter_full and iter_full_mut that would return a triplet of (key, value, position).

Signature of OccupiedEntry::insert

• OccupiedEntry::insert for HashMap
• OccupiedEntry::insert for BTreeMap
• OccupiedEntry::insert for LInkedHashMap
• OccupiedEntry::insert for IndexMap

The first three all take &mut self while indexmap takes self by value. Is this intentional? I found this makes it complicated for a library to multiplex across different choices of underlying map type: serde-rs/json#418.

My workaround is:

mem::replace(occupied_entry.get_mut(), value)

General hashes-like relation

• Example: Hash table that is keyed by newtyped strings and lookup is case-insensitive.
• Use case: Want to lookup using &str.

minor different behavior from HashMap

summary

I'm not sure what the issue here is, but autodereferencing doesn't seem to be working for &&T -> &T

example

I hit this while migratign from HashMap to OrderMap. I'm mostly opening this issue because I was confused about what was even happening and it might be useful to others. I was under the impression that rust did this kind of dereferencing automatically, but it appears not to be happening in this case.

The following code:

#[macro_use]
extern crate ordermap;

#[test]
fn test_it() {
    let key = "foo".to_string();
    let mut m = ordermap!{key.clone() => 3};
    assert!(m.contains_key(&&key));
}

fn main() {
    println!("yay");
}

results in this error:

error[E0277]: the trait bound `std::string::String: std::borrow::Borrow<&std::string::String>` is not satisfied
 --> src/main.rs:8:15
  |
8 |     assert!(m.contains_key(&&key));
  |               ^^^^^^^^^^^^ the trait `std::borrow::Borrow<&std::string::String>` is not implemented for `std::string::String`
  |
  = help: the following implementations were found:
            <std::string::String as std::borrow::Borrow<str>>
  = note: required because of the requirements on the impl of `ordermap::Equivalent<std::string::String>` for `&std::string::String`

1.0.3 release?

What is needed to cut a 1.0.3 release?

I need the clonable iterators feature introduced in d8e5225, as it provides parity with HashMap. I'd prefer to not depend on a git revision if possible.

Is there some way I could help out on preparing a release?

(awesome create btw. I just sped up an application by > 30% by just switching to indexmap!)

Question: Indexmap and CLion

Hello,

Thanks for the excellent crate!

I have one question related to the usage of the crate with JetBrains CLion. Somehow the intellisense doesn't work very nicely with it, and the IDE doesn't recognise the output types of functions such as IndexMap::get(). Is there anything that can be done to improve that experience?

Thanks a lot

32bit support?

hi,

is 32bit support on the roadmap? Im building a webapp and discovered that ordermap fails at compile time. I guess that's because of web/emscripten being a 32bit platform.

warning: this expression will panic at run-time
   --> .../.cargo/registry/src/github.com-1ecc6299db9ec823/ordermap-0.2.7/src/lib.rs:144:29
    |
144 | 
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^ attempt to shift left with overflow

Implement `serde` traits under `#[cfg(features = "serde")]`

Instead of #[cfg(features = "serde-1")] which is bit counter-intuitive and not in consistency with rust ecosystem.

Implementation of `PartialEq`, `Eq`, `PartialOrd` and `Ord` for `IndexMap`

Hey, I'd like to replace BTreeMap by IndexMap to preserve the order of deserialized elements in RON. However, IndexMap lacks PartialEq, Eq, PartialOrd and Ord.

Is that intentional or just not implemented yet?

Adding `.reverse()` method to `IndexMap` & `IndexSet`

Looking at the impl of OrderMapCore, .sort_by() saves the hash index, sorts the underlying entries vector, and then restores the hash index. Would this mean that a .reverse() method could be added to OrderedMapCore (and thus IndexMap/IndexSet)?

Make this crate no_std

Using the new alloc crate released with Rust 1.36 it is possible to make the IndexMap independent from std.
It is suggested to make library crates use only the alloc crate in order to be compatible with the #![no_std] ecosystem

Feature request: Retain Iterator

Have retain return a iterator over the extracted values, similar to Drain in the standard library, or my PreRFC.

It seems like a relatively simple change, so if you're interested I could probably get it done and shoot you a PR later this week.

Possible changes for 2.0

Let's use this issue to discuss changes for a hypothetical 2.0.

Proposed

• Increase the MSRV from 1.18.
  • This is not strictly breaking, as we did state the intent to allow increases in minor releases, but in 2.0 we can do this without any fear of harming our dependents.
    • In particular, serde_json supports Rust 1.31.
  • #104 wants 1.28 for RangeBounds.
  • #131 wants 1.32 for hashbrown.
  • 1.34 adds TryFrom/TryInto which may be useful for custom indexing.
  • With 1.36, we could assume alloc support.
  • DONE: We went ahead with MSRV 1.32 in indexmap 1.5, and MSRV 1.36 in 1.6
• Change the default hasher to something faster.
  • e.g. hashbrown defaults to ahash.
  • Especially with MSRV 1.36, a no_std default would be nice.
  • We can wrap it to avoid committing to a specific public dependency.
• Parameterize the index type.
  • While we could try to default Idx = usize, this doesn't always work well with type inference.
• Maybe change the default remove()?
  • (bluss) I still think the current decision is fine, no change from my end for 2.0. (Good way to document this would be to say that "remove is the fastest way to remove an element, and if order should be preserved, ...")
• std crate feature
• Consider ordered PartialEq + Eq, more like Vec than HashMap (#153, #154).
  • This would also allow PartialOrd, Ord, and Hash.
• get_index_mut should return &K instead of &mut K (#160 (comment))

Question: Are there any plans for primitive-type keys?

Hello,
I wrote my own implementation of k-nucleotide benchmark task (like many do), and looks like it is ~5-9% faster that the current top Rust#6 solution:
https://github.com/inv2004/knucleotide-rs

The main difference: I do not use u64 key anymore, but u8, u16, u32 and u64, depends on the size of string.

Unfortunately I found that the main problem here is hashmap-indexmap implementation for primitive numbers. A lot of branches in perf:
C++: https://ptpb.pw/yOYB
Rust: https://ptpb.pw/UCjl

Are there any plans on it ?

Thank you,

`indexmap` is rebuilt on every `cargo` run

From what I can tell, the changes in #106 caused one of our CI jobs' run times to shoot up from ~23 minutes (while depending on indexmap 1.1.0) to 1 1/2 - 2 hours (about 4x, while depending on indexmap 1.3.2).

Due to the shortcomings of cargo we need to run it 4 times in succession to cover all bases:

$ cargo build --release --all-targets      # Build production code
$ cargo test --no-run --release            # Build tests and doc tests
$ cargo test --no-run --release --benches  # Build benchmarks
$ cargo test --release                     # Run tests and doc tests

Normally this is not an issue, as the cargo build at the top usually builds everything, the next two are no-ops, and the last one again doesn't need to build anything, only run the tests. But after bumping indexmap to 1.3.2 in Cargo.lock, suddenly each of those cargo commands first starts rebuilding indexmap and then everything that depends on it, essentially doing the exact same thing 4 times.

AFAICT that is because of the use of autocfg in build.rs. Every time cargo is run, it thinks indexmap has been modified and rebuilds it. Which in turn causes everything that depends on it to be rebuilt. Although nothing at all has changed.

I don't have a solution and I've already spent way too much time tracking this down and finding a workaround (rolled back indexmap to 1.2.0), but it would be great if you could find a way to convince cargo that nothing has changed and it doesn't need to rebuild indexmap every single time.

(BTW, this is not only an issue for CI, any local build or test run will also start by rebuilding indexmap and go from there.)

Feature request: index_of(&self, key: &Q)

A method that returns the index of the entry stored for key, if it is present, else None.
I would suggest a signature like:
index_of(&self, key: &Q) -> Option<usize>

Move mutable key access to a trait

We should put this feature a bit on the side, in a trait.

Something like:

impl Ordermap {
    fn get_pair_mut(&mut self, key: &Q) -> (&K, &mut V) { .. }
}

impl MutableKeys for OrderMap {
    fn good_method_name(&mut self, key: &Q) -> (&mut K, &mut V) {  .. }
}

indices can be a `Box<[_]>`

Saves 8 bytes in the pretty large OrderMap struct (currently 72 bytes for 64-bit using the default siphash with a 16-byte key).

Implement in place sorting of OrderMap and OrderSet

Feature request: Add in-place sorting methods to OrderMap (and OrderSet). Similar to the existing .sorted_by, these methods can allow the user to sort by either keys or values or both by supplying all of those to the closure.

For example:

fn sort_by<F>(&mut self, cmp: F)
    where F: FnMut(&K, &V, &K, &V) -> Ordering;

Original issue:

---

Feature request: add sort_keys<K:Ord+Hash>(&mut self) to do an inplace sort.

This library is INCREDIBLE when assertions need to be made because I can do an inplace sort and the diffs look good (when combined with pretty_assertions of course).

I did a naive implementation like this:

pub fn sort_ordermap<K:Ord+Hash, V>(m: &mut OrderMap<K, V>) {
    let mut ordered: Vec<_> = m.drain(..).collect();
    ordered.sort_by(|left, right| left.0.cmp(&right.0));
    m.extend(ordered.drain(..));
}

pub fn sort_orderset<K:Ord+Hash>(m: &mut OrderSet<K>) {
    let mut ordered: Vec<_> = m.drain(..).collect();
    ordered.sort_by(|left, right| left.cmp(&right));
    m.extend(ordered.drain(..));
}

I'm sure a better one can be done with full access to the underlying data structure.

Use implementation details to make the ordermap!() macro efficient

It probably already is, but you know, maybe it can be even more.

Consistency with HashMap: missing `values` iterator.

I'm trying to use ordermap to fix the non-determinism in rustbuild in a useful (order-preserving) way, and I had to replace .values() with .iter().map(|(_, r)| r) - it's close, but not a drop-in replacement.

push README+description update on ordermap to link to indexmap

The description in Cargo.toml should mention that it was renamed as well as README to prevent people from accidentally using it.

Thanks!

panic assertion failed intermitent crash compiled to WASM and running with libwebkit2gtk on Linux

I was referred from rust-lang/hashbrown#194 to report this issue here too

Compiled using wasm-bindgen, via wasm-pack using the web target, and the --dev profile. This code is running in the webkit2gtk 2.28.4-1 browser on Archlinux.

Stack trace:

panicked at 'assertion failed: `(left != right)`
  left: `0`,
 right: `0`', ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/macros/mod.rs:16:9

<?>.wasm-function[console_error_panic_hook::Error::new::h11710926359e6a99]@[wasm code]
<?>.wasm-function[console_error_panic_hook::hook_impl::h43ac723b12164d7a]@[wasm code]
<?>.wasm-function[console_error_panic_hook::hook::h1509fc86d60f8ecc]@[wasm code]
<?>.wasm-function[core::ops::function::Fn::call::h63e0941c1aa82381]@[wasm code]
<?>.wasm-function[std::panicking::rust_panic_with_hook::h3e6619a8809a443a]@[wasm code]
<?>.wasm-function[rust_begin_unwind]@[wasm code]
<?>.wasm-function[core::panicking::panic_fmt::ha8209066b105fcdd]@[wasm code]
<?>.wasm-function[hashbrown::raw::inner::RawTable<T>::bucket::hfcb3cc32fba2684b]@[wasm code]
<?>.wasm-function[hashbrown::raw::inner::RawTable<T>::insert::h71a2e674faab18bc]@[wasm code]
<?>.wasm-function[indexmap::map::core::IndexMapCore<K,V>::push::h4f42c6dc20fce8fb]@[wasm code]
<?>.wasm-function[indexmap::map::core::IndexMapCore<K,V>::insert_full::h6cc9bd7b0982815d]@[wasm code]
<?>.wasm-function[indexmap::map::IndexMap<K,V,S>::insert_full::he19372e70f82483d]@[wasm code]
<?>.wasm-function[indexmap::map::IndexMap<K,V,S>::insert::h007a7c31241165e3]@[wasm code]
<?>.wasm-function[<indexmap::map::IndexMap<K,V,S> as core::iter::traits::collect::Extend<(K,V)>>::extend::{{closure}}::h396d242148f62446]@[wasm code]
<?>.wasm-function[core::iter::traits::iterator::Iterator::for_each::call::{{closure}}::hbb035c70f8777522]@[wasm code]
<?>.wasm-function[core::iter::adapters::map_fold::{{closure}}::h62be75b9f0a27adf]@[wasm code]
<?>.wasm-function[core::iter::adapters::filter_fold::{{closure}}::h268ff1624341ca8a]@[wasm code]
<?>.wasm-function[core::iter::adapters::map_fold::{{closure}}::hcf5aea4a2ef5e328]@[wasm code]
<?>.wasm-function[core::iter::traits::iterator::Iterator::fold::h853140cd02b11d69]@[wasm code]
<?>.wasm-function[<core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::hf5d0ea58d414ae46]@[wasm code]
<?>.wasm-function[<core::iter::adapters::Filter<I,P> as core::iter::traits::iterator::Iterator>::fold::hfe2842f31b010ace]@[wasm code]
<?>.wasm-function[<core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::h64e7ef0a45baccd6]@[wasm code]
<?>.wasm-function[core::iter::traits::iterator::Iterator::for_each::h5ba4ea97dae030ca]@[wasm code]
<?>.wasm-function[<indexmap::map::IndexMap<K,V,S> as core::iter::traits::collect::Extend<(K,V)>>::extend::h413750f7b641b695]@[wasm code]
<?>.wasm-function[<indexmap::map::IndexMap<K,V,S> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter::h49e190a53d106e85]@[wasm code]
<?>.wasm-function[<indexmap::set::IndexSet<T,S> as core::iter::traits::collect::FromIterator<T>>::from_iter::hc8ac9b46dc06688c]@[wasm code]
<?>.wasm-function[core::iter::traits::iterator::Iterator::collect::h6577e1d6120a2276]@[wasm code]
<?>.wasm-function[<yew::virtual_dom::Classes as core::convert::From<&str>>::from::hdb0a0b180f292b8c]@[wasm code]
<?>.wasm-function[<T as core::convert::Into<U>>::into::h22798bc94b25b76c]@[wasm code]

It's possible this may be related to rust-random/rand#1016 Perhaps it's a stability issue in libwebkit2gtk.