tessil / hopscotch-map Goto Github PK

View Code? Open in Web Editor NEW

723.0 723.0 64.0 1.59 MB

C++ implementation of a fast hash map and hash set using hopscotch hashing

License: MIT License

CMake 1.96% C++ 98.04%

c-plus-plus cpp data-structures hash-map hash-table header-only

hopscotch-map's People

Contributors

Stargazers

Watchers

hopscotch-map's Issues

Possible issue with iterator

Hi,

I have a code similar to that one:

tsl::hopscotch_map<std::string, std::vector<int>> map;
auto it = map.find("aaa");
if(it != std::end(map)) {
  auto& v = it->second;
  // modify v
}

It seems it does not work for me (right now I am testing your branch with additional hash argument for find() operation). I get const vector from non-const iterator. It is probably caused by:

        using value_type = const typename hopscotch_hash::value_type;

in https://github.com/Tessil/hopscotch-map/blob/add_hash_parameter/src/hopscotch_hash.h#L610 and directly using std::vector and std::list iterators on std::pair<key, value>.

Can you confirm that is an issue?

begin() slows down when the container size decreases

It seems that method begin() becomes slower after I remove some elements. Here is a simple test https://gist.github.com/andrusha97/de6b63f48aca45a6b04b4e968c34289b
I guess it's because this method scans buckets sequentially until it finds a non-empty one, and erase never reduces the number of buckets. Also if this is the case, iteration over all elements probably takes O(c.capacity()) and not O(c.size()).

Could you confirm that I understand things right? I'd like iterators to have more "traditional" complexities. Could you recommend how to deal with this? Should I monitor load factor and shrink containers manually?

set::erase crash

Hello,

We are hitting a crash when calling erase on the set implementation.

Exception thrown: read access violation.
this was nullptr.

Here is the stack trace

>	Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0>::neighborhood_infos() Line 487	C++	Symbols loaded.
 	Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::find_in_buckets<unsigned int>(const unsigned int & key, unsigned __int64 hash, std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0> > > > it_bucket) Line 1772	C++	Symbols loaded.
 	Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::find_in_buckets<unsigned int>(const unsigned int & key, unsigned __int64 hash, std::_Vector_iterator<std::_Vector_val<std::_Simple_types<tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0> > > > it_bucket) Line 1760	C++	Symbols loaded.
 	Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::erase<unsigned int>(const unsigned int & key, unsigned __int64 hash) Line 1124	C++	Symbols loaded.
 	Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::erase<unsigned int>(const unsigned int & key) Line 1117	C++	Symbols loaded.
 	Server_r.exe!tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::erase(const unsigned int & key) Line 279	C++	Symbols loaded.

Thanks !

Thread-safety ?

Which guarantees does the data structures offer ? Are they at least at strong (so not very strong) than equivalent std:: containers ? (i.e. multiple threads doing only lookups in the map at the same time should be safe).

Thanks !

Occasional map misbehavior, inserted element not found later on

Hello,

We are experiencing a weird misbehavior with tsl/hopscotch_map, bisected to commit 5c5770a being the culprit.

According to its changelog entry, this looks like an innocent commit purely renaming variables, namely m_buckets to m_buckets_data and m_first_or_empty_bucket to m_buckets. However, danger lurks around in one of the old names being the same as the other new name.

At several places (e.g. hopscotch_hash.h line 1246 being the first one) m_buckets remains m_buckets. Based on the changelog's wording, these look like accidental mistakes rather than intentional "changes".

Into our tsl::hopscotch_map<int, void *> we insert about 1000 entries (using map[key] = val) and erase about 800 (using it = map.find(key); blahblah; map.erase(it) and then no longer using it afterwards). These operations interleave each other in kinda random order, and we sometimes re-add a key that we erased earlier.

Then, after one insertion, the following situation arises:

If I iterate through all the items using for (auto it : map) {...} then I get all the about 200 items, as expected.

However, for one of these entries map.count(key) gives 0 instead of the expected 1. Trying to access this item using .at(key) raises an exception, as usual for missing entries. Or, trying to access this key using [key] inserts this item again, with the default value of nullptr. After this operation, if I loop through the items again using for (auto it : map) {...} then this key gets listed twice, once with the expected value and once with nullptr. And at this point it becomes another key that is "missing" in this weird sense that looping over all the entries finds it, but looking it up directly does not.

Unfortunately I couldn't come up with a self-contained test demonstrating the failure, and I cannot give you access to our in-house software that every now and then triggers this bug. Our software has a very non-deterministic behavior by its nature, but as of your aforementioned commit we see the misbehavior about 1 out of 20-50-ish cases. We haven't seen misbehavior with your hopscotch_map prior to the mentioned commit during thousands of runs, nor with the standard unordered_map.

Let me know if you need any help in debugging this issue (without me digging deeply into your code :)). For example, if you create a version that dumps into a global fd the memory region of the map after every insertion/deletion, I'd be happy to test that. But maybe you just want to revert the functionality and redo the variable naming properly :)

Thanks in advance!

Different behavior compared with std::unordered_map

#include <iostream>
#include <string>
#include "hopscotch_map.h"
#include <unordered_map>

using namespace tsl;
using namespace std;

struct Test { 

};

int main() {

    unordered_map<string, Test> m;

    auto it = m.find("a");
    Test& r = it->second;
    Test t = move(r);

    hopscotch_map<string, Test> m2;

    auto it2 = m2.find("a");
    Test& r2 = it2->second;
    Test t2 = move(r2);

    return 0;
}

Which produce error

main.cc: In function 'int main()':
main.cc:24:21: error: binding reference of type 'Test&' to 'const Test' discards qualifiers
     Test& r2 = it2->second;
                ~~~~~^~~~~~

But it2->second should not be const.
sparse_map also have the same problem.

operator= is broken

The following won't build on current git head:

tsl::hopscotch_map<std::string, std::string> s;
tsl::hopscotch_map<std::string, std::string> s2 = s; // initialization is ok
s2 = s; // operator= fails

It worked by f5b7e62 .

A quick bisect told me that fd765fb is the commit that introduced breakage; 838413d works.

The CMake configuration is not compatible with CMake 3.1.

The CMake configuration has its minimum required version set to 3.1. However, the cxx_std_11 constant passed to target_compile_features has only been introduced in CMake 3.8. The result is that for versions 3.1 to 3.7 an error will appear that this value does not exist.

The easiest solution would be to increase the minimum required version to 3.8. I have looked for other solutions to require C++11, but they are not really satisfying. It is not possible to require the C++11 compiler switch via the CXX_STANDARD property for interface libraries. Another, more involved, attempted solution was to specifically determine what compiler features the project needs from the list of features available in 3.1. This includes for example cxx_auto_type and cxx_nullptr. However, CMake only knows about these features for GCC and Clang, so that is also not an option.

Another kind of solution would be that the target feature will only be set when configuring with CMake 3.8 via a version condition. For versions 3.1 to 3.7 it could print a warning, determine whether the CXX_STANDARD variable was set or it would simply fail during compilation.

Move-Only, Non-Default-Constructible Types not supported as values

The documentation states that there's "Support for move-only and non-default constructible key/value", however, unless I'm mistaken in what is meant with that, I don't think that is true or needs further clarification.

Edit: I did some more digging and I'm going with "needs further clarification". The issue is that the move / copy constructor has to be defined noexcept in order to be used by the hopscotch_map. This is not API compatible with std::unordered_{map,set} and should be noted as such.

I've cooked up a small example (see below) which does not compile with error

[...] error: call to implicitly-deleted copy constructor of 'value_type' [...]

I'm on Mac OSX 10.11.6 with the system compiler; clang++ --version reports

Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I'm attempting to compile the example with

clang++ -W -Wall -Wextra -std=c++11 -I hopscotch-map -o test test.cpp

I have uploaded the full error log here.

#include <memory>

struct MoveOnly {
    explicit MoveOnly(int i)
        : m_data(i){}
    MoveOnly(MoveOnly && other)
        : m_data(std::move(other.m_data)){}
    MoveOnly(const MoveOnly &) = delete;
    MoveOnly & operator=(const MoveOnly &) = delete;
    MoveOnly & operator=(MoveOnly &&) = delete;
private:
    int m_data;
};

#include "hopscotch_map.h"

#include <string>

class Foo {
public:
    tsl::hopscotch_map<std::string, MoveOnly> m_db;
    Foo()
        : m_db(){}
};

int main(){
    Foo f;
}

Key and Value Iterators

Hello,

I love these map libraries and am using them extensively, thank you so much for sharing your work!

I am frequently running into the need to iterate over the keys or values separate of the other. Is there any way this could be implemented for this map?

Currently I have implemented utility functions to gather the keys or values into a vector and then use that, but I'd like to do this iteration in a functional programming kind of way (so without a for loop and a map() or transform() instead) and also without taking up the memory to copy all the keys or values into a separate vector.

Let me know your thoughts on this, and thanks again for your hard work!

bug on building a simple map

Dear Tessil,
I spent nearly one day on a bug and finally find it may come out from the hopscotch-map.
Here is a test file extracted from our log and a simple test program to help you to reproduce it.
The key file contains 187414 unique values, and expected the map should have the same number of size. However, the size of built map is 187353.
https://www.dropbox.com/s/hf35bq88t7ixve0/keys.dat?dl=0
https://www.dropbox.com/s/a62cdfq3kr3nuao/maptest.cpp?dl=0

could you help to debug it? Many thanks!

VS C4503 warning

When I try to use tsl::hopscotch_map instead of std::unordered_map I often end up with VS C4503 warning. Do you have any good BKMs on how to handle such issues?

Support for older GCC versions (-std=c++0x)

Hi,

It seems compiling this library using old GCC version with incomplete C++11 support (-std=c++0x) is not currently possible. Are there any plans to support this?

Cheers!

Bad insertion time of uint64_t on the latest version (February 7th)

Hi,

I tried your project yesterday and compared its performance on Windows (VS2015) and Linux (gcc 5.3) vs std::unordered_map. I have 2 use cases for some 'struct A':
a) tsl::hopscotch_map<std::string, A&> or tsl::hopscotch_map<gsl::cstring_span<>, A&>
b) tsl::hopscotch_map<uint64_t, A&>
I must say that for first use case hopscotch_map performed much better than std::unordered_map on both Windows and Linux. However the second use case on Linux has really bad performance. I measured the time of 70.000 insertions of unique uint64_t key/values pairs (and verified they have unique hash values too) and then the cleanup of such a container. The result was more than 100x times worse performance of tsl::hopscotch_map. It looks strange to me as use case a) works at the same speed. I tried to analyze the issue with linux perf tools but didn't have much time for that. Do you have any ideas what is wrong?

Feature request: Heterogeneous overloads for insertion.

tsl::hopscotch_map already provides heterogeneous overloads for lookup (e.g. via find or at) but there is currently no such support for adding elements (e.g. via operator[] or try_emplace). AFAIK, corresponding overloads were added to
the standard associative containers with C++26.

@Tessil Do you think it makes sense to also extend tsl::hopscotch_map? From what I can see, it seems that the underlying tsl::hopscotch_hash already has the necessary functionality, so I drafted a first version for try_emplace and operator[] here. I only extended tsl::hopscotch_map as tsl::bhopscotch_map seems to sometimes copy keys anyways.

If you think this makes sense, I could also open a PR.

Support Vcpkg library manager

Microsoft has created a C++ Library Manager for Windows, Linux, and MacOS.
https://github.com/Microsoft/vcpkg

range-based for loops

This is a minor thing, but it's been bugging me for a little while. One of the differences with std::unordered_map is that value_type is const std::pair<Key,T> instead of std::pair<const Key,T>. Iterators allow us to work around this with the value() method, which gives us a mutable reference to the mapped type.

This works fine with traditional for loops and iterators.

for (auto it = map.begin(); it != map.end(); ++it) {
    auto &second = it.value();
    ...
}

However, range-based for loops don't work as nicely, and AFAIK the only way to get a mutable reference to the mapped type in that case is to const_cast it:

for (auto &value : map) {
    auto &second = const_cast<T&>(value.second);
    ...
}

I'm sure there's a reason why value_type has to be const (sorry if I missed it), but it's a bit unfortunate that we can't use range-based for loops as straightfordwardly as with std::unordered_map. Apart from that, great library! 👍

Infinite loop with insert / emplace on MSVC

Certainly the fault of a dumb compiler, but still : given hopscotch_map<void*>, I get an infinite recursion between these two functions:

template<class P, typename std::enable_if<std::is_constructible<value_type, P&&>::value>::type* = nullptr>
std::pair<iterator, bool> insert(P&& value) { 
    return emplace(std::forward<P>(value)); 
}

template<class... Args>
std::pair<iterator, bool> emplace(Args&&... args) {
    return insert(value_type(std::forward<Args>(args)...));
}

std::aligned_storage deprecated in C++23

When compiling in C++23 mode, there is a deprecation warning. What about removing std::aligned_storage and use an unsigned char instead? (With C++17, also std::byte could be used.)

diff --git a/include/tsl/hopscotch_hash.h b/include/tsl/hopscotch_hash.h
index ed5f858..cf87532 100644
--- a/include/tsl/hopscotch_hash.h
+++ b/include/tsl/hopscotch_hash.h
@@ -386,11 +386,9 @@ class hopscotch_bucket : public hopscotch_bucket_hash<StoreHash> {
   }
 
  private:
-  using storage = typename std::aligned_storage<sizeof(value_type),
-                                                alignof(value_type)>::type;
 
   neighborhood_bitmap m_neighborhood_infos;
-  storage m_value;
+  alignas(value_type) unsigned char m_value[sizeof(value_type)];
 };
 
 /**

compile failed if use with sparse-map

reproduce code:

#include <string>

#include "sparse-map/tsl/sparse_map.h"
#include "hopscotch-map/include/tsl/hopscotch_map.h"


class Value;

typedef tsl::sparse_map<std::string, Value> SMap;
typedef tsl::hopscotch_map<std::string, Value> Map;

// general value class
class Value {

public:

    Value& operator=(const SMap& v) noexcept {
        smap = new SMap(v);
        return *this;
    }

    Value& operator=(const Map& v) noexcept {
        map = new Map(v);
        return *this;
    }

private:
        Map* map;
        SMap* smap;
};


int main() {
        Value xx;
        return 0;
}

error message:

In file included from test.cc:4:
In file included from ./hopscotch-map/include/tsl/hopscotch_map.h:36:
./hopscotch-map/include/tsl/hopscotch_hash.h:405:51: error: invalid application of 'sizeof' to an incomplete type 'tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<std::__cxx11::basic_string<char>, Value>, 62, false>::value_type' (aka 'std::pair<std::__cxx11::basic_string<char>, Value>')
    using storage = typename std::aligned_storage<sizeof(value_type), alignof(value_type)>::type;
                                                  ^~~~~~~~~~~~~~~~~~
./hopscotch-map/include/tsl/hopscotch_hash.h:463:42: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<std::__cxx11::basic_string<char>, Value>, 62, false>' requested here
    using neighborhood_bitmap = typename hopscotch_bucket::neighborhood_bitmap;
                                         ^
./hopscotch-map/include/tsl/hopscotch_map.h:119:31: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::KeySelect, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2>, std::__cxx11::list<std::pair<std::__cxx11::basic_string<char>, Value>, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> > > >' requested here
    using key_type = typename ht::key_type;
                              ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1061:56: note: in instantiation of template class 'tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >' requested here
      : public __bool_constant<__is_assignable(_Tp, _Up)>
                                                       ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1073:14: note: in instantiation of template class 'std::is_assignable<Value &, const Value &>' requested here
    : public is_assignable<_Tp&, const _Tp&>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1079:14: note: in instantiation of template class 'std::__is_copy_assignable_impl<Value, true>' requested here
    : public __is_copy_assignable_impl<_Tp>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:144:14: note: (skipping 5 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
    : public conditional<_B1::value, _B2, _B1>::type
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1049:14: note: in instantiation of template class 'std::is_nothrow_constructible<std::pair<std::__cxx11::basic_string<char>, Value>, std::pair<std::__cxx11::basic_string<char>, Value> &&>' requested here
    : public is_nothrow_constructible<_Tp, _Tp&&>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1055:14: note: in instantiation of template class 'std::__is_nothrow_move_constructible_impl<std::pair<std::__cxx11::basic_string<char>, Value>, true>' requested here
    : public __is_nothrow_move_constructible_impl<_Tp>
             ^
./sparse-map/tsl/sparse_hash.h:833:24: note: in instantiation of template class 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >' requested here
    static_assert(std::is_nothrow_move_constructible<ValueType>::value ||
                       ^
./sparse-map/tsl/sparse_map.h:119:31: note: in instantiation of template class 'tsl::detail_sparse_hash::sparse_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::KeySelect, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium, tsl::sh::probing::quadratic>' requested here
    using key_type = typename ht::key_type;
                              ^
test.cc:18:20: note: in instantiation of template class 'tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>' requested here
        smap = new SMap(v);
                   ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/bits/stl_pair.h:198:12: note: definition of 'std::pair<std::__cxx11::basic_string<char>, Value>' is not complete until the closing '}'
    struct pair
           ^
In file included from test.cc:4:
In file included from ./hopscotch-map/include/tsl/hopscotch_map.h:36:
./hopscotch-map/include/tsl/hopscotch_hash.h:1233:67: error: no member named 'value' in 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >'
    static_assert(std::is_nothrow_move_constructible<value_type>::value || 
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
./hopscotch-map/include/tsl/hopscotch_map.h:119:31: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::KeySelect, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2>, std::__cxx11::list<std::pair<std::__cxx11::basic_string<char>, Value>, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> > > >' requested here
    using key_type = typename ht::key_type;
                              ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1061:56: note: in instantiation of template class 'tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >' requested here
      : public __bool_constant<__is_assignable(_Tp, _Up)>
                                                       ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1073:14: note: in instantiation of template class 'std::is_assignable<Value &, const Value &>' requested here
    : public is_assignable<_Tp&, const _Tp&>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1079:14: note: in instantiation of template class 'std::__is_copy_assignable_impl<Value, true>' requested here
    : public __is_copy_assignable_impl<_Tp>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:144:14: note: in instantiation of template class 'std::is_copy_assignable<Value>' requested here
    : public conditional<_B1::value, _B2, _B1>::type
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/bits/stl_pair.h:368:3: note: (skipping 4 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
                __and_<is_copy_assignable<_T1>,
                ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1049:14: note: in instantiation of template class 'std::is_nothrow_constructible<std::pair<std::__cxx11::basic_string<char>, Value>, std::pair<std::__cxx11::basic_string<char>, Value> &&>' requested here
    : public is_nothrow_constructible<_Tp, _Tp&&>
             ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1055:14: note: in instantiation of template class 'std::__is_nothrow_move_constructible_impl<std::pair<std::__cxx11::basic_string<char>, Value>, true>' requested here
    : public __is_nothrow_move_constructible_impl<_Tp>
             ^
./sparse-map/tsl/sparse_hash.h:833:24: note: in instantiation of template class 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >' requested here
    static_assert(std::is_nothrow_move_constructible<ValueType>::value ||
                       ^
./sparse-map/tsl/sparse_map.h:119:31: note: in instantiation of template class 'tsl::detail_sparse_hash::sparse_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::KeySelect, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium, tsl::sh::probing::quadratic>' requested here
    using key_type = typename ht::key_type;
                              ^
test.cc:18:20: note: in instantiation of template class 'tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>' requested here
        smap = new SMap(v);
                   ^
2 errors generated.

compiler:

clang6.0 / gcc8.1 can not compile this code
vs 2017 can compile this code

But following code is OK

#include <string>

#include "sparse-map/tsl/sparse_map.h"
#include "hopscotch-map/include/tsl/hopscotch_map.h"


class Value;

typedef tsl::sparse_map<std::string, Value> SMap;
typedef tsl::hopscotch_map<std::string, Value> Map;

// general value class
class Value {

public:

    Value& operator=(const SMap& v) noexcept {
        smap = new SMap(v);
        return *this;
    }

/*
    Value& operator=(const Map& v) noexcept {
        map = new Map(v);
        return *this;
    }
*/

private:
        Map* map;
        SMap* smap;
};


int main() {
        Value xx;
        return 0;
}

MS VS 2017 compiler error

Simplest code produces compilation error (MS VS 2017 Version 15.9.17).

tsl::bhopscotch_map<uint64_t, uint64_t> m3;

m3[10] = 100;

Severity Code Description Project File Line Suppression State
Error C2440 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &' fastest_hash c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h 538
Error C2440 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &' fastest_hash c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h 541

Full output log:
1>------ Build started: Project: fastest_hash, Configuration: Release x64 ------
1>main.cpp
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(538): error C2440: 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &'
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(538): note: Conversion loses qualifiers
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(1073): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::hopscotch_iterator::value<ValueSelect,0x0>(void) const' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(1071): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::hopscotch_iterator::value<ValueSelect,0x0>(void) const' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::operator []<unsigned __int64,ValueSelect,0x0>(K &&)' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,
1> K=unsigned __int64
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::operator []<unsigned __int64,ValueSelect,0x0>(K &&)' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,
1> K=unsigned __int64
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: while compiling class template member function 'T &tsl::bhopscotch_map<uint64_t,T,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::operator [](Key &&)'
1> with
1> [
1> T=uint64_t,
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\main.cpp(121): note: see reference to function template instantiation 'T &tsl::bhopscotch_map<uint64_t,T,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::operator [](Key &&)' being compiled
1> with
1> [
1> T=uint64_t,
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\main.cpp(119): note: see reference to class template instantiation 'tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_growth_policy.h(202): note: see reference to class template instantiation 'std::array<size_t,40>' being compiled
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(541): error C2440: 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &'
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(541): note: Conversion loses qualifiers
1>Done building project "fastest_hash.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

difference_type/unsigned int conversion warning

difference_type/unsigned int conversion warning is generated for 64-bit build in the following line:

hopscotch-map/src/hopscotch_hash.h

Line 191 in fc99945

m_iprime = std::distance(tsl::detail_hopscotch_hash::PRIMES.begin(), it_prime);

m_iprime = std::distance(tsl::detail_hopscotch_hash::PRIMES.begin(), it_prime);

Add the possibility to use a binary tree in case of overflow

For now the implementation uses a std::list when an overflow in the neighbourhood occurs. This list has a search complexity of O(n)

We could add an additional hash map interlace which would take as template parameter a LessThanComparable functor. We could then use something like a RB tree, AVL tree, skip list, ... instead of a list.

This would provide a O(log n) worst case for the map, even if all the inserted values hash to the exact same bucket. It would make the hash map resistant to DDOS collision attacks.

hopscotch_set::emplace doesn't work with MSVC compiler

#include <iostream>
#include "hopscotch/hopscotch_set.h"

void test() {
	tsl::hopscotch_set<int> xx;
	xx.emplace(1);
	std::cout << "size after emplace:" << xx.size() << std::endl;
}

void test1() {
	tsl::hopscotch_set<int> xx;
	xx.insert(1);
	std::cout << "size after insert:" << xx.size() << std::endl;
}

int main(int argc, char* argv[]) {
	test();
	test1();

output:

size after emplace:0                                                                                                                                          
size after insert:1

MSVC 2019 compiler

Searching by key and hash

Hi,

I would like to know your opinion on the possibility to provide additionally a precomputed hash value to find() member function. I would love such a feature as I am often looking for the same key value in many different hash containers and each of them recalculates the hash value for that value all over again. If find function had signature like:

iterator find(const Key& key, Hash* hash = nullptr);
<or>
iterator find(const Key& key, Hash hash = m_hash(key));

I would save a lot of time on calculating the same hash value all over again.

Best

Mat

Precalculated hashes

Hi tessil,

we are working on a research database which uses radix clustering. Here, we first hash a value, determine a partition accordingly, and then create a hash table for each partition.

Since we already calculated a hash for the partitioning, I was wondering why there is no insert method that supports precalculated hashes.
Am I overseeing something or has there just not been a reason to implement yet?

Besides that, I really dig your work here on GitHub. Keep it up!

contains()

Member functions contains(const Key&) and contains(const Key&, size_t hash) were added to standard associative containers in C++20.

Adding these functions will improve compatibility with the latest standard features.

Compile error on latest commit

I have to change hopscotch_hash.h:375 into if constexpr(StoreHash) {, else I get the following error:

tsl/hopscotch_hash.h:376: error : no type named 'hash_type' in 'tsl::detail_hopscotch_hash::hopscotch_bucket_hash<false>'
            return std::numeric_limits<typename bucket_hash::hash_type>::max();
                                       ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~

(and indeed, a ctrl-f of hash_type does not yield anything).
I don't set the StoreHash parameter to true anywhere in my code but it seems that my compiler (gcc-8.2) still checks the validity of the branch.

Building with -fno-exceptions

Using this library is currently impossible when building with the -fno-exceptions flag. Is it possible for exceptions to be replaced with abort() when using said flag?

Repeated use of operator[] coredumps in one particular case

Hi
This simple use case coredumps with corrupt heap

class Foo
{
public:
Foo() : foo_(nullptr) { }
~Foo()
{
delete[] foo_;
foo_ = nullptr;
}

    void fromString( const std::string& s ) 
    {
        delete foo_;
        foo_ = new char[ s.size() + 1  ];  
        std::strcpy( foo_, s.c_str() );
    }
    const char* foo() const { return foo_; }
private:
    char* foo_;

};

#ifdef USE_STD_UOM
typedef std::unordered_mapstd::string,Foo test_map;
#else
typedef tsl::hopscotch_mapstd::string,Foo test_map;
#endif

int main()
{
test_map m;
for( int i = 0; i < 100; ++i )
{
// build key
std::string key = "key_";
key += std::to_string(i);
// build value;
std::string value = "value_";
value += std::to_string(i);
// insert key and get ref to inserted value
Foo& foo = m[key];
// populate value
foo.fromString( value );
std::cout << key << ":" << foo.foo() << '\n';
}
}

Works fine with std::unordered_map or google dense map

Use a hash generation function written by myself

Hi tessil,
Thank you very much for the excellent project! I recently used this project to replace std::unordered_map and achieved a very large performance improvement. I find that my lookups are very regular with key values (e.g. "L>" "L?" "L@"). So I want to implement my own hash calculation function to replace std::hashstd::string()("");. I'm a newbie and don't know how to do this. I don't seem to see an example of this in the documentation either, so I'm asking for help here.

Compilation issues

On last 2 releases (1.2, 1.2.1) I have compilation/linker issues with hopscotch_map:

gcc 5.3.0 debug
hopscotch_hash.h:130: undefined reference to `tsl::mod_growth_policy<std::ratio<3l, 1l> >::MIN_BUCKETS_SIZE
VS 2015
hopscotch_hash.h(1557): warning C4100: 'hash': unreferenced formal parameter (compiling source file C:\chprojects\ets_falcon\source\matchserver-cpp\src\common\CentralLimitOrderBook.cpp)
I admit that the last one is probably a compiler bug but anyway it might be good to investigate what is going on. It helps adding (void)hash; in line 1565

Are you aware of those?

Warnings with -Wshadow flag

Gcc produces warnings if compile with flag -Wshadow.

/home/ksergey/dev/bitfinex_gateway_ng/deps/zerofw/deps/hopscotch-map/src/hopscotch_set.h:460:34: warning: declaration of ‘count’ shadows a member of ‘tsl::hopscotch_set<Key, Hash, KeyEqual, Allocator, NeighborhoodSize, StoreHash, GrowthPolicy>’ [-Wshadow]

/home/ksergey/dev/bitfinex_gateway_ng/deps/zerofw/deps/hopscotch-map/src/hopscotch_set.h:461:35: warning: declaration of ‘count’ shadows a member of ‘tsl::hopscotch_set<Key, Hash, KeyEqual, Allocator, NeighborhoodSize, StoreHash, GrowthPolicy>’ [-Wshadow]
     void reserve(size_type count) { m_ht.reserve(count); }

Containers are not copyable anymore

The following code fails to compile for me with current git master :

#include <hopscotch_map.h>
void t()
{
  tsl::hopscotch_map<int, int> m;
  auto m2 = m;
}

with the following error :

In file included from /usr/include/c++/6.3.1/bits/stl_tempbuf.h:60:0,
                 from /usr/include/c++/6.3.1/bits/stl_algo.h:62,
                 from /usr/include/c++/6.3.1/algorithm:62,
                 from /home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_map.h:28,
                 from /home/jcelerier/travail/i-score/API/OSSIA/ossia/ossia.cpp:1:
/usr/include/c++/6.3.1/bits/stl_construct.h: In instantiation of ‘void std::_Construct(_T1*, _Args&& ...) [with _T1 = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>; _Args = {const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>&}]’:
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:75:18:   required from ‘static _ForwardIterator std::__uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*; bool _TrivialValueTypes = false]’
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:126:15:   required from ‘_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*]’
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:281:37:   required from ‘_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, std::allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*; _Tp = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>]’
/usr/include/c++/6.3.1/bits/stl_vector.h:324:31:   required from ‘std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>; _Alloc = std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> >]’
/home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_hash.h:568:5:   required from here
/usr/include/c++/6.3.1/bits/stl_construct.h:75:7: error: use of deleted function ‘tsl::detail_hopscotch_hash::hopscotch_bucket<ValueType, NeighborhoodSize, StoreHash>::hopscotch_bucket(const tsl::detail_hopscotch_hash::hopscotch_bucket<ValueType, NeighborhoodSize, StoreHash>&) [with ValueType = std::pair<int, int>; unsigned int NeighborhoodSize = 62u; bool StoreHash = false]’
     { ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_map.h:36:0,
                 from /home/jcelerier/travail/i-score/API/OSSIA/ossia/ossia.cpp:1:
/home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_hash.h:217:5: note: declared here
     hopscotch_bucket(const hopscotch_bucket& bucket) = delete;
     ^~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

count() does not work for hopscotch_set

It seems that there is an issue with count() for tsl::hopscotch_set. In the simple following example:

tsl::hopscotch_set<std::string> s;
s.find(std::string{"abc"});
s.count(std::string{"abc"});

I get "no matching overloaded function found" in hopscotch_hash.h:1124 on Visual Studio 2015 for count() however find() works OK.

Note that such code is also OK:

tsl::hopscotch_map<std::string, int> s;
s.find(std::string{"abc"});
s.count(std::string{"abc"});

Warning: comparison 'unsigned int' <= 4294967295 is always true

While compiling my code, I'm getting this warning.

hopscotch_hash.h:1743:39: warning: comparison 'unsigned int' <= 4294967295 is always true [-Wtautological-constant-compare]"

Getting a weird backtrace in hopscotch_set

=================================================================
==26163==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffd2708b080 at pc 0x7fb4dbedb7c0 bp 0x7ffd2708b040 sp 0x7ffd2708b030
READ of size 8 at 0x7ffd2708b080 thread T0
    #0 0x7fb4dbedb7bf in __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >::difference_type __gnu_cxx::operator-<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > const&, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > const&) stl_iterator.h:965:0
    #1 0x7fb4dbec52f3 in std::iterator_traits<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >::difference_type std::__distance<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, std::random_access_iterator_tag) stl_iterator_base_funcs.h:104:0
    #2 0x7fb4dbeaf2e6 in std::iterator_traits<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >::difference_type std::distance<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) stl_iterator_base_funcs.h:142:0
    #3 0x7fb4dbe9f4d0 in __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::find_in_buckets<ossia::pd::remote*>(ossia::pd::remote* const&, unsigned long, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) hopscotch_hash.h:1760:0
    #4 0x7fb4dbe92299 in tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::hopscotch_iterator<false> tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::find_impl<ossia::pd::remote*>(ossia::pd::remote* const&, unsigned long, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) hopscotch_hash.h:1730:0
    #5 0x7fb4dbe88bf5 in std::pair<tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::hopscotch_iterator<false>, bool> tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::insert_impl<ossia::pd::remote* const&>(ossia::pd::remote* const&) hopscotch_hash.h:1534:0
    #6 0x7fb4dbe817af in tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::insert(ossia::pd::remote* const&) hopscotch_hash.h:949:0
    #7 0x7fb4dbe78dca in tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::insert[abi:cxx11](ossia::pd::remote* const&) hopscotch_set.h:237:0

here's the relevant code from my libstdc++; I really don't understand how it is getting a stack use-after-scope in this case.

iterator_base_funcs:142

  template<typename _InputIterator>
    inline _GLIBCXX17_CONSTEXPR
    typename iterator_traits<_InputIterator>::difference_type
    distance(_InputIterator __first, _InputIterator __last)
    {
      // concept requirements -- taken care of in __distance
      return std::__distance(__first, __last,
			     std::__iterator_category(__first));
    }

then std::__distance:

  template<typename _RandomAccessIterator>
    inline _GLIBCXX14_CONSTEXPR
    typename iterator_traits<_RandomAccessIterator>::difference_type
    __distance(_RandomAccessIterator __first, _RandomAccessIterator __last,
               random_access_iterator_tag)
    {
      // concept requirements
      __glibcxx_function_requires(_RandomAccessIteratorConcept<
				  _RandomAccessIterator>)
      return __last - __first;
    }

  template<typename _Iterator, typename _Container>
    inline typename __normal_iterator<_Iterator, _Container>::difference_type
    operator-(const __normal_iterator<_Iterator, _Container>& __lhs,
	      const __normal_iterator<_Iterator, _Container>& __rhs)
    _GLIBCXX_NOEXCEPT
    { return __lhs.base() - __rhs.base(); }

and it crashes on either __lhs.base() or __rhs.base(), which I do not understand. It works if I use std::unordered_map (and the map is a global static accessed by a single thread so there's no "use-after-free" possibility). I'm compiling at -O0 with -fsanitize=address.

Recommended means to update key?

Hi! Is there any recommended/built in way to update the key of the hashmap ? Without touching the library the only thing I can think of is erasing a key value pair, and adding it back with the updated key but that's an expensive operation. I'd appreciate any help I can get, and I'm loving the library so far!

tessil / hopscotch-map Goto Github PK

hopscotch-map's People

Contributors

Stargazers

Watchers

Forkers

hopscotch-map's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs