tessil / hopscotch-map Goto Github PK
View Code? Open in Web Editor NEWC++ implementation of a fast hash map and hash set using hopscotch hashing
License: MIT License
C++ implementation of a fast hash map and hash set using hopscotch hashing
License: MIT License
Hi,
I have a code similar to that one:
tsl::hopscotch_map<std::string, std::vector<int>> map;
auto it = map.find("aaa");
if(it != std::end(map)) {
auto& v = it->second;
// modify v
}
It seems it does not work for me (right now I am testing your branch with additional hash argument for find() operation). I get const vector from non-const iterator. It is probably caused by:
using value_type = const typename hopscotch_hash::value_type;
in https://github.com/Tessil/hopscotch-map/blob/add_hash_parameter/src/hopscotch_hash.h#L610 and directly using std::vector
and std::list
iterators on std::pair<key, value>
.
Can you confirm that is an issue?
It seems that method begin()
becomes slower after I remove some elements. Here is a simple test https://gist.github.com/andrusha97/de6b63f48aca45a6b04b4e968c34289b
I guess it's because this method scans buckets sequentially until it finds a non-empty one, and erase
never reduces the number of buckets. Also if this is the case, iteration over all elements probably takes O(c.capacity())
and not O(c.size())
.
Could you confirm that I understand things right? I'd like iterators to have more "traditional" complexities. Could you recommend how to deal with this? Should I monitor load factor and shrink containers manually?
Hello,
We are hitting a crash when calling erase on the set implementation.
Exception thrown: read access violation.
this was nullptr.
Here is the stack trace
> Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0>::neighborhood_infos() Line 487 C++ Symbols loaded.
Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::find_in_buckets<unsigned int>(const unsigned int & key, unsigned __int64 hash, std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0> > > > it_bucket) Line 1772 C++ Symbols loaded.
Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::find_in_buckets<unsigned int>(const unsigned int & key, unsigned __int64 hash, std::_Vector_iterator<std::_Vector_val<std::_Simple_types<tsl::detail_hopscotch_hash::hopscotch_bucket<unsigned int,62,0> > > > it_bucket) Line 1760 C++ Symbols loaded.
Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::erase<unsigned int>(const unsigned int & key, unsigned __int64 hash) Line 1124 C++ Symbols loaded.
Server_r.exe!tsl::detail_hopscotch_hash::hopscotch_hash<unsigned int,tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::KeySelect,void,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy,std::list<unsigned int,std::allocator<unsigned int> > >::erase<unsigned int>(const unsigned int & key) Line 1117 C++ Symbols loaded.
Server_r.exe!tsl::hopscotch_set<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int>,std::allocator<unsigned int>,62,0,tsl::power_of_two_growth_policy>::erase(const unsigned int & key) Line 279 C++ Symbols loaded.
Thanks !
Which guarantees does the data structures offer ? Are they at least at strong (so not very strong) than equivalent std:: containers ? (i.e. multiple threads doing only lookups in the map at the same time should be safe).
Thanks !
Hello,
We are experiencing a weird misbehavior with tsl/hopscotch_map, bisected to commit 5c5770a being the culprit.
According to its changelog entry, this looks like an innocent commit purely renaming variables, namely m_buckets
to m_buckets_data
and m_first_or_empty_bucket
to m_buckets
. However, danger lurks around in one of the old names being the same as the other new name.
At several places (e.g. hopscotch_hash.h line 1246 being the first one) m_buckets
remains m_buckets
. Based on the changelog's wording, these look like accidental mistakes rather than intentional "changes".
Into our tsl::hopscotch_map<int, void *>
we insert about 1000 entries (using map[key] = val
) and erase about 800 (using it = map.find(key); blahblah; map.erase(it)
and then no longer using it
afterwards). These operations interleave each other in kinda random order, and we sometimes re-add a key that we erased earlier.
Then, after one insertion, the following situation arises:
If I iterate through all the items using for (auto it : map) {...}
then I get all the about 200 items, as expected.
However, for one of these entries map.count(key)
gives 0 instead of the expected 1. Trying to access this item using .at(key)
raises an exception, as usual for missing entries. Or, trying to access this key using [key]
inserts this item again, with the default value of nullptr. After this operation, if I loop through the items again using for (auto it : map) {...}
then this key gets listed twice, once with the expected value and once with nullptr. And at this point it becomes another key that is "missing" in this weird sense that looping over all the entries finds it, but looking it up directly does not.
Unfortunately I couldn't come up with a self-contained test demonstrating the failure, and I cannot give you access to our in-house software that every now and then triggers this bug. Our software has a very non-deterministic behavior by its nature, but as of your aforementioned commit we see the misbehavior about 1 out of 20-50-ish cases. We haven't seen misbehavior with your hopscotch_map prior to the mentioned commit during thousands of runs, nor with the standard unordered_map.
Let me know if you need any help in debugging this issue (without me digging deeply into your code :)). For example, if you create a version that dumps into a global fd the memory region of the map after every insertion/deletion, I'd be happy to test that. But maybe you just want to revert the functionality and redo the variable naming properly :)
Thanks in advance!
#include <iostream>
#include <string>
#include "hopscotch_map.h"
#include <unordered_map>
using namespace tsl;
using namespace std;
struct Test {
};
int main() {
unordered_map<string, Test> m;
auto it = m.find("a");
Test& r = it->second;
Test t = move(r);
hopscotch_map<string, Test> m2;
auto it2 = m2.find("a");
Test& r2 = it2->second;
Test t2 = move(r2);
return 0;
}
Which produce error
main.cc: In function 'int main()':
main.cc:24:21: error: binding reference of type 'Test&' to 'const Test' discards qualifiers
Test& r2 = it2->second;
~~~~~^~~~~~
But it2->second should not be const.
sparse_map also have the same problem.
The following won't build on current git head:
tsl::hopscotch_map<std::string, std::string> s;
tsl::hopscotch_map<std::string, std::string> s2 = s; // initialization is ok
s2 = s; // operator= fails
It worked by f5b7e62 .
A quick bisect told me that fd765fb is the commit that introduced breakage; 838413d works.
The CMake configuration has its minimum required version set to 3.1. However, the cxx_std_11
constant passed to target_compile_features
has only been introduced in CMake 3.8. The result is that for versions 3.1 to 3.7 an error will appear that this value does not exist.
The easiest solution would be to increase the minimum required version to 3.8. I have looked for other solutions to require C++11, but they are not really satisfying. It is not possible to require the C++11 compiler switch via the CXX_STANDARD
property for interface libraries. Another, more involved, attempted solution was to specifically determine what compiler features the project needs from the list of features available in 3.1. This includes for example cxx_auto_type
and cxx_nullptr
. However, CMake only knows about these features for GCC and Clang, so that is also not an option.
Another kind of solution would be that the target feature will only be set when configuring with CMake 3.8 via a version condition. For versions 3.1 to 3.7 it could print a warning, determine whether the CXX_STANDARD
variable was set or it would simply fail during compilation.
The documentation states that there's "Support for move-only and non-default constructible key/value", however, unless I'm mistaken in what is meant with that, I don't think that is true or needs further clarification.
Edit: I did some more digging and I'm going with "needs further clarification". The issue is that the move / copy constructor has to be defined noexcept
in order to be used by the hopscotch_map
. This is not API compatible with std::unordered_{map,set}
and should be noted as such.
I've cooked up a small example (see below) which does not compile with error
[...] error: call to implicitly-deleted copy constructor of 'value_type' [...]
I'm on Mac OSX 10.11.6 with the system compiler; clang++ --version
reports
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
I'm attempting to compile the example with
clang++ -W -Wall -Wextra -std=c++11 -I hopscotch-map -o test test.cpp
I have uploaded the full error log here.
#include <memory>
struct MoveOnly {
explicit MoveOnly(int i)
: m_data(i){}
MoveOnly(MoveOnly && other)
: m_data(std::move(other.m_data)){}
MoveOnly(const MoveOnly &) = delete;
MoveOnly & operator=(const MoveOnly &) = delete;
MoveOnly & operator=(MoveOnly &&) = delete;
private:
int m_data;
};
#include "hopscotch_map.h"
#include <string>
class Foo {
public:
tsl::hopscotch_map<std::string, MoveOnly> m_db;
Foo()
: m_db(){}
};
int main(){
Foo f;
}
Hello,
I love these map libraries and am using them extensively, thank you so much for sharing your work!
I am frequently running into the need to iterate over the keys or values separate of the other. Is there any way this could be implemented for this map?
Currently I have implemented utility functions to gather the keys or values into a vector and then use that, but I'd like to do this iteration in a functional programming kind of way (so without a for loop and a map() or transform() instead) and also without taking up the memory to copy all the keys or values into a separate vector.
Let me know your thoughts on this, and thanks again for your hard work!
Dear Tessil,
I spent nearly one day on a bug and finally find it may come out from the hopscotch-map.
Here is a test file extracted from our log and a simple test program to help you to reproduce it.
The key file contains 187414 unique values, and expected the map should have the same number of size. However, the size of built map is 187353.
https://www.dropbox.com/s/hf35bq88t7ixve0/keys.dat?dl=0
https://www.dropbox.com/s/a62cdfq3kr3nuao/maptest.cpp?dl=0
could you help to debug it? Many thanks!
When I try to use tsl::hopscotch_map instead of std::unordered_map I often end up with VS C4503 warning. Do you have any good BKMs on how to handle such issues?
Hi,
It seems compiling this library using old GCC version with incomplete C++11 support (-std=c++0x) is not currently possible. Are there any plans to support this?
Cheers!
Hi,
I tried your project yesterday and compared its performance on Windows (VS2015) and Linux (gcc 5.3) vs std::unordered_map. I have 2 use cases for some 'struct A':
a) tsl::hopscotch_map<std::string, A&>
or tsl::hopscotch_map<gsl::cstring_span<>, A&>
b) tsl::hopscotch_map<uint64_t, A&>
I must say that for first use case hopscotch_map performed much better than std::unordered_map on both Windows and Linux. However the second use case on Linux has really bad performance. I measured the time of 70.000 insertions of unique uint64_t key/values pairs (and verified they have unique hash values too) and then the cleanup of such a container. The result was more than 100x times worse performance of tsl::hopscotch_map. It looks strange to me as use case a) works at the same speed. I tried to analyze the issue with linux perf tools but didn't have much time for that. Do you have any ideas what is wrong?
tsl::hopscotch_map
already provides heterogeneous overloads for lookup (e.g. via find
or at
) but there is currently no such support for adding elements (e.g. via operator[]
or try_emplace
). AFAIK, corresponding overloads were added to
the standard associative containers with C++26.
@Tessil Do you think it makes sense to also extend tsl::hopscotch_map
? From what I can see, it seems that the underlying tsl::hopscotch_hash
already has the necessary functionality, so I drafted a first version for try_emplace
and operator[]
here. I only extended tsl::hopscotch_map
as tsl::bhopscotch_map
seems to sometimes copy keys anyways.
If you think this makes sense, I could also open a PR.
Microsoft has created a C++ Library Manager for Windows, Linux, and MacOS.
https://github.com/Microsoft/vcpkg
This is a minor thing, but it's been bugging me for a little while. One of the differences with std::unordered_map
is that value_type
is const std::pair<Key,T>
instead of std::pair<const Key,T>
. Iterators allow us to work around this with the value()
method, which gives us a mutable reference to the mapped type.
This works fine with traditional for loops and iterators.
for (auto it = map.begin(); it != map.end(); ++it) {
auto &second = it.value();
...
}
However, range-based for loops don't work as nicely, and AFAIK the only way to get a mutable reference to the mapped type in that case is to const_cast
it:
for (auto &value : map) {
auto &second = const_cast<T&>(value.second);
...
}
I'm sure there's a reason why value_type
has to be const (sorry if I missed it), but it's a bit unfortunate that we can't use range-based for loops as straightfordwardly as with std::unordered_map
. Apart from that, great library! 👍
Certainly the fault of a dumb compiler, but still : given hopscotch_map<void*>
, I get an infinite recursion between these two functions:
template<class P, typename std::enable_if<std::is_constructible<value_type, P&&>::value>::type* = nullptr>
std::pair<iterator, bool> insert(P&& value) {
return emplace(std::forward<P>(value));
}
template<class... Args>
std::pair<iterator, bool> emplace(Args&&... args) {
return insert(value_type(std::forward<Args>(args)...));
}
When compiling in C++23 mode, there is a deprecation warning. What about removing std::aligned_storage and use an unsigned char
instead? (With C++17, also std::byte
could be used.)
diff --git a/include/tsl/hopscotch_hash.h b/include/tsl/hopscotch_hash.h
index ed5f858..cf87532 100644
--- a/include/tsl/hopscotch_hash.h
+++ b/include/tsl/hopscotch_hash.h
@@ -386,11 +386,9 @@ class hopscotch_bucket : public hopscotch_bucket_hash<StoreHash> {
}
private:
- using storage = typename std::aligned_storage<sizeof(value_type),
- alignof(value_type)>::type;
neighborhood_bitmap m_neighborhood_infos;
- storage m_value;
+ alignas(value_type) unsigned char m_value[sizeof(value_type)];
};
/**
reproduce code:
#include <string>
#include "sparse-map/tsl/sparse_map.h"
#include "hopscotch-map/include/tsl/hopscotch_map.h"
class Value;
typedef tsl::sparse_map<std::string, Value> SMap;
typedef tsl::hopscotch_map<std::string, Value> Map;
// general value class
class Value {
public:
Value& operator=(const SMap& v) noexcept {
smap = new SMap(v);
return *this;
}
Value& operator=(const Map& v) noexcept {
map = new Map(v);
return *this;
}
private:
Map* map;
SMap* smap;
};
int main() {
Value xx;
return 0;
}
error message:
In file included from test.cc:4:
In file included from ./hopscotch-map/include/tsl/hopscotch_map.h:36:
./hopscotch-map/include/tsl/hopscotch_hash.h:405:51: error: invalid application of 'sizeof' to an incomplete type 'tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<std::__cxx11::basic_string<char>, Value>, 62, false>::value_type' (aka 'std::pair<std::__cxx11::basic_string<char>, Value>')
using storage = typename std::aligned_storage<sizeof(value_type), alignof(value_type)>::type;
^~~~~~~~~~~~~~~~~~
./hopscotch-map/include/tsl/hopscotch_hash.h:463:42: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<std::__cxx11::basic_string<char>, Value>, 62, false>' requested here
using neighborhood_bitmap = typename hopscotch_bucket::neighborhood_bitmap;
^
./hopscotch-map/include/tsl/hopscotch_map.h:119:31: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::KeySelect, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2>, std::__cxx11::list<std::pair<std::__cxx11::basic_string<char>, Value>, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> > > >' requested here
using key_type = typename ht::key_type;
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1061:56: note: in instantiation of template class 'tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >' requested here
: public __bool_constant<__is_assignable(_Tp, _Up)>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1073:14: note: in instantiation of template class 'std::is_assignable<Value &, const Value &>' requested here
: public is_assignable<_Tp&, const _Tp&>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1079:14: note: in instantiation of template class 'std::__is_copy_assignable_impl<Value, true>' requested here
: public __is_copy_assignable_impl<_Tp>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:144:14: note: (skipping 5 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
: public conditional<_B1::value, _B2, _B1>::type
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1049:14: note: in instantiation of template class 'std::is_nothrow_constructible<std::pair<std::__cxx11::basic_string<char>, Value>, std::pair<std::__cxx11::basic_string<char>, Value> &&>' requested here
: public is_nothrow_constructible<_Tp, _Tp&&>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1055:14: note: in instantiation of template class 'std::__is_nothrow_move_constructible_impl<std::pair<std::__cxx11::basic_string<char>, Value>, true>' requested here
: public __is_nothrow_move_constructible_impl<_Tp>
^
./sparse-map/tsl/sparse_hash.h:833:24: note: in instantiation of template class 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >' requested here
static_assert(std::is_nothrow_move_constructible<ValueType>::value ||
^
./sparse-map/tsl/sparse_map.h:119:31: note: in instantiation of template class 'tsl::detail_sparse_hash::sparse_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::KeySelect, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium, tsl::sh::probing::quadratic>' requested here
using key_type = typename ht::key_type;
^
test.cc:18:20: note: in instantiation of template class 'tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>' requested here
smap = new SMap(v);
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/bits/stl_pair.h:198:12: note: definition of 'std::pair<std::__cxx11::basic_string<char>, Value>' is not complete until the closing '}'
struct pair
^
In file included from test.cc:4:
In file included from ./hopscotch-map/include/tsl/hopscotch_map.h:36:
./hopscotch-map/include/tsl/hopscotch_hash.h:1233:67: error: no member named 'value' in 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >'
static_assert(std::is_nothrow_move_constructible<value_type>::value ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
./hopscotch-map/include/tsl/hopscotch_map.h:119:31: note: in instantiation of template class 'tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::KeySelect, tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2>, std::__cxx11::list<std::pair<std::__cxx11::basic_string<char>, Value>, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> > > >' requested here
using key_type = typename ht::key_type;
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1061:56: note: in instantiation of template class 'tsl::hopscotch_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, 62, false, tsl::hh::power_of_two_growth_policy<2> >' requested here
: public __bool_constant<__is_assignable(_Tp, _Up)>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1073:14: note: in instantiation of template class 'std::is_assignable<Value &, const Value &>' requested here
: public is_assignable<_Tp&, const _Tp&>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1079:14: note: in instantiation of template class 'std::__is_copy_assignable_impl<Value, true>' requested here
: public __is_copy_assignable_impl<_Tp>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:144:14: note: in instantiation of template class 'std::is_copy_assignable<Value>' requested here
: public conditional<_B1::value, _B2, _B1>::type
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/bits/stl_pair.h:368:3: note: (skipping 4 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
__and_<is_copy_assignable<_T1>,
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1049:14: note: in instantiation of template class 'std::is_nothrow_constructible<std::pair<std::__cxx11::basic_string<char>, Value>, std::pair<std::__cxx11::basic_string<char>, Value> &&>' requested here
: public is_nothrow_constructible<_Tp, _Tp&&>
^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../include/c++/8.1.1/type_traits:1055:14: note: in instantiation of template class 'std::__is_nothrow_move_constructible_impl<std::pair<std::__cxx11::basic_string<char>, Value>, true>' requested here
: public __is_nothrow_move_constructible_impl<_Tp>
^
./sparse-map/tsl/sparse_hash.h:833:24: note: in instantiation of template class 'std::is_nothrow_move_constructible<std::pair<std::__cxx11::basic_string<char>, Value> >' requested here
static_assert(std::is_nothrow_move_constructible<ValueType>::value ||
^
./sparse-map/tsl/sparse_map.h:119:31: note: in instantiation of template class 'tsl::detail_sparse_hash::sparse_hash<std::pair<std::__cxx11::basic_string<char>, Value>, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::KeySelect, tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>::ValueSelect, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium, tsl::sh::probing::quadratic>' requested here
using key_type = typename ht::key_type;
^
test.cc:18:20: note: in instantiation of template class 'tsl::sparse_map<std::__cxx11::basic_string<char>, Value, std::hash<std::__cxx11::string>, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<std::__cxx11::basic_string<char>, Value> >, tsl::sh::power_of_two_growth_policy<2>, tsl::sh::exception_safety::basic, tsl::sh::sparsity::medium>' requested here
smap = new SMap(v);
^
2 errors generated.
compiler:
But following code is OK
#include <string>
#include "sparse-map/tsl/sparse_map.h"
#include "hopscotch-map/include/tsl/hopscotch_map.h"
class Value;
typedef tsl::sparse_map<std::string, Value> SMap;
typedef tsl::hopscotch_map<std::string, Value> Map;
// general value class
class Value {
public:
Value& operator=(const SMap& v) noexcept {
smap = new SMap(v);
return *this;
}
/*
Value& operator=(const Map& v) noexcept {
map = new Map(v);
return *this;
}
*/
private:
Map* map;
SMap* smap;
};
int main() {
Value xx;
return 0;
}
Simplest code produces compilation error (MS VS 2017 Version 15.9.17).
tsl::bhopscotch_map<uint64_t, uint64_t> m3;
m3[10] = 100;
Severity Code Description Project File Line Suppression State
Error C2440 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &' fastest_hash c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h 538
Error C2440 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &' fastest_hash c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h 541
Full output log:
1>------ Build started: Project: fastest_hash, Configuration: Release x64 ------
1>main.cpp
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(538): error C2440: 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &'
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(538): note: Conversion loses qualifiers
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(1073): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::hopscotch_iterator::value<ValueSelect,0x0>(void) const' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(1071): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::hopscotch_iterator::value<ValueSelect,0x0>(void) const' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::operator []<unsigned __int64,ValueSelect,0x0>(K &&)' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,
1> K=unsigned __int64
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: see reference to function template instantiation 'unsigned __int64 &tsl::detail_hopscotch_hash::hopscotch_hash<std::pair<const _Kty,_Ty>,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::KeySelect,tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Ty>,std::less<_Ty>,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,Hash,KeyEqual,Allocator,62,false,GrowthPolicy,std::map<_Kty,_Ty,Compare,Allocator>>::operator []<unsigned __int64,ValueSelect,0x0>(K &&)' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> _Ty=uint64_t,
1> Hash=std::hash<uint64_t>,
1> KeyEqual=std::equal_to<uint64_t>,
1> Allocator=std::allocator<std::pair<const uint64_t,uint64_t>>,
1> GrowthPolicy=tsl::hh::power_of_two_growth_policy<2>,
1> Compare=std::less<uint64_t>,
1> ValueSelect=tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<uint64_t>,std::equal_to<uint64_t>,std::less<uint64_t>,std::allocator<std::pair<const uint64_t,uint64_t>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::ValueSelect,
1> K=unsigned __int64
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\bhopscotch_map.h(447): note: while compiling class template member function 'T &tsl::bhopscotch_map<uint64_t,T,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::operator [](Key &&)'
1> with
1> [
1> T=uint64_t,
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\main.cpp(121): note: see reference to function template instantiation 'T &tsl::bhopscotch_map<uint64_t,T,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>::operator [](Key &&)' being compiled
1> with
1> [
1> T=uint64_t,
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\main.cpp(119): note: see reference to class template instantiation 'tsl::bhopscotch_map<uint64_t,uint64_t,std::hash<_Kty>,std::equal_to<_Kty>,std::less,std::allocator<std::pair<const _Kty,_Ty>>,62,false,tsl::hh::power_of_two_growth_policy<2>>' being compiled
1> with
1> [
1> _Kty=uint64_t,
1> Key=uint64_t,
1> _Ty=uint64_t
1> ]
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_growth_policy.h(202): note: see reference to class template instantiation 'std::array<size_t,40>' being compiled
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(541): error C2440: 'return': cannot convert from 'const unsigned __int64' to 'unsigned __int64 &'
1>c:\dev\projects\fastest_hash\src\hopscotch-map-master\include\tsl\hopscotch_hash.h(541): note: Conversion loses qualifiers
1>Done building project "fastest_hash.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
difference_type
/unsigned int
conversion warning is generated for 64-bit build in the following line:
hopscotch-map/src/hopscotch_hash.h
Line 191 in fc99945
m_iprime = std::distance(tsl::detail_hopscotch_hash::PRIMES.begin(), it_prime);
For now the implementation uses a std::list
when an overflow in the neighbourhood occurs. This list has a search complexity of O(n)
We could add an additional hash map interlace which would take as template parameter a LessThanComparable functor. We could then use something like a RB tree, AVL tree, skip list, ... instead of a list.
This would provide a O(log n) worst case for the map, even if all the inserted values hash to the exact same bucket. It would make the hash map resistant to DDOS collision attacks.
#include <iostream>
#include "hopscotch/hopscotch_set.h"
void test() {
tsl::hopscotch_set<int> xx;
xx.emplace(1);
std::cout << "size after emplace:" << xx.size() << std::endl;
}
void test1() {
tsl::hopscotch_set<int> xx;
xx.insert(1);
std::cout << "size after insert:" << xx.size() << std::endl;
}
int main(int argc, char* argv[]) {
test();
test1();
output:
size after emplace:0
size after insert:1
MSVC 2019 compiler
Hi,
I would like to know your opinion on the possibility to provide additionally a precomputed hash value to find() member function. I would love such a feature as I am often looking for the same key value in many different hash containers and each of them recalculates the hash value for that value all over again. If find function had signature like:
iterator find(const Key& key, Hash* hash = nullptr);
<or>
iterator find(const Key& key, Hash hash = m_hash(key));
I would save a lot of time on calculating the same hash value all over again.
Best
Mat
Hi tessil,
we are working on a research database which uses radix clustering. Here, we first hash a value, determine a partition accordingly, and then create a hash table for each partition.
Since we already calculated a hash for the partitioning, I was wondering why there is no insert method that supports precalculated hashes.
Am I overseeing something or has there just not been a reason to implement yet?
Besides that, I really dig your work here on GitHub. Keep it up!
Member functions contains(const Key&)
and contains(const Key&, size_t hash)
were added to standard associative containers in C++20.
Adding these functions will improve compatibility with the latest standard features.
I have to change hopscotch_hash.h:375 into if constexpr(StoreHash) {
, else I get the following error:
tsl/hopscotch_hash.h:376: error : no type named 'hash_type' in 'tsl::detail_hopscotch_hash::hopscotch_bucket_hash<false>'
return std::numeric_limits<typename bucket_hash::hash_type>::max();
~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
(and indeed, a ctrl-f of hash_type does not yield anything).
I don't set the StoreHash parameter to true anywhere in my code but it seems that my compiler (gcc-8.2) still checks the validity of the branch.
Using this library is currently impossible when building with the -fno-exceptions
flag. Is it possible for exceptions to be replaced with abort()
when using said flag?
Hi
This simple use case coredumps with corrupt heap
class Foo
{
public:
Foo() : foo_(nullptr) { }
~Foo()
{
delete[] foo_;
foo_ = nullptr;
}
void fromString( const std::string& s )
{
delete foo_;
foo_ = new char[ s.size() + 1 ];
std::strcpy( foo_, s.c_str() );
}
const char* foo() const { return foo_; }
private:
char* foo_;
};
#ifdef USE_STD_UOM
typedef std::unordered_mapstd::string,Foo test_map;
#else
typedef tsl::hopscotch_mapstd::string,Foo test_map;
#endif
int main()
{
test_map m;
for( int i = 0; i < 100; ++i )
{
// build key
std::string key = "key_";
key += std::to_string(i);
// build value;
std::string value = "value_";
value += std::to_string(i);
// insert key and get ref to inserted value
Foo& foo = m[key];
// populate value
foo.fromString( value );
std::cout << key << ":" << foo.foo() << '\n';
}
}
Works fine with std::unordered_map or google dense map
Hi tessil,
Thank you very much for the excellent project! I recently used this project to replace std::unordered_map and achieved a very large performance improvement. I find that my lookups are very regular with key values (e.g. "L>" "L?" "L@"). So I want to implement my own hash calculation function to replace std::hashstd::string()("");. I'm a newbie and don't know how to do this. I don't seem to see an example of this in the documentation either, so I'm asking for help here.
On last 2 releases (1.2, 1.2.1) I have compilation/linker issues with hopscotch_map:
Are you aware of those?
Gcc produces warnings if compile with flag -Wshadow.
/home/ksergey/dev/bitfinex_gateway_ng/deps/zerofw/deps/hopscotch-map/src/hopscotch_set.h:460:34: warning: declaration of ‘count’ shadows a member of ‘tsl::hopscotch_set<Key, Hash, KeyEqual, Allocator, NeighborhoodSize, StoreHash, GrowthPolicy>’ [-Wshadow]
/home/ksergey/dev/bitfinex_gateway_ng/deps/zerofw/deps/hopscotch-map/src/hopscotch_set.h:461:35: warning: declaration of ‘count’ shadows a member of ‘tsl::hopscotch_set<Key, Hash, KeyEqual, Allocator, NeighborhoodSize, StoreHash, GrowthPolicy>’ [-Wshadow]
void reserve(size_type count) { m_ht.reserve(count); }
The following code fails to compile for me with current git master :
#include <hopscotch_map.h>
void t()
{
tsl::hopscotch_map<int, int> m;
auto m2 = m;
}
with the following error :
In file included from /usr/include/c++/6.3.1/bits/stl_tempbuf.h:60:0,
from /usr/include/c++/6.3.1/bits/stl_algo.h:62,
from /usr/include/c++/6.3.1/algorithm:62,
from /home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_map.h:28,
from /home/jcelerier/travail/i-score/API/OSSIA/ossia/ossia.cpp:1:
/usr/include/c++/6.3.1/bits/stl_construct.h: In instantiation of ‘void std::_Construct(_T1*, _Args&& ...) [with _T1 = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>; _Args = {const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>&}]’:
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:75:18: required from ‘static _ForwardIterator std::__uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*; bool _TrivialValueTypes = false]’
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:126:15: required from ‘_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*]’
/usr/include/c++/6.3.1/bits/stl_uninitialized.h:281:37: required from ‘_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, std::allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> > > >; _ForwardIterator = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>*; _Tp = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>]’
/usr/include/c++/6.3.1/bits/stl_vector.h:324:31: required from ‘std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false>; _Alloc = std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<std::pair<int, int>, 62u, false> >]’
/home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_hash.h:568:5: required from here
/usr/include/c++/6.3.1/bits/stl_construct.h:75:7: error: use of deleted function ‘tsl::detail_hopscotch_hash::hopscotch_bucket<ValueType, NeighborhoodSize, StoreHash>::hopscotch_bucket(const tsl::detail_hopscotch_hash::hopscotch_bucket<ValueType, NeighborhoodSize, StoreHash>&) [with ValueType = std::pair<int, int>; unsigned int NeighborhoodSize = 62u; bool StoreHash = false]’
{ ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_map.h:36:0,
from /home/jcelerier/travail/i-score/API/OSSIA/ossia/ossia.cpp:1:
/home/jcelerier/travail/i-score/API/3rdparty/hopscotch-map/src/hopscotch_hash.h:217:5: note: declared here
hopscotch_bucket(const hopscotch_bucket& bucket) = delete;
^~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
It seems that there is an issue with count()
for tsl::hopscotch_set. In the simple following example:
tsl::hopscotch_set<std::string> s;
s.find(std::string{"abc"});
s.count(std::string{"abc"});
I get "no matching overloaded function found" in hopscotch_hash.h:1124 on Visual Studio 2015 for count()
however find()
works OK.
Note that such code is also OK:
tsl::hopscotch_map<std::string, int> s;
s.find(std::string{"abc"});
s.count(std::string{"abc"});
While compiling my code, I'm getting this warning.
hopscotch_hash.h:1743:39: warning: comparison 'unsigned int' <= 4294967295 is always true [-Wtautological-constant-compare]"
=================================================================
==26163==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffd2708b080 at pc 0x7fb4dbedb7c0 bp 0x7ffd2708b040 sp 0x7ffd2708b030
READ of size 8 at 0x7ffd2708b080 thread T0
#0 0x7fb4dbedb7bf in __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >::difference_type __gnu_cxx::operator-<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > const&, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > const&) stl_iterator.h:965:0
#1 0x7fb4dbec52f3 in std::iterator_traits<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >::difference_type std::__distance<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, std::random_access_iterator_tag) stl_iterator_base_funcs.h:104:0
#2 0x7fb4dbeaf2e6 in std::iterator_traits<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >::difference_type std::distance<__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > >(__gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> const*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) stl_iterator_base_funcs.h:142:0
#3 0x7fb4dbe9f4d0 in __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > > tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::find_in_buckets<ossia::pd::remote*>(ossia::pd::remote* const&, unsigned long, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) hopscotch_hash.h:1760:0
#4 0x7fb4dbe92299 in tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::hopscotch_iterator<false> tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::find_impl<ossia::pd::remote*>(ossia::pd::remote* const&, unsigned long, __gnu_cxx::__normal_iterator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>*, std::vector<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false>, std::allocator<tsl::detail_hopscotch_hash::hopscotch_bucket<ossia::pd::remote*, 62u, false> > > >) hopscotch_hash.h:1730:0
#5 0x7fb4dbe88bf5 in std::pair<tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::hopscotch_iterator<false>, bool> tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::insert_impl<ossia::pd::remote* const&>(ossia::pd::remote* const&) hopscotch_hash.h:1534:0
#6 0x7fb4dbe817af in tsl::detail_hopscotch_hash::hopscotch_hash<ossia::pd::remote*, tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::KeySelect, void, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy, std::__cxx11::list<ossia::pd::remote*, std::allocator<ossia::pd::remote*> > >::insert(ossia::pd::remote* const&) hopscotch_hash.h:949:0
#7 0x7fb4dbe78dca in tsl::hopscotch_set<ossia::pd::remote*, ossia::safe_set<ossia::pd::remote*>::EgurHash<ossia::pd::remote*>, std::equal_to<ossia::pd::remote*>, std::allocator<ossia::pd::remote*>, 62u, false, tsl::power_of_two_growth_policy>::insert[abi:cxx11](ossia::pd::remote* const&) hopscotch_set.h:237:0
here's the relevant code from my libstdc++; I really don't understand how it is getting a stack use-after-scope in this case.
iterator_base_funcs:142
template<typename _InputIterator>
inline _GLIBCXX17_CONSTEXPR
typename iterator_traits<_InputIterator>::difference_type
distance(_InputIterator __first, _InputIterator __last)
{
// concept requirements -- taken care of in __distance
return std::__distance(__first, __last,
std::__iterator_category(__first));
}
then std::__distance:
template<typename _RandomAccessIterator>
inline _GLIBCXX14_CONSTEXPR
typename iterator_traits<_RandomAccessIterator>::difference_type
__distance(_RandomAccessIterator __first, _RandomAccessIterator __last,
random_access_iterator_tag)
{
// concept requirements
__glibcxx_function_requires(_RandomAccessIteratorConcept<
_RandomAccessIterator>)
return __last - __first;
}
template<typename _Iterator, typename _Container>
inline typename __normal_iterator<_Iterator, _Container>::difference_type
operator-(const __normal_iterator<_Iterator, _Container>& __lhs,
const __normal_iterator<_Iterator, _Container>& __rhs)
_GLIBCXX_NOEXCEPT
{ return __lhs.base() - __rhs.base(); }
and it crashes on either __lhs.base() or __rhs.base(), which I do not understand. It works if I use std::unordered_map (and the map is a global static accessed by a single thread so there's no "use-after-free" possibility). I'm compiling at -O0 with -fsanitize=address.
Hi! Is there any recommended/built in way to update the key of the hashmap ? Without touching the library the only thing I can think of is erasing a key value pair, and adding it back with the updated key but that's an expensive operation. I'd appreciate any help I can get, and I'm loving the library so far!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.