matthewcrews / fastdictionarytest Goto Github PK

Hey there!

I'm trying to implement hash dicts in Elm based on your repo and videos (and the awesome C++Now 2018 talk by Malte).

I've just got round to Robin Hood, and I have a property based test suite set up checking that outwardly my dict behaves exactly the same as the "known to be correct" stdlib Dict.

The test suite has found an issue with your approach to inserting an item into a Tombstone. Consider this scenario:

1. Start with empty table (4 buckets)
2. Insert "A" (let's say it hashes to bucket index 3) -> will end up in bucket 3 with offset 0
3. Insert "B" (hashes to bucket index 3) -> will end up in bucket 0 with offset 1
4. Remove "A" -> now bucket 3 is a Tombstone
5. Insert "B" -> it sees a tombstone in bucket 3, and gets inserted there
6. Convert to list -> you get two Bs instead of just one!

That's because after step 5 the array looks like:

instead of

On the other hand, if in step 5 you continued to the next index after seeing a tombstone, you'd find the B in bucket 0 and would replace its value / end there.

Hi, Matthew. First, thanks for doing this. I've sped up a lot of code with your work.

I don't have a whole lot of information to share about this issue, but here's what I know. In a recent project, I've really ramped up usage of dictionaries -- based on ByteListRobinHoodVec128. I have about 7 of them, some with about 50k elements, some with about half a million.

On my first "big" test run, I received a System.AccessViolationException in the implementation of GetHashCode for the stringComparer. (function startes line 68 in ByteListRobinHoodVec128.fs)

I don't have a lot more information than that. I can tell you that the dictionary had about 50k elements, that the keys were all string representations of int64 values, and the values were structs of about 1k each. I'm not sure any of that is relevant :)

I've stared at the GetHashCode implementation for a long while and I'll be darned if I can see a problem. If I replace it with one of the StringComparer implementations in System the problem disappears. The exception does say that "this is often and indication that other memory is corrupt", but I'm not sure where that would be. All the rest of the code is either standard .NET 6.0 stuff or MongoDB queries.

If you have any insight, I'd love to hear it.

I started using the implementation in ByteListRobinHoodInline.fs per your suggestion in the previous issue I raised.

When the dictionary is empty, Bucket.IsLast never seems to evaluate to true. As a result, lookups in an empty dictionary cause an infinite loop.

I added a check for count = 0 where it checks IsLast in the main body of the Item function and that seems to have sorted it. But obviously that adds another CMP instruction and that's not optimal, is it? :) I was wondering if you knew of another way to do it.