shared_mutex? about parallel-hashmap

[greg7mdp]

I think I can support the upgrade lock cleanly as well. This is a pretty general concept. You can get a shared lock, a unique lock, and if necessary you can upgrade a shared lock to a unique lock.

[greg7mdp]

Hi Vinnie,

The current version uses a more flexible locking mechanism which should support boost::shared_mutex and boost::upgrade_mutex, although I haven't tested with these yet. The only location internally where we actually upgrade the mutex, though, is in erase (so we don't unique_lock if we attempt to erase a value that is not present).
In order for this to work, the boost mutex headers have to be included before phmap.h.

[greg7mdp]

Hi Vinnie, I validated the use of the boost::upgrade_mutex in phmap (which is the same class as shared_mutex), so closing the ticket. Feel free to ask if you have further questions.

[greg7mdp]

This is a very good point. I'll look into improving the interface (template parameter) to allow for this use case.

Indeed you can work around this, but it would be nicer if the parallel hashmap supported this natively.

[vinniefalco]

it would be nicer if the parallel hashmap supported this natively.

I'm not sure that I agree. Human creativity is effectively infinite, and it is unreasonable to expect that the parallel hashmap should handle every imaginable use-case simply by adjusting one template parameter. Instead, I would consider simply making sure that the functionality exposed by the hashmap (adding members if necessary) is sufficiently flexible that authors who desire custom behavior are able to achieve it through efficient composition of the hashmap container.

For example, I would consider the normal hashmap interface to be "sufficient flexible" if it is possible to build a parallel hashmap from it with no loss of performance, using only the public interfaces (is it?)

[greg7mdp]

I was thinking that instead of providing a mutex as a template parameter, I could allow for a class that has two lock/unlock methods, so you'd be able to use for example:

class StdLockable {
public:
    void lock()          { m_.lock(); }
    void unlock()        { m_.unlock(); }
    void lock_shared()   { m_.lock(); }
    void unlock_shared() { m_.unlock(); }

private:
    std::mutex m_;
};

class SharedLockable{
public:
    void lock()          { m_.lock(); }
    void unlock()        { m_.unlock(); }
    void lock_shared()   { m_.lock_shared(); }
    void unlock_shared() { m_.unlock_shared(); }

private:
    boost::shared_mutex m_;
};

Internally the phmap would use lock_shared for the red APIs and lock for the mutable ones.

[greg7mdp]

For example, I would consider the normal hashmap interface to be "sufficient flexible" if it is possible to build a parallel hashmap from it with no loss of performance, using only the public interfaces (is it?)

No it isn't. For example there is no way to avoid recomputing the hash in the internal submaps using the std interface.

[vinniefalco]

there is no way to avoid recomputing the hash in the internal submaps using the std interface.

Then I would explore this direction. Is there a clean way to provide an interface that avoids the recomputation of the hash?

I suggest this because, mutexes are notoriously difficult to compose. You can have a parallel hashmap with a built-in mutex that works great by itself, but then when you add surrounding code that has its own mutex and locking, it no longer works ideally. For example, you might want to atomically insert two elements at once. This is not possible if the mutex is an implementation detail of the parallel map. To support these use-cases, it is sufficient to allow users to build up arbitrarily complex parallel maps using the regular map as a building block - as you have pointed out, there needs to be some visibility and control over how and when hashes are calculated for this to work efficiently.

[greg7mdp]

Well, for complicated use cases as you suggest, it is better to keep the locking separate. I don't really see the use case where the additional interface you suggest would be useful.

However, I still think that internal locking can be convenient for simple use cases, and that I need to support shared_locks.

[vinniefalco]

I still think that internal locking can be convenient for simple use cases, and that I need to support shared_locks.

Okay. What about upgrade locks then? On insertion, acquire the shared lock to determine if the element already exists, and then if not, upgrade the shared lock to an exclusive lock to modify the map.

[greg7mdp]

So the sequence is something like this?

m.lock_upgrade();
find();
if (present)
{
    m.unlock_upgrade();
    return;
}
m.lock();
insert_item();
m.unlock();
m.unlock_upgrade(); // not sure about that last one

[vinniefalco]

The UpgradeLockable concept is a Boost extension, you can read about it here:
https://www.boost.org/doc/libs/1_69_0/doc/html/thread/synchronization.html#thread.synchronization.mutex_concepts.upgrade_lockable

I bring up the upgrade lock approach not because I am suggesting that you should implement it, but to show how there are uncountably many different approaches, and tying the parallel hashmap to one solution will necessarily disadvantage others.