jhuckaby / megahash Goto Github PK
View Code? Open in Web Editor NEWA super-fast C++ hash table with Node.js wrapper, tested up to 1 billion keys.
License: Other
A super-fast C++ hash table with Node.js wrapper, tested up to 1 billion keys.
License: Other
@jhuckaby Would you accept a PR which makes the megahash interface compatible with ES6 Map?
If so, what would be your preferred option?
E.g. Using class inheritance like:
const MegaHash = require('./main.js')
class Map extends MegaHash {
get size () {
return this.length()
}
* keys () {
let key
while (1) {
key = this.nextKey(key)
if (key === undefined) break
yield key
}
return key
}
//...
}
module.exports = Map
or adding the prototypes in main.js
?
Currently we don't have any definition file for this package. How about adding them?
I've made my own implementation and it works good enough to be used in global:
declare module 'megahash' {
export default class MegaHash<K, V> {
/**
* Set or replace one key/value in the hash.
* Ideally both key and value are passed as Buffers, as this provides the highest performance.
* Most built-in data types are supported of course, but they are converted to buffers one way or the other.
* It actually returns a number that means type of result: 0 = error (out of memory), 1 = success, 2 = success (replaced existing value)
* @param {V} key unique key
* @param {K} value value to store
*/
set(key: K, value:V ): 0 | 1 | 2
/**
* Fetch a value given a key.
* Since the value data type is stored internally as a flag with the raw data, this is used to convert the buffer back to the original type when the key is fetched.
* So if you store a string then fetch it, it'll come back as a string. If there's no such value - it gonna reply back with undefined.
* @param {K} key unique key
*/
get(key: K): V
/**
* Tests given key. Returns true if there is a some value connected with this key and false otherwise.
* @param {K} key unique key
*/
has(key: K): boolean
/**
* Deletes key & connected to it value from hash. Returns true if it finds value and deletes it, false otherwise.
* @param {K} key unique key
*/
delete (key: K): boolean
/**
* Delete all keys from the hash, effectively freeing all memory (except for indexes).
* You can also optionally pass in one or two 8-bit unsigned integers to this method, to clear an arbitrary "slice" of the total keys. This is an advanced trick,
* only used for clearing out extremely large hashes in small chunks, as to not hang the main thread.
* Pass in one number between 0 - 256 to clear a "thick slice" (approximately 1/256 of total keys).
* @param {number} from
* @param {number} to
*/
clear (from?: number, to?: number): void
/**
* Returns a number of keys currently allocated in mega hash (map).
*/
length (): number
/**
* Fetch statistics about the current hash, including the number of keys, total data size in memory, and more. The return value is a native Node.js object with several properties populated.
*/
stats (): MegaHashStatistics
}
interface MegaHashStatistics {
numKeys: number
dataSize: number
indexSize: number
metaSize: number
numIndexes: number
}
}
Line 28 in cc0c1da
When trying to store structures like AVL Trees and others that reference "themselves", the Stringify function throws this error. There's an option when calling the function to avoid this error, but I'm not sure how it impacts the rest of its usage in this module.
Hi,
It looks like the objects stored are somewhat serialized, instead of storing object references, so it's not a dropping replacement for ES6 Map.
Is that a bug or an undocumented feature?
E.g.:
const Hmap = require( 'megahash' ) ;
hmap = new Hmap() ;
var o = { a: 1 } ;
hmap.set( 'key' , o ) ;
console.log( hmap.get( 'key' ) === o ) ; // expected true, but got false
console.log( hmap.get( 'key' ) ) ;
o.b = 2 ;
console.log( hmap.get( 'key' ) ) ; // expected {a:1,b:2} but got {a:1}
Essentially, Redis is also a key-value data store and is used in plenty of production software. I was just curious to see if you could compare Redis and megahash?
Hello, I am interested in this project and would like to contribute, is it still actively maintained? I see a few PRs have been open for a while...
I would also like to ask, have you experimented with the xxHash algorithm? While they claim it to be the fastest hashing algorithm around, current JS implementations pale in comparison to DJB and FNV but implementing it in C++ could be an interesting thing to test.
Regards!
I tough it was a serious blame of the system integrity; hence I investigated it further to be sure.
Example:
when updating keys '4723640122' and '4723640123' in a 100k elements list, the key '4723640123' as disappearing from the Map.
This is due to those keys having a close hash [0, 0, 0, 0, 11, 4, 5, 4] and [0, 0, 0, 0, 11, 4, 5, 5] and hence ending on the same leaf list. The issue is not to complex to reproduce as long as two keys stay in the same linked-list update of one will affect the other (but not the other way around).
The second key will disappear (effectively returning undefined from get and false from has) but the key counter from stats will not decrease but will further increase if the key is set again.
It is quite serious as it causes almost 40% of the key/value pair being lost (and not freed from memory).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.