paralleldrive / cuid Goto Github PK
View Code? Open in Web Editor NEWCollision-resistant ids optimized for horizontal scaling and performance.
License: Other
Collision-resistant ids optimized for horizontal scaling and performance.
License: Other
Hey! @ericelliott I know there's already a PHP version of the package but I wanted a literal import of the node version. When you have time can you take a look at it? https://github.com/Shoperti/Cuid Thanks :)
I installed the package with component install, after the dist folder is just empty. Used this component a lot and it seems like it doesn't work anymore for some days.
Hi,
I have to generate unique and not guessable api keys.
Those keys will give user full access to the application via REST API.
Was wondering if using cuid is safe enough, and what are the benefits over using something like
https://www.npmjs.com/package/hat
or
https://www.npmjs.com/package/node-uuid
thanks
How do you feel about using getRandomValues
from the Web Crypto API in the environments that support it for randomBlock
?
Hi,
Is cuid will always return a length 25 string ?
Need to know for setting VARCHAR length.
Couldn't find an exact answer about it.
Thanks !
module.exports = (function() {
I try the following. only slug(), and fingerprint return 5 chars.
How do I get longer one?
Regards,
var cuid = require('cuid');
var id = new cuid();
return function(req, res, next) {
console.log('cuid module');
//console.log(cuid);
//console.log(id);
//console.log(cuid.fingerprint()); // short, 5 chars
//console.log(cuid.slug()); // work, short one
console.log(cuid.cuid());
//console.log(cuid.nodePrint());
console.log(id.cuid());
//console.log(id.fingerprint());
//console.log(id.slug()); // work, short one
//console.log(id.nodePrint());
next();
}
})();
I'm using cuid
for logging stuff into a document database in objects like this:
{
id: cuid(),
timestamp: new Date.getTime(),
type: 'content-specific',
content: {...some_data}
}
If I could somehow safely reconstruct the timestamp
back from id
, I could omit having to store the timestamp in the database thus saving myself a lot of space.
Any option to add such feature?
Hi,
Was wondering what your thoughts were on the potential of spoofed Id's originating from untrusted clients.
One of the things that appeals to me about cuids are the fact that id's are more or less sequential (assuming consistent time sources), which means I can use them as PK's in my DB and won't get burnt on inserts (due to the rows being stored.as a clustered index),
This does however leave an unsavory taste in my mouth - as an untrusted client could screw with my DB index by generating unique, but non sequential id's.
I was thinking of performing some sanity checking server side on any new id's generated client side. Nothing too restrictive, I would probably just check the basic format is correct and that the timestamp fits within a suitably large window. I then noticed this in the README:
"The cuid specification should not be considered an API contract. Code that relies on the groupings as laid out here should be considered brittle and not be used in production."
Do you believe this applies in the stated context?
Regards
Rob
Applitude is defunct. It's past time to remove references to it from Cuid.
Because the popular bundling tools may not bundle stuff from node_modules
our package.json
may need to point to bundled versions of the library, instead of individual modules. I've tried doing this with Browserify, but then Webpack gives weird warnings to all the users of our module (I think -- it used to, anyway).
We could try to accomplish this with rollup.
For this to be complete we need to test:
Please add support for web workers.
cuid()
currently throws an exception in web workers since navigator.mimeTypes
is not available in that context.
Generally, for the fingerprint I suggest to collect various strings as a whole, gracefully handling errors, and then create a fingerprint out of that (arbitrary length) string using some hash function - like MD5, but perhaps something more lightweight.
Something in this direction:
api.fingerprint = function browserPrint() {
var things = [];
try {
for (var i = 0; i < navigator.mimeTypes.length; i++) {
things.push(navigator.mimeTypes[i]);
}
} catch (e) {
// ignore errors
}
try {
things.push(navigator.userAgent);
} catch (e) {
// ignore errors
}
// maybe add some more...
things.push(api.globalCount());
return someHashFunction(things.join(""));
};
I think you get the idea...
I could being doing something really stupid (as is the case normally) but I can't get my tests to pass in Node with cuid.
error trying to resolve DateInputViewModel navigator is not defined ReferenceError: navigator is not d
efined
If I dig a little further, it looks like cuid is expecting the window.navigator
to exist... but obviously this is not the case when running in a Node process. I am using Testem's launchers to run this particular suite of tests, and perhaps that is where the error lies. Maybe Testem isn't setting an environment variable correctly and therefore some kind of cuid
check of if process.env === node
is failing?
Any ideas would be appreciated.
Eric, you write that the sequential nature of cuid makes for better performance when used as primary keys in databases. I'm currently trying to choose what id concept to use and the more I read the more I understand that this is hard.
It seems that both the placement of the timestamp part of the uuid AND big/little endian concept of different database engines and OSes matters. Do you have any plans to make cuid more compatible with different RDBMs by making the timestamp style tweakable?
Maybe some interesting reading: https://github.com/richardtallent/RT.Comb
I'm sorry to say that I don't have the competence to be able to PR something. I stumbled over cuid while researching. I have always held you high for competence and innovation in the Javascript community, and was hoping that "this is it, this could solve my id needs" :)
The standalone built versions (./dist) of cuid currently do only one of the two typical checks for CommonJS based amd. See:
else if (typeof module !== 'undefined') { module.exports = api; }
-- Line 104 dist/browser-cuid.js
when compared to the standard practice of:
if ( typeof module === "object" && typeof module.exports === "object" ) {
This caused a bit of a headache recently as another dependency (angular-mocks) when run through Karma/Jasmine actually stores a function in window.module. I wasn't sure the proper procedure for submitting a fix as it looks like the ./dist versions are not built through the npm scripts.
Thanks
Hello, I love what you have here. So, I have port it to PHP. Cheers.
Keep up the good work.
Regards,
Endy
padStart/padEnd are stage 3. It looks like they have a great chance of making it into ES2017.
The official polyfill is here:
https://www.npmjs.com/package/string.prototype.padstart
In slug() impl, the c value is not checked to be in the discrete values range like it is in cuid(). c is then directly incremented after usage.
c = (c < discreteValues) ? c : 0;
Intensive use of slug without any calls to fingerprint could lead to overflow.
I was looking through this project since it's a dependency for something else I was checking out.
I noticed that you are using Math.random(). This is fine but if so you shouldn't claim that the ids are unguessable:
Cuids contain enough random data and moving parts as to make guessing another id based on an existing id practically impossible.
Since Math.random() is not cryptographically secure it's easy to predict the next values. I would remove the two security/secure paragraphs completely and add a warning instead.
EDIT: Just for fun I wrote a proof of concept here that can predict all CUIDs(excluding timestamp portion) after seeing four consecutive ids. It's for node v4/v5 only since they've updated their RNG algorithm but the approach for later versions is similar.
Hi everyone,
I'm wondering what is the dev status of this library, is it still actively maintained?
Is it safe to bet on it for a new Node project?
Thanks for your answer :)
Hi, node-uuid author here. I just had this module brought to my attention and thought we might compare notes. I haven't done a thorough investigation, but it seems like cuid is pretty similar to node-uuid's RFC4122 v1 implementation.
RFC4122 uses a timestamp system that includes both the timestamp and counter concepts. The RFC timestamp interval is 100-nanoseconds, but in environments that lack this resolution, clients can use a counter of ids generated during the current timestamp interval to emulate the higher precision, which is what node-uuid does.
In terms of how these values contribute to uniqueness, it's about the same; the RFC timestamp is 64 bits.
The cuid timestamp + counter is ~62 bits
cuid attempts to generate a per-client unique id in the form of fingerprint
. In cuid (browser) this is built from these values:
userAgent.length
- Only unique to browser version + OSmimeTypes.length
- Only unique to browser (+ plugins? I'm not sure what contributes to this honestly)Object.keys(window).length
- Only unique to browser + application runtime... or in cuid (node):
For the reasons noted above, I'm concerned about how unique this "fingerprint" actually is in production.
node-cuid uses Math.random()
, which is (by definition) an RNG of uncertain quality.
Meanwhile in the RFC/node-uuid ...
The node
property should be either the host MAC address (unique to a fairly high degree of certainty) if available, or set randomly if not. In node-uuid it's set randomly.
node-uuid only uses Math.random() as the RNG of last resort, preferring higher-quality crypto
implementations where available.
The RFC also includes a 14-bit clockseq
value which is initialized randomly, and which increments any time there is a discontinuity detected in the system clock (e.g. if the clock regresses backward). This is similar to cuid's random
value, with the added benefit of preventing duplicates if something happens to the system clock, which is a feature I don't see in node-cuid.
In terms of how these contribute to uniqueness, the cuid fingerprint
+ cuid random
is ~ 62 bits (21 + 41 bits. Meanwhile the RFC node
+ clockseq
is also 62 bits (48 + 14 bits).
node-cuid offers strikingly similar features to RFC4122 v1 UUIDs, just in a slightly different form factor. While the node-cuid IDs are somewhat shorter (25 chars vs. 40 chars), that's simply a presentation choice; (e.g. converting from RFC notation to cuid notation and back is fairly trivial). In fact, I'm curious why node-cuid doesn't use base-62, for shorter-still ids, since that's one of the selling points.
More importantly, I don't see anything in cuid that suggests it is better at avoiding collisions. In fact, the fingerprint
issues, the reliance on Math.random(), and lack of clock regression detection logic suggest that it will be worse, not better, than an RFC compliant implementation.
(FWIW, I've spent a lot of time looking at the RFC trying to see if there's a way it might be improved. It's actually a very well thought-out spec, which is why I decided to focus on a rigorous implementation for node-uuid. The one aspect of it I'm not a fan of is the choice of gregorian epoch + 100-nanosecond increments for the timestamp field. I think the spec would have been better served with a standard unix epoch + 1 second increments, with an explicit counter
field, akin to what cuid has to handle id sequencing with a given timestamp unit. But the end result is more or less the same, so it's not that big a deal.
Anyhow, hope this is useful. And please do point out anywhere I may have misunderstood what you're doing.
Cheers!
wouldn't it make sense to store fingerprint in a variable and only run it once when needed?
... If they're generated on different browsers or machines, and then inserted to the database later, won't they be in some random order? Am I totally missing the point? (I think so, sorry about that!) This looks really cool, but I'm struggling to understand the bit about them being sequential. If they were only sequential because they were being generated on ONE machine, then why wouldn't you just use an auto-increment int or bigint?
Would you consider moving away from jshint/eslint to use Standard JavaScript instead?
Can I get the timestamp from the generated id?
For reasons related to a problem in a CUID 1.3.8 dependency, I'm very excited for the jump to 2.0.0
Any word on when it lands in NPM? Any API changes to know about?
Thanks!
https://usecuid.org/ uses Github's SSL certificate, which is only valid for *.github.io, not custom domains. Unfortunately, Github doesn't allow custom certificates, but you can switch to Netlify (which is free and should work as a drop-in replacement here).
Is there any way to validate a cuid? For example to distinguish it from a random string?
The documentation states that IDs generated through cuid.slug() weigh in at 8 characters, however the ones I'm seeing are 7 characters. Is this expected behaviour? Does the documentation mean they will be up to 8 characters?
The latest release/tag and npm package, v1.3.0, has in package.json
"main": "./build/server/cuid.js",
but includes no folder build
.
The package.json also still references the dist
folder, which was included in v1.2.4, but is not in v1.3.0.
The Travis build is currently failing. Looks like a broken package dependency.
More of a comment here for other react-native users that might run into the same issue, than a bug specifically, but because react-native doesn't provide a userAgent or mimeTypes by default, using cuid will throw an error.
This can be worked around by defining dummy values before requiring cuid;
global.navigator.mimeTypes = ''; //browser-fingerprint only checks the length property so an empty string is fine
global.navigator.userAgent = 'reactnative';
I did try using the node version but rn also doesn't set a process.pid so runs into the same issue.
Didn't update any package or system environment, but today when I call cuid()
I get
TypeError: process.pid is undefined
nodePrint
[path to my react component]
cuid
webpack:///./~/cuid/dist/node-cuid.js?:49:23
Using
cuid: 1.3.8
webpack: 2.2.0
webpack-dev-server: 2.2.0
node v6.9.4
npm v3.10.10
yarn 0.20.3
Tried on [email protected], got the same result.
"Cuid maintained by..."
Would be great to have a way to customize the starting character.
It could be given as a parameter to the cuid() function. The "c" could be the default if not given...
If customizable, one could create specific classes of IDs that would allow some backend decisions based on the type of ID received...
Hi,
If you are interested in mentioning it in your README file, I have ported this to Go. The package is not 100% finished but will be pretty soon.
Thanks for this lib :)
I got the following error.
When the process too fast, it generate duplicate key.
MongoError: E11000 duplicate key error index
Either JavaScript Pseudo random is not good.
I also think Timestamp have a problem too by looking at the key.
Let say, node is running with cluster and more then 10 thread. It is highly possible that both Timestamp and Pseudo random will be the same with high hit.
A way to let user add just one digit will be good enough I guess.
Hey,
I liked your cuid project so I ported it to Ruby, you can see it here: http://github.com/iyshannon/cuid.
Please check it out and let me know what you think, I haven't published it to RubyGems yet as I wanted to get your OK before doing so.
Thanks,
Ian
Is there any specific reason (along making it a bit longer than absolutely necessary) why the timestamp does not contain some padding (beginning zeros)... if it would, I think it could be used as a time order criteria on standard ASCII sorting
You can create a bin/env.sh
file in a bin dir and add it to .gitignore:
!/bin/bash
set KEY=value
Be sure to include a bin/env.sh.example
file and check that one into git.
Then as part of the scripts:
"test:browsers": "source bin/<filename>.sh"
bin/env.sh
bin/env.sh.example
bin/env.sh
instructions to READMEDue to the c{timestamp}... structure, I have concerns about "todays" items creating a hot spot in a distributed data store for my use-case (long history of items, todays items are used heavily, using aws DynamoDB).
Thoughts on the validity of my concern?
Please complete #24 first.
.
└── cuid
├── build
│ ├── client
│ └── server
├── source
│ ├── client
│ └── server
└── test
├── client
└── server
You can ignore build files in diffs.
Create a .gitattributes file in the repo root:
build -diff
See the prod-module-boilerplate package.json
package.json
The JS version of cuid needs to support all of these targets:
We'll use feature detection to select the correct entropy source and return the right fingerprint()
function for the environment.
Ideally, we should select entropy sources supported by all of the above environments.
Is there a regular expression to validate CUIDs?
Would it be possible to allow the cuid to be seeded by some arbitrary value? For example, I want to generate a cuid based on a uuid that is completely random. Does that make sense?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.