suyash / ulid Goto Github PK

Issue Description

As mentioned in b93d569#r43494808, the changes to EncodeEntropyRand in #7 cause the random component to be 0 when the function is called on a ulid. The problem occurs with how casting is happening here:

inline void EncodeEntropyRand(ULID& ulid) {
	ulid.data[6] = (uint8_t)(std::rand() * 255ull) / RAND_MAX;

In C++ order of operations, a c style cast happens before multiplication or division. That means this expression is resolved:

 (std::rand() * 255ull)

then cast as a uint8_t before the division by RAND_MAX occurs. Thus, you end up with a number always less than 0 (since a unint8_t can only hold up to 255, and RAND_MAX (on 32 bit platforms) is 2147483647).

Fix

A fix was proposed in #9. However, I think that only capturing the last 0xFF part of the output of rand() may be suboptimal (see std::rand() notes here).

Another fix would be to force the correct order of operations and cast the entire expression. I've tested that this is working.

Using this library in a linux application, compiled with g++:

/home/xxxx/percona-server-8.0.32-24/plugin/ulid/ulid/src/ulid_uint128.hh:11:5: warning: "_MSC_VER" is not defined, evaluates to 0 [-Wundef]
 #if _MSC_VER > 0
     ^~~~~~~~

A quick google says _MSC_VER is something specific to Visual Studio Code. Please add an #ifdef before checking the value so that other IDEs/compilers don't complain.

Could you clarify please is there monotonic sort order implemented?

using namespace std;

map<string, bool> values;

mt19937 rng;
rng.seed(random_device()());
uniform_int_distribution<mt19937::result_type> dist(1, 100000);

for (auto i = 0; i < 10; ++i) {
    auto msec   = chrono::duration_cast<chrono::milliseconds>(chrono::system_clock::now().time_since_epoch()).count();
//        auto sec    = chrono::duration_cast<chrono::seconds>(chrono::system_clock::now().time_since_epoch()).count();

    auto ulid = ulid::Create(msec, [&]() {
        return dist(rng);
    });

    auto marshal = ulid::Marshal(ulid);
    cout << "ULID: " << marshal << " TIME:" << msec << endl;
    values.insert(pair<string, bool> (marshal, true));
}

cout << endl;

for (auto it = values.begin(); it != values.end(); ++it) {
    cout << (*it).first << endl;
}

As result:

ULID: 01F0Q46HRRM8TN2DYE2QAYFBCV TIME:1615683864344
ULID: 01F0Q46HRRHEHPZY95X06GF250 TIME:1615683864344
ULID: 01F0Q46HRRD7QSH5481ED4NJDV TIME:1615683864344
ULID: 01F0Q46HRRE4QDCAK58JQJA1CF TIME:1615683864344
ULID: 01F0Q46HRRDTW2XXMN28B54888 TIME:1615683864344
ULID: 01F0Q46HRRC0QVB15CHGRB8DVS TIME:1615683864344
ULID: 01F0Q46HRR6WJFE1YM6BEB8XNZ TIME:1615683864344
ULID: 01F0Q46HRR8J4MS9DYW2F7QKBD TIME:1615683864344
ULID: 01F0Q46HRRNMS141PFKDGA2TY6 TIME:1615683864344
ULID: 01F0Q46HRR9P6D9ZZYJCV5MX9E TIME:1615683864344

01F0Q46HRR6WJFE1YM6BEB8XNZ
01F0Q46HRR8J4MS9DYW2F7QKBD
01F0Q46HRR9P6D9ZZYJCV5MX9E
01F0Q46HRRC0QVB15CHGRB8DVS
01F0Q46HRRD7QSH5481ED4NJDV
01F0Q46HRRDTW2XXMN28B54888
01F0Q46HRRE4QDCAK58JQJA1CF
01F0Q46HRRHEHPZY95X06GF250
01F0Q46HRRM8TN2DYE2QAYFBCV
01F0Q46HRRNMS141PFKDGA2TY6

You can see that the selected value is not in the same order as it should be. I understand that map-container keeps values sorted and they are not even in reverse order. As we can see their time is the same.
I also compared how it works in PHP (https://github.com/rorecek/laravel-ulid and https://github.com/ulid/javascript).
Here is console output:

>>> $values = [];
=> []
>>> for($i = 0; $i < 10; ++$i) { $ulid = ulid(); $values[] = $ulid; echo $ulid . PHP_EOL;}
01F0Q59G42ZHN51J4AYR6F2XHV
01F0Q59G43CKCHAPCFWRXQEJHX
01F0Q59G44D1QHZGS9ZSZGQ78H
01F0Q59G45AG3KXAZMZ5JQ9B30
01F0Q59G46ZZZ616T9YE2TQ2E8
01F0Q59G47XB98SAJETRMG659C
01F0Q59G48MZXWAXK4APFQG90A
01F0Q59G49CFSP07JXWQ9ADF1E
01F0Q59G4AVC3D99EFCG7SYGA0
01F0Q59G4BHDJY7W2HERNG7EGB
>>> asort($values);
=> true
>>> print_r($values);
Array
(
    [0] => 01F0Q59G42ZHN51J4AYR6F2XHV
    [1] => 01F0Q59G43CKCHAPCFWRXQEJHX
    [2] => 01F0Q59G44D1QHZGS9ZSZGQ78H
    [3] => 01F0Q59G45AG3KXAZMZ5JQ9B30
    [4] => 01F0Q59G46ZZZ616T9YE2TQ2E8
    [5] => 01F0Q59G47XB98SAJETRMG659C
    [6] => 01F0Q59G48MZXWAXK4APFQG90A
    [7] => 01F0Q59G49CFSP07JXWQ9ADF1E
    [8] => 01F0Q59G4AVC3D99EFCG7SYGA0
    [9] => 01F0Q59G4BHDJY7W2HERNG7EGB
)

As you can see after the sorting we have the same order as it was in loop during generation.
Sorry if I maybe misunderstood how to use this lib. Is there a way to get that monotonic sort order (Ulid spec https://github.com/ulid/spec)?

Problem Description

Per the ULID spec, the timestamp portion of the ULID is supposed to be the Unix epoch time in ms. The current implementation has several API calls that encode the time in seconds, not milliseconds. time_t stores the time in seconds, and the following calls use it:

• void EncodeTime(time_t timestamp, ULID& ulid)
• void EncodeTimeNow(ULID& ulid)
• void Encode(time_t timestamp, const std::function<uint8_t()>& rng, ULID& ulid)
• void EncodeNowRand(ULID& ulid)
• ULID Create(time_t timestamp, const std::function<uint8_t()>& rng)
• ULID CreateNowRand()

The exception to this is EncodeTimeSystemClockNow, which properly encodes ms timestamps.

The second problem around this is that time_t is only 4 bytes on 32 bit systems (noted in #9), so a call to time_t Time(const ULID& ulid) on those systems will not work correctly past a certain time.

Potential Fix

I think we should internally fix the API so that it encodes in ms, and ensure that the API only allows encoding the time as ms (using chrony data types can enforce this). We also shouldn't use time_t as a data type when we need 8 bytes of space to do operations.