ngx_txid is a module that exposes $txid: a cached, request scoped, 20
character, base32hex encoded, temporally and lexically sortable, case
insensitive, 96 bit identifier.
Use $txid to correlate logs or upstream requests.
- Configure and nginx with
--add-module=path/to/ngx_txid
Assuming you want to to store your logs in column format. You'll need a join
key for each request even across connections regardless of how accurate the
scheduler is from your timer_resolution directive. The remaining columns can
be specific per data logged.
log_format agent '$txid $http_user_agent';
log_format referrer '$txid $http_referer';
log_format peer '$txid $remote_addr';
log_format status '$txid $status';
log_format request '$txid $request';
log_format conn '$txid conn:$connection reqs:$connection_requests pipe:$pipe';
server {
listen 80;
server_name example.com;
access_log logs/example.com/conns.log conn;
access_log logs/example.com/agents.log agent;
location / {
remove_header X-Request-Id;
add_header X-Request-Id $txid;
proxy_set_header X-Request-Id $txid;
proxy_pass http://localhost:8080;
}
}
The design of this transaction ID should meet the following requirements:
- Be roughly numerically temporally sortable with ~second granularity.
- Have a representation that is roughly lexically sortable with ~second granularity.
- Have a probability of less than 1e-9 for collision at 1 million transactions per second.
- Be efficient and easy to decode into fixed size C types
- Always be available at the risk of higher collision probability
- Use as few bytes as possible
- Work with IPv4 and IPv6 networks
Use a monotonic millisecond resolution clock in the high 42 bits and system entropy for the low 54 bits. Use enough entropy bits to satisfy a collision probability at a desired global request rate.
+------------- 64 bits------------+--- 32 bits ----+
+------ 42 bits ------+--22 bits--|----------------+
| msec since 1970-1-1 | random | random |
+---------------------+-----------+----------------+
A request rate of 1 million per second across all servers means 1000 random
values per millisecond. Estimating the collision probability using the
birthday paradox can be done
with this formula: 1 - e^(-((m^2)/(2*n))) where m is the number of ids and
n is the number of random values possible.
When using 54 bits of entropy:
1mil req/s = 1 - exp(-((1000^2) /(2*2^54))) = 2.775558e-11
10mil req/s = 1 - exp(-((10000^2)/(2*2^54))) = 2.775558e-09
The odds of collision are small even at 10 million requests per second.
Nginx keeps track of the current clock in increments of the configuration
directive timer_resolution. The clock resolution for $txid is 1ms, so a
timer resolution greater than 1ms means that the probability of collision will
increase. If you have a timer_resolution of 10ms, 1 million requests per
second would require 10,000 random values per second in the worst case.
Base32hex is used with a lower case alphabet and without padding characters is chosen for the following reasons:
- lexically sort order equivalent to numeric sort order
- case insensitive equality
- lower case is easer for visual compares
- denser than hex encoding by 4 bytes
-
snowflake: uses time(41) + unique id(10) + sequence(12).
- Pro: guaranteed unique sequences
- Pro: fits in 63 bits
- Cons: requires unique id coordination for each server - 16 workers processes per host means a limit of 64 instances of nginx
- Cons: only 11 bits available for unique id, needs monitoring
- Cons: total ordering only possible in the same process
- Cons: service interruption possible when clocks lose synchronization
-
flake: uses time + mac id + sequence.
- Pro: guaranteed unique sequences
- Cons: uses 128 bits
- Cons: wastes 22 bits of timestamp data
- Cons: only a single process per host can generate ids - needs to synchronize access to the sequence from each worker process
- Cons: service interruption possible when clocks lose synchronization
- Cons: needs cross platform MAC Address lookup.
-
UUIDv4: 122 bits of entropy
- Pro: very low probability of collision
- Cons: unsortable
-
UUID with timestamp: 48 bits of time + 74 bits entropy
- Pro: very low probability of collision
- Cons: string representation is not temporally local
-
httpd mod_unique_id: host ip(32) + pid(32) + time(32) + sequence (16) + thread id (32)
- Pro: deterministic
- Cons: uses 144 bits
- Cons: assumes unique IPv4 for the hostnamme's interface
- Cons: unsortable case-sensitive custom representation - base64 with a custom alphabet
- Cons: hard limit of 65535 ids per second per pid - small tolerance for clock steps
This module uses the perl Test::Nginx
module with tests in the t directory with a specialized build of nginx of the
version specified in Makefile. A local install of dependent CPAN modules
will be downloaded with cpanminus into t/ext. Sudo/root access is not
required.
make test
Memory profiling of the tests can be performed when valgrind is installed.
make grind
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent