vineetsista/vsx

VSX: an electronic exchange in modern C++ - matching engine, ITCH replay, binary protocols, trading agents, live viz

★ 1Forks 0C++GitHub ↗Compare
cppcpp20exchangelock-freelow-latencymarket-datamatching-enginenasdaq-itchorder-booktrading-systems

README

VSX

A complete electronic exchange in modern C++20, built from scratch: deterministic matching engine, sequenced binary market data feed (VITCH), binary order-entry gateway (VOUCH), NASDAQ ITCH 5.0 replay, an ecosystem of algorithmic trading agents, and a live web visualization of the running market.

VSX flash crash demo

Live capture of ./scripts/demo.sh flash_crash: all market makers are killed at T+60s — the spread blows out from 4 to ~40 ticks and momentum takers punch holes in the thinned book — then the MMs return at T+120s and the market heals. Nothing staged: this GIF is 71 real frames of the running system.

What this is

Everything interesting is hand-rolled — that's the point. The only third-party C++ dependencies are Catch2 (tests) and Google Benchmark. No Boost. The book, the pools, the hash tables, the ring buffers, the wire protocols, the histograms: all here, all tested, all measured.

Matching core single-threaded, deterministic (LMAX-style); byte-identical replay enforced by a golden-digest test in CI
Hot path zero heap allocations after startup, enforced by a counting-allocator test that runs under ASan
Order book anchored price-window array (O(1) level access) + sorted overflow; intrusive FIFOs of pool-resident 64-byte orders linked by 32-bit slot indices
Correctness differential fuzz vs an independently written std::map reference engine: identical event streams over 10M ops in CI
Market data VITCH: LE binary, Mold-style sequencing over UDP; book provably rebuildable from the feed (exact equality at every 1M-message checkpoint)
Order entry VOUCH: OUCH-style binary TCP, epoll gateway, per-session sequencing, binary session logs
Real data full-day NASDAQ ITCH 5.0 (268.7M messages) parsed with zero errors and replayed through the engine in 115s
Agents noise traders, two market-maker designs (Python + C++), momentum takers, a labeled spoofer specimen; YAML scenarios with per-agent P&L and an exact zero-sum conservation check

Measured (see BENCHMARKS.md for methodology and caveats)

Metric Result
Synthetic throughput (release, quiet machine) 11.2 – 11.8 M ops/s across passive/matching/trend scenarios
Add-order latency, p99 (serialized rdtsc) 340 – 464 ns
Full-day ITCH replay (268.7M msgs, zcat-streamed) 115 s (2.34M msg/s)
VOUCH order->fill RTT over TCP (p50/p99) 188 µs / 573 µs
vs. naive std::map book (same streams, same session) 1.3 – 1.5x (honest analysis of why the gap isn't bigger is in BENCHMARKS.md)

Every number above was measured on the stated hardware with the stated commands; raw outputs are committed under bench/results/. Unmeasured things are labeled unmeasured.

Architecture

  itch replayer / agents (python + c++)
        |
        v
  gateway (TCP, VOUCH) ---> [SPSC ring] ---> MATCHING CORE (one thread, deterministic)
        ^                                        |
        |                                        v  events
        +--------------- [SPSC ring] <-----------+
                                                 |
                                                 v
                             feed publisher (VITCH over UDP, unicast fan-out)
                                   |                    |
                                   v                    v
                          bookbuilder (C++)      Node WS bridge --> React viz

Concurrency exists only at the edges: one epoll I/O thread and one feed I/O thread meet the matching thread at single-producer/single-consumer rings. The core never reads a clock, never allocates, never iterates a hash table — same input stream, byte-identical output stream, forever.

Build and run

Requires Linux (developed on WSL2 Ubuntu 24.04): g++ 13+, CMake 3.25+, Ninja, Python 3.11+ with PyYAML, Node 20+.

# On a fresh Ubuntu/WSL2 box:
sudo apt install build-essential g++-13 cmake ninja-build python3-yaml nodejs npm

git clone https://github.com/vineetsista/vsx.git && cd vsx
cmake --preset release && cmake --build --preset release
ctest --preset release                     # full suite (200k-op fuzz default)
VSX_FUZZ_OPS=10000000 ctest --preset release -R fuzz   # the CI-sized 10M-op gate
./scripts/demo.sh flash_crash              # the show: open http://127.0.0.1:5173

Other presets: debug, asan (ASan+UBSan — the whole suite runs clean under both), bench (adds Google Benchmark targets; run via ./scripts/bench.sh, which refuses to measure on a loaded machine).

Real market data (optional, ~3.3GB): ./scripts/fetch_itch.sh, then

zcat data/12302019.NASDAQ_ITCH50.gz | build/release/itch/vsx_itch_replay replay - AAPL,MSFT,TSLA

Layout

Directory Contents
core/ matching engine: book, matching, order pool, id map, symbol table
itch/ NASDAQ ITCH 5.0 zero-copy parser, replay translator, validation oracle
synth/ seeded synthetic market generator (integer-only OU fundamental)
feed/ VITCH publisher, SPSC ring, datagram packer, independent bookbuilder
gateway/ VOUCH epoll gateway, the vsx_exchange daemon, C++ client SDK
sdk/python/ VOUCH + VITCH Python SDKs
agents/ trading agents, YAML scenarios, the scenario runner
viz/ Node WS bridge + React/canvas live market display
bench/ rdtsc + HDR-histogram instrumentation, GBench suite, naive baseline book
docs/ protocol specs, walkthrough, interview drill
phase_reports/ per-phase build reports with real measurements and known gaps

Documentation

  • docs/WALKTHROUGH.md — a guided ~2-hour tour through the six files that matter most, in reading order.
  • docs/INTERVIEW_DRILL.md — 40 interviewer-style questions with answers referencing file:line.
  • DECISIONS.md — the engineering decision log: every non-obvious call, its rationale, and what was rejected.
  • BENCHMARKS.md — methodology-first performance numbers, v1 and the post-profiling v2, including the regressions.
  • docs/protocols/VITCH.md / VOUCH.md — the wire specs, written before their implementations.

Honest limitations

  • One box, one instance. No redundancy, no failover. There IS now write-ahead journaling with crash recovery (--journal, added in v1.1: replay the op log through a fresh engine, resnapshot the feed), but durability is batch-flush not per-op fsync — the crash window is the OS page cache. No cross-machine replication.
  • WSL2 tails. Every max-latency figure in this repo includes hypervisor descheduling; p50/p99 are meaningful, maxima are the machine's.
  • The engine-vs-baseline gap widens with depth, now measured (v1.1): 1.8x at 8 levels, 2.85x at 4096 (BENCHMARKS.md v3). The engine is depth-independent; std::map degrades as its tree deepens. The oft-quoted "only 1.3–1.5x" is the shallow-book number.
  • ITCH replay is intent reconstruction: the original taker orders are not in the stream, so E/C become synthetic IOCs. FIFO fidelity is 99.87% per event; end-of-day level agreement 96.7% — the honest ceiling without hidden-liquidity data, with the divergence counted, not hidden.
  • Toy auth on VOUCH (any user, password "vsx"), no TLS, no cancel-on-disconnect (a deliberate, documented default). Session replay-on-reconnect is spec'd but not built.
  • Scenario "determinism" is seeded macro-phenomena (the crash reproduces from seeds; observed 40- and 38-tick blowouts back to back), not byte-identical tapes — sockets and threads see to that. Engine-level determinism is the strong claim and is enforced byte-for-byte.
  • The spoofer is a labeled specimen; a heuristic layering detector now watches for its pattern in the viz (v1.1, viz/bridge/surveillance.mjs) and flags it live, but it is pattern-based (no participant identity in the feed) and surfaces false positives honestly — an alert flags for a human, it does not convict. It never touches anything but this closed synthetic market.

Contributors

vineetsista

Issues