Rust tools for EVT2, EVT2.1, and EVT3 recordings: event filtering, cluster tracking, IMU rotation compensation, and foreground detection.

Use a current stable Rust toolchain. Run commands from the repository root. MP4 export requires ffmpeg on PATH. Install Git LFS to download the example recordings after cloning:

git lfs install
git lfs pull
cargo build --release --locked
cargo run --release -- examples/ball_klein_3.raw view
cargo run --release -- examples/ball_klein_3.raw track-ball --max-detections 10
cargo run --release -- --help

examples/ contains a 25 cm ball recording (ball_gross_1) and a 7 cm ball recording (ball_klein_3), about 30 MB combined. RAW files use LFS; matching IMU CSVs, camera calibration, and estimated alignment profiles use regular Git.

view provides playback, a timeline, filter controls, and optional 3D ball projection. Hover a control for its description. track-ball prints detections. These commands use the cluster tracker; IMU processing uses motion-compare.

The parser reads the RAW header by default. Use --format evt2, evt21, or evt3 to override it. Use --endian little32 for EVT2.1 input with little-endian 32-bit halves. Unknown event words are ignored by the tracker.

Provide a RAW recording, its IMU CSV, and calibration.json. The CSV must contain timestamp in seconds and gx,gy,gz in rad/s. The IMU must be rigidly attached to the camera. Calibration must match the recording resolution and contain image_width, image_height, a 3×3 camera_matrix in pixels, and distortion_coefficients in [k1,k2,p1,p2,k3] order. Use --calibration to select another file.

Run an example with its saved time offset and IMU-to-camera rotation:

cargo run --release -- examples/ball_klein_3.raw motion-compare \
  --imu examples/ball_klein_3.csv --calibration examples/calibration.json \
  --alignment examples/ball_klein_3.alignment --start-s 4

The viewer starts paused. The top row shows events with rotation compensation off and on. The bottom row applies foreground detection to each result. All panels use the same timestamps, lens correction, and brightness scale.

Reuse the alignment to export a four-panel video:

mkdir -p outputs/motion-comparison
cargo run --release -- examples/ball_klein_3.raw motion-compare \
  --imu examples/ball_klein_3.csv --calibration examples/calibration.json \
  --alignment examples/ball_klein_3.alignment \
  --start-s 4 --end-s 6 --export-mp4 outputs/motion-comparison/ball_klein_3.mp4

Omit --export-mp4 to open the viewer. Replace it and its path with --benchmark to measure contiguous processing windows. Existing MP4 and alignment files are not overwritten.

For a new recording, omit --alignment and add --save-alignment outputs/motion-comparison/<name>.alignment to estimate and save its alignment. Use calibration for that camera and resolution.

Automatic alignment searches ±4 s and estimates a fixed mounting rotation. It requires background structure and varied camera rotation. Weak fits fail. Use --alignment-range to change the search range, --auto-align to recalibrate, or --motion-offset and --imu-to-camera to supply manual values. The mounting matrix has nine comma-separated values in row-major order.

An alignment file has two lines: offset in seconds, then the mounting matrix. The time convention is relative IMU time = relative camera time + offset, with each stream measured from its first sample or event. Each recording needs its own time alignment. The CSV's arrival times and absolute orientation are unused.

The gyroscope measures angular velocity. The code integrates it into relative rotations, converts these to camera axes, and projects each undistorted event into the camera orientation at the end of the window. Foreground detection then selects regions with recent activity that extends beyond earlier background activity. It detects moving regions; it does not identify balls.

Defaults are a 30 ms window, 3 px cells, at least 3 events per cell, a normalized mean-time residual above 0.12, and at least 8 connected cells. At least 10% of each region must lie outside the neighborhood of activity in the first quarter of the window. Tune --motion-window-ms, --motion-cell-px, --motion-threshold, --motion-min-cells, and --motion-min-new-fraction. Set the last option to 0 to disable the early-activity check.

Compensation covers rotation. Translation, parallax, lighting changes, and slow targets can cause errors. IMU gaps above 50 ms or missing coverage disable the compensated foreground. The benchmark includes decoding, compensation, both foreground paths, and CPU panel rendering. It excludes setup, sensor transport, GPU display, and video encoding. Live ingestion is not implemented; it requires timestamped camera/IMU input, alignment buffering, and an end-to-end latency test.

motion::compensate accepts one event window. Output masks preserve input event indices. Camera rays are cached; rotations use 0.5 ms bins. Processing uses no future event windows, but IMU interpolation requires samples bracketing the window.

cargo test --locked

Git excludes recordings (*.raw, root CSVs/ZIPs, motion-compensation/), calibration.json, generated output (outputs/, MP4s, alignment profiles), viewer settings, logs, OS metadata, and target/. The recordings, calibration, and alignment profiles in examples/ are exceptions. Keep exports and local diagnostics under outputs/. Keep reusable source outside that directory. Commit source, .gitattributes, Cargo.toml, and Cargo.lock.