SunsetWan/xcsift-python

Streaming Python port of xcsift for token-efficient Xcode and Swift build output

★ 0Forks 0PythonGitHub ↗Compare

README

xcsift-python

xcsift-python is a zero-runtime-dependency Python port of xcsift, the token-efficient parser and formatter for xcodebuild and Swift Package Manager output. It is designed for coding agents and CI systems that need structured diagnostics without retaining multi-megabyte build logs in memory.

Compatibility target: upstream xcsift v1.3.1, commit 0a4b128. This port has its own version, 0.1.0, and is not an official xcsift release.

The parser/formatter behavior is compatible with that upstream target for the documented build, test, linker, coverage, JSON, TOON, and GitHub Actions surfaces. The original MIT license and upstream copyright attribution are retained in LICENSE.

Why Python

This implementation explores the same design under CPython 3.14 while keeping deployment simple:

  • CPython 3.14.7, uv and uv_build 0.12.2
  • no runtime packages: parsing, TOML, JSON, process execution, and CLI handling use the standard library
  • an installable xcsift-python command and an importable streaming parser API
  • pinned development tooling: Ruff 0.16.1, pytest 9.1.1, and mypy 2.3.0

Architecture

The implementation mirrors current xcsift's streaming, layered design:

stdin bytes (64 KiB chunks)
  -> bounded UTF-8 line reader (5,000 bytes per line)
  -> cheap marker routing + precompiled regular expressions
  -> stateful LineParser
       linker blocks | test lifecycle/crashes | #expect look-ahead | script look-back
  -> StreamingOutputParser
       aggregation | normalization | deduplication | status computation
  -> conditional wire model
       JSON | TOON | GitHub Actions annotations

Only the current partial line and three prior context lines are retained from the raw input. A line over 5,000 UTF-8 bytes is discarded before decoding. Structured findings necessarily remain in memory until final output is encoded.

Supported behavior

  • compiler, fatal, generic, and script-phase errors
  • compile, runtime, and SwiftUI warnings; warning detail gating; -Werror
  • undefined and duplicate symbols, missing frameworks/libraries, and architecture mismatches
  • XCTest, Swift Testing, and parallel test results, aggregate counts, duration, #expect comments, crashes, slow tests, and flaky tests
  • exact success, failed, and incomplete status rules from xcsift v1.3.1
  • build/test timing, per-target phases, dependencies, durations, slowest five targets, and app executables
  • xcbeautify/Tuist parsing behind --xcbeautify, with an opt-in hint when markers are detected
  • JSON omission rules and GitHub Actions annotations (automatically appended on Actions)
  • TOON with comma, tab, or pipe delimiters and safe key folding/depth controls
  • .xcsift.toml discovery, CLI-over-file merging, and --init
  • LLVM coverage JSON and Xcode xccov JSON, plus automatic .profraw/.xcresult conversion through xcrun

Three real upstream fixtures are checked as golden tests: the 2.7 MB Xcode build, Swift Testing output, and linker failure output.

Installation

Install the pinned Python and environment with uv:

uv python install 3.14.7
uv sync --locked
uv tool install . --python 3.14.7

For a source checkout without a tool installation:

uv run xcsift-python --version

Usage

Always merge stderr into stdout; Xcode and Swift compiler diagnostics are normally written to stderr.

xcodebuild build 2>&1 | xcsift-python
swift test 2>&1 | xcsift-python --warnings
xcodebuild test 2>&1 | xcsift-python -f toon --slow-threshold 1.0
swift build 2>&1 | xcsift-python -W -E

Useful options:

-w, --warnings                 include warning details
-W, --Werror                  convert warnings to errors
-q, --quiet                   suppress clean successful output
-E, --exit-on-failure         exit 1 unless status is success
-f, --format FORMAT           json, toon, or github-actions
-c, --coverage                discover and convert coverage
    --coverage-details        include per-file coverage
    --build-info              include target phases/timing/dependencies
-e, --executable              include generated .app bundles
    --xcbeautify              parse xcbeautify/Tuist markers
    --init                    create .xcsift.toml

An empty input stream exits with EX_USAGE (64). A markerless or truncated non-empty stream emits "status": "incomplete" and exits 0 by default, or 1 with -E.

Configuration

Configuration is loaded from the first available source:

  1. --config PATH
  2. .xcsift.toml in the working directory
  3. ~/.config/xcsift/config.toml

Generate a commented template with:

xcsift-python --init

CLI options take precedence. Boolean CLI switches enable settings; they do not negate a true file setting, matching upstream behavior.

Coverage

swift test --enable-code-coverage 2>&1 | xcsift-python --coverage
xcodebuild test -enableCodeCoverage YES 2>&1 | xcsift-python -c --coverage-details
xcodebuild test 2>&1 | xcsift-python -c --coverage-path /path/to/Test.xcresult

The conversion path invokes xcrun llvm-profdata, xcrun llvm-cov export, or xcrun xccov view --report --json as appropriate. Those Apple tools must be available for raw profile/result conversion; already-converted JSON is portable.

Compatibility differences

  • The command and Python distribution are named xcsift-python; they intentionally do not replace the upstream xcsift binary.
  • Upstream plugin installer subcommands (install-codex, install-cursor, and related commands) are outside this parser/formatter port's scope.
  • The runtime requires exactly CPython 3.14.7 for reproducible cross-language benchmarking.
  • Coverage conversion still depends on Apple xcrun tools, just as it does upstream; parsing converted JSON works on Linux.

Development and verification

uv sync --locked
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run pytest

The CI workflow runs the same gates on macOS and Linux.

Benchmark entry point

The stdlib-only benchmark helper reports raw samples and summary statistics as JSON:

uv run python benchmarks/run.py tests/fixtures/build.txt --warmup 5 --runs 30

Use the same fixture, warmup count, run count, machine power state, and output sink when comparing against xcsift, TypeScript, or Rust implementations. The helper is intentionally small so a cross-language benchmark harness can invoke the installed executable directly and produce the final charts separately.

Contributors

SunsetWan

Issues