A service that lets QA Systems Under Test (SUTs) upload logs, supportconfigs, and sosreports effortlessly via HTTP POST or restricted SSH. Uploads are safely extracted, given a short base62 ID, mapped to Bugzilla bugs, and browsed via a lightweight Web UI.
- Effortless Uploads: No authentication required. Secured by IP masking (
nginxallow-list and SSHfrom=limits) and file type validation. - Safe Extraction: Strict path traversal prevention, decompression bomb protection (resource limits and compression ratio caps), and deep permission hardening (executable bits stripped,
0640/0750permissions). - File Serving: Apache-style directory listings with secure X-Accel-Redirect streaming. Uploaded payload files are never served as executable HTML (stored XSS protection).
- Bugzilla Integration: Best-effort automatic assignment from
show_bug.cgiReferers, direct assignment by?bug=NNN, or via the web UI. - Auto-Cleanup: One-shot systemd container rules for purging untouched logs, old fixed-bug logs, and dealing with low-space critical conditions.
The service is fully containerized with Podman/Docker Compose, divided into three services: app (FastAPI backend), nginx (proxy/streaming), and sshd (SSH ingest). A separate systemd timer handles automated cleanup.
- Provide your secrets:
mkdir -p secrets echo "your_bugzilla_api_key" > secrets/bugzilla_api_key # Add your public keys to the restricted SSH pool cp deploy/sshd/authorized_keys.template secrets/authorized_keys # (Edit secrets/authorized_keys to insert the real keys)
- Configure settings:
cp config/config.example.toml config/config.toml # Edit config.toml to adjust base_url, size limits, proxy CIDRs, and cleanup days - Start the cluster:
podman compose -f deploy/compose.yaml up -d --build
- Setup Cleanup Timer (Host-level Systemd):
sudo cp deploy/systemd/scstorage-cleanup.service /etc/systemd/system/ sudo cp deploy/systemd/scstorage-cleanup.timer /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable --now scstorage-cleanup.timer
The project is packaged with uv and uses modern Python tooling (ruff, mypy, pytest). Minimum Python version is 3.11.
# Create venv and install dependencies
uv venv
source .venv/bin/activate
uv pip install -e ".[dev]"
# Testing
uv run pytest --cov=scstorage --cov-report=term-missing
# Linting and Type Checking
uv run ruff check src tests
uv run mypy src# Basic upload
curl -F "[email protected]" https://logs.example.org/upload
# Upload with a Bugzilla bug assignment
curl -F "[email protected]" "https://logs.example.org/upload?bug=1234567"
# Raw body streaming (requires X-Filename header)
curl --data-binary @nts.txz -H "X-Filename: nts.txz" https://logs.example.org/upload# Basic upload
ssh [email protected] < supportconfig.tar.xz
# Upload with a Bugzilla bug assignment
ssh [email protected] "bug=1234567" < sosreport.tar.xz
# Provide a filename hint for raw uploads
ssh [email protected] "bug=1234567 name=nts_myhost.txz" < nts_myhost.txzSee the /docs directory for complete architectural specifications.
01-architecture.md02-configuration.md04-upload-and-storage.md08-cleanup.md11-security.md