AI-native test case generation framework using dependency graphs.

TestWeaver is a modern rework of depend-test-framework, redesigned to be AI-native and agent-friendly. Define test operations with decorators in Python (or declaratively in YAML), and TestWeaver builds a dependency graph to automatically discover all valid test paths.

Core

• Decorator-based definitions — @provides, @requires, @clears, @excludes, @graft, @cut
• Hierarchical state model — states are trees (vm.config.tpm)
• Automatic case generation — dependency graph finds all valid paths to each target
• Runtime data flow — operations pass dynamic data (UUIDs, IPs) through env node values
• Operation verification — @verify_for / verify YAML field
• Graph modifiers — EdgeGuard, TransientHook, TransitionObserver for runtime decisions

Parameterization

• Parameter graph (param_choices) — parameters affect which operations are available
• Parameter matrix (param_matrix) — Cartesian product with constraint filtering
• Multi-instance namespaces — TPM:tpm0, TPM:tpm1 with wildcard queries and generation strategies

Execution

• Parallel execution — --workers N for concurrent test cases
• Retry / flaky handling — --retries with automatic flaky detection
• Per-step timeout — @timeout(seconds) per operation
• Lifecycle hooks — @suite_setup, @suite_teardown, @case_setup, @case_teardown
• Dry-run mode — --dry-run to preview without executing

Output & Analysis

• Structured reporting — JSON, JUnit XML, TAP, HTML
• Graph visualization — DOT (Graphviz) and Mermaid export
• Case filtering — -k, --target, --has-step, --fault-only
• Case prioritization — --sort shortest|longest|target|total|fault-first|random
• Progress reporting — live progress bar with --progress
• Logging — --verbose, --debug, --log-file

Assertions

• Fluent assertion API — assert_that(value).equals(expected).greater_than(0)
• Chained assertions with rich expected-vs-actual diffs
• assert_raises context manager for exception testing

pip install -e ".[dev]"

# my_ops.py
from testweaver import action, check, cleanup, provides, requires, clears, verify_for

@action
@provides('file.exists')
def create_file(params, env):
    import subprocess
    subprocess.run('echo "hello world" > /tmp/test.txt', shell=True, check=True)

@verify_for('create_file')
def check_content(params, env):
    import subprocess
    subprocess.run('grep -q "hello world" /tmp/test.txt', shell=True, check=True)

@check
@requires('file.exists')
def check_file_exists(params, env):
    import subprocess
    subprocess.run('test -f /tmp/test.txt', shell=True, check=True)

@cleanup
@requires('file.exists')
@clears('file.exists')
def remove_file(params, env):
    import subprocess
    subprocess.run('rm -f /tmp/test.txt', shell=True, check=True)

testweaver run my_ops.py --format text

operations:
  - name: create_file
    type: action
    provides: [file.exists]
    run: echo "hello world" > /tmp/hello.txt
    verify: grep -q "hello world" /tmp/hello.txt

  - name: check_file_exists
    type: check
    requires: [file.exists]
    run: test -f /tmp/hello.txt

  - name: remove_file
    type: cleanup
    requires: [file.exists]
    clears: [file.exists]
    run: rm -f /tmp/hello.txt

suite:
  name: "Hello World"
  targets: [check_file_exists]
  cleanup: true

testweaver validate my_test.yaml          # Check for errors
testweaver generate my_test.yaml          # Generate test cases
testweaver run my_test.yaml               # Run tests (JSON output)
testweaver run my_test.yaml --format text # Human-readable output
testweaver graph my_test.yaml --format dot | dot -Tpng -o graph.png  # Visualize

The examples/virt/ directory contains mock implementations of libvirt/QEMU test operations (VM lifecycle, vTPM, save/restore, disk management, and more). See Virtualization Examples.

testweaver generate examples/virt/vtpm_test.yaml --format text
testweaver run examples/virt/backing_chain_test.yaml --format text

TestWeaver is designed to be used by AI agents:

1. Get the schema: testweaver schema --type definition returns the JSON Schema
2. Generate a definition: Write a YAML file matching the schema
3. Validate: testweaver validate <file> checks for errors
4. Run: testweaver run <file> --output results.json returns structured results
5. Debug: testweaver analyze results.json -d <file> provides failure details

All commands output JSON by default.

MIT