Dispatch is a distributed task runner written in Go. The project starts as a single-machine task runner and evolves into a production-inspired distributed system while exploring concurrency, networking, fault tolerance, and distributed systems concepts.

Stage 3 — TCP Distributed Execution (Implemented)

Implemented:

• Parallel task execution
• Configurable worker pool
• Task timeouts
• Task cancellation
• Real-time stdout/stderr streaming
• File-based task input
• In-process Coordinator scheduling
• In-process Worker execution
• TCP Coordinator server
• Standalone Worker process
• Worker registration
• Remote task execution
• Automated tests

Run commands directly:

go run ./cmd/dispatch run -parallel 2 "echo one" "echo two"

Run commands from a file:

go run ./cmd/dispatch run -parallel 2 -file tasks.txt

Example failure:

./dispatch run "echo Start" "false"

Run with remote workers:

go run ./cmd/dispatch coordinator -listen 127.0.0.1:9000
go run ./cmd/dispatch worker -coordinator 127.0.0.1:9000 -id 1
go run ./cmd/dispatch run -coordinator 127.0.0.1:9000 \
"echo hello" \
"echo world"

go run ./cmd/dispatch run -coordinator 127.0.0.1:9000 \
"echo task 1" \
"echo task 2" \
"echo task 3" \
"echo task 4" \
"echo task 5" \
"echo task 6" \
"echo task 7" \
"echo task 8" \
"echo task 9" \
"echo task 10"

A concurrent task runner capable of executing multiple tasks on a single machine.

Topics:

• Goroutines
• Channels
• Worker pools
• Context cancellation
• Process execution
• Testing

Separate scheduling from execution while still running on one machine in one process.

Topics:

• Coordinator-owned task scheduling
• Worker-owned command execution
• Backward-compatible CLI behavior
• Context cancellation across scheduler and workers
• Testable boundaries for future transport work

Deferred:

• TCP communication
• Heartbeats and health checks
• Retries and reassignment
• Streaming logs

Replace in-process workers with standalone workers connected over TCP.

Topics:

• TCP communication
• Worker registration
• Task serialization
• Remote command execution
• Streaming stdout/stderr
• Result summaries

Deferred:

• Heartbeats and failure detection
• Retries and reassignment
• Load balancing
• Persistent storage
• Authentication and TLS
• Metrics and observability

Dispatch follows a Test-Driven Development (TDD) workflow.

go test ./...
go test -race ./...
go vet ./...
go build ./cmd/dispatch