A submission to the AgentX AgentBeats Competition building on the User Environment Simulator (UES) project.

This project implements:

• Green Agent (Evaluator): Orchestrates assessments of AI personal assistants using UES as the testing environment. Manages scenarios, generates character responses, and evaluates agent performance.
• Purple Agent Template: A lightweight template to help participants build A2A-compliant personal assistant agents.

1. Green agent initializes a UES environment with a predefined scenario
2. Purple agent receives task context via the A2A protocol
3. Purple agent interacts with UES (email, calendar, SMS, chat) to complete tasks
4. Green agent generates character responses and advances simulation time
5. Assessment results are scored across multiple dimensions (accuracy, efficiency, safety, etc.)

# Install dependencies
uv sync

# Run tests
uv run pytest

# Start Green agent (example)
uv run python -m src.green --host 0.0.0.0 --port 8000

# Start Purple agent (example)
uv run python -m src.purple --host 0.0.0.0 --port 8001

src/
├── common/
│   ├── a2a/          # A2A protocol helpers (complete)
│   └── agentbeats/   # AgentBeats-specific utilities (complete)
├── green/            # Green agent implementation
│   ├── scenarios/    # Scenario schema and loader (complete)
│   ├── evaluation/   # Criteria judge (complete)
│   ├── response/     # Character response generation (complete)
│   ├── core/         # LLM config, action log, message collector (complete)
│   └── assessment/   # Assessment orchestration (in progress)
└── purple/           # Purple agent template
scenarios/
└── email_triage_basic/  # First scenario (complete — config, state, evaluators)
tests/                # Test suite (1,542 tests)
docs/                 # Documentation

Design Documents:

• Competition Description
• Assessment Flow
• Implementation Plan
• Response Generation Design

Module READMEs:

• Green Agent Module
• Scenario Management
• Response Prompts
• A2A Helpers
• AgentBeats Helpers

• UES: Installed as a local editable dependency from ../ues/. Ensure the UES repository is cloned adjacent to this project.
• pytest-dotenv: Automatically loads .env during test runs.

Copy .env.example to .env and configure:

# Required for OpenAI-based LLMs
OPENAI_API_KEY=sk-your-key-here

# Optional for other providers
# ANTHROPIC_API_KEY=sk-ant-your-key-here
# GOOGLE_API_KEY=your-key-here

• Implement src/common/agentbeats/messages.py - Assessment message types
• Implement src/common/agentbeats/results.py - Assessment results models
• Implement src/common/agentbeats/updates.py - Task update helpers
• Implement src/common/agentbeats/config.py - Configuration models
• Write tests for serialization/validation (237 tests passing)

• Implement scenario schema and loader (src/green/scenarios/)
• Implement LLM configuration factory (src/green/llm_config.py)
• Implement action log builder (src/green/action_log.py)
• Implement new message collector (src/green/message_collector.py)
• Implement response generator (src/green/response_generator.py)
• Implement response data models (src/green/response_models.py)
• Implement LLM prompt templates (src/green/prompts/)
• Integrate LangChain for LLM-based response generation
• Write integration tests with Ollama and OpenAI
• Implement assessment orchestrator (src/green/agent.py)
• Implement criteria judge (src/green/evaluation/)
• Implement GreenAgent with full turn loop, UES management, Purple communication
• Full test suite: 1,424 tests passing (0 failures, 0 skipped)
• Email Triage Basic scenario: evaluators complete (ground_truth.py, _eval_helpers.py, evaluators.py — 73 scenario tests)
• Full test suite: 1,542 tests passing (0 failures, 0 skipped)

• Implement base agent class (src/purple/base_agent.py)
• Implement assessment handler (src/purple/handler.py)
• Implement Purple executor (src/purple/executor.py)
• Create UES client wrapper (src/purple/ues_wrapper.py)
• Create simple example agent

• Dockerize Green agent (Dockerfile.green)
• Dockerize Purple agent (Dockerfile.purple)
• Write scenario documentation (docs/SCENARIOS.md)
• Write evaluation criteria docs (docs/EVALUATION_CRITERIA.md)
• Write Purple agent guide (docs/PURPLE_AGENT_GUIDE.md)
• Create demo video
• Register on AgentBeats

MIT