Comparative Artifact Performance Evaluation for UPLC Programs

A standardized benchmarking framework for measuring and comparing the on-chain performance of UPLC programs generated by different Cardano smart contract compilers.

UPLC-CAPE provides a friendly competition platform for Cardano compiler authors to:

• 📊 Benchmark their compiler's UPLC output against standardized scenarios
• 🔄 Compare performance across different compilers and versions
• 📈 Track optimization improvements over time
• 🏆 Showcase efficient implementations to the community

Currently, comparing UPLC output from different compilers (Aiken, Plinth, Plutarch, etc.) is ad-hoc and inconsistent. UPLC-CAPE standardizes this process by providing:

• Consistent benchmarks - Well-defined computational scenarios
• Standardized metrics - CPU units, memory units, script size, term size
• Reproducible results - Version-controlled scenarios and submission metadata
• Community-driven - Open collaboration between compiler teams

• Nix with flakes enabled
• Git

# Clone the repository
git clone https://github.com/your-org/UPLC-CAPE.git
cd UPLC-CAPE

# Enter the development environment
nix develop
# Or if using direnv (recommended)
direnv allow

# Verify installation
cape --help

# List available benchmarks
cape benchmark list

# View a specific benchmark
cape benchmark fibonacci

# Create a submission for your compiler
cape submission new fibonacci MyCompiler 1.0.0 myhandle

# Benchmark Management
cape benchmark list                    # List all benchmarks
cape benchmark list fibonacci          # Show specific benchmark details
cape benchmark new my-benchmark        # Create new benchmark

# Submission Management
cape submission list                   # List all submissions
cape submission list fibonacci         # Show submissions for specific benchmark
cape submission new fibonacci Aiken 1.0.8 myhandle  # Create new submission
cape submission validate                # Validate submission files
cape submission validate --all          # Validate all submissions

All commands support interactive prompting for missing arguments:

# These will prompt for missing information
cape benchmark new                     # Prompts for benchmark name
cape submission new                    # Prompts for all required fields
cape submission new fibonacci          # Prompts for compiler, version, handle

cape --help                           # Show main help
cape benchmark --help                 # Show benchmark command help
cape submission new --help            # Show specific subcommand help

cape benchmark list

cape submission new fibonacci MyCompiler 1.0.0 myhandle

This creates: submissions/fibonacci/MyCompiler_1.0.0_myhandle/

Replace the placeholder with your compiled UPLC:

# Edit: submissions/fibonacci/MyCompiler_1.0.0_myhandle/fibonacci.uplc
# Your fully-applied UPLC program that computes fibonacci(25)

Run your UPLC program and fill in the metrics:

# Edit: submissions/fibonacci/MyCompiler_1.0.0_myhandle/metrics.json
{
  "cpu_units": 185916,
  "memory_units": 592,
  "script_size_bytes": 1234,
  "term_size": 45
}

Before submitting, validate your files against the required schemas:

# Validate your submission directory
cd submissions/fibonacci/MyCompiler_1.0.0_myhandle/
cape submission validate

# Or validate from project root
cape submission validate submissions/fibonacci/MyCompiler_1.0.0_myhandle/

# Validate specific files if needed
cape submission validate --single metrics.json
cape submission validate --single metadata.json

# Edit: submissions/fibonacci/MyCompiler_1.0.0_myhandle/metadata.json
{
  "compiler_name": "MyCompiler",
  "compiler_version": "1.0.0",
  "contributor_handle": "myhandle",
  "submission_date": "2025-01-15",
  "optimization_level": "O2"
}

Edit the README.md with implementation notes, optimization strategies, and any relevant details.

UPLC-CAPE/
├── scenarios/              # Benchmark definitions
│   ├── TEMPLATE/           # Template for new benchmarks
│   └── fibonacci.md        # Fibonacci benchmark specification
├── submissions/            # Compiler submissions
│   ├── TEMPLATE/           # Template for new submissions
│   └── fibonacci/          # Submissions for fibonacci benchmark
│       └── Aiken_1.0.8_contributor/  # Example submission
├── scripts/               # CAPE management tools
│   ├── cape.sh           # Main CLI tool
│   └── cape-subcommands/ # Command implementations
├── doc/                  # Documentation
│   ├── domain-model.md   # Framework architecture
│   └── adr/             # Architecture Decision Records
└── templates/           # File templates

We welcome contributions from compiler authors, benchmark designers, and performance researchers!

# Create benchmark specification
cape benchmark new my-new-benchmark

# Edit the generated file
nano scenarios/my-new-benchmark.md

# Submit a pull request

# Create submission for existing benchmark
cape submission new existing-benchmark MyCompiler 1.0.0 myhandle

# Fill in your UPLC program and metrics
# Submit a pull request

• 📖 Contributing Guide - Detailed contribution process
• 🏛️ Domain Model - Framework architecture
• 📋 Templates - File templates and examples

When contributing to this project:

1. Enter the development environment: nix develop
2. Check existing ADRs to understand current architectural decisions
3. Create an ADR for any significant architectural changes: adr new "Your Decision"
4. Follow the established patterns and guidelines documented in the ADRs

This project uses Nix for reproducible development environments:

# Enter the development shell
nix develop

# Or if using direnv (recommended)
direnv allow

# Available tools:
cape <command>       # CAPE management tool
adr new "Title"      # Create Architecture Decision Record
mmdc -i file.mmd     # Generate diagrams

# Test the CLI tool
cape benchmark list
cape submission list

# Run tests (if available)
just test

This project uses Architecture Decision Records to document important architectural and design decisions. ADRs are managed using Log4brains.

The development environment includes convenient short commands for managing ADRs:

# Create a new ADR
adr new "My Decision Title"

# Preview ADRs in your browser
adr preview

# Build static documentation site
adr build

# Show help and available commands
adr help

For even faster usage, single letter aliases are available:

adr n "Quick ADR"    # new
adr p               # preview
adr b               # build
adr h               # help

1. Before making significant decisions: Check existing ADRs in doc/adr/
2. When making a new decision: Create an ADR using adr new "Decision Title"
3. Follow the template: Fill out Context, Decision, and Consequences sections
4. Review process: Use adr preview to review your ADR in the browser
5. Commit: Include the ADR in your pull request with the related changes

• ADR 0001: Record architecture decisions
• ADR 0002: Use Log4brains for ADR management

• 🏛️ Domain Model - Framework architecture and entities
• 📋 Contributing Guide - How to contribute
• 🎯 Project Epic - Detailed project description and goals
• 📖 Benchmark Specifications - Individual benchmark details

• 💬 Discussions - Use GitHub Discussions for questions
• 🐛 Issues - Report bugs via GitHub Issues
• 📧 Contact - Reach out to the Plutus Core team

We invite all Cardano compiler teams to participate:

• Aiken - Modern smart contract language
• Plinth - Reference implementation
• Plutarch - High-level eDSL
• Your Compiler - Join the friendly competition!

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

• Plutus Core Team - Framework infrastructure and reference implementations
• Compiler Authors - For participating in the friendly competition
• Community Contributors - For benchmark scenarios and improvements

• Log4brains Documentation - ADR tool documentation
• Architecture Decision Records - ADR concept by Michael Nygard