A production-ready AI-powered system that automatically classifies and triages GitHub issues using OpenAI's language models and the Model Context Protocol (MCP), orchestrated through Smithery's platform.

• Intelligent Issue Analysis: Uses OpenAI GPT-4o to classify issues with 85-95% confidence
• Multi-Category Support: Automatically categorizes issues as bug, feature-request, documentation, question, or enhancement
• Confidence Scoring: Only applies labels when classification confidence exceeds configurable threshold
• Contextual Reasoning: Provides clear explanations for classification decisions

• Smart Labeling: Automatically applies appropriate labels based on AI classification
• Helpful Comments: Posts structured triage comments explaining the classification reasoning
• Severity Assessment: Evaluates bug severity levels (critical, high, medium, low)
• Label Management: Handles existing labels intelligently to avoid duplicates

• Structured Logging: Comprehensive logging with correlation IDs for request tracing
• Error Handling: Custom error classes with proper context and retry logic
• Input Validation: Zod-based schema validation for all configurations and inputs
• Health Monitoring: Built-in health checks for all integrated services

• Smithery Orchestration: Seamless integration with GitHub and OpenAI through MCP servers
• Real-time Processing: Fast classification and labeling (typically under 5 seconds)
• Tool-based Interface: Four core MCP tools for different operational needs
• Zero Infrastructure: Leverages Smithery's managed platform for deployment

MCP Client → Smithery Platform → GitHub Auto-Triage Agent
                                         ↓
                               OpenAI MCP ← → GitHub MCP

The system uses the Model Context Protocol to orchestrate between:

• GitHub MCP Server: Handles repository operations (issue creation, labeling, commenting)
• OpenAI MCP Server: Manages AI classification requests
• Smithery Platform: Provides unified orchestration and management

• Node.js 18+
• GitHub Personal Access Token with repo scope
• OpenAI API key
• Smithery account (sign up free)

1. Clone and install dependencies:

   git clone <repository-url>
   cd github-issue-triage-agent
   npm install

2. Configure environment variables:

   cp .env.example .env

   Edit .env with your credentials:

   GITHUB_TOKEN=ghp_your_github_token
   GITHUB_WEBHOOK_SECRET=your_webhook_secret
   OPENAI_API_KEY=sk-proj-your_openai_key
   GITHUB_REPO_OWNER=your_username
   GITHUB_REPO_NAME=your_repository

3. Start the development server:

   npm run dev

4. Access the Smithery playground at the provided URL to test the MCP tools.

Ensure your target GitHub repository has these labels:

• bug - Something isn't working
• feature-request - New feature or request
• documentation - Improvements or additions to documentation
• question - Further information is requested
• enhancement - New feature or request

The agent exposes four MCP tools for different operational needs:

Manually classify an existing issue without modifying GitHub.

Parameters:

• issueNumber (number): GitHub issue number
• title (string): Issue title
• body (string): Issue description
• author (string): Issue author username

Use case: Test classification logic or manually triage specific issues.

Create a new GitHub issue and immediately classify it with full automation.

Parameters:

• title (string): Issue title
• body (string): Issue description
• issueType (optional): Expected classification for validation

Use case: End-to-end automation - creates issue, classifies, labels, and comments.

Verify connectivity and health of all integrated services.

Returns: Status of OpenAI and GitHub API connections.

Use case: System monitoring and troubleshooting.

Retrieve current agent configuration and settings.

Returns: Current triage settings, repository info, and server configuration.

Use case: Configuration verification and debugging.

Default supported labels (configurable via TRIAGE_LABELS):

• bug - Issues reporting problems or errors
• feature-request - Requests for new functionality
• documentation - Documentation improvements or clarifications
• question - User questions or help requests
• enhancement - Improvements to existing features

// Through MCP client
const result = await client.callTool("triage_issue", {
  issueNumber: 42,
  title: "App crashes on login",
  body: "When I click login, the app freezes and shows a JavaScript error",
  author: "user123"
});

// Creates issue #43 and immediately triages it
const result = await client.callTool("create_and_triage_issue", {
  title: "Add dark mode support", 
  body: "Users have requested a dark theme option for better night-time usage",
  issueType: "feature" // optional validation
});

const health = await client.callTool("health_check", {});
// Returns: { overall: true, services: { classifier: true, github: true } }

1. Push code to GitHub
2. Connect repository in Smithery dashboard
3. Configure environment variables in deployment settings
4. Deploy from Smithery interface

For production deployments:

• Set NODE_ENV=production
• Use LOG_LEVEL=info or warn
• Ensure all required environment variables are configured
• Verify GitHub token has appropriate repository permissions

All operations include structured logging with:

• Correlation IDs: Track requests across services
• Component Tags: Identify log sources
• Performance Metrics: Duration and token usage tracking
• Error Context: Detailed error information with stack traces

Built-in health checks verify:

• OpenAI API connectivity and authentication
• GitHub API access and repository permissions
• Configuration validity
• Service response times

The system tracks:

• Classification accuracy and confidence scores
• Processing times for each operation
• API usage and rate limiting
• Error rates and failure patterns

Classification Confidence Too Low

• Adjust CONFIDENCE_THRESHOLD in configuration
• Review issue content quality and completeness
• Check OpenAI API model performance

GitHub API Errors

• Verify token has repo scope
• Check repository permissions
• Ensure target repository exists and is accessible

OpenAI API Issues

• Validate API key format and permissions
• Monitor rate limits and usage quotas
• Check model availability

Enable debug logging with LOG_LEVEL=debug to see:

• Detailed API request/response data
• Classification reasoning and confidence scores
• Internal state transitions
• Performance timing information

• 5,000 requests/hour with Personal Access Token
• Automatic retry with exponential backoff on rate limit errors
• Rate limit status monitoring and logging

• Varies by subscription tier
• Automatic error handling for rate limit responses
• Token usage tracking and optimization

• Store all credentials in environment variables
• Never commit secrets to version control
• Rotate keys regularly
• Use least-privilege access principles

• All inputs validated with Zod schemas
• HTML sanitization for issue content
• Protection against injection attacks
• Webhook signature verification

1. Fork the repository
2. Create a feature branch
3. Install dependencies: npm install
4. Start development: npm run dev
5. Run tests: npm test
6. Submit pull request

• TypeScript strict mode enabled
• Comprehensive error handling required
• Structured logging for all operations
• Unit tests for business logic
• Production-grade code quality

MIT License - see LICENSE file for details.

For issues and support:

• GitHub Issues: Create issues in this repository
• Smithery Support: Contact Smithery
• Documentation: Smithery Docs

Built with ❤️ using Smithery and the Model Context Protocol.