jongomezdev/ue-copilot-overlay

โ˜… 0Forks 0JavaScriptGitHub โ†—Compare

README

UE Copilot Overlay

An always-on-top desktop overlay that answers Unreal Engine questions and can look at a screenshot of your editor to tell you where you are and what to do next.

This is v1: overlay window + chat + screenshot-aware guidance (steps 1โ€“4 of the roadmap). Blueprint-graph verification against the actual project (step 6) is not built yet โ€” see "What's not here yet" below.

Setup

  1. Install dependencies (already done if you're reading this from the initial scaffold):
    npm install
    
  2. Copy .env.example to .env and add your Anthropic API key:
    cp .env.example .env
    
    Then edit .env:
    ANTHROPIC_API_KEY=sk-ant-...
    
  3. Start the app:
    npm start
    

A small chat panel appears in the top-right corner of your screen, always on top of other windows.

Using it

  • Type a question and press Enter (Shift+Enter for a newline).
  • Click the ๐Ÿ“ท button to attach a screenshot of your current screen (the overlay hides itself for a moment so it doesn't appear in its own screenshot). Then ask something like "am I in the right place to create a Blueprint?" or "what should I click next?".
  • Ctrl+Shift+U toggles the overlay's visibility from anywhere, even while Unreal Editor is focused.
  • Click โœ• in the title bar to quit.
  • Drag the title bar to reposition the window.

Architecture

  • src/main.js โ€” Electron main process. Creates the always-on-top overlay window, registers the global toggle shortcut, captures screenshots via desktopCapturer, and brokers chat requests over IPC.
  • src/preload.js โ€” Context-isolated bridge exposing a minimal window.ueCopilot API to the renderer (no direct Node/Electron access from the UI).
  • src/anthropicClient.js โ€” Builds the Claude request (system prompt tuned for Unreal Engine assistance, optional image content block) and streams the response using the Anthropic SDK.
  • src/renderer/ โ€” The chat UI (plain HTML/CSS/JS, no framework or bundler).

Model: claude-opus-5, streamed. Conversation history is kept client-side in the renderer (text only) and resent each turn; screenshots are only sent for the turn they're attached to, not stored in history, to keep context size down.

What's not here yet

  • Blueprint/graph verification (step 6 from the original plan): currently "verify this is wired correctly" relies purely on the vision model reading a screenshot, which is unreliable for dense node graphs. A more accurate version would also read the Unreal project's actual Blueprint data (e.g. via Unreal's Python API) rather than just pixels โ€” see the architecture discussion this project started from.
  • Packaging/distribution (electron-builder) โ€” for now this only runs via npm start from source.
  • Persisted chat history across restarts.

Contributors

jongomezdev

Issues