The most comprehensive open-source AR glasses SDK knowledge base.
27+ repositories · 510 API paths · 12 domains · Every major consumer AR platform
A deeply-researched, cross-referenced documentation repository for building software that targets AR glasses — XREAL, RayNeo, Rokid, Brilliant Labs Frame, Meta, Vuzix, Everysight, Snap Spectacles, and more. Every claim is backed by actual source code analysis of 27+ GitHub repos, 10+ documentation sites (including RayNeo Feishu wikis, Khronos OpenXR spec, Monado headers), and device SDKs.
| Platform | URL | What it is |
|---|---|---|
| Fern Docs | vincent.docs.buildwithfern.com | Polished API reference + guides. Auto-deploys on every push. |
| Swagger UI | vincentwi.github.io/ar-glasses-master-sdk | Interactive API explorer. 510 paths, dark theme, deep-linking. Auto-deploys via GitHub Pages. |
| GitHub Gist | Full API Reference | 108KB markdown. Every function with params, types, returns, examples. |
| OpenAPI Spec | openapi.yaml | 309KB, 510 paths, 42 schemas. Machine-readable. |
| Workflow | Purpose | Status |
|---|---|---|
docs.yml |
Deploy Swagger UI to GitHub Pages | Passing |
fern-docs.yml |
Publish to vincent.docs.buildwithfern.com | Passing |
fern.yml |
Validate OpenAPI spec with Fern | Passing |
Open Fern Docs for a polished view, or Swagger UI for interactive exploration.
from xg_glass_sdk import GlassesClient
client = GlassesClient()
client.connect()
client.display("Hello from AR Glasses SDK")
# Read IMU data
client.on_imu_data(lambda data: print(f"Quaternion: {data.quaternion}"))// Initialize head tracking
HeadTrackedPoseParams.AwakeDriver();
// Get pose updates
void OnPostUpdate(Pose pose) {
transform.rotation = pose.rotation; // pitch, yaw, roll
}
// Reset heading
HeadTrackedPoseParams.ResetRotation();FusionAhrs ahrs;
FusionAhrsInitialise(&ahrs);
// Update with sensor data every cycle
FusionAhrsUpdate(&ahrs, gyroscope, accelerometer, magnetometer, deltaTime);
// Get orientation
FusionQuaternion q = FusionAhrsGetQuaternion(&ahrs);
FusionEuler euler = FusionQuaternionToEuler(q);
printf("Roll: %.1f, Pitch: %.1f, Yaw: %.1f\n", euler.roll, euler.pitch, euler.yaw);npm install -g fern-api
cd fern && fern generate --docs| Device | Vendor | Transport | DoF | Camera | Display | Key SDK |
|---|---|---|---|---|---|---|
| XREAL Air 2 Ultra | XREAL | USB HID (VID 0x3318) |
6DoF | Dual stereo | 1920x1080 SBS, 52° | Unity XR Plugin |
| XREAL Air/Air 2 | XREAL | USB HID (VID 0x3318) |
3DoF | — | 1920x1080 SBS, 46° | XRLinuxDriver |
| RayNeo X3 Pro | RayNeo | USB HID (VID 0x1bbb) |
3DoF (+SLAM) | 16MP | Micro-LED, 43° | Android ARDK + Unity |
| RayNeo X2 | RayNeo | USB HID | 6DoF | VGA SLAM | 1920x1080, 33° | OpenXR Unity ARDK |
| VITURE One/Pro | VITURE | USB HID (VID 0x35ca) |
3DoF | — | 1920x1080 SBS, 43° | XRLinuxDriver |
| Rokid Air/Max | Rokid | BLE→Wi-Fi | 3DoF | — | 3840x1200 3D, 45° | xg-glass-sdk |
| Brilliant Labs Frame | Brilliant | BLE GATT | — | Low-res | 640x400 OLED | nRF52840 + TFLite Micro |
| Vuzix Blade 2 | Vuzix | Android | — | 8MP | DLP Waveguide | HUD SDK (Java) |
| Everysight Maverick | Everysight | BLE | — | Yes | Waveguide | iOS/Android binary SDK |
| Snap Spectacles | Snap | Wi-Fi | 6DoF | Yes | Waveguide ~720p | Snap OS 2.0 / WebXR |
| MentraOS Glasses | Mentra | BLE→WebSocket | — | Yes | — | React Native + cloud |
| Domain | Paths | Description | Key Repos |
|---|---|---|---|
| device | 137 | Connection lifecycle, drivers, firmware, config, CLI | XRLinuxDriver, xg-glass-sdk, MentraOS |
| imu | 86 | AHRS, quaternion math, bias estimation, head tracking | Fusion, headset-utils, real_utilities |
| display | 44 | Text, bitmap, HUD menus, binocular stereo, curved | Frame, Vuzix, RayDesk, TAPLINKX3 |
| stardust | 44 | 3D spatial nodes, drawables, fields, panels, Wayland | StardustXR server/core/flatland/protostar |
| ml | 36 | TFLite Micro, SAM3 segmentation, Groq LLM | frame-codebase, TAPLINKX3, MentraOS |
| spatial | 36 | OpenXR sessions, Monado runtime, SLAM, plane detection | Monado, XREAL SDK, RayNeo ARDK |
| geo | 31 | Overpass OSM queries, Gemini Maps grounding, health data | overpass-turbo, Google AI, open-wearables |
| audio | 26 | TTS, mic capture, BeatSync, transcription | MentraOS, TAPLINKX3, CoreDevices |
| ble | 19 | BLE GATT (NUS, DAT, CXR-M), Wi-Fi P2P, WebSocket | MentraOS, Everysight, CoreDevices |
| gesture | 16 | Temple touch, gaze ray, hand tracking, ring input | RayNeo, decky-XRGaming, Monado |
| camera | 11 | Photo capture, SLAM cameras, frame processing | RayNeo ShareCamera, CoreDevices |
| gps | 3 | Phone GPS IPC, location streaming | RayNeo GPSIPCHelper |
| Brand | VID | Product IDs (sample) |
|---|---|---|
| XREAL | 0x3318 |
Air: 0x0424, Air 2: 0x0428, Air 2 Pro: 0x0432, Air 2 Ultra: 0x0436 (10 PIDs total) |
| VITURE | 0x35ca |
One: 0x1011, One Lite: 0x1013, Pro: 0x1017 (14 PIDs total) |
| Rokid | various | 7 PIDs across Air/Max models |
| RayNeo | 0x1bbb |
X2: 0xaf50 |
Full registry: reference/vid-pid-registry.md
| Document | Size | Description |
|---|---|---|
| OpenAPI Spec | 309KB | 510 paths, 42 schemas, 12 domain tags |
| Master SDK Reference | 66KB | Unified API surface by domain |
| Full API Reference | 109KB | Every function with params, types, returns, examples |
| API Invocation Reference | 60KB | Part 1: core SDKs + drivers |
| API Invocation Reference Pt.2 | 63KB | Part 2: utilities + companion apps |
| Capabilities Matrix | 4KB | Hardware × software feature grid |
| Cross-Functionality | 99KB | 45 cross-repo integration opportunities |
| JTBD Opportunities | 75KB | 15 Jobs-to-be-Done product ideas |
| Link Registry | 5KB | Every crawled URL with status |
| Card | Content |
|---|---|
| VID/PID Registry | USB Vendor/Product IDs for all devices |
| Protocol Cheatsheet | BLE GATT, USB HID, IPC, WebSocket, FlatBuffers |
| Sensor Fusion Guide | Fusion AHRS, complementary filter, quaternion math |
Click to expand full list
| # | Topic | Repos Covered | Size |
|---|---|---|---|
| 01 | Core SDKs | xg-glass-sdk, MentraOS, Vuzix Blade 2 | 26KB |
| 02 | Everysight + CoreDevices | Everysight Maverick SDK, CoreDevices mobileapp | 44KB |
| 03 | XR Tools | open-wearables, xreal-webxr, decky-XRGaming | 42KB |
| 04 | XR Drivers | PhoenixHeadTracker, XRLinuxDriver, RayDesk, TAPLINKX3 | 36KB |
| 05 | StardustXR | server, core, flatland, protostar (Rust) | 37KB |
| 06 | IMU & Sensors | Fusion AHRS, headset-utils, real_utilities, imu-inspector | 42KB |
| 07 | RayNeo + Frame | RayNeo Unity SDK, TFLite on Frame, overpass-turbo | 27KB |
| 08 | Monado + Specs | Monado OpenXR (219+ xrt_* types), platform specs | 44KB |
| 09 | RayNeo Docs | Feishu wiki docs (3DOF/6DOF/GPS/Ring/Gaze) | 8KB |
| 10 | Geo + Maps | Overpass API, Gemini Maps grounding | 23KB |
| 11 | Platform Docs | XREAL SDK, OpenXR 1.1, SnapOS 2.0, Qualcomm | 27KB |
| 12–18 | Additional analyses | Cross-verification, utilities, samples | ~200KB |
Click to expand full list of analyzed repos
Core SDKs:
- hkust-spark/xg-glass-sdk — Cross-device glasses SDK (Python)
- Mentra-Community/MentraOS — Full OS for smart glasses
- Vuzix/Blade_2_Template_App — Vuzix Blade 2 template
- everysight-maverick/sdk — Everysight Maverick SDK
- coredevices/mobileapp — CoreDevices/Pebble companion app
XR Drivers & Tools:
- wheaney/XRLinuxDriver — Linux XR driver (102 C files, 4 brands)
- wheaney/decky-XRGaming — Steam Deck XR plugin
- iVideoGameBoss/PhoenixHeadTracker — XREAL head tracker
- jakedowns/xreal-webxr — WebHID protocol for XREAL
- Quad-Labs/RayDesk — Moonlight streaming to RayNeo
- informalTechCode/TAPLINKX3 — AR browser for RayNeo
StardustXR (Rust 3D compositor):
IMU & Sensor Fusion:
- xioTechnologies/Fusion — Madgwick AHRS (C)
- 3rl-io/headset-utils — Rust XR abstraction
- edwatt/real_utilities — XREAL Air protocol (C++)
- abls/imu-inspector — IMU data viewer
- 3rl-io/spidgets-3dof — 3DOF web bridge
RayNeo:
- MaxManausa/Metaverse-Max-RayNeo-X3-Pro-Set-Up
- MaxManausa/Rayneo_OpenXR_ARDK_Project
- MaxManausa/RayNeoX3Pro-MITSample
- RayNeo Feishu docs (8 wiki pages), GitBook, open.rayneo.com
Standards & Runtimes:
- Monado — Open-source OpenXR runtime
- OpenXR 1.1 Spec — Khronos standard
- the-momentum/open-wearables — MCP health data server
Other:
- pkrumpl/frame-codebase — TFLite Micro on Brilliant Frame
- tyrasd/overpass-turbo — OSM query tool
- Google Gemini Maps grounding API, Scaniverse/Niantic VPS, Qualcomm AR1 docs
ar-glasses-master-sdk/
├── README.md ← You are here
├── ARCHITECTURE.md ← System design & methodology
├── openapi.yaml ← OpenAPI 3.1 (309KB, 510 paths, 42 schemas)
├── .github/workflows/
│ ├── docs.yml ← Swagger UI → GitHub Pages (auto)
│ ├── fern-docs.yml ← Fern → vincent.docs.buildwithfern.com (auto)
│ └── fern.yml ← Fern check (validation)
├── fern/
│ ├── fern.config.json ← org: "vincent"
│ ├── docs.yml ← Fern docs navigation + pages
│ ├── generators.yml ← OpenAPI → SDK generation
│ ├── openapi/openapi.yaml ← Spec copy for Fern
│ ├── pages/*.mdx ← 6 documentation pages
│ └── assets/ ← Logos
├── docs/
│ ├── swagger/index.html ← Dark-themed Swagger UI
│ ├── FULL_API_REFERENCE.md ← 109KB, every function documented
│ ├── MASTER_SDK_REFERENCE.md ← 66KB, unified by domain
│ ├── API_INVOCATION_REFERENCE.md ← 60KB, invocation guide pt.1
│ ├── API_INVOCATION_REFERENCE_PART2.md ← 63KB, invocation guide pt.2
│ ├── CROSS_FUNCTIONALITY.md ← 99KB, 45 opportunities
│ ├── JTBD_OPPORTUNITIES.md ← 75KB, 15 JTBD ideas
│ ├── CAPABILITIES_MATRIX.md ← Device comparison
│ ├── LINK_REGISTRY.md ← Crawled URLs + status
│ ├── GREP_REPASS_COMPLETE.md ← 29KB grep extraction
│ └── deep-reads/ ← 18 numbered repo analyses (~400KB)
├── reference/
│ ├── vid-pid-registry.md ← USB VID/PID quick-lookup
│ ├── protocol-cheatsheet.md ← BLE/USB/IPC/WS protocols
│ └── sensor-fusion-guide.md ← IMU fusion libraries
└── src/core/ ← Kotlin SDK core
├── GlassesClient.kt
├── GlassesCallback.kt
├── Controllers.kt
└── types.kt
| Finding | Detail |
|---|---|
| Best cross-device abstraction | xg-glass-sdk — Python CLI, 6 device backends (Rokid, Meta, Frame, RayNeo, Omi, Sim) |
| Linux standard driver | XRLinuxDriver — 31 USB AR glasses models, 4 brands, C with libusb hotplug |
| Most ambitious compositor | StardustXR — Rust + Bevy + OpenXR, FlatBuffers over Unix sockets |
| Most complete glasses OS | MentraOS — React Native + cloud + BLE, full app ecosystem |
| Critical sensor fusion | Fusion (C) — Madgwick revised AHRS, acceleration/magnetic rejection, gyro bias |
| 6 overlap clusters | IMU fusion (4 impl), device abstraction (3), display rendering (4), geo querying (3) |
| 10 capability gaps | Hand tracking, persistent SLAM, eye tracking, depth sensing, multi-user, offline maps |
- Clone the target repo you want to document
- Read actual source files (not just README)
- Document every public API with types and signatures
- Add a numbered deep-read in
docs/deep-reads/ - Update
openapi.yamlwith new paths - Push — Fern docs + Swagger UI auto-deploy
Documentation and analysis content © 2024–2026 vincentwi. Original source code belongs to respective repository owners. See docs/LINK_REGISTRY.md for all analyzed repositories and their licenses.
