rabble/divine-dvm

Nostr video event indexer and NIP-90 Data Vending Machine

★ 1Forks 0RustGitHub ↗Compare

README

divine-dvm

A high-performance Nostr Data Vending Machine (DVM) for indexing and querying kind 32222 video events.

Architecture

  • Indexer: Ingests kind 32222 events from relay, stores in ClickHouse
  • DVM Service: Provides NIP-90 WebSocket and GraphQL HTTP query interfaces
  • Storage: ClickHouse for events, Redis for caching
  • Performance: <200ms p95 latency at 50+ QPS with caching

Quick Start

Prerequisites

  • ClickHouse instance at clickhouse.divine.video
  • Nostr relay at wss://relay3.openvine.co
  • Redis (via Docker or managed)

Setup

  1. Copy environment configuration:
cp .env.example .env
# Edit .env with your settings
  1. Initialize ClickHouse schema:
clickhouse-client --host clickhouse.divine.video < schema.sql
  1. Run with Docker Compose:
docker-compose up

Services

Indexer

Backfill historical events:

cargo run -p indexer -- backfill --since 0

Run live subscription:

cargo run -p indexer -- live

GraphQL API (Port 8080)

Query example:

query {
  searchVideos(
    minLoops: 100,
    hashtags: ["music"],
    sort: ["likes:desc"],
    limit: 50
  ) {
    items {
      id
      title
      loops
      likes
      duration
      hashtags
      hls
    }
    cursor
  }
}

NIP-90 WebSocket (Port 8090)

Job request (kind 5302):

{
  "kind": 5302,
  "tags": [
    ["param", "minLoops", "100"],
    ["param", "hashtags", "music,funny"],
    ["param", "sort", "likes:desc"],
    ["param", "limit", "50"]
  ]
}

Result (kind 6302): Full Nostr events in content field

Development

Running Tests

# All tests
cargo test --workspace

# Specific crate
cargo test -p indexer
cargo test -p dvm

Building

cargo build --release

Configuration

See .env.example for all configuration options:

  • RELAY_URL: Source relay for events
  • CLICKHOUSE_URL: ClickHouse connection
  • REDIS_URL: Redis for caching
  • RATE_LIMIT_PER_MIN: Request rate limiting
  • CACHE_TTL_SEC: Cache duration

Data Model

ClickHouse Tables

  • events_raw: Raw Nostr events
  • video_32222: Typed wide table with parsed fields
  • video_32222_current: Current state (NIP-33 replaceable)
  • events_deleted: Tombstoned events (kind 5)

Supported Filters

  • Time range: since, until
  • Metrics: minLoops, minLikes
  • Duration: durationMin, durationMax
  • Hashtags: Array intersection
  • Sorting: Multi-field with direction
  • Pagination: Cursor-based continuation

License

MIT

Contributors

rabble

Issues