Media downloader plugin for yoink-core.

Supports video, audio, image, and playlist downloads via yt-dlp and gallery-dl. Per-user quality/codec/proxy settings, NSFW detection, file caching, rate limiting, inline YouTube search, forum topic routing, cookie management, and clipping.

Included in yoink-core as a git submodule at plugins/yoink-dl.

URLs sent without a command are auto-detected and downloaded.

dl commands are available to all users with user role or higher by default - no explicit feature grant required. Access is enforced per-handler via AccessPolicy(min_role=user, check_group_enabled=True, check_thread_policy=True).

The inline handler is guarded by FeatureSpec(dl:inline, default_min_role=user) at the inline dispatcher level.

Mounted at /api/v1/dl/. Auth: JWT Bearer token.

• yt-dlp + gallery-dl backends with automatic URL detection
• Per-user settings - quality, codec (avc1/hevc/av1/vp9), container, proxy, subtitles, split size
• File cache - Telegram file_id reuse; supports multi-file results (albums)
• NSFW detection - configurable blur rules per domain/tag
• Rate limiting - per-minute/hour/day limits per user
• Inline mode - YouTube search (@bot query) with result caching
• Clipping - /cut cuts video by HH:MM:SS-HH:MM:SS range via ffmpeg; clip_start/clip_end written to download_log on success
• Clip media type - clips are stored with media_type="clip" (separate from video); shown with Clapperboard icon in history
• Forum topics - respects group forum topics, auto-creates "Downloads" DM topic
• Cookie pool - is_pool flag separates personal vs shared pool cookies; round-robin rotation via CookieManager; account info (name/avatar) fetched from YouTube/Instagram/Twitter APIs on upload; two-level dedup by session_key (SAPISID/uid) then content_hash
• Pool access - users with shared_cookies permission (or admin/owner role) get inherited pool cookies; use_pool_cookies user setting controls opt-in; has_pool_access returned in /dl/settings response
• Personal+pool rotation - when both exist for same domain, alternates between personal and pool cookie per user:domain key
• Progress tracking - real-time download progress in chat
• gallery-dl - e621.net and similar sites; multi-file results sent as media groups; gallery title fetched before download; zip archives named after the pool/gallery ({Pool Name}.zip)
• IPv6 rotation - optional outbound IPv6 source address rotation via IPv6Pool service; applied to both yt-dlp and gallery-dl requests
• Download retries - configurable via admin bot-settings (default 3); exponential backoff (1s/2s/4s); retries transient errors (ffmpeg crash, network); status message updated on each retry
• Informative errors - BotError shows localized message via i18n key; non-BotError shows sanitized message text instead of generic "unknown error"
• Download log - all download outcomes (ok, cached, error) written to download_log with user_id, group_id, thread_id, media_type, group_title
• _safe_filename() - Unicode lookalike substitution for <>:"/\|?* chars

By default all downloads go through yt-dlp (with optional Netscape cookie files). The YouTube TV OAuth path is an opt-in alternative that uses a dedicated Node.js sidecar (yoink-youtubei) running youtubei.js with a TV client and real OAuth tokens. It is designed for age-restricted videos and accounts where cookie export is impractical.

yt-dlp remains the default for every user. The OAuth path activates only when both conditions are met:

1. The user has completed the TV device flow authorization.
2. The user has switched "YouTube auth method" to "YouTube TV OAuth" in Settings.

1. User opens the Mini App, goes to the Cookies page, and clicks "Start authorization".
2. Backend calls POST /cookies/yttv/start which hits the Google OAuth device endpoint and returns a verification_url + user_code.
3. User opens the URL in any browser, enters the code, and approves the Google account.
4. Frontend polls GET /cookies/yttv/poll/{session_id} every 5 seconds. When Google confirms, the backend exchanges the device code for access_token + refresh_token.
5. Tokens are stored in the Cookie table with the content field prefixed __oauth2__ (Netscape cookie parsing skips these automatically).
6. The OAuth badge appears next to the cookie entry in the list.

Token refresh is handled transparently by the sidecar: youtubei.js fires an update-credentials event when the access token expires; the sidecar returns the new tokens in the X-Updated-Tokens response header; the Python client persists them back to the DB.

acquire_cookie() in url/pipeline/download_phase.py decides the path:

if youtube_auth_mode == "oauth" and url is youtube.com/youtu.be:
    tokens = cookie_mgr.get_oauth_tokens_for_url(user_id, url)
    if tokens:
        -> youtubei path (download_via_youtubei_job)
else:
    -> yt-dlp path (regular CookieManager)

Express HTTP server on port 9173, reachable at http://yoink-youtubei:9173 within the Docker network.

POST /download

{
  "url": "https://youtu.be/VIDEO_ID",
  "tokens": { "access_token": "...", "refresh_token": "...", "expiry_date": 1234567890 },
  "quality": "best",
  "audio_only": false,
  "start_sec": 3306,
  "end_sec": 3318
}

Response: binary file stream with headers:

• Content-Disposition: attachment; filename="<safe_title>.mp4"
• X-File-Title: <url-encoded title>
• X-Updated-Tokens: <json> (only when tokens were refreshed)

Flow inside the sidecar:

1. Create an Innertube session with client_type: TVHTML5 and sign in with the provided tokens.
2. Call getBasicInfo(videoId, 'TV'). The TV client with OAuth returns a reduced video_details (no title). Title is fetched via a second anonymous WEB client getBasicInfo call as a fallback.
3. Download video and audio streams separately (TV client provides only adaptive formats, no progressive). Prefer H.264 (avc) to avoid Telegram transcoding; fall back to any MP4.
4. Merge with ffmpeg. If start_sec/end_sec are provided, pass -ss <start> before inputs and -t <duration> after to clip during merge (stream copy, no re-encode).
5. For audio_only: download audio stream, trim with ffmpeg if clip params are present.

GET /info

Returns { title, duration, author } without downloading. Used for metadata prefetch (not currently called by the Python side but available).

Clips work the same way as with yt-dlp: parse_clip_spec() in url/clip.py parses the message text (e.g. https://youtu.be/X?t=3306 12 → ClipSpec(start_sec=3306, end_sec=3318)). The clip is passed through download_via_youtubei_job → download_via_youtubei → HTTP payload start_sec/end_sec → ffmpeg trim at merge time. The resulting DownloadJob carries the clip field so the upload phase shows the ✂️ HH:MM → HH:MM marker in the caption.

run_download in url/pipeline/run.py maintains a per-user asyncio.Semaphore(3) in bot_data["user_dl_semaphores"]. If all 3 slots are taken, the bot replies immediately with a "too many downloads" message and returns. The semaphore is acquired non-blocking (locked() check before acquire()) and released in the finally block alongside temp dir cleanup.

PTB is started with concurrent_updates=True so multiple URL messages from the same user are dispatched in parallel rather than queued sequentially.

Note: rate limits, timeout, max file size, retries, and playlist count are stored in BotSetting KV (dl.* keys) and editable at runtime via admin bot-settings page. Config values are used as fallback defaults only.