GithubHelp home page GithubHelp logo

blackjack4494 / yt-dlc Goto Github PK

View Code? Open in Web Editor NEW
2.9K 73.0 360.0 26.08 MB

media downloader and library for various sites.

License: The Unlicense

Makefile 0.11% Python 99.67% Shell 0.14% Batchfile 0.01% ActionScript 0.07%
downloader media library streaming

yt-dlc's Introduction

Build Status PyPi

Gitter chat License: Unlicense

youtube-dlc - download videos from youtube.com or other video platforms.

youtube-dlc is a fork of youtube-dl with the intention of getting features tested by the community merged in the tool faster, since youtube-dl's development seems to be slowing down. (https://web.archive.org/web/20201014194602/https://github.com/ytdl-org/youtube-dl/issues/26462)

INSTALLATION

How to update

All Platforms
Preferred way using pip:
You may want to use python3 instead of python

python -m pip install --upgrade youtube-dlc

To get the latest release with pip git

python -m pip install --upgrade git+https://github.com/blackjack4494/yt-dlc@release

If you want to install the current master branch

python -m pip install --upgrade git+https://github.com/blackjack4494/yt-dlc@master

UNIX (Linux, macOS, etc.)
Using wget:

sudo wget https://github.com/blackjack4494/yt-dlc/releases/latest/download/youtube-dlc -O /usr/local/bin/youtube-dlc
sudo chmod a+rx /usr/local/bin/youtube-dlc

Using curl:

sudo curl -L https://github.com/blackjack4494/yt-dlc/releases/latest/download/youtube-dlc -o /usr/local/bin/youtube-dlc
sudo chmod a+rx /usr/local/bin/youtube-dlc

Windows users can download youtube-dlc.exe (do not put in C:\Windows\System32!).
Or get the 32 Bit version youtube-dlc_x86.exe and rename it if needed.

Compile To build the Windows executable yourself (without version info!)

python -m pip install --upgrade pyinstaller
pyinstaller.exe youtube_dlc\__main__.py --onefile --name youtube-dlc

Or simply execute the make_win.bat if pyinstaller is installed. There will be a youtube-dlc.exe in /dist

New way to build Windows is to use python pyinst.py (please use python3 64Bit)
For 32 Bit Version use a 32 Bit Version of python (3 preferred here as well) and run python pyinst32.py

For Unix: You will need the required build tools
python, make (GNU), pandoc, zip, nosetests
Then simply type this

make

UPDATE

DO NOT UPDATE using -U ! instead download binaries again or when installed with pip use a described above when installing.
Next release will include in-built updater

DESCRIPTION

youtube-dlc is a command-line program to download videos from YouTube.com and a few more sites. It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on macOS. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.

youtube-dlc [OPTIONS] URL [URL...]

OPTIONS

-h, --help                       Print this help text and exit
--version                        Print program version and exit
-U, --update                     Update this program to latest version. Make
                                 sure that you have sufficient permissions
                                 (run with sudo if needed)
-i, --ignore-errors              Continue on download errors, for example to
                                 skip unavailable videos in a playlist
--abort-on-error                 Abort downloading of further videos (in the
                                 playlist or the command line) if an error
                                 occurs
--dump-user-agent                Display the current browser identification
--list-extractors                List all supported extractors
--extractor-descriptions         Output descriptions of all supported
                                 extractors
--force-generic-extractor        Force extraction to use the generic
                                 extractor
--default-search PREFIX          Use this prefix for unqualified URLs. For
                                 example "gvsearch2:" downloads two videos
                                 from google videos for youtube-dlc "large
                                 apple". Use the value "auto" to let
                                 youtube-dlc guess ("auto_warning" to emit a
                                 warning when guessing). "error" just throws
                                 an error. The default value "fixup_error"
                                 repairs broken URLs, but emits an error if
                                 this is not possible instead of searching.
--ignore-config                  Do not read configuration files. When given
                                 in the global configuration file
                                 /etc/youtube-dlc.conf: Do not read the user
                                 configuration in ~/.config/youtube-
                                 dlc/config (%APPDATA%/youtube-
                                 dlc/config.txt on Windows)
--config-location PATH           Location of the configuration file; either
                                 the path to the config or its containing
                                 directory.
--flat-playlist                  Do not extract the videos of a playlist,
                                 only list them.
--mark-watched                   Mark videos watched (YouTube only)
--no-mark-watched                Do not mark videos watched (YouTube only)
--no-color                       Do not emit color codes in output

Network Options:

--proxy URL                      Use the specified HTTP/HTTPS/SOCKS proxy.
                                 To enable SOCKS proxy, specify a proper
                                 scheme. For example
                                 socks5://127.0.0.1:1080/. Pass in an empty
                                 string (--proxy "") for direct connection
--socket-timeout SECONDS         Time to wait before giving up, in seconds
--source-address IP              Client-side IP address to bind to
-4, --force-ipv4                 Make all connections via IPv4
-6, --force-ipv6                 Make all connections via IPv6

Geo Restriction:

--geo-verification-proxy URL     Use this proxy to verify the IP address for
                                 some geo-restricted sites. The default
                                 proxy specified by --proxy (or none, if the
                                 option is not present) is used for the
                                 actual downloading.
--geo-bypass                     Bypass geographic restriction via faking
                                 X-Forwarded-For HTTP header
--no-geo-bypass                  Do not bypass geographic restriction via
                                 faking X-Forwarded-For HTTP header
--geo-bypass-country CODE        Force bypass geographic restriction with
                                 explicitly provided two-letter ISO 3166-2
                                 country code
--geo-bypass-ip-block IP_BLOCK   Force bypass geographic restriction with
                                 explicitly provided IP block in CIDR
                                 notation

Video Selection:

--playlist-start NUMBER          Playlist video to start at (default is 1)
--playlist-end NUMBER            Playlist video to end at (default is last)
--playlist-items ITEM_SPEC       Playlist video items to download. Specify
                                 indices of the videos in the playlist
                                 separated by commas like: "--playlist-items
                                 1,2,5,8" if you want to download videos
                                 indexed 1, 2, 5, 8 in the playlist. You can
                                 specify range: "--playlist-items
                                 1-3,7,10-13", it will download the videos
                                 at index 1, 2, 3, 7, 10, 11, 12 and 13.
--match-title REGEX              Download only matching titles (regex or
                                 caseless sub-string)
--reject-title REGEX             Skip download for matching titles (regex or
                                 caseless sub-string)
--max-downloads NUMBER           Abort after downloading NUMBER files
--min-filesize SIZE              Do not download any videos smaller than
                                 SIZE (e.g. 50k or 44.6m)
--max-filesize SIZE              Do not download any videos larger than SIZE
                                 (e.g. 50k or 44.6m)
--date DATE                      Download only videos uploaded in this date
--datebefore DATE                Download only videos uploaded on or before
                                 this date (i.e. inclusive)
--dateafter DATE                 Download only videos uploaded on or after
                                 this date (i.e. inclusive)
--min-views COUNT                Do not download any videos with less than
                                 COUNT views
--max-views COUNT                Do not download any videos with more than
                                 COUNT views
--match-filter FILTER            Generic video filter. Specify any key (see
                                 the "OUTPUT TEMPLATE" for a list of
                                 available keys) to match if the key is
                                 present, !key to check if the key is not
                                 present, key > NUMBER (like "comment_count
                                 > 12", also works with >=, <, <=, !=, =) to
                                 compare against a number, key = 'LITERAL'
                                 (like "uploader = 'Mike Smith'", also works
                                 with !=) to match against a string literal
                                 and & to require multiple matches. Values
                                 which are not known are excluded unless you
                                 put a question mark (?) after the operator.
                                 For example, to only match videos that have
                                 been liked more than 100 times and disliked
                                 less than 50 times (or the dislike
                                 functionality is not available at the given
                                 service), but who also have a description,
                                 use --match-filter "like_count > 100 &
                                 dislike_count <? 50 & description" .
--no-playlist                    Download only the video, if the URL refers
                                 to a video and a playlist.
--yes-playlist                   Download the playlist, if the URL refers to
                                 a video and a playlist.
--age-limit YEARS                Download only videos suitable for the given
                                 age
--download-archive FILE          Download only videos not listed in the
                                 archive file. Record the IDs of all
                                 downloaded videos in it.
--break-on-existing              Stop the download process after attempting
                                 to download a file that's in the archive.
--include-ads                    Download advertisements as well
                                 (experimental)

Download Options:

-r, --limit-rate RATE            Maximum download rate in bytes per second
                                 (e.g. 50K or 4.2M)
-R, --retries RETRIES            Number of retries (default is 10), or
                                 "infinite".
--fragment-retries RETRIES       Number of retries for a fragment (default
                                 is 10), or "infinite" (DASH, hlsnative and
                                 ISM)
--skip-unavailable-fragments     Skip unavailable fragments (DASH, hlsnative
                                 and ISM)
--abort-on-unavailable-fragment  Abort downloading when some fragment is not
                                 available
--keep-fragments                 Keep downloaded fragments on disk after
                                 downloading is finished; fragments are
                                 erased by default
--buffer-size SIZE               Size of download buffer (e.g. 1024 or 16K)
                                 (default is 1024)
--no-resize-buffer               Do not automatically adjust the buffer
                                 size. By default, the buffer size is
                                 automatically resized from an initial value
                                 of SIZE.
--http-chunk-size SIZE           Size of a chunk for chunk-based HTTP
                                 downloading (e.g. 10485760 or 10M) (default
                                 is disabled). May be useful for bypassing
                                 bandwidth throttling imposed by a webserver
                                 (experimental)
--playlist-reverse               Download playlist videos in reverse order
--playlist-random                Download playlist videos in random order
--xattr-set-filesize             Set file xattribute ytdl.filesize with
                                 expected file size
--hls-prefer-native              Use the native HLS downloader instead of
                                 ffmpeg
--hls-prefer-ffmpeg              Use ffmpeg instead of the native HLS
                                 downloader
--hls-use-mpegts                 Use the mpegts container for HLS videos,
                                 allowing to play the video while
                                 downloading (some players may not be able
                                 to play it)
--external-downloader COMMAND    Use the specified external downloader.
                                 Currently supports
                                 aria2c,avconv,axel,curl,ffmpeg,httpie,wget
--external-downloader-args ARGS  Give these arguments to the external
                                 downloader

Filesystem Options:

-a, --batch-file FILE            File containing URLs to download ('-' for
                                 stdin), one URL per line. Lines starting
                                 with '#', ';' or ']' are considered as
                                 comments and ignored.
--id                             Use only video ID in file name
-o, --output TEMPLATE            Output filename template, see the "OUTPUT
                                 TEMPLATE" for all the info
--autonumber-start NUMBER        Specify the start value for %(autonumber)s
                                 (default is 1)
--restrict-filenames             Restrict filenames to only ASCII
                                 characters, and avoid "&" and spaces in
                                 filenames
-w, --no-overwrites              Do not overwrite files
-c, --continue                   Force resume of partially downloaded files.
                                 By default, youtube-dlc will resume
                                 downloads if possible.
--no-continue                    Do not resume partially downloaded files
                                 (restart from beginning)
--no-part                        Do not use .part files - write directly
                                 into output file
--no-mtime                       Do not use the Last-modified header to set
                                 the file modification time
--write-description              Write video description to a .description
                                 file
--write-info-json                Write video metadata to a .info.json file
--write-annotations              Write video annotations to a
                                 .annotations.xml file
--load-info-json FILE            JSON file containing the video information
                                 (created with the "--write-info-json"
                                 option)
--cookies FILE                   File to read cookies from and dump cookie
                                 jar in
--cache-dir DIR                  Location in the filesystem where youtube-
                                 dlc can store some downloaded information
                                 permanently. By default
                                 $XDG_CACHE_HOME/youtube-dlc or
                                 ~/.cache/youtube-dlc . At the moment, only
                                 YouTube player files (for videos with
                                 obfuscated signatures) are cached, but that
                                 may change.
--no-cache-dir                   Disable filesystem caching
--rm-cache-dir                   Delete all filesystem cache files
--trim-file-name                 Limit the filename length (extension
                                 excluded)

Thumbnail images:

--write-thumbnail                Write thumbnail image to disk
--write-all-thumbnails           Write all thumbnail image formats to disk
--list-thumbnails                Simulate and list all available thumbnail
                                 formats

Verbosity / Simulation Options:

-q, --quiet                      Activate quiet mode
--no-warnings                    Ignore warnings
-s, --simulate                   Do not download the video and do not write
                                 anything to disk
--skip-download                  Do not download the video
-g, --get-url                    Simulate, quiet but print URL
-e, --get-title                  Simulate, quiet but print title
--get-id                         Simulate, quiet but print id
--get-thumbnail                  Simulate, quiet but print thumbnail URL
--get-description                Simulate, quiet but print video description
--get-duration                   Simulate, quiet but print video length
--get-filename                   Simulate, quiet but print output filename
--get-format                     Simulate, quiet but print output format
-j, --dump-json                  Simulate, quiet but print JSON information.
                                 See the "OUTPUT TEMPLATE" for a description
                                 of available keys.
-J, --dump-single-json           Simulate, quiet but print JSON information
                                 for each command-line argument. If the URL
                                 refers to a playlist, dump the whole
                                 playlist information in a single line.
--print-json                     Be quiet and print the video information as
                                 JSON (video is still being downloaded).
--newline                        Output progress bar as new lines
--no-progress                    Do not print progress bar
--console-title                  Display progress in console titlebar
-v, --verbose                    Print various debugging information
--dump-pages                     Print downloaded pages encoded using base64
                                 to debug problems (very verbose)
--write-pages                    Write downloaded intermediary pages to
                                 files in the current directory to debug
                                 problems
--print-traffic                  Display sent and read HTTP traffic
-C, --call-home                  Contact the youtube-dlc server for
                                 debugging
--no-call-home                   Do NOT contact the youtube-dlc server for
                                 debugging

Workarounds:

--encoding ENCODING              Force the specified encoding (experimental)
--no-check-certificate           Suppress HTTPS certificate validation
--prefer-insecure                Use an unencrypted connection to retrieve
                                 information about the video. (Currently
                                 supported only for YouTube)
--user-agent UA                  Specify a custom user agent
--referer URL                    Specify a custom referer, use if the video
                                 access is restricted to one domain
--add-header FIELD:VALUE         Specify a custom HTTP header and its value,
                                 separated by a colon ':'. You can use this
                                 option multiple times
--bidi-workaround                Work around terminals that lack
                                 bidirectional text support. Requires bidiv
                                 or fribidi executable in PATH
--sleep-interval SECONDS         Number of seconds to sleep before each
                                 download when used alone or a lower bound
                                 of a range for randomized sleep before each
                                 download (minimum possible number of
                                 seconds to sleep) when used along with
                                 --max-sleep-interval.
--max-sleep-interval SECONDS     Upper bound of a range for randomized sleep
                                 before each download (maximum possible
                                 number of seconds to sleep). Must only be
                                 used along with --min-sleep-interval.
--sleep-subtitles                Enforce sleep interval on subtitles as well.

Video Format Options:

-f, --format FORMAT              Video format code, see the "FORMAT
                                 SELECTION" for all the info
--all-formats                    Download all available video formats
--prefer-free-formats            Prefer free video formats unless a specific
                                 one is requested
-F, --list-formats               List all available formats of requested
                                 videos
--youtube-skip-dash-manifest     Do not download the DASH manifests and
                                 related data on YouTube videos
--youtube-skip-hls-manifest      Do not download the HLS manifests and
                                 related data on YouTube videos
--merge-output-format FORMAT     If a merge is required (e.g.
                                 bestvideo+bestaudio), output to given
                                 container format. One of mkv, mp4, ogg,
                                 webm, flv. Ignored if no merge is required

Subtitle Options:

--write-sub                      Write subtitle file
--write-auto-sub                 Write automatically generated subtitle file
                                 (YouTube only)
--all-subs                       Download all the available subtitles of the
                                 video
--list-subs                      List all available subtitles for the video
--sub-format FORMAT              Subtitle format, accepts formats
                                 preference, for example: "srt" or
                                 "ass/srt/best"
--sub-lang LANGS                 Languages of the subtitles to download
                                 (optional) separated by commas, use --list-
                                 subs for available language tags

Authentication Options:

-u, --username USERNAME          Login with this account ID
-p, --password PASSWORD          Account password. If this option is left
                                 out, youtube-dlc will ask interactively.
-2, --twofactor TWOFACTOR        Two-factor authentication code
-n, --netrc                      Use .netrc authentication data
--video-password PASSWORD        Video password (vimeo, smotri, youku)

Adobe Pass Options:

--ap-mso MSO                     Adobe Pass multiple-system operator (TV
                                 provider) identifier, use --ap-list-mso for
                                 a list of available MSOs
--ap-username USERNAME           Multiple-system operator account login
--ap-password PASSWORD           Multiple-system operator account password.
                                 If this option is left out, youtube-dlc
                                 will ask interactively.
--ap-list-mso                    List all supported multiple-system
                                 operators

Post-processing Options:

-x, --extract-audio              Convert video files to audio-only files
                                 (requires ffmpeg or avconv and ffprobe or
                                 avprobe)
--audio-format FORMAT            Specify audio format: "best", "aac",
                                 "flac", "mp3", "m4a", "opus", "vorbis", or
                                 "wav"; "best" by default; No effect without
                                 -x
--audio-quality QUALITY          Specify ffmpeg/avconv audio quality, insert
                                 a value between 0 (better) and 9 (worse)
                                 for VBR or a specific bitrate like 128K
                                 (default 5)
--remux-video FORMAT             Remux the video to another container format
                                 if necessary (currently supported: mp4|mkv,
                                 target container format must support video
                                 / audio encoding, remuxing may fail)
--recode-video FORMAT            Encode the video to another format if
                                 necessary (currently supported:
                                 mp4|flv|ogg|webm|mkv|avi)
--postprocessor-args ARGS        Give these arguments to the postprocessor
-k, --keep-video                 Keep the video file on disk after the post-
                                 processing; the video is erased by default
--no-post-overwrites             Do not overwrite post-processed files; the
                                 post-processed files are overwritten by
                                 default
--embed-subs                     Embed subtitles in the video (only for mp4,
                                 webm and mkv videos)
--embed-thumbnail                Embed thumbnail in the audio as cover art
--add-metadata                   Write metadata to the video file
--metadata-from-title FORMAT     Parse additional metadata like song title /
                                 artist from the video title. The format
                                 syntax is the same as --output. Regular
                                 expression with named capture groups may
                                 also be used. The parsed parameters replace
                                 existing values. Example: --metadata-from-
                                 title "%(artist)s - %(title)s" matches a
                                 title like "Coldplay - Paradise". Example
                                 (regex): --metadata-from-title
                                 "(?P<artist>.+?) - (?P<title>.+)"
--xattrs                         Write metadata to the video file's xattrs
                                 (using dublin core and xdg standards)
--fixup POLICY                   Automatically correct known faults of the
                                 file. One of never (do nothing), warn (only
                                 emit a warning), detect_or_warn (the
                                 default; fix file if we can, warn
                                 otherwise)
--prefer-avconv                  Prefer avconv over ffmpeg for running the
                                 postprocessors
--prefer-ffmpeg                  Prefer ffmpeg over avconv for running the
                                 postprocessors (default)
--ffmpeg-location PATH           Location of the ffmpeg/avconv binary;
                                 either the path to the binary or its
                                 containing directory.
--exec CMD                       Execute a command on the file after
                                 downloading and post-processing, similar to
                                 find's -exec syntax. Example: --exec 'adb
                                 push {} /sdcard/Music/ && rm {}'
--convert-subs FORMAT            Convert the subtitles to other format
                                 (currently supported: srt|ass|vtt|lrc)

Extractor Options:

--ignore-dynamic-mpd             Do not process dynamic DASH manifests

CONFIGURATION

You can configure youtube-dlc by placing any supported command line option to a configuration file. On Linux and macOS, the system wide configuration file is located at /etc/youtube-dlc.conf and the user wide configuration file at ~/.config/youtube-dlc/config. On Windows, the user wide configuration file locations are %APPDATA%\youtube-dlc\config.txt or C:\Users\<user name>\youtube-dlc.conf. Note that by default configuration file may not exist so you may need to create it yourself.

For example, with the following configuration file youtube-dlc will always extract the audio, not copy the mtime, use a proxy and save all videos under Movies directory in your home directory:

# Lines starting with # are comments

# Always extract audio
-x

# Do not copy the mtime
--no-mtime

# Use this proxy
--proxy 127.0.0.1:3128

# Save all videos under Movies directory in your home directory
-o ~/Movies/%(title)s.%(ext)s

Note that options in configuration file are just the same options aka switches used in regular command line calls thus there must be no whitespace after - or --, e.g. -o or --proxy but not - o or -- proxy.

You can use --ignore-config if you want to disable the configuration file for a particular youtube-dlc run.

You can also use --config-location if you want to use custom configuration file for a particular youtube-dlc run.

Authentication with .netrc file

You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with --username and --password) in order not to pass credentials as command line arguments on every youtube-dlc execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a .netrc file on a per extractor basis. For that you will need to create a .netrc file in your $HOME and restrict permissions to read/write by only you:

touch $HOME/.netrc
chmod a-rwx,u+rw $HOME/.netrc

After that you can add credentials for an extractor in the following format, where extractor is the name of the extractor in lowercase:

machine <extractor> login <login> password <password>

For example:

machine youtube login [email protected] password my_youtube_password
machine twitch login my_twitch_account_name password my_twitch_password

To activate authentication with the .netrc file you should pass --netrc to youtube-dlc or place it in the configuration file.

On Windows you may also need to setup the %HOME% environment variable manually. For example:

set HOME=%USERPROFILE%

OUTPUT TEMPLATE

The -o option allows users to indicate a template for the output file names.

tl;dr: navigate me to examples.

The basic usage is not to set any template arguments when downloading a single file, like in youtube-dlc -o funny_video.flv "https://some/video". However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to python string formatting operations. For example, %(NAME)s or %(NAME)05d. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Allowed names along with sequence type are:

  • id (string): Video identifier
  • title (string): Video title
  • url (string): Video URL
  • ext (string): Video filename extension
  • alt_title (string): A secondary title of the video
  • display_id (string): An alternative identifier for the video
  • uploader (string): Full name of the video uploader
  • license (string): License name the video is licensed under
  • creator (string): The creator of the video
  • release_date (string): The date (YYYYMMDD) when the video was released
  • timestamp (numeric): UNIX timestamp of the moment the video became available
  • upload_date (string): Video upload date (YYYYMMDD)
  • uploader_id (string): Nickname or id of the video uploader
  • channel (string): Full name of the channel the video is uploaded on
  • channel_id (string): Id of the channel
  • location (string): Physical location where the video was filmed
  • duration (numeric): Length of the video in seconds
  • view_count (numeric): How many users have watched the video on the platform
  • like_count (numeric): Number of positive ratings of the video
  • dislike_count (numeric): Number of negative ratings of the video
  • repost_count (numeric): Number of reposts of the video
  • average_rating (numeric): Average rating give by users, the scale used depends on the webpage
  • comment_count (numeric): Number of comments on the video
  • age_limit (numeric): Age restriction for the video (years)
  • is_live (boolean): Whether this video is a live stream or a fixed-length video
  • start_time (numeric): Time in seconds where the reproduction should start, as specified in the URL
  • end_time (numeric): Time in seconds where the reproduction should end, as specified in the URL
  • format (string): A human-readable description of the format
  • format_id (string): Format code specified by --format
  • format_note (string): Additional info about the format
  • width (numeric): Width of the video
  • height (numeric): Height of the video
  • resolution (string): Textual description of width and height
  • tbr (numeric): Average bitrate of audio and video in KBit/s
  • abr (numeric): Average audio bitrate in KBit/s
  • acodec (string): Name of the audio codec in use
  • asr (numeric): Audio sampling rate in Hertz
  • vbr (numeric): Average video bitrate in KBit/s
  • fps (numeric): Frame rate
  • vcodec (string): Name of the video codec in use
  • container (string): Name of the container format
  • filesize (numeric): The number of bytes, if known in advance
  • filesize_approx (numeric): An estimate for the number of bytes
  • protocol (string): The protocol that will be used for the actual download
  • extractor (string): Name of the extractor
  • extractor_key (string): Key name of the extractor
  • epoch (numeric): Unix epoch when creating the file
  • autonumber (numeric): Number that will be increased with each download, starting at --autonumber-start
  • playlist (string): Name or id of the playlist that contains the video
  • playlist_index (numeric): Index of the video in the playlist padded with leading zeros according to the total length of the playlist
  • playlist_id (string): Playlist identifier
  • playlist_title (string): Playlist title
  • playlist_uploader (string): Full name of the playlist uploader
  • playlist_uploader_id (string): Nickname or id of the playlist uploader

Available for the video that belongs to some logical chapter or section:

  • chapter (string): Name or title of the chapter the video belongs to
  • chapter_number (numeric): Number of the chapter the video belongs to
  • chapter_id (string): Id of the chapter the video belongs to

Available for the video that is an episode of some series or programme:

  • series (string): Title of the series or programme the video episode belongs to
  • season (string): Title of the season the video episode belongs to
  • season_number (numeric): Number of the season the video episode belongs to
  • season_id (string): Id of the season the video episode belongs to
  • episode (string): Title of the video episode
  • episode_number (numeric): Number of the video episode within a season
  • episode_id (string): Id of the video episode

Available for the media that is a track or a part of a music album:

  • track (string): Title of the track
  • track_number (numeric): Number of the track within an album or a disc
  • track_id (string): Id of the track
  • artist (string): Artist(s) of the track
  • genre (string): Genre(s) of the track
  • album (string): Title of the album the track belongs to
  • album_type (string): Type of the album
  • album_artist (string): List of all artists appeared on the album
  • disc_number (numeric): Number of the disc or other physical medium the track belongs to
  • release_year (numeric): Year (YYYY) when the album was released

Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with NA.

For example for -o %(title)s-%(id)s.%(ext)s and an mp4 video with title youtube-dlc test video and id BaW_jenozKcj, this will result in a youtube-dlc test video-BaW_jenozKcj.mp4 file created in the current directory.

For numeric sequences you can use numeric related formatting, for example, %(view_count)05d will result in a string with view count padded with zeros up to 5 characters, like in 00042.

Output templates can also contain arbitrary hierarchical path, e.g. -o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' which will result in downloading each video in a directory corresponding to this path template. Any missing directory will be automatically created for you.

To use percent literals in an output template use %%. To output to stdout use -o -.

The current default template is %(title)s-%(id)s.%(ext)s.

In some cases, you don't want special characters such as 中, spaces, or &, such as when transferring the downloaded filename to a Windows system or the filename through an 8bit-unsafe channel. In these cases, add the --restrict-filenames flag to get a shorter title:

Output template and Windows batch files

If you are using an output template inside a Windows batch file then you must escape plain percent characters (%) by doubling, so that -o "%(title)s-%(id)s.%(ext)s" should become -o "%%(title)s-%%(id)s.%%(ext)s". However you should not touch %'s that are not plain characters, e.g. environment variables for expansion should stay intact: -o "C:\%HOMEPATH%\Desktop\%%(title)s.%%(ext)s".

Output template examples

Note that on Windows you may need to use double quotes instead of single.

$ youtube-dlc --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc
youtube-dlc test video ''_ä↭𝕐.mp4    # All kinds of weird characters

$ youtube-dlc --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc --restrict-filenames
youtube-dlc_test_video_.mp4          # A simple file name

# Download YouTube playlist videos in separate directory indexed by video order in a playlist
$ youtube-dlc -o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re

# Download all playlists of YouTube channel/user keeping each playlist in separate directory:
$ youtube-dlc -o '%(uploader)s/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' https://www.youtube.com/user/TheLinuxFoundation/playlists

# Download Udemy course keeping each chapter in separate directory under MyVideos directory in your home
$ youtube-dlc -u user -p password -o '~/MyVideos/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/java-tutorial/

# Download entire series season keeping each series and each season in separate directory under C:/MyVideos
$ youtube-dlc -o "C:/MyVideos/%(series)s/%(season_number)s - %(season)s/%(episode_number)s - %(episode)s.%(ext)s" https://videomore.ru/kino_v_detalayah/5_sezon/367617

# Stream the video being downloaded to stdout
$ youtube-dlc -o - BaW_jenozKc

FORMAT SELECTION

By default youtube-dlc tries to download the best available quality, i.e. if you want the best quality you don't need to pass any special options, youtube-dlc will guess it for you by default.

But sometimes you may want to download in a different format, for example when you are on a slow or intermittent connection. The key mechanism for achieving this is so-called format selection based on which you can explicitly specify desired format, select formats based on some criterion or criteria, setup precedence and much more.

The general syntax for format selection is --format FORMAT or shorter -f FORMAT where FORMAT is a selector expression, i.e. an expression that describes format or formats you would like to download.

tl;dr: navigate me to examples.

The simplest case is requesting a specific format, for example with -f 22 you can download the format with format code equal to 22. You can get the list of available format codes for particular video using --list-formats or -F. Note that these format codes are extractor specific.

You can also use a file extension (currently 3gp, aac, flv, m4a, mp3, mp4, ogg, wav, webm are supported) to download the best quality format of a particular file extension served as a single file, e.g. -f webm will download the best quality format with the webm extension served as a single file.

You can also use special names to select particular edge case formats:

  • best: Select the best quality format represented by a single file with video and audio.
  • worst: Select the worst quality format represented by a single file with video and audio.
  • bestvideo: Select the best quality video-only format (e.g. DASH video). May not be available.
  • worstvideo: Select the worst quality video-only format. May not be available.
  • bestaudio: Select the best quality audio only-format. May not be available.
  • worstaudio: Select the worst quality audio only-format. May not be available.

For example, to download the worst quality video-only format you can use -f worstvideo.

If you want to download multiple videos and they don't have the same formats available, you can specify the order of preference using slashes. Note that slash is left-associative, i.e. formats on the left hand side are preferred, for example -f 22/17/18 will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download.

If you want to download several formats of the same video use a comma as a separator, e.g. -f 22,17,18 will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: -f 136/137/mp4/bestvideo,140/m4a/bestaudio.

You can also filter the video formats by putting a condition in brackets, as in -f "best[height=720]" (or -f "[filesize>10M]").

The following numeric meta fields can be used with comparisons <, <=, >, >=, = (equals), != (not equals):

  • filesize: The number of bytes, if known in advance
  • width: Width of the video, if known
  • height: Height of the video, if known
  • tbr: Average bitrate of audio and video in KBit/s
  • abr: Average audio bitrate in KBit/s
  • vbr: Average video bitrate in KBit/s
  • asr: Audio sampling rate in Hertz
  • fps: Frame rate

Also filtering work for comparisons = (equals), ^= (starts with), $= (ends with), *= (contains) and following string meta fields:

  • ext: File extension
  • acodec: Name of the audio codec in use
  • vcodec: Name of the video codec in use
  • container: Name of the container format
  • protocol: The protocol that will be used for the actual download, lower-case (http, https, rtsp, rtmp, rtmpe, mms, f4m, ism, http_dash_segments, m3u8, or m3u8_native)
  • format_id: A short description of the format

Any string comparison may be prefixed with negation ! in order to produce an opposite comparison, e.g. !*= (does not contain).

Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the video hoster.

Formats for which the value is not known are excluded unless you put a question mark (?) after the operator. You can combine format filters, so -f "[height <=? 720][tbr>500]" selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s.

You can merge the video and audio of two formats into a single file using -f <video-format>+<audio-format> (requires ffmpeg or avconv installed), for example -f bestvideo+bestaudio will download the best video-only format, the best audio-only format and mux them together with ffmpeg/avconv.

Format selectors can also be grouped using parentheses, for example if you want to download the best mp4 and webm formats with a height lower than 480 you can use -f '(mp4,webm)[height<480]'.

Since the end of April 2015 and version 2015.04.26, youtube-dlc uses -f bestvideo+bestaudio/best as the default format selection (see #5447, #5456). If ffmpeg or avconv are installed this results in downloading bestvideo and bestaudio separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to best and results in downloading the best available quality served as a single file. best is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add -f bestvideo[height<=?1080]+bestaudio/best to your configuration file. Note that if you use youtube-dlc to stream to stdout (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as -o -, youtube-dlc still uses -f best format selection in order to start content delivery immediately to your player and not to wait until bestvideo and bestaudio are downloaded and muxed.

If you want to preserve the old format selection behavior (prior to youtube-dlc 2015.04.26), i.e. you want to download the best available quality media served as a single file, you should explicitly specify your choice with -f best. You may want to add it to the configuration file in order not to type it every time you run youtube-dlc.

Format selection examples

Note that on Windows you may need to use double quotes instead of single.

# Download best mp4 format available or any other best if no mp4 available
$ youtube-dlc -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best'

# Download best format available but no better than 480p
$ youtube-dlc -f 'bestvideo[height<=480]+bestaudio/best[height<=480]'

# Download best video only format but no bigger than 50 MB
$ youtube-dlc -f 'best[filesize<50M]'

# Download best format available via direct link over HTTP/HTTPS protocol
$ youtube-dlc -f '(bestvideo+bestaudio/best)[protocol^=http]'

# Download the best video format and the best audio format without merging them
$ youtube-dlc -f 'bestvideo,bestaudio' -o '%(title)s.f%(format_id)s.%(ext)s'

Note that in the last example, an output template is recommended as bestvideo and bestaudio may have the same file name.

VIDEO SELECTION

Videos can be filtered by their upload date using the options --date, --datebefore or --dateafter. They accept dates in two formats:

  • Absolute dates: Dates in the format YYYYMMDD.
  • Relative dates: Dates in the format (now|today)[+-][0-9](day|week|month|year)(s)?

Examples:

# Download only the videos uploaded in the last 6 months
$ youtube-dlc --dateafter now-6months

# Download only the videos uploaded on January 1, 1970
$ youtube-dlc --date 19700101

$ # Download only the videos uploaded in the 200x decade
$ youtube-dlc --dateafter 20000101 --datebefore 20091231

yt-dlc's People

Contributors

atomicdryad avatar blackjack4494 avatar dineshs91 avatar dstftw avatar filosottile avatar fstirlitz avatar gcmalloc avatar gebn avatar goggle avatar iemejia avatar jaimemf avatar jeffcrouse avatar johnhawkinson avatar jwilk avatar lalinsky avatar mikecol avatar parmjitv avatar phihag avatar pukkandan avatar pulpe avatar remitamine avatar rg3 avatar rrooij avatar rudloff avatar rzhxeo avatar tithen-firion avatar trox1972 avatar xanadu avatar xarantolus avatar yasoob avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yt-dlc's Issues

[Broken] Youtube

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

 youtube-dlc -x --audio-format mp3 https://www.youtube.com/watch?v=0Qd_k0hv3Nw -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-x', '--audio-format', 'mp3', 'https://www.youtube.com/watch?v=0Qd_k0hv3Nw', '-v']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.9.0 (CPython) - macOS-10.13.6-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg present, ffprobe present, rtmpdump Reason
[debug] Proxy map: {}
[youtube] 0Qd_k0hv3Nw: Downloading webpage
[youtube] 0Qd_k0hv3Nw: Downloading embed webpage
ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python3.9/site-packages/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.9/site-packages/youtube_dlc/extractor/youtube.py", line 2065, in _real_extract
    jsplayer_url_json = self._search_regex(
  File "/usr/local/lib/python3.9/site-packages/youtube_dlc/extractor/common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dlc.utils.RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

As of yesterday downloading anything from Youtube seems to fail.

[Broken] 7plus not working for some episodes

Checklist

  • [x ] I'm reporting a broken site support
  • [x ] I've verified that I'm running youtube-dlc version 2020.10.26
  • [x ] I've checked that all provided URLs are alive and playable in a browser
  • [x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • [x ] I've searched the bugtracker for similar issues including closed ones

Verbose log

>youtube-dlc --verbose https://7plus.com.au/housos?episode-id=HOUS03-001
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://7plus.com.au/housos?episode-id=HOUS03-001']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg git-2020-06-28-4cfcfb3, ffprobe git-2020-06-28-4cfcfb3
[debug] Proxy map: {}
[7plus] HOUS03-001: Downloading JSON metadata
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\sevenplus.py", line 61, in _real_extract
  File "youtube_dlc\extractor\brightcove.py", line 551, in _parse_brightcove_metadata
  File "youtube_dlc\extractor\common.py", line 1359, in _sort_formats
youtube_dlc.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

7plus downloader no longer working for some episodes in particular the latest episode of Housos found here: https://7plus.com.au/housos?episode-id=HOUS03-001

Thanks :)

Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:1124)> (caused by URLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1124)')

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

$ youtube-dlc --verbose --extract-audio --audio-format mp3 --audio-quality 0 "https://www.youtube.com/watch?v=I8k4jk2ParA"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--extract-audio', '--audio-format', 'mp3', '--audio-quality', '0', 'https://www.youtube.com/watch?v=I8k4jk2ParA']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out cp1252, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] I8k4jk2ParA: Downloading webpage
ERROR: Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:1124)> (caused by URLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1124)')))
  File "youtube_dlc\extractor\common.py", line 632, in _request_webpage
  File "youtube_dlc\YoutubeDL.py", line 2308, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "youtube_dlc\utils.py", line 2736, in https_open
  File "urllib\request.py", line 1353, in do_open

Description

WRITE DESCRIPTION HERE
Getting error when try to download

ERROR: Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:1124)> (caused by URLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1124)')))
  File "youtube_dlc\extractor\common.py", line 632, in _request_webpage
  File "youtube_dlc\YoutubeDL.py", line 2308, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "youtube_dlc\utils.py", line 2736, in https_open
  File "urllib\request.py", line 1353, in do_open

[Broken] ERROR: Unable to extract JS player URL

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

@bq:03:11:27:/tmp/dl$ youtube-dlc --verbose -F https://www.youtube.com/watch?v=hKAB91trmbg
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-F', u'https://www.youtube.com/watch?v=hKAB91trmbg']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.26
[debug] Python version 2.7.18 (CPython) - Linux-5.8.0-3-amd64-x86_64-with-debian-bullseye-sid
[debug] exe versions: avconv 4.3.1-3, avprobe 4.3.1-3, ffmpeg 4.3.1-3, ffprobe 4.3.1-3, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] hKAB91trmbg: Downloading webpage
[youtube] hKAB91trmbg: Downloading embed webpage
ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1966, in _real_extract
    yt_initial_data['contents']['twoColumnWatchNextResults']['conversationBar']['liveChatRenderer']['continuations'][0]['reloadContinuationData']['continuation']
KeyError: u'conversationBar'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 2066, in _real_extract
    ASSETS_RE, embed_webpage, 'JS player URL')
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

[Question]

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

Hi, I just installed youtube-dlc on my MacBook using cURL but for all the commands I made, I ended up with syntax error near unexpected token 'newline' as shown in screenshot below

image

Is this a bug or anything I did wrong?

[Feature Request] GAnarchy project commit

Checklist

  • I'm reporting a feature request
  • [N/A] I've verified that I'm running youtube-dlc version 2020.10.24
  • I've searched the bugtracker for similar feature requests including closed ones

Description

WRITE DESCRIPTION HERE

GAnarchy is a project I started about when the Mastodon developers refused to add a privacy-protecting feature I requested. it was originally created with the long-term goal of taking over the Mastodon project.

I kinda grew tired of Mastodon but I still stand behind GAnarchy. I think it's an useful project. I think forks of projects such as OpenWRT/LEDE, OpenSSL/LibreSSL, DooM/Sonic Robo Blast 2, and yt-dl/yt-dlc would all benefit from it in different ways. In particular for yt-dl/yt-dlc, I believe it can speed up development efforts even further (by completely decentralizing them), as well as provide strong censorship resistance (because it's so decentralized).

You can learn more about GAnarchy by checking out the subreddit at https://reddit.com/r/GAnarchy or one of the various instances such as https://ganarchy.github.io/ or https://ganarchy.autistic.space/. I've yet to do a proper write-up of it tho.

All it needs is a project maintainer to create a "project commit". They're formatted like so:

[Project] GAnarchy

A Project Page Generator written in Python, focused on giving forks of a
project the same visibility as the original repo.

[Broken] Unable to download Age Restricted videos on Youtube

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

[youtube] 1234567890: Downloading webpage
[youtube] 1234567890: Downloading embed webpage
[youtube] 1234567890: Refetching age-gated info webpage
Traceback (most recent call last):
  File "youtube_dlc\__main__.py", line 19, in <module>
  File "youtube_dlc\__init__.py", line 487, in main
  File "youtube_dlc\__init__.py", line 477, in _real_main
  File "youtube_dlc\YoutubeDL.py", line 2095, in download
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\youtube.py", line 2068, in _real_extract
  File "json\__init__.py", line 341, in loads
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
[22460] Failed to execute script __main__

Description

Using Windows version, I get this error while trying to download age restricted videos. It used to download properly till 2 days ago.

[Broken]CBS.COM ERROR: No video formats found

Checklist

  • [x ] I'm reporting a broken site support
  • [x ] I've verified that I'm running youtube-dlc version 2020.10.26
  • [x ] I've checked that all provided URLs are alive and playable in a browser
  • [x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • [x ] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.cbs.com/shows/bob-the-builder/video/9AF4F96F-6B2A-A0FF-F4A7-208A3B6154F1/bob-the-builder-pilchard-and-the-bear/']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.7.3 (CPython) - Linux-4.14.182-bbrplus-x86_64-with-debian-10.0
[debug] exe versions: ffmpeg 4.1.6-1, ffprobe 4.1.6-1
[debug] Proxy map: {}
[CBS] 9AF4F96F-6B2A-A0FF-F4A7-208A3B6154F1: Downloading XML
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dlc -U to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

download for this specific episode not working
https://www.cbs.com/shows/bob-the-builder/video/9AF4F96F-6B2A-A0FF-F4A7-208A3B6154F1/bob-the-builder-pilchard-and-the-bear/
however the other works just fine.
e.g https://www.cbs.com/shows/bob-the-builder/video/38CC8BCF-E327-4B42-8F0C-208A3B593ABB/bob-the-builder-drive-thru-disaster/ this ok.

[YouTube] Unable to extract JS player URL

Checklist

  • [x ] I'm reporting a broken site support issue
  • [x ] I've verified that I'm running youtube-dlc version 2020.10.24
  • [x ] I've checked that all provided URLs are alive and playable in a browser
  • [x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • [x ] I've searched the bugtracker for similar bug reports including closed ones
  • [x ] I've read bugs section in FAQ

Verbose log

[Download] Debug: [youtube] dX3k_QDnzHE: Downloading webpage
[Download] Debug: [youtube] dX3k_QDnzHE: Downloading embed webpage
[Download] Error: ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
[Download] Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dlc/extractor/youtube.py", line 2050, in _real_extract
    ASSETS_RE, embed_webpage, 'JS player URL')
  File "/usr/local/lib/python3.6/dist-packages/youtube_dlc/extractor/common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dlc.utils.RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

This issue is happening around 2-3% of the time when downloading some Videos. This is one of the Videos (https://youtu.be/dX3k_QDnzHE), I am using yt-dlc in python embedded and not as a command line tool. I tried to make it as verbose as possible. If you need more info, just let me know.

As this issue is only happening sometimes, I know this is extremely hard to reproduce and fix, but maybe other people are experiencing this issue too and can share their experience with said issue.

If you need any more info, let me know

[Feature Request] Code Signing Executables

Checklist

  • I'm reporting a feature request
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've searched the bugtracker for similar feature requests including closed ones

Description

While code signing is not a panacea, it would reduce the possibility of maliciously modified copies of the binaries, as well as reduce the probability for anti-malware utilities reporting a false positive.

[Question] Migrate to Codeberg

Checklist

  • [ x] I'm asking a question
  • [x ] I've looked through the README and FAQ for similar questions
  • [ x] I've searched the bugtracker for similar questions including closed ones

Question

I think that This project can be migrated to https://codeberg.org or other Open Source friendly Git hosting service.

Please fix auto login for Comcast/Xfinity users

The issue is that if you are an Xfinity customer and you try to use youtube-dl while on your home Xfinity connection (in other words you are not trying to log in through some other ISP) it will not allow you to login unless it completes its auto-login sequence. This 8 line fix allows that and has worked fine for a year and a half now, it was originally posted in a comment on a related issue on the other site, but for some inexplicable reason was never added to the main program. This is the output of the diff command comparing the original adobepass.py to the patched one, as you can see there are only 8 lines to be added (6 if you don't count the comments):

$ diff -u adobepass.py.original adobepass.py.patched
--- adobepass.py.original	2018-10-17 01:22:56.000000000 -0400
+++ adobepass.py.patched	2019-04-23 08:53:36.000000000 -0400
@@ -1433,6 +1433,14 @@
                             provider_redirect_page, 'oauth redirect')
                         self._download_webpage(
                             oauth_redirect_url, video_id, 'Confirming auto login')
+                    elif 'automatically signed in with' in provider_redirect_page:
+                        # Seems like comcast is rolling up new way of automatically signing customers
+                        oauth_redirect_url = self._html_search_regex(
+                            r'continue:\s*"(https://oauth.xfinity.com/oauth/authorize\?.+)"',
+                            provider_redirect_page, 'oauth redirect (signed)')
+                        # Just need to process the request. No useful data comes back
+                        self._download_webpage(
+                            oauth_redirect_url, video_id, 'Confirming auto login')
                     else:
                         if '<form name="signin"' in provider_redirect_page:
                             provider_login_page_res = provider_redirect_page_res

I should clarify that I did not write this patch, just copied it from a comment/submission on the original project.

[Broken]Tiktok seems to be broken

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.24
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

ytdlc -v https://www.tiktok.com/@charlidamelio/video/6854565358808993029
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.tiktok.com/@charlidamelio/video/6854565358808993029']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.24-5
[debug] Python version 2.7.16 (CPython) - Darwin-19.6.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[TikTok] 6854565358808993029: Downloading video webpage
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "./ytdlc/__main__.py", line 19, in <module>
  File "./ytdlc/youtube_dlc/__init__.py", line 487, in main
  File "./ytdlc/youtube_dlc/__init__.py", line 477, in _real_main
  File "./ytdlc/youtube_dlc/YoutubeDL.py", line 2096, in download
  File "./ytdlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
  File "./ytdlc/youtube_dlc/extractor/common.py", line 532, in extract
  File "./ytdlc/youtube_dlc/extractor/tiktok.py", line 137, in _real_extract
  File "./ytdlc/youtube_dlc/extractor/tiktok.py", line 22, in _extract_aweme
AttributeError: 'NoneType' object has no attribute 'get'

Description

https://www.tiktok.com/@charlidamelio/video/6854565358808993029

[Question] Rename the project to media-dl

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

I'd recommend renaming the project to media-dl. That will avoid any trademark issues with YouTube and is a more accurate description of youtube-dlc anyway since YouTube is just one of many sites it can rip from.

Not returning artist or song name of Youtube video (Python)

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

[youtube] P5ZJui3aPoQ: Downloading webpage
Track: None
Artist: None

Description

When using the YoutubeDL.extract_info, that returns None to both track and artist (it returns None to more things like album, release_date, release_year, etc).

As described in the solution here, apparently changing the User-Agent to the Facebook's one fixed the problems, but, for me, even changing stills returning None.

I believe it's a regex problem while scrapping the metadata, but don't have a clue how/where to fix.

Even videos that have this information given in the description, as Carry on Wayward Son - https://imgur.com/55N5k4l, it stills returning None to the YoutubeDL.extract_info.

Python Example for Carry on Wayward Son

YTDL_OPTIONS = {
        'ignoreerrors': False,
        'logtostderr': False,
        'quiet': False,
        'no_warnings': False,
    }

ytdl = ytb.YoutubeDL(YTDL_OPTIONS)

example_url = "https://www.youtube.com/watch?v=P5ZJui3aPoQ"
video = ytdl.extract_info(example_url, download=False)

print('Track: ' + video['track'])
print('Artist: ' + video['artist'])

Output

[youtube] P5ZJui3aPoQ: Downloading webpage
Track: None
Artist: None

Anyway, it seems to works well with videos that have the description generated by the Youtube, like this one

Python Example for the link above

YTDL_OPTIONS = {
        'ignoreerrors': False,
        'logtostderr': False,
        'quiet': False,
        'no_warnings': False,
    }

ytdl = ytb.YoutubeDL(YTDL_OPTIONS)

example_url = "https://www.youtube.com/watch?v=QBxSQXbj6Go"
video = ytdl.extract_info(example_url, download=False)

print('Track: ' + video['track'])
print('Artist: ' + video['artist'])

Output

[youtube] QBxSQXbj6Go: Downloading webpage
Track: Keep Pushin'
Artist: REO Speedwagon

Remove -U option

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Description

The readme states

DO NOT UPDATE using -U ! instead download binaries again or when installed with pip use a described above when installing.
I will add some memorable short links to the binaries so you can download them easier.

So what about removing the -U flag completely? Or change to print the update information.

If you want this done I can make a PR this.

[Question] SoundCloud Go+ HQ downloads

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

Does this fork have ability to download HQ tracks from SoundCloud with Go+ subscription using just login and password?

YouTube-DLC_x86: Error loading python35.dll

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

PASTE VERBOSE LOG HERE

Error loading Python DLL 'C:\DOCUME1\Owner\LOCALS1\Temp_MEI33442\python35.dll'.
LoadLibrary: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.

Description

WRITE DESCRIPTION HERE
I know I'm the exception rather than the rule, but I tried to run YouTube-DLC_x86 on my Windows XP system and got the above error. The original YouTube-DL works fine for me, but YouTube-DLC_x86 won't even run because it's looking for a python36.dll, which I don't have.

Unable to extract JS player URL

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

youtube-dlc -v https://www.youtube.com/watch?v=rNlhTKTHgpo
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=rNlhTKTHgpo']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[youtube] rNlhTKTHgpo: Downloading webpage
[youtube] rNlhTKTHgpo: Downloading embed webpage
ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\youtube.py", line 2065, in _real_extract
  File "youtube_dlc\extractor\common.py", line 1010, in _search_regex
youtube_dlc.utils.RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

Unable to extract JS player URL error

[Broken]YT playlist - collected 0 ids

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

ytdlc -v -F https://youtu.be/-gJaYKwreEc?list=RD-gJaYKwreEc
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://youtu.be/-gJaYKwreEc?list=RD-gJaYKwreEc']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.09.20
[debug] Python version 2.7.16 (CPython) - Darwin-19.6.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube:playlist] Downloading playlist RD-gJaYKwreEc - add --no-playlist to just download video -gJaYKwreEc
[youtube:playlist] RD-gJaYKwreEc: Downloading page 1 of Youtube mix
[download] Downloading playlist: RD-gJaYKwreEc
[youtube:playlist] playlist RD-gJaYKwreEc: Collected 0 video ids (downloading 0 of them)
[download] Finished downloading playlist: RD-gJaYKwreEc

Description

test links:
https://www.youtube.com/watch?v=-gJaYKwreEc&feature=youtu.be&list=RD-gJaYKwreEc

https://youtu.be/-gJaYKwreEc?list=RD-gJaYKwreEc

Tried some other playlists and they seem to be fine.

Assume this could be a formatting problem of the initial link.
Opened that channel, navigated to playlists and took this link for that very same playlist wanted initially:
https://www.youtube.com/watch?v=-gJaYKwreEc&list=PLF-k_OIEn_PMQ4bYJGD1Jx1OUAM7zwaEH
It works.
Please take a look and in case this is just a link formatting problem, just close this issue.

Some downloads fail with KeyError: 'url'

Verbose log

Y:\YouTube Channels\NonCompete>youtube-dl --cookies "Y:\cookies.txt" --add-metadata --write-description --write-info-json --write-thumbnail --download-archive "Y:\ytdl_archive.txt" --all-subs --sleep-interval 1 -ciw -o "%(title)s.%(ext)s" -v https://www.youtube.com/channel/UCkZFKKK-0YB0FvwoS8P7nHg
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'Y:\\cookies.txt', '--add-metadata', '--write-description', '--write-info-json', '--write-thumbnail', '--download-archive', 'Y:\\ytdl_archive.txt', '--all-subs', '--sleep-interval', '1', '-ciw', '-o', '%(title)s.%(ext)s', '-v', 'https://www.youtube.com/channel/UCkZFKKK-0YB0FvwoS8P7nHg']
[debug] Loading archive file 'Y:\\ytdl_archive.txt'
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg git-2020-04-15-51db0a4, ffprobe 4.3.1-2020-10-01-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube:channel] UCkZFKKK-0YB0FvwoS8P7nHg: Downloading channel page
[youtube:playlist] UUkZFKKK-0YB0FvwoS8P7nHg: Downloading webpage
[download] Downloading playlist: Uploads from NonCompete
[youtube:playlist] UUkZFKKK-0YB0FvwoS8P7nHg: Downloading page #1
[youtube:playlist] playlist Uploads from NonCompete: Downloading 159 videos
[download] Downloading video 1 of 159
[youtube] T78IcPvnBtQ: Downloading webpage
[debug] Default format spec: bestvideo+bestaudio/best
[info] Video description is already present
[info] Writing video subtitles to: Viva South America! El Breadcast!.live_chat.json
ERROR: 'url'
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 841, in extract_info
  File "youtube_dlc\YoutubeDL.py", line 896, in process_ie_result
  File "youtube_dlc\YoutubeDL.py", line 1691, in process_video_result
  File "youtube_dlc\YoutubeDL.py", line 1893, in process_info
KeyError: 'url'

Description

For some videos, I'm seeing failures with KeyError: 'url' (and this time I've actually confirmed the videos work on the site). This is with the latest release version.

[Question] How do i get the Video Title (aka the default output filename) as a variable in Python?

Checklist

  • [x ] I'm asking a question
  • [x ] I've looked through the README and FAQ for similar questions
  • [x ] I've searched the bugtracker for similar questions including closed ones

I want to download a video and give the user the option to keep the video for later, but for further processing i need to somehow store the output file name as a variable, i cant find any way to do so, and i dont want to make a big workaround by using os to check the file manually

EDIT: I am aware of the extract_info option, but this wont give me the literal file name, as it ommits e.g. : and turns it into -

[Broken] Newgrounds

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

D:\`user`\Documents\Youtube-DL>youtube-dlc https://www.newgrounds.com/portal/view/769705 -v
[debug] System config: []
[debug] User config: ['-o', 'F:/Videos/Downloaded/%(title)s.%(ext)s', '--write-sub', '--write-auto-sub']
[debug] Custom config: []
[debug] Command-line args: ['https://www.newgrounds.com/portal/view/769705', '-v']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg git-2020-04-03-52523b6, ffprobe git-2020-04-03-52523b6
[debug] Proxy map: {}
[Newgrounds] 769705: Downloading webpage
ERROR: Unable to extract ; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\newgrounds.py", line 64, in _real_extract
  File "youtube_dlc\extractor\common.py", line 1010, in _search_regex
youtube_dlc.utils.RegexNotFoundError: Unable to extract ; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

fails to download from newgrounds. using a random video for this example but have tried on multiple.

PLEASE LET ME KNOW IF YOU NEED MORE INFO

[Feature Request] How to host yt-dlc offshore without risk of it getting taken down

Checklist

  • I'm reporting a feature request
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've searched the bugtracker for similar feature requests including closed ones

I have put together a guide on how to host yt-dlc offshore using companies that have a track record of being very resilient to DMCA takedowns: https://old.reddit.com/r/youtubedl/comments/jktx5b/how_to_anonymously_host_the_continued_development/
I won't copy paste the whole guide onto this issue ticket since it might be better to just link to it off-site and have further away from the development of the software.

[Question] Accuracy of VirusTotal report?

Checklist

FAQ, etc. down due to suppression of FOSS.

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

Is this report from VirusTotal accurate (https://www.virustotal.com/gui/file/763744714819c4a63ecda89e869eab4767e8d4304d544ff000298d267d198646/detection)
? The previous release did not have this issue.

Release scanned: youtube-dlc 2020.10.26 (youtube-dlc.exe). It looks like a false positive to me, but surprised to see it.

[Question]

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

I normally would update via Homebrew and would install using these instructions: https://formulae.brew.sh/formula/youtube-dl

Is the new fork will homebrew accessible? is there a formulae for yt-dlc available?

Confusing URLs pointing to yt-dl.org

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Description

For example, when program fails to extract rating, it returns:

WARNING: unable to extract rating; please report this issue on https://yt-dl.org/bug .

Link points to youtube-dl bug tracker, not youtube-dlc one.
That may confuse some people.

[Broken] Unable to extract JS player URL

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones
$ youtube-dlc -g -v https://youtu.be/QpQvJExilsk

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-g', '-v', 'https://youtu.be/QpQvJExilsk']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.5 (CPython) - Windows-10-10.0.17763-SP0
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\youtube_dlc\YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "c:\python38\lib\site-packages\youtube_dlc\extractor\common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "c:\python38\lib\site-packages\youtube_dlc\extractor\youtube.py", line 2065, in _real_extract
    jsplayer_url_json = self._search_regex(
  File "c:\python38\lib\site-packages\youtube_dlc\extractor\common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dlc.utils.RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

This started occurring about two days ago and happens the majority of the time when trying to get the URL for a video.

[Broken] Bandcamp again.

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dlc -v https://artikalmusic.bandcamp.com/album/far-north-dubs-ep
[debug] System config: []
[debug] User config: ['-o', '/dev/shm/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1080][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', '--no-mtime', '--add-metadata', '--embed-thumbnail']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://artikalmusic.bandcamp.com/album/far-north-dubs-ep']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.25
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-122-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg N-99430-g9d8f9b2-Nico_20201001, ffprobe N-99430-g9d8f9b2-Nico_20201001
[debug] Proxy map: {}
[Bandcamp:album] far-north-dubs-ep: Downloading webpage
ERROR: The page doesn't contain any tracks; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/bandcamp.py", line 311, in _real_extract
    raise ExtractorError('The page doesn\'t contain any tracks')
youtube_dlc.utils.ExtractorError: The page doesn't contain any tracks; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

As a user on Reddit experienced, Bandcamp stopped working again.
Doesn't matter what album.

ERROR: Unable to extract JS player URL

When downloading the video getting this error,

ERROR: C0DPdy98e4c: YouTube said: Unable to extract video data
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/youtube_dlc/YoutubeDL.py", line 830, in extract_info
ie_result = ie.extract(url)
File "/usr/local/lib/python3.8/site-packages/youtube_dlc/extractor/common.py", line 532, in extract
ie_result = self._real_extract(url)
File "/usr/local/lib/python3.8/site-packages/youtube_dlc/extractor/youtube.py", line 1865, in _real_extract
raise ExtractorError(
youtube_dlc.utils.ExtractorError: C0DPdy98e4c: YouTube said: Unable to extract video data

After retry..no error.
This issue happening randomly.

Site now uses go extractor instead of its own, partial fix was posted but still doesn't fully work, can it be fixed?

I had posted this in the discussion area of the other project just a few days ago and wanted to get it here while I still had it in TextArea Cache (a Firefox addon that comes in REALLY handy at times). Please note that I am NOT the author of the patch discussed below so I can't really answer any questions about it:

I am waiting for blackjack4494/youtube-dlc#124 to be resolved before I install youtube-dlc but in the meantime I wanted to refer you to the patches in https://github.com/ytdl-org/youtube-dl/pull/26826/files although they do not fully fix the issue. The patches are as follows (these are to the original, not necessarily your fork, and they are not mine, I am just referencing them):

In youtube_dl/extractor/extractors.py:

Remove the line from .fxnetworks import FXNetworksIE at or near line 397.

Remove the file youtube_dl/extractor/fxnetworks.py completely.

And in the file youtube_dl/extractor/go.py:

@@ -38,13 +38,17 @@ class GoIE(AdobePassIE):
        'disneynow': {
            'brand': '011',
            'resource_id': 'Disney',
-        }
+        },
+        'fxnow.fxnetworks': {
+            'brand': '025',
+            'requestor_id': 'dtci',
+        },
    }
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
                            (?:(?P<sub_domain>%s)\.)?go|
-                            (?P<sub_domain_2>abc|freeform|disneynow)
+                            (?P<sub_domain_2>abc|freeform|disneynow|fxnow\.fxnetworks)
                        )\.com/
                        (?:
                            (?:[^/]+/)*(?P<id>[Vv][Dd][Kk][Aa]\w+)|
@@ -99,6 +103,19 @@ class GoIE(AdobePassIE):
            # m3u8 download
            'skip_download': True,
        },
+    }, {
+        'url': 'https://fxnow.fxnetworks.com/shows/better-things/video/vdka12782841',
+        'info_dict': {
+            'id': 'VDKA12782841',
+            'ext': 'mp4',
+            'title': 'First Look: Better Things - Season 2',
+            'description': 'md5:fa73584a95761c605d9d54904e35b407',
+        },
+        'params': {
+            'geo_bypass_ip_block': '3.244.239.0/24',
+            # m3u8 download
+            'skip_download': True,
+        },
    }, {
        'url': 'http://abc.go.com/shows/the-catch/episode-guide/season-01/10-the-wedding',
        'only_matching': True,

I apologize that this is not proper patch file format, I don't know how you get one from github so I had to manually type the + and - symbols at the start of the lines. Anyway this all came from ytdl-org/youtube-dl#26826 but the problem is that it doesn't work on things that require a login. As I said there,

I very much appreciate the effort here, and using your example URL it does work, but I find that with these patches installed I still get an error on other types of URL's: ERROR: Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: 'Unauthorized'>); please report this issue on https://yt-dl.org/bug ... It seems to affect episodes picked from the page at https://www.fxnetworks.com/ that include season and episode numbers in the URL. For example the first episode on that page tonight was https://fxnow.fxnetworks.com/shows/a-wilderness-of-error/episode-guide/season-01/episode-05-a-wilderness-of-error/vdka20447879 and that doesn't work. Your example in your code follows the format:

https://fxnow.fxnetworks.com/shows/show-name/video/vdka########

What doesn't work is

https://fxnow.fxnetworks.com/shows/show-name/episode-guide/season-0#/episode-0#-episode-title/vdka########

(the #'s represent digits)

The guy that wrote the patch said the issue was that I was not supplying credentials, but in fact I was, and other Go services that require credentials work fine. My guess is it just isn't parsing the second form of the URL correctly but I don't know for sure. If there is some way you can figure out why his patch only works for videos that don't require a provider login it would be much appreciated, but also I have a feeling that even his original patches will not make it into the main distribution any time soon, so it would be helpful if you could get them into yours if there's not some reason not to.

[Youtube] Extraction of subs fails for certain videos, crashes process

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones
    Since bugtracker is quite new, and the old one is under DMCA, I'm unsure if this was reported before.

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: [u'-f', u'bestvideo[ext=webm]+bestaudio[ext=webm]/bestvideo+bestaudio', u'-i', u'-o', u'%(uploader)s (%(uploader_id)s)/%(upload_date)s - %(title)s - [%(resolution)s] [%(id)s].%(ext)s', u'--ffmpeg-location', u'./ffmpeg', u'--download-archive', u'youtube-dl-archive.txt', u'-a', u'youtube-dl-channels.txt', u'--prefer-ffmpeg', u'--merge-output-format', u'mkv', u'--write-sub', u'--all-subs', u'--convert-subs', u'srt', u'--add-metadata', u'--write-description', u'--write-thumbnail', u'-v']
[debug] Command-line args: [u'--config-location', u'./youtube-dl.conf']
[debug] Batch file urls: [u'https://www.youtube.com/channel/UCxLYtICsUCWdr1YPrj5DtwA']
[debug] Loading archive file u'youtube-dl-archive.txt'                                                               
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8                                                     
[debug] youtube-dlc version 2020.10.26                                                                               
[debug] Python version 2.7.16 (CPython) - Linux-5.6.0-0.bpo.2-amd64-x86_64-with-debian-10.6                          
[debug] exe versions: ffmpeg N-45774-g223f3dff8-static                                                               
[debug] Proxy map: {}
[youtube:playlist] UUxLYtICsUCWdr1YPrj5DtwA: Downloading webpage
[download] Downloading playlist: Uploads from Ordinary Things
[youtube:playlist] playlist Uploads from Ordinary Things: Downloading 36 videos
[download] Downloading video 1 of 36
[download] The Evil Economics of Tinder has already been recorded in archive
[download] Downloading video 2 of 36
[download] Handshakes Deserve to Die, Here's Why has already been recorded in archive
[download] Downloading video 3 of 36
[download] Why the Internet Hates Nestlé has already been recorded in archive
[download] Downloading video 4 of 36
[download] How to WIN an Election | An Ordinary Guide has already been recorded in archive
[download] Downloading video 5 of 36
[download] The Bloody Business of the British Flag has already been recorded in archive
[download] Downloading video 6 of 36
[download] The Tasteless History of Ronald McDonald has already been recorded in archive
[download] Downloading video 7 of 36
[download] How Nintendo Killed the GameCube has already been recorded in archive
[download] Downloading video 8 of 36
[download] The Great Egg Industry InfoWar has already been recorded in archive
[download] Downloading video 9 of 36
[download] How the USSR Collapsed on Soviet TV has already been recorded in archive
[download] Downloading video 10 of 36
[download] The Billion-Dollar Bitcoin Scam has already been recorded in archive
[download] Downloading video 11 of 36
[download] The Bizarre Business of Trading Cards has already been recorded in archive
[download] Downloading video 12 of 36
[download] The Rise & Fall of American Malls has already been recorded in archive
[download] Downloading video 13 of 36
[download] Unmasking The History of Masks has already been recorded in archive
[download] Downloading video 14 of 36
[download] The Toxic Business of Cruise Ships has already been recorded in archive
[download] Downloading video 15 of 36
[info] Writing video description to: Ordinary Things (UCxLYtICsUCWdr1YPrj5DtwA)/20200320 - The Desperate History of Panic Buying - [1920x1080] [ZX4vseaNJKY].description
[info] Writing video subtitles to: Ordinary Things (UCxLYtICsUCWdr1YPrj5DtwA)/20200320 - The Desperate History of Panic Buying - [1920x1080] [ZX4vseaNJKY].live_chat.json
ERROR: u'url'
Traceback (most recent call last):
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 841, in extract_info                      
    return self.process_ie_result(ie_result, download, extra_info)                                                   
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 896, in process_ie_result                 
    return self.process_video_result(ie_result, download=download)                                                   
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 1691, in process_video_result             
    self.process_info(new_info)
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 1893, in process_info                     
    sub_info['url'], info_dict['id'], note=False).read()
KeyError: u'url'

[download] Downloading video 16 of 36
[youtube] 9k6sbhyOD5o: Downloading webpage
[youtube] 9k6sbhyOD5o: Downloading embed webpage
[youtube] 9k6sbhyOD5o: Refetching age-gated info webpage
[info] Writing video description to: Ordinary Things (UCxLYtICsUCWdr1YPrj5DtwA)/20200306 - The (Ob)Noxious Rise of Vapes - [1920x1080] [9k6sbhyOD5o].description
[info] Writing video subtitles to: Ordinary Things (UCxLYtICsUCWdr1YPrj5DtwA)/20200306 - The (Ob)Noxious Rise of Vapes - [1920x1080] [9k6sbhyOD5o].live_chat.json
ERROR: u'url'
Traceback (most recent call last):
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 841, in extract_info                      
    return self.process_ie_result(ie_result, download, extra_info)                                                   
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 896, in process_ie_result                 
    return self.process_video_result(ie_result, download=download)                                                   
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 1691, in process_video_result             
    self.process_info(new_info)
  File "/home/user/.local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 1893, in process_info                     
    sub_info['url'], info_dict['id'], note=False).read()
KeyError: u'url'

^C
ERROR: Interrupted by user

Description

youtube-dlc seems to fail grabbing subtitles for videos, even if there aren't any, and is instead grabbing live_chat.json files, which don't exist for these videos.
This in turn crashes the video ID grab process, which skips grabbing the video all together.

If I comment out these subs lines in my .conf, it grabs video and description without issue. It of course does not grab subs.

# Get All Subs to SRT
--write-sub
--all-subs
--convert-subs srt

youtube-dl 2020.09.20 fails to parse or even download the videos.
Thanks!

[Question] Consider migration from GitHub

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

Have you considered migrating from GitHub to prevent this entire situation from happening again?

A mirror is now available at https://git.kavin.rocks/kavin/yt-dlc
Tweet regarding the CEO trying to bring back youtube-dl: https://nitter.kavin.rocks/t3rr4dice/status/1320660235363749888

vlive have change url format

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

$ youtube-dlc https://www.vlive.tv/post/0-19646784 -s -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.vlive.tv/post/0-19646784', '-s', '-v']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg N-99793-gae9a1a9698, ffprobe N-99793-gae9a1a9698
[debug] Proxy map: {}
[generic] 0-19646784: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 0-19646784: Downloading webpage
[generic] 0-19646784: Extracting information
ERROR: Unsupported URL: https://www.vlive.tv/post/0-19646784
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\generic.py", line 3382, in _real_extract
youtube_dlc.utils.UnsupportedError: Unsupported URL: https://www.vlive.tv/post/0-19646784

Description

Vlive now change their URL format (the old format is http://www.vlive.tv/video/0000000 or https://channels.vlive.tv/xxxx/video)
New format:

Single video: https://www.vlive.tv/post/1-19369327
Channel: https://www.vlive.tv/channel/FE619/board/3498

[Broken] Youtube download not working

Checklist

  • [ x] I'm reporting a broken site support
  • [ x] I've verified that I'm running youtube-dlc version 2020.10.26
  • [ x] I've checked that all provided URLs are alive and playable in a browser
  • [ x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • [ x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=HRHVJPFKCqE']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.26
[debug] Python version 2.7.18 (CPython) - Linux-5.4.0-52-generic-x86_64-with-Ubuntu-20.04-focal
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Proxy map: {}
[youtube] HRHVJPFKCqE: Downloading webpage
[youtube] HRHVJPFKCqE: Downloading embed webpage
ERROR: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1966, in _real_extract
    yt_initial_data['contents']['twoColumnWatchNextResults']['conversationBar']['liveChatRenderer']['continuations'][0]['reloadContinuationData']['continuation']
KeyError: u'conversationBar'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 2066, in _real_extract
    ASSETS_RE, embed_webpage, 'JS player URL')
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 1010, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract JS player URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

Im simply trying to download a youtube video. Every video that I try including the one mentioned above returns with the stated error. Some other options work like --write-sub or --list-formats but download of the video fails. I had youtube-dl previously installed. Is there some cache I need to clear or is the issue something else? Thank you very much for your help!

[Feature Request] NotABug Repository

Checklist

  • I'm reporting a feature request
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've searched the bugtracker for similar feature requests including closed ones

Description

I'm afraid of projects like these being taken down again, so I suggest this project be either mirrored or moved to another site such as NotABug
https://notabug.org/

[Broken] Niconico - Long videos fail downloading

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dlc --ignore-config --verbose 'https://www.nicovideo.jp/watch/sm29303999'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--ignore-config', u'--verbose', u'https://www.nicovideo.jp/watch/sm29303999']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.26
[debug] Python version 2.7.18 (CPython) - Linux-5.4.0-52-generic-x86_64-with-Ubuntu-20.04-focal
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Proxy map: {}
[niconico] sm29303999: Downloading webpage
[niconico] sm29303999: Downloading JSON metadata for h264_360p-aac_64kbps
[niconico] sm29303999: Downloading JSON metadata for h264_360p_low-aac_64kbps
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://pe03821af87.dmc.nico/vod/ht2_nicovideo/nicovideo-sm29303999_d12af73024e45c5410966649f9601391bdf4129f978a797c374c3bb3ce252d22?ht2_nicovideo=6-N0wawdNOqC_1604121810978.4nohp4qioy_qj1wb7_20e41xpuox7ii'
[download] Destination: 【東方】幻想万華鏡 第7話・巨大妖怪伝説の章【二次創作アニメ】-sm29303999.mp4
[download]  23.8% of 58.16MiB at 120.34KiB/s ETA 06:17[download] Got server HTTP error: Downloaded 14524539 bytes, expected 60982677 bytes. Retrying (attempt 1 of 10)...
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dlc/YoutubeDL.py", line 2003, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dlc/YoutubeDL.py", line 1861, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dlc/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dlc/downloader/http.py", line 349, in real_download
    establish_connection()
  File "/usr/local/bin/youtube-dl/youtube_dlc/downloader/http.py", line 114, in establish_connection
    raise err
HTTPError: HTTP Error 403: Forbidden

Description

Downloading long videos requires heartbeat signals. Here's the description found in the original issue on youtube-dl's repo:

As of today, Nicovideo's seem to implemented a new session system. After every few minutes, the url of the video changes. This causes a 403 Forbidden error on the old url, Causing the Downloader to fail.
A screenshot of part of a video session:
chrome 2017-10-25 10 43 35 396

Resuming the download works just for a few minutes before the session is dropped.
Note: I have only tested on a non-premium account. Im not sure if it applies to the permium accounts or not.

URL example: https://www.nicovideo.jp/watch/sm29303999

[Broken] PBS - Some subtitles not detected

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

:\>c:\support\youtube-dlc.exe --list-subs https://www.pbs.org/video/worlds-fastest-animal-8w5tdp/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--list-subs', 'https://www.pbs.org/video/worlds-fastest-animal-8w5tdp/', '--verbose']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.24-6
[debug] Python version 3.8.6 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg N-92986-gaceb9131c1, ffprobe N-92986-gaceb9131c1, rtmpdump 2.4
[debug] Proxy map: {}
[debug] Using fake IP 6.56.221.25 (US) as X-Forwarded-For.
[pbs] Downloading JSON metadata
[pbs] worlds-fastest-animal-8w5tdp: Downloading webpage
[pbs] worlds-fastest-animal-8w5tdp: Downloading widget/partnerplayer page
[pbs] worlds-fastest-animal-8w5tdp: Downloading portalplayer page
[pbs] worlds-fastest-animal-8w5tdp: Downloading 0 video url info
[pbs] worlds-fastest-animal-8w5tdp: Downloading m3u8 information
[pbs] worlds-fastest-animal-8w5tdp: Downloading 1 video url info
[pbs] worlds-fastest-animal-8w5tdp: Checking http-2000k video URL
[pbs] worlds-fastest-animal-8w5tdp: http-2000k video URL is invalid, skipping
[pbs] worlds-fastest-animal-8w5tdp: Checking http-6500k video URL
[pbs] worlds-fastest-animal-8w5tdp: http-6500k video URL is invalid, skipping
[pbs] worlds-fastest-animal-8w5tdp: Checking http-4500k video URL
[pbs] worlds-fastest-animal-8w5tdp: http-4500k video URL is invalid, skipping
[pbs] worlds-fastest-animal-8w5tdp: Checking http-3000k video URL
[pbs] worlds-fastest-animal-8w5tdp: Checking http-1100k video URL
[pbs] worlds-fastest-animal-8w5tdp: http-1100k video URL is invalid, skipping
[pbs] worlds-fastest-animal-8w5tdp: Checking http-730k video URL
[pbs] worlds-fastest-animal-8w5tdp: http-730k video URL is invalid, skipping
3018507559 has no subtitles

Description

Some videos are not picking up VTT subtitles.

Video URL: https://www.pbs.org/video/worlds-fastest-animal-8w5tdp

Subtitle URL: https://ga.video.cdn.pbs.org/captions/nova/ed371369-2d0a-4442-9f64-700ed52cdc9e/captions/yFtOv5_caption.vtt

[Broken]Facebook private (friends only and private groups) error handling response is broken

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

./testdlc -v -F https://www.facebook.com/100002659934141/videos/3355692847862680/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.facebook.com/100002659934141/videos/3355692847862680/']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.10.25
[debug] Python version 2.7.16 (CPython) - Darwin-19.6.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[facebook] 3355692847862680: Downloading webpage
[facebook] 3355692847862680: Downloading webpage
[facebook] 3355692847862680: Downloading webpage
ERROR: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./testdlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "./testdlc/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "./testdlc/youtube_dlc/extractor/facebook.py", line 484, in _real_extract
    video_id, fatal_if_no_video=True)
  File "./testdlc/youtube_dlc/extractor/facebook.py", line 380, in _extract_from_url
    raise ExtractorError('Cannot parse data')
ExtractorError: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

Description

Private videos used to work fine until lately. I assume the server response from FB changed or something. As a result, ytdlc does not how to handle it and ask for cookies or login, for example.

ERROR: This playlist does not exist for watchlater list

youtube-dlc --version
2020.10.26

config in use:
--cookies=~/.config/youtube-dl/cookies-youtube-com.txt
--no-check-certificate
--ignore-errors
--format best[ext=mp4]
--no-overwrites
--add-metadata
--output 's1e%(playlist_index)s %(title)s.%(ext)s'
--download-archive ~/.config/youtube-dl/archive.txt

Do I need a new cookie file?

[Broken] Youtube sometimes fails with "Unable to extract video data" for the same video.

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: [] 
[debug] User config: []
[debug] Custom config: [] 
[debug] Command-line args: [u'--no-warnings', u'--dump-json', u'--verbose', u'--id', u'--', u'6-8E4Nirh9s'] 
[debug] Loading archive file None 
[debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8 
[debug] youtube-dlc version 2020.10.25 [debug] Python version 2.7.16 (CPython) - Linux-4.19.0-10-amd64-x86_64-with-debian-10.6 
[debug] exe versions: ffmpeg 4.1.6-1, ffprobe 4.1.6-1 
[debug] Proxy map: {} ERROR: 6-8E4Nirh9s: YouTube said: Unable to extract video data
Traceback (most recent call last): 
File "./youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info ie_result = ie.extract(url) 
File "./youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract ie_result = self._real_extract(url) 
File "./youtube-dlc/youtube_dlc/extractor/youtube.py", line 1866, in _real_extract 'YouTube said: %s' % unavailable_message, expected=True, video_id=video_id) 
ExtractorError: 6-8E4Nirh9s: YouTube said: Unable to extract video data

Description

Currently, about 3% of video analysis for youtube fails with "Unable to extract video data", even when always using the same URL. The problem ist that sometimes Youtube delivers a player page which does not contain the expected "args": "player_response" object (see youtube.py - _real_extract()).

I did not have time to investigate further today, therefore I open this issue for others to investigate. I will update this if I find out more, if someone else finds something, please add to this issue :)

I attached a working and a non-working reponse to this issue for further investigation.
responses.zip

[Broken] ABC iview

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.24
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: ['-f', 'best', '--merge-output-format', 'mkv']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://iview.abc.net.au/show/bluey/series/0/video/CH1903Q042S00']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2020.09.20
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763
[debug] exe versions: ffmpeg 2020-10-21-git-289e964873-full_build-www.gyan.dev, ffprobe 2020-10-21-git-289e964873-full_build-www.gyan.dev
[debug] Proxy map: {}
[debug] Using fake IP 1.152.104.99 (AU) as X-Forwarded-For.
[abc.net.au:iview] CH1903Q042S00: Downloading JSON metadata
[abc.net.au:iview] CH1903Q042S00: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 401: Unauthorized (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpksi3o1r1\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpksi3o1r1\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

All attempts to download content from ABC iview fail with the above error. This was working yesterday, but is failing today. I can confirm that videos are still playable in the browser (using an Australian-based VPN server).

[Broken] "Unable to extract video data" when using cookies on YouTube

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dlc version 2020.10.26
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

C:\youtube-dlc>youtube-dlc -v --cookies cookies.txt https://www.youtube.com/watch?v=JR4KHfqw-oE
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--cookies', 'cookies.txt', 'https://www.youtube.com/watch?v=JR4KHfqw-oE']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.10.26
[debug] Python version 3.8.6 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 4.3.1-2020-10-01-full_build-www.gyan.dev, ffprobe 4.3.1-2020-10-01-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube] JR4KHfqw-oE: Downloading webpage
ERROR: JR4KHfqw-oE: YouTube said: Unable to extract video data
Traceback (most recent call last):
  File "youtube_dlc\YoutubeDL.py", line 830, in extract_info
  File "youtube_dlc\extractor\common.py", line 532, in extract
  File "youtube_dlc\extractor\youtube.py", line 1865, in _real_extract
youtube_dlc.utils.ExtractorError: JR4KHfqw-oE: YouTube said: Unable to extract video data

Description

As of October 20th, using --cookies no longer works correctly with my YouTube account. No matter if the video is public or members-only, I still get the same error. While I can just omit this option to download public videos, this becomes a problem when I want to download members-only videos. Between the time of it working and not working, nothing with my setup was changed in any way which leads me to believe this is something YouTube changed on the back-end.

I have signed out and back in to YouTube multiple times, have generated new cookies files multiple times, even tried on another machine but still receive the same error. There was another user who submitted this exact issue back on the old repository but unfortunately I didn't think to make an archive to use for reference.

What adds to the frustration is that this only seems to be happening selectively. When asked, most have reported no recent problems while a few users I've found online seem to have the exact issue as described here.

[Question] Unit Tests failing, should we be fixing them?

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

While testing code changes, I noticed many of the unit tests are failing either due to unsupported URLs, timeouts or extractor errors. These are things we should be correcting, right? I didn't want to start down the path of looking at them if we don't intend on fixing them :-)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.