GithubHelp home page GithubHelp logo

gatecrasher777 / ytcog Goto Github PK

View Code? Open in Web Editor NEW
70.0 6.0 13.0 221 KB

YouTube innertube class library for node-js; session, searches, channels, playlist, videos, comments and downloads.

License: MIT License

JavaScript 100.00%
youtube video objects channels searches downloads node-js innertubeapi innertube youtube-dl

ytcog's Introduction

ytcog

NPM GitHub release (latest SemVer)
YouTube innertube class library for node-js; session, searches, channels, playlists, videos, comments and downloads.

Features

  • Simple, efficient, fast, powerful.
  • No Google developer key required.
  • The innertube api is what the YouTube website itself uses to efficiently deliver search, channel and video information (json only).
  • The downloader is a forked process allowing for concurrent non-blocking, high quality downloads.

Classes

  • Session - manage your Youtube session/player - deciphering, encoding and hashing - enables seemless search, channel, playlist, video and download requests.
  • Search - fetch videos, playlists and channels from specific search requests.
  • Channel - fetch metadata, videos, playlists, associated channels or search specific channels.
  • Playlist - fetch videos from specific playlists
  • Video - fetch metadata and stream information deciphered/encoded to avoid throttling - ensure reliable and fast downloads.
  • Comment - helper class for the download and management of video comments and threads.
  • Download - a convenience class for easy once-off, sessionless, downloads.

See the wiki for greater detail.

Basic Usage

Easy downloader

const ytcog = require('ytcog');
await ytcog.dl(videoOptions[, cookie, userAgent, proxy, debug]);

videoOptions (object) See the wiki for all videoOptions.

cookie (string) is optional. With a cookie, everything will work. Without it, age-restricted video streams will not be retrieved and there might be some rate-limiting (although none reported so far)

userAgent (string) is optional. Since ytcog emulates a browser session, you can make all requests use your browser's user agent. If you supply a userAgent, you must supply a cookie, even if it is an empty string.

proxy (string) is optional. Provide a proxy agent string for all session https requests, i.e:
await ytcog.dl({id:'5qwDrjTinMk'},'','','http://127.0.0.1:8000');

debug (boolean) if true debug information is sent to the console.

NB: If you are downloading multiple videos (i.e. from search results, playlists or channels) then maintianing a session and using video.download() is much more efficient than running ytcog.dl() on each video.

Session

const ytcog = require('ytcog');
const session = new ytcog.Session([cookie, userAgent, proxy]);
await session.fetch();
console.log(`session status: ${session.status}`);

cookie and userAgent are optional, but in order to obtain them log onto YouTube in your browser. Goto settings > ... > developer tools. Refresh the page. Goto network>headers. Find the "www.youtube.com" entry. In the request headers you will find "cookie" and "user-agent". Pass these string values into your ytcog sessions. If you supply a userAgent, you must supply a cookie, even if it is an empty string.

proxy (string) is optional. Provide a proxt agent string for all session https requests, i.e:
let session = new ytcog.session('','','http://127.0.0.1:8000')

A session object is required to create search, channel, playlist and video objects.

Search

const search = new ytcog.Search(session, searchOptions);
await search.fetch();

session (Object) the session object, see above.

searchOptions (Object) See the wiki for all search options.

Search again with different options:

await search.fetch({items: 'videos', period:'year', order: 'views', features: 'hd', quantity: 500 });

Examine the results in an array of Video objects:

search.videos.forEach((video)=>{
    // do something with the results, like collect and display their streams
    await video.fetch();
    console.log(video.info());
    console.log(video.streamInfo);
});

Also search for playlists, channels and movies that match your search term

await search.fetch({items:'playlists'});
await search.fetch({items:'channels'});
await search.fetch({items:'movies'});

Iterate through the current results with:

search.results.forEach((item)=>{});

Or the accumulated results

search.playlists.forEach((playlist)=>{...});
search.channels.forEach((channel)=>{...});
search.videos.forEach((video)=>{...});

Channel

const channel = new ytcog.Channel(session, channelOptions);
await channel.fetch();

channelOptions See wiki for all channel options.

Get channel playlists

await channel.fetch({items: 'playlists', order: 'updated', quantity: 90});

Get associated channels

await channel.fetch({items: 'channels'});

Search a channel

await channel.fetch({items: 'search', query: 'vlogs'});

Iterate through the results with:

channel.results.forEach((item)=>{});  //current
channel.videos.forEach((video)=>{...}); //accumulated
channel.playlists.forEach((playlist)=>{...}); //accumulated
channel.channels.forEach((chan)=>{...}); //accumulated

Playlist

const playlist = new ytcog.Playlist(session, playlistOptions);
await playlist.fetch();

playlistOptions See wiki for all playlist options.

Get 100 videos from a playlist

await playlist.fetch({quantity:100});

Get all the videos from a playlist

await playlist.fetch({quantity: playlist.videoCount});

Iterate through the results with:

playlist.results.forEach((video)=>{...}) //current
playlist.videos.forEach((video)=>{...}); //accumulated

Video

Get metadata, media and stream information:

const video = new ytcog.Video(session, videoOptions);
await video.fetch();

Get comments

const video = new ytcog.Video(session, videoOptions);
await video.fetchComments(commentOptions);

Or just download:

const video = new ytcog.Video(session, videoOptions);
await video.download();

videoOptions See wiki for all video options.
commentOptions See wiki for comment options.

Examples

Check the examples folder for more clarity on usage of Session, Search, Channel, Playlist and Video classes.

To run the examples:

ytcog> node examples/session_test
ytcog> node examples/search_test [query]
ytcog> node examples/channel_test [id]
ytcog> node examples/playlist_test [id]
ytcog> node examples/video_test [id]
ytcog> node examples/dl_test [id]

Install

npm install ytcog

Disclaimer

YouTube can and will change how their innertube api works at any time. So potential disruptions are likely in the future. I will try to evolve and adapt this library asap, but without gaurantees.

Command Line Interface

Try out the command line interface (CLI) to this library:

Acknowledgement

To the following node-js projects on which ytcog has a dependency:

ytcog's People

Contributors

gatecrasher777 avatar md-is avatar sirxavierc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

ytcog's Issues

How to search and download by passing download location parameter?

I using this:

const pesquisar = new ytcog.Search(session, searchOptions)
await pesquisar.fetch()

const options2 = {
  path: './src/music/',
  filename:id,
  container:"mp3",
  videoQuality:"none"
}
//sobrecrevendo os options
pesquisar.videos[0].options = Object.assign({}, pesquisar.videos[0].options, options2)

await pesquisar.videos[0].download()

but is there any more official way to download from search passing the download arguments?

Only show console.log if debug is set to true

Hello,

As I'd rather handle console.log myself, what do you think about hiding them in the index.js if debug is not set to true:

//...
async fetch() {
    this.session.debugOn = this.debug;
    await this.session.fetch();
    if (this.session.status === 'OK') {
        this.video = new Video(this.session, this.options);
        this.video.debugOn = this.debug;
        await this.video.fetch();
        if (this.video.status === 'OK') {
            if (this.debug)
                console.log(ut.jsp(this.video.info(exclusions)));
            await this.video.download();
            if (this.video.downloaded) {
                if (this.debug) {
                    console.log('\n\nDone!');
                    console.log(`Downloaded: ${this.video.fn}`);
                }
            } else {
                throw new Error(`Video status: ${this.video.status} (${this.video.reason})`);
            }
        } else {
            throw new Error(`Video status: ${this.video.status} (${this.video.reason})`);
        }
    } else {
        throw new Error(`Session status: ${this.session.status} (${this.session.reason})`);
    }
}
//...

As you can see, I've also put some throw new Error('...'), which allow my script to do the error handling.

Thank you, awesome lib!

Is it possible to find shorts and videos?

searchOptions = {
                    query: "https://www.youtube.com/shorts/IXCkBMFChvU?feature=share",
                    items: "any",
                    quantity: 1
                }

I'm using these search parameters but it doesn't find the selected shorts

This is a piece of output.

  quantity: 0,
  latest: 0,
  results: [],
  videos: [],
  channels: [],
  playlists: [],
  estimated: 0,
  page: 0,
  more: '',
  updated: 1686948331353

Nice❤

great alternative to ytdlcore, lately ytdlcore was showing a lot of errors but this was the salvation thanks.

Long delay after download complete

Hi,

I'm using yt-cog to download audio from Youtube. While the download is fairly quick, if the target audio file is downloaded in 10 seconds (i.e. we see the progress event reach 100%), there's then a further 20-30 seconds where nothing is happening but the script is still waiting for video.download to complete.

Looking at the filesystem, I see the target file created e.g. sSheoMl5_uQ.aac, but a secondary file sSheoMl5_uQ_audio_a0.mp4 is also created and eventually deleted after the long delay.

Here's the relevant code:

  await session.fetch();
  let video = new ytcog.Video(session, {id: req.params.id, videoQuality: 'none', filename: '${id}'});
  await video.fetch();

  let bytesDownloaded = 0;
  await video.download({
      progress: (prog, size, total)=>{
          bytesDownloaded += size;
          process.stdout.write( `Progress ${Math.floor(prog)}%  ${bytesDownloaded}/${total} bytes    \r` );
      } 
  });

  return res.status(200).json({
    message: "Download complete!",
  });

Thwe same happens using the dl method:

  let options = {
		// any video id, mandatory propery
		id: req.params.id,
		// supply a download folder for downloaded video
		path: './tmp',
		// supply a optional filename, do not include an extension. default filename is author_title_videoId_format.ext
		filename: '${id}',
		// any, mp4, webm, mkv
		container: 'any',
		// desired quality: highest, 1080p, 720p, 480p, medium, 360p, 240p, 144p, lowest, none
		videoQuality: 'none',
		// desired audio quality: high, medium, low, none
		audioQuality: 'medium',
	};

  await ytcog.dl(options);

Can you suggest what might be causing the delay?

Can not get session again!

It's failed to get Session again:
Status = NOK
Reason: Could not extract player functions

Could u pls check

Can not get session

Today, I got the Session error: 'No information received'
Could u pls check it?

Strange behavior from quantity

If I specify any value on quantity, it returns 30 results, if not 60

        const channel=new ytcog.Channel(cogSession, {
            id: code,
            items: 'videos',
            quantity: 1,
        });
        await channel.fetch()

Channel videos have no thumbs and status = 'NOK'

Channel.fetch returns videos without thumbs and an error status:

...
Video {
      type: 'video',
      cookie: '',
      userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0 Gecko/20100101 Firefox/79.0',
      proxy: '',
      status: 'NOK',
      reason: 'No information received',
...
const channel = new ytcog.Channel(ytcogSession, {id: id, items: 'videos'});
await channel.fetch();

question/feat-req: youtube music support

hello gatecrasher,
does YouTube Music also run on innertube?
if so, could ytcog be modified to support ytmusic-specific features (for example, filtering music searches for songs, albums, artists, etc)?

Wrong video time properties

I'm having an issue with the 'updated', 'published' and 'isLive' properties. If I try to get this from a channel that has a regular video:

console.log("updated:", channel.videos[0].updated);
console.log("published:", channel.videos[0].published);
console.log("isLive:", channel.videos[0].isLive)

I get:

updated: 0
published: 1568813855500
isLive: false

a little while later but same video:

updated: 0
published: 1568814231378
isLive: false

Every video I tried the timestamp keeps changing, and updated is always 0.

For live videos, I get the following:

updated: 0
published: 0
duration: 0
isLive: false

But it's not as big of a deal, because I consider it live if duration is 0.

Failed to get link

Today morning it just return error:
NOK: Could not extract player functions
Can you have a look on it?

No media available

Failed to get videoStreams for some video
Available media streams: { error: 'No media available' }

Sample Video ID: 4fndeDfaWCg
Command: node examples/video_test 4fndeDfaWCg

Playback on different IP addresses (question)

Hi! Previously, there would be some audio URLs (googlevideo.com urls) that would work on multiple IP addresses but it seems there are no more videos like this anymore. All returned videos only work on the IP it's generated on. For me, this ends up causing much more server usage as now all videos need to be streamed.

Have you also noticed this? And do you know of any potential solutions? Thanks for your help.

Edit: Only noticed the change about 5-8 days ago.

TypeScript typings

I am currently writing some typings. Would you consider a PR with index.d.ts?

can I get bigger Video Thumbnail

I'd like to play video on the web page, and I need a video thumbnail to be the poster of my video-tag
I tried to get this fromvideo. data after await video.fetch(). I found some thumbnails, but it is small. how can I get a bigger one?
thanks

add proxy support

ytdl-core allows the ability to pass "requestOptions" to miniget which in turn allows the use of a proxy for all requests.

would it be possible to offer similar functionality with ytcog?

Make default search result period any

const search = new ytcog.Search(session, {
"take my breath the weeknd",
order: "views",
}

I just spent over an hour trying to figure out why this search would be returning such weird results, only to find out that this library returns the newest uploads instead of the usual search behaviour of defaulting to all-time results. I'd like to suggest that this is changed to emulate the search bar on YouTube itself.

Age-restricted videos return no audio streams with only some cookies.

Hi,

I'm trying to access age-restricted videos such as this one with ytcog. However, on every one of my account cookies, it returns no audio streams except on my main one. I am wondering why this may be.

While it's acceptable to simply use my primary account's cookies for this, I'm worried it will influence my YouTube recommendations and watch history. Any help would be appreciated and thank you so much for making this library!!

Also- I'm happy to provide the cookies themselves via email or something if needed.

Inconsistent Playback (Slow Buffering)

I'm sure you already know but thought I'd post here about a recent change YouTube made that affects how videos are parsed. Leads to slow playback.

TeamNewPipe/NewPipeExtractor#785
Here's a referenced pull that seemed to help.

Edit: I have also been noticing that some playback URLs simply don't work. I think this is related to this issue since it's only been happening since this change and seems to be in a similar realm. (Links will work upon retrying the video information fetch code.)

Thanks!

Dechipered Audio URLs sometimes time out

Very randomly, audio URLs will take between 30-40 seconds to load (even going to them directly through a browser), and then the progress bar fills up immediately. I'm not sure why this is happening but it could be because of a recent change YouTube made. This issue has been happening since about a month ago.

Split out download fork to allow webpack compilation

I am currently having trouble compiling the code with webpack. I only need the code to download video meta info which does not need the code to fork "dl.js" in video.js.
Could the video meta code live in a webpack compatible module?

cookie error

Video status: ERROR (Video unavailable)
undefined

const ytcog = require('ytcog');

function progresso(p){
    console.log(p);
}
let options = {
    // any video id, mandatory propery
    id: 'YrV3pOo3ycU',
    // optional published timestamp
    published: 0,
    // supply a download folder for downloaded video
    path: './src/music/',
    // supply a optional filename, do not include an extension. default filename is author_title_videoId_format.ext
    filename: '${author}_${datetime}_${title}_${id}_${videoQuality}_${videoCodec}_${audioCodec}',
    // any, mp4, webm, mkv
    container: 'mp3',
    // desired quality: highest, 1080p, 720p, 480p, medium, 360p, 240p, 144p, lowest, none
    videoQuality: 'none',
    // desired audio quality: high, medium, low, none
    audioQuality: 'medium',
    // for streams of equal resolution/quality pick the highest or lowest bitrate.
    mediaBitrate: 'highest',
    // Specific video format (-1 use above options to rank video streams)
    videoFormat: -1,
    // Specific audio format (-1 use above options to rank audio streams)
    audioFormat: -1,
    // Metadata to add to downloaded media
    metadata: 'author,title,published',
    progress: progresso
}

async function teste(){
    let sessionc = {
        cookie:`
        VISITOR_INFO1_LIVE=##################; 
        DEVICE_INFO=######################################################; 
        SSID=##########; 
        APISID=###################; 
        SAPISID=0#########################; 
        __Secure-1PAPISID=###########################; 
        __Secure-3PAPISID=######################; 
        LOGIN_INFO=############################; 
        _gcl_au=######################; 
        PREF=#####################################; 
        SID=#######################################; 
        __Secure-1PSID=##########################################.; 
        __Secure-3PSID=###################################.; 
        _gid=###############################; 
       _ga=###################################; 
      _ga_8TM49EC3ES=##################################; 
        SIDCC=#################################; 
        __Secure-1PSIDCC=##############################; 
        __Secure-3PSIDCC=#########################`,
        userAgent:`Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36`
    }
    const session = new ytcog.Session(sessionc.cookie,sessionc.userAgent);
    let r = await ytcog.dl(session,options)
    console.log(r)
}
teste()

Some videos does not have sound on download

Context

Hello 👋

I have a Google Sheet with a lot of Youtube videos (concert show) links on it and with a simple NodeJS script, I'm downloading them.

What I've seen so far (with the 4/5 firsts links), is that some video does not have sound, for example:

  • Zqonsv6j1_c
  • A1ypfceoOKk

Minimal code to reproduce this problem:

import ytcog from 'ytcog';

(async () => {
    await ytcog.dl({
        id: 'A1ypfceoOKk',
        path: '.',
        filename: 'test',
    })
})()

What I tried without success:

  • Change the video/audio quality
  • Check if it's a copyright problem with the video but the video does not seem to have copyright claim
  • Use cookie and userAgent (cookie that I had to encodeURI otherwise a Invalid character in header content ["cookie"] was thrown)

What I noticed:

  • The sound file does not seem to be generated, so the problem could be with downloading the sound and not with the step of merging the two files (not 100% sure)

Let me know if I can be of any help to resolve this issue,
Thank you 🙏

[help wanted] how to make this work on cloudflare workers?

Great job!!!
getting stream information with this package, and then warping the video stream from youtube to my own web with miniget package, my app works fine in Heroku.

for Heroku is a little slow in my country, so I use Cloudflare as a load balancer

Then I thought about whether Cloudflare workers could be used directly to complete these tasks. There are several advantages to using Cloudflare workers directly:

  1. Don’t worry about server dormancy (Heroku server will sleep on its own after a period of time and wake up when requested)
  2. Request information from youtube from different machines to avoid being blocked by youtube, when the request volume is large.

I tried to rewrite your code and use fetch instead of miniget. This can basically be done. But I found that you use the vm2 module in the player class, and it feels like you want to change the string from the intercepted string to runnable code. I don't know whether the vm2 module can be run directly in Cloudflare worker, it seems to use the built-in module of node js. So I want to use the new Function() of js to convert the string to runnable code, but I found that this function seems to be disabled by the Cloudflare worker.

I don’t know your code logic very well, I can only look for some clues in the code. If you can explain the whole design logic in detail, or give a code that can run on Cloudflare workers, I would be very grateful

No audio streams for specific video

Hi,

With this song, no audioStreams seem to be available for some reason. This is despite being generated with a cookie to bypass age restriction. I seem to be logged in with session.loggedIn being true.

Thought this was some behaviour that you might find interesting! Anyone able to reproduce it?

How to fetch channel by name?

Hi again,

Looking at Youtube Channels, many default to a name, rather than an ID. The ytcog.Channel method currently works only with the ID. Is there any way to fetch a channel by name?

For example: https://www.youtube.com/c/PianoteOfficial

Its channel ID (UC_DmCvOP5Q_eBMRDvqqRXjg) can be found by looking at the HTML source: <link rel="canonical" href="https://www.youtube.com/channel/UC_DmCvOP5Q_eBMRDvqqRXjg"> but this isn't going to be an option for users.

Thanks again.

Login to confirm your age - video info response

Restricted videos now returning with this error code, even though there is a valid logged-in session. The session info (with valid cookie) confirms logged-in status, but individual video info requests using the session ids report otherwise. This means age restricted/login required video info requests do not return stream data.

Busy investigating this. No easy fix found so far.

Would be interested if others are experiencing this issue.

failed streams

I have a general issue in that all streams fail to download. Worked fine yesterday.

Everything else seems to work without error. Just downloading. On video test I get:

Video status: NOK (all video streams are exhausted, download failed)

It might be a localized issue, and only affecting me. But if you also have this issue, please indicate.

Question about NodeJS Streams

Instead of downloading a video/audio to a file, is it possible to output the audio into a stream - for example, an ExpressJS response?

Thanks for your help.

Cannot run program

node examples/dl_test.js
Session status: NOK (Session data not found)
node --version
v13.13.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.