joetats / youtube_search Goto Github PK

View Code? Open in Web Editor NEW

206.0 206.0 61.0 34 KB

Tool for searching for youtube videos to avoid using their heavily rate-limited API

License: MIT License

Python 100.00%

youtube_search's People

Stargazers

Watchers

youtube_search's Issues

KeyError: 'descriptionSnippet' - Search is inconsistent

Hi guys,

Really hate to be posting here as it's probably a config I've got somewhere, but I can't for the life of me get this working to return any data on a search term properly, it seems almost random and I can't think of why it's doing this.

>>> YoutubeSearch('search terms', max_results=1).to_dict()
[{'id': '6ihNMNTBdjk', 'thumbnails': ['https://i.ytimg.com/vi/6ihNMNTBdjk/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLDohvKhRqlBMKaSWiA0_jIMQXVbqA', 'https://i.ytimg.com/vi/6ihNMNTBdjk/hqdefault.jpg?sqp=-oaymwEiCMQBEG5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLCD3ayrPiREK6X76-vSClblWE4QIw', 'https://i.ytimg.com/vi/6ihNMNTBdjk/hqdefault.jpg?sqp=-oaymwEjCPYBEIoBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBydsvefOrgm0l0RjEXwBEfe57s-w', 'https://i.ytimg.com/vi/6ihNMNTBdjk/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLDKfWtAZYgHu1TO3oXRxWtzP0CjPA'], 'title': 'How to improve your search terms', 'long_desc': 'How to improve your ', 'channel': 'Brock University Library', 'duration': '3:05', 'views': '10,766 views', 'url_suffix': '/watch?v=6ihNMNTBdjk'}]
>>>
>>> YoutubeSearch('prizm', max_results=1).to_dict()
[{'id': 'Z3BWqUZ3Ka0', 'thumbnails': ['https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLCn1Iqp-WmH2AO5CeV6K_MdCCY6EA', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEiCMQBEG5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLDmD-SwIRNtAXwphN2KaaVA1_gdzQ', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEjCPYBEIoBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBS2JJqU2bWn6EvRhCLraqR3EUnZg', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBa1DLaU3WLiVJf4VqBPo1_ppR45g'], 'title': 'PRIZM - All Night (Official Music Video)', 'long_desc': 'PRIZM', 'channel': 'FiXT Neon', 'duration': '3:53', 'views': '19,005 views', 'url_suffix': '/watch?v=Z3BWqUZ3Ka0'}]
>>>
>>> YoutubeSearch('prizm all night', max_results=1).to_dict()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 10, in __init__
    self.videos = self.search()
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 19, in search
    results = self.parse_html(response)
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 48, in parse_html
    res["long_desc"] = video_data["descriptionSnippet"]["runs"][0]["text"]
KeyError: 'descriptionSnippet'
>>> YoutubeSearch('prizm all', max_results=1).to_dict()
[{'id': 'Z3BWqUZ3Ka0', 'thumbnails': ['https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLCn1Iqp-WmH2AO5CeV6K_MdCCY6EA', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEiCMQBEG5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLDmD-SwIRNtAXwphN2KaaVA1_gdzQ', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEjCPYBEIoBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBS2JJqU2bWn6EvRhCLraqR3EUnZg', 'https://i.ytimg.com/vi/Z3BWqUZ3Ka0/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBa1DLaU3WLiVJf4VqBPo1_ppR45g'], 'title': 'PRIZM - All Night (Official Music Video)', 'long_desc': 'PRIZM', 'channel': 'FiXT Neon', 'duration': '3:53', 'views': '19,005 views', 'url_suffix': '/watch?v=Z3BWqUZ3Ka0'}]
>>>
>>> YoutubeSearch('prizm', max_results=1).to_dict()
[]
>>> YoutubeSearch('prizm want', max_results=1).to_dict()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 10, in __init__
    self.videos = self.search()
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 19, in search
    results = self.parse_html(response)
  File "/usr/local/lib/python3.7/dist-packages/youtube_search/__init__.py", line 48, in parse_html
    res["long_desc"] = video_data["descriptionSnippet"]["runs"][0]["text"]
KeyError: 'descriptionSnippet'
>>>

Again, sorry to post here but I'm properly stumped.

Cheers

ImportError: cannot import name 'YoutubeSearch'

Hey, I'm pretty sure this issue is my own fault, but whenever running the example code given I get the error:

"ImportError: cannot import name 'YoutubeSearch'"

I've tried installing the library on both windows and linux (I use the windows terminal), both through:

pip (or pip3) install youtube_search

and through github via

pip (or pip3) install git+https://github.com/joetats/youtube_search.git

This is the example code:

from youtube_search import YoutubeSearch

results = YoutubeSearch('search terms', max_results=10).to_json()

print(results)

returns a json string

########################################

results = YoutubeSearch('search terms', max_results=10).to_dict()

print(results)

returns a dictionary

I also have pytube installed if that affects anything?

Video Quality and Upload date

I think including these two in the meta data is quite essential, not sure if it can be done easily.

Get channel subscribers and views

This is important for me, I need to get subs and channel views along side the videos in the search result json.
I think the parsing is almost the same as search page parsing.

Intermitent Issue: raised an exception: KeyError: 'sectionListRenderer'

Hi,

I get this error from time to time but somehow on 2nd call it resolves itself.Maybe something to be looked into.

Regards,
Subhadip

Cache result of search() in case it's called twice.

Since the function is called on __init__ and it's result is stored in self.videos, you should return the self.videos on search() or add an optional parameter like dont_cache=False to the function.

Add max results parameter

How can I get results from my list?

Hello, I'm trying to get a dictionary from my list.

I have a list like this:
data = ['cats', 'rabbits', 'dogs', ...]

I want to get Youtube results for each element (such as a video about cats, rabbits, and dogs).
However, my code only returns videos of the last element.

for elem in data:
        videosSearch = YoutubeSearch(elem).to_dict()

Release tags on GitHub

It would be nice if releases appeared on https://github.com/joetats/youtube_search/releases.

Need an option to filter live streams

Filter Results with Creative Commons License

Is there any way to filter search results to see only media that has creative commons license?

Blank json output for all searches

I have been using this snippet for a while:

search = json.loads(YoutubeSearch("test", max_results=1).to_json())
videos = search["videos"][0]
link = videos["link"]
print("https://youtube.com" + link)

which now results in list index out of range

Since today, the json output appears to be returning this for all searches:
{"videos": []}

I'm wondering if youtube changed something in the page?

asynchronous support?

I am including the search in my discord bot, and it seems to freeze the bot while it searches
if setting == 'url': url=query elif setting == 'search': yt = YoutubeSearch(query, max_results=1).to_json() yt_id = str(json.loads(yt)['videos'][0]['id']) url = 'https://www.youtube.com/watch?v='+yt_id

longBylineText occassionaly gives AttributeError

Some search terms will fail, such as this example for "melon buffet". This happens fairly often with many different query's.

from youtube_search import YoutubeSearch
search = YoutubeSearch('melon buffet', max_results=10).to_json()
print (search)

Traceback (most recent call last):
  File "./yt.py", line 6, in <module>
    search = YoutubeSearch('melon buffet', max_results=10)
  File "/home/pi/.local/lib/python3.7/site-packages/youtube_search/__init__.py", line 10, in __init__
    self.videos = self.search()
  File "/home/pi/.local/lib/python3.7/site-packages/youtube_search/__init__.py", line 19, in search
    results = self.parse_html(response)
  File "/home/pi/.local/lib/python3.7/site-packages/youtube_search/__init__.py", line 47, in parse_html
    res["channel"] = video_data.get("longBylineText", {}).get("runs", [[{}]])[0].get("text", None)
AttributeError: 'list' object has no attribute 'get'

Commenting out this line

youtube_search/youtube_search/__init__.py

Line 47 in e021809

 res["channel"] = video_data.get("longBylineText", {}).get("runs", [[{}]])[0].get("text", None) 

is a temporary fix. I'm not sure on why this is happening for some queries?

Slowness

Hi, I've been looking for something like this project and it seems perfect. The only thing that's really nagging me is that it takes so long to return a search query. I would like to use this to create a script that queries youtube, lets me select a video and opens it in mpv, but the slowness would make that pretty frustrating.

I tried running time on a simple script and got:

#Search query results... 
python test.py  0,41s user 0,05s system 5% cpu 8,174 total

The script in question is:

from youtube_search import YoutubeSearch
results = YoutubeSearch('query', max_results=8).to_json()
print(results)

So am I doing something wrong here or is it just inherently slow?

Search is broken

Getting 200 None and empty response after invoking a search

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): youtube.com:443
DEBUG:urllib3.connectionpool:https://youtube.com:443 "GET /results?search_query=tuvan%20throat%20singing&pbj=1 HTTP/1.1" 301 0
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): www.youtube.com:443
DEBUG:urllib3.connectionpool:https://www.youtube.com:443 "GET /results?search_query=tuvan%20throat%20singing&pbj=1 HTTP/1.1" 200 None
{"videos": []}

youtube-search module is updated to 1.0.0

Fails when YouTube serves "Suggest Products" ad

# 1
YoutubeSearch("Hard to Concentrate - Red Hot Chili Peppers", max_results=10).to_dict()

# 2
YoutubeSearch("Hard to Concentrat - Red Hot Chili Peppers", max_results=10).to_dict()

Notice any difference? (1) returns zero results, while (2) returns all ten. Searching this manually in the YouTube search box shows that the first query results in a "Suggested Products" ad from youtube (stuff like printed lyrics etc) which presumably is confusing the web scraper.

Both of these should return a full ten results since there are plenty to scrape from the page.

can you add filter option about upload date??

can you add option about upload date filter?

url = f"{BASE_URL}/results?search_query={encoded_search}&sp=EgQIARAB

last 1 hour &sp=EgQIARAB
last day &sp=EgQIAhAB
..

Empty Results

from youtube_search import YoutubeSearch
results = YoutubeSearch('youtube', max_results=10).to_json()
print(results)

Returns an empty object:

{"videos": []}

I have a working internet connection and no errors are presented.
Any idea why?

max_results attribute cap at 20

Nice library, thanks for releasing this! The max_results flag doesn't seem to be able to return more than 20 results.

Set json method to return json, make a dict method

Weird empty result response

On Python 3.7

>>> serp = 'asereje'
>>> YoutubeSearch(serp, max_results=1).to_dict()
[{'title': 'Las Ketchup - The Ketchup Song (Asereje) (Spanish Version) (Official Video)', 'link': '/watch?v=V0PisGe66mY', 'id': 'V0PisGe66mY'}]
>>> YoutubeSearch(serp, max_results=1).to_dict()
[]
>>> YoutubeSearch(serp, max_results=1).to_dict()
[{'title': 'Las Ketchup - The Ketchup Song (Asereje) (Spanish Version) (Official Video)', 'link': '/watch?v=V0PisGe66mY', 'id': 'V0PisGe66mY'}]

proxies = {"http":"http://username:password@proxy_ip:proxy_port"}
Pinterest(email='emai', password='pass', username='name', cred_root='cred_root', proxies=proxies)

joetats / youtube_search Goto Github PK

youtube_search's People

Stargazers

Watchers

Forkers

youtube_search's Issues

returns a json string

returns a dictionary

Recommend Projects

Recommend Topics

Recommend Org

Jobs