Comments (7)
Traceback (most recent call last):
File "***site-packages\pornhub\videos.py", line 247, in getVideos
for possible_video in self._scrapLiVideos(self._loadPage(page_num=page, sort_by=sort_by)):
File "***site-packages\pornhub\videos.py", line 80, in scrapLiVideos
return soup_data.find("div", class="sectionWrapper").find_all("li", { "class" : re.compile(".videoblock videoBox.") } )
AttributeError: 'NoneType' object has no attribute 'find_all'
from pornhub-api.
I believe there is an error happening in the _loadPage() method, where the page loaded was actually the home page instead of the search page wanted (thus making the _scrapLiVideos() to malfunction). This is pretty weird, lol
from pornhub-api.
Okay, there is a 404 request error in line 66 of _loadPage(), that is probably the cause of trouble
And the reason why such trouble occurs is because I input a keyword string where punctuations exists.
from pornhub-api.
But the error continues, the _scrapLiVideos(self, soup_data) returns an empty list that causes an infinite loop
The find('sectionWrapper') method seems to return the 'sectionWrapper PremiumSuggestion' node, which is not wanted
from pornhub-api.
Okay, there is a 404 request error in line 66 of _loadPage(), that is probably the cause of trouble
And the reason why such trouble occurs is because I input a keyword string where punctuations exists.
ok i'll add code that removes any punctuation:
for item in self.keywords:
item = re.sub(r"[^\w\s]", "", item).replace("_", " ")
That seems to work
from pornhub-api.
Cool, I also found a way to correct the infinite loop error caused by not finding the wanted "sectionWrapper"
I rewrite and change the method _scrapLiVideos() a bit, and so far it seems to work.
def _scrapLiVideos(self, soup_data) -> list:
section_wrappers = soup_data.findAll("div", class_="sectionWrapper")
for wrapper in section_wrappers:
LiVideos = wrapper.find_all("li", {"class": re.compile(".*videoblock videoBox.*")})
if LiVideos != []:
return LiVideos
raise Exception('LiVideos Not Found')
from pornhub-api.
Cool, I also found a way to correct the infinite loop error caused by not finding the wanted "sectionWrapper" I rewrite and change the method _scrapLiVideos() a bit, and so far it seems to work.
def _scrapLiVideos(self, soup_data) -> list: section_wrappers = soup_data.findAll("div", class_="sectionWrapper") for wrapper in section_wrappers: LiVideos = wrapper.find_all("li", {"class": re.compile(".*videoblock videoBox.*")}) if LiVideos != []: return LiVideos raise Exception('LiVideos Not Found')
Nice, also I added something similar for future gifs
just in case. Nothing seems to be broken :D
from pornhub-api.
Related Issues (20)
- getPhotos doesn't work, requests.exceptions.ConnectionError HOT 2
- getVideos attribute don't work? HOT 2
- Can't we use sort_by parameter HOT 2
- getVideos() got an unexpected keyword argument 'full_data' HOT 2
- getVideo method seems absent HOT 1
- demo code error (pip installing the wrong version) HOT 10
- getVideo() giving "***Video not available in your country***" despite IP not being in a restricted region HOT 8
- [Error Page Not Found] occurs when search keys' length in utf-8 bytes exceeds 100 (>= 100) HOT 1
- Expecting value: line 1 column 1 (char 0) when calling getVideo() print(video['accurate_views']) HOT 4
- Endless video download HOT 1
- @@ -0,0 +1,11 @@ +# -*- coding: utf-8 -*- + +from scrapy import Item, Field +class PornVideoItem(Item): + video_title = Field() + image_url = Field() + video_duration = Field() + quality_480p = Field() + video_views = Field() + video_rating = Field() + link_url = Field() \ No newline at end of file HOT 1
- Comments HOT 1
- GetVideos Module Has an Error
- GetVideos Module Has an Error HOT 1
- How Photos class work? HOT 1
- Make package install-able with pip HOT 6
- Script doesn't work anymore HOT 3
- This is not working when I try to obtain stars or even search a video through keywords HOT 5
- getStars HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pornhub-api.