Comments (9)
Remember you pointed out error of wrong numbers of stargazers.
The log says that it scrapped data of about 306 users.
And as i told you earlier that since code was not tested for the case where data is more than 1000 and thus this code is neglecting x1000.
Will be fixing very soon.
from stardox.
What i know is that api rate limit of authenticated request is 1500/h and 60/h for non-authenticated.
Whereas on checking terms and conditions and rules for scrapping i didn't found any such limits to scrappers.
from stardox.
Yeah, those sound right, encountered them before in my research, as other projects were using the API. Hopefully there are no ToS related issues indeed.
I didn't got the proper amount of results when scaping freeCodeCamp though xD
from stardox.
Freecodecamp repo has huge numbers.
And yes stardox was not tested for such big numbers.although Yesterday i tested stardox for around 200 stars and it worked perfectly.
I know it should would perfectly no matter what the number are. Can you tell what was the problem ? Upto how much you were able to scrape? Do you have any logs of error if possible?
from stardox.
@0xPrateek Not sure about a number, the saved log has about 2800 lines though. There wasn't any error.
the log 2.txt
from stardox.
Great to hear then. Yeah, was thinking that might be the issue, looking forward to the fix and also to #19
from stardox.
I'll push these updates very soon.
from stardox.
Stardox fetched more email addresses so far, but still far away from the truth, I suspend because of GH API limitations, but no error gets displayed. You should test with a big repo, like I do with freeCodeCamp
from stardox.
@DreadKnight Okay..! will test it once.
from stardox.
Related Issues (20)
- getting numbers wrong HOT 3
- Restructuring stardox code
- Re-writing command line arguments code. HOT 2
- Use PEP-8 code formatting HOT 3
- cannot get the first stargazer's info HOT 1
- cannot fetch total stargazers list after 40 pages HOT 5
- Fetch stargazer's bio and location along with other details HOT 2
- Search profile information using username HOT 6
- adding more attempts for Entering correct infomation HOT 8
- Use a vitual environment for the application HOT 2
- Restructuring to add new arguments HOT 1
- Details of the contributors of a repository HOT 1
- Replace Example images with example images with new logo HOT 6
- Fix Typos and Minor Grammar Issues in README HOT 1
- Logging all the errors in a file
- Using only 'owner/repo-name' format instead of entire link HOT 1
- BeautifulSoup name error HOT 1
- list index out of range HOT 3
- Export Limit to 1201 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stardox.