GithubHelp home page GithubHelp logo

Comments (9)

0xPrateek avatar 0xPrateek commented on May 29, 2024 1

Remember you pointed out error of wrong numbers of stargazers.
The log says that it scrapped data of about 306 users.
And as i told you earlier that since code was not tested for the case where data is more than 1000 and thus this code is neglecting x1000.
Will be fixing very soon.

from stardox.

0xPrateek avatar 0xPrateek commented on May 29, 2024

What i know is that api rate limit of authenticated request is 1500/h and 60/h for non-authenticated.
Whereas on checking terms and conditions and rules for scrapping i didn't found any such limits to scrappers.

from stardox.

DreadKnight avatar DreadKnight commented on May 29, 2024

Yeah, those sound right, encountered them before in my research, as other projects were using the API. Hopefully there are no ToS related issues indeed.
I didn't got the proper amount of results when scaping freeCodeCamp though xD

from stardox.

0xPrateek avatar 0xPrateek commented on May 29, 2024

Freecodecamp repo has huge numbers.
And yes stardox was not tested for such big numbers.although Yesterday i tested stardox for around 200 stars and it worked perfectly.
I know it should would perfectly no matter what the number are. Can you tell what was the problem ? Upto how much you were able to scrape? Do you have any logs of error if possible?

from stardox.

DreadKnight avatar DreadKnight commented on May 29, 2024

@0xPrateek Not sure about a number, the saved log has about 2800 lines though. There wasn't any error.
the log 2.txt

from stardox.

DreadKnight avatar DreadKnight commented on May 29, 2024

Great to hear then. Yeah, was thinking that might be the issue, looking forward to the fix and also to #19

from stardox.

0xPrateek avatar 0xPrateek commented on May 29, 2024

I'll push these updates very soon. 👍

from stardox.

DreadKnight avatar DreadKnight commented on May 29, 2024

Stardox fetched more email addresses so far, but still far away from the truth, I suspend because of GH API limitations, but no error gets displayed. You should test with a big repo, like I do with freeCodeCamp

from stardox.

0xPrateek avatar 0xPrateek commented on May 29, 2024

@DreadKnight Okay..! will test it once.

from stardox.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.