GithubHelp home page GithubHelp logo

Comments (7)

ShixiangWang avatar ShixiangWang commented on August 10, 2024

我的个人仓库没有 100 个,但也支持同步了 30 个左右。

from hub-mirror-action.

ShixiangWang avatar ShixiangWang commented on August 10, 2024

目前只支持最多30个啊

https://stackoverflow.com/questions/27331849/github-api-v3-doesnt-show-all-user-repositories

image

建议修改下支持 100 个仓库。

from hub-mirror-action.

Yikun avatar Yikun commented on August 10, 2024
function get_all_repos {
    USER=yikun
    GITHUB_REQUEST_RATE=2
    last_page=`curl -sI "https://api.github.com/users/$USER/repos?page=1&per_page=10" | sed -nr 's/^[lL]ink:.*page=([0-9]+)&per_page=10.*/\1/p'`
    sleep $GITHUB_REQUEST_RATE

    # does this result use pagination?
    if [ -z "$last_page" ]; then
        # no - this result has only one page
        curl -s -i "https://api.github.com/users/$USER/repos" | jq '.[] | .name' |  sed 's/"//g'
    else
        p=1
        while [ "$p" -le "$last_page" ]; do
            x=`curl -s "https://api.github.com/users/$USER/repos?page=$p&per_page=10" | jq '.[] | .name' |  sed 's/"//g'`
            echo $x
            p=$(($p + 1))
            sleep $GITHUB_REQUEST_RATE
        done
    fi
}

github有流控,很容易出现查着查着就超时的情况。

from hub-mirror-action.

Yikun avatar Yikun commented on August 10, 2024

{"message":"API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://developer.github.com/v3/#rate-limiting"}

from hub-mirror-action.

ShixiangWang avatar ShixiangWang commented on August 10, 2024

增加 GItHub token 的支持?这样我记得可以一小时几千次。

from hub-mirror-action.

Yikun avatar Yikun commented on August 10, 2024

@ShixiangWang 嗯,可以复用那个token配置项,有空我试试

from hub-mirror-action.

Yikun avatar Yikun commented on August 10, 2024

因为需要获取src端的列表,因此需要src端的token,目前暂时没加这个配置项。

测了下其实基本也还好,github的接口是大概1小时60次,基本满足诉求了,即使分页每次100个repo,超过5000个repo才有可能超时。

当然,精益求精,后面准备加一个src_token作为可选配置。其实代码已经写了,但是代码比较乱,会影响pagination的阅读,所以后面再单独合入[1]

[1] 6cb3a46

from hub-mirror-action.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.