Comments (7)
我的个人仓库没有 100 个,但也支持同步了 30 个左右。
from hub-mirror-action.
目前只支持最多30个啊
https://stackoverflow.com/questions/27331849/github-api-v3-doesnt-show-all-user-repositories
建议修改下支持 100 个仓库。
from hub-mirror-action.
function get_all_repos {
USER=yikun
GITHUB_REQUEST_RATE=2
last_page=`curl -sI "https://api.github.com/users/$USER/repos?page=1&per_page=10" | sed -nr 's/^[lL]ink:.*page=([0-9]+)&per_page=10.*/\1/p'`
sleep $GITHUB_REQUEST_RATE
# does this result use pagination?
if [ -z "$last_page" ]; then
# no - this result has only one page
curl -s -i "https://api.github.com/users/$USER/repos" | jq '.[] | .name' | sed 's/"//g'
else
p=1
while [ "$p" -le "$last_page" ]; do
x=`curl -s "https://api.github.com/users/$USER/repos?page=$p&per_page=10" | jq '.[] | .name' | sed 's/"//g'`
echo $x
p=$(($p + 1))
sleep $GITHUB_REQUEST_RATE
done
fi
}
github有流控,很容易出现查着查着就超时的情况。
from hub-mirror-action.
{"message":"API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://developer.github.com/v3/#rate-limiting"}
from hub-mirror-action.
增加 GItHub token 的支持?这样我记得可以一小时几千次。
from hub-mirror-action.
@ShixiangWang 嗯,可以复用那个token配置项,有空我试试
from hub-mirror-action.
因为需要获取src端的列表,因此需要src端的token,目前暂时没加这个配置项。
测了下其实基本也还好,github的接口是大概1小时60次,基本满足诉求了,即使分页每次100个repo,超过5000个repo才有可能超时。
当然,精益求精,后面准备加一个src_token作为可选配置。其实代码已经写了,但是代码比较乱,会影响pagination的阅读,所以后面再单独合入[1]
[1] 6cb3a46
from hub-mirror-action.
Related Issues (20)
- github 私有仓库同步到gitee企业版私有仓库失败,配置文件如下 HOT 1
- 同步到gitlab如何使用呢 HOT 2
- 希望可以对自建Gitea进行支持 HOT 3
- 公共仓库间同步失败 HOT 4
- github 仓库同步到 gitee 仓库失败
- 希望可以添加对 sourcehut 的支持
- 希望添加清理缓存的功能 HOT 4
- 希望增加打印已成功仓库日志 HOT 9
- AttributeError: 'Hub' object has no attribute 'src_repo_base' HOT 4
- hub_mirror-action@v运行缓慢 HOT 2
- 为什么运行同步一直在下载系统安装东西,卡十几分钟都不行 HOT 5
- 同步目标源时创建仓库返回502 HOT 1
- 希望增加对codeup的支持
- 同步非private,但是fork的仓库同步如何做? HOT 2
- 应该如何避免总任务超时 HOT 2
- 同步gitee 失败 HOT 1
- error in gitee 同步github HOT 2
- github action处理python包PyYAML报错 HOT 2
- 【HELP】最近失败的频率有点高 HOT 1
- Permission denied (publickey) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hub-mirror-action.