donwa / gclone Goto Github PK

It's a great mod.Hope you can add MacOS support!

Can only use the first account, and then report an error, can not switch accounts

Hi, I tried using this to copy a couple of files but it doesn't seems to be working. It keeps on changing SAs and then shows that transfer thingies with 0/800GB transfered. Then it goes back to changing SAs and keeps doing that. Nothing is getting copied to my other drive.

I have added SAs to both the remotes.

使用共享链接的快捷方式的文件夹ID,运行gclone copy 显示
Ignoring unknown document type "application/vnd.google-apps.shortcut"
认不出，直接跳过。

大佬，gclone的rclone.conf并不像rclone那样有token项目，现在我想在heroku上搭建离线下载自动上传到GD，但是没有token，上传失败！
求助大佬，gclone的token去哪里找啊！

之前使用gclone转存的时候偶尔会出现如图中的错误，现排查可能是accounts目录下的部分sa没有添加到团队盘的缘故（很多人譬如我都是用的autorclone的一键sa加入团队盘的脚本，这个脚本会出现sa添加不全的状况），希望能做一个判断，跳过不存在的sa账号。

这个项目太强了，感谢大佬！

首先感谢作者你开发的Gclone。

我的情况是我要转存有很多文件夹的文件，加起来好几个T。
网上很多教程教的参数是--drive-server-side-across-configs -vvP --ignore-existing --tpslimit 3
实际使用下来速度还不错，平均一两百M的样子。但是我在使用的过程中，看到有人说用另一种参数速度会更快。说是一个小时就能转存几个T。
参数为--transfers=8 --size-only --drive-server-side-across-configs

我针对自己的实际使用情况把它修改为--drive-server-side-across-configs -vvP --ignore-existing --tpslimit 3 --transfers=8
然后很怪异的是为什么用了第二个参数里的transfers后却根本没有想象中的速度。反而变成了十几K。按照官方的说明transfers多线程应该会更快啊。所以想请问是软件的原因还是我理解有误？

真的好爽 特别是转存分享资源的时候，不要太爽

gclone 根据google的文件夹命名规则进行判断是否是正确的folder ID，默认为33位
但是极少数的，很早以前的文档有28位的。这几天碰到过两次
本着精益求精的原则，大神可否改下呢

backend/drive/drive.go 文件的 1112行 默认的目录位数是33，能否同时支持28位的

设备：N1 Armbian
安装提示：unsupported OSARCH: aarch64

root@instance:~# bash <(wget -qO- https://git.io/gclone.sh)
2020-10-15 16:25:47 URL:https://github-production-release-asset-2e65be.s3.amazonaws.com/240205214/112ec880-4e6a-11ea-8ffd-548bb11b5a75?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201015%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201015T162546Z&X-Amz-Expires=300&X-Amz-Signature=dd390916618e52a715926881c0b5135f6c24574da0b6bf3393897a9f48c2f68c&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=240205214&response-content-disposition=attachment%3B%20filename%3Dgclone_1.51.0-mod1.3.1_Linux_x86_64.gz&response-content-type=application%2Foctet-stream [11413038/11413038] -> "-" [1]
FINISHED --2020-10-15 16:25:47--
Total wall clock time: 1.9s
Downloaded: 1 files, 11M in 0.8s (14.4 MB/s)
rclone v1.51.0-mod1.3.1

• os/arch: linux/amd64
• go version: go1.13.8
  

要删除哪些目录或者文件？

since gclone is a modified (version of the old and outdated) rclone, do gclone accept all of rclone standard commands?

by all, I mean, literally all.

I am trying to clone my drive (with may subdirectory and files) and I got the bugs below:

C:\Users\user>gclone mount
Error: unknown command "mount" for "rclone"
Run 'rclone --help' for usage.
2022/01/15 Fatal error: unknown command "mount" for "rclone"

Cannot use command "mount".

"rclone mount" is working properly though.

gclone v1.51.0-mod1.3.1

作者大大你好 我试图在debian9上安装gclone 显示貌似是安装成功了，但是输入各种命令都会提示错误 入下图 请问此问题能解决吗？

谢谢！

系统版本 ubuntu 20.04

使用官方一件安装命令执行后无响应，下载并上传gclone.sh执行后显示

root@Montreal:~# bash /root/gclone.sh
No URLs found in -.

gzip: stdin: unexpected end of file

执行gclone version命令后也无响应

请教下大佬这是debian的问题还是没配置好？
显示如下，感觉是把路径都显示出来了，但是乱码了
Transferring:

• 游�/PC/�Phantom Pain.part2.rar: 0% /3.900G, 0/s, -
• 游�/PC/�Phantom Pain.part4.rar: 0% /3.900G, 0/s, -

Thank you

运行过程中断开，显示内存什么的问题，麻烦作者看看！！

错误代码：
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xbefe2d]

goroutine 44 [running]:
github.com/donwa/gclone/backend/drive.(*Fs).Copy.func1(0x19fc4a0, 0xc000a9b880, 0x6eb267ef)
/home/runner/work/gclone/gclone/backend/drive/drive.go:2182 +0x6d
github.com/rclone/rclone/fs.pacerInvoker(0x2, 0xa, 0xc000b29080, 0xc000a9b880, 0x19fc4a0, 0xc000a9b880)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/fs.go:1417 +0x3c
github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc000463380, 0xc000b29080, 0xa, 0x30, 0x15cd520)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/lib/pacer/pacer.go:197 +0x8b
github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc000463380, 0xc000b29080, 0xc000614ac0, 0xc0001f70e0)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/lib/pacer/pacer.go:216 +0x6a
github.com/donwa/gclone/backend/drive.(*Fs).Copy(0xc0002516c0, 0x1a269c0, 0xc000614ac0, 0x1a3dd20, 0xc0006f1080, 0xc0001f70e0, 0x41, 0xbfb0ff4df0f79c73, 0x32b789228, 0x243bcc0, ...)
/home/runner/work/gclone/gclone/backend/drive/drive.go:2181 +0x2f2
github.com/rclone/rclone/fs/operations.Copy(0x1a269c0, 0xc000614ac0, 0x1a3dca0, 0xc0002516c0, 0x0, 0x0, 0xc0001f70e0, 0x41, 0x1a3dd20, 0xc0006f1080, ...)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/operations/operations.go:365 +0x2ad
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc00003c8c0, 0x1a269c0, 0xc000614ac0, 0xc0001979a0, 0x1a3dca0, 0xc0002516c0, 0xc00003c998)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/sync/sync.go:331 +0x21c
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/sync/sync.go:356 +0xb1

gclone copy gc:{1apg9nf68zPQI4YFTIWX9omebIAOGAUUF} gc:{我自己的TD} --drive-server-side-across-configs
Total 332個file, 出現197error
ERROR: ............. MD5 hash differ '........' vs '..........'
跑了4次都不行,每次error數量都不一樣

後面我全刪掉重新跑,加上 --ignore-checksum才ok
嘗試其他id暫時沒事,不確定是不是哪裏出問題

嘗試dedupe說沒重複,可是MD5如果出現不同,不知道是否能查出

遇到大文件夹的时候无法计算全，多跑几次返回结果不一样

我的配置是

[K]
type = drive
scope = drive
service_account_file = /path/to/json
service_account_file_path = /path/
root_folder_id = teamdriveid

所有的sa账户都确认已经加到团队盘里面了

运行给clone之后
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xbd69e3]

goroutine 357 [running]:
github.com/donwa/gclone/backend/drive.(*Fs).list(0xc000308700, 0x1a269c0, 0xc0004ec640, 0xc000617d98, 0x1, 0x1, 0x0, 0x0, 0x0, 0xc000617da8, ...)
/home/runner/work/gclone/gclone/backend/drive/drive.go:797 +0x1e3
github.com/donwa/gclone/backend/drive.(*Fs).List(0xc000308700, 0x1a269c0, 0xc0004ec640, 0xc000706360, 0x8e, 0x0, 0x0, 0x0, 0xc08278de4b2ec249, 0xcbca736f12208bee)
/home/runner/work/gclone/gclone/backend/drive/drive.go:1631 +0x1e9
github.com/rclone/rclone/fs/list.DirSorted(0x1a269c0, 0xc0004ec640, 0x1a3dca0, 0xc000308700, 0xb00fae385db0a900, 0xc000706360, 0x8e, 0x938bc8a20e498d34, 0x2e647f646ffda325, 0x42e2ca, ...)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/list/list.go:24 +0x7d
github.com/rclone/rclone/fs/march.(*March).makeListDir.func1(0xc000706360, 0x8e, 0xdbbd9fec2cfa1415, 0xe614f9d5032fff95, 0x9188f04a89dddc88, 0x1fb5d9c7809c6227, 0x4af0ef74802deb30)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/march/march.go:79 +0x68
github.com/rclone/rclone/fs/march.(*March).processJob.func1(0xc000663490, 0xc000469710, 0xc000706360, 0x8e, 0xc000706360, 0x8e, 0x7ffffffb, 0x0, 0x100, 0xc000644160, ...)
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/march/march.go:383 +0x82
created by github.com/rclone/rclone/fs/march.(*March).processJob
/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/march/march.go:381 +0x130

Any instructions/guidance for installing on MacOS?

只把文件夹复制过去了，但里面的文件没有复制过去是什么原因？

Hi
I tried the Windows version and it works., but arm. Versión Dont work. Its always says same message. (same config that windows version)
2020/06/26 12:52:33 Failed to lsd with 2 errors: last error was: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&corpora=drive&driveId=xxxxxxxxxxxxx&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%270AOAue4Fkb160Uk9PVA%27+in+parents%29&supportsAllDrives=true: oauth2: cannot fetch token: Post https://oauth2.googleapis.com/token: dial tcp: lookup oauth2.googleapis.com on [::1]:53: read udp [::1]:45033->[::1]:53: read: connection refused

I am facing this error.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xbefe2d]

goroutine 48 [running]:
github.com/donwa/gclone/backend/drive.(*Fs).Copy.func1(0x19fc4a0, 0xc0002da170, 0x6a751e76)
	/home/runner/work/gclone/gclone/backend/drive/drive.go:2182 +0x6d
github.com/rclone/rclone/fs.pacerInvoker(0x2, 0xa, 0xc000776270, 0xc0002da170, 0x19fc4a0, 0xc0002da170)
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/fs.go:1417 +0x3c
github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc0001e3620, 0xc000776270, 0xa, 0x30, 0x15cd520)
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/lib/pacer/pacer.go:197 +0x8b
github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc0001e3620, 0xc000776270, 0xc000706780, 0xc00011cfc0)
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/lib/pacer/pacer.go:216 +0x6a
github.com/donwa/gclone/backend/drive.(*Fs).Copy(0xc00063cc40, 0x1a269c0, 0xc000706780, 0x1a3dd20, 0xc0000ff580, 0xc00011cfc0, 0x64, 0xbf8ac8dc14841245, 0x49d7eedd, 0x243bcc0, ...)
	/home/runner/work/gclone/gclone/backend/drive/drive.go:2181 +0x2f2
github.com/rclone/rclone/fs/operations.Copy(0x1a269c0, 0xc000706780, 0x1a3dca0, 0xc00063cc40, 0x0, 0x0, 0xc00011cfc0, 0x64, 0x1a3dd20, 0xc0000ff580, ...)
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/operations/operations.go:365 +0x2ad
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0007408c0, 0x1a269c0, 0xc000706780, 0xc000273360, 0x1a3dca0, 0xc00063cc40, 0xc000740998)
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/sync/sync.go:331 +0x21c
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	/home/runner/go/pkg/mod/github.com/rclone/[email protected]/fs/sync/sync.go:356 +0xb1

What's the reason? Am i doing something wrong?
Copy starts and throws this error

对于新手来说，还要去理一下autorclone，才能使用，有点不太方便

如图所示，运行后无法使用命令，加在系统变量也还是不行

Hi,
Someone can share this docker version here?

出现上述报错，打开其中的链接显示如下内容

{"error":{"errors":[{"domain":"usageLimits","reason":"dailyLimitExceededUnreg","message":"Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup.","extendedHelp":"https://code.google.com/apis/console"}],"code":403,"message":"Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup."}}

请问如何解决？

unsupported OSARCH: aarch64
能支持的话Termux可以直接使用gclone

The expected path in rclone.conf file is like:
service_account_file_path = /root/accounts/

But somehow if you write the path as the following, the program will crash.
service_account_file_path = /root/accounts

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xbd69e3]

It's not a bug, but cloud be avoided from code.

Failed to copy: googleapi: Error 403: The user's Drive storage quota has been exceeded., storageQuotaExceeded

取消后再传输，15GB后就报着错误。

请问大佬我从一个盘往另外一个盘搬时，想跳过某个带中括号名字的[XX]目录该咋写命令呀？

Hi Devs and all.

I have this message when cloning the data from one team drive to a other (Empty) team drive i get this many times

Duplicate directory found in source - ignoring

So I would like to know if there is a way to copy the folder instead of dedupe it or ignoring it, like it says it's related with the folder / directory

Btw, I don't want to dedupe my drive because I have same content in different languages and if I dedupe it, it can delete one of the languages by mistake, in the other hand i want to make sure than folder and files with the same name sa been copied (at normally there are no more than 1 movies by folder)
So i pretty sure than there will be no issues if some one help me to find a flag to force a copy instead of dedupe it.

Thanks

service_account_file = /root/accounts/1.json

是否在 accounts 下的 json 文件必须是1.json 2.json 3.json 这样啊？不会自动搜索这个目录下的 json 文件吗？ 600个 乱码一样的文件如果是必须为数字就要改名了。

空白文件夹不会复制过去吗？

哪位大神能把gclone更新至rclone最新版本，并且把每秒请求写入硬盘速度限制在3个，谢谢！

配置好了token sa账户js路径 文件也放进去了，命令是gclone copy －p od:xxx gd:xxx

复制大量文件切换sa时候会出现丢文件的情况，然而输出日志显示全部复制完毕

gclone sync 时使用参数--transfers=6(默认是4)，此时会有6个文件并发sync操作，当一个SA的750G用完时，6个并发中可能会有多个触发userRateLimitExceeded，此时SA会更换多个。
不过看代码

Great project thanks!

Could you make it work with rclone mount?

I'm using the mount command, it would be great to switch service account while the mount is kept open.

can we deploy to heroku?

I got us both $50 from Varo.* Just open a Varo Bank Account using my link below, spend $20 on your new debit card, and we'll both get paid. 🙌 💰 https://varomoney.com/r/?r=Stephanie3390

https://tweakfish.com/

今天想部署一下goindex，结果打开书签发现网页404了..而且忘记fork了...

谢谢！