429 on login about degoo

[bernd-wechner]

Thanks for looking deeper.I haven't considered the role of cookies as I don't think that the Python request we send sends any cookies with it. But I'm not sure. One of my hopes was to send the request via fiddler to see the exact request the Python library sends. But for now login works fine here, and also it seems for @DLBerger: #25 (comment).

It seems indeed that you nailed the mechanism, as the URL rest-api.degoo.com resolves to 104.18.228.238 which is owned by:

OrgName:        Cloudflare, Inc.
OrgId:          CLOUD14
Address:        101 Townsend Street
City:           San Francisco
StateProv:      CA
PostalCode:     94107
Country:        US
RegDate:        2010-07-09
Updated:        2021-07-01
Ref:            https://rdap.arin.net/registry/entity/CLOUD14

And so degoo use cloudfare as a tunnel to reach their servers. So now at least I have a handle on how cloudfare come to be involved.

That is good in a way, as being a cloudflare issue raises the chances mildly I think of getting support on the issue (Degoo are way understaffed for much support and don't prioritise supporting our efforts here very highly I suspect - they have at least note provided any documentation on the API or any advice nor as yet been able to answer a few simple outstanding questions on it - notable this one: #16).

Cloudflare being a much larger business also means community support is more likely to yield fruit.

I have indeed read that these 429s happen on cloudflare because of config issues and redirect loops.

[bernd-wechner]

To clarify, you're finding that this patch:

https://github.com/IAMSDR/Degoo/commit/eeab5472d99c7a895a7203b05b3d6481fe971dd0

works for you?

[adyanth]

With that patch, I am getting a response code of 400 when trying to login :(

Edit: My bad (typing skills :) ), it was indeed successful! Let me try to see if other functions work now!

[webhamster2000]

To clarify, you're finding that this patch:

IAMSDR@eeab547

works for you?

Can confirm. Works like a charm.
Used it on top of current master and not faced any issues so far. Uploads are very stable.

[adyanth]

Looks like the login API is at https://degoo.com/me/login and the POST request takes application/x-www-form-urlencoded rather than JSON with CSRF.

[bernd-wechner]

Are you using the latest code base (pulled from here)? We just fixed a 429 issue yesterday as it happens. I see this now:

$ ./degoo_login -fvvv
Request:
	URL: https://rest-api.degoo.com/login
	method: POST
	headers:
		User-Agent: Degoo-client/0.3
		Accept: */*
		Accept-Language: en-US,en;q=0.5
		Accept-Encoding: gzip, deflate
		Content-Type: application/json
		Origin: https://app.degoo.com
		Content-Length: 61
	body:
		{"Username":"<redacted>","Password":"<redacted>"}
Response:
	status: 200
	reason: OK
	headers:
		Date: Mon, 26 Jul 2021 09:01:22 GMT
		Content-Type: application/json
		Transfer-Encoding: chunked
		Connection: keep-alive
		CF-Ray: 674c71612fd017cf-MEL
		Access-Control-Allow-Origin: *
		Via: 1.1 7799fcaed3aefe6ff8b2d01a9d1fddcd.cloudfront.net (CloudFront)
		CF-Cache-Status: DYNAMIC
		Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
		x-amz-apigw-id: DEf7WFljoAMFg_w=
		X-Amz-Cf-Id: sUYjpCnlsA9CVWAydCFwSIRfqBnCB7u0-eE-yhkBQPx14dJfZDdaVg==
		X-Amz-Cf-Pop: ATL51-C1
		x-amzn-RequestId: 3369654a-13ea-43b0-95ef-bc2b86dd9fd5
		X-Amzn-Trace-Id: Root=1-60fe79e2-3bea0647717211800fec05fc;Sampled=0
		X-Cache: Miss from cloudfront
		Vary: Accept-Encoding
		Server: cloudflare
		Content-Encoding: gzip
	content:
		b'  {"Token":"<redacted>","RefreshToken":"","Redirect":"/moments"}\n'
Successfully logged in.

Try the latest code base, and if still no luck post the results of a login effort with -vvv redacting out your personal details. I may add a redacting option tonight to make that easier.

[bernd-wechner]

Well, it was a trivial add so while waiting for dinner, I added redacting. Now try: degoo_login -rvvv

Note that if you succeed in logging in your credentials are written to file and you can subsequently login with the -f option to draw them from file. Up to you to keep the credentials file secure. It is stored in:

conf_dir = user_config_dir("degoo")
cred_file = os.path.join(conf_dir, "credentials.json")

To find out where that is on your system I added quickly the degoo_config command, which will report it. Make sure the credentials and key files are secure. They record the access details to your account.

Im on the latest master now. This is what it shows with -rvvvv

Request:
        URL: https://rest-api.degoo.com/login
        method: POST
        headers:
                User-Agent: Degoo-client/0.3
                Accept: */*
                Accept-Language: en-US,en;q=0.5
                Accept-Encoding: gzip, deflate
                Content-Type: application/json
                Origin: https://app.degoo.com
                Content-Length: 55
        body:
                {"Username":"<redacted>","Password":"<redacted>"}
Response:
        status: 429
        reason: Too Many Requests
        headers:
                Date: Mon, 26 Jul 2021 09:53:07 GMT
                Content-Type: text/plain;charset=UTF-8
                Content-Length: 0
                Connection: keep-alive
                Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
                Vary: Accept-Encoding
                Server: cloudflare
                CF-RAY: 674cbd342bca1d3f-MAA
        content:
                b''
Login failed with: 429: Too Many Requests
Login failed.

[bernd-wechner]

Interesting. I was seeing this myself on the weekend. And I removed the keep-alive request from the request headers and it went away. Reported here:

#25 (comment)

As I'm not seeing the error now I can't experiment on it. Trying to understand it theoretically has yielded no fruit.

You see the 429 is reported by cloudflare, which is accompanied by a series of worthy observations:

1. cloudflare is a very popular CDN (content delivery network) and I use it myself a lot. They provide otehr services too which I'm not familiar with off hand.
2. I see zero reason for the login command to be drawing nay content from cloudflare, no login on any server typically, woud I expect to be leaning on a third party. But who knows.
3. Examining the full body of loaded code on both the login page and the moments page (to which it redirects reveals zero references to cloudflare.
4. Logging in on the wbe page the login request is submitted by Firefox with these headers:

POST /login HTTP/1.1
Host: rest-api.degoo.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:90.0) Gecko/20100101 Firefox/90.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json
Content-Length: 82
Origin: https://app.degoo.com
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
DNT: 1
Sec-GPC: 1
Connection: keep-alive

That last one is the one I removed to get it working here yesterday. Using it I found the 429 error reported. What you can do is capture your web login headers similarly or use these and here:

Degoo/degoo/API.py

Line 307 in 28ec3eb

headers = OrderedDict([

you can try to mimic those headers step by step to see if you find a combination that works. Their API seems very fiddly in this space. And we are reverse engineering it with a black box approach alas (we have no access to their code). So it needs someone like yourself who can reproduce a given problem to perform these experiments,. Technically if you use those exact headers the login should/must succeed (it works for Firefox).

The body Firefox submitted is as follows:

{"Username":"<redacted>","Password":"<redacted>","GenerateToken":true}

Which is clearly different to ours, it seems they have added a field since I last inspected. This is another problem we have, a moving target.

I will quickly adapt with news findings and test and push.

[bernd-wechner]

OK so I quickly added the GenerateToken request and placed that Firefox set of headers as comment in the code. I added all the remaining ones and lo and behold if I add just a single one of the Sec or DNT or Connection headers I get a 429 error. Leave them off and I can log in. I have pushed the code with which I can log in.

If you want to capture your headers in your browser and post them, please do.

If you don't know how, it's outlined in the README. But basically surf to the login page, press F12, click the network tab, clear it if it's full, login and the first entry is generally the login request. Inspect that, and get the raw headers and body.

Sure thing @bernd-wechner I can test all these out gladly.

Regarding Cloudflare, they have this Argo Tunnel which protects the origin by having a tunnel from the hosting server to cloudflare, rather than exposing the server directly. I use this myself to host a bunch of services in my homelab. All requests would be reverse proxied by Cloudflare no matter what.

When I logged in by myself on the UI, I saw it send application/x-www-form-urlencoded with my username and password and the endpoint was not rest-api.degoo.com. I'm thinking if maybe this is geographically dependent?

I'll send the headers and everything soon.

Okay, checking now, it is JSON. Let me test again with the latest changes you made!

headers = {
    'sec-ch-ua': '" Not;A Brand";v="99", "Microsoft Edge";v="91", "Chromium";v="91"',
    'Referer': '',
    'sec-ch-ua-mobile': '?0',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36 Edg/91.0.864.71',
    'Content-Type': 'application/json',
}

data = '{"Username":"<redacted>","Password":"<redacted>","GenerateToken":true}'

response = requests.post('https://rest-api.degoo.com/login', headers=headers, data=data)

Even after adding the Generate token to the code, I see the 429.

Request:
        URL: https://rest-api.degoo.com/login
        method: POST
        headers:
                User-Agent: Degoo-client/0.3
                Accept: */*
                Accept-Language: en-US,en;q=0.5
                Accept-Encoding: gzip, deflate
                Content-Type: application/json
                Origin: https://app.degoo.com
                Content-Length: 66
        body:
                {"Username":"<redacted>","Password":"<redacted>","GenerateToken":true}
Response:
        status: 429
        reason: Too Many Requests
        headers:
                Date: Mon, 26 Jul 2021 11:38:51 GMT
                Content-Type: text/plain;charset=UTF-8
                Content-Length: 0
                Connection: keep-alive
                Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
                Vary: Accept-Encoding
                Server: cloudflare
                CF-RAY: 674d5817bc581d3d-BLR
        content:
                b''
Login failed with: 429: Too Many Requests
Login failed.

But sometimes I see this happening on login:

cookies = {
    'lang': '',
    'pid': '',
    '__RequestVerificationToken': 'UktVeqbk8CP-zEPzsHNOwMK2YxXVS2uJ-XrHKp4Pt_1YAiKkPJ-lTY7f6b5rSolCNniPzOO-e9aDmThv_pkVLpBnVp1_3wI0-DUUuz1ysqE1',
    'invite': '',
    'AcceptCookies': '',
    '__stripe_mid': '22b9b296-9cba-49ef-a396-d161290d47d0be71be',
    '__stripe_sid': 'a16e2100-7aec-4de6-9a7d-f51e77e4ac690bbda0',
}

headers = {
    'authority': 'degoo.com',
    'cache-control': 'max-age=0',
    'sec-ch-ua': '" Not;A Brand";v="99", "Microsoft Edge";v="91", "Chromium";v="91"',
    'sec-ch-ua-mobile': '?0',
    'upgrade-insecure-requests': '1',
    'origin': 'https://degoo.com',
    'content-type': 'application/x-www-form-urlencoded',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36 Edg/91.0.864.71',
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'navigate',
    'sec-fetch-user': '?1',
    'sec-fetch-dest': 'document',
    'referer': 'https://degoo.com/',
    'accept-language': 'en-US,en;q=0.9',
}

data = {
  '__RequestVerificationToken': 'CIip6F_dW94k9FZJf66cApY3sB-U44e_8Wz2Fk-gWAQojmu4wY3DQ2VR223jfSFx-FJeCdeBxHxbonuiRVO0QpMmZtRHkgkHwlvFcOH38MI1',
  'returnUrl': '',
  'source': '',
  'premiumID': '',
  'Email': '<redacted>',
  'Password': '<redacted>'
}

response = requests.post('https://degoo.com/me/login', headers=headers, cookies=cookies, data=data)

The different requests happen depending on if I login to https://app.degoo.com/login or https://degoo.com/me/login

So, seems like two different endpoints for now. Still checking why am I getting 429.

Tried with all the headers added by the browser (Edge), still no luck.

Request:
        URL: https://rest-api.degoo.com/login
        method: POST
        headers:
                User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36 Edg/91.0.864.71
                Accept-Language: en-US,en;q=0.9
                Accept-Encoding: gzip, deflate, br
                Content-Type: application/json
                Referer: 
                sec-ch-ua: " Not;A Brand";v="99", "Microsoft Edge";v="91", "Chromium";v="91"
                sec-ch-ua-mobile: ?0
                sec-fetch-site: same-site
                sec-fetch-mode: cors
                sec-fetch-dest: empty
                Origin: https://app.degoo.com
                Content-Length: 76
        body:
                {"Username":"<redacted>","Password":"<redacted>","GenerateToken":true}
Response:
        status: 429
        reason: Too Many Requests
        headers:
                Date: Mon, 26 Jul 2021 12:00:42 GMT
                Content-Type: text/plain;charset=UTF-8
                Content-Length: 0
                Connection: keep-alive
                Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
                Vary: Accept-Encoding
                Server: cloudflare
                CF-RAY: 674d781bbb051d18-BLR
        content:
                b''
Login failed with: 429: Too Many Requests
Login failed.

[adyanth]

I don't see the retry after header which makes me think it might be the end server sending 429?

https://support.cloudflare.com/hc/en-us/articles/115001635128-Configuring-Cloudflare-Rate-Limiting

[adyanth]

The main difference I can see is that my browser uses HTTP/2 whereas requests uses HTTP/1.1

Also, I'm not able to inject the HTTP/2 pseudo headers in both requests and Insomnia, which I set to HTTP/2

[adyanth]

Okay, now this is surprising. I copied the CURL command from the network tab, ran it from the terminal and getting a 429 from cloudflare. I have no idea what is happening.

Sending a simple OPTIONS request gets a 429 too.

$ curl 'https://rest-api.degoo.com/login'   -X 'OPTIONS'   -H 'authority: rest-api.degoo.com'   -H 'accept: */*'   -H 'access-control-request-method: POST'   -H 'access-control-request-headers: content-type'   -H 'origin: https://app.degoo.com'   -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36 Edg/91.0.864.71'   -H 'sec-fetch-mode: cors'   -H 'sec-fetch-site: same-site'   -H 'sec-fetch-dest: empty'   -H 'accept-language: en-US,en;q=0.9'   --compressed -v
*   Trying 104.18.229.238:443...
* TCP_NODELAY set
* Connected to rest-api.degoo.com (104.18.229.238) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com
*  start date: Jul  7 00:00:00 2021 GMT
*  expire date: Jul  6 23:59:59 2022 GMT
*  subjectAltName: host "rest-api.degoo.com" matched cert's "*.degoo.com"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffcc49b820)
> OPTIONS /login HTTP/2
> Host: rest-api.degoo.com
> accept-encoding: deflate, gzip, br
> authority: rest-api.degoo.com
> accept: */*
> access-control-request-method: POST
> access-control-request-headers: content-type
> origin: https://app.degoo.com
> user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36 Edg/91.0.864.71
> sec-fetch-mode: cors
> sec-fetch-site: same-site
> sec-fetch-dest: empty
> accept-language: en-US,en;q=0.9
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 429
< date: Mon, 26 Jul 2021 12:30:51 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 0
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< vary: Accept-Encoding
< server: cloudflare
< cf-ray: 674da440ccfc1d2d-BLR
<
* Connection #0 to host rest-api.degoo.com left intact

[adyanth]

Yup

And the very straightforward way to repro the issue is by going to dev tools, find the login request, right click, copy curl (bash) and run the curl command on the terminal. This failing is a big indicator that something else is going on.

[adyanth]

Further confirming the Cloudflare part, I can access this: http://rest-api.degoo.com/cdn-cgi/trace which provides what colo I am connected to. Still does not answer why I cannot perform the same query using cURL without getting 429 :(

[bernd-wechner]

Like wow, what is that link? Where did you find that and what does it mean? What's a colo?

[adyanth]

That is from the Cloudflare's troubleshooting page here: https://support.cloudflare.com/hc/en-us/articles/203118044-Gathering-information-for-troubleshooting-sites

A colo is a colocated datacenter, in this case, the Cloudflare's DC where I am getting the data from :)

[bernd-wechner]

I can't even figure how it's possible that the exact same request via browser and via curl has different server side responses. There must be something broken in the premise - that the requests are identical. How else could the server see a difference, except that there is a difference.

Could it be that the browse provides a cookie that curl isn't?

I wonder if you want to reach out to Cloudflare with a support question or see if they have a support forum?

[developeranaz]

I'm having the same login problem
But I was able to login using (#25 (comment)) his method.

I used his repo as my login and used this repository is for uploading to degoo. I don't know what's going on in both repository. but I'm still having issues while uploading /degoo_put command.
using this method i am only able to upload one per files to newly manually created folders.
Other commands degoo_ls degoo_pwd degoo_user
are working.

[developeranaz]

To clarify, you're finding that this patch:

IAMSDR@eeab547

works for you?

Yes, this partially works with some bugs.

I logged in using IAMSDR@eeab547 patch

[lupinx]

hi,
The problem persists, even with the patch.
failed 429 ...

any news ??

[lupinx]

Using the whole fork from IAMSDR works partially,
the login and unserinfo only, the token key is generated in "keys.json"

All other commands fail: degoo_ls: getFileChildren3 failed with: Too large input!

[miikka75]

I created a pull request with IAMSDR's patch which fixes login issues for me.

[lupinx]

hmmm,
here is what i observed so far:

• the @IAMSDR (whole fork) can login successfully, generates the Token key, but commands are NOT working.

• the @bernd-wechner (whole fork + new pull fix) does NOT login (error 429...), commands are working fine but not all,
  python degoo_put test.jpg /

    `... degoo.util.DegooError: put_file: / is not a remote folder!`
    ( util.py: line 1195 )
  

And
python degoo_mkdir /newfolder

   `degoo.util.DegooError: mkdir: No parent_id provided.`

in summary, i generate the token key with @IAMSDR fork, then use @bernd-wechner

[miikka75]

I noticed also that uploading files or mkdir folder to root (/) folder doesn't work but works for other folders - for example Web folder

[bernd-wechner]

Yes, sorry, I should document that better. The root for is not writable. That's a Degoo feature. It's is reserved for devices. And your license determines how many devices you can have. Be very careful, I removed one of my devices and could not recreate it in spite of being licensed for as many devices as I want. Degoo Support had to recreate it for me. I imagine they want to fix that some time.

[lupinx]

an example of upload command would be appreciated,
as i am still struggling with it

edit:

• i uploaded a folder using firefox, but cannot upload files to it using CLI
• i created a folder using firefox, i can upload files now with CLI, but with errors
  File "...\degoo\API.py", line 850, in setUploadFile3 contents = self.getFileChildren3(parent_id)

File "...\degoo\API.py", line 585, in getFileChildren3 prefix = self.devices[i['DeviceID']] + os.sep + "Recycle Bin" if binned else self.devices[i['DeviceID']]
KeyError: 30121212 degoo_put: 30121212

[bernd-wechner]

Sounds like device issues to me. But do remember this is not a mature product under public release, but a simple personal hack that is a work in progress. To wit, it's generally not bad idea to install a Python ID and debugger and work on it and contribute ;-). l Some folk have been wonderful enough to do that. In fact I have one PR still to check but wanted quickly to help here if I can

I would suggest doing something like this:

$ ./degoo_pwd
Working Directory is /
$ ./degoo_ll
11450318017	Device     	Phone	 [phone]	c:2020-03-03 03:35:14	m:2020-06-05 06:40:27	u:2020-06-05 06:40:27
12568516976	Device     	Web  	7943781047	c:2020-06-09 14:03:59	m:2020-06-09 22:14:58	u:2020-06-09 22:14:58

So we can see what you situation is.

Then, a / is not writeable:

$ ./degoo_cd Web
Working Directory is now /Web

or into whatever Device you want. From which:

$ ./degoo_put --help
usage: Degoo CLI [-h] [-v] [-V] [-r] [-c] [-d] [-f] [-s] local [remote]

Degoo commands -- Some CLI commands to interact with a Degoo cloud drive

Created by Bernd Wechner on 2020-06-03.
Copyright 2020. All rights reserved.

Licensed under The Hippocratic License 2.1
https://firstdonoharm.dev/

Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.

USAGE:

positional arguments:
  local            The file/folder/path to put
  remote           The remote folder to put it in

optional arguments:
  -h, --help       show this help message and exit
  -v, --verbose    set verbosity level [default: 0]
  -V, --version    show program's version number and exit
  -r, --redacted   When outputting deep verbose debugging (-vvv), redact out personal security details for reporting on forums.
  -c, --config     Report the API configs before all else.
  -d, --dryrun     Show what would be uploaded but don't upload it.
  -f, --force      Force uploads, else only upload if changed.
  -s, --scheduled  Upload only when the configured schedule allows.

In trying this I found a small bug myself which I just fixed and pushed.

[lupinx]

./degoo_pwd
Working Directory is /

./degoo_ll
15906779581     Folder          demo                     0      c:2021-09-15 19:32:27   m:2021-09-15 19:32:26   u:2021-09-15 19:32:26
15914315623     Folder          fofo                     0      c:2021-09-16 23:07:09   m:2021-09-16 23:08:12   u:2021-09-16 23:08:12
15906773865     Image           Dart.png            248803      c:2021-09-12 19:14:02   m:2021-09-15 19:31:26   u:2021-09-15 19:31:26
15913092251     Image           fennec.png         1473721      c:2021-09-16 18:14:26   m:2021-09-16 18:16:44   u:2021-09-16 18:16:44
15904331971     Image           SoftDart.jpg         82768      c:2021-09-12 12:24:43   m:2021-09-15 12:39:28   u:2021-09-15 12:39:28

 ./degoo_cd fofo
Working Directory is now /fofo


./degoo_ll

1- Uploading the first file to the folder: (SUCCESS)

./degoo_put lion.jpeg /fofo

Uploading lion.jpeg to /fofo
100% [##############################################################################]     35K / 35K
Traceback (most recent call last):
  File "./degoo_put", line 305, in <module>
    sys.exit(main())
  File "./degoo_put", line 213, in main
    result = degoo.put(args.local, args.remote, args.verbose, not args.force, args.dryrun, args.scheduled)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 1194, in put
    return put_file(local_path, remote_folder, verbose, if_changed, dry_run, schedule)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 1098, in put_file
    degoo_id = api.setUploadFile3(os.path.basename(local_file), dir_id, Size, Checksum)
  File "/home/userx/workdir/degoo/degoo/degoo/API.py", line 850, in setUploadFile3
    contents = self.getFileChildren3(parent_id)
  File "/home/userx/workdir/degoo/degoo/degoo/API.py", line 585, in getFileChildren3
    prefix = self.devices[i['DeviceID']] + os.sep + "Recycle Bin" if binned else self.devices[i['DeviceID']]
KeyError: 30272619
degoo_put: 30272619
           for help use --help

2- Trying to upload another file to the folder: (FAILS)

./degoo_put tiger.jpeg /fofo

Traceback (most recent call last):
  File "./degoo_put", line 305, in <module>
    sys.exit(main())
  File "./degoo_put", line 213, in main
    result = degoo.put(args.local, args.remote, args.verbose, not args.force, args.dryrun, args.scheduled)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 1194, in put
    return put_file(local_path, remote_folder, verbose, if_changed, dry_run, schedule)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 962, in put_file
    if (not if_changed or has_changed(local_file, remote_folder, verbose - 1)):
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 631, in has_changed
    files = get_children(remote_path)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 604, in get_children
    __CACHE_CONTENTS__[dir_id] = api.getFileChildren3(dir_id)
  File "/home/userx/workdir/degoo/degoo/degoo/API.py", line 585, in getFileChildren3
    prefix = self.devices[i['DeviceID']] + os.sep + "Recycle Bin" if binned else self.devices[i['DeviceID']]
KeyError: 30272619
degoo_put: 30272619
           for help use --help

./degoo_ll

Traceback (most recent call last):
  File "./degoo_ll", line 305, in <module>
    sys.exit(main())
  File "./degoo_ll", line 112, in main
    degoo.ls(args.folder, args.long, args.human, args.recursive)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 1210, in ls
    items = get_children(directory)
  File "/home/userx/workdir/degoo/degoo/degoo/util.py", line 604, in get_children
    __CACHE_CONTENTS__[dir_id] = api.getFileChildren3(dir_id)
  File "/home/userx/workdir/degoo/degoo/degoo/API.py", line 585, in getFileChildren3
    prefix = self.devices[i['DeviceID']] + os.sep + "Recycle Bin" if binned else self.devices[i['DeviceID']]
KeyError: 30272619
degoo_ll: 30272619
          for help use --help

Only the first file succeed on upload, after that everything fails, even "degoo_ll"

fyi: i have a free degoo Plan.

[bernd-wechner]

Wowsers, you have apparently got files in the root folder. New to me that that is even possible? How did you get them there? The Degoo web app? With degoo_put?

What you describe as succeeded looks odd to me it's crashed with a stack trace. A very liberal definition of success you have there, from which I infer you mean the file actually did upload.

Before anything else I'd want to know what's happening here:

  File "/home/userx/workdir/degoo/degoo/degoo/API.py", line 585, in getFileChildren3

Alas line 585 mysides doesn't contain:

prefix = self.devices[i['DeviceID']] + os.sep + "Recycle Bin" if binned else self.devices[i['DeviceID']]

But line 623 does. I'd hat to go guessing , but the error is KeyError: 30272619 suggesting i['DeviceID'] is equal to 30272619 and that self.devices has no such entry. Which ties back to the root folder in which Devices reside.

I am beginning to suspect that the free plan works differently to the ultimate plan that I have and use and test with. A reasonable presumption at this point is that it does away with the device tier and the root folder is just THE device you have access to.

That would be an interesting learning if so.

As ever I would strongly suggest using a Python IDE and debugging it yourself. I can't reproduce the problem and I'm very short of time. But as I'm very curious (regarding the free plan hypothesis) I would consider trying it out some it. Not a priority for me alas in the coming month. I'd have to create me a free account and take these tools for a test spin. Which would only happen out of curiosity my sides and a service to you, as frankly I have an ultimate plan and coded this up to back up my disks on that plan (so no direct interest in them working on a free plan).

If it helps, the clues lead to this:

Degoo/degoo/API.py

Line 274 in 2eed880

def devices(self):

And I'd break therein in a debugger, or instrument the code with some prints.

But I'd also look in:

Degoo/degoo/API.py

Line 548 in 2eed880

def getFileChildren3(self, dir_id):

and find out how it got there, what it's trying to do etc. Time to lean ...

[lupinx]

Wowsers, you have apparently got file sin the root folder. New to me that that is even possible? How did you get them there? The Degoo web app? With degoo_put?

Degoo Web App

Alas line 585 mysides doesn't contain:
...

i used a previous version of API.py, indeed, in the last version the error shows up in line 623

honestly, i am not a python guy, but i will try some debugging once i finish some work i am on.

[bernd-wechner]

Fret not Python is a piece of cake, and generally fairly easy to read (the pedagogic language of choice in 2021). If you can contribute fixes that make this work for Free License users, you'll probably win some fans down the road ;-). Shouldn't be hard, the error is a trivial one, that seems to relate to a different way Degoo have of presenting the root folder with the two account types and the code may need to be robust for both scenarios. It's just that it's fast and easy to diagnose by breaking there in an ID and look at how it got there, stepping back up the stack trace and inspecting the logic that is at play.

[lupinx]

hello,
could you please check if you can still login and generate a new token without that 429 error ??
i cannot anymore ...

and can we talk in private please ?

[schunka]

hello, could you please check if you can still login and generate a new token without that 429 error ?? i cannot anymore ...

and can we talk in private please ?

Any info in this matter. I'm affected as well.
Cloudflares DDoS protection steps in and returns 429 for login and register methods.

[Kcchouette]

Hello

Thanks for the project

I have the same case of problem.
I don't follow all the discussion as some fork has disappear.

[bernd-wechner]

Not sure of status here. But a heads up that I took a look at fixing the login issues and fixed those here:

bc92782

though the whole package remains a WIP (work in progress) and under testing. Not least as the degoo API changed in the interim and I've also made some updates here, but am not done with testing and it's one of far too many pies I have my fingers in and tabs I have open ;-).

I'm not sure if that fixes this as well. But basically as time permits (and she's greedy and doesn't permit much) I will continue testing the API upgrades and ultimately look at the PR for parallism and FUSE I hope/