~/n/t/lifetheuniverseandeverything (master|✚4…) (nerd-auth)ϵ❯❯❯ time -v nerd dataset download e1d00c6b73b78c01f8af ./output/test2 --debug
Recieved GitHub error message: API rate limit exceeded for 185.82.186.24. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) (https://developer.github.com/v3/#rate-limiting)
2017/06/28 11:54:50 dataset_download.go:81: Downloading dataset with ID 'e1d00c6b73b78c01f8af'
2017/06/28 11:54:50 client.go:64: [DEBUG] HTTP Request:
GET /v1/projects/dxlb-default/datasets/e1d00c6b73b78c01f8af HTTP/1.1
Host: batch.nerdalize.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzM4NCJ9.eyJpc3MiOiJhdXRoLm5lcmRhbGl6ZS5jb20iLCJhY2Nlc3MiOlt7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ0YXNrIiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJsaXN0IiwicnJlY2VpdmUiLCJkZXNjcmliZSIsInN0b3AiLCJzdGFydCIsInJwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJkYXRhc2V0IiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJkb3dubG9hZCIsImNyZWF0ZSIsImxpc3QiLCJ1cGxvYWQiLCJkZXNjcmliZSIsInVwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ3b3JrZXIiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImRlbGV0ZSIsImxpc3QiLCJ1cGRhdGUiXX0seyJzZXJ2aWNlIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJ0eXBlIjoid29ya2xvYWQiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImxpc3QiLCJkZWxldGUiLCJkZXNjcmliZSJdfV0sImV4cCI6MTQ5ODY0MzcyNywibmJmIjoxNDk4NjQzMzY3LCJpYXQiOjE0OTg2NDM0MjcsInN1YiI6IjY4In0.a8dea4LbuZoUnFK3C2Pail3FBkzekgSCHr3Ze82nRkke4h_XkQlGlz4XuDw-BPzHUIrmyVRJNN3HygSpHKCywLqhauVv_I-x3C2S857FoOR80PNFSuoMvKq4td5iw4WP
2017/06/28 11:54:50 client.go:78: [DEBUG] HTTP Response:
HTTP/2.0 200 OK
Content-Length: 263
Content-Type: application/json
Date: Wed, 28 Jun 2017 09:54:50 GMT
Via: 1.1 7f4dc1bba7bed3781d0d492ed99de258.cloudfront.net (CloudFront)
X-Amz-Cf-Id: kro9eaB9_sgdWrEeNLJli7UfQca8hV8taT8pQesN4wZPpDorjAyvUw==
X-Amzn-Requestid: d38374c7-5be7-11e7-8276-2524d0cb6728
X-Amzn-Trace-Id: sampled=0;root=1-59537cea-af70adc279df21840203e68a
X-Cache: Miss from cloudfront
{"project_id":"dxlb-default","dataset_id":"e1d00c6b73b78c01f8af","bucket":"f7c3f734-uniprod-advan.nerdalize.com","dataset_root":"dxlb-default/e1d00c6b73b78c01f8af","project_root":"dxlb-default","upload_expire":0,"upload_status":"SUCCESS","created_at":1498642404}
2017/06/28 11:54:50 client.go:64: [DEBUG] HTTP Request:
POST /v1/projects/dxlb-default/tokens HTTP/1.1
Host: batch.nerdalize.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzM4NCJ9.eyJpc3MiOiJhdXRoLm5lcmRhbGl6ZS5jb20iLCJhY2Nlc3MiOlt7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ0YXNrIiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJsaXN0IiwicnJlY2VpdmUiLCJkZXNjcmliZSIsInN0b3AiLCJzdGFydCIsInJwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJkYXRhc2V0IiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJkb3dubG9hZCIsImNyZWF0ZSIsImxpc3QiLCJ1cGxvYWQiLCJkZXNjcmliZSIsInVwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ3b3JrZXIiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImRlbGV0ZSIsImxpc3QiLCJ1cGRhdGUiXX0seyJzZXJ2aWNlIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJ0eXBlIjoid29ya2xvYWQiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImxpc3QiLCJkZWxldGUiLCJkZXNjcmliZSJdfV0sImV4cCI6MTQ5ODY0MzcyNywibmJmIjoxNDk4NjQzMzY3LCJpYXQiOjE0OTg2NDM0MjcsInN1YiI6IjY4In0.a8dea4LbuZoUnFK3C2Pail3FBkzekgSCHr3Ze82nRkke4h_XkQlGlz4XuDw-BPzHUIrmyVRJNN3HygSpHKCywLqhauVv_I-x3C2S857FoOR80PNFSuoMvKq4td5iw4WP
{"project_id":"dxlb-default"}
2017/06/28 11:54:51 client.go:78: [DEBUG] HTTP Response:
HTTP/2.0 200 OK
Content-Length: 740
Content-Type: application/json
Date: Wed, 28 Jun 2017 09:54:51 GMT
Via: 1.1 7f4dc1bba7bed3781d0d492ed99de258.cloudfront.net (CloudFront)
X-Amz-Cf-Id: fqfuE9CZ4HaN-7f_-U6Vzr293KzTHCtAB3qAH65ATjPB8POgmBu4iQ==
X-Amzn-Requestid: d3b448d5-5be7-11e7-84b2-37f180a0157c
X-Amzn-Trace-Id: sampled=0;root=1-59537cea-380621c4e9597ef8d98f6695
X-Cache: Miss from cloudfront
{"aws_region":"eu-west-1","aws_access_key_id":"ASIAINILN6Q6TXZ7OZTA","aws_expiration":"2017-06-28T21:54:51Z","aws_secret_access_key":"+zMAiSavrmcwYEgUSYm8Mid1DuXFNKbvSCV0wLAp","aws_session_token":"FQoDYXdzEFsaDHWpnbK34pM4bQ84tSL1AqneWwPCsmFNQxJ2dwnliLuABtcemjsWF58B10YfUfKW5jrZOjZ3W323ETs3te4f4xp54nxDmBUqYkBCZfH3R9OFgLqHxmh9Z6+nCL8RBqUqKd4pm8MUkq1EH6CyCVT0KLeXORRoqKjkfGu5SuqLdebhb2owKf3OtIyjcnb4t/xsSkE+dG5P3WGIocj1UHi1nGtNhwV0zCYKDA5dTk2ngdnx+zh7gn1FSDjecyjz37qUvTE2MiJj968w3XSjEwpjDp6pAgM2nzFa6Qg6AwsKiVge5JIdqNVtUZpzdo0f3HNV97adBo/EbtRgGTfGlyg8fc0Qqn8k8YWJnNVsdOmQkb3Wy4Oxh47LYk2rcBOpboeaRW5Uq+U9kj841LrdCj+8ZY1l5ZbbpMt6yCADlt9DJlwIWfRYNfhUTdp3WgJ65X+lHhtfd1H1Oa5aQNU4OB9Ajbsx6dmeUkP09P3n950pRyWcpMkpEE667+flnXsUyNaRELwZoTIo6/nNygU="}
2017/06/28 11:54:51 client.go:64: [DEBUG] HTTP Request:
GET /v1/projects/dxlb-default/datasets/e1d00c6b73b78c01f8af HTTP/1.1
Host: batch.nerdalize.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzM4NCJ9.eyJpc3MiOiJhdXRoLm5lcmRhbGl6ZS5jb20iLCJhY2Nlc3MiOlt7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ0YXNrIiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJsaXN0IiwicnJlY2VpdmUiLCJkZXNjcmliZSIsInN0b3AiLCJzdGFydCIsInJwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJkYXRhc2V0IiwiYWRkcmVzcyI6Im5jZS5uZXJkYWxpemUuY29tIiwicmVzb3VyY2VfaWRlbnRpZmllciI6ImR4bGItZGVmYXVsdCIsInJpZ2h0cyI6WyJkb3dubG9hZCIsImNyZWF0ZSIsImxpc3QiLCJ1cGxvYWQiLCJkZXNjcmliZSIsInVwcm9ncmVzcyJdfSx7InNlcnZpY2UiOiJuY2UubmVyZGFsaXplLmNvbSIsInR5cGUiOiJ3b3JrZXIiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImRlbGV0ZSIsImxpc3QiLCJ1cGRhdGUiXX0seyJzZXJ2aWNlIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJ0eXBlIjoid29ya2xvYWQiLCJhZGRyZXNzIjoibmNlLm5lcmRhbGl6ZS5jb20iLCJyZXNvdXJjZV9pZGVudGlmaWVyIjoiZHhsYi1kZWZhdWx0IiwicmlnaHRzIjpbImNyZWF0ZSIsImxpc3QiLCJkZWxldGUiLCJkZXNjcmliZSJdfV0sImV4cCI6MTQ5ODY0MzcyNywibmJmIjoxNDk4NjQzMzY3LCJpYXQiOjE0OTg2NDM0MjcsInN1YiI6IjY4In0.a8dea4LbuZoUnFK3C2Pail3FBkzekgSCHr3Ze82nRkke4h_XkQlGlz4XuDw-BPzHUIrmyVRJNN3HygSpHKCywLqhauVv_I-x3C2S857FoOR80PNFSuoMvKq4td5iw4WP
0 B / 600.00 MiB [-----------------------------------------------------------------------------------------------------------------------] 0.00%2017/06/28 11:54:51 client.go:78: [DEBUG] HTTP Response:
HTTP/2.0 200 OK
Content-Length: 263
Content-Type: application/json
Date: Wed, 28 Jun 2017 09:54:51 GMT
Via: 1.1 7f4dc1bba7bed3781d0d492ed99de258.cloudfront.net (CloudFront)
X-Amz-Cf-Id: QlqsrvN99F5vy9zuLeIPhEN-H4Xh9_l50LbBBpFHpuBoyebYivujFQ==
X-Amzn-Requestid: d42ea93c-5be7-11e7-ac2d-4bd068f7c3f2
X-Amzn-Trace-Id: sampled=0;root=1-59537ceb-8fa502c7234d53f09af981bd
X-Cache: Miss from cloudfront
{"project_id":"dxlb-default","dataset_id":"e1d00c6b73b78c01f8af","bucket":"f7c3f734-uniprod-advan.nerdalize.com","dataset_root":"dxlb-default/e1d00c6b73b78c01f8af","project_root":"dxlb-default","upload_expire":0,"upload_status":"SUCCESS","created_at":1498642404}
210.12 MiB / 600.00 MiB [=====================================>----------------------------------------------------------------------] 35.02% 39s2017/06/28 12:00:09 outputter.go:143: failed to download dataset 'e1d00c6b73b78c01f8af'
210.12 MiB / 600.00 MiB [=====================================>--------------------------------------------------------------------] 35.02% 5m18s2017/06/28 12:00:09 outputter.go:147: [DEBUG] Underlying error: read tcp 10.1.10.159:39778->52.218.20.36:443: read: connection reset by peer
failed to copy chunk to byte buffer
github.com/nerdalize/nerd/nerd/service/datatransfer/v1.downloadChunks.func1
/home/dexter/go/src/github.com/nerdalize/nerd/nerd/service/datatransfer/v1/download_process.go:91
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:2197
failed to download dataset 'e1d00c6b73b78c01f8af'
github.com/nerdalize/nerd/command.(*Download).DoRun
/home/dexter/go/src/github.com/nerdalize/nerd/command/dataset_download.go:101
github.com/nerdalize/nerd/command.(*Download).DoRun-fm
/home/dexter/go/src/github.com/nerdalize/nerd/command/dataset_download.go:35
github.com/nerdalize/nerd/command.(*command).Run
/home/dexter/go/src/github.com/nerdalize/nerd/command/command.go:103
github.com/nerdalize/nerd/vendor/github.com/mitchellh/cli.(*CLI).Run
/home/dexter/go/src/github.com/nerdalize/nerd/vendor/github.com/mitchellh/cli/cli.go:154
main.main
/home/dexter/go/src/github.com/nerdalize/nerd/main.go:75
runtime.main
/usr/local/go/src/runtime/proc.go:185
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:2197
Command exited with non-zero status 1
Command being timed: "nerd dataset download e1d00c6b73b78c01f8af ./output/test2 --debug"
User time (seconds): 3.07
System time (seconds): 3.80
Percent of CPU this job got: 2%
Elapsed (wall clock) time (h:mm:ss or m:ss): 5:20.26
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 292664
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 54610
Voluntary context switches: 330951
Involuntary context switches: 2450
Swaps: 0
File system inputs: 263
File system outputs: 258451
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 1