Intermittent errors in deploy error handling about cli

[schickling]

Same here when running via CircleCI. Unfortunately this blocks us from using Netlify.

[aj0strow]

I actually fixed it twice, while I was waiting for the first one to go through Circle Ci.

• https://github.com/aj0strow/netlify-cli
• https://github.com/aj0strow/netlifyctl

Hope this helps.

[fool]

Hi @schickling and @ilkka - I think we just fixed the longer term problem with larger deploys. Can you give v1.1.0 a spin and let us know how it goes?

[fool]

We've hit some walls in debugging this and want to take it to CircleCI - but are not paying customers.

Does one of you who is a paying customer mind writing in with a real ticket (rather than a forum post which is all we can do for free) and cc:ing myself ([email protected]) and our buildmaster ([email protected]) so we can present our evidence and ask them for help? After that point we probably don't actually need you to spend any time at it, just trying to get in the door :)

If someone does it please comment here so we don't hit them multiple times in parallel. (cc @schickling and @ilkka)

[rybit]

That is definitely part of an ongoing discussion we're having, but it will definitely be supported for the foreseeable future. Idealy we would bundle it inside a node module, but that is still a ways out.

[schickling]

Any idea @biilmann @cassiozen?

[biilmann]

I pushed an update to netlify-cli yesterday that mainly pulls in an updated version of the node client library. The fix in the node client was related to deploys with many concurrent uploads failing with that kind of error. Can you make sure you're on netlify-cli v1.0.3 and let me know if that version still has the problem?

[schickling]

Hey @biilmann. I've installed the netlify-cli today, so it's using version 1.0.3. Could there be a different issue?

[fool]

ah, I filed #13 today - to give us some better debugging tools around this same issue.

[schickling]

Any update on this? 80% of our builds are seeing this error...

[ilkka]

Thanks, we'll bump our dependency and see how it goes!

[fool]

OK, this is still an issue with version 1.1.0 and I've now seen it firsthand and think we (@netlify) will be able to reproduce well enough to debug. Is there anyone attached to this issue (mainly not sure about @ilkka ) who isn't using CircleCI when they get the error?

It felt to me like the "working" deploys completed at least an order of magnitude faster than the failing ones, so while I'm not suggesting this as a long-term workaround or a fix, you may get some mileage out of a stopgap with something like the first answer here to restart a failed deploy rather than having to redo the build:

http://stackoverflow.com/questions/5161193/bash-script-that-kills-a-child-process-after-a-given-timeout

[fool]

hmm, haven't heard back from anyone yet but would like to make some progress on this - @schickling any chance you can email Circle and cc: us?

[schickling]

Sorry we're also not on the needed pricing plan and they haven't responded to our previous inquiries yet.

[ilkka]

Yeah, sorry, we're still only freeloaders on CircleCI too.

[fool]

seems to be happening to TravisCI customers as well sometimes, grr! (https://app.intercom.io/a/apps/q245f50x/inbox/conversation/6853522502)

[schickling]

Yeah, it's gotten pretty bad for us. Now happening for 9/10 builds for some repos...

[schickling]

Any progress on this?

[rybit]

I was looking for this ticket!
Ok ~ so we went and built up a simple cli tool in go based on our open-api client. It doesn't have feature parity with the current cli, but we are slowly working on that. I built up releases for linux and osx too. I can build more, but just haven't had the motivation :)

check it out! https://github.com/netlify/netlifyctl

If you want to see an example of using it in circle-ci check out this https://github.com/rybit/circle-ci-test-repo

[schickling]

Thanks for this update @rybit! Are there any plans to phase out the JS version of netlify-cli in favor of the Go implementation or is this rather considered as a workaround for now?

[schickling]

@aj0strow can you create a PR for the original netlify-cli tool? 🙂

[jppurcell9]

I'm also seeing this error every time using bitbucket pipelines. The call that hangs for me is the createDeploy call in node-client: https://github.com/netlify/node-client/blob/22117378564500d082b281f4e8c0f62e4ca5fe56/lib/site.js#L144

It's failing with a socket hangup, which looks like it's actually making the connection but not receiving any data. Do you see the connection being made on your side? I'm able to make the GET request for the site before this call, so at least something is making a connection.

Here's the request data if it helps (without the 161 files from the post body):

making request /sites/<my site id>/deploys {"files":{...},"draft":false}
{ method: 'post',
  hostname: 'api.netlify.com',
  path: '/api/v1/sites/<my site id>/deploys',
  port: 443,
  headers: 
   { 'Content-Type': 'application/json',
     'Content-Length': 12835,
     Authorization: 'Bearer $NETLIFY_TOKEN' },
  auth: null }

[jppurcell9]

I was able to hack something together to get this to work, but I'm still not sure of the root cause. I found that simply retrying the request in node wasn't doing the trick. Instead I was able to fall back to curl to make the request instead. It's a total hack, but the code is here: https://github.com/jppurcell9/node-client/tree/feature/retry-if-no-data

[rybit]

@jppurcell9 have you tried the go client? It might be easier than subshelling out to curl

[jppurcell9]

@rybit I got the go client to work. Thanks!

[rozhok]

All of sudden I've starting to get the same error in my jenkins machine. My build is very small and consists only of few dozens of files less than 2 MiBs in total.
Also, when tried to upload stuff from my laptop and faced same issue. Will give a try for go client instead.

[schickling]

Any ETA on when the go binary will be shipped via NPM? It's a pretty big deal for us.

[rozhok]

@schickling from what I understand, CircleCI allows to install many runtimes/libs into build image and also allow to download custom binaries so you could just use netlifyctl binaries taken from https://github.com/netlify/netlifyctl/releases

[schickling]

@rozhok it's rather a matter of convenience/maintainability since we're mainting dozens of repos using Netlify for deployment.

[BorisVaskin]

Having the same issue on Visual Studio Team Services build system. The netlify deploy is almost always failing with mentioned error.

2017-05-01T16:26:46.5666661Z ##[command]netlify deploy -s ******** -p C:\a\1\a -t ********
2017-05-01T16:26:47.7018809Z Deploying folder: C:\a\1\a
2017-05-01T16:28:47.8972080Z { client: 
2017-05-01T16:28:47.8972080Z    { access_token: '********',
2017-05-01T16:28:47.8972080Z      client_id: '********',
2017-05-01T16:28:47.8982076Z      client_secret: undefined,
2017-05-01T16:28:47.8982076Z      redirect_uri: undefined,
2017-05-01T16:28:47.8982076Z      ENDPOINT: 'https://api.netlify.com',
2017-05-01T16:28:47.8982076Z      VERSION: 'v1',
2017-05-01T16:28:47.8982076Z      hostname: 'api.netlify.com',
2017-05-01T16:28:47.8982076Z      ssl: [ 'https://', index: 0, input: 'https://api.netlify.com' ],
2017-05-01T16:28:47.8982076Z      port: 443,
2017-05-01T16:28:47.8982076Z      http: 
2017-05-01T16:28:47.8982076Z       { Server: [Object],
2017-05-01T16:28:47.8982076Z         createServer: [Function],
2017-05-01T16:28:47.9052085Z Potentially unhandled rejection [3] SyntaxError: Unexpected token E in JSON at position 0
2017-05-01T16:28:47.9052085Z         globalAgent: [Object],
2017-05-01T16:28:47.9062085Z     at Object.parse (native)
2017-05-01T16:28:47.9062085Z         Agent: [Object],
2017-05-01T16:28:47.9072087Z     at Object.exports.log (C:\NPM\Modules\node_modules\netlify-cli\lib\helpers\error_logger.js:11:21)
2017-05-01T16:28:47.9072087Z         request: [Function],
2017-05-01T16:28:47.9072087Z     at C:\NPM\Modules\node_modules\netlify-cli\lib\commands\deploy.js:144:17
2017-05-01T16:28:47.9072087Z         get: [Function] } },
2017-05-01T16:28:47.9072087Z     at tryCatchReject (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\makePromise.js:845:30)
2017-05-01T16:28:47.9072087Z   data: 
2017-05-01T16:28:47.9072087Z     at runContinuation1 (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\makePromise.js:804:4)
2017-05-01T16:28:47.9072087Z    { Error: socket hang up
2017-05-01T16:28:47.9072087Z     at Rejected.when (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\makePromise.js:625:4)
2017-05-01T16:28:47.9082084Z        at createHangUpError (_http_client.js:254:15)
2017-05-01T16:28:47.9082084Z     at Pending.run (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\makePromise.js:483:13)
2017-05-01T16:28:47.9082084Z        at TLSSocket.socketOnEnd (_http_client.js:346:23)
2017-05-01T16:28:47.9082084Z     at Scheduler._drain (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\Scheduler.js:62:19)
2017-05-01T16:28:47.9082084Z        at emitNone (events.js:91:20)
2017-05-01T16:28:47.9082084Z     at Scheduler.drain (C:\NPM\Modules\node_modules\netlify-cli\node_modules\when\lib\Scheduler.js:27:9)
2017-05-01T16:28:47.9082084Z        at TLSSocket.emit (events.js:185:7)
2017-05-01T16:28:47.9082084Z     at _combinedTickCallback (internal/process/next_tick.js:67:7)
2017-05-01T16:28:47.9082084Z        at endReadableNT (_stream_readable.js:974:12)
2017-05-01T16:28:47.9082084Z        at _combinedTickCallback (internal/process/next_tick.js:74:11)
2017-05-01T16:28:47.9082084Z        at process._tickCallback (internal/process/next_tick.js:98:9) code: 'ECONNRESET' },
2017-05-01T16:28:47.9092083Z   meta: null }

[schickling]

Any update on this @netlify? This is still a huge pain for us.

[fool]

While pull requests are welcomed, we will probably not put a lot more development effort into netlify-cli

https://github.com/netlify/netlifyctl is the tool that can reliably deploy today, and will definitely be developed further in the future.

[bcomnes]

Closing since this issue appears stale. Feel free to reopen if we run into similar issues on the 2.0.0 node cli release.

[Cjay123]

After deploying my project to netlify. I saw something wrong to my website. My imported assets cant read by netlify but i can run it well on localhost. Anything can help me about my problem?

[bcomnes]

Sorry @Cjay123, doesn't ring a bell. If you have a reproducible problem you can demonstrate in an issue, gist or repo, please open an issue.

[worrawutp]

I have this problem when I do $ netlify deploy --prod on the first time. It show something like...

Deploying to live site URL...
√ Finished hashing 7 files
√ CDN requesting 7 files
√ Finished uploading 7 assets
| Waiting for deploy to go live... »   Warning:
 »   {
 »      "message": "request to
 »   https://api.netlify.com/api/v1/sites/04c14fe2-e27a-4ed4-b10c-f81a2bfbd27c/deplo
 »   ys/5c6ed79fef12041c97a94db0 failed, reason: read ECONNRESET",
 »      "type": "system",
 »      "errno": "ECONNRESET",
 »      "code": "ECONNRESET",
 »      "name": "FetchError",

but, after few seconds I run the same command again it then work fine.
and I got

Deploying to live site URL...
√ Finished hashing 7 files
√ CDN requesting 0 files
√ Finished uploading 0 assets
√ Deploy is live!

Not sure if it help to point out to the cause of the problem...

[bcomnes]

Nice catch @worrawutp, we should make the polling step while waiting for the deploy to finish up more resilient to random errors or transport errors.