Comments (12)
Interesting question :) The Go http server should take care of closing the body but that will happen after the handler returns. Can you point me to where the requirement for the body to be closed before the context can be cancelled comes from?
from goa.
I am not sure why it is like that, but if we don't close the body, then we don't always get notified of the client cancelling on his side. It is not always the case, if just merely testing by directly connecting on the service with curl, it does work as expected (no need to close the body), but when we are in actual deployment behind a routing proxy (traefik in our case), not only we don't get notified, but even worst there seem to be leak of connections.
It's easy to miss the issue, because for it to happen, the client need to close his side before the method returned, so if all methods respond quickly, it won't happen very often. In our case though we have a long running POST method, and it happens all the time.
from goa.
Thanks for the explanation, one more question: does implementing a custom decoder that closes the body help? This would be a good test to validate the approach (might also be a better workaround than using SkipRequestBodyEncodeDecode
).
from goa.
Thanks for the suggestion, bizarrely I tried and didn't manage to fix the issue with a custom decoder. Maybe there's something I didn't completely understand, the custom decoder is actually a custom decoder factory, it returns a decoder for a given request, I don't fully understand why it is done like that, does it mean that the Decode method could be called more than once for a given decoder ? If so I should not close the body at the end of Decode (in general), because this could break subsequent calls.
Here is what I tried :
type BodyCloser struct {
request *http.Request
lower goahttp.Decoder
}
func (d *BodyCloser) Decode(v any) error {
err := d.lower.Decode(v)
d.request.Body.Close()
return err
}
func NewBodyCloserRequestDecoder(r *http.Request) goahttp.Decoder {
return &BodyCloser{r, goahttp.RequestDecoder(r)}
}
To complete the explanation, this is actually this issue we hit : golang/go#23262
It is a pretty old issue, while there's a working solution in closing the body, the issue is still open, so I'm not sure what it means, maybe the golang team consider this is not sufficiently documented ?
Anyway, what they are saying in that issue is that if you don't read the body, then you should close it.
from goa.
Hmm the above code looks good to me. The reason the decoder is a factory and not a specific decoder is so that the code can inspect the request Content-Type
header and use the proper decoding strategy (e.g. if an API was to support both JSON and XML). The factory is called on each request so it shouldn't be an issue to call Close
on the body. AFAIK the above implements the behavior described in the original message (i.e. closes the body before the handler is called).
from goa.
I see that makes sense.
Regarding fixing by using a custom decoder, it does not work, because there's nothing to decode in the body. That's what triggers the bug in the first place : if the body is not read, you need to close it (or you don't receive cancel), that's what is described in golang/go#23262. But the code generated by goa does not call the decoder in this case, so I can't implement the fix there.
from goa.
I verified, if I add an optional request parameter in the body in my design, then generated code use the decoder, which will read the body, but in this case the bug is not triggered, even without the custom decoder, because the body was read.
So maybe what goa should do (the generated code I guess) is when it does not use the decoder, close the body instead.
from goa.
That makes sense, would you care putting together a PR? Adding a else
to https://github.com/goadesign/goa/blob/v3/http/codegen/server.go#L588 might be a good starting point but there may need to be more changes.
from goa.
I would be happy to propose something. But I don't know what ...
It turns out that when using nginx as proxy, closing the body does not work. I need to make sure the body is read. For example by adding some dummy parameter, so generated code will call the decoder, or even again using SkipRequestBodyEncodeDecode and calling Read
on the body instead of Close
.
So, I guess for now I will stick to this workaround, and since I can also workaround #3328 , I am not blocked.
(note : reading just partially is not enough, you need to read up to EOF, in my case there was two bytes in the body, reading just one byte did not fix, but reading 100 (which returned only 2 + EOF) did. The two bytes were '{' and '}' )
from goa.
OK I thought the problem was that Goa wasn't closing the request Body when it is empty. The code I linked above is where the code checks whether the decoder should be called and I was suggesting adding a else
clause that would close the body if that check failed (and the decoder wasn't called).
from goa.
Well, I also thought that, but after digging deeper, it is not clear, since it does not resolve the issue when nginx is used as reverse proxy.
What seems to fix the issue is to make sure you consume the body fully. Normally the body should be almost empty (for example in our case it contains an empty json dict {} ). I'm a bit resilient to propose this workaround generically because it's not as straightforward as just closing the body.
Basically, this issue on golang side golang/go#23262 is still open. I am not sure what it means, do they plan to change the behaviour one day ? Or improve the doc ?
from goa.
Thank you, that makes sense. It does look like the need for reading the request body before a request can be closed is a hard limitation - good to know! Closing this issue since there isn't anything Goa can do to change this behavior.
from goa.
Related Issues (20)
- Is there a nice way to integrate AsyncAPI into goa? HOT 1
- Make endpointsData public HOT 1
- goa gen only generating structs for types contained in service definitions HOT 1
- Upstream indirect dependency change HOT 6
- Meta openapi:example still generates example if put at API level HOT 1
- Goa not enforcing required attribute HOT 6
- grpc stream.Recv doesn't decode errors HOT 1
- can't use SkipRequestBodyEncodeDecode and authentification at the same time HOT 7
- proto file(s) should be named with package name not service name (if package name defined)
- goa gen results in "package src/goa-test/calc is not in std" HOT 1
- chi: wildcard '*' must be the last value in a route. trim trailing text or use a '{param}' instead HOT 6
- objects HOT 1
- Meta("struct:pkg:path", "foo/bar") doesn't work. HOT 9
- how to return an existing type plus additional cookies ? HOT 6
- Method with SkipRequestBodyEncodeDecode and without payload generates invalid client
- setting the accept header doesn't seem to work HOT 5
- Cannot access help forum HOT 3
- Does Goa create client code as well as CLI code? HOT 10
- Support OpenAPI V3 YAML/JSON oneof HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goa.