Limit download length (size) but still make use of decoder and parser about needle

[tomas]

You're welcome. To stop at a specific point (e.g. after 10000 bytes have been transferred) you'd need to count the chunk lengths and stop the stream whenever over that limit. Usually I have no problem implementing stuff on Needle as requested, but in this case it looks like a really specific use case.

What are you trying to do, if I may ask?

[bitinn]

Thx for the response, I am just trying to limit download size for potential memory issue.

Say I have a open system that allow user submit url so we fetch content of that link. Then potentially one could construct a large json file and make an attempt to fetch it, which probably will crash the worker instance due to memory issue. Using stream we can effectively avoid such problem.

Though I was wrong at assuming decoder required buffering to work with chunk, it would seem that only parser require buffering. So this might not be an issue after all, I will give it a go first and report back.

[bitinn]

@tomas things are working fine, just 1 question, is stream.request.abort() is right way to close a on-going needle request when my limit is reached?

[tomas]

As far as I know, yup, that should do it.