Needle parses in streams mode about needle

[tomas]

Yes, that's the expected behaviour. Were you expecting something different?

The option is parse: false by the way!

[terinjokes]

Sorry, I typo'd the issue here. 😀

The service I'm interacting with returns files, which occassionally can be JSON files, which should be saved to disk. Doing a simple needle.get and piping to fs.createWriteStream was successful until I encountered this case.

The only documentation around piping in this module shows a web request being written to disk via the same manner. Therefore it "feels" like a bug when this is violated by the stream being knocked into object mode by the JSON or XML parsers.

[tomas]

I'm wondering, what makes a JSON file different from a JSON string? As far as I can see, to Needle it's basically a JSON object that should be parsed, given that the response's Content-Type says so.

I think this is more an edge case than an actual bug. :)

[tomas]

@terinjokes I'm closing this one for the time being. Feel free to reopen if you think there's a better way of handling this.

[monolithed]

var stream = needle.request('post', url, {}, {
    decode: false,
    parse: false
});

stream.pipe(response);

stream.on('end', function () {
      // Never be fired without `parse: false`
});

@tomas, Please pay attention to this point in the documentation!

[tomas]

@monolithed Is that so? What URL are you testing with?

[monolithed]

@tomas, public url like https://e.mail.ru

[tomas]

@monolithed Can you share a full code example? I don't know where the response object you show comes from.

[monolithed]

@tomas,

app.use(express.cookieParser());

app.use(function (request, response, next) {
    get_cookies(function () {
          request.cookies = result.cookies;
          next();
   });
});

app.use('/api', function (request, response, next) {
    var url = 'https://e.mail.ru/api' + request.url;

    var stream = needle.request(request.method, url, request.body, {
        cookies: request.cookies,
        parse  : false
    });

    stream.pipe(response);
});

Could not close connection without parse: false. It seems very strange...

[tomas]

Ok, gonna test it now. Which needle version are you running?

[tomas]

I just tested and running the latest Needle (from git, not npm) I do get the 'end' event emitted. This might have to do with a few recent changes related to the Connection header.

I haven't pushed a new version to NPM because I need to update the documentation first. However you can safely run the latest version of git since all new major changes have been thoroughly covered with tests.

[monolithed]

0.11.0, thanks