will this support streaming or encode? about needle

[tomas]

I don't think I understood your question. Can you show the code your using?

[atian25]

I means when use this module to download some url with charset=utf-8 , such as www.baidu.com,
the chinese words is unreadable.

and I saw other module use iconv-lite to encode, or provide a stream method to allow user to encode buffer.

[tomas]

Ok, got it. Can you show an example of how iconv-lite is used? I'd be happy to merge it into the codebase.

[atian25]

https://github.com/andris9/fetch/issues/4
https://github.com/ashtuchkin/iconv-lite

[tomas]

Ok, looks simple, but it seems we'd need to pass a buffer to iconv.decode() and Needle currently handles hte response as a plain string. I just pushed a commit that grabs the charset from the content type header, but if possible I'd like you to fork and make a pull request, to make sure we get it right (I can only say Ni-hao in chinese, to start). :)

[tomas]

Ok, just pushed a new brach called iconv and though everything looks right, I just cant get the same output for baidu.com as returned by cURL. Any pointers?

[atian25]

hi, I had check out the iconv branch and test, everything is ok now, thanks

[tomas]

Are you sure? The version I get of baidu.com is slightly different from the one I get via cURL.

[atian25]

env

• win7
• chinese
• cmd encoding is gbk
• node v0.8.7

target:

• www.baidu.com , (warn, it has two page)
• one is new : http://www.baidu.com/home/show/home (use charset=utf-8)
• one is old: http://www.baidu.com/home/page/show/classic (use charset=gb2312)
  all of them will redirect to url www.baidu.com

step1:

• use master branch code, change the test.js#simple_get to www.baidu.com
• output :

  <title>�ٶ�һ�£�����֪��      </title>
  

step2:

• use iconv branch code, change the test.js#simple_get to www.baidu.com
• output :

  <title>百度一下，你就知道      </title>
  

another bug

• when use icon branch, simple_get http://www.baidu.com/home/show/home
• error:

  
  E:\Workspace\Git\needle>node test/test.js get
  E:\Workspace\Git\needle\lib\needle.js:151
        charset = header && header[1].replace('charset=', '').trim();
                                      ^
  TypeError: Cannot call method 'replace' of undefined
  at ClientRequest. (E:\Workspace\Git\needle\lib\needle.js:151:41)
  at ClientRequest.g (events.js:185:14)
  at ClientRequest.EventEmitter.emit (events.js:88:17)
  at HTTPParser.parserOnIncomingClient [as onIncoming](http.js:1455:7)
  at HTTPParser.parserOnHeadersComplete [as onHeadersComplete](http.js:111:23)
  at Socket.socketOnData [as ondata](http.js:1366:20)
  at TCP.onread (net.js:402:27)
  
  

[tomas]

Fixed! Try again and see if it works. :)

[atian25]

yes, works.
I hate redirect by location.href ... which can't follow by code.

    E:\Workspace\Git\needle>node test\test.js get
    null
    Got status code 200
   <script>
    location.href="http://www.baidu.com/";
   </script>

[atian25]

when will this publish to npm?

[tomas]

Soon! I'll be out of the office all day tomorrow. :(

[monolithed]

Have the same problem
Solved:

needle.post(URL, { 'mode': '1' }, { decode: false }, function () {})