Detect encoding from HTML/XML <meta> header about needle

[tomas]

Sorry for not responding earlier. Would you submit a pull request including this change?

[leesei]

This would introduce another dependency of charset, is this okay?
mime checking at needle.js would also be changed.

[tomas]

Hmm. I guess you're right. In the end it's as simple as doing this, right?

if (!mime.charset && mime.type == 'text/html') {
  var found = body.toString().match(/[charset|encoding]=([^\"]+)/);
  if (found) mime.charset = found[1];
}

I don't think we really need another dependency.

[leesei]

Yup, I would also suggest using body.toString().slice(1, 1024).match() to look for the first 1024 bytes only.

[tomas]

Ok, thanks.

[leesei]

I was studying the latest code and didn't realize the meta handling in decoder.js was added because of me 😛.

[fritx]

heys @tomas how about including jschardet for charset auto-detect, if not found in <meta>?
at https://github.com/tomas/needle/blob/master/lib%2Fneedle.js#L105
i'm currently using the library and all seems well now