Invalid input error with unicode values in tablestorage about azure-sdk-for-node

[brenoferreira]

I'm getting this same error here. Taking a quick look at the source code, I think it's missing an "Accept-Encoding" or "Content-Encoding" Headers (set to utf-8) in the webResource object in the insertEntity function of TableService. It's just a guess. I'll run some tests here and see if I can figure it out.

[brenoferreira]

I've been trying to find the problem but until now, no success. I've traced the request data (including headers and content), and everything seems to be ok.
During my tests, I found out that when inserting items in batch when there's some unicode values, it says the entities were inserted, when in fact, they weren't. I'll keep trying to find what's wrong.

[einaros]

Could you provide a test case which reproduces the problem reliably?

[brenoferreira]

I've created a gist with a code example you can use to reproduce: https://gist.github.com/1672240

[einaros]

Awesome, I'll give it a go.

[brenoferreira]

Great! I'll keep trying to find something here!

[einaros]

Alright, I've found the issue. Now I just need to find a place to charge my laptop before figuring out how to patch.

[einaros]

Right, the issue is the content length set for all the xml data. It should be byte length, not char length.

See all the xml sending methods in tableservice.js. They will be setting it such as:

  webResource.addOptionalHeader(HeaderConstants.CONTENT_LENGTH, entityXmlDescriptor.length);

The fix is to replace all of these with:

  webResource.addOptionalHeader(HeaderConstants.CONTENT_LENGTH, Buffer.byteLength(entityXmlDescriptor, 'utf8'));

[einaros]

I'd be happy to send a pullreq, but will have to find a power source first.

[einaros]

Feel free to try my fork, at https://github.com/einaros/azure-sdk-for-node .. it should be fixed there. I haven't run the test cases yet, though, so no pullreq.

[andrerod]

Great stuff @einaros ! Your fix is absolutely correct. When you can just generate the pul request and ill accept it.

Thanks
Andre

[brenoferreira]

Awesome @einaros ! It totally makes sense.

Thank you so much!
Breno

[brenoferreira]

I've been looking at the source code, and I think this problem seems to be present in some other modules as well, such as the QueueService and BlobService.

[einaros]

It is. I've started writing test cases and fixes for the other pieces too.

[einaros]

There we go. Updated the pullreq.

[andrerod]

This is now fixed in the dev branch.