Wrong media hash about enml-js

[wanasit]

Hi, Zack

I want to help you solving the problem. But, I can understand your test file you posted.

Can you upload the test file as .txt or .json and post the URL for me to download instead.

[zack9433]

Hello berrybo

The problem note is just evernote welcome message note.

If you create a new account on sandbox.evernote.com, and you have a default welcome message note,

I try to translate this note content to html format but the image(e.g. Evernote log) do not translate correctly.

http://dl.dropbox.com/u/16706203/note.zip

above content is evernote log image

thanks for your reply

[wanasit]

Probably, the resources object wasn't created properly.
Please refer to example's note2

You need to create resources object as map of { bodyHash : URL },
then pass it to enml.HTMLOfENML()

//====================================

var note2 = fs.readFileSync('../note2.json','utf8');
var shardId = '48' //HARDCODE...

var resources = {};
for(var i in note2.resources){
var resource = note2.resources[i];
resources[resource.data.bodyHash] = enml.URLOfResource(resource.guid, shardId);
}

//====================================

NOTE : The shadId is your (the note owner) shadId. Thus, it may not be 48.

[zack9433]

Can you use following evernote image to create a new note and execute enml.HTMLOfENML() to parse it correctly?

[wanasit]

OK, Zack. I got your problem.
Here, the bad news.

The problem is related to Javascript character encoding. And, I need very long time to fix it.

Here is QUICK & DIRTY way to work around the problem.
Change example's note2 to

var resources = {};
for(var i in note2.resources){
    var resource = note2.resources[i];

    var hash = ''
    for(var i=0;i<resource.data.bodyHash.length;i++){

    if(resource.data.bodyHash.charCodeAt(i) < 128 || resource.data.bodyHash.charCodeAt(i) == 65533 )
            hash += resource.data.bodyHash.charAt(i)
    else
            hash += String.fromCharCode(65533) + String.fromCharCode(65533);
    }

    resources[hash] = enml.URLOfResource(resource.guid, shardId);
}

[nzjony]

Hey BB,

Thanks for your library. I'm using it at the moment on a node.js project, and the HTMLTOENML method is a lifesaver 😄

However, I too had a problem with the hashes, I had the following code:

for(var resIndex in response.resources) {
    var resource = response.resources[resIndex];
    resources[resource.data.bodyHash] = webApiUrlPrefix + 'res/' + resource.guid;
}

A call to the 'HTMLOfENML' function would then fail. I managed to get it working ok if by simply changing
your function 'BodyHashOfENMLHash' and removing the check for a value greater than 128. I.e:

function BodyHashOfENMLHash(enmlHash){

    var buffer = [];
    for(var i =0 ; i<enmlHash.length; i += 2) 
        buffer.push( parseInt(enmlHash[i],16)*16 + parseInt(enmlHash[i+1],16));

    var bodyHash = '';
    for(var i =0 ; i<buffer.length; i ++){
        bodyHash += String.fromCharCode(buffer[i]);
    } 

    return bodyHash;
}

Will that work or am I missing something? I don't quite understand why you take values 128 or over, and assign them to the Unicode value 65533.

[wanasit]

Hi nzjony,

Ok. If you have follow zack9433's issue, you will note that the problem come from character encoding.

It's quite hard to explain. (I still don't fully understand the problem either.)

The library is designed to run on both Front-end and Node.js.

When we work on Front-end (fetching the note, then rendering it in browser),

$.get('../note', function(note) {

   for(var resIndex in note.resources) {
      var resource = response.resources[resIndex];
      resources[resource.data.bodyHash] = webApiUrlPrefix + 'res/' + resource.guid;
   }

Since we normally send data through the browser as JSON, resource.data.bodyHash would be damaged and forcefully encoded to ASCII. Thus, it would have :

• 1-128 (ASCII range)
• 65533 (Undefined)

That's why (when working on Browser) I need to check 128 or over and assign them to the Unicode value 65533.
The code on my previous is the quick fix when working on Node.js. #1 (comment)

Since, you are working on node.js application, I believe that your fix will work fine too.

[nzjony]

Ok, I hadn't considered that this library would need to work in the browser as well, thanks for clearing that up. If thats an issue, you would think that the evernote SDK would mention that the bodyHash could be corrupt, I had a look, but found nothing, props to you though for getting a solution.

[jguepin]

The BodyHashOfENMLHash still does not match some resources for me, using enml-js last version on nodejs. Last version was working better.

[nzjony]

To be honest, I have the same issue, go to http://www.jonathanstichbury.com/post/Vanuatu and search in the HTML for 6e2a9628b09ecad0bf427cf47ce5d622. That image doesn't render, but the other 3 do... strange.

[wanasit]

Yes. The code is not perfect yet. It still can not handle all of the hash. (But I quite believe this one work better, @Chmod0, please check the example again, and don't forget to remove the fix that I suggested you last month)

And I'm sorry to tell you that I don't have time to work on it this week.
If you want to help fixing it, please refer to function BodyHashOfENMLHash in enml.js

  if( buffer[i] < 128 )
    bodyHash += String.fromCharCode(buffer[i]);
  else if(buffer[i] < 0xC0)
    bodyHash += String.fromCharCode(65533); // UNKNOWN
  else if((i+1 < buffer.length) && ((buffer[i+1] & 0xC0) == 0x80) ){
    //UTF-8 2-bytes encoding
    var charcode = (buffer[i] & 0x1f)*64 + (buffer[i+1] & 0x3f);
    bodyHash += String.fromCharCode(charcode);
    i += 1;
  }

The code has already handled 1-byte & 2-byte encoding. But there are also 3,4,5,6 -byte cases.
http://en.wikipedia.org/wiki/UTF-8

And also, it would be nice if you could help sending me some example of the wrong media hash.

[phuoc-insignia]

I'm able to render the image with this PR
#16