Browser support about nbt-js

[deathcap]

browserify uses a compatible Buffer module https://github.com/feross/buffer so that's no problem. However its zlib module (think this is it: https://github.com/brianloveswords/zlib-browserify) lacks zlib.unzip.

zlib.unzip supports both gzip and deflate, but I'm not sure if we would need a separate header check (GH-6) vs gzip; or if deflate is worth supporting, in GH-7 I changed the call to zlib.gunzip and it then worked in the browser (able to parse the NBT successfully).

But I haven't tried to run the test suite in the browser yet (not sure how to - does grunt support it?)

[sjmulder]

I’m using Grunt + Karma + PhantomJS + Jasmine in another project, that setup has worked well for me there. I’ll see if I can implement that over lunch.

[sjmulder]

That setups works well, however I’m having some issues getting the test to use the Browserify-generated bundle. I’ll have another look later.

[deathcap]

Maybe could use https://ci.testling.com for browser tests (free for open source, lots of other browser-related npm modules use it)

[sjmulder]

I’ve implemented support directly in the source now (without need for pre or postprocessing with any tool).

Citing my comment from the other ticket:

Note that

• This is largely untested
• You need to pass in data as ArrayBuffer objects
• Writer’s buffer is made private now. Instead, use its getData() to get an ArrayBuffer.
• writeUncompressed also gives you an ArrayBuffer instead of a Buffer now
• Compression only works if window.zlib is a zlib module

[dokranke]

Thank you for making a browser only version. I just got this error:

Uncaught RangeError: Offset is outside the bounds of the DataView
at DataView.getInt8 (native)
at nbt.Reader.read (https://gdkdev.github.io/web-scm/lib/nbt-sj.js:370:40)
at Object.nbt.parseUncompressed (https://gdkdev.github.io/web-scm/lib/nbt-sj.js:538:21)
at Object.nbt.parse (https://gdkdev.github.io/web-scm/lib/nbt-sj.js:580:24)
at HTMLElement. (https://gdkdev.github.io/web-scm/test/gistToNBT2.html?f7ab983da36b827389ff71c56e0b3ff3:19:12)
at HTMLElement. (https://gdkdev.github.io/web-scm/lib/jquery-3.1.1.js:9842:14)
at Function.each (https://gdkdev.github.io/web-scm/lib/jquery-3.1.1.js:368:19)
at jQuery.fn.init.each (https://gdkdev.github.io/web-scm/lib/jquery-3.1.1.js:157:17)
at Object. (https://gdkdev.github.io/web-scm/lib/jquery-3.1.1.js:9841:9)
at fire (https://gdkdev.github.io/web-scm/lib/jquery-3.1.1.js:3305:31)

[sjmulder]

Do you have the NBT file for me to reproduce? You can email it to me privately if you don’t want to share online. (Email address is in my profile.)

Edit: and perhaps more importantly, the code you used to get the data and invoke parse().

[dokranke]

This is the NBT-Data: data.bin.txt.
It's from a mod called SuperCircuitMaker, to share the circuits it creates a base64 encoded gist (original source). The data above is already decoded and tested with the web-app from irath96.

EDIT: You can see what I am doing here.

[dokranke]

Seems like the parser doesn't detect compounds correctly. Because the data starts with a compound and the dataType is set to Int8. Also I couldn't force it to use a compound.

I tried it with the nbt.parse function and with the compound function of a nbt.Reader object.

[sjmulder]

Thanks for your file and testing!

Here's what I found:

• The file you linked is gzipped, which is fine but…
• …the initial hasGzipHeader() check failed because it couldn't deal with ArrayBuffers. It would return false and the library would incorrectly assume the file is uncompressed and proceed to try and parse it. This is why it couldn't see the compound at the start.

Addressed in 7a5c23c, along with a related issue (see notes).

What wil likely happen now when you try to run your code is that it will throw NBT archive is compressed but zlib is not available. You have two options: either find a browser zlib library that exposes a gunzip() function that takes a Buffer or Uint8Array, or make sure the files you're getting aren't compressed.

Hope this helps. If you have any ideas please let me know.

[dokranke]

I updated the library, but I still get the same error as you can see here.
I noted that you have a parseUncompressed-function. Why not add a parseCompressed-function, so the normal parse-function will try to auto-detect and the other ones force a specific data handling.

[sjmulder]

The library should handle this condition a bit more gracefully, but you’re passing in a string instead of an ArrayBuffer. Per the docs:

@param {ArrayBuffer|Buffer} data - gzipped or uncompressed data

I’ve added task #29 to support arrays and strings because that would be useful especially in the browser, despite strings being a bad fit for binary data. In the meantime you can use a conversion routine like this:

function stringToArrayBuffer(str) {
	if (typeof str !== 'string') throw 'str must be a string';
	var i, x, arr = new Uint8Array(str.length);
	for (i = 0; i < str.length; i++) {
		x = str.charCodeAt(i);
		if (x < 0 || x > 255) throw 'Element ' + i + ' out of range: ' + x;
		arr[i] = x;
	}
	return arr.buffer;
}

You’re also not checking the error argument. As mentioned before you’ll get an error if you pass in gzipped data without a zlib library in window.zlib.

[dokranke]

I thought that the atob-function returns a ByteArray.
I found out that neither IE or Chrome Mobile support zlib, later I will try it with my normal setup

I'm not dealing with errors, because I couldn't even get a basic setup to work.

[sjmulder]

Closing this because the issues you encountered now have separate tasks. If there’s anything else, feel free to open another ticket.