Ability to load bytecode about duktape

[svaarala]

There's no firm plan one way or the other right now, simply because there hasn't been much need for it. Could you describe a bit what sort of use case are you considering? For instance, are you interested in loading bytecode for obfuscation, memory considerations (perhaps dropping the compiler from the build), or speed?

Technically, dumping and loading bytecode is quite straightforward although the bytecode will be version dependent. Verifying the loaded bytecode for safety is not trivial and probably won't be implemented at least in the near future, which means that the bytecode source needs to be trusted. In other words, untrusted bytecode could be memory unsafe to execute.

[malord]

I actually have two different uses in mind!

I have a web server written in C++ which currently uses an ECMAScript-with-bits-missing scripting language as its templating language. The templates are currently parsed the first time they are requested and on subsequent requests it just re-runs the bytecode (in a new, blank context to ensure no data leaks between requests).

The second use is a hobby project, a game engine. It uses the same, broken scripting engine as the web server. In this case I store the bytecode in the level files, compiled by the game's level editor (on Mac) so that the game doesn't need to complle at runtime (on iOS/Android). I had been considering a switch to Lua, which is often used in this way in games, but changed my mind when I discovered Duktape.

[svaarala]

Thanks for the use case details :) The templating case is quite interesting and I can see why compilation time would matter. In this case bytecode loading basically serves as compilation caching, while in the game scenario you probably want the compilation to happen during build.

I can put bytecode support into the release plan - at least for 1.1 and perhaps for 1.0 if there's time. It's relatively straightforward so it mainly needs a bit of documentation, test cases, etc.

[mitchblank]

Technically, dumping and loading bytecode is quite straightforward although the bytecode will be version dependent.

It also looks like it's endian dependent. A dump/load procedure would probably want to normalize it to little-endian. This is needed at least for the bytecode stream and the constants section, probably other stuff as well.

[svaarala]

What I meant is that bytecode will at least be version dependent because a version independent bytecode format would be way too limiting. Endianness can be normalized or not - the choice depends on the intended use case for bytecode. For instance, @malord's scenario doesn't need normalization, while other use cases do.

Personally I'd like bytecode to be directly portable between architectures, provided that Duktape versions match. This saves users hassle (in some cases) and costs quite little. For byte order I'd prefer big endian because it's also the standard network byte order and seems appropriate if bytecode is (potentially) transferred between machines.

Btw, for comparison, Lua bytecode is not endian normalized (at least in Lua 5.1, not sure about newer ones). I've had to deal with Lua bytecode endianness issues myself :-)

[mitchblank]

Yes, I was just talking about the case where you want to cross-compile assets. I.e. if I'm on an x86 machine and cross-compiling for PowerPC, it would be handy if I can easily pre-compile ecma assets on the host machine without any extra steps.

Which endian is up to you. Personally I realized a decade ago that 99.9% of machines running my code are going to be little-endian and that isn't likely going to change. Therefore in data formats I've designed recently I usually specify LE integers even if that's the opposite of tradition. I figure I might as well optimize for the common case even if it usually doesn't matter too much.

[svaarala]

I was also referring to this cross-compile case which one needs to work around with Lua. It's trivial of course but just one more unnecessary hoop to jump through, so I agree with you there. I was just trying to point out it's not a technically mandatory feature and thus more of a matter of preference than a technical necessity.

I'll probably go with big endian for the reasons mentioned above. It also has the nice feature that binary dumps are easier to read since constants won't get byte order mangled.

[toastfr35]

The ability to compile and then save the bytecode for later execution would be great for my application.
The idea is to compile once and then not have to recompile every time the script is started. In my case I have an application started many times that contains a JS script and duktape. The script can be large but does not execute for long and I think that I might be spending more time compiling it every time than actually running it. It would be nice to be able to store it as bytecode as in Lua.

[svaarala]

Now tracked by #208, closing this one.