facebookarchive / prepack

A JavaScript bundle optimizer.

Prepack

Prepack is a partial evaluator for JavaScript. Prepack rewrites a JavaScript bundle, resulting in JavaScript code that executes more efficiently. For initialization-heavy code, Prepack works best in an environment where JavaScript parsing is effectively cached.

See the official prepack.io website for an introduction and an interactive REPL playground.

Status

We, the Prepack team at Facebook, have temporarily set down work on Prepack including the React compiler project. You won't see many Prepack PRs while we are currently prioritizing some other projects.

How to use Prepack

Install the CLI via npm,

$ npm install -g prepack

Or if you prefer yarn, make sure you get yarn first,

$ npm install -g yarn

and then install the Prepack CLI via yarn:

$ yarn global add prepack

You may need to prepend (pun intended!) the command with sudo in some cases.

Let the party begin

To compile a file and print the output to the console:

$ prepack script.js

If you want to compile a file and output to another file:

$ prepack script.js --out script-processed.js

Detailed instructions and the API can be found at Prepack CLI: Getting Started

Plugins to other tools

The following are a few plugins to other tools. They have been created and are maintained separately from Prepack itself. If you run into any issues with those plugins, please ask the plugin maintainers for support.

• A Rollup plugin for Prepack
• A Webpack plugin for Prepack
• A Parcel plugin for Prepack
• A Visual Studio code plugin for Prepack
• A babel plugin which transforms Flow annotations into prepack model declarations.

Test Results and Code Coverage

• test262 status on master branch
• code coverage report for serialization tests
• To see the status for a pull request, look for the message All checks have passed or All checks have failed. Click on Show all checks, Details, Artifacts, and then test262-status.txt or coverage-report-sourcemapped/index.html.

How to get the code

0. Clone repository and make it your current directory.
1. git submodule init
2. git submodule update --init
3. Get yarn and node, then do yarn

Note: For development work you really need yarn, as many scripts require it.

How to build, lint, type check

0. Get the code
1. yarn build
   You can later run yarn watch in the background to just compile changed files on the fly.
2. yarn lint
3. yarn flow

How to run tests

0. Get the code
1. Make sure the code is built, either by running yarn build or yarn watch
2. yarn test

You can run individual test suites as follows:

• yarn test-serializer
  This tests the interpreter and serializer. All tests should pass.
• yarn test-test262
  This tests conformance against the test262 suite. Not all will pass, increasing conformance is work in progress.

How to run the interpreter

0. Get the code
1. Make sure the code is built, either by running yarn build or yarn watch
2. yarn repl
   This starts an interactive interpreter session.

How to run Prepack

0. Get the code

1. Make sure the code is built, either by running yarn build or yarn watch.

2. Have a JavaScript file handy that you want to prepack, for example:
   echo "function hello() { return 'hello'; } function world() { return 'world'; } s = hello() + ' ' + world();" >/tmp/sample.js

3. cat /tmp/sample.js | yarn prepack-cli
   Try --help for more options.

How to validate changes

Instead of building, linting, type checking, testing separately, the following does everything together:
yarn validate

How to edit the website

The content for prepack.io resides in the website directory of this repository. To make changes, submit a pull request, just like for any code changes.

In order to run the website locally at localhost:8000:

1. Build prepack into the website: yarn build && mv prepack.min.js website/js
2. Run python -m SimpleHTTPServer (Python 2) or python -m http.server (Python 3) from the website/ directory

How to contribute

To read more about the project, check out this suggested reading wiki

For more information about contributing pull requests and issues, see our Contribution Guidelines.

License

Prepack is BSD-licensed. We also provide an additional patent grant.

prepack's Issues

Serializer crashes on abstract array length

Watch it crash:

var x = global.__abstract ? __abstract("boolean", "true") : true;
var a = [];
if (x) a.push(42);
inspect = function() { return a.length; }

To fix, _serializeValueArray in serializer.js needs to be modified as follows:

• Don't just call ToLength on the "length" property, but first check if it's an abstract value. If not, proceed as always.
• If it's an abstract value, emit code that...

1. Initializes the array to the empty array literal.
2. Sets the array length to abstract length value.
3. And then emits all properties (including numeric keys) via addProperties.
   Ideally, fuse the old and the new code in an elegant way.

Add a regression test to test/serializer/abstract.

Setup test infra to prepack popular JavaScript frameworks, including React.

Native Values That Are Not Allowed to Serialize

I used to have a magic string for this but we can do better. It would be good to have a materialized value that can be read and used as any other value, but if it ever ends up in the serialized output it should throw.

An example of this is a native file handle. It can be valid to use it during compilation such as in Node's start up phase. However, it will be invalid by the time the application revives. So it can only be temporary.

I used to add this to the module system cache so that if it ever got serialized, I'd know to fix it so that it can be fully cleaned up at the end rather than accidentally live with a bloated output.

Look into turning all closures created from a single node into metaprogramming at runtime

var a = [1, 2].map(function (i) {
  return function () { return i; };
});

is going to produce something like:

var _0 = 1;
var _1 = 2;

var _3 = function () {
  return function () { return _0; };
};

var _4 = function () {
  return function () { return _0; };
};

var a = [_3, _4];

Some engines may perform some optimisations to reduce the amount of bytecode for the original closure. We could turn this into something that's cheap to create at runtime. This would be beneficial for code size too.

Figure out why serialization fails on multiple passes of serializer

The serialization of some internal tests fails without the --singlePass option but succeeds with that option. Figure out and fix the issue with the optimizations relying on valToRefCount.

Avoid generation of wrapper function when no local scope is needed

Currently, prepacking a simple piece of code almost always creates a function wrapper:
(function () {
...
}).call(this);

Determine if we actually declare any variables, and only emit the function wrapper then.

Add support for storing event listeners (process / document etc)

Intrinsics on the prototype chain aren't serialized correctly

arr = Object.create(Array.prototype);

incorrectly turns into:

arr = {};

This seems to happen with any form including:

num = new Number(123);

Interpreter: Create union values when referencing intrinsic in a logical `||`

var foo = Object.assign || function () {};

Should serialise and retain the right side even if Object.assign exists.

On property access we should query all of the possible values and if they don't SameValue then we should throw a compiler error.

Serialise weirdly configured global properties

Conditionally Deleting a Property Yields `__empty` Object Instead of Missing Property

let x = __abstract('boolean', 'x');

ob = { a: 1 };
if (x) { delete ob.a; }

Turns into:

(function () {
  var __empty = {};
  var _0 = {
    a: x ? __empty : 1
  };
  ob = _0;
}).call(this);

It should have a missing property in this branch.

Wrong code generation for Top Level Variable Declarations

Given a top level like:

var x = 1;
x = 2;

Prepack turns it into:

var x = undefined;
var x = 1;
var x = 2;

I guess multiple "set" operations is correct but it is not correct to "set" it to undefined first. It would invoke a setter unnecessarily. It is also a bit unnecessary to print it as multiple var so it looks like an error.

Object.prototype.toString duck typing catches everything as a Date

console.log(Object.prototype.toString.call({}));

->

console.log("[object Date]");

Should be [object Object] but for some reason everything looks like a Date.

Interpreter: Retain strict mode for function bodies

Retain specific polyfill branching

Will need to special case specific patterns. Will need to analyse popular polyfills.

1. Detect branching - ensure it's execution is side effect free. We could do this by having a read-only mode on realm and just execute the branches anyway and throw an error on writes.
2. Add it to the end of builds.

Reduce number of cycles

flow check --profile indicates that we have a giant cycle involving >211 files.
Flow conceptually concatenates all files involved in a cycle, which causes it to become slow when changing any file in the cycle.
We need to 1) drastically reduce the number of files involved large cycles, and 2) put a mechanism in place to prevent cycles from creeping in again.

Add option to profile Prepack

At least the following sections should be timed:

• How long does interpretation of the first initialization phase take?
• How long does speculative initialization of more modules take?
• How long does first serialization pass take?
• How long does second serialization pass take?

Implement Destructuring

This isn't really more important than any other ES2015 feature other than that I had to work around this specifically for Node.

Seems like it should be easy. I might look into it if nobody else does.

Correctly preserve console.log, console.warn, console.error

In console.js, currently everything is munged together, and in generator.js in emitConsoleLog they are all emitted as console.log statements. This is wrong. Keep around information regarding log/warn/error and emit the correct code.

Reads of intrinsics can happen after they've already been mutated

(function() {
  let old = Array.prototype.forEach;
  Array.prototype.forEach = function() {};
  Array.prototype.forEach = old;
})();

Becomes

(function () {
  var _$0 = Object.defineProperty;
  var _0 = Array.prototype;

  function _1() {}

  _$0(_0, "forEach", {
    enumerable: false,
    configurable: true,
    writable: true,
    value: _1
  });

  _$0(_0, "forEach", {
    enumerable: false,
    configurable: true,
    writable: true,
    value: Array.prototype.forEach
  });
}).call(this);

In the original Prepack reads and writes were ordered in a single queue for this reason so that reads can be done at the right time.

Generalize support for modules/require.

Pass all test262 tests

Stack traces only contain call stack, but not location of actual crash

When prepacking the following program

(function() {
  function f() {



    t



  }
  f();
})();

we get an error like this:

t is not defined
ReferenceError
    at repl:13:3
    at repl:3:1

where 3:1 is the beginning of first called function, and 13:3 the beginning of the second function. However, the position within the second function is lost. It should be on top of the stack. Look at Error.js/build to see how the error is constructed. We probably need to pass in an AST node with location information.

Invocation of an abstract function yields residual functions

(function() {
  var o = __abstract('number', 'foo');
  var obj = {};
  function bar(x) {
    if (o > 1) {
      obj.foo = function() { return 1 + x; }
    } else {
      obj.foo = function() { return 2 + x; }
    }
  }
  bar(5);
  window.x = obj.foo();
})();

Prepacks to:

(function () {
  function _3() {
    return 1 + 5;
  }

  function _4() {
    return 2 + 5;
  }

  var _0 = foo > 1 ? _3 : _4;

  var _$0 = _0();

  x = _$0;
}).call(this);

It seems like these functions could just collapse into their values or side-effects?

Should this be done at the abstract value level or some other future pass?

Prepack doesn't properly serialise arrays with accessor properties in the 0-length range.

Prepacking

a = [1,2,3];
Object.defineProperty(a, 2, { configurable: true, enumerable: true, get: function() { throw new Error();} })

currently crashes Prepack, as it wrongly tries to evaluate the getter during serialisation instead of persisting it as is.

Add support for storing timers (setTimeout / setInterval)

Retain the [[Prototype]] of an object if it's intrinsic

Rename Compatibility Mode `jsc`

This mode isn't really a general purpose compatibility mode for all JSCs but a specific version of JSC. Maybe we can name it after the nearest version?

This part is also a bit odd:

https://github.com/facebook/prepack/blob/master/src/prepack.js#L27

Why does that get added in this mode? Seems more like it is readding a polyfill. Is that because Prepack doesn't detect mutations to global or something?

Don't serialize reserved names in helpers

In the object creation helpers that are generated. It is possible to have property names that are invalid identifiers or reserved words.

I don't have a simple repro because I don't yet understand when these helpers are generated but the result is something like this:

  function _2AQ(default, __esModule) {
    return {
      "default": default,
      __esModule: __esModule
    };
  }

Which is invalid code.

Add DOM shims

Inline single-use pure serialised declarations

Refactor common pattern of throwing of errors.

Currently, the codebase is littered with code of the following pattern:

        throw new ThrowCompletion(
          Construct(realm, realm.intrinsics.SOMEERROR, [new StringValue(realm, SOMETEXT)])
        );

All of these terribly long constructs should be replaced by

throw realm.createErrorThrowCompletion(realm.intrinsics.SOMEERROR, SOMETEXT)

This will make it easier to reason about errors.

Memoize references to global properties early

Currently, when generating code, the serializer assumes that names like "RegExp", "Symbol", "Proxy" have a certain meaning. However, those global properties could have been redefined. When needed, the serializer should obtain references to such global properties via the prelude.

Consider what happens when prepacking the following code:

(function() {
  let re = RegExp;
  RegExp = function() { throw new Error(); }
  RegExp = new re('ab+c');
})();

Execute side effectless constructs

Interpreter: Serialise intrinsics pollution

Add a non-opaque/compound mode AKA full interpretation

Prepack drops array properties outside of 0-length range

Prepacking

a = [1,2,3]
a.foo = 42;

yields

(function () {
    var _0 = [1, 2, 3];
    a = _0;
}).call(this);

This is wrong. The "foo" property must be preserved as well.

Setting properties that are not valid identifiers serializes into invalid identifiers

global['foo-bar'] = true;

->

(function () {
  foo-bar = true;
}).call(this);

empty wrongly becomes undefined

Prepack this test to trigger the issue:

(function() {
var c = global.__abstract ? __abstract("boolean", "false") : false;
a = {};
if (c) a.f = a;
})();

inspect = function() { return 'f' in a; }

Experiment with diffing realms, injecting changes. Hot reloading magic 😱

Serialise symbols

Add option to omit second serialization pass

We currently do two passes in order compute usage counts for values, and then avoid generating ids for single-use value. However, this makes Prepack slower, and it would be beneficial for quick functional tests to avoid the second pass.

Multi-pass serialization doesn't respect observable object identity in residual function

Prepacking

(function() {
  var a = {x: 4};
  function f() {
    return a;
  }
  function g() {
    return a;
  }
  inspect = function() { return f() === g(); }
})();

yields

(function () {
  function _1() {
    return {
      x: 4
    };
  }

  function _3() {
    return {
      x: 4
    };
  }

  function _0() {
    return _1() === _3();
  }

  inspect = _0;
}).call(this);

Object identity must be preserved if it can be observed by residual functions. This is a concrete issue relating to #422.

Interpreter: Create an indeterministic value type

For cases such as:

var d = Date.now();

We should throw a compiler error whenever this is read.

Make sure all generated names are unique

The serializer and generator create new names of the form _XXX. However, such names might also appear in residual functions. Prepack needs to make sure that all names are unique. Consider prepacking the following code:

(function() {
  function bar() {
    try {
      return _0.name;
    } catch(e) {
      return "exception";
    }
  }
  inspect = function() { return bar(); }
})();

At the very least, we should detect this and fail. Or better, we make those ids unique which collide with any local variable names.

Interpreter: Fix object/array circular reference bug

var arr = [];
arr.push(arr);

will be compiled incorrectly.

Serialization of objects with internal slots

Currently, the serializer doesn't properly handle all kinds of objects with internal slots.
Generally, internal slots are properties named "$...".
In serializer.js, look at how it properly handles a "Number" which has the internal slot "$NumberData".
In a similar manner, we need to handle...

• $StringData => String
• $BooleanData => Boolean
• $SymbolData => Symbol (not yet fully supported anyway)
• $DateValue => Date

Check ObjectValue.js to see what other internal slots might exist. Make the serializer log an error when it comes across any objects that have unsupported internal slots set.

Make scoping spec compliant

I just YOLO'd it without reading the spec text.

Ignore `constructor` property when deserialising if possible

Incorrect Serialization of Functions with Default Arguments

Ok, this rabbit hole was a bit tricky to figure out...

This is a quirk of the spec that was added to better support lazy parser. Basically, you can't make a function that has default arguments ("non simple") go into "use strict" mode.

This is invalid:

function foo(bar = 1) { "use strict"; }

Luckily Babylon will complain if you do this so no code is written this way. But...

It is possible for Prepack to create a mixed mode environment:

(function() {
  function foo() {
    "use strict";
    return function(bar = 1) {
      return bar;
    };
  }
  x = foo();
  y = function() {};
})();

Which turns into this invalid code by flattening the scopes:

(function () {
  function _0(bar = 1) {
    "use strict";

    return bar;
  }

  x = _0;

  function _1() {}

  y = _1;
}).call(this);

Arguably this is something that Babel could handle on its side.

However, in these scenarios Prepack will also generate very bloated code because "use strict" is repeated for every new function. Maybe we should instead group all strict functions into a single "strict" scope?