Performance: Maintain a summary of child node types about ast-types

[drslump]

I've been playing with this idea for skipping whole branches during the traversal, the tests has been quick-and-dirty, just to have a sense of the potential gain. I'm actually impressed by how much this improves the performance when doing multiple passes.

First be aware of some conditionals that at least for these tests I'm working with:

• Does not use NodePath
• Only traversal is tested (no transformations yet)
• The Nodes are annotated (see below)
• The hash is contained in 32bit

When first traversing the AST each node is annotated like so:

    Object.defineProperties(node, {
        'x-ast-types-parent': {
            value: null,
            enumerable: false,
            writable: true
        },
        'x-ast-types-name': {
            value: null,
            enumerable: false,
            writable: true
        },
        'x-ast-types-contains': {
            value: 0,
            enumerable: false,
            writable: true
        }
    });

Note: It's much faster to simply augment the node with node['x-ast-types-xxx'] = value, but it feels a bit dirty.

On this first traversal the contains property is computed as well, which doesn't seem to have much of an impact actually.

The speed is roughly the same as NodePath when traversing backbone.js on the first iteration (~120ms). However on the next iteration I get consistent results of ~5ms versus ~60ms for NodePath. That is with a visitor for FunctionExpression. Visiting stuff that is not present like ArrowFunctionExpression it gets under ~1ms (a single call to traverse) while NodePath stays on the ~50ms.

The false positive rate for skipping branches is around 50%. It could be improved by having more bits to store the hashes but it'll make the first iteration a bit slower and would require an additional field.

One side effect of this approach is that structure is kept on the AST, which has benefits and drawbacks. The main benefit is that now we can use a path object that just holds the logic, since the state is in the node and is always kept in sync. I've played with the idea of having a this.path(node) on the visitor, so it's always at hand. Another thing I've played with is detecting the number of arguments the visiting method has, if it's 2 then it'll receive as first parameter the node (as normally) but as second parameter it'll receive an already configured NodePath instance.

Overall I'm quite happy with the performance that can be obtained with this design, it's a specially good approach for things with multiple phases and frontend visitors for not so common node types (like esnext). While the speed of a full traversal can be optimized (experiments with meta programming based on the type declarations and trampolines are promising) I doubt that it could reach the speed improvements of just ignoring whole branches of the AST. The main drawback however is that the AST has to be augmented with custom properties, it's the only way to make this design work in a reliable way when introducing transformations in the tree.

[eventualbuddha]

This is great. I've done a bit of thinking along this line and I'm convinced it's worth pursuing. I had been using a modified version of esprima to do the generation of the hasTypes property. Anything more been done on this?

[drslump]

@eventualbuddha I played a bit more with the idea but it becomes too complex once transformations are performed. On #82 there is a traversal implementation that offers a very good performance without the problems associated with detecting mutations in the tree.

If the speed improvements coming from #82 are not enough perhaps something like this .hasTypes can be implemented on top or as some sort of specialized visitor. For the time being I'm closing this one.