bblfsh / python-pydetector Goto Github PK

Currently a trailing comment pseudo-node will be added as child of all elements in that same line. While this is technically correct (its a trailing comment of all elements in that line) its also wasteful. Make a cache of trailing comments storing the line of the comment to avoid duplicates.

This have to be disabled by default because it could be dangeous. It would be nice to also find a way to avoid the interpreters running main (maybe with a third proxy module importing the one being tested).

From the meeting about this matter on the Language Analysis planning:

For nodes without a token (e. g. grouping nodes like "arguments" or "aliases"):

• If the node doesn't have StartPosition in the native AST it won't have it either in the UAST.
• Nodes without token but with position in the native AST will have exactly that StartPosition in the UAST and no EndPosition (since we can't extract it from the tokenizer).

Nodes with a token:

• The EndPosition will be the one of the token. E. g. functions' EndPosition will be the position of the last character of the name, not the end position of the last child, as stated in the documentation.
• When a node with a token have different positions in the native AST and the tokenizer, the last one will be used since it has proved to be more reliable (e. g. the a.b.c.d case).
• No workarounds will be applied to positions that an issue reporter consider wrong if they're wrong in the tokenizer and/or native AST.

This

This

In the meeting planning we decided to split the functionality in the current pydetector.astexport.py module between the retrieval of the native AST data structure unmodified (but for the right Python version) in pydetector and the visitor + noop extractor + position updater in python-driver, reusing the data returned from pydetector to avoid doing a double parsing.