bblfsh / javascript-driver Goto Github PK

View Code? Open in Web Editor NEW

9.0 9.0 13.0 9.65 MB

License: GNU General Public License v3.0

JavaScript 39.35% Smarty 0.71% Makefile 0.04% Go 56.44% Shell 0.16% Dockerfile 3.29%

babelfish driver javascript

javascript-driver's People

Contributors

Stargazers

Watchers

Forkers

serabe erizocosmico ajnavarro abeaumont dennwc tsolakoua bzz juanjux lwsanty dhanush breitfelder stg-tud pbehnke

javascript-driver's Issues

Update native parser to a recent Babel version

Current impl uses pre-released beta of JS parser

v7.0.0-beta.34
@hzoo hzoo released this on Dec 2, 2017 · 986 commits to master since this release

Latest release is

v7.1.6
@nicolo-ribaudo nicolo-ribaudo released this 16 days ago · 25 commits to master since this release

[bug] Missed token for regular expressions

Hi,
I found that token information for node with regexp is missed.
Code:

function addResponsiveClass(str) {
  return str.replace(/\<img/g, '<img class="img-responsive"');
}

Let's visualize code without nodes with tokens:

function __________________(___) {
  return ___._______(/\<img/g, _____________________________);
}

If we check the node with /\<img/g we will see:

internal_type: "RegExpLiteral"
properties {
  key: "flags"
  value: "g"
}
properties {
  key: "internalRole"
  value: "arguments"
}
properties {
  key: "pattern"
  value: "\\<img"
}
start_position {
  offset: 283
  line: 14
  col: 22
}
end_position {
  offset: 291
  line: 14
  col: 30
}
roles: EXPRESSION
roles: LITERAL
roles: REGEXP
roles: CALL
roles: ARGUMENT

I think this node should contain token with /\<img.

Timeout error even with no timeout used on bblfshd

Removing the context timeout using the go bblfsh client, I'm still having the following error when parsing javascript files:

Processing file docs/js/vendor/jquery-1.8.0.min.js
rpc error: code = DeadlineExceeded desc = context deadline exceeded

Better comment handling

After #73 we uniformly drop all 3 type of comments from normalized nodes to avoid annotating them as part of SDK update in #57

This is to track part of of normalizer.go FIXME items.

Native driver crashed

From bblfsh logs we got a message of rpc error native driver crashed. Example of files that triggered this error:

xpfe/components/regviewer/regviewer.js
JSTests/ChakraCore/test/Strings/long_concatstr.js

ImportDeclaration: unhandled case

The file mentioned in #69 also triggers this error if the original issue is fixed:

blacklisted nodes of type "ImportDeclaration" (2) found in the tree

It's not detected by you test suite and is triggered with this file:

jsx.js

// @flow strict
import * as React from "react";
import { graphql, QueryRenderer } from "react-relay";
import Alert from "@kiwicom/orbit-components/lib/Alert";
import InputField from "@kiwicom/orbit-components/lib/InputField";
import type { Environment } from "react-relay";

import environmentReal from "../../services/environment";
import PickerDropDown from "./primitives/PickerDropDown";
import NoResult from "./primitives/NoResult";
import ClickOutside from "../ClickOutside";
import Text from "../Text";
import LocationPickerResultList from "./components/LocationPickerResultList";
import getPlaceholder from "./services/placeholder";
import type { Location } from "../../records/Location";

// FIXME add arrow handling somehow neatly
// const ARROW_UP = 38;
// const ARROW_DOWN = 40;
// const ESC = 27;
// const ENTER = 13;

// FIXME @viktr solve these:
// centralize styles with IataPicker

// FIXME @oreqizer solve these:
// arrow handling

type Props = {|
  value: Location | null,
  onChange: (loc: Location) => void,
  label: string,
  icon: React.Node,
  // defaulted
  environment: Environment,
|};

type State = {|
  active: boolean,
  input: string,
|};

class LocationPicker extends React.Component<Props, State> {
  static defaultProps = {
    environment: environmentReal,
  };

  state = {
    active: false,
    input: "",
  };

  node: { current: any | HTMLDivElement } = React.createRef();

  handleClose = () => {
    this.setState({
      active: false,
    });
  };

  handleSelect = (item: Location) => {
    const { onChange } = this.props;

    onChange(item);
    this.setState({
      active: false,
    });
  };

  handleChange = (ev: SyntheticInputEvent<HTMLInputElement>) => {
    this.setState({
      input: ev.target.value,
      active: true,
    });
  };

  render() {
    const { value, label, icon, environment } = this.props;
    const { active, input } = this.state;

    const placeholder = value ? getPlaceholder(value) : null;

    return (
      <ClickOutside active={active} onClickOutside={this.handleClose}>
        <>
          <InputField
            inlineLabel
            placeholder={placeholder || label}
            onChange={this.handleChange}
            prefix={icon}
            value={active ? input : ""}
          />
          {input && active && (
            <QueryRenderer
              environment={environment}
              query={graphql`
                query LocationPickerQuery($input: String!) {
                  allLocations(last: 50, search: $input) {
                    ...LocationPickerResultList_list
                  }
                }
              `}
              variables={{ input }}
              render={res => {
                if (res.error) {
                  return (
                    <Alert type="critical">
                      <Text t="common.api_error" />
                    </Alert>
                  );
                }

                if (!res.props) {
                  return null;
                }

                if (!res.props.allLocations) {
                  // TODO render this in the list if length is 0
                  return (
                    <NoResult>
                      <Text t="forms.places_no_results" />
                    </NoResult>
                  );
                }

                return (
                  <PickerDropDown ref={this.node}>
                    <LocationPickerResultList
                      list={res.props.allLocations}
                      selectedId={value && value.id}
                      onSelect={this.handleSelect}
                    />
                  </PickerDropDown>
                );
              }}
            />
          )}
        </>
      </ClickOutside>
    );
  }
}

export default LocationPicker;

transform failed: rune out of bounds: 2200447 [0, 2200314)

File: https://gist.github.com/smacker/833bfbbf187727a1dbf0adc72777136a
(hopefully uploading to gist didn't break it)

bblfsh 2.11.8-drivers

Errors like this: construct: key \"operator\": key \"@role\": unhandled value: **= in map[+=:[{Arithmetic} {Add}] ^=:[{Bitwise} {Xor}] -=:[{Arithmetic} {Substract}] *=:[{Arithmetic} {Multiply}] /=:[{Arithmetic} {Divide}] >>=:[{Bitwise} {RightShift}] >>>=:[{Bitwise} {RightShift} {Unsigned}] |=:[{Bitwise} {Or}] &=:[{Bitwise} {And}] =:[] %=:[{Arithmetic} {Modulo}] <<=:[{Bitwise} {LeftShift}]] arose in the latest bblfsh logs.

Files that triggered this error

src/parser/test/esprima/ES2016/exponent/exp_assign.js
JSTests/test262/test/language/expressions/exponentiation/exp-assignment-operator.js
pkg/nuclide-debugger/spec/Bridge-spec.js
src/parser/test/esprima/ES2016/exponent/exp_assign.js
JSTests/stress/pow-simple.js
JSTests/ChakraCore/test/es6/expo.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A15.js
src/parser/test/esprima/ES2016/exponent/exp_void.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A23.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A4.js
JSTests/test262/test/language/expressions/exponentiation/exp-operator-evaluation-order.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A13.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A11.js
JSTests/stress/pow-stable-results.js
JSTests/test262/test/language/expressions/exponentiation/exp-operator.js
src/parser/test/esprima/ES2016/exponent/exp_precedence.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A20.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A3.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A21.js
JSTests/test262/test/language/expressions/exponentiation/exp-operator-precedence-update-expression-semantics.js
src/parser/test/esprima/ES2016/exponent/exp_not.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A6.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A7.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A18.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A22.js
JSTests/stress/pow-evaluation-order.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A8.js
src/parser/test/esprima/ES2016/exponent/exp_typeof.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A2.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A14.js
JSTests/stress/pow-with-constants.js
JSTests/stress/pow-integer-exponent-fastpath.js
src/parser/test/esprima/ES2016/exponent/exp_bitnot.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A1.js
tests/fixtures/passing-es7.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A17.js
src/parser/test/esprima/ES2016/exponent/exp_minusminus.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A5.js
tests/arith/exponent.js
src/parser/test/esprima/ES2016/exponent/exp_delete.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A9.js
src/parser/test/esprima/ES2016/exponent/exp_plusplus.js
src/parser/test/esprima/ES2016/exponent/exp_operator.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A19.js
JSTests/stress/pow-coherency.js
JSTests/stress/pow-basics.js
JSTests/stress/pow-with-never-NaN-exponent.js
JSTests/stress/pow-nan-behaviors.js
src/parser/test/esprima/ES2016/exponent/exp_plus.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A12.js
JSTests/test262/test/language/expressions/exponentiation/applying-the-exp-operator_A16.js
src/parser/test/esprima/ES2016/exponent/invalid_update.js
src/parser/test/esprima/ES2016/exponent/exp_minus.js
JSTests/stress/pow-to-number-should-be-executed-in-code-side.js
src/parser/test/esprima/ES2016/exponent/update_exp.js
JSTests/test262/test/language/expressions/exponentiation/exp-operator-precedence-unary-expression-semantics.js

Bad version string on v1 branch

From bblfsh/python-client/issues/141:

The javascript driver version is reported to be "dev-adcd1b4", which does not help to check that it is 1.2.0 (which we strictly require since the newer versions do not work for us).

Note: this seems to be fixed on the v2 branch.

\n should not be parsed in strings

Any JS string which has \n, e.g.

bufferedLog.push(...args, '\n');

contains parsed new lines instead of \n. The desired behavior is passing the string contents verbatim.

Premature EOF

From bbfsh logs we got a message of unexpected EOF. Examples of files that triggered this error:

third_party/closure/goog/graphics/vmlgraphics.js
Source/WebInspectorUI/UserInterface/Models/TimelineRecording.js
node_modules/gitbook-plugin-eayun/index.js
release/src/router/www/sysdep/DSL-AC68U/www/jstz.min.js
release/src/router/www/sysdep/DSL-N55U-B/www/jstz.min.js
release/src/router/www/sysdep/RT-AC87U/www/chanspec.js
release/src/router/www/sysdep/RT-AC53U/www/chanspec.js
release/src/router/www/sysdep/RT-AC68U/www/chanspec.js
release/src/router/www/sysdep/RT-AC56U/www/chanspec.js
release/src/router/www/sysdep/RT-N66U/www/chanspec.js
release/src/router/www/sysdep/RT-AC3200/www/chanspec.js
release/src/router/www/sysdep/DSL-AC68U/www/chanspec.js
release/src/router/www/sysdep/RT-N18U/www/chanspec.js

No informative message on errors

On any error, the driver always returns "message is not defined", example:

{
    "status": "error",
    "language": "javascript",
    "errors": [
        "message is not defined"
    ],
    "ast": null
}

Parsing fails on JSX + Flow file

Steps to reproduce

docker run --rm -it -p 9432:9432 bblfsh/javascript-driver:v2.6.0
wget https://raw.githubusercontent.com/kiwicom/nitrolib/91c1b4d26d7690690529ba0001f24985b161f2ac/src/components/LocationPicker/index.jsx
bblfsh-cli -l javascript index.jsx

Expected

UASTv2 tree printed (with some nodes may be not annotated)

Actual

couldn't parse index.jsx: transform failed: check: key "specifiers": field was not used: leadingComments

Parsing fails on valid JS: string literal with \0

Parsing string.js file with content

var escSlash = '\0SLASH'+Math.random()+'\0';
var escOpen = '\0OPEN'+Math.random()+'\0';
var escClose = '\0CLOSE'+Math.random()+'\0';
var escComma = '\0COMMA'+Math.random()+'\0';
var escPeriod = '\0PERIOD'+Math.random()+'\0';

results in

 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0SLASH')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0OPEN')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0CLOSE')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0COMMA')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0')
 check: key "value": invalid syntax
 check: key "value": invalid syntax ('\0PERIOD')

Found though #54

Steps to reproduce

$ bblfsh-cli string.js

Operator Role missing position information

Run:

echo "var a = 0 + 1;" > o.js
docker run -d --rm --name style_analyzer_bblfshd_new --privileged -p 9432:9432 bblfsh/bblfshd\:v2.11.0 --log-level DEBUG
docker exec style_analyzer_bblfshd_new bblfshctl driver install javascript docker://bblfsh/javascript-driver:v2.6.0
docker cp o.js style_analyzer_bblfshd_new:o.js
docker exec -it style_analyzer_bblfshd_new bblfshctl parse o.js

You get next UAST: https://gist.github.com/zurk/b594c7642a1bdfeca334ab27a2394426
And operator tokens are missing their positions:

.  .  .  .  .  .  .  .  .  .  0: Operator {
.  .  .  .  .  .  .  .  .  .  .  Roles: Arithmetic,Add,Expression,Operator,Binary
.  .  .  .  .  .  .  .  .  .  .  TOKEN "+"
.  .  .  .  .  .  .  .  .  .  .  Properties: {
.  .  .  .  .  .  .  .  .  .  .  .  internalRole: operator
.  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  }

And for driver v.1.2.0 I do not see this node. So I think it was added inside bblfsh, but without position information. It should be easy to include positions because you have positions for the right and left part of the expression.

Parsing fails on valid JS file: extra + parenthesized/parenStart

Discovered though #54 (comment)

Using latest bblfshd v2.11.6/client.v3/javasript driver v2.6.0 parsing of the valid 2.4 mb .js file fails.

Steps to reproduce

wget https://raw.githubusercontent.com/laravel/telescope/6f0a10ec586cfa1a22218b6778bf9c1572b97912/public/app.js
node --check app.js // check JS syntax
go install github.com/bblfsh/client-go/cmd/bblfsh-cli/...
bblfsh-cli ./app.js

Expected

Very long semantic UAST printed in YAML format.

Actual

couldn't parse public/app.js: transform failed: received 15 errors:
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenStart
	check: key "extra": field was not used: parenStart
	check: key "extra": field was not used: parenStart
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenStart

As noted #54 (comment) actual order of the failing Transformations can vary.

semantic: string literal escape sequence handing

Discovered as part of the bblfsh/bblfshd#268 (comment)

This is an ☂️ issue for multiple driver failures \w invalid syntax.

E.g driver fails to parse e.g this file with

     check: key "value": invalid syntax

Normalization mapping for semantic mode needs to be updated.

check: key "value": invalid syntax ("\0\0\0\0\0\0\0\0") #79
double quoted literals need to have same js escape sequence handled as #64
check: key "value": invalid syntax ("\.\.")"
...

132 file examples could be obtained using

grep "language=javascript$" bblfshd.log | grep -oP "rpc error: code = \K(.*)" | grep "invalid syntax" | sort | less

REGEXP role does not have token

if you parse next js code

x = /<regexp>/;

you have next UAST tree:

#           Token  Internal Role         Roles Tree                                                    
                                                                                                       
(1, 1, 13)  ||     File                  FILE                                                          
(1, 1, 13)  ||     Program               ┣ MODULE                                                      
(1, 1, 13)  ||     ExpressionStatement   ┃ ┣ STATEMENT                                                 
(1, 1, 13)  ||     AssignmentExpression  ┃ ┃ ┣ EXPRESSION, ASSIGNMENT, OPERATOR, BINARY                
(1, 1, 2)   |x|    Identifier            ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER, ASSIGNMENT, BINARY, LEFT      
(1, 3, 13)  ||     RegExpLiteral         ┗ ┗ ┗ ┗ EXPRESSION, LITERAL, REGEXP, ASSIGNMENT, BINARY, RIGHT

and there is no token for RegExpLiteral Node. I think it is because it stored not in value, but in pattern field: https://github.com/babel/babylon/blob/master/ast/spec.md#regexpliteral .

It should be easy to fix.

Error `message check: key "extra": field was not used: parenthesized` for v2.6.0

There is a file that is parsable with v1.2.0 driver but not with v2.6.0
Full error mesage

Errors:
	- transform failed: received 2 errors:
	check: key "extra": field was not used: parenthesized
	check: key "extra": field was not used: parenthesized

How to reproduce

wget https://raw.githubusercontent.com/hakimel/reveal.js/0b3e7839ebf4ed8b6c180aca0abafa28c67aee6d/js/reveal.js

v2.6.0 javascript-driver


docker run -d --rm --name bblfshd_new_driver --privileged bblfsh/bblfshd\:v2.11.0 --log-level DEBUG
docker exec bblfshd_new_driver bblfshctl driver install javascript docker://bblfsh/javascript-driver:v2.6.0
docker cp reveal.js bblfshd_new_driver:reveal.js
docker exec -it bblfshd_new_driver bblfshctl parse reveal.js

v1.2.0 javascript-driver (the same script, just install another driver and give different name)

docker run -d --rm --name bblfshd_old_driver --privileged bblfsh/bblfshd\:v2.11.0 --log-level DEBUG
docker exec bblfshd_old_driver bblfshctl driver install javascript docker://bblfsh/javascript-driver:v1.2.0
docker cp reveal.js bblfshd_old_driver:reveal.js
docker exec -it bblfshd_old_driver bblfshctl parse reveal.js

Last command prints UAST. Be careful, it is huge.

Better type annotations

Low-hanging fruit part of the TODO items from #57

Flow variable types #57 (comment)
Flow function arguments #57 (comment)
Flow function return type #57 (comment)

Consider using existing JS parsing tests for fixtures

See most of recently discovered issues come from this repo which contains a set of JS parser tests. We can use this file set in our fixtures to get a better test coverage.

Driver returns twice the same comment

If it's parsed the following example:

a = 1; //comment here
/* block comment */

It is returned each comment twice with different internalRole

addendum:
This issue might be related to these similar ones in other languages drivers:
bblfsh/go-driver#28 bblfsh/python-driver#167 bblfsh/java-driver#86

JSON parse failure

I tried to parse a simple JSON object, expecting it to work as it does on the Chrome console,
but got a parsing error.

{
  "message": "hello, world"
}

Gives the error syntax error: message is not defined.

I would have expected JSON to be supported by this driver.
Is that an incorrect assumption on my side?

Ensure fixtures cover 100% of all possible native node types

Looks like some types escaped the coverage of the integration tests' fixtures. We should check the native driver to get all possible types. Also, it seems the parser module is outdated since it was renamed from babylon to babel but we still refer to the old name.

Unused fields on nodes

Mesages such as:

unused field(s) on node AssignmentPattern: trailingComments
unused field(s) on node AssignmentPattern: leadingComments
unused field(s) on node ImportDeclaration: importKind
unused field(s) on node RestElement: typeAnnotation

arose in the latest bblfsh logs.

Files that triggered AssignmentPattern: trailingComments

tests/binding/scope.js

Files that triggered AssignmentPattern: leadingComments

pkg/nuclide-ui/highlightOnUpdate.js
JSTests/test262/test/built-ins/Function/prototype/toString/function-declaration-non-simple-parameter-list.js

Files that triggered ImportDeclaration: importKind

JSTests/test262/test/language/module-code/eval-rqstd-order.js
JSTests/test262/test/language/module-code/eval-rqstd-once.js
JSTests/test262/test/language/module-code/eval-self-once.js
src/parser/test/esprima/ES6/import-declaration/import-named-empty.js
JSTests/test262/test/language/module-code/instn-once.js
JSTests/test262/test/language/module-code/instn-resolve-empty-import.js

Files that triggered RestElement: typeAnnotation

pkg/commons-node/observable.js
pkg/nuclide-jasmine/lib/waitsForPromise.js
tests/optional/optional_param4.js
tsrc/test/testRunner.js
pkg/nuclide-textedit/lib/main.js
pkg/commons-node/nuclideUri.js
pkg/nuclide-move-item-to-available-pane/spec/move-spec.js
tsrc/convert/convertRunner.js
tests/function/rest.js
tests/annot/issue-530.js
pkg/commons-node/redux-observable.js
pkg/commons-node/collection.js
pkg/nuclide-jasmine/lib/waitsForPromise.js
tsrc/new-test/new-testRunner.js
tests/spread/test4.js
website/docs/ref/syntax.doc.js
pkg/nuclide-file-tree/spec/helpers/BuildTempDirTree.js
pkg/nuclide-task/spec/fixtures/multiple-exports.js
pkg/nuclide-logging/spec/utils-spec.js
pkg/commons-node/observable.js
tests/destructuring/poly.js
pkg/nuclide-move-item-to-available-pane/spec/move-spec.js
pkg/nuclide-debugger/spec/Bridge-spec.js

Position of Identifiers sometimes covers other tokens

There are some cases where positional information in identifiers covers the whole parameter declaration.

Here is one of the failing cases:
https://github.com/bblfsh/javascript-driver/blob/master/fixtures/ext_typedecl.js#L9

const packageToBabelConfig = makeWeakCache(
  (file: ConfigFile): ConfigFile | null => {

In the AST positional info of the file identifier covers file: ConfigFile for some reason. See TODO.

JSX languages are not part of the JavaScript driver

This is analogy of bblfsh/bash-driver#39 and thus, although it's not a driver issue per-say, it is still logged here.

It arouse from trying to reproduce #42:

Parsing a JSX file using bblfshd go client client.NewParseRequest().ReadFile("source.js.jsx").UAST() results in panic: rpc error: code = Unknown desc = unexpected error: runtime failure: missing driver for language "jsx"

General solution is proposed in bblfsh/bash-driver#39 (comment) (will be moved to SDK issue soon)

[bug] roles: UNANNOTATED

Hi,
I noticed that JS driver produces a lot of roles: UNANNOTATED when extract UAST from https://github.com/babel/babel/blob/master/packages/babel-core/src/transformation/plugin-pass.js.
If you need any other information - I will be glad to answer!

Several failures with the attached files

js_fail.tar.gz

The files attached above produce several problems with the JS Driver:

Blacklisted types that should have been promoted to semantic objects (FunctionDeclaration, ImportDeclaration, ImportSpecifier, BlockStatement, etc).
partial parse: message is not defined errors.
Some failed checks on transforms like check: key "specifiers": field was not used: leadingComments.

They also generate many Unannotated roles but I'll be fixing those as part of #28 and #29.

Missing token for StringLiteralTypeAnnotation

how to reproduce:

wget https://raw.githubusercontent.com/reduxjs/redux/6949ba0ce1d25aca9fa40d250d81c7e09beff30f/examples/todos-flow/src/types/todos.js
docker run -d --rm --name style_analyzer_bblfshd_new --privileged -p 9432:9432 bblfsh/bblfshd\:v2.11.0 --log-level DEBUG
docker exec style_analyzer_bblfshd_new bblfshctl driver install javascript docker://bblfsh/javascript-driver:v2.6.0
docker cp todos.js style_analyzer_bblfshd_new:todos.js
docker exec -it style_analyzer_bblfshd_new bblfshctl parse todos.js | grep -A 16 StringLiteralTypeAnnotation

output:

.  .  .  .  .  .  .  .  .  .  .  .  .  .  1: StringLiteralTypeAnnotation {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Roles: Declaration,Type,String,Literal
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  StartPosition: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Offset: 256
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Line: 20
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Col: 13
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  EndPosition: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Offset: 266
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Line: 20
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Col: 23
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Properties: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  internalRole: value
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  value: ADD_TODO
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  }
--
  .  .  .  .  .  .  .  .  .  .  .  .  .  .  1: StringLiteralTypeAnnotation {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Roles: Declaration,Type,String,Literal
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  StartPosition: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Offset: 303
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Line: 21
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Col: 13
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  EndPosition: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Offset: 316
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Line: 21
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Col: 26
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Properties: {
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  internalRole: value
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  value: TOGGLE_TODO
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  }
.  .  .  .  .  .  .  .  .  .  .  .  .  .  }

It has value in its properties but no token.

Missing `roleDeclaration` for class declaration node

Code input:

class Hero {
    constructor(name, level) {
        this.name = name;
        this.level = level;
    }
}

The node represents class declaration:

children {
      internal_type: "Identifier"
      properties {
        key: "internalRole"
        value: "id"
      }
      token: "Hero"
      start_position {
        offset: 41
        line: 2
        col: 7
      }
      end_position {
        offset: 45
        line: 2
        col: 11
      }
      roles: EXPRESSION
      roles: IDENTIFIER
      roles: TYPE
      roles: NAME
    }

does not have DECLARATION in roles.

I/O timeout

From bblfsh logs we got a message of rpc error read |0: i/o timeout. Example of files that triggered this error:

Source/WebInspectorUI/UserInterface/Views/DataGrid.js
JSTests/stress/regress-159779-2.js
JSTests/ChakraCore/test/Basics/DomProperties.js
JSTests/ChakraCore/test/TaggedIntegers/addition.js
JSTests/ChakraCore/test/Generated/mod0.js
PerformanceTests/JetStream/simple/gcc-loops.cpp.js

Annotate the UAST

Comment positions does not cover the token

Positional information in comment nodes currently does not cover the // and /* tokens. See TODO.

Multibyte characters break positions

Hi,
I used vis tool for position and found that positions are broken at this file.

/* @noflow */

import type { NodePath } from "@babel/traverse";
import wrapFunction from "@babel/helper-wrap-function";
import annotateAsPure from "@babel/helper-annotate-as-pure";
import * as t from "@babel/types";

const awaitVisitor = {
  Function(path) {
    path.skip();
  },

  AwaitExpression(path, { wrapAwait }) {
    const argument = path.get("argument");

    if (path.parentPath.isYieldExpression()) {
      path.replaceWith(argument.node);
      return;
    }

    path.replaceWith(
      t.yieldExpression(
        wrapAwait
          ? t.callExpression(t.cloneNode(wrapAwait), [argument.node])
          : argument.node,
      ),
    );
  },
};

export default function(
  path: NodePath,
  helpers: { wrapAsync: Object, wrapAwait: Object },
) {
  path.traverse(awaitVisitor, {
    wrapAwait: helpers.wrapAwait,
  });

  const isIIFE = checkIsIIFE(path);

  path.node.async = false;
  path.node.generator = true;

  wrapFunction(path, t.cloneNode(helpers.wrapAsync));

  const isProperty =
    path.isObjectMethod() ||
    path.isClassMethod() ||
    path.parentPath.isObjectProperty() ||
    path.parentPath.isClassProperty();

  if (!isProperty && !isIIFE && path.isExpression()) {
    annotateAsPure(path);
  }

  function checkIsIIFE(path: NodePath) {
    if (path.parentPath.isCallExpression({ callee: path.node })) {
      return true;
    }

    // try to catch calls to Function#bind, as emitted by arrowFunctionToExpression in spec mode
    // this may also catch .bind(this) written by users, but does it matter? 🤔
    const { parentPath } = path;
    if (
      parentPath.isMemberExpression() &&
      t.isIdentifier(parentPath.node.property, { name: "bind" })
    ) {
      const { parentPath: bindCall } = parentPath;

      // (function () { ... }).bind(this)()

      return (
        // first, check if the .bind is actually being called
        bindCall.isCallExpression() &&
        // and whether its sole argument is 'this'
        bindCall.node.arguments.length === 1 &&
        t.isThisExpression(bindCall.node.arguments[0]) &&
        // and whether the result of the .bind(this) is being called
        bindCall.parentPath.isCallExpression({ callee: bindCall.node })
      );
    }

    return false;
  }
}

and it's visualization

_____________

import type { ________ } from _________________;
import ____________ from _____________________________;
import ______________ from ________________________________;
import * as _ from ______________;

const ____________ = {
  ________(____) {
    ____.____();
  },

  _______________(____, { _________ }) {
    const ________ = ____.___(__________);

    if (____.__________._________________()) {
      ____.___________(________.____);
      return;
    }

    ____.___________(
      _._______________(
        _________
          ? _.______________(_._________(_________), _______________)
          : ________.____,
      ),
    );
  },
};

export default function(
  ______________,
  _________________________________________________,
) {
  ____.________(____________, {
    _________: _______._________,
  });

  const ______ = ___________(____);

  ____.____._____ = _____;
  ____.____._________ = ____;

  ____________(____, _._________(_______._________));

  const __________ =
    ____.______________() ||
    ____._____________() ||
    ____.__________.________________() ||
    ____.__________._______________();

  if (!__________ && !______ && ____.____________()) {
    ______________(____);
  }

  function ___________(______________) {
    if (____.__________.________________({ ______: ____.____ })) {
      return ____;
    }

    ____________________________________________________________________________________________
    ___________________________________________________________________________    const { p__________} = p____
    if (
      p__________i__________________) &&
      t_i____________p__________n____p________ { n____ "______})
    ) {
      const { p__________ b________} = p__________

      /_____________________________________
      return (
        /_____________________________________________________        b________i________________) &&
        /__________________________________________        b________n____a_________l______=== 1_&&
        t_i________________b________n____a_________0_) &&
        /____________________________________________________________        b________p__________i________________{ c______ b________n____})
      );
    }

    return f_____
  }
}

we can see that offset is broken by 1 symbol after this comment

    // try to catch calls to Function#bind, as emitted by arrowFunctionToExpression in spec mode
    // this may also catch .bind(this) written by users, but does it matter? 🤔

could it be because of special character 🤔?

Add tags to repository

Repository should be tagged similar to other drivers. It should contain at least following tags (topics): babelfish, javascript, driver.

Two identical parse requests can return different results

Guys, I found one really weird big in bblfsh. I think it is related to daemon itself but cannot be 100% sure. So do not hesitate to move this issue if I am wrong.

Context

Machine learning team really want to have reproducible results everywhere where it is possible.
So we fix all random seeds to have a deterministic code. Suddenly we find out that two identical runs of style-analyzer quality estimation give different results: src-d/style-analyzer#557. Clues lead to bblfsh and I was able to reproduce the bug without our code. Let me show you.

How to reproduce

Run in bash

docker run -d --rm --name style_analyzer_bblfshd --privileged -p 9432\:9432 \
                bblfsh/bblfshd\:v2.11.0 --log-level DEBUG
docker exec style_analyzer_bblfshd bblfshctl driver install \
                javascript docker://bblfsh/javascript-driver\:v2.6.0
# clone repo with data
git clone https://github.com/laravel/telescope
git checkout 6f0a10ec586cfa1a22218b6778bf9c1572b97912

Also checked for v1.2.0 driver version.

Run in python3

from collections import defaultdict
import pickle
import time
import os
import bblfsh

bc = bblfsh.BblfshClient("localhost:9432")
res = defaultdict(lambda: defaultdict(int))
times = defaultdict(list)
iter_n = 10
for i in range(iter_n):
    for (dirpath, dirnames, filenames) in os.walk("telescope"):
        for filename in filenames:
            try:
                t_start = time.time()
                filepath = os.path.join(dirpath, filename)
                if ".git" in filepath:
                    continue
                resp = bc.parse(filename=filepath)
                if len(res[filepath]) > 0 and resp.status not in res[filepath]:
                    print("iter %.3d New status %d for filepath %s. counter: %s " % (
                        i + 1, resp.status, filepath, dict(res[filepath])))
                res[filepath][resp.status] += 1
                times[filepath].append(time.time() - t_start)
            except Exception as e:
                print(filepath, e)
                res[filepath][repr(e)] += 1

I do not think it is important but bblfsh package version is 2.12.6.

My output head

telescope/resources/.DS_Store Content must be UTF-8, ASCII or Base64 encoded
telescope/resources/.DS_Store Content must be UTF-8, ASCII or Base64 encoded
iter 003 New status 2 for filepath telescope/webpack.mix.js. counter: {0: 2} 
telescope/resources/.DS_Store Content must be UTF-8, ASCII or Base64 encoded
telescope/resources/.DS_Store Content must be UTF-8, ASCII or Base64 encoded
iter 004 New status 2 for filepath telescope/resources/js/routes.js. counter: {0: 3} 
iter 004 New status 2 for filepath telescope/resources/js/base.js. counter: {0: 3}

How to interpret

Mainly, if you see the message, that some file gets a new status that means that during the identical requests bblfsh return different statuses. For example, for telescope/webpack.mix.js and telescope/resources/js/routes.js it was able to parse file first but not the second time.

Notes

If you set iter_n to bigger number you get much more output messages.
Sometimes performance for big iter_n become super bad. Can be related to bblfsh/bblfshd#226
If you feed only js files it works great and always parse them for iter_n=100.

So it is important for ML team to have reproducible output from bblfsh.
Please tell me if you are able to reproduce the issue or need more info from my side.

Some role annotations can be improved

There are many annotations that are just "Incomplete" while they could easily have more roles adding more information (literals, new expressions, et cetera).

I'll do a pass and try to improve them as part of the task completing all role types.

panic: slice bounds out of range in uast/transformer

On parsing this not valid syntax using javascript-driver

empty-comment.js

// 
import * as React from "react";

a panic in transformer.(*commentElems).Split happens

panic: runtime error: slice bounds out of range

goroutine 12 [running]:
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.(*commentElems).Split(0xc420228728, 0x0, 0x0, 0xc420228728)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/semantic.go:160 +0x8eb

Steps to reproduce

docker run --rm -it -p 9432:9432 bblfsh/javascript-driver:v2.6.0 --log-level=debug
bblfsh-cli empty-comment.js

Expected

Actual

couldn't parse empty-comment.js: rpc error: code = Unavailable desc = transport is closing

Full driver log in details

time="2019-02-21T21:24:00Z" level=info msg="server listening in /tmp/rpc.sock (unix)" id=01d48z36zpa0hgdhjcdcwg7qrg language=javascript
time="2019-02-21T21:24:01Z" level=info msg="new driver instance started bblfsh/javascript-driver:latest (01d48z36zpa0hgdhjcdcwg7qrg)"
panic: runtime error: slice bounds out of range

goroutine 12 [running]:
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.(*commentElems).Split(0xc420228728, 0x0, 0x0, 0xc420228728)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/semantic.go:160 +0x8eb
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.(*commentUAST).Check(0xc4202a57a0, 0xc420296f20, 0xb744e0, 0xf030c0, 0xc42030cbc8, 0xffffffffffffff01, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/semantic.go:260 +0xd3
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.Fields.CheckObj(0xc420256480, 0x1, 0x1, 0xc420296f20, 0xc420256360, 0xed8d90, 0xc4202ee480, 0xc420228b58)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/ops.go:809 +0x15f
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.Obj.CheckObj(0xc4202ea840, 0xc420296f20, 0xc420256360, 0x5, 0xc42030cbc8, 0x1)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/ops.go:269 +0x61
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.(*opObjJoin).CheckObj(0xc4202eaff0, 0xc420296f20, 0xc4202509f0, 0x40c374, 0xaf701a, 0xc42028c810)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/ops.go:575 +0x1cf
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.checkObj(0xb73a60, 0xc4202eaff0, 0xc420296f20, 0xb74480, 0xc4202509f0, 0x532456, 0xc4202ff7a0, 0xc4202be580)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/ops.go:372 +0x5c
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.(*opObjJoin).Check(0xc4202eaff0, 0xc420296f20, 0xb74480, 0xc4202509f0, 0xc4202eb230, 0x200000000000001, 0xffffffffffffffff)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/ops.go:554 +0x55
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.mappings.Do.func1(0xb74480, 0xc4202509f0, 0xc420229128, 0xb744e0, 0xf030c0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/transformer.go:326 +0x1d6
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes.Apply(0xb74480, 0xc4202509f0, 0xc420229128, 0xc42025cef0, 0xc420251100, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes/node.go:875 +0x234
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes.Apply(0xb74300, 0xc420253860, 0xc420229128, 0xb744e0, 0xc42025cef0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes/node.go:863 +0x12e
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes.Apply(0xb74480, 0xc4202517a0, 0xc420229128, 0xf02228, 0xad8dc0, 0xc42035c2d0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/nodes/node.go:848 +0x3a4
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer.mappings.Do(0xee4220, 0xb, 0xb, 0xc42030ee40, 0xc42030f0e0, 0x0, 0x0, 0x0, 0xb74480, 0xc4202517a0, ...)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/uast/transformer/transformer.go:305 +0xeb
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver.Transforms.do.func1(0xaf5616, 0x8, 0xc4202d5900, 0x1, 0x1, 0x0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/transforms.go:99 +0x16b
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver.Transforms.do(0xaf6bea, 0xa, 0xc4202a7f10, 0x1, 0x1, 0xecea70, 0x1, 0x1, 0xc4202d5900, 0x1, ...)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/transforms.go:133 +0x4e3
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver.Transforms.Do(0xaf6bea, 0xa, 0xc4202a7f10, 0x1, 0x1, 0xecea70, 0x1, 0x1, 0xc4202d5900, 0x1, ...)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/transforms.go:89 +0x1c7
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver.(*driverImpl).Parse(0xc42014ce60, 0xb718c0, 0xc42020dc50, 0xc4201df110, 0x23, 0xc4202145d0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/impl.go:69 +0x268
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol.(*driverServer).Parse(0xc420320880, 0xb718c0, 0xc42020dc20, 0xc420348680, 0x0, 0x0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol/driver.go:87 +0x1af
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol._Driver_Parse_Handler.func1(0xb718c0, 0xc42020dc20, 0xa8dc80, 0xc420348680, 0xb718c0, 0xc42020dc20, 0xb75f80, 0xc4203161e0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol/driver.pb.go:179 +0x86
github.com/bblfsh/javascript-driver/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0xb718c0, 0xc42020da70, 0xa8dc80, 0xc420348680, 0xc42020bc80, 0xc42020bdc0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc/server.go:57 +0x2ba
github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol._Driver_Parse_Handler(0xa1b3e0, 0xc420320880, 0xb718c0, 0xc42020da70, 0xc42013c850, 0xc420315600, 0x0, 0x0, 0x1, 0x1)
	/go/src/github.com/bblfsh/javascript-driver/vendor/gopkg.in/bblfsh/sdk.v2/protocol/driver.pb.go:181 +0x167
github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc420140380, 0xb751a0, 0xc4200ae000, 0xc4202bc400, 0xc420394c30, 0xecff30, 0x0, 0x0, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc/server.go:1011 +0x4fc
github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc.(*Server).handleStream(0xc420140380, 0xb751a0, 0xc4200ae000, 0xc4202bc400, 0x0)
	/go/src/github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc/server.go:1249 +0x1318
github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc4201264b0, 0xc420140380, 0xb751a0, 0xc4200ae000, 0xc4202bc400)
	/go/src/github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc/server.go:680 +0x9f
created by github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
	/go/src/github.com/bblfsh/javascript-driver/vendor/google.golang.org/grpc/server.go:678 +0xa1
time="2019-02-21T21:24:01Z" level=error msg="request processed content 35 bytes error: rpc error: code = Unavailable desc = transport is closing" elapsed=1.2484346s filename=empty-comment.js language=javascript

</ details>

JSXIdentifier: comments are not handled

Found another validation error, this time for JSXIdentifier:

check: unused field(s) on node JSXIdentifier: leadingComments

It's not detected by you test suite and is triggered with this file:

jsx.js

// @flow strict
import * as React from "react";
import { graphql, QueryRenderer } from "react-relay";
import Alert from "@kiwicom/orbit-components/lib/Alert";
import InputField from "@kiwicom/orbit-components/lib/InputField";
import type { Environment } from "react-relay";

import environmentReal from "../../services/environment";
import PickerDropDown from "./primitives/PickerDropDown";
import NoResult from "./primitives/NoResult";
import ClickOutside from "../ClickOutside";
import Text from "../Text";
import LocationPickerResultList from "./components/LocationPickerResultList";
import getPlaceholder from "./services/placeholder";
import type { Location } from "../../records/Location";

// FIXME add arrow handling somehow neatly
// const ARROW_UP = 38;
// const ARROW_DOWN = 40;
// const ESC = 27;
// const ENTER = 13;

// FIXME @viktr solve these:
// centralize styles with IataPicker

// FIXME @oreqizer solve these:
// arrow handling

type Props = {|
  value: Location | null,
  onChange: (loc: Location) => void,
  label: string,
  icon: React.Node,
  // defaulted
  environment: Environment,
|};

type State = {|
  active: boolean,
  input: string,
|};

class LocationPicker extends React.Component<Props, State> {
  static defaultProps = {
    environment: environmentReal,
  };

  state = {
    active: false,
    input: "",
  };

  node: { current: any | HTMLDivElement } = React.createRef();

  handleClose = () => {
    this.setState({
      active: false,
    });
  };

  handleSelect = (item: Location) => {
    const { onChange } = this.props;

    onChange(item);
    this.setState({
      active: false,
    });
  };

  handleChange = (ev: SyntheticInputEvent<HTMLInputElement>) => {
    this.setState({
      input: ev.target.value,
      active: true,
    });
  };

  render() {
    const { value, label, icon, environment } = this.props;
    const { active, input } = this.state;

    const placeholder = value ? getPlaceholder(value) : null;

    return (
      <ClickOutside active={active} onClickOutside={this.handleClose}>
        <>
          <InputField
            inlineLabel
            placeholder={placeholder || label}
            onChange={this.handleChange}
            prefix={icon}
            value={active ? input : ""}
          />
          {input && active && (
            <QueryRenderer
              environment={environment}
              query={graphql`
                query LocationPickerQuery($input: String!) {
                  allLocations(last: 50, search: $input) {
                    ...LocationPickerResultList_list
                  }
                }
              `}
              variables={{ input }}
              render={res => {
                if (res.error) {
                  return (
                    <Alert type="critical">
                      <Text t="common.api_error" />
                    </Alert>
                  );
                }

                if (!res.props) {
                  return null;
                }

                if (!res.props.allLocations) {
                  // TODO render this in the list if length is 0
                  return (
                    <NoResult>
                      <Text t="forms.places_no_results" />
                    </NoResult>
                  );
                }

                return (
                  <PickerDropDown ref={this.node}>
                    <LocationPickerResultList
                      list={res.props.allLocations}
                      selectedId={value && value.id}
                      onSelect={this.handleSelect}
                    />
                  </PickerDropDown>
                );
              }}
            />
          )}
        </>
      </ClickOutside>
    );
  }
}

export default LocationPicker;

Offset mismatch

https://github.com/facebook/react/blob/master/fixtures/unstable-async/suspense/src/api/data.js

"trueadm" vs. "ueadm: " according to the offsets

https://github.com/facebook/react/blob/master/fixtures/attribute-behavior/src/App.js

"width" vs. "idth}" according to the offsets

UASTv2 has no tokens for JSX Nodes

JSX is a syntax extension to JavaScript. It can be converted to a proper JS with babel tool.
The bug I found is that UASTv1 provides a lot of tokens while UASTv2 don't
Here is the data to reproduce:

The JSX code

class TestIdTestApp extends React.Component {
  render() {
    return (
      <View>
        <Text testID="Text">text</Text>
      </View>
    );
  }
}

UASTv1

Start Position  Token              Internal Role      Roles Tree                                              
                                                                                                              
(0, 1, 1)       |''|               File               FILE                                                    
(0, 1, 1)       |''|               Program            ┣ MODULE                                                
(0, 1, 1)       |''|               ClassDeclaration   ┃ ┣ DECLARATION, TYPE, STATEMENT                        
(6, 1, 7)       |'TestIdTestApp'|  Identifier         ┃ ┃ ┣ EXPRESSION, IDENTIFIER, TYPE, NAME                
(28, 1, 29)     |''|               MemberExpression   ┃ ┃ ┣ QUALIFIED, EXPRESSION, IDENTIFIER, TYPE, BASE     
(28, 1, 29)     |'React'|          Identifier         ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER                          
(34, 1, 35)     |'Component'|      Identifier         ┃ ┃ ┃ ┗ EXPRESSION, IDENTIFIER                          
(44, 1, 45)     |''|               ClassBody          ┃ ┃ ┣ TYPE, BODY                                        
(48, 2, 3)      |''|               ClassMethod        ┃ ┃ ┃ ┣ DECLARATION, FUNCTION, STATEMENT                
(48, 2, 3)      |'render'|         Identifier         ┃ ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER, KEY, NAME             
(57, 2, 12)     |''|               BlockStatement     ┃ ┃ ┃ ┃ ┣ FUNCTION, BODY, STATEMENT, BLOCK, SCOPE, VALUE
(63, 3, 5)      |''|               ReturnStatement    ┃ ┃ ┃ ┃ ┃ ┣ STATEMENT, RETURN                           
(78, 4, 7)      |''|               JSXElement         ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                               
(78, 4, 7)      |''|               JSXOpeningElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(79, 4, 8)      |'View'|           JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ UNANNOTATED                           
(84, 4, 13)     |'\n        '|     JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(93, 5, 9)      |''|               JSXElement         ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(93, 5, 9)      |''|               JSXOpeningElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                           
(94, 5, 10)     |'Text'|           JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                         
(99, 5, 15)     |''|               JSXAttribute       ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                         
(99, 5, 15)     |'testID'|         JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                       
(106, 5, 22)    |'Text'|           StringLiteral      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ ┗ EXPRESSION, LITERAL, STRING       
(113, 5, 29)    |'text'|           JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                           
(117, 5, 33)    |''|               JSXClosingElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                           
(119, 5, 35)    |'Text'|           JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ ┗ UNANNOTATED                         
(124, 5, 40)    |'\n      '|       JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(131, 6, 7)     |''|               JSXClosingElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(133, 6, 9)     |'View'|           JSXIdentifier      ┗ ┗ ┗ ┗ ┗ ┗ ┗ ┗ ┗ UNANNOTATED

UASTv2

(0, 1, 1)       |''|               File               FILE                                                    
(0, 1, 1)       |''|               Program            ┣ MODULE                                                
(0, 1, 1)       |''|               ClassDeclaration   ┃ ┣ DECLARATION, TYPE, STATEMENT                        
(6, 1, 7)       |'TestIdTestApp'|  Identifier         ┃ ┃ ┣ EXPRESSION, IDENTIFIER, TYPE, NAME                
(28, 1, 29)     |''|               MemberExpression   ┃ ┃ ┣ QUALIFIED, EXPRESSION, IDENTIFIER, TYPE, BASE     
(28, 1, 29)     |'React'|          Identifier         ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER                          
(34, 1, 35)     |'Component'|      Identifier         ┃ ┃ ┃ ┗ EXPRESSION, IDENTIFIER                          
(44, 1, 45)     |''|               ClassBody          ┃ ┃ ┣ TYPE, BODY                                        
(48, 2, 3)      |''|               ClassMethod        ┃ ┃ ┃ ┣ DECLARATION, FUNCTION, STATEMENT                
(48, 2, 3)      |'render'|         Identifier         ┃ ┃ ┃ ┃ ┣ EXPRESSION, IDENTIFIER, KEY, NAME             
(57, 2, 12)     |''|               BlockStatement     ┃ ┃ ┃ ┃ ┣ STATEMENT, BLOCK, SCOPE, FUNCTION, BODY, VALUE
(63, 3, 5)      |''|               ReturnStatement    ┃ ┃ ┃ ┃ ┃ ┣ STATEMENT, RETURN                           
(78, 4, 7)      |''|               JSXElement         ┃ ┃ ┃ ┃ ┃ ┃ ┣ INCOMPLETE                                
(78, 4, 7)      |''|               JSXOpeningElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ BLOCK, INCOMPLETE                       
(79, 4, 8)      |''|               JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ IDENTIFIER, INCOMPLETE                
(84, 4, 13)     |''|               JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(93, 5, 9)      |''|               JSXElement         ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ INCOMPLETE                              
(93, 5, 9)      |''|               JSXOpeningElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ BLOCK, INCOMPLETE                     
(94, 5, 10)     |''|               JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ IDENTIFIER, INCOMPLETE              
(99, 5, 15)     |''|               JSXAttribute       ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ INCOMPLETE                          
(99, 5, 15)     |''|               JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ IDENTIFIER, INCOMPLETE            
(106, 5, 22)    |'"Text"'|         StringLiteral      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ ┗ EXPRESSION, LITERAL, STRING       
(113, 5, 29)    |''|               JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                           
(117, 5, 33)    |''|               JSXClosingElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                           
(119, 5, 35)    |''|               JSXIdentifier      ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┗ ┗ IDENTIFIER, INCOMPLETE              
(124, 5, 40)    |''|               JSXText            ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(131, 6, 7)     |''|               JSXClosingElement  ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┣ UNANNOTATED                             
(133, 6, 9)     |''|               JSXIdentifier      ┗ ┗ ┗ ┗ ┗ ┗ ┗ ┗ ┗ IDENTIFIER, INCOMPLETE

Should be easy to fix because all positional information is saved.

drivers are v1.2.0 and v.2.6.0

bblfsh / javascript-driver Goto Github PK

javascript-driver's People

Contributors

Stargazers

Watchers

Forkers

javascript-driver's Issues

Steps to reproduce

Expected

Actual

Steps to reproduce

Steps to reproduce

Expected

Actual

Context

How to reproduce

My output head

How to interpret

Notes

Steps to reproduce

Expected

Actual

Recommend Projects

Recommend Topics

Recommend Org

Jobs