tree-sitter / tree-sitter-regex

Tree-sitter parser for regular expressions

tree-sitter-regex

Regex grammar for tree-sitter

tree-sitter-regex's Issues

Terminology alignment with PCRE

According to https://github.com/tree-sitter/tree-sitter-regex/blob/master/corpus/main.txt ,
current terminology of Tree sitter is aligned with PCRE pattern for

• sequence
• assertion
• character classes
• quantifier

But not with

• alternative where disjunction is used.

i suggest to keep pcre naming and use alternative instead of disjunction.

Abbreviations come easy too.
Consider (rep (alt 0 1) 1 n) for binary integer.
Very simple to read, simple to write and to understand.

Those ones could also help to get a leaner ast

• sequence (seq)
• alternative (alt)
• repetition (rep)
• optional (opt)
• ...

Lookbehind assertions

$ echo "/(?<!not) preceeding/" > re.re
$ tree-sitter parse re.re
(pattern [0, 0] - [1, 0]
  (ERROR [0, 0] - [0, 9]
    (pattern [0, 0] - [0, 8]
      (term [0, 0] - [0, 8]
        (pattern_character [0, 0] - [0, 1])
        (ERROR [0, 1] - [0, 5])
        (pattern_character [0, 5] - [0, 6])
        (pattern_character [0, 6] - [0, 7])
        (pattern_character [0, 7] - [0, 8]))))
  (term [0, 9] - [0, 21]
    (pattern_character [0, 9] - [0, 10])
    (pattern_character [0, 10] - [0, 11])
    (pattern_character [0, 11] - [0, 12])
    (pattern_character [0, 12] - [0, 13])
    (pattern_character [0, 13] - [0, 14])
    (pattern_character [0, 14] - [0, 15])
    (pattern_character [0, 15] - [0, 16])
    (pattern_character [0, 16] - [0, 17])
    (pattern_character [0, 17] - [0, 18])
    (pattern_character [0, 18] - [0, 19])
    (pattern_character [0, 19] - [0, 20])
    (pattern_character [0, 20] - [0, 21])))
re.re   0 ms    (ERROR [0, 0] - [0, 9])

shorthand `-` in classes should ERROR

$ cat shorthand.re
/[\w-.]/
$ tree-sitter parse shorthand.re
(pattern [0, 0] - [1, 0]
  (term [0, 0] - [0, 8]
    (pattern_character [0, 0] - [0, 1])
    (character_class [0, 1] - [0, 7]
      (class_range [0, 2] - [0, 6]
        (character_class_escape [0, 2] - [0, 4])
        (class_character [0, 5] - [0, 6])))
    (pattern_character [0, 7] - [0, 8])))

expected: error

Error when trying to parse a regex with a dash in a character set

Hello !

I noticed that there is an ERROR type node when I try to parse regex expression like this "[a-]" but not with "[-a]" or "[-]" am I doing something wrong ?

>>> parser.parse("[a-]").root_node.sexp()
(pattern (term (character_class (class_character) (ERROR))))

>>> parser.parse("[-a]").root_node.sexp()
(pattern (term (character_class (class_character) (class_character))))

I believe that in this case tree sitter makes a confusion with class range nodes. Is there any workaround to this issue ?
ps: I am using the tree-sitter 0.19 version

Decimal backreference escapes

I tried to fix this problem:

$ echo '(a)\1' > /tmp/decimal-backreference.txt
$ node_modules/.bin/tree-sitter parse /tmp/decimal-backreference.txt 
(pattern [0, 0] - [1, 0]
  (term [0, 0] - [0, 3]
    (anonymous_capturing_group [0, 0] - [0, 3]
      (pattern [0, 1] - [0, 2]
        (term [0, 1] - [0, 2]
          (pattern_character [0, 1] - [0, 2])))))
  (ERROR [0, 3] - [0, 5]
    (ERROR [0, 3] - [0, 5])))
/tmp/decimal-backreference.txt	0 ms	(ERROR [0, 3] - [0, 5])

... in these two branches of mine:

th-we@4d1076d
th-we@135e904

Why is that? (As a general question: What's the best place to get help with problems writing tree-sitter grammars?)

/{a/

=>  echo '{a' > test.re
=>  tree-sitter parse test.re
(pattern [0, 0] - [1, 0]
  (ERROR [0, 0] - [0, 1])
  (term [0, 1] - [0, 2]
    (pattern_character [0, 1] - [0, 2])))
test.re	0 ms	ERROR [0, 0] - [0, 1]