SystemVerilog grammar for tree-sitter
License: MIT License
SystemVerilog grammar for tree-sitter.
npm i tree-sitter-verilog
SystemVerilog Test Suite report: https://symbiflow.github.io/sv-tests
Find what the differences are?
As @maxbrunsfeld recommended here: tree-sitter/tree-sitter#268
I have followed your suggestion about inlining of identifiers. That created ~50 inlines here: https://github.com/tree-sitter/tree-sitter-verilog/blob/inline_identifiers/grammar.js#L4552
I have totally rewritten conflicts section.
It was ~200 lines and now it is ~120 shorter lines.
https://github.com/tree-sitter/tree-sitter-verilog/blob/inline_identifiers/grammar.js#L4608
I have not measured the performance. I hope it is better.
The src/parser.c line count increased from 1871065 to 1883659 lines.
Compiled parser build/Release got bigger from 64740708 to 68840740 byte.
I have feelings that it is too big and open for suggestions on how to make it smaller.
I find that it doesn't support fold on instantiated instances，and port declaration, is it possible to support that?
Thanks,
Alex
I'm getting a parsing error when trying to parse this construct:
module foobar #(
) (
input logic i_clk,
input logic i_rst
);
always_comb begin : label
if (bar.baz[15:6] >= boo) begin
foo = 1'b0;
end else begin
foo = 1'b1;
end
end : label
endmoduleParsing result:
module_or_generate_item [2, 6] - [7, 11])
always_construct [2, 6] - [7, 11])
always_keyword [2, 6] - [13, 6])
statement [14, 6] - [7, 11])
statement_item [14, 6] - [7, 11])
seq_block [14, 6] - [7, 11])
simple_identifier [22, 6] - [27, 6])
ERROR [4, 7] - [29, 7])
constant_function_call [8, 7] - [15, 7])
function_subroutine_call [8, 7] - [15, 7])
subroutine_call [8, 7] - [15, 7])
method_call [8, 7] - [15, 7])
primary [8, 7] - [11, 7])
simple_identifier [8, 7] - [11, 7])
method_call_body [12, 7] - [15, 7])
method_identifier [12, 7] - [15, 7])
method_identifier [12, 7] - [15, 7])
simple_identifier [12, 7] - [15, 7])
data_type_or_implicit1 [15, 7] - [21, 7])
implicit_data_type1 [15, 7] - [21, 7])
packed_dimension [15, 7] - [21, 7])
constant_range [16, 7] - [20, 7])
constant_expression [16, 7] - [18, 7])
constant_primary [16, 7] - [18, 7])
primary_literal [16, 7] - [18, 7])
integral_number [16, 7] - [18, 7])
decimal_number [16, 7] - [18, 7])
unsigned_number [16, 7] - [18, 7])
constant_expression [19, 7] - [20, 7])
constant_primary [19, 7] - [20, 7])
primary_literal [19, 7] - [20, 7])
integral_number [19, 7] - [20, 7])
decimal_number [19, 7] - [20, 7])
unsigned_number [19, 7] - [20, 7])
ERROR [22, 7] - [24, 7])
simple_identifier [25, 7] - [28, 7])
The error seems to be located at the bar.baz[15:6] part.
Removing either the .baz or [15:6] resolves the error.
I'm running the following 64 bit tree-sitter executable on Windows 10:
D:\Projects\tree-sitter-verilog-master>tree-sitter --version
tree-sitter 0.17.1 (5080de496a87fc2b9609af80e3d893a04e2b868c)
When trying running the generate command the following error is produced:
D:\Projects\tree-sitter-verilog-master>tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli\src\generate\render.rs:555:41
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The full back trace is as follows:
D:\Projects\tree-sitter-verilog-master>tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli\src\generate\render.rs:555:41
stack backtrace:
0: 0x7ff6c674d9be - <unknown>
1: 0x7ff6c676d2ec - <unknown>
2: 0x7ff6c6744a13 - <unknown>
3: 0x7ff6c675106b - <unknown>
4: 0x7ff6c6750cb8 - <unknown>
5: 0x7ff6c675183f - <unknown>
6: 0x7ff6c67513bf - <unknown>
7: 0x7ff6c676bbb0 - <unknown>
8: 0x7ff6c676b952 - <unknown>
9: 0x7ff6c65bf5fc - <unknown>
10: 0x7ff6c65d4b1e - <unknown>
11: 0x7ff6c64f35cc - <unknown>
12: 0x7ff6c64f0af7 - <unknown>
13: 0x7ff6c646510b - <unknown>
14: 0x7ff6c64621f2 - <unknown>
15: 0x7ff6c646bca6 - <unknown>
16: 0x7ff6c6751a36 - <unknown>
17: 0x7ff6c646bc97 - <unknown>
18: 0x7ff6c678dcb9 - <unknown>
19: 0x7ffa09fe6fd4 - BaseThreadInitThunk
20: 0x7ffa0a7dcec1 - RtlUserThreadStart
I know this did generate using an earlier release. From memory I think 16.9 worked fine.
Failed in:
https://symbiflow.github.io/sv-tests/#tree_sitter_verilog|5.6.4|preproc_test_13
/*
:name: preproc_test_13
:description: Test
:should_fail: 0
:tags: 5.6.4
*/
`define LONG_MACRO(
a, b="(3,2)", c=(3,2)) \
a + b /c +345Hi,
I can not figure out why the installation fails on Windows when it was working two/three moths ago. https://github.com/TerosTechnology/colibri/runs/400677109?check_suite_focus=true
In Linux is working fine. Is this something directly related with tree-sitter-verilog?
Thanks in advance, I really apreciate your work on GitHub.
> [email protected] install C:\Users\isma\node_modules\tree-sitter-verilog
> node bin/install.js
build
events.js:288
throw er; // Unhandled 'error' event
^
Error: spawn node-gyp ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (internal/child_process.js:273:12)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
errno: 'ENOENT',
code: 'ENOENT',
syscall: 'spawn node-gyp',
path: 'node-gyp',
spawnargs: [ 'configure', 'build' ]
}
npm WARN enoent ENOENT: no such file or directory, open 'C:\Users\isma\package.json'
npm WARN isma No description
npm WARN isma No repository field.
npm WARN isma No README data
npm WARN isma No license field.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node bin/install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
always block sensitivity list error
always @(a or b or c or d) begin endimplement and test 22.11 pragma directive
https://symbiflow.github.io/sv-tests/#tree_sitter_verilog|5.6.4|pragma-directive
Given the expression a + b, the operator + is excluded from the syntax tree. This seems to be due to the use of the token() function on the operator in the exprOp() and constExprOp() functions.
tree-sitter-verilog/grammar.js
Lines 73 to 79 in 8d97cbd
The solution is to remove the use of the token() function.
I suspect this escaped testing because the tests are done on the S expressions which exclude unnamed nodes.
Given the input:
function f();
x = a + b;
y = a * b + c;
endfunctionHere is a diff of the concrete syntax tree before and after removing the token() function:
--- with_token_function 2019-08-08 17:03:39.702042600 -0700
+++ without_token_function 2019-08-08 17:18:39.611493000 -0700
@@ -1,67 +1,70 @@
S Expression:
(source_file (package_item (function_declaration (function_body_declaration (function_identifier (function_identifier (simple_identifier))) (function_statement_or_null (function_statement (statement (statement_item (blocking_assignment (o
perator_assignment (variable_lvalue (identifier (simple_identifier))) (assignment_operator) (expression (expression (primary (identifier (simple_identifier)))) (expression (primary (identifier (simple_identifier))))))))))) (function_statem
ent_or_null (function_statement (statement (statement_item (blocking_assignment (operator_assignment (variable_lvalue (identifier (simple_identifier))) (assignment_operator) (expression (expression (expression (primary (identifier (simple_
identifier)))) (expression (primary (identifier (simple_identifier))))) (expression (primary (identifier (simple_identifier)))))))))))))))
CST with unnamed nodes:
source_file: function f();
package_item: function f();
function_declaration: function f();
function: function
function_body_declaration: f();
function_identifier: f
function_identifier: f
simple_identifier: f
(: (
): )
;: ;
function_statement_or_null: x = a + b;
function_statement: x = a + b;
statement: x = a + b;
statement_item: x = a + b;
blocking_assignment: x = a + b
operator_assignment: x = a + b
variable_lvalue: x
identifier: x
simple_identifier: x
assignment_operator: =
=: =
expression: a + b
expression: a
primary: a
identifier: a
simple_identifier: a
+ +: +
expression: b
primary: b
identifier: b
simple_identifier: b
;: ;
function_statement_or_null: y = a * b + c;
function_statement: y = a * b + c;
statement: y = a * b + c;
statement_item: y = a * b + c;
blocking_assignment: y = a * b + c
operator_assignment: y = a * b + c
variable_lvalue: y
identifier: y
simple_identifier: y
assignment_operator: =
=: =
expression: a * b + c
expression: a * b
expression: a
primary: a
identifier: a
simple_identifier: a
+ *: *
expression: b
primary: b
identifier: b
simple_identifier: b
+ +: +
expression: c
primary: c
identifier: c
simple_identifier: c
;: ;
endfunction: endfunctionImplement and test 22.14 {begin_keywords, end_keywords} directives
https://symbiflow.github.io/sv-tests/#tree_sitter_verilog|5.6.4|begin-keywords
import "DPI-C" function void myInt();
There seems to be a problem in the concat grammar. Given this simple verilog tests:
module test(input [4:0]b, input [4:0]c, output [8:0] a);
assign a = { b[1:0], c[2:1] };
endmodule
WHen I run:
./node_modules/tree-sitter-cli/tree-sitter parse test.v
I get this error at the end:
test.v 0 ms (MISSING "++" [3, 38] - [3, 38])
but the verilog is fine.
If I change to this, it parses:
module test(input [4:0]b, input [4:0]c, output [8:0] a);
assign a = { b, c[2:1] };
endmodule
Hi,
For positional instantiation, seems the first port will always be output and others will be input?
I cloned this repo for the verilog language and am trying to use it with tree sitter; however I am getting the following error.
ValueError: Incompatible Language version 12. Must not be between 13 and 13
Tried to both manually change the version number and also go back to an older commit but didnt have much luck.
It might be of interest to you that the SymbiFlow project has started working on an open source System Verilog compliance suite which can be found at https://github.com/SymbiFlow/sv-tests
It's still very early stages, but its goal is to cover the whole of the System Verilog LRM.
The project is currently trying to produce a "status report" for popular open source tools and your tree sitter parser is included. The resulting output report can be found at https://symbiflow.github.io/sv-tests/.
Hi I have a highliting issue on a system-verilog file while using tree-sitter on neovim.
These are my system-verilog lines of code:
There is some highliting problem because the blue color is used for all the lines after the "" empty string on top.
The correct output (by disabling tree-sitter) is the following one:
I hope that this is not too much out of the scope for this repo but I was suggested to post the issue here because it probably relates to this project.
If I change the order of parameter and constant in the following expression:
tree-sitter-verilog/corpus/assign.txt
Line 262 in 0ca8442
...
(constant_expression
(constant_expression (constant_primary
(primary_literal (integral_number (decimal_number (unsigned_number))))))
(constant_expression (constant_primary
(parameter_identifier (simple_identifier))))
)
...
to:
assign a = b[Param+1:0]; Then parser fails.
Tests:
$sformatf("%m")
Failure example:
module foo (
input a,
`line 123 "foo.v" 0
output b
);
endmoduleI was happy to find there is verilog parser using tree-sitter.
When I first cloned the tree-sitter-verilog and tested it, I get an error in an empty module without () like this:
module mod;
endmodulelist_of_port_declarations is enclosed by [], so optional.)module_declaration ::=
...
| { attribute_instance } module_keyword module_identifier [ module_parameter_port_list ] [ list_of_port_declarations ] ; { non_port_module_item }
endmodule
...
module_ansi_header ::=
{ attribute_instance } module_keyword [ lifetime ] module_identifier
{ package_import_declaration }1 [ parameter_port_list ] [ list_of_port_declarations ] ;
Looking at the grammer.js you wrote, the fix looks simple. You can wrap choice in module_declaration with optional to fix this.
The module_declaration part of grammer.js looks different from the standard BNF. It doesn't have to be the same structure as standard BNF, but is there a reason?
Allow simple_text_macro_usage statements in reasonable places of the grammar:
Testing a json parser that do not manage utf-8 characters I found that this grammar has some that I'm not sure it's intentional:
"value": "–>" ///verilog
"value": "->" ///cpp do not uses utf-8
...
"value": "–" ///verilog
"value": "––" ///verilog
"value": "[–>" ///verilog
Treesitter for Verilog is not working. When I ran :checkhealth, a series of errors citing an ABI version mismatch appear. I have tried running :TSUpdate verilog and it is up to date.
Excerpt from :checkhealth output:
health#nvim_treesitter#check
========================================================================
## Installation
- OK: `tree-sitter` found 0.20.0 (parser generator, only needed for :TSInstallFromGrammar)
- OK: `node` found v16.4.2 (only needed for :TSInstallFromGrammar)
- OK: `git` executable found.
- OK: `cc` executable found. Selected from { vim.NIL, "cc", "gcc", "clang", "cl" }
- OK: Neovim was compiled with tree-sitter runtime ABI version 13 (required >=13). Parsers must be compatible with runtime ABI.
## Parser/Features H L F I J
- bibtex ✓ . ✓ ✓ .
- python ✓ ✓ ✓ ✓ ✓
- comment ✓ . . . .
- lua ✓ ✓ ✓ ✓ ✓
- json ✓ ✓ ✓ ✓ .
- cpp ✓ ✓ ✓ ✓ ✓
- c ✓ ✓ ✓ ✓ ✓
- verilog x x x . x
- devicetree ✓ ✓ ✓ ✓ ✓
- dockerfile ✓ . . . ✓
- rust ✓ ✓ ✓ ✓ ✓
Legend: H[ighlight], L[ocals], F[olds], I[ndents], In[j]ections
+) multiple parsers found, only one will be used
x) errors found in the query, try to run :TSUpdate {lang}
The following errors have been detected:
- ERROR: verilog(highlights): /usr/share/nvim/runtime/lua/vim/treesitter/language.lua:33: ABI version mismatch for ~/.local/share/nvim/site/parser/verilog.so: supported between 13 and 13, found 12
- ERROR: verilog(locals): /usr/share/nvim/runtime/lua/vim/treesitter/language.lua:33: ABI version mismatch for ~/.local/share/nvim/site/parser/verilog.so: supported between 13 and 13, found 12
- ERROR: verilog(folds): /usr/share/nvim/runtime/lua/vim/treesitter/language.lua:33: ABI version mismatch for ~/.local/share/nvim/site/parser/verilog.so: supported between 13 and 13, found 12
- ERROR: verilog(injections): /usr/share/nvim/runtime/lua/vim/treesitter/language.lua:33: ABI version mismatch for ~/.local/share/nvim/site/parser/verilog.so: supported between 13 and 13, found 12Other details:
nvim --versionNVIM v0.5.0
Build type: Release
LuaJIT 2.0.5
Compilation: /usr/bin/cc -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNVIM_TS_HAS_SET_MATCH_LIMIT -O2 -DNDEBUG -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/build/neovim/src/neovim-0.5.0/build/config -I/build/neovim/src/neovim-0.5.0/src -I/usr/include -I/build/neovim/src/neovim-0.5.0/build/src/nvim/auto -I/build/neovim/src/neovim-0.5.0/build/include
Compiled by builduser
Features: +acl +iconv +tui
See ":help feature-compile"
system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/usr/share/nvim"
module top ();
initial begin
p1.lo = 4'h5;
end
endmoduletests/chapter-7/structures/unpacked/basic.sv
tests/chapter-7/structures/unpacked/default-value.sv
tests/chapter-7/unions/packed/basic.sv
tests/chapter-7/unions/basic.sv
https://symbiflow.github.io/sv-tests/#tree_sitter_verilog|6.20.2|parameter_aggregate
/*
:name: parameter_aggregate
:description: parameter aggregate type tests
:should_fail: 0
:tags: 6.20.2
*/
module top();
parameter logic [31:0] p [3:0] = '{1, 2, 3, 4};
endmoduleCURL / WGET switch
Google is actively developing and maintaining a full system verilog parser. Check these out:
https://github.com/google/verible#systemverilog-developer-tools
https://github.com/google/verible/tree/master/verilog/parser
What parser do you use? Verible would be recommended as Google won't abandon it given it has silicon development teams that develop silicon in-house (like their TPU computing).
in test corpus/module.txt -> empty module with ANSI io
changing: module mod (output out); to module mod (input out);
brakes parser
https://symbiflow.github.io/sv-tests/tests/chapter-8/8.6--methods.sv.html
https://symbiflow.github.io/sv-tests/tests/chapter-8/8.9--static_properties.sv.html
https://symbiflow.github.io/sv-tests/tests/chapter-8/8.12--assignment.sv.html
https://symbiflow.github.io/sv-tests/tests/chapter-8/8.12--shallow_copy.sv.html
https://symbiflow.github.io/sv-tests/tests/chapter-8/8.24--out_of_block_methods.sv.html
https://symbiflow.github.io/sv-tests/tests/chapter-15/15.4--mailbox-blocking.sv.html
Fail on:
module block_tb ();
initial begin
@ev
end
endmodulewith:
node: 14.16.0
tree-sitter-cli: 0.19.2
running:
tree-sitter generate
produces this error:
Unresolved conflict for symbol sequence:
'[' _identifier • '.' …
Possible interpretations:
1: '[' (_sequence_identifier _identifier) • '.' …
2: '[' (generate_block_identifier _identifier) • '.' …
3: '[' (hierarchical_identifier_repeat1 _identifier • '.') (precedence: 0, associativity: Left)
4: '[' (let_expression _identifier) • '.' … (precedence: 0, associativity: Left)
5: '[' (primary _identifier • select1) (precedence: 0, associativity: Left)
6: '[' (primary _identifier) • '.' … (precedence: 0, associativity: Left)
7: '[' (sequence_instance _identifier) • '.' … (precedence: 0, associativity: Left)
8: '[' (tf_call _identifier) • '.' … (precedence: 0, associativity: Left)
Possible resolutions:
1: Specify a higher precedence in `primary` and `hierarchical_identifier_repeat1` than in the other rules.
2: Specify a higher precedence in `sequence_instance` than in the other rules.
3: Specify a higher precedence in `let_expression` than in the other rules.
4: Specify a higher precedence in `tf_call` than in the other rules.
5: Specify a higher precedence in `primary` than in the other rules.
6: Specify a higher precedence in `generate_block_identifier` than in the other rules.
7: Specify a higher precedence in `_sequence_identifier` than in the other rules.
8: Specify a left or right associativity in `sequence_instance` and `let_expression` and `tf_call` and `primary` and `generate_block_identifier` and `_sequence_identifier`
9: Add a conflict for these rules: `sequence_instance`, `let_expression`, `tf_call`, `primary`, `generate_block_identifier`, `_sequence_identifier`
always_comb
{a,b,c} = d;Fails:
module class_tb ();
function void a::b();
endfunction
endmoduleHi,
I'm developing a extension for vscode and I'm using tree-sitter-verilog. I have a lot of problems with the NODE_MODULE_VERSION:
Activating extension 'teros-technology.teroshdl' failed: The module '/home/carlos/repo/colibri/node_modules/tree-sitter/build/Release/tree_sitter_runtime_binding.node' was compiled against a different Node.js version using NODE_MODULE_VERSION 72. This version of Node.js requires NODE_MODULE_VERSION 75. Please try re-compiling or re-installing the module (for instance, using `npm rebuild` or `npm install`)..
If I want to use tree-sitter-verilog, can my extension only run in a specific version of Visual Studio Code cause of the nodejs version?
Could you add a v0.0 tag to the first commit (1b857d6) in this repository? This will make git-describe return a useful value for this repository.
git tag -a v0.0 1b857d6c0adbff3d2c9153ba1502f92538dc1032 -m "First commit"
git push origin v0.0
My tree-sitter-verilog parser can go into the infinite loop.
Here is simple case: tree-sitter parse package.json
How can I fix my grammar to prevent it?
Fail to assume new to be an identifier, not the class_new
module class_tb ();
initial begin
inst = new;
end
endmodule
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.