travonted / tree-sitter-fennel

Treesitter grammar for the Fennel programming language

tree-sitter-fennel

Fennel grammar for tree-sitter

References

• Fennel language reference

tree-sitter-fennel's Issues

Make `>` and `<` valid function/symbol names

I am using the latest Neovim nightly version.

Symbols using those characters look like this:

Here is the code I used for this test:

(fn str->tbl [str]
  (icollect [chr (string.gmatch str ".")] chr))

(str->tbl "Hello") ; Returns ["H" "e" "l" "l" "o"]

(fn ->str [obj]
  (tostring obj))

(->str 1) ; Returns "1"

doesn't ignore shebang

it's not ignoring shebangs

Comments can cause Fennel nodes to end early

Reposting from nvim-treesitter/nvim-treesitter#997

Describe the bug

Hi there, I'm beginning to experiment with tree-sitter over in Conjure for Julia support Olical/conjure#184

I noticed an issue with Fennel where forms / nodes are cut off when there's comments mid way through the node. This only seems to happen in certain scenarios so I presume it's related to the grammer(?).

To Reproduce

;; This is okay
(+ 1
   ;
   )

;; This is not
(+ 1
   ;
   2)

It ends up producing (+ 1 when I call get_node_text. The range also changes so it appears to be in parsing, not re-writing back to a string.

;; Okay
{:content "(+ 1
   ;
   )"
 :range {:end [123 3] :start [121 0]}}

;; Not okay
{:content "(+ 1" :range {:end [120 3] :start [120 0]}}

Expected behavior

A clear and concise description of what you expected to happen.

Output of :checkhealth nvim_treesitter

health#nvim_treesitter#check

Installation

• OK: git executable found.
• OK: cc executable found.

Parser/Features H L F I

• fennel ✓ ✓ . .
  Legend: H[ighlight], L[ocals], F[olds], I[ndents]
  *) multiple parsers found, only one will be used
  x) errors found in the query, try to run :TSUpdate {lang}

Output of nvim --version

NVIM v0.5.0-dev
Build type: Release
LuaJIT 2.1.0-beta3
Compilation:
Compiled by nixbld

Features: +acl +iconv +tui
See ":help feature-compile"

system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "
/nix/store/anb133cb1hz7c5mfavly92wb437ds0qf-neovim-nightly-master/share/nvim"

Run :checkhealth for more info

Additional context

I'm happy to try and fix this, I just though I'd report it and get some more experienced eyes on it first. Thanks!

lambda

the word "lambda" gets highlighted for me, but not the symbol "λ". Fennel supports both

Support for Threading Macros

Threading seems to be broken when there is more than one operation

For example:

(fn c [b] (-> b (+ 1) (/ 2)))

yeilds an error

(list ( (symbol) (symbol)
 (list ( (symbol) (number) ))
 (list ( (symbol)
  (ERROR {)
  ))
 ))

whereas

(fn e [b] (-> b (+ 11)))

yeilds something workable

(fn ( fn docstring: (symbol)
 (parameters [
  (binding (symbol))
  ])
 (list ( (symbol) (symbol)
  (list ( (symbol) (number) ))
  ))
 ))

Create separate rule for `:keyword` style strings?

Currently, it seems like there's no way to have different coloring for :keyword strings and "quoted" strings. This is something supported by fennel.vim, FWIW.

Is this something that could be supported?

Thanks.

Publish on NPM (for use in Atom grammars)

Please add a CSON which defines the language and file recognition, so that this grammar can be used in Atom.

Can't use `:` as delimiter

I'm writing some queries and I can't seem to distinguish between methods and functions with keyword args.

(foo :bar)
(foo:bar)

Is it possible to add : or a keyword node?

What I've tried

((function_call
    name: ([(identifier) @variable
            (field_expression (identifier)*)])
    (field) @method))
; captures keywords as well.

((function_call
    name: ([(identifier) @variable
            (field_expression (identifier)*)])
    ":" @punctuation.delimiter
    (field) @method))
; throws errors

Parser show errors where there are none

In this fennel file, nvim-treesitter's playground shows errors in the following places

1. }} on line 2
2. {:* on line 18
3. the first : on line 18
4. }) on line 20
5. :✖) on line 22
6. :✖ on line 22
   see this screenshot https://imgur.com/3o1Te23

Missing license

Please specify a license for the code in this repository, so that I (and others) can legally use (or not use) this grammar. I want to bundle it in a VS Code extension.

Most Tree-sitter grammars use the MIT license.

Github's guide: https://choosealicense.com/

Treat comprehension and accumulation macros like `each` or `for`

As you can see in the following image, collect, icollect and accumulate are treated like normal functions instead of like each and for

It would be useful to have those macros treated like each and for, including the highlight and the indentation.

Thanks for you help!

symbols in quoted expressions are not highlighted like function symbols in non-quoted expressions

I don't know if I should report this in this repository or in https://github.com/nvim-treesitter/nvim-treesitter.

I have some macros that accept quoted expressions and when using those quoted expressions the first symbol of the list is not highlighted at all.

See this example:

As you can see if I remove the quoted the symbol is highlighted properly:

It also works properly if I use hash functions:

The issue is also present if ` is used to create the quoted expression.

Errors when using non-ascii identifiers in Fennel

To reproduce:

1. given f.fnl with content (local ⧔ 42)
2. highlighting is broken
3. using playground, it reports:

ERROR [0, 0] - [0, 14]
      ERROR [0, 7] - [0, 13]

Fennel itself can compile the above just fine:

$ fennel --compile f.fnl 
local _e2_a7_94 = 42
return nil

so it is valid code.

fucntion values in tables

File : https://github.com/p00f/dotfiles/blob/main/nvim/.config/nvim/fnl/dotfiles/module/glsetup.fnl

When the value is an inline function, there is an error:

When the value is a predfined function, this happens

?. macro produces an error

The ?. macro does not produce a valid node. The parser only seems to recognize ? as a symbol, . ends up as an error.

(?. table :value)
(. table :value)

(program [0, 0] - [2, 0]
  (list [0, 0] - [0, 17]
    (symbol [0, 1] - [0, 2])
    (ERROR [0, 2] - [0, 3])
    (symbol [0, 4] - [0, 9])
    (string [0, 10] - [0, 16]))
  (list [1, 0] - [1, 16]
    (symbol [1, 1] - [1, 2])
    (symbol [1, 3] - [1, 8])
    (string [1, 9] - [1, 15])))

Parentheses inside let breaks syntax highlight

As you can see in the next image, when I use a expression like the one in the image, the syntax breaks.

If I use [] instead, the syntax works.

It even breaks without the ' before the parentheses.

I am using the latest neovim nightly.

Edit: here is the code I used:

(macro map! [buffer?]
  (fn cons [obj seq]
    [obj (unpack seq)])
  (let [expr '("lhs" "rhs" {})
        expr (if buffer? (cons 0 expr) expr)
        expr (if buffer?
               (cons 'vim.api.nvim_buf_set_keymap expr)
               (cons 'vim.api.nvim_set_keymap expr))]
    (list (unpack expr))))
(macrodebug (map! true))

Update to 0.19.2

We are currently preparing a switch to the next ABI version of tree-sitter. All parsers need to be regenerated. We marked some parsers that have not yet done the update to enable users to still use them. Howevery, without the regenerated C files users will need the tree-sitter CLI installed.

nvim-treesitter/nvim-treesitter#1000

This query causes some problems with the new runtime if you want to investigate.

(field_expression
   (identifier)
   (field) @function)

Does not recognize doted names as function names

Indeed the current code specifically requires the function name to be an identifier, which cannot be doted

tree-sitter-fennel/grammar.js

Lines 213 to 214 in bc689e2

	_function_body: $ => seq(
	optional(field('name', $.identifier)),

Testing against source files?

I've also worked on a grammar for Fennel and have collected a list of repositories which I test against: repositories-list.txt

With clones of those made locally (e.g. under ~/src/fennel-repositories) from a tree-sitter grammar repository I typically do something like:

$ find ~/src/fennel-repositories -type f -regex '.*\.fnl' -print0 | xargs -0 npx tree-sitter parse --quiet > 2020-09-12.txt

Then I examine the results, investigate issues, tweak the grammar, etc.

May be you do something like this already, but FWIW.

Fix CI

Fuzzing

Could you try fuzzing for scanner.c with the instruction in
ikatyang/tree-sitter-markdown#14

Apparently it's easy to get stuck in a endless loop in the scanner.c

Integrate Aniseed macros as part of grammar

Aniseed is a compilation environment for Fennel for NeoVim that introduces a number of useful Clojure-like macros for Fennel. Currently the parser just recognizes these are raw macros, but since these macros are meant to replace and enhance features of Fennel, and since they also cannot be redefined by those using Aniseed, it might be a smart idea to include them in the grammar so they can be parsed appropriately. This Is a list of the current macros, but the following is a rough example:

; module header macro to export and import values
(module module-name
  {autoload {autoload some.file}
   require {require some.file}
   require-macros [require.some.macro.file]})
; defn replaces fn, is exported with the module
(defn function [] (print :x))
(fn function [] (print :x))

; def defines some module local value that can be exported
; it is scoped like local and behaves mostly the same, but exports easier
(def value "value")

There's more to these, but as you can see these macros can become quite useful when working in an Aniseed environment, so it would be useful if some of these could be added.

Operators are missing

Maybe I am misunderstanding something, but specials like +, or, <=, .., and ~= are being highlighted as function calls for me, and are not treated like other specials