neovim / tree-sitter-vimdoc

Tree-sitter parser for Vim help files

Documentation | Chat

Neovim is a project that seeks to aggressively refactor Vim in order to:

• Simplify maintenance and encourage contributions
• Split the work between multiple developers
• Enable advanced UIs without modifications to the core
• Maximize extensibility

See the Introduction wiki page and Roadmap for more information.

Features

• Modern GUIs
• API access from any language including C/C++, C#, Clojure, D, Elixir, Go, Haskell, Java/Kotlin, JavaScript/Node.js, Julia, Lisp, Lua, Perl, Python, Racket, Ruby, Rust
• Embedded, scriptable terminal emulator
• Asynchronous job control
• Shared data (shada) among multiple editor instances
• XDG base directories support
• Compatible with most Vim plugins, including Ruby and Python plugins

See :help nvim-features for the full list, and :help news for noteworthy changes in the latest version!

Install from package

Pre-built packages for Windows, macOS, and Linux are found on the Releases page.

Managed packages are in Homebrew, Debian, Ubuntu, Fedora, Arch Linux, Void Linux, Gentoo, and more!

Install from source

See BUILD.md and supported platforms for details.

The build is CMake-based, but a Makefile is provided as a convenience. After installing the dependencies, run the following command.

make CMAKE_BUILD_TYPE=RelWithDebInfo
sudo make install

To install to a non-default location:

make CMAKE_BUILD_TYPE=RelWithDebInfo CMAKE_INSTALL_PREFIX=/full/path/
make install

CMake hints for inspecting the build:

• cmake --build build --target help lists all build targets.
• build/CMakeCache.txt (or cmake -LAH build/) contains the resolved values of all CMake variables.
• build/compile_commands.json shows the full compiler invocations for each translation unit.

Transitioning from Vim

See :help nvim-from-vim for instructions.

Project layout

├─ cmake/           CMake utils
├─ cmake.config/    CMake defines
├─ cmake.deps/      subproject to fetch and build dependencies (optional)
├─ runtime/         plugins and docs
├─ src/nvim/        application source code (see src/nvim/README.md)
│  ├─ api/          API subsystem
│  ├─ eval/         Vimscript subsystem
│  ├─ event/        event-loop subsystem
│  ├─ generators/   code generation (pre-compilation)
│  ├─ lib/          generic data structures
│  ├─ lua/          Lua subsystem
│  ├─ msgpack_rpc/  RPC subsystem
│  ├─ os/           low-level platform code
│  └─ tui/          built-in UI
└─ test/            tests (see test/README.md)

License

Neovim contributions since b17d96 are licensed under the Apache 2.0 license, except for contributions copied from Vim (identified by the vim-patch token). See LICENSE for details.

Vim is Charityware.  You can use and copy it as much as you like, but you are
encouraged to make a donation for needy children in Uganda.  Please see the
kcc section of the vim docs or visit the ICCF web site, available at these URLs:

        https://iccf-holland.org/
        https://www.vim.org/iccf/
        https://www.iccf.nl/

You can also sponsor the development of Vim.  Vim sponsors can vote for
features.  The money goes to Uganda anyway.

tree-sitter-vimdoc's Issues

parse error: windows.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/windows.html

Context:

Closing a window
----------------

:q[uit]
:{count}q[uit]						*:count_quit*
CTRL-W q						*CTRL-W_q*

two lines are combined as one: options.txt

Context:

If you want to include a ':' in a set command precede it with a '\'.  The
backslash in front of the ':' will be removed.  Example:

The code block below are combined with the line following it (This sets the 'fillchars' option).

Also in context for option filetype

When a dot appears in the value then this separates two filetype
	names.  Example:

Same issue.

parse error: cmdline.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/cmdline.html

Context:

					*c_CTRL-R_CTRL-R* *c_<C-R>_<C-R>*
					*c_CTRL-R_CTRL-O* *c_<C-R>_<C-O>*
CTRL-R CTRL-R {register CTRL-F CTRL-P CTRL-W CTRL-A CTRL-L}
CTRL-R CTRL-O {register CTRL-F CTRL-P CTRL-W CTRL-A CTRL-L}
		Insert register or object under the cursor.  Works like
		|c_CTRL-R| but inserts the text literally.  For example, if

Severe slowdown on large files

Description

Neovim lags severely when moving around large files. The lag gets worse the closer you are to the bottom of the file.

To reproduce

:h netrw (4418 lines)
G
Observe slowdown with normal commands or simply holding k

embedded languages in codeblocks

This is a moonshot, but would it be possible (at least in principle) to detect the language of a code_block? In contrast to Markdown, Vimdoc doesn't use language markers, so it's nigh impossible to set up proper injections for them. At least being able to distinguish Lua/Vimscript/Other (the last highlighted as plain string) would be great.

Odd highlighting for `local-additions`

Some odd highlighting when looking at help.txt's local-additions section

w/o TreeSitter	w/ TreeSitter

text:

LOCAL ADDITIONS:				*local-additions*
|nvim-treesitter|  Treesitter configurations and abstraction layer for Neovim.
|packer.txt|                      A use-package inspired Neovim plugin manager
|matchit.txt|	Extended "%" matching

notice the h in matchit.txt is concealed

parse error: options.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/options.html

Context:

	   "number"	display signs in the 'number' column. If the number
			column is not present, then behaves like "auto".

	Note regarding 'orphaned signs': with signcolumn numbers higher than
	1, deleting lines will also remove the associated signs automatically,
	in contrast to the default Vim behavior of keeping and grouping them.
	This is done in order for the signcolumn appearance not appear weird

Umm. What just happened? Where am I?

parse error: options.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/options.html

Context:

		    Value must be 50 or less.
	maxwid	    Maximum width of the item.  Truncation occurs with a '<'
		    on the left for text items.  Numeric items will be
		    shifted down to maxwid-2 digits followed by '>'number
		    where number is the amount of missing digits, much like
		    an exponential notation.
	item	    A one letter code as described below.

parse error: motion.txt , spurious listitem

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/motion.html

Context:

-  <minus>		[count] lines upward, on the first non-blank
			character |linewise|.

+		or					*+*
CTRL-M		or					*CTRL-M* *<CR>*
<CR>			[count] lines downward, on the first non-blank
			character |linewise|.

parse error for indent text block

The display here is not right compared to builtin help file.
Maybe the text block should be rendered as code blocks.

parse error: lsp.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/lsp.html

Context:

    the file system upwards starting from the current directory until either a
    `pyproject.toml` or `setup.py` file is found.

    `workspace_folders` a list of { uri:string, name: string } tables. The
    project root folders used by the language server. If `nil` the property is
    derived from the `root_dir` for convenience.

Tilde not concealed in column headers

E.g., for :h api-types, the first table has a column heading

Basic types ~

which corresponds to the playground

block [94, 0] - [107, 0]
  line [94, 0] - [95, 0]
    column_heading [94, 0] - [95, 0]
      name: word [94, 0] - [94, 5]
      name: word [94, 6] - [94, 11]

Since ~ does not appear as an (anonymous) node, it cannot be captured and thus concealed as in the legacy syntax.

This is the same issue (with the same likely fix) as in #33.

missed codeblock

In this example from eval.txt, the middle codeblock is not recognized:

Note that in the command: >
	:if "foo"
	:" NOT executed
"foo" is converted to 0, which means FALSE.  If the string starts with a
non-zero number it means TRUE: >
	:if "8foo"
	:" executed
To test for a non-empty string, use empty(): >
	:if !empty("foo")
<

parse error: pi_spec.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pi_spec.html

Context:

2. Customizing					*spec-customizing*

The format string				*spec_chglog_format*
-----------------

You can easily customize how your spec file entry will look like.  To do
this just set the variable "spec_chglog_format" in your vimrc file like
this: >

parse error: options.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/options.html

Context:

	to check what is following.  E.g. for Javascript, if a function is
	defined with "func_name = function(args)": >
		^\s*\ze\i\+\s*=\s*function(
<	If the function is defined with "func_name : function() {...": >
	        ^\s*\ze\i\+\s*[:]\s*(*function\s*(
<	When using the ":set" command, you need to double the backslashes!
	To avoid that use `:let` with a single quote string: >

The 3rd code block is not rendered correctly as code block.

URL token

Seems like the parser currently doesn't recognize URLs in text. The old makehtml.awk script had primitive support for that, and it's useful for rendering URLs as hyperlinks.

Example text: https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/api.txt#L20-L31

==============================================================================
API Usage                                               *api-rpc* *RPC* *rpc*

                                                        *msgpack-rpc*
RPC is the typical way to control Nvim programmatically.  Nvim implements the
MessagePack-RPC protocol:
  https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
  https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md

Many clients use the API: user interfaces (GUIs), remote plugins, scripts like
"nvr" (https://github.com/mhinz/neovim-remote).  Even Nvim itself can control
other Nvim instances.  API clients can:

  - Call any API function
  - Listen for events
  - Receive remote calls from Nvim

query.lua:219: query: invalid node type at position 14 for language help

No matter which help file I open, be it an empty file or the help file for treesitter, I'm always getting this error on load and repeatedly as I move around the file.

Error detected while processing FileType Autocommands for "*":
Error executing lua callback: ...im/0.8.1/share/nvim/runtime/lua/vim/treesitter/query.lua:219: query: invalid node type at position 14 for language help
stack traceback:
        [C]: in function '_ts_parse_query'
        ...im/0.8.1/share/nvim/runtime/lua/vim/treesitter/query.lua:219: in function 'get_query'
        ...1/share/nvim/runtime/lua/vim/treesitter/languagetree.lua:52: in function 'new'
        ...r/neovim/0.8.1/share/nvim/runtime/lua/vim/treesitter.lua:45: in function '_create_parser'
        ...r/neovim/0.8.1/share/nvim/runtime/lua/vim/treesitter.lua:96: in function 'get_parser'
        .../start/nvim-treesitter/lua/nvim-treesitter/highlight.lua:34: in function 'start'
        .../start/nvim-treesitter/lua/nvim-treesitter/highlight.lua:42: in function 'attach'
        ...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:505: in function 'attach_module'
        ...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:528: in function 'reattach_module'
        ...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:131: in function <...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:130>

I've removed all of my plugins and configuration and I'm still getting this issue whenever the filetype is help.

parse error: motion.txt single quote followed by register type

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/motion.html

Context:

|shada-file-marks|.

Numbered marks '0 to '9 are quite different.  They can not be set directly.
They are only present when using a shada file |shada-file|.  Basically '0
is the location of the cursor when you last exited Vim, '1 the last but one
time, etc.  Use the "r" flag in 'shada' to specify files for which no
Numbered mark should be stored.  See |shada-file-marks|.

Same as #45?

Inline code highlighting has issues

There are multiple issues of recognizing inline code blocks. It usually works fine when backticks surround single word consisting from letters. Other than that, not so much.

Text:

`inline code with s`
`inline code with spaces`
`require('')`
`require('mod')`
`{}`

Current behavior with tree-sitter:

Current behavior without tree-sitter:

parser gets confused by single bar (|)

Example from autocmd.txt:. The '|' seems to confuse the parser, and changing it to "|" doesn't help. Likely | should be treated as a literal if any whitespace follows it before the closing |.

The special pattern <buffer> or <buffer=N> defines a buffer-local autocommand.
See |autocmd-buflocal|.

Note: The ":autocmd" command can only be followed by another command when the
'|' appears where the pattern is expected.  This works: >
	:augroup mine | au! BufRead | augroup END
But this sees "augroup" as part of the defined command: >
	:augroup mine | au! BufRead * | augroup END
	:augroup mine | au BufRead * set tw=70 | augroup END
Instead you can put the group name into the command: >
	:au! mine BufRead *
	:au mine BufRead * set tw=70
Or use `:execute`: >
	:augroup mine | exe "au! BufRead *" | augroup END
	:augroup mine | exe "au BufRead * set tw=70" | augroup END

$XDG_CONFIG_HOME/nvim/init.vim

parse error: pi_spec.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pi_spec.html

Context:

done so.

Setting a map					*spec-setting-a-map*
-------------

As you should know, you can easily set a map to access any Vim command (or
anything, for that matter).  If you don't like the default map of
<LocalLeader>c, you may just set up your own key.  The following line

parse error: windows.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/windows.html

Context:

		  a	an active buffer: it is loaded and visible
		  h	a hidden buffer: It is loaded, but currently not
			   displayed in a window |hidden-buffer|
		   -	a buffer with 'modifiable' off
		   =	a readonly buffer
		   R	a terminal buffer with a running job
		   F	a terminal buffer with a finished job

Hot-links are not recognized inside parenthesis

Text:

(|aaa|) |bbb| ccc |ddd|

Current behavior with tree-sitter:

Behavior without tree-sitter:

more syntax features: keycodes, special, parameters

In testing this parser for nvim-treesitter (nvim-treesitter/nvim-treesitter#2679), I noticed that some features are not yet supported, especially

• Parameters as in {pattern}
• Keycodes and chords as in <c-o> and CTRL-O
• Command modifiers as in [range] (that's just a bunch of literals, and so could simply be listed as anonymous nodes in the highlight query)
• optional arguments #109
• modelines? #125

(See https://github.com/nanotee/vimdoc-notes for the most comprehensive specification of vimdoc I've seen so far.)

parse error: two lines merged as one line?

The rendered html here is not right. The group as an atom and nothing: separates alternatives line are merged as a single long line.

CI: run gen_help_html.lua in pull requests

gen_help_html.lua in the Nvim source is used to generate HTML from vimdoc :help files:

https://github.com/neovim/neovim/blob/6afe5cce7964258f7ee0b25ae08faa551bc76090/scripts/gen_help_html.lua#L766

Currently the full gen() step in gen_help_html.lua is not fully run in Nvim CI because it takes 30-60s. However, 30-60s in the CI for tree-sitter-vimdoc could be acceptable. And it would add a lot of coverage from an actual consumer.

However, it would cause CI here to fail if we rename nodes. That would require a corresponding PR to update gen_help_html.lua.

nested list items

(Continuation of #12 )

Problem

#16 added parsing for list items as (line_li), but these don't "nest".

Example of a nested list

api.txt has many nested lists.

https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/api.txt#L3257-L3273

   • callback (function|string) optional: if a string, the name
     of a Vimscript function to call when this autocommand is
     triggered. Otherwise, a Lua function which is called when
     this autocommand is triggered. Cannot be used with
     {command}. Lua callbacks can return true to delete the
     autocommand; in addition, they accept a single table
     argument with the following keys:
     • id: (number) the autocommand id
     • event: (string) the name of the event that triggered the
       autocommand |autocmd-events|
     • group: (number|nil) the autocommand group id, if it
       exists
     • match: (string) the expanded value of |<amatch>|
     • buf: (number) the expanded value of |<abuf>|
     • file: (string) the expanded value of |<afile>|
     • data: (any) arbitrary data passed to
       |nvim_exec_autocmds()|

parse error: syntax.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/syntax.html

Context:

	a number instead of a color name.

	Note that for 16 color ansi style terminals (including xterms), the
	numbers in the NR-8 column is used.  Here '*' means 'add 8' so that 
	Blue is 12, DarkGray is 8 etc.

	Note that for some color terminals these names may result in the wrong

single apostrophe (') incorrectly parsed as option link

Sample text (from api.txt):

Example:

Let's set an extmark at the first row (row=0) and third column (column=2).
|api-indexing| Passing id=0 creates a new mark and returns the id: >

      01 2345678
    0 ex|ample..
<        ^ extmark position
>
    let g:mark_ns = nvim_create_namespace('myplugin')
    let g:mark_id = nvim_buf_set_extmark(0, g:mark_ns, 0, 2, {})
<
We can get the mark by its id: >

The ' char in Let's confuses the parser, and lots of errors follow.

issue with angle-brackets in change.txt

reproduce: :help <
How it's actually displayed:

SHIFTING LINES LEFT OR RIGHT                            shift-left-right

                                                        <
{motion}                Shift {motion} lines one 'shiftwidth' leftwards.

                        If the 'shiftwidth' option is set to zero, the amount
                        of indent is calculated at the first non-blank
                        character in the line.
                                                        <<
<                       Shift [count] lines one 'shiftwidth' leftwards.

                                                        v_<
{Visual}[count]<        Shift the highlighted lines [count] 'shiftwidth'

I hope this is the right place to report this...

tilde "~" headings are not recognized

The Attributes: ~ subheading in the sample text below is parsed as a normal "line"

line [9, 4] - [10, 0]
  word [9, 4] - [9, 15]
  word [9, 16] - [9, 17]

sample text:

nvim_ui_try_resize({width}, {height})                   *nvim_ui_try_resize()*
    TODO: Documentation

    Attributes: ~
        |RPC| only

Maybe there should be h1, h2, h3 headings?

• h1: ======== headings
• h2: ---------- headings
• h3: Foo ~ headings

paragraphs (and nested paragraphs?)

Preamble from #12 :

In order for :help to render as something other than fixed-width, hard-wrapped text, (e.g. for HTML format) the parser needs to give some basic hints about structure besides "line". Sometimes (often) the :help source needs to be fixed (because it can't be reasonably parsed), but in other common cases it's probably reasonable for tree-sitter-vimdoc to provide some basic idea of:

1. paragraph (and "nested" paragraphs? AKA indentation) [this issue]
2. list items (which may be nested) #12

Example of paragraphs and nested (indented) paragraphs

https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/eval.txt#L51-L74

By "nested paragraphs" I simply mean blocks of text that have common indentation. In the following example,

• we can't parse the Number 123... block but if we can at least recognize that it is indented, then the HTML result can wrap that block in <pre> so that it is readable.
• the *octal* line doesn't share indentation so it should be parsed as a normal "line", possibly part of the next paragraph.
• it may be reasonable for us to require blank lines between these blocks.

The Number and String types are converted automatically, depending on how they
are used.

Conversion from a Number to a String is by making the ASCII representation of
the Number.  Examples:
	Number 123	-->	String "123" ~
	Number 0	-->	String "0" ~
	Number -1	-->	String "-1" ~
							*octal*
Conversion from a String to a Number is done by converting the first digits to
a number.  Hexadecimal "0xf9", Octal "017" or "0o17", and Binary "0b10"
numbers are recognized.  If the String doesn't start with digits, the result
is zero. Examples:
	String "456"	-->	Number 456 ~
	String "6bar"	-->	Number 6 ~
	String "foo"	-->	Number 0 ~
	String "0xf1"	-->	Number 241 ~
	String "0100"	-->	Number 64 ~
	String "0o100"	-->	Number 64 ~
	String "0b101"	-->	Number 5 ~
	String "-8"	-->	Number -8 ~
	String "+8"	-->	Number 0 ~

To force conversion from String to Number, add zero to it: >
	:echo "0100" + 0
<	64 ~

To avoid a leading zero to cause octal conversion, or for using a different
base, use |str2nr()|.

parse error: pi_spec.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pi_spec.html

Context:

it will insert new items before the existing ones.

Inserting release info				*spec_chglog_release_info*
----------------------

If you want, the plugin may automatically insert release information
on each changelog entry.  One advantage of turning this feature on is
that it may control if the release has been updated after the last

nested blocks

(Continuation of #13 . See also #21 )

Problem

#16 added (block), but these don't "nest".

Preamble from #12 :

In order for :help to render as something other than fixed-width, hard-wrapped text, (e.g. for HTML format) the parser needs to give some basic hints about structure besides "line". Sometimes (often) the :help source needs to be fixed (because it can't be reasonably parsed), but in other common cases it's probably reasonable for tree-sitter-vimdoc to provide some basic idea of:

Example of nested (indented) blocks

https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/eval.txt#L51-L74

"Nested blocks" simply means blocks of text that have common indentation. In the following example,

• we can't parse the Number 123... block but if we can at least recognize that it is indented, then the HTML result can wrap that block in <pre> so that it is readable.
• the *octal* line doesn't share indentation so it should be parsed as (line) (or possibly introduce a new (tags_heading) node?).
• it may be reasonable for us to require blank lines between these blocks.

The Number and String types are converted automatically, depending on how they
are used.

Conversion from a Number to a String is by making the ASCII representation of
the Number.  Examples:
	Number 123	-->	String "123" ~
	Number 0	-->	String "0" ~
	Number -1	-->	String "-1" ~
							*octal*
Conversion from a String to a Number is done by converting the first digits to
a number.  Hexadecimal "0xf9", Octal "017" or "0o17", and Binary "0b10"
numbers are recognized.  If the String doesn't start with digits, the result
is zero. Examples:
	String "456"	-->	Number 456 ~
	String "6bar"	-->	Number 6 ~
	String "foo"	-->	Number 0 ~
	String "0xf1"	-->	Number 241 ~
	String "0100"	-->	Number 64 ~
	String "0o100"	-->	Number 64 ~
	String "0b101"	-->	Number 5 ~
	String "-8"	-->	Number -8 ~
	String "+8"	-->	Number 0 ~

To force conversion from String to Number, add zero to it: >
	:echo "0100" + 0
<	64 ~

To avoid a leading zero to cause octal conversion, or for using a different
base, use |str2nr()|.

parse error: pattern.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pattern.html#%2Fmagic

Context:

	  ()	   \(\)     \(\)     \(\)	group as an atom
	  |	   \|	    \|	     \|		nothing: separates alternatives
	  \\	   \\	    \\	     \\		literal backslash
	  \{	   {	    {	     {		literal curly brace

{only Vim supports \m, \M, \v and \V}

The line literal curly brace has an alignment issue w.r.t the previous line.

misalignment of boxed text

In this page, text inside the character boxes are all misaligned, either out of the box or sticking out of the enclosing box.

list items (and nested list items)

In order for :help to render as something other than fixed-width, hard-wrapped text, (e.g. for HTML format) the parser needs to give some basic hints about structure besides "line". Sometimes (often) the :help source needs to be fixed (because it can't be reasonably parsed), but in other common cases it's probably reasonable for tree-sitter-vimdoc to provide some basic idea of:

1. paragraph (and "nested" paragraphs? AKA indentation) #13
2. list items (which may be nested) [this issue]

Example of a basic list

https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/api.txt#L25-L34

Many clients use the API: user interfaces (GUIs), remote plugins, scripts like
"nvr" (https://github.com/mhinz/neovim-remote).  Even Nvim itself can control
other Nvim instances.  API clients can:

  - Call any API function
  - Listen for events
  - Receive remote calls from Nvim

The RPC API is like a more powerful version of Vim's "clientserver" feature.

Example of a nested list

api.txt has many nested lists. Maybe it's not worth supporting these?

https://github.com/neovim/neovim/blob/52e4b779e3df1335efc4774e8dfc199d07e24bfd/runtime/doc/api.txt#L3257-L3273

                 • callback (function|string) optional: if a string, the name
                   of a Vimscript function to call when this autocommand is
                   triggered. Otherwise, a Lua function which is called when
                   this autocommand is triggered. Cannot be used with
                   {command}. Lua callbacks can return true to delete the
                   autocommand; in addition, they accept a single table
                   argument with the following keys:
                   • id: (number) the autocommand id
                   • event: (string) the name of the event that triggered the
                     autocommand |autocmd-events|
                   • group: (number|nil) the autocommand group id, if it
                     exists
                   • match: (string) the expanded value of |<amatch>|
                   • buf: (number) the expanded value of |<abuf>|
                   • file: (string) the expanded value of |<afile>|
                   • data: (any) arbitrary data passed to
                     |nvim_exec_autocmds()|

parse error: motion.txt, invalid option: 'a - 'z

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/motion.html

Context:

remembered.  Do not confuse marks with named registers, they are totally
unrelated.

'a - 'z		lowercase marks, valid within one file
'A - 'Z		uppercase marks, also called file marks, valid between files
'0 - '9		numbered marks, set from .shada file

The single quote followed by a digit or character (for those register types) is not handled correctly.

parse error: options.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/options.html

Context:

	      Note that there is no '%' before the closing '}'.  The
	      expression cannot contain a '}' character, call a function to
	      work around that.  See |stl-%{| below.
	{% -  This is almost same as { except the result of the expression is
	      re-evaluated as a statusline format string.  Thus if the
	      return value of expr contains % items they will get expanded.
	      The expression can contain the } character, the end of

Conceal eats adjacent whitespace

With the new grammar and queries, open any help file (say, :h api.txt) and look at option or tag links: the concealed markers will include all adjacent whitespace. E.g.,

API Usage                                               *api-rpc* *RPC* *rpc*

becomes

API Usageapi-rpcRPCrpc

instead of

API Usage                                               api-rpc RPC rpc

(This was not the case with the old parser version.)

Tags with a dot "." beforehand isn't handled correctly

The following doesn't conceal "|": vim.api.|nvim_get_keymap()|

Some whole paragraphs colored

When having TS enabled with highlight, often there are whole paragraphs highlighted, for me (gruvbox-baby) in yellow. Example from usr_03.txt below:

With TS:

Without TS:

Allow options as tags

Currently, the parser only allows a simple word as tag, as in *foo*; option like *'sw'* and argument like {subject} (possibly the only instance) are a parsing error.

parse error: pi_spec.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pi_spec.html

Context:

2. Customizing					*spec-customizing*

The format string				*spec_chglog_format*
-----------------

You can easily customize how your spec file entry will look like.  To do
this just set the variable "spec_chglog_format" in your vimrc file like
this: >

parse error: autocmd.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/autocmd.html

Context:

	~	matches a '~'
	,	separates patterns
	\,	matches a ','
	{ }	like \( \) in a |pattern|
	,	inside { }: like \| in a |pattern|
	\}	literal }
	\{	literal {

multiple keycodes must be separated by whitespace

Found in https://neovim.io/doc/user/options.html#'wildmenu'

CTRL-N isn't highlighted, but it should be.

Context:

Keys that show the previous/next match, such as <Tab> or
CTRL-P/CTRL-N, cause the highlight to move to the appropriate match.
['wildmode'](https://neovim.io/doc/user/options.html#'wildmode') must specify "full": "longest" and "list" do not start
['wildmenu'](https://neovim.io/doc/user/options.html#'wildmenu') mode. You can check the current mode with [wildmenumode()](https://neovim.io/doc/user/builtin.html#wildmenumode()).

parse error, single star followed by a number

In https://neovim.io/doc/user/intro.html#mode-switching, the mode-switching table is messed up due to several *1 like item in the table.

Probably should not be parsed as list item, but should be footnote like.

single-char 'x' is not an option hotlink

Noticed this while working on gen_help_html.lua neovim/neovim#11967 .

This is just for reference; as mentioned in #11 (comment) I'm pretty deep into rewriting tree-sitter-vimdoc. Results are promising so far, but will take time.

Problem

The parser currently thinks that 'x' is a link to an option, but that is not correct. single-char surrounded by single-quotes should be treated as a plain word.

  ["'!'"] = true,
  ["'#'"] = true,
  ["'$'"] = true,
  ["'%'"] = true,
  ["'&'"] = true,
  ["'('"] = true,
  ["')'"] = true,
  ["'+'"] = true,
  ["','"] = true,
  ["','sqlKeyword'"] = true,
  ["','sqlOption'"] = true,
  ["','sqlStatement'"] = true,
  ["'-'"] = true,
  ["'-'Z'"] = true,
  ["'.'"] = true,
  ["'/'"] = true,
  ["'0'"] = true,
  ["'1'"] = true,
  ["'2'"] = true,
  ["'4'"] = true,
  ["'8'"] = true,
  ["'9'"] = true,
  ["':'"] = true,
  ["';'"] = true,
  ["'<'"] = true,
  ["'='"] = true,
  ["'>'"] = true,
  ["'?'"] = true,
  ["'@'"] = true,
  ["'A'"] = true,
  ["'B'"] = true,
  ["'C'"] = true,
  ["'D'"] = true,
  ["'E'"] = true,
  ["'F'"] = true,
  ["'G'"] = true,
  ["'H'"] = true,
  ["'I'"] = true,
  ["'J'"] = true,
  ["'K'"] = true,
  ["'L'"] = true,
  ["'M'"] = true,
  ["'N'"] = true,
  ["'O'"] = true,
  ["'P'"] = true,
  ["'Q'"] = true,
  ["'R'"] = true,
  ["'S'"] = true,
  ["'T'"] = true,
  ["'W'"] = true,
  ["'X'"] = true,
  ["'Y'"] = true,
  ["'Z'"] = true,
  ["'['"] = true,
  ["']'"] = true,
  ["'^'"] = true,
  ["'_'"] = true,
  ["'a'"] = true,
  ["'b'"] = true,
  ["'c'"] = true,
  ["'d'"] = true,
  ["'e'"] = true,
  ["'f'"] = true,
  ["'g'"] = true,
  ["'h'"] = true,
  ["'i'"] = true,
  ["'j'"] = true,
  ["'k'"] = true,
  ["'l'"] = true,
  ["'m'"] = true,
  ["'n'"] = true,
  ["'o'"] = true,
  ["'p'"] = true,
  ["'q'"] = true,
  ["'r'"] = true,
  ["'s'"] = true,
  ["'t'"] = true,
  ["'u'"] = true,
  ["'v'"] = true,
  ["'w'"] = true,
  ["'x'"] = true,
  ["'y'"] = true,
  ["'z'"] = true,
  ["'~'"] = true,
  ["'·'"] = true,
  ["'ö'"] = true,
  ["'─'"] = true,
  ["'│'"] = true,
  ["'├'"] = true,
  ["'┤'"] = true,
  ["'┬'"] = true,
  ["'┴'"] = true,
  ["'┼'"] = true,
  ["'보'"] = true,
  ["'세'"] = true,
  ["'여'"] = true,
  ["'요'"] = true,

parse error: pi_spec.txt

Found tree-sitter-vimdoc parse error at: https://neovim.io/doc/user/pi_spec.html

Context:

function man page.

Where to insert new items			*spec_chglog_prepend*
-------------------------

The plugin will usually insert new %changelog entry items (note that it's
not the entry itself) after the existing ones.  If you set the
spec_chglog_prepend variable >