Leading backslash causes wrong node.pos.i about vim-vimlparser

[tyru]

sorry, I translated Japanese to English (I forgot about this repository's rule)

[mattn]

StringReader.buf should have buffer which have same length of the source. So leading \ should be contained in AST. However, we doesn't have the token for the \. ex: s:TOKEN_JOINBACKSLASH ? i.e. this will be breaking compatibility.
I have an idea to avoid breaking compatibility. But workaround.

diff --git a/autoload/vimlparser.vim b/autoload/vimlparser.vim
index 431f340..1dc02a9 100644
--- a/autoload/vimlparser.vim
+++ b/autoload/vimlparser.vim
@@ -234,6 +234,10 @@ function! s:iswordc1(c)
   return a:c =~# '^[A-Za-z_]$'
 endfunction
 
+function! s:iswhite(c)
+  return a:c =~# '^[ \t]$' || a:c == ''
+endfunction
+
 function! s:iswhite(c)
   return a:c =~# '^[ \t]$'
 endfunction
@@ -1071,6 +1075,8 @@ function! s:VimLParser.parse_trail()
   elseif c ==# '"'
     call self.parse_comment()
     call self.reader.get()
+  elseif c ==# ''
+    call self.reader.get()
   else
     throw s:Err(printf('E488: Trailing characters: %s', c), self.reader.getpos())
   endif
@@ -3780,10 +3786,11 @@ function! s:StringReader.__init__(lines)
           if c == '\'
             let skip = s:FALSE
           endif
+          call add(self.buf, '')
         else
           call add(self.buf, c)
-          call add(self.pos, [lnum + 2, col + 1])
         endif
+        call add(self.pos, [lnum + 2, col + 1])
         let col += len(c)
       endfor
       let lnum += 1

This add leading \ into AST as empty character, and skip them while traversing. How about this?

[mattn]

vim-vimparser parse characters in source. So the offset should be unicode-based index. So you must get the code pointed from pos and endpos like below.

return join(map(split(join(a:src, "\n"), '\zs'), 'v:val[pos.i : endpos.i]'), '')