Random crashes when inserting / deleting text about strainer

[derula]

At least one crash is actually a sievelib bug. The following script crashes the sievelib parser:

if address :domain "" ""){}

Traceback:

Traceback (most recent call last):
  File "~\.virtualenvs\strainer-40YVjBHe\lib\site-packages\sievelib\parser.py", line 400, in parse
    if not self.__command(ttype, tvalue):
  File "~\.virtualenvs\strainer-40YVjBHe\lib\site-packages\sievelib\parser.py", line 322, in __command
    self.__up()
  File "~\.virtualenvs\strainer-40YVjBHe\lib\site-packages\sievelib\parser.py", line 157, in __up
    if self.__curcommand.must_follow is not None:
AttributeError: 'NoneType' object has no attribute 'must_follow'

[derula]

In master, the problem still exists, but traceback is a bit different:

Traceback (most recent call last):
  File "~/sievelib/sievelib/parser.py", line 435, in parse
    if not self.__command(ttype, tvalue):
  File "~/sievelib/sievelib/parser.py", line 375, in __command
    if self.__cstate(ttype, tvalue):
  File "~/sievelib/sievelib/parser.py", line 328, in __arguments
    if self.__argument(ttype, tvalue):
  File "~/sievelib/sievelib/parser.py", line 285, in __argument
    self.__curcommand.non_deterministic_args
AttributeError: 'NoneType' object has no attribute 'non_deterministic_args'

Interestingly, if address :domain "" ""){ crashes in master (with the same error), but does not crash in 1.1.1. if address :domain "" "")) crashes with the same error as in 1.1.1. Both versions do not crash with either if address :domain "" ""), if address :domain ""), or if address :domain "" "" "").

[derula]

Even simpler: } crashes as well. I think the problem is that the parser doesn't properly keep track of parenthesis / bracket depth. It just encounters a close bracket and assumes that it was properly opened. To solve this, we could introduce a stack of opened brackets, and when a close bracket mismatches, it's an instant parse error.

However, this would also mean that during partial lexing, I will have to backtrack to when the bracket was opened...

One solution would be to always use the entire file in partial lexing, but to only change styling between start and end (and wherever the first token starts / last token ends). I will create an issue for this.

My bad, partial lexing doesn't use the parser, only the lexer. Doesn't apply here.

[derula]

Reopened to implement a workaround since sievelib bug is still open.