d-dantte / axis.pulsar Goto Github PK

Include an optional trace-writer for the recognizers so printing out the trace can be used as a debug tool.

See here for guidance on how to implement syntax highlighting for visual studio file editor.

By default, when not specified, a NonTemrinal should always result in a FailedRecognitionError when its group element returns a FailedRecognitionError. This means for sequences, no matter how many elements were successfully recognized before the failure, the NonTerminal will never return a PartialRecognitionError.

Tests revealed that NonTerminal entities default to a recognition threshold of 1. This needs to be changed.

The ability to import a grammar into another grammar, and use on-demand namespaces to demarcate grammars, is a nice to have feature.

Implement event notifications for production recognition, as well as production discards.

Parsing atomic content arguments has a bug where escaping the ' character with \' yields the whole escape sequence - in other words, the code failed to strip the \.

Add these methods to the BufferedTokenReader:

    public bool TryNextTokens(string tokens, bool resetIfCompareFails = true) => TryNextTokens(resetIfCompareFails, tokens.ToArray());

    public bool TryNextTokens(bool resetIfCompareFails, params char[] tokensToReadAndCompare)
    {
        if (tokensToReadAndCompare == null || tokensToReadAndCompare.Length == 0)
            throw new ArgumentException("Invalid argument");

        else
        {
            var position = Position;
            if(TryNextTokens(tokensToReadAndCompare.Length, out var read))
            {
                if (tokensToReadAndCompare.SequenceEqual(read))
                    return true;

                if (resetIfCompareFails)
                    Reset(position);
            }

            return false;
        }
    }

Closed match type has 2 issues:

1. It doesn't consider situations where the MaxMatch is greater than the number of tokens left in the reader.
2. It doesn't back-track the reader after discounting failed matches

Recognition Threshold should consider ONLY non-optional ICSTNode instances - this invariably means that ICSTNode implementations will need an extra property to tell if they are optional or required.

This feature is necessary because setting a threshold yields inconsistent results when optional symbols are present in the production symbol list.

XBNF has a bug where delimited content appearing before a sequence (within another group) is not recognized correctly because
the the content concatenation operator clashes with the start-delimiter for sequences.