diasbruno / cl-bnf

A simple BNF parser.

cl-bnf

A simple BNF.

(:char . #\a) matches the given char.

(:string . "abc") matches the given string.

(:and exp ...) all expressions must match.

(:or exp ...) returns the first expression that matches.

(:* . exp) collect all strings matches the expression, if any.

(:? . exp) optional match.

#'some-function execute the function with the current char. must return boolean.

and can also be written in a form of sequence rule-c := rule-a rule-b.

or can also be written using :/, for example: rule-c := rule-a :/ rule-b.

rules & grammars

You can define a single rule using define-rule

(cl-bnf:define-rule word (:* . #'alpha-char-p) :call #'stringify)

...or using the define-grammar

(cl-bnf:define-grammar (abc . parser)
   abc := #\a #\b #\c
   cba := "cba"
   abc-cba := abc :/ cba
   parser := abc-cba)

(abc "abc") ;; (#\a #\b #\c)
(abc "cba") ;; "cba"

transformations

• :call apply a function to the results using funcall.
• :apply apply a function to the results using apply.
• :tag return a cons like (cons TAG RESULTS).
• If none where specified, it return all matches.

License

This project is released under the MIT License.

See license.

cl-bnf's Issues

Use stream interface.

Documentation.

Improve documentation for the project.

Macro in the keyword package

The macro := is named by the symbol named = in the keyword package. The exported symbol cl-bnf:|:=| has nothing to do with it.

As soon as the system is loaded, the macro is available from the keyword package:

CL-USER> (ql:quickload "cl-bnf")
To load "cl-bnf":
  Load 1 ASDF system:
    asdf
  Install 1 Quicklisp release:
    cl-bnf
; Fetching #<URL "http://beta.quicklisp.org/archive/cl-bnf/2018-07-11/cl-bnf-20180711-git.tgz">
; 4.94KB
==================================================
5,054 bytes in 0.00 seconds (0.00KB/sec)
; Loading "cl-bnf"
[package cl-bnf].
("cl-bnf")
CL-USER> (:= foo (:char #\a))
FOO

CL-USER> (describe 'cl-bnf:|:=|)
CL-BNF:|:=|
  [symbol]
; No value
CL-USER> (describe :=)
:=
  [symbol]

= names a constant variable:
  Value: :=

= names a macro:
  Lambda-list: (LABEL RULE &KEY CALL TAG APPLY)
  Documentation:
    Generate a function LABEL to parse RULE. Later,
    you can apply a TRANSFORMATION which can be a function
    or a keytword.
  Source file: /home/svante/quicklisp/dists/quicklisp/software/cl-bnf-20180711-git/bnf.lisp
; No value
CL-USER>

I do not think that this was intended, and I feel that it is less than ideal that the (global) keyword package gets filled with library code.

Save and return the stream position in case of non-match

Set up continuous integration.

Self-explanatory.

First implementation.

In this first implementation, the goal is to make it easy to parse and apply transformations on the parsed items.

Combinators:

• (:char #\a) test the next element on the stream with the given char.
• (:one #'fn) test using a function to get the next char in the stream.
• (:string "abc") test if the string is in the stream.
• (:and exp ...) collect many expressions.
• (:or exp ...) collect the expression that matches.
• (:many exp) collect all matches of the expression, if any.
• (:maybe exp) turns the failed test into a valid one.

Declaration

A macro is used to declare all rules for the BNF.

(:= word (:many (:one #'alpha-char-p)) :call #'stringify)

Valid transformations:

• :call apply a function to the results using funcall.
• :apply apply a function to the results using apply.
• :tag return a cons like (cons TAG RESULTS).
• If none where specified, it return a nested list of all matches.

Example

Parsing a valid json number:

(load #P"~/projects/cl-bnf/cl-bnf.lisp")

(:= decimal-number (:many (:one #'numeric-char-p)))
(:= real-number (:or (:and #'decimal-number
                           (:char #\.)
                           #'decimal-number)
                     (:and #'decimal-number
                           (:char #\.))))
(:= signed-part (:or (:char #\+) (:char #\-)))
(:= exp-chars (:or (:char #\e)
                   (:char #\E)))
(:= exp-part (:or (:and #'exp-chars
                        #'signed-part
                        #'decimal-number)
                  (:and #'exp-chars
                        #'decimal-number)))
(:= numeric (:or #'real-number
                 #'decimal-number))
(:= number-literal (:or (:and #'numeric
                              #'exp-part)
                        #'numeric)
    :call (lambda (matches)
            (cons :number (stringify matches))))

(parse #number-literal "1e3")

Find a way to improve the rule declaration.

Maybe there is something to do to simplify the rule declaration.

Borrow ideas from Haskell's Alex...

Maybe we can borrow some ideas from this package and try compile to be more efficient.

What is sequence:position?

In the file src/bnf.lisp it uses sequence:position several times.

Where is the package "SEQUENCE" supposed to come from?

Isn't it just the standard function position?

Move the rule tranformation to the define-rule macro.

https://github.com/diasbruno/cl-bnf/blob/master/src/bnf.lisp#L180 should be moved to the defined-rule macro.

Update documentation and start releasing versions.

define-grammar defines the stuff only at compile file

**define-grammar ** in src/bnf.lisp defines the rules only at compile time, so it doesn't work if you load the compiled file without actually compiling.

For example, (ql:quickload "cl-bnf-examples") so it compile and load everything. Quit the image you are running and start it again. Then (ql:quickload "cl-bnf-examples") again. This loads the compiled files, and errors because none of the rules in the json grammar are defined.

More operators and common patterns.

Other operators we can bring in:

:one-of

(:one-of "abc" #'number-char-p)

(:or (:char #\a) (:char #\b) (:char #\c) #'number-char-p)

:between

(:between (:char #\a) #'number)

(:and (:char #\a) #number (:char #\a))