smolkaj / nice-parser

Nice parsers in OCaml without the boilerplate

For an older version of this repository based on sedlex, refer to the sedlex branch.

Nice Parser

Writing an OCaml parser with nice error messages should be easy - and now it is! Nice Parser comes in two parts:

1. lib/: The nice_parser library (API) consolidates boilerplate code and wraps your auto-generated parser in a nice interface with beautiful error messages. In types: Nice_parser.Make : functor(P : RAW_PARSER) -> NICE_PARSER.
2. example/: The example parser lets you get started on your own parser in seconds. It is based on nice_parser and standard tools:
   • Menhir, a LR(1) parser generator.
   • ocamllex, OCaml's built-in lexer generator.
   • Jane Street's dune and base, the de-facto standard built tool and standard library for OCaml.

Using the library and the skeleton, you can get started on your own parser in seconds:

git clone https://github.com/smolkaj/nice-parser.git  # clone this repository
cd nice-parser && rm -r lib && mv example src         # use example as starting point
opam install . --deps-only --with-test                # install dependencies
dune build                                            # try to build...
dune exec src/bin/main.exe                            # ...and run your parser!

You should see the following output (the error message relies on OCaml >= 4.08's new source highlighting mechanism):

Trying to parse "(a b (c d) e)".
-> (List ((Atom a) (Atom b) (List ((Atom c) (Atom d))) (Atom e)))

Trying to parse "(long_atom_with_0123)".
-> (List ((Atom long_atom_with_0123)))

Trying to parse "
    ( so far so good
          but (this is)) illegal (isnt it?)
    (* parsing will fail ^^^^^^^ here *)
  ".
Fatal error: exception Line 3, characters 25-32:
3 |           but (this is)) illegal (isnt it?)
                             ^^^^^^^
Error: [parser] unexpected token

Documentation

The API is documented here. The example skeleton should be self-explanatory.

How to build

Ideally, use OCaml 4.08 or higher (for beautiful error messages). All required dependencies can be installed using the opam package manager. The project can be built using dune.

opam install . --deps-only --with-test
dune build
dune runtest

Suggestions and Improvements

Suggestions and changes are welcome. Please submit pull requests, or open issues.

nice-parser's Issues

Please provide ability to compile to js_of_ocaml if possible

Is it possible co compile to js_of_ocaml with this setup?
I know Core is incompatible with jsoo, but I tried just replacing it with Core_kernel and it seems to compile without problems (to native). Though I can't figure out how to compile to jsoo.

Here is my fork with Core_kernel: https://github.com/loxs/ocaml-parsing

Error: Uninterpreted extension 'sedlex.regexp'

Hi,

i get Error: Uninterpreted extension 'sedlex.regexp' as seen below. menhir is version 20180703.

Any idea how to fix this?

Thanks,
Lacramioara

====

isar:07:06:16:~/ocaml-parsing$ make
time -p jbuilder build @install
ocamlopt src/.ocaml_parsing.objs/ocaml_parsing__Lexer.{cmx,o} (exit 2)
(cd _build/default && /home/eugen/.opam/4.06.0/bin/ocamlopt.opt -w -40 -g -I src/.ocaml_parsing.objs -I /home/eugen/.opam/4.06.0/lib/base -I /home/eugen/.opam/4.06.0/lib/base/caml -I /home/eugen/.opam/4.06.0/lib/base/shadow_stdlib -I /home/eugen/.opam/4.06.0/lib/bytes -I /home/eugen/.opam/4.06.0/lib/gen -I /home/eugen/.opam/4.06.0/lib/menhirLib -I /home/eugen/.opam/4.06.0/lib/ocaml/compiler-libs -I /home/eugen/.opam/4.06.0/lib/ppx_deriving -I /home/eugen/.opam/4.06.0/lib/result -I /home/eugen/.opam/4.06.0/lib/sedlex -I /home/eugen/.opam/4.06.0/lib/sexplib0 -I /home/eugen/.opam/4.06.0/lib/stdio -intf-suffix .ml -no-alias-deps -open Ocaml_parsing -o src/.ocaml_parsing.objs/ocaml_parsing__Lexer.cmx -c -impl src/Lexer.pp.ml)
File "src/Lexer.ml", line 42, characters 15-28:
Error: Uninterpreted extension 'sedlex.regexp'.

Explanation of “ppx_sedlex and ppx_import are incompatible with jbuilder?”

So this section of the jbuild seems to imply that it's temporary:

;; generate lexer -- temporary hack while ppx_sedlex and ppx_import are incompatible with jbuilder!!
(rule
 ((targets (Lexer.sedlex.ml))
  (deps    (Lexer.cppo.sedlex.ml Tokens.ml))
  (action  (run ${bin:cppo} ${<} -n -o ${@}))))
(rule
 ((targets (Lexer.mli))
  (deps    (Lexer.cppo.mli Tokens.ml))
  (action  (run ${bin:cppo} ${<} -n -o ${@}))))
(rule
 ((targets (Lexer.ml))
  (deps    (Lexer.sedlex.ml))
  (action  (run ${ocaml_where}/../sedlex/ppx_sedlex ${<} -o ${@}))))

What part of that is the "temporary hack"; and how, specifically, is dune/jbuilder incompatible with Sedlex? Thanks!

No package named nice_parser

How do I get past this

Why the usage of `cppo`?

(Related: #2; is this the described “hack?”)

So, I'm relatively new to OCaml (hence using this project to try and understand parsing in OCaml! 🤣); but I don't understand why CPP-style source-transformation is necessary to reach Menhir's generated “token definitions”:

#include "Tokens.ml"
  [@@deriving show, enumerate]

module Sedlexing = LexBuffer
open LexBuffer
(* ... so on, so forth ... *)

Can't we use some sort of built-in language mechanic to include the symbols from the generated Tokens module into both Lexer.mli and the (eventual) Lexer.ml?

This has been a show-stopper for me; as a newbie, I still really depend on Merlin to understand what's going on and write code — and the CPPO invocation at the top of these files completely breaks Merlin, right now. But if there's no other way to do this … )'=

(It looks like somebody's trying to build support for it, ocaml/merlin#548, but that may be a long way off? I'm kinda an outsider, so I'm not sure. See also: let-def/merlin-extend#7.)

Having some trouble building...

Howdy! I'm having some trouble building. Perhaps it is just a question of not having the right packages installed via OPAM.

First of all, the build begins with this:

oasis setup-clean
oasis setup
W: Cannot find source file matching module 'My_Tokens' in library parser.
W: Use InterfacePatterns or ImplementationPatterns to define this file with feature "source_patterns".

Second, at the end, I get this, which seems to be the bad part:

• /Users/perry/.opam/4.05.0/bin/ocamlfind ocamldep -package sedlex -package ppx_import -package core -package compiler-libs.common -package MenhirLib -package ppx_deriving,ppx_deriving.eq,ppx_deriving.enum,ppx_deriving.show,ppx_jane -modules src/My_Lexer.mli > src/My_Lexer.mli.depends
  File "src/My_Lexer.mli", line 1, characters 58-67:
  Error: Cannot locate deriver enumerate
  Command exited with code 2.
  Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
  E: Failure("Command ''/Users/perry/.opam/4.05.0/bin/ocamlbuild' src/parser.cma test.byte -use-ocamlfind -use-menhir -cflags -w,@5+8@11@12-40,-short-paths -package ppx_deriving,ppx_deriving.eq,ppx_deriving.enum,ppx_deriving.show,ppx_jane -tag debug -tag tests' terminated with error code 10")
  make[1]: *** [all] Error 1
  make: *** [default] Error 2

Core still listed as a dependency in test/jbuild

Recommend changing (libraries (core ocaml_parsing)) to (libraries (base stdio ocaml_parsing)).

How to use with dune

Hello, I found this project very helpful as I tried to figure out how to use sedlex.

I think people might benefit from seeing an example in dune, especially since RWO is using it now.