Explanation of “ppx_sedlex and ppx_import are incompatible with jbuilder?” about nice-parser

[smolkaj]

Ideally, the jbuild-file should look like this

(jbuild_version 1)

;; generate menhir tokens
(rule
 ((targets (Tokens.ml))
  (deps    (Menhir_parser.mly))
  (action  (run ${bin:menhir} --only-tokens ${<} --base Tokens))))

;; generate parser
(menhir
 ((modules (Menhir_parser))
  (flags (--external-tokens Lexer))))

(library
 ((name        ocaml_parsing)
  (public_name ocaml_parsing)
  (wrapped true)
  (libraries 
    (core
     sedlex
     compiler-libs.common
     menhirLib)
  )
  (virtual_deps (cppo menhir))
  (preprocess (pps 
    (ppx_deriving.std
     ppx_enumerate
     ppx_sedlex  ;; include sedlex here 
     ppx_import  ;; use ppx_import instead of cppo
    )
  ))
 )
)

and Lexer.cppo.sedlex.ml should become simply Lexer.ml and start like this:

type token = [%import: Tokens.token] [@@deriving show, enumerate]
...

Unfortunately, both ppx_sedlex and ppx_import are currently not directly compatible with jbuilder. As a workaround, I invoke ppx_sedlex manually and I use cppo, again by invoking it manually, instead of ppx_import.

While ppx_sedlex will become compatible with jbuilder soon (ocaml-community/sedlex#57), ppx_import will likely never become compatible. The right thing to do here is probably to ask Francois Pottier (the menhir author) to provide a Tokens.to_string function, so we don't have to derive it through this hack

[ELLIOTTCABLE]

This is a really great write-up and response! I'm happy I understand the constraints, now.

As I'm sure you're more experienced than me — can you think of an alternative workaround, that allows Merlin to function? I was thinking of trying to use Jbuilder's “promotion” features, maybe? I suspect we could come up with a structure wherein the Tokens module is actually inlined into the file while editing it, and then automatically updated/replaced …

[smolkaj]

Sorry, but I don't use Merlin myself, so I am not sure what to do. I suppose it's best to wait for a new release of Menhir that allows us to get rid of this hack ;)