While toying around with mdx.1.4.0 I stumbled upon the bug fixed by #136. This bug is now fixed but one thing I noticed is that it manifested in a really weird and disturbing way if you used ocaml-mdx rule to generate your mdx test rules.

Because the generated rule uses (diff? %{x} %{x}.corrected) actions, it will silently pass if no .corrected file has been generated.

I'm wondering if it wouldn't be more robust to have ocaml-mdx rule to produce a rule that generates all the .corrected no matter what, using the --force-output option of ocaml-mdx test and to produce strict diff rules thus preventing any such silent failures.

Let me know what you think!

Currently, sh blocks are not using the label 'file' if this one was added.
We should add it's implementation for promotion purposes

I think there are a few issues with how mdx delimits parts:

1. They are not clearly identified as being interpreted by mdx and that can be confusing. [@@@mdx.part ...] would be better
2. They aren't actual delimiters and its not very user friendly. Separate begin and end markers for a given part would be better than having to introduce [@@@part 2] to mark the end of the part 1 even though no part 2 is ever used anywhere.
3. The fact that they are attributes can be slightly confusing as those usually have meaning. One could think they are interpreted by a PPX and expanded into something. Is there any reason why this isn't done using comments instead?

I have a README.md defining a part to sync:

Blabla:

```ocaml file=test.ml,part=1
let f x = x
```

and a test.ml:

[@@@part "1"]

let f x = x

If I add a new part in the README, eg.

Part 2:

```ocaml file=test.ml,part=2
let g f x = f x
```

running mdx with --direction=to-ml won't update the test.ml file.

The following raises an error:

```ocaml
# print_newline ();;
```

$ mdx test a.md
mdx-test: internal error, uncaught exception:
          Invalid_argument("String.sub / Bytes.sub")

I'm a bit confused by all the possible type of ocaml blocks mdx interprets, toplevel, code, code extracted from a file and I'm not sure how it behaves in each cases.

From a user perspective it can be a bit puzzling. I think a first step is to add proper documentation. I'll try to take care of that first part in the next few days but I think it might be worth re-thinking the how these blocks are specified and eventually let the user be explicit about it.

That would probably break mdx's API so we need to think carefully about how to handle all this but I think in the end the mdx's code base would benefit from it as we could get rid of all the "guessing" logic!

Hi,

I use mdx for one of my projects (routes) and I notice that for travis builds running against 4.08 there is the following error:

- (cd _build/default && /home/opam/.opam/4.08/bin/ocaml-mdx test --direction=infer-timestamp README.md)
- ocaml-mdx-test: internal error, uncaught exception:
-                 Failure("incomplete OCaml version")

Link to the failed build: https://api.travis-ci.com/v3/job/221454058/log.txt

When i try to run the tests locally under 4.08 i don't have the same exception. The builds for ocal versions 4.05 through 4.07 also succeed (https://api.travis-ci.com/v3/job/221454057/log.txt -> build log for 4.07)

I'm not sure if this is a problem with mdx or with my test setup and i'd be happy to help debug more.

Thanks!

I was pointed to this. Since we're targeting JS programmers with ReasonML, it's probably a good idea to rename to something else.

Reproducing:

$ cat test.mdx
```ocaml

let x = 5
```
$ mdx pp test.mdx
mdx: internal error, uncaught exception:
     Failure("invalid toplevel block: [; let x = 5]")

The --direction=infer-timestamp is not working very well. In particular it can't be used in dune because dune copies all the files to the _build directory first and therefore messes with the timestamps.

While I like the idea of having such an option, I'm quite struggling to find a good workflow for it outside of dune and its promotion mechanism.

I'd suggest we simply remove it for now until we have that figured out.

Note that if anyone's actually using this option and has a good workflow to integrate it outside of dune I'd love to hear about it. If that were the case an alternative would be to remove it from ocaml-mdx rule at least so that it only produces options that actually work fine with dune.

I get:

$ make
dune build   @install
dune runtest --no-buffer --force
Done: 682/689 (jobs: 3)>> Fatal error: H/87 unbound at toplevel
ocaml-mdx-test: internal error, uncaught exception:
                Misc.Fatal_error

on containers master (a1f3f4781d0529e2fffe8f77c64c5133342d9395) when running mdx test on the readme. It works for previous versions.

This is currently done for Async but would be nice to support lwt too.

I'd like to have installation instructions run "live", so that we can regression test our installation instructions in RWO and ocaml.org.

For that, we would need to have some way to redirect a codeblock into docker. E.g. something like:

# apk add --update opam
# opam init -ya
# opam install rwo

And have that execute the commands and fill them in.

There seems to be an intricate relationship between the ocaml-mdx and the ocaml-mdx-test binaries.

The former seems to just be some proxy that re-executes the latter with its own command line arguments.

I'm not familiar enough with mdx to understand why it works like that but I'd be curious to know.

The pattern is currently pretty fragile and more specifically, it doesn't work very well with dune, or without opam at least.

I'm using a vendored mdx so when I tell dune a rule depends on ocaml-mdx it will know how to build it. Because dune's not aware that ocaml-mdx depends on ocaml-mdx-test though, running ocaml-mdx just crashes with the following error:

ocaml-mdx: internal error, uncaught exception:
           Unix.Unix_error(Unix.ENOENT, "execvp", "../../../install/default/bin/ocaml-mdx-test")

While we could probably hack with dune rules to make that dependency explicit to dune, I'm not sure that's the right solution here.

Is there any reason why those binaries are split in such a way?

They seem to remove all the subsequent phrases.

Also, sometimes we want to ignore newlines to have clearer separations between phrase blocks.

After upgrade to ocaml-4.08.1+rc3 make test fails:
(cd _build/default/test && ../../install/default/bin/ocaml-mdx test empty_line.md)
ocaml-mdx-test: internal error, uncaught exception:
Failure("incomplete OCaml version")

Sys.ocaml_version;;

• : string = "4.08.1+rc3"

For instance, if one environment loads core:

```ocaml env=core
 # #require "core";;
 # open Core
``` 

Other environment are (indirectly) impacted:

```ocaml env=non-core
# let x = ref 1
val x : int Stdlib.ref = {Stdlib.contents = 1}
```

Instead we would like the last signature to be: val x : int ref = {contents = 1}

It would be really convenient to be able to add the skip label to an ocaml or sh snippet without having to remove other labels.

For example if I have the following snippet in my .md file:

```sh dir=./some/dir
$ some-command
Some output
``

I'd like to be able to skip it without having to remove the dir label, which can be useful for temporarily disable it while debugging for instance. In other word I'd like to be able to write:

```sh skip dir=./some/dir
$ some-command
Some output
``

and have it just skip that snippet's execution.

Currently what happens is that I get the following diff if I place skip before dir:

  ```sh skip dir=./some/dir
+ >> 
+ "skip dir" is not a valid label or prefix. Valid labels are "dir", "source-tree", "file", "part", "env", "skip", "non-deterministic" and "version" and valid prefixes are "set-".
  $ some-command

and the following error if I place it after:

$ ocaml-mdx test --prelude=prelude.ml --direction=to-ml README.md
ocaml-mdx-test: internal error, uncaught exception:
                (Sys_error
                  "./some/dir skip: No such file or directory")

Do you think that would be possible and would make sense? If not I guess we could at least improve the error reporting a bit to make it clear only one those labels can be used at a time!

I'm of course happy to contribute a fix.

Let me know what you think!

This will help in the CI (as we require a very recent version of pandoc) and on windows.

This will likely be an ocaml-specific tool for some time, so it'll ease packaging if we rename it to ocamlmdx sooner rather than later. Especially before it starts getting embedded in 3rd party projects...

We should check that one can run an expect test, and have corrections work for that test, and then include the results in the mdx. Mostly I think this would be about getting the dune rules to work properly, rather than mdx as a tool.

(e.g. we should at least document how use ppx_expect with mdx)

And we should also probably switch to omp to support new compiler releases more easily.

We are using ocaml-mdx for the new owl tutorials, but I cannot make it compile properly. The tests are failing with

   ocaml-mdx alias book/algodiff/runtest (exit 1)
(cd _build/default/book/algodiff && ../../../install/default/bin/ocaml-mdx test --prelude=prelude.ml --direction=to-md README.md)
Got an error while evaluating:
---
#require "core,core.top,ppx_jane,owl-top,owl-plplot";;

open Core
open Owl
open Bigarray

let () = Printexc.record_backtrace false
let () = 
  Owl_base_stats_prng.init 89;
  Owl_stats_prng.init 89
---
Line 2, characters 8-12:
Error: Unbound module Core

core seems to be correctly installed, also if I try to run them by hand they instead fail as

   opam exec -- ../../../install/default/bin/ocaml-mdx test --prelude=prelude.ml --direction=to-md README.md
ocaml-mdx-test: internal error, uncaught exception:
                Fl_package_base.No_such_package("unix", "")

I am not sure where to start to triage it.
The code is all here: https://github.com/mseri/owl_tutorials

The Irmin tutorials are currently using the pp subcommand to verify that each one compiles. We currently have one block which fails CI, and so it would be nice to use the skip label to avoid outputting it to the .ml file; however, the pp subcommand simply ignores the skip label.

Perhaps the pp command was never intended to be used for a test workflow such as this, but until #158 is fixed it is not possible to use the test subcommand to achieve the right result.

CC. @Julow

I'm using mdx only for tests, not for fancy documents, and for that purpose craml's syntax was just more convenient.

I'm wondering if it could be reintroduce either with a --syntax cram flag, or by simply assuming that if the input is a .t file then it will be written in the cram syntax.

Something like this should work:

```ocaml $FOO=bar
 # print_newline (Sys.getenv "FOO")
 bar
 ``` 

This kind of diff is a bit annoying:

-|  (cd _build/default && /home/yminsky/.opam/fresh-4.06.1/bin/ocamlc.opt -w -40 -g -bin-annot -I .freq.eobjs -I /home/yminsky/.opam/fresh-4.06.1/lib/base -I /home/yminsky/.opam/fresh-4.06.1/lib/base/caml -I /home/yminsky/.opam/fresh-4.06.1/lib/base/shadow_stdlib -I /home/yminsky/.opam/fresh-4.06.1/lib/sexplib0 -I /home/yminsky/.opam/fresh-4.06.1/lib/stdio -no-alias-deps -o .freq.eobjs/counter.cmo -c -impl counter.ml)
+|  (cd _build/default && /Users/thomas/git/rwo/book/_opam/bin/ocamlc.opt -w @a-4-29-40-41-42-44-45-48-58-59-60-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -bin-annot -I .freq.eobjs -I /Users/thomas/git/rwo/book/_opam/lib/base -I /Users/thomas/git/rwo/book/_opam/lib/base/caml -I /Users/thomas/git/rwo/book/_opam/lib/base/shadow_stdlib -I /Users/thomas/git/rwo/book/_opam/lib/sexplib0 -I /Users/thomas/git/rwo/book/_opam/lib/stdio -intf-suffix .ml -no-alias-deps -opaque -o .freq.eobjs/counter.cmo -c -impl counter.ml)

It would be nice if mdx could support some way to skip some words (e.g using <path> or some similar keywords or notation) instead of the sometimes too generic ... ellipsis.

When running the RWO build, I got the following error:

[yminsky@tiamat RWO]$ make
         mdx book/testing.html (exit 125)
(cd _build/default/book && /home/yminsky/Documents/code/RWO/_opam/bin/mdx output testing/README.md -o testing.html)
mdx: internal error, uncaught exception:
     Failure("invalid padding: \"\"")

The source of the problem appears to be a block that looks like this:

sh file=../../examples/code/testing/broken_inline_test/run.sh
  $ jbuilder runtest --dev
           run alias runtest (exit 2)
  (cd _build/default && ./.foo.inline-tests/run.exe inline-test-runner foo -source-tree-root . -diff-cmd -)
  File "test.ml", line 3, characters 0-73: rev is false.

  FAILED 1 / 1 tests
@@ exit 1

The space before the FAILED line appears to be the issue. The failure is one problem, and the lack of line number in the error message is another.

@avsm pointed to this bit of code in mdx as the culprit:

let of_lines t =
  let pad = pad_of_lines t in
  let unpad line =
    if String.length line < pad then Fmt.failwith "invalid padding: %S" line
    else String.with_index_range line ~first:pad

It appears that this was recently introduced. See an example here.

Is there any particular reason for that?

Currently it is possible to specify that a code block should run for a given version of the OCaml compiler only (with version=4.02 for instance). It would be nice to extend this feature with version constraints.

For using mdx to generate README files, it would be convenient if it had the same toplevel environment as that provided by dune utop. This allows locally built packages to be used to process mdx files.

somewhat similar to #35 but with comments:

```ocaml
(* foo *)
# 1+1;;
```

returns a syntax error.

There's mainly two issues with the runtest-all alias added for .md files containing non-deterministic code snippets:

1. dune build @runtest-all doesn't trigger dune file generation
2. dune build @runtest-all doesn't run mdx test on files without non-deterministic code snippets

This should probably be fixed by always adding a dependency from runtest-all to runtest.

The Irmin tutorials currently contain many instances of Markdown text within a module definition, e.g:

```ocaml
module Counter: Irmin.Contents.S with type t = int64 = struct
	type t = int64
	let t = Irmin.Type.int64
\```

Now we need to define a merge function:

\```ocaml
	let merge ~old a b =
	    let open Irmin.Merge.Infix in
		old () >|=* fun old ->
        let old = match old with None -> 0L | Some o -> o in
        let (+) = Int64.add and (-) = Int64.sub in
        a + b - old

    let merge = Irmin.Merge.(option (v t merge))
end
\```

This is particularly important when tutorialising backend definitions, as these require several large module definitions. However, it appears that these are currently unsupported by ocaml-mdx:

ᐅ ocaml-mdx test data/tutorial/contents.md
Got an error while evaluating:
---
module Counter: Irmin.Contents.S with type t = int64 = struct
	type t = int64
	let t = Irmin.Type.int64
---
Characters 110-110:
Error: Syntax error: 'end' expected
Characters 55-61:
Error: This 'struct' might be unmatched

It would be nice if mdx would support this sort of pattern 🙂

Based on the original discussion in ocaml/dune#895 (comment).

It would be great to have support for odoc in mdx. It is a common practice to include examples in API documentation and those can get out of date very quickly. Additionally the new odoc version will include support for documentation pages which would benefit from this feature.

Here are some questions I have regarding the potential integration:

• How would be the OCaml code blocks matched? Should we find all occurrences of {[ ... ]} in the documentation or is it better to have a special annotation for this (some examples might not be runnable at all on purpose)?
• Does it make sense to support the sh syntax?
• Would it be possible to support Reason syntax?

I would be glad to work on a PR for this feature in the next couple of weeks. Suggestions about how this could be implemented would be very appreciated! :)

Thanks!

/cc @gpetiot

The CLI subcommands (output, pp, rule, test) currently all accept only a single file as input; it might be nice if they could accept more than one at a time.

This would be helpful when verifying a set of markdown files, as in the Irmin tutorials. Currently, I think this requires an unsightly bash action to do it in a single Dune action:

  (action (with-stdout-to %{targets}
            (bash "echo %{tutorials} | xargs --max-args=1 ocaml-mdx pp")))

Whereas it would be nice to get away with simply:

  (action (with-stdout-to %{targets}
            (ocaml-mdx pp %{tutorials})))

What do you think?

Mdx appear to sometime add trailing whitespaces to toplevel evaluations. See an example of this behaviour here.

I haven't had time to investigate why this happened but I'm guessing it could be fixed by systematically stripping trailing whitespaces from toplevel evaluations as there's no value in keeping them around even if the toplevel did return them.

In some use-case we want to explain how to build a project by quoting the contents of the files in that project. We would like the instructions to be kept up-to-date automatically, so we need a way to promote the contents of the file to the documents and/or the reverse.

It would be nice to use Opttoploop, Opttopdirective, opttoplevel.cmxa instead of their bytecode equivalents, as it would speed up the tests quite a lot and avoid us to ship two binaries (and use an horrible hack to generate the proper help for mdx test).

/cc @diml, @trefis and @let-def who might have an opinion on how to do this properly (which might involve making changes in the compiler API).

I recently ran into an issue with mdx. I have an .md file containing several ocaml code snippets pointing to different parts of an .ml file. I'm using mdx with --direction to-ml and for some reasons, it gets the first right but doesn't write the other ones to the .ml file.

My first guess was that the first part starts at the file's toplevel but introduces the declaration of a submodule and the subsequent parts all are within that submodule. I thought this could be the reason for the issue because [@@@...] attributes are structure_items of a different part of the AST but I doubt mdx actually relies on that.

You can take a look at the .md file here and the current target file here. Running dune runtest there result in the diff action on the .ml file suggesting to remove anything from [@@@part "2"] (included) to the end of the file. I'm not sure exactly why this is happening. I'll to reproduce that in a simpler example if I find the time to!