Add SMTCoq to Coq Platform about platform

[MSoegtropIMC]

No problem - solving system issues in bringing nice research to a broader audience is what Coq Platform is about.

I don't think that using a binary CVC4 or ANTLR is an option. Both are not that frequently used that one can rely on that all supported systems will have binary packages of the desired version - I understand that you are not flexible in that. A binary ANTLR might be an option in case CVC4 is flexible about the specific version.

If you insist on a specific CVC4 version - which I guess will always be the case - compiling CVC4 from sources is the only reliable option - anything else will likely be a maintenance nightmare for you in case some lecturer decides to use SMTCoq in a larger course.

So I guess I should look into compiling ANTL3 in opam. If it turns out to be too messy, we can still discuss alternatives.

Btw.: Once we have ANTLR3, I expect CVC4 to be easy.

[ckeller]

Sure, here are some references:

• In their ITP'10 paper entitled "Fast LCF-Style Proof Reconstruction for Z3", Sascha Böhme and Tjark Weber already pointed out some limitations of Z3 proofs.
• In our CPP'11 paper "A Modular Integration of SAT/SMT Solvers to Coq through Proof Witnesses", we showed that verit+Coq is more efficient than Z3+Isabelle/HOL even if Z3 is more efficient than veriT; there are multiple factors, but one is the easiness of proof reconstruction for veriT.
• More recently, in the CADE'21 paper "Reliable Reconstruction of Fine-grained Proofs in a Proof Assistant", Hans-Jörg Schurr, Mathias Fleury and Martin Desharnais show that the smt tactic in Isabelle/HOL is more reliable with veriT than Z3.

[MSoegtropIMC]

A small update: about half of the files added in the Debian fork are from GNU libtool and easy to install via system packages, but somehow the configure file didn't pick up an installed libtool - need to figure out why. The other files are FSF public domain licensed files, so no issue putting them into opam. I will try tomorrow. Pickung up a system libtool should be an easy patch.

[MSoegtropIMC]

Thanks for the request!

I looked at CVC4 in the past and found it quite hard to make it an opam package mostly because the build scripts download sub components from the internet which the opam sandbox would not allow. So one would have to rework the high level build infrastructure of CVC4. Do you have contacts to the CVC4 maintainers? I am open to do this (create CVC4 opam packages), but I can't maintain them long term.

The others I didn't look into. Typically it is not hard to make opam packages, e.g. I created packages for EProver and the TPTP variant of Z3 (both used by CoqHammer, which also would like to have CVC4). In case they are low effort packages, I could also maintain them long term.

Is the priority for solvers the order you listed them above?

Btw.: would it make sense to go to CVC5 right away?

[ckeller]

I updated the order of the provers in my initial message so that it matches the priorities.
I am in contact with the CVC4/5 developers so I can ask them. Can you point me to some documentation where I can understand how to build opam packages for non OCaml code?
A port for CVC5 (as well as the last version of veriT) is under progress but it is a lot of effort. I can't promise it will be done on time.

[Zimmi48]

A note about CVC4 is that last I checked, SMTCoq depended on a specific version of CVC4 which is not the latest, contrary (I think) to CoqHammer. Is it still the case, or has a port to the latest CVC4 version already happened?

[ckeller]

No, it is not the latest.

[MSoegtropIMC]

@ckeller : I am not aware of any documentation for creating non OCaml packages in opam, but then besides the fact that an opam switch comes with an OCaml compiler, there is nothing special about OCaml in opam. The opam files download and patch code, call autoconf and make and the like and it is pretty much the same for OCaml and C++ code - one just leaves away the dependency on OCaml and adds a dependency for say conf-gcc. The other thing is that not all libraries one might want for C/C++ code already have opam packages, but meanwhile the collection of supported C/C++ libraries in opam is quite OK.

As I said I can create the initial opam packages for CVC4 and its dependencies - I expect that this will be exceptionally tricky and I guess it is quickest if I do it myself. But I would appreciate if someone else could do the maintenance of the packages afterwards. Usually opam packages just follow documented build procedures, but for CVC4 this will not work for the reasons I outlined above. So the opam package might be a bit maintenance intensive and it should be done by someone who is aware of changes in the top level build logic of CVC4. Translating such changes to opam should be straight forward with a working opam package for the previous release even for non opam experts and if there is something tricky of course I can help. What I don't want to do is regularly reverse engineer changes in CVC4 top level build scripts.

[Zimmi48]

@MSoegtropIMC Something to note is that CVC4 is no longer developed, so there won't be any new release (besides the one that SMTCoq currently use, and the current latest one).

[MSoegtropIMC]

@Zimmi48 : good point. Still my notes are valid as soon as SMTCoq switches to CVC5 - unless they have a more meta build tool friendly build system.

Possibly one can also discuss with the CVC4/CVC5 team the requirements of meta build tools and implement support for this in their scripts.

[MSoegtropIMC]

I guess it is best if I start with CVC4 - as you say there shouldn't be an issue then - and have a look at the status quo of the CVC5 build, so that we can start a discussion with the CVC5 team.

[ckeller]

Thanks a lot!
As @Zimmi48 said, CVC4 is not maintained so the next changes will happen when SMTCoq have support for CVC5, which will not be the case in the next months.
If I can be of any help for the opam support and maintenance, of course I will.

[ckeller]

Just to make things clear: for veriT and CVC4, the supported versions are exactly those which are linked in my first message.

[MSoegtropIMC]

OK, I will look at your instructions first.

Two more questions:

1.) The issue I had was with the ./contrib/get-antlr-3.4 step in (https://github.com/CVC4/CVC4-archived/blob/master/INSTALL.md).

Your instructions are:

./autogen.sh
./configure
make

Does the 1.6 version not require ANTLR?

2.) The CVC archive only has a 1.8 tag - see (https://github.com/CVC4/CVC4-archived). Where can I get the 1.6 code?

[Zimmi48]

You can get it from https://cvc4.cs.stanford.edu/downloads/builds/src/cvc4-1.6.tar.gz (that's what nixpkgs uses for SMTCoq's CVC4).

[ckeller]

You can algo get it from here: https://github.com/cvc5/cvc5/releases/tag/1.6 (even if it is called cvc5). I just tried and neither of them compile on my laptop... I am inverstigating further. (I have not compiled it for a while since I use the binaries which have always run out of the box.)
@MSoegtropIMC Maybe SMTCoq's docker file can help you: https://github.com/smtcoq/smtcoq/blob/coq-master/ci/smtcoq.docker

[MSoegtropIMC]

@Zimmi48 @ckeller : thanks for the pointers - I see how far I get.

The docker image seems to use a binary for CVC4: https://github.com/smtcoq/smtcoq/blob/9e097b8c4037813ca5d09e34d5de6a947edbc45a/ci/smtcoq.docker#L53

[ckeller]

Yes - as I said, I have not compiled CVC4 for a while since I use the binaries which have always run out of the box.

[MSoegtropIMC]

A small update: CVC4 1.6 requires the C bindings for ANTLR3. As far as I can tell ANTLR3 is written in Java, so compiling it with opam would require a java compiler - not out of the question but, well ...

As far as I can tell all by Coq Platform supported platforms do have a system package for the ANTLR3 java part, but MacPorts does not have a package for the ANTLR3 C library. Mac Homebrew and Cygwin seem to have the C library and I guess all Linux distros as well.

What I am not sure about is if the ANTLR C library actially requires the Java part, or if this separate.

So I would say the options are:

• seek advice from an ANTLR expert or the CVC4 team on this topic
• check if ANTLR3 Java and C are independent
• see if one can extend the MacPorts package to include the C part
• see if one can compile only the C library part of Antrl3 assuming the Java part exists in opam

[MSoegtropIMC]

@ckeller @Zimmi48 : what is your opinion on creating opam packages for Java and Maven? If that is it then, it might be OK IMHO - if we need a lot of Java packages then, it gets tricky, but the system level dependencies of the system ANTLR3 packages don't look like that this would be the case. Of course I can't tell what is compiled in.

[ckeller]

@MSoegtropIMC :-( sorry about that... Would relying on the binaries be an option?
Anyway, I can contact CVC4 developers about ANTLR3.

[MSoegtropIMC]

It would help to get feedback from the CVC4 team how picky they are about the ANTLR3 version.

[MSoegtropIMC]

Waiting on some builds I played with ANTLR3. Some results:

• the java part is reasonably easy to build - on Mac ARM MacPorts:

sudo port install maven3
export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk17-zulu/Contents/Home
mvn -Dgpg.skip=true -DskipTests install

• I couldn't figure out as yet if the C part requires the java part, but I don't think so. As far as my (very limited) understanding goes the Java part is required to generate parsers but to execute parsers only the C part is required. I guess that CVC4 has the generated parser files checked in - not sure though
• The C library is nasty to build:
  • in the ANTLR3 repo the autoconf generated configure file is missing and autoconf and autoreconf give tons of errors trying to generate the the configure file from configure.ac - I guess the root cause is that about 10 required standard automake/autoconf files have been removed, possibly for licensing reasons.
  • Since there is a Debian package, I looked into how they are doing it. The Debian package source is here (https://packages.debian.org/source/buster/libantlr3c) - they keep a fork of the runtime/C folder of the ANTLR3 repo here (https://salsa.debian.org/debian/libantlr3c). This fork has the files missing in the original repo to make autoconf work, but it strangely misses two C source files, so make fails - no idea how the Debian package is built - I didn't try tags, though.
  • If I copy the missing C source files from the ANTLR3 original repo into the debian repo it compiles
  • If I copy the missing autoconf files from the Debian repo to the ANTLR3 original repo - also to newer versions than Debian has - it also works

After copying missing files one way or the other, these build instructions work:

cd runtime/C
autoconf
./configure --enable-64bit
make

So all in all it is quite messy. One can put the files missing in the original repo from the Debian repo as opam patches, but one has to study the licenses. But in case the Java part is not needed, it is technically not that complicated.

[MSoegtropIMC]

@ckeller : doing some (unrelated) tests with z3 I found that it seems to support the SMTLIB2 commands (set-option :produce-proofs true) and (get-proof) and it produces something which makes sense at first glance. I tested this with Z3 version 4.8.17 - 64 bit and Z3 version 4.11.0 - 64 bit. I wonder if you looked into this - since we already have Z3 for CoqHammer it might be a sensible addition to the portfolio of provers. But I guess each SMT solver and even each logic of a SMT solver produces proofs in a different logic which all need individual adoption to Coq.

Anyway I will continue to work on veriT and CVC4.

[ckeller]

@MSoegtropIMC Thanks for the information! Z3 produces proofs with a very low level of details. We have shown for many years that it is much faster to use veriT because proof reconstruction is easier, and Isabelle is currently switching to veriT instead of Z3.

[MSoegtropIMC]

@ckeller : I see - interesting! Just out of curiosity: do you have examples which show what nature the lack of detail in a Z3 proof is? From looking at trivial examples one can still easily follow pencil and paper and guessing what the underlying logic constructs mean this is not obvious.