ocaml-gospel / gospel Goto Github PK
View Code? Open in Web Editor NEWA tool-agnostic formal specification language for OCaml.
Home Page: https://ocaml-gospel.github.io/gospel
License: MIT License
A tool-agnostic formal specification language for OCaml.
Home Page: https://ocaml-gospel.github.io/gospel
License: MIT License
On such a pattern the OCaml compiler emits warning 57:
let f = function
| (x, 0) | (1, x) when x = 0 -> 3
| _ -> 4
that is,
Warning 57: Ambiguous or-pattern variables under guard;
variable x may match different arguments. (See manual section 9.5)
See https://v2.ocaml.org/releases/4.14/htmlman/patterns.html#sss:pat-or
Suggestion: do not allow such patterns in Gospel.
to avoid confusion with Seq
from the OCaml standard library
The following produces a syntax error:
val log2b: int -> int
(* @ r = log2b [i] x
requires i >= 0
requires x = pow 2 i
ensures r = i *)
Create an empty stdlib.mli
file:
$ >> stdlib.mli
and run
$ gospel check stdlib.mli
gospel: internal error, uncaught exception:
Invalid_argument("index out of bounds")
Raised by primitive operation at Gospel__Parser_frontend.string_equal_sub.aux in file "src/parser_frontend.ml", line 47, characters 41-51
Called from Gospel__Parser_frontend.remove_after_module_aliases.find in file "src/parser_frontend.ml", line 55, characters 7-40
Called from Gospel__Parser_frontend.remove_after_module_aliases in file "src/parser_frontend.ml", line 57, characters 14-65
Called from Gospel__Parser_frontend.parse_ocaml in file "src/parser_frontend.ml", line 66, characters 6-95
Called from Dune__exe__Check.run_file in file "bin/check.ml", line 34, characters 16-32
Called from Dune__exe__Check.run.(fun) in file "bin/check.ml", line 62, characters 33-53
Called from Dune__exe__Cli.run_check in file "bin/cli.ml", line 42, characters 10-47
Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 24, characters 19-24
Called from Cmdliner_eval.run_parser in file "cmdliner_eval.ml", line 34, characters 37-44
Currently only val_spec
contain their original text and location. All specifications should have them.
Originally posted by fpottier February 16, 2022
In the following .mli
file, the first declaration is valid Gospel but ugly. The second declaration seems much more natural but seems currently invalid: it is rejected by gospel check
and by Cameleer. Would it be possible to allow it? This would seemingly make sense at least for tuples and records.
val write : int array * int array -> unit
(*@ write ab
requires Array.length (snd ab) > 0 *)
val write2 : int array * int array -> unit
(*@ write2 (a, b)
requires Array.length b > 0 *)
I think the following program should raise an error since a
variable isn't bind in both sides of the or-pattern.
val f : int -> int
(*@ y = f x
ensures match x with _ | a -> a = 1 *)
I note that an unattached Gospel comment, like this, is ignored:
(*@ a = make n x
requires n > 0
ensures forall i. a(i) = x *)
This seems undesirable: perhaps this comment is unattached by mistake (e.g., due to an unwanted blank line), so we should at least warn about it.
As a particular case, a Gospel comment that appears in front of a value declaration is ignored:
(*@ a = make n x
requires n > 0
ensures forall i. a(i) = x *)
val make: int -> 'a -> 'a t
This seems undesirable; ideally, I think that this Gospel comment should be considered attached to the declaration of make
.
Contract headers are currently mandatory, but being able to only specify pure
or diverges
without a full contract when they are alone would let us write more concise yet readable specifications.
raises E x -> p | E y -> p'
Is represented by [(y, p'); (x, p)]
.
Error Message
"Invalid integer literal in base 10: '0x16' "
The gospel check
command fails on input :
val f: int -> (int * int, int) result
(*@ r = f ()
ensures
match r with
| Ok (p, q) -> true
| _ -> false *)
with the following spurious error :
Error:
The constructor `Ok' expects `1' argument(s)
but is applied to 2 argument(s) here.
It may be related to too late instanciation of type variables from ('a*'b) result
.
Hi,
The README starts with:
Gospel is a behavioural specification language for OCaml program. It provides
developers with a non-invasive and easy-to-use syntax to annotate their module
interfaces with formal contracts that describe type invariants, mutability,
function pre-conditions and post-conditions, effects, exceptions, and [much
more](https://ocaml-gospel.github.io/gospel/language.html)!
But https://ocaml-gospel.github.io/gospel/language.html returns "Page Not Found".
The following .mli
file does not cause an error unlike the expected behavior
type t = L of t * t | E
val f : t -> int
(*@ r = f a
ensures
match a with
| L E E -> r = 1
*)
Still to do:
old e
is effectively just e
when e
is read-only, so the old
primitive is likely an error from the programmer; we want to issue a warning if that happens.
The following is currently rejected:
val a : int ref
val f : int -> int
(*@ y = f x modifies a *)
The following piece of code issues a syntax error:
val f : int list -> int
(*@ r = f l
requires match l with
| _ :: x :: _ :: _ :: _ -> x >= 0
| _ -> false *)
The reason is that our parser only accepts pat_uni_
as (::)
arguments. Thus, one way to fix the above is by doing the following:
val f : int list -> int
(*@ r = f l
requires match l with
| _ :: (x :: (_ :: (_ :: _ ))) -> x >= 0
| _ -> false *)
This is tremendously inconvenient.
Writing two logical functions with the same name silently overrides the first one, perhaps we could issue a warning instead?
Currently, Gospel accepts the following two clauses when E
is an exception constructor with arguments:
raises E
raises E p -> t
but rejects
raises E _
raises E
should be rejected during typing (bad arity)raises E p
should be accepted during parsing.Gospel only allows comma-separated variable names as return values at the moment.
Allowing patterns would be closer to OCaml and more convenient and generic (e.g. when destructing records).
Similarly to OCaml, non exhaustive pattern matchings should be rejected.
Most of the time, functions returning unit
have side effects.
If the side effect affects some observable data, then there should be a modifies
clause.
If it is another kind of side effect, the function probably can't be specified anyway
Some ideas on improving the handling of ghost variables in the AST:
ghost
field to vsymbol
instead of having a Lghost
label[@ghost x: t, y: t']
, to disambiguate with lists and allow patterns as returned values.The following .mli file gives rise to Error: OCaml syntax error
:
(*@ model content: int *)
type 'a t
The error message seems incorrect: this is a valid OCaml file.
More generally, OCaml allows documentation comments to appear either before or after the object that they document. It would be good if the same was true of Gospel comments. Thus, I would expect the following to be accepted:
(**A type. *)
(*@ model content: int *)
type 'a t
Or, if we insist that Gospel comments must come after the object that they document, then perhaps this should be documented.
There are no such litterals in Gospel yet, but they would be useful in specifications, e.g.
raises Invalid_argument s -> s = "input should be positive"
The following code
val (==) : 'a -> 'a -> bool
(*@ r = x0 == x1
ensures x0 = x1 *)
is not accepted by the Gospel parser.
The reason is that uparser
only expects a function name and a sequence of arguments in specification header (cf,
Lines 214 to 219 in 6eadafb
consumes
and modifies
declarations currently accept any term
s.
These should be restricted to qualified identifiers only e.g. x.y.z
.
The following piece of code
val f : int -> int * int
raises a too few parameters
error.
The specification of a recursive function can easily grow when one needs to duplicate the same property as a pre- and postcondition (see for instance https://github.com/dCastanho/algocameleer/blob/main/proofs/path.ml).
We could imagine allowing the clause invariant
in functions' contract, avoiding such duplication. Hence, a Gospel specification of the form
val f : 'a -> 'b
(*@ requires P
invariant I
ensures Q *)
would be automatically translated into
val f : 'a -> 'b
(*@ requires P
requires I
ensures Q
ensures I *)
Fields symbols and other logical symbols live in the same namespace, which means in
(*@ predicate p = ... *)
type t
(*@ model p : ... *)
the latter p
field overrides the predicate (the opposite would be true too if they were in the reverse order).
when
guard in pattern cases.odoc
documentationAfter the opam-repository
PR is merged:
Post-release:
why3gospel
(?)vocal
(?)gospel check
issues a The type declaration for `t' contains a cycle
error for the following piece of code:
module type A = sig
type t
end
module type B = sig
type t
module C : A with type t = t
end
I believe this is a bug in the type-checker.
Field t_loc
of TTerm.term
is on option.
Not sure whether you'd prefer this to be an issue instead of a discussion.
Originally posted by edwintorok January 17, 2022
Hi,
Sorry for raising so many different discussions, but I thought it'd be best to separate it into 3 different topics as they are quite orthogonal. I've recently started looking at the possibility of using Gospel to prove some OCaml code.
I'm aware this is quite a young project with some rough edges around the tooling, but I think it already provides quite a lot of value by being able to write down function behaviour in a machine parseable form. In fact writing down the specifications alone in gospel helps in thinking and reviewing code correctness (previously all these constraints were just in my head, and would have to "swap in" when coming back and looking at an old piece of code).
int32, int64 would be good to have, including 2's complement arithmetic or a form where any overflow means an error.
I think the latter is what Why3's stdlib provides, but isn't quite yet easily available in Gospel, but see https://github.com/ocaml-gospel/cameleer/blob/master/examples/cameleer.drv should be possible to add.
It might even be definable in pure Gospel syntax, although it'd duplicate the definitions in Why3.
There is also a question on what to do about int
. Cameleer currently defines it the same as integer
which is not quite right: it might prove that a function never throws an exception, whereas in practice it would due to integer overflow causing some invariants to not be upheld.
I see the value of defining specifications with arbitrary sized integers, but then verification should ensure no overflow happens (and treat that as a spec failure if it does).
But which integer size should int
be? If you're targeting verification only for one platform, e.g. x86-64 then choosing statically the size would make sense.
OTOH for fully portable programs it'd be great if we could prove that they work for both 31-bit and 63-bit sized int
s. E.g. one might focus ortac
fuzzing on just one build, which would leave the other one less well tested, and this is exactly where formal tools could help prove it works everywhere.
I don't see how to define a type depending on another predicate though, e.g. ideally something like this (IIUC this can't be expressed in Gospel):
(word_size = 32 -> type int = int31) && (word_size = 64 -> type int = int63)
Or perhaps just the various arithmetic operations would define their limits based on word_size, but that means it couldn't reuse the types from Why3...
Alternatively verification tools like Cameleer could build 2 formulas: one for word_size =32 and one for word_size =64 and then merge the 2 formulas with && (perhaps simplifying and dropping duplicate entries along the way to make the proofs simpler).
(Or run 2 separate proofs, once for 32-bit and once for 64-bit wordsizes, which might be the easiest with the addition of a cmdline flag to cameleer)
What do you think the best solution would be here?
I'd be happy to help in adding support for these (either in Gospel or Cameleer, although Cameleer says no support is provided yet, so not sure if it accepts PRs yet)
The following code
val (==) : 'a -> 'a -> bool
(*@ r = (==) x y
ensures r <-> x = y *)
raises a Type checking error with the following message: val specification header does not match name
.
We added a new Tfield
node to the typed AST 5bdfc54, but it is still possible to write record fields as function applications, i.e. f x
instead of x.f
.
The type-checking should only allow the later.
When there are multiple conditions in a specification (in my experience, 3 or more), the error messages provided to the user are often obscure.
With files
(*example.mli*)
type 'a t
val silly_create : int -> 'a t
(*@ t = silly_create c
requires c >= 5
requires c < 10
requires c = 0
*)
(*myexe.ml*)
let _ = Example_ortaced.silly_create 0
running
~/issue ᐅ ortac example.mli > example_ortaced.ml
~/issue ᐅ dune build
~/issue ᐅ dune exec ./myexe.exe
produces
File "example.mli", line 3, characters 0-153:
Runtime error in function `silly_create'
- the invariant
`s c = '
was violated in the pre-state.
Fatal error: exception Ortac_runtime.Error(_)
Of course, it is the "requires c >= 5" condition that is violated, but the message 's c =' does not reveal this. Full example is included below in a zip file.
The culprit is the function term_printer
on line 10 of ortac/src/core/translate.ml. The argument string text
is the full text of the specification. In the example, this is
t = silly_create c
requires c >= 5
requires c < 10
requires c = 0
I think the intent of the code is for global_loc.loc_start.pos_cnum
and t.t_loc.loc_start.pos_cnum
to be indices into the original file as written by the user, making t.t_loc.loc_start.pos_cnum - global_loc.loc_start.pos_cnum
the start of the violated condition c >= 5
in text
. However, global_loc.loc_start.pos_cnum
and t.t_loc.loc_start.pos_cnum
are instead indices into preprocessed code (given by Pps.run
on line 37 of parse_ocaml_lb
in gospel/src/parser_frontend.ml). Printing the string output by Pps.run
reveals that some characters are inserted between the start of the specification comment and the start of c >= 5
during preprocessing. The index t.t_loc.loc_start.pos_cnum - global_loc.loc_start.pos_cnum
does indicate the start of c >= 5
in the preprocessed spec.
I tried to locally create a fix where the string stored in the sp_text
field of the specification representation (type val_spec
in gospel/src/tast.ml) is the preprocessed spec, rather than the spec written by the user. In the example, this would mean that the argument text
to term_printer
would be
[@@gospel
# 4 "example.mli"
{| t = silly_create c
requires c >= 5
requires c < 10
requires c = 0
|}
# 8 "example.mli"
]
I think this would work, but I couldn't figure out a simple means of achieving it without modifying a lot of code.
Alternatively, the body of term_printer
could be replaced by Fmt.str "%a" Tterm_printer.print_term t
. In the example, this would produce an error message of
File "example.mli", line 3, characters 0-153:
Runtime error in function `silly_create'
- the invariant
`((integer_of_int
c:int):integer >= 5:integer):prop'
was violated in the pre-state.
Fatal error: exception Ortac_runtime.Error(_)
While going through the documentation looking for examples with @shym, we've spotted some mistakes (gospel check
throwing a Syntax Error).
#249 fixes one of them. Another is that in the Union-find
example, equivalent
is used to name a predicate though it is a reserved keyword.
These two findings has been done empirically (aka we were lucky to stumble on them). Maybe we should extract the OCaml (and gospel) code of the documentation and check them.
By default, when a checks
precondition is not satisfied, the function is expected to fail with Invalid_argument
.
There are however occurrences when the developer might want to raise another exception in those cases (see OCaml's stdlib Queue.pop
for instance).
It would be a nice feature to be able to specify the raised exception manually, while keeping a default to Invalid_argument
. Here is a first syntax proposal, to be discussed:
(*@ ...
checks x > 0 with exception <exception pattern case> *)
The type-checker does not currently make any difference between
type t = E of int * int
and
type t = E of (int * int)
while this difference exists in OCaml.
For instance, it is possible with the latter to write let x = (1, 2) in E x
, while this is not allowed with the former.
In both cases, this term is allowed in Gospel, which is wrong.
In the gospel docs (here) there is the following example for exceptional postconditions:
(*@ ...
raises Error "foo" -> P | Error _ -> Q
raises Error x -> R *)
However, with the following file,
(*example.mli*)
exception Error of string
val silly : int -> int
(*@
raises Error "foo" -> 1 = 1 | Error _ -> 10 = 10
raises Error x -> 42 = 42 *)
I get
~ ᐅ gospel check example.mli
File "example.mli", line 5, characters 11-52:
5 | raises Error "foo" -> 1 = 1 | Error _ -> 10 = 10
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Type checking error: Exception pattern does not match its type.
Simpler tests produce the same problem. For example,
(*example1.mli*)
exception Error of string
val silly : int -> int
(*@
raises Error "foo" -> 1 = 1 *)
gives
~ ᐅ gospel check example1.mli
File "example1.mli", line 5, characters 11-31:
5 | raises Error "foo" -> 1 = 1 *)
^^^^^^^^^^^^^^^^^^^^
Error: Type checking error: Exception pattern does not match its type.
and
(*example2.mli*)
exception Error of int
val silly : int -> int
(*@
raises Error 0 -> 1 = 1 *)
gives
~ᐅ gospel check example2.mli
File "example2.mli", line 5, characters 11-27:
5 | raises Error 0 -> 1 = 1 *)
^^^^^^^^^^^^^^^^
Error: Type checking error: Exception pattern does not match its type.
Only the most general pattern seems to work. For example,
(*example3.mli*)
exception Error of int
val silly : int -> int
(*@
raises Error z -> 1 = 1 *)
gives
~ᐅ gospel check example3.mli
OK
Regarding pure
clauses, I know about them, and I know about the implicit hypothesis that every higher-order function is pure (and requires pure arguments), but I would like to remove this hypothesis. Hence my suggestion to view isPure
as a logical proposition, allowing a higher-order function (such as map
) to explicitly indicate that its argument f
must be pure (if desired).
Originally posted by @fpottier in #147 (comment)
Documentation pages are becoming quite long and browsing them without a visible table of contents is not easy.
We should add a sticky navbar on the side to easily go through it.
At the moment, constants only have ensures
clauses. We defined that as a property that holds at the end of the module evaluation.
It would also be nice to have invariant
clauses, that hold at every function entry/exit.
lsymbol
is used for a lot of things:
The difference being coded with two booleans, which is not very clear in my opinion, maybe we can organize that with a variant:
type lsymbol =
| Constructor of Ident.t * ty list
| Field of Ident.t * ty
| Function of Ident.t * ty list * ty
Or maybe with named tuples rather than unnamed ones.
I was surprised by the choice of f[x <- v]
for function update.
Syntax is (as always) controversial 🤷♂️ Meta-syntax for function update+substitution within the field of formal PL is particularly plagued by many variations.
Guy Steele made a survey and gave a number of recommendations based on the result a few years ago (slides 37-44):
https://groups.csail.mit.edu/mac/users/gjs/6.945/readings/Steele-MIT-April-2017.pdf
Popularity is (of course) a terrible measure, but choosing a more common syntax would nevertheless lower the bar for new users already familiar with it... 🤔
From @pascutto I understand that other factors play in, such as choosing the same update syntax as Why3 for consistency.
Currently that requires two clauses:
type 'a t
(*@ mutable model contents: 'a Set.t *)
val add : 'a t -> t -> unit
(*@ add t x
modifies t.contents
ensures t.contents = Set.add x (old t.contents) *)
Example new syntax:
val add : 'a t -> t -> unit
(*@ add t x
modifies t.contents <- Set.add x t.contents *)
Other possibility with the with
keyword; seems more human-friendly, but less close to OCaml's record syntax:
val add : 'a t -> t -> unit
(*@ add t x
modifies t.contents with Set.add x t.contents *)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.