leanprover-community / lean-auto Goto Github PK

View Code? Open in Web Editor NEW

58.0 58.0 9.0 2.43 MB

Experiments in automation for Lean

License: Apache License 2.0

Lean 99.50% SMT 0.40% OpenEdge ABL 0.10%

lean-auto's People

Contributors

Stargazers

Watchers

Forkers

abentkamp semorrison joshclune denisgorbachev dranov atomb

lean-auto's Issues

Feature request: configuration option to allow the user to change solver timeouts

It looks like the timeout is hard-coded right now?
https://github.com/leanprover-community/lean-auto/blob/main/Auto/Solver/SMT.lean#L70

lean-auto failure on simple model.

I have a trivial Lean spec:

import Auto
import Std.Data.BitVec
set_option auto.smt true
set_option auto.smt.trust true
set_option trace.auto.smt.printCommands true
set_option trace.auto.smt.result true

inductive Zone where
  | Z1 | Z2 | Z3 | Z4
  -- Ask Lean to automatically show that type is not empty, has a representation function, and
  -- equality is decidable
  deriving Inhabited, Repr, DecidableEq

abbrev Area : Type := Int

def Zone.MinArea1 : Zone → Area
  | .Z1    => 10000
  | .Z2    => 5000
  | .Z3     => 3500
  | .Z4     => 2500

example (x: Zone) : x.MinArea1 >= 2500 := by cases x <;> simp -- succeeds
example (x: Zone) : x.MinArea1 >= 2500 := by cases x <;> auto -- fails

I can't seem to prove it by auto because it is not expanding the definition of MinArea1:

[auto.smt.printCommands] (declare-sort |Empty| 0) 

[auto.smt.printCommands] (define-fun |nsub| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|>=| |x| |y|) (|-| |x| |y|) 0)) 

[auto.smt.printCommands] (define-fun |itdiv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|div| |x| |y|) (|-| (|div| (|-| |x|) |y|))))) 

[auto.smt.printCommands] (define-fun |itmod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|mod| |x| |y|) (|-| (|mod| (|-| |x|) |y|))))) 

[auto.smt.printCommands] (define-fun |iediv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) 0 (|div| |x| |y|))) 

[auto.smt.printCommands] (define-fun |iemod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|mod| |x| |y|))) 

[auto.smt.printCommands] (declare-datatypes ((smti_0 0)) (((|smti_1|) (|smti_2|) (|smti_3|) (|smti_4|)))) 

[auto.smt.printCommands] (declare-fun |smti_5| (|smti_0|) |Int|) 

[auto.smt.printCommands] (assert (! (|not| (|<=| 2500 (|smti_5| |smti_1|))) :named valid_fact_0)) 

[auto.smt.result] z3 says Sat, model:
    ((|define-fun| |valid_fact_0| () |Bool| (|not| (|<=| 2500 (|smti_5| |smti_1|)))) (|define-fun| |smti_5| ((|x!0| |smti_0|)) |Int| 0))
    stderr:

I have tried a couple of things like asking lean-auto to unfold Zone.MinArea1: auto u[Zone.MinArea1] but this leads to an error: lamTerm2STerm :: Unexpected head term Auto.Embedding.Lam.LamTerm.lam (.atom 1) (.app (.base (.nat)) (.base (.icst (.iofNat))) (.base (.ncst (.natVal 10000))))

Can you tell me how to complete this proof properly? Also, is it possible to get lean-auto to handle the cases part?

Feature request: configuration option to write generated SMT commands to a file instead of being displayed on stdout

For large SMT files, it would be helpful to not have a cluttered Lean Info window.

This would also allow the generated SMT file to be processed by other tools (including other solvers).

LT and LE for bitvectors

The following is a theorem, but worryingly, auto reported a counterexample.

theorem auto_bitvec_inequality_test (i j max : Std.BitVec 64)
  (h0 : i < max) (h1 : j <= max - i) (h2 : 0#64 < j) :
  (max - (i + j)) < (max - i) := by
  auto

From the generated SMT commands (pasted in at the end of this message), one can see that the issue was that < and <= were defined as uninterpreted SMT functions, and not in terms of bvult and bvule, as I had expected.

This was inconsistent with + and -, which were indeed resolved to bvadd and bvneg.

 (declare-sort |Empty| 0)
 (define-fun |nsub| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|>=| |x| |y|) (|-| |x| |y|) 0))
 (define-fun |itdiv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|div| |x| |y|) (|-| (|div| (|-| |x|) |y|)))))
 (define-fun |itmod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|mod| |x| |y|) (|-| (|mod| (|-| |x|) |y|)))))
 (define-fun |iediv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) 0 (|div| |x| |y|)))
 (define-fun |iemod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|mod| |x| |y|)))
 (declare-fun |smti_0| () (_ BitVec 64))
 (declare-fun |smti_1| () (_ BitVec 64))
 (declare-fun |smti_2| ((_ BitVec 64) (_ BitVec 64)) |Bool|)
 (assert (! (|smti_2| |smti_0| |smti_1|) :named valid_fact_0))
 (declare-fun |smti_3| () (_ BitVec 64))
 (declare-fun |smti_4| ((_ BitVec 64) (_ BitVec 64)) |Bool|)
 (assert (! (|smti_4| |smti_3| (|bvadd| |smti_1| (|bvneg| |smti_0|))) :named valid_fact_1))
 (assert (! (|smti_2| #b0000000000000000000000000000000000000000000000000000000000000000 |smti_3|) :named valid_fact_2))
 (assert (! (|not| (|smti_2| (|bvadd| |smti_1| (|bvneg| (|bvadd| |smti_0| |smti_3|))) (|bvadd| |smti_1| (|bvneg| |smti_0|)))) :named valid_fact_3))
 
[auto.smt.result] z3 says Sat, model:
    ((|define-fun| |smti_0| () (|_| |BitVec| 64) 0) (|define-fun| |smti_3| () (|_| |BitVec| 64) 18446744073709549568) (|define-fun| |valid_fact_3| () |Bool| (|let| ((|a!1| (|smti_2| (|bvadd| |smti_1| (|bvneg| (|bvadd| |smti_0| |smti_3|))) (|bvadd| |smti_1| (|bvneg| |smti_0|))))) (|not| |a!1|))) (|define-fun| |valid_fact_2| () |Bool| (|smti_2| 0 |smti_3|)) (|define-fun| |valid_fact_1| () |Bool| (|smti_4| |smti_3| (|bvadd| |smti_1| (|bvneg| |smti_0|)))) (|define-fun| |valid_fact_0| () |Bool| (|smti_2| |smti_0| |smti_1|)) (|define-fun| |smti_1| () (|_| |BitVec| 64) 262160) (|define-fun| |smti_4| ((|x!0| (|_| |BitVec| 64)) (|x!1| (|_| |BitVec| 64))) |Bool| |true|) (|define-fun| |smti_2| ((|x!0| (|_| |BitVec| 64)) (|x!1| (|_| |BitVec| 64))) |Bool| (|ite| (|and| (|=| |x!0| 264208) (|=| |x!1| 262160)) |false| |true|)))

Bug in the translation of `Std.BitVec.not`

Here is an example to illustrate the issue:

import Std.Data.BitVec
import Auto

set_option auto.smt true
set_option auto.smt.trust true
set_option trace.auto.smt.printCommands true
set_option trace.auto.smt.result true

open Std.BitVec

theorem bvnot_auto_test (x : Std.BitVec 4) :
  x + (Std.BitVec.not x) = 0xF#4 := by
  auto

The error message is:

[auto.smt.result] SMT invocation failed with lamTerm2STerm :: The arity of Auto.Embedding.Lam.LamBaseTerm.bvcst (.bvnot 4) is not 1

automate test suite

I need make test or equivalent in the repo, and also the github CI should run the tests.

Lean Auto fails to translate some type classes

Hey! I was playing with lean auto and found some wired bug involving simple type classes. Lean auto sometimes fails to translate typeclasses which depend on arbitrary types. For instance:

import Auto.Tactic

set_option auto.smt true
set_option auto.smt.trust true
set_option trace.auto.smt.printCommands true
set_option trace.auto.smt.result true

class foo (t : Type) where
  le : t -> t -> Prop

example [foo Nat] (x y : Nat) : foo.le x y := by
  auto -- calls smt solver, but fails to find a proof

example {α : Type} [foo α] (x y : α) : foo.le x y := by
  auto -- fails to translate (and doen't even call the solver) with the following error: 
 /-
 getSexp :: Malformed (prefix of) input (
  ;; universe for smti_1:
  ;;   smti_1!val!1 smti_1!val!0 
  ;; -----------
  ;; definitions for universe elements:
  (declare-fun smti_1!val!1 () smti_1)
  (declare-fun smti_1!val!0 () smti_1)
  ;; cardinality constraint:
  (forall ((x smti_1)) (or (= x smti_1!val!1) (= x smti_1!val!0)))
  ;; -----------
  (define-fun valid_fact_0 () Bool
    (not (smti_3 smti_0 smti_2)))
  (define-fun smti_2 () smti_1
    smti_1!val!1)
  (define-fun smti_0 () smti_1
    smti_1!val!0)
  (define-fun smti_3 ((x!0 smti_1) (x!1 smti_1)) Bool
    false)
)
 -/

Could you help me with this issue? Is it a fundamental limitation or just a bug?

Feature request: define SMT functions that match Lean names whenever possible for ease of debugging & readability

This would also make it easier for the user to interpret counterexamples in terms of identifiers in their conjecture.

Feature requests: Meaningful UNSAT cores and Models in Lean

Is it possible / meaningful to get UNSAT cores or models in a format that is meaningful in Lean?

We are representing domain models and some queries against those models. We would (if possible) like to get witnesses to proof failures expressed in a meaningful way back into Lean, and to get UNSAT cores that have meaning in Lean.

The underlying idea is that we want users to be able to "ask questions" about domain models by checking whether partially instantiated ground terms are conformant with the model and then presenting the results back in a meaningful way.

Mostly I am curious whether this is on your roadmap for any point in the future.

Faulty translation of `Nat` existentials into SMT

import Auto
set_option auto.smt true
set_option auto.smt.trust true

set_option trace.auto.smt.printCommands true
set_option trace.auto.smt.result true

example : false = true := by
  auto -- z3 says Sat

example : ∃ (n : Int), false = true := by
  auto -- z3 says Sat

example : ∃ (n : Nat), false = true := by
  auto -- z3 says Unsat
  /-
  [auto.smt.printCommands] (assert (! (|not| (exists ((|smtd_0| |Int|)) (|=>| (|>=| |smtd_0| 0) (|=| |false| |true|)))) :named valid_fact_0))
  [auto.smt.result] z3 says Unsat, unsat core:
      (|valid_fact_0|)
  -/

In the translation of exists, there should be a conjunction rather than implication. As far as I can tell, the bug is due to lamQuantified2STerm.

Feature request: SMT trigger annotations

When sending quantified assumptions through the SMT backend, quantifier instantiation can greatly affect performance (and even completeness). Boogie and Dafny are example tools that have made heavy use of trigger annotations to guide the solver (Z3, generally) to choose useful instantiations. I suspect it would be possible to add some sort of mechanism for writing annotations on quantified terms that don't have semantic significance within Lean but could be used to control trigger annotations in SMT-Lib formulas. (Other provers could probably just ignore them.)

Zero-width bitvectors

Zero-width bitvectors are not supported by the Fixed-Size Bitvector theory in SMT, but Std.BitVec allows them. This can cause interesting issues. Here is a motivating example.

The following is a theorem in Lean:
∀ (x : BitVec 8), x ++ (Std.BitVec.ofNat 0 0) = x

When auto is used to prove the theorem above, (Std.BitVec.ofNat 0 0) is translated to a 1-bit SMT bitvector, and a sort mismatch error is obtained from the solver (log below).

Perhaps a simplification rule is needed to tackle this case?

[auto.smt.printCommands] (declare-sort |Empty| 0)
[auto.smt.printCommands] (define-fun |nsub| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|>=| |x| |y|) (|-| |x| |y|) 0))
[auto.smt.printCommands] (define-fun |itdiv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|div| |x| |y|) (|-| (|div| (|-| |x|) |y|)))))
[auto.smt.printCommands] (define-fun |itmod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|ite| (|>=| |x| 0) (|mod| |x| |y|) (|-| (|mod| (|-| |x|) |y|)))))
[auto.smt.printCommands] (define-fun |iediv| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) 0 (|div| |x| |y|)))
[auto.smt.printCommands] (define-fun |iemod| ((|x| |Int|) (|y| |Int|)) |Int| (|ite| (|=| |y| 0) |x| (|mod| |x| |y|)))
[auto.smt.printCommands] (declare-fun |smti_0| () (_ BitVec 8))
[auto.smt.printCommands] (assert (! (|not| (|=| (|concat| |smti_0| #b0) |smti_0|)) :named valid_fact_0))
[auto.smt.result] z3 produces unexpected check-sat response
     (|error| "line 10 column 55: Sorts (_ BitVec 9) and (_ BitVec 8) are incompatible")

Feature Request: Allow users to map suitable Lean functions to corresponding SMT functions

Solvers are sensitive to encoding, and for certain problems, it would help if the user could dictate whether the Lean functions in their conjecture should be mapped to a corresponding SMT function. Here is an illustrative example:

import Std.Data.BitVec
import Auto

set_option auto.smt true
set_option auto.smt.trust true
set_option trace.auto.smt.printCommands true
set_option trace.auto.smt.result true
set_option trace.auto.smt.model true
set_option trace.auto.smt.proof false

def prop (a1 a2 b1 b2 : Std.BitVec 64) : Bool :=
  Std.BitVec.ule (a2 - b1) (b2 - b1) &&
  Std.BitVec.ule (a1 - b1) (a2 - b1)

set_option auto.smt.timeout 200 -- seconds
set_option trace.auto.smt.printCommands true in
set_option auto.smt.savepath "/tmp/prop_transitive.smt2" in
theorem prop_transitive
  (h1 : prop a1 a2 b1 b2)
  (h2 : prop b1 b2 c1 c2) :
  prop a1 a2 c1 c2 := by
  revert h1 h2
  auto d[prop] --- times out

Auto inlines prop, which adversely affects the SMT solving time. On the other hand, z3 can solve this problem in ~10s in my hand-written SMT file below, where prop appears as an SMT function.

(set-logic ALL)
(set-option :interactive-mode true)

(declare-const a1 (_ BitVec 64))
(declare-const a2 (_ BitVec 64))
(declare-const b1 (_ BitVec 64))
(declare-const b2 (_ BitVec 64))
(declare-const c1 (_ BitVec 64))
(declare-const c2 (_ BitVec 64))

(define-fun prop ((a1 (_ BitVec 64))
                  (a2 (_ BitVec 64))
                  (b1 (_ BitVec 64))
                  (b2 (_ BitVec 64)))
  Bool  
  (and (bvule (bvadd a2 (bvneg b1)) (bvadd b2 (bvneg b1)))
       (bvule (bvadd a1 (bvneg b1)) (bvadd a2 (bvneg b1)))))



;; Preferred solver: z3 (~10s)

;; prop_transitive
(assert (not
         (=> (and (prop a1 a2 b1 b2)
                  (prop b1 b2 c1 c2))
             (prop a1 a2 c1 c2))))

(check-sat)

leanprover-community / lean-auto Goto Github PK

lean-auto's People

Contributors

Stargazers

Watchers

Forkers

lean-auto's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs