juodaspaulius / clafer Goto Github PK

Failed to compile the following test case model

a
  xor b
    c --> d
    d
assert[always a.b.c => a.b.d]

The error message:
clafer: AlloyLtl.genPExp': No pattern match for IClaferId {_modName = "", _sident = "~@r_c0_c", _isTop = True, _binding = Nothing}
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_clafertransitions01.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

once key word is not supported in the constraints. For e.g.in tmp_before01.cfr

a
  b ?
    c ?
    d ?
    [once c before d]

Error message:

Parse failed at line 7 column 11...
syntax error at line 7 before before d ] }
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_before01.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

Consider the following example:

a ?
  b ?
assert [ G ( some b => some a )

This assertion fails. It should not. Michał was planning to create a test case, but most likely he has not managed.

Right now the hierarchy constraint just handles the following: each instance of the clafer b can appear (for the first time) at most once in history, and under an instance of clafer a.

We are missing the following constraint: each instance of a clafer b must appear at least once in time in relation to its parent a. This will eliminate the zombie instances that appear with no relation to time. The formulation that we experimented with is:

total[~(r_c0_a.State), c0_a] // for a
total[~(r_cX_b.State),cX_b] // for b

AP1. It would be cool to add this constraint for each nested clafer (also for those nested under root).
AP2. It would also be very nice to include comments in the generated code (like the to bold lines above could be generated next to their respective constraints). This will improve the debugging and maintainability of the generator.
AP3. Add a regression test case (the one with which the issue starts) and perhaps one more, which is stronger, something like that:

a?
  b?
assert [ G ( all b1: b | b.parent.b = b1 ) ]

Checks

1. test case is ok ? Yes
2. Compiles ? Yes
3. De-sugars correctly ? yes
4. Compiles in Alloy ? Yes
5. Instances in alloy are ok? no instance found.

Model:

a
  xor b
   c
   d
   e
   [always c between d and e ]
   [sometime d]
   [sometime c]
   [sometime e]
assert[always c between d and e]

When tested the following alloy code generated for the above model, it fails to generate a sample instance. Interstingly, if I comment [sometime e] constraint, it generates an instance without e.
This case is similar to after and until pattern, where it also fails to generate a instance with suitable start and end scopes.

/*
All clafers: 5 | Abstract: 0 | Concrete: 5 | References: 0
Constraints: 5
Goals: 0
Global scope: 1..1
Can skip resolver: yes
*/

open util/ordering[Time]
pred show {}
run show for 10

sig Time {loop: lone Time}
fact Loop {loop in last->Time}
fun timeLoop: Time -> Time { Time <: next + loop }

one sig root
{ r_c0_a : c0_a -> Time }
{ all t : Time | one r_c0_a.t }

sig c0_a
{ r_c0_b : c0_b -> Time }
{ all t : Time | one r_c0_b.t }

sig c0_b
{ r_c0_c : c0_c -> Time
, r_c0_d : c0_d -> Time
, r_c0_e : c0_e -> Time }
{ all t : Time | lone r_c0_c.t && lone r_c0_d.t && lone r_c0_e.t
  all t : Time | let children = (r_c0_c.t + r_c0_d.t + r_c0_e.t) | one children
  let local_next = (this.(c0_a.@r_c0_b)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some loop and all t':t.*timeLoop | (((some this.@r_c0_d.t') && (no this.@r_c0_e.t')) && ((some t'':t'.*timeLoop | some this.@r_c0_e.t''))) => ((some t'':t'.*timeLoop | some this.@r_c0_e.t'' and ( all t''':t'.*timeLoop & ^timeLoop.t''|some this.@r_c0_c.t'''))))
  let local_next = (this.(c0_a.@r_c0_b)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some t':t.*timeLoop | some this.@r_c0_d.t')
  let local_next = (this.(c0_a.@r_c0_b)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some t':t.*timeLoop | some this.@r_c0_c.t')
  let local_next = (this.(c0_a.@r_c0_b)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some t':t.*timeLoop | some this.@r_c0_e.t') }

sig c0_c
{}
{ all t : Time | one r_c0_c.t }


sig c0_d
{}
{ all t : Time | one r_c0_d.t }


sig c0_e
{}
{ all t : Time | one r_c0_e.t }





assert assertOnLine_12 { one t : first <: Time | (some loop and all t':t.*timeLoop | (((some (@r_c0_a.t'.@r_c0_b.t').@r_c0_d.t') && (no (@r_c0_a.t'.@r_c0_b.t').@r_c0_e.t')) && ((some t'':t'.*timeLoop | some (@r_c0_a.t''.@r_c0_b.t'').@r_c0_e.t''))) => ((some t'':t'.*timeLoop | some (@r_c0_a.t''.@r_c0_b.t'').@r_c0_e.t'' and ( all t''':t'.*timeLoop & ^timeLoop.t''|some (@r_c0_a.t'''.@r_c0_b.t''').@r_c0_c.t''')))) }
check assertOnLine_12 for 0

Let's say we have a reference c1 in a model:

abstract B
final A
  d : B -> D
D

Every reference is reified, so Alloy generator creates an additional signature sig c0_d to encode the reference. In case the reference is not final, we need to attach State set to the reference relation. A small dilemma is whether to attach it to relation in reified class or in the parent class?.. So AlloyLtl generator can translate above into:

Case 1:

sig c0_A
{ r_c0_d : c0_d -> State } {...}

sig c0_d
{ c0_d_ref : c0_D } {...}

or Case 2:

sig c0_A
{ r_c0_d : c0_d } {...}

sig c0_d
{ c0_d_ref : c0_D -> State } {...}

Any input on this?

The following model fails to generate an instance.

xor a
  b
  c
  [eventually c ]
  [initially b]

Output:
Executing "Run show for 10"
Solver=minisat(jni) Bitwidth=0 MaxSeq=0 SkolemDepth=1 Symmetry=20
39767 vars. 2230 primary vars. 36915 clauses. 141ms.
No instance found. Predicate may be inconsistent. 2ms.

Alloy Code

The constraint language is not fully supported in assertions.

Test case:
a ?
  xor b 1
    c ?
      d ?
    [never d after c ]

assert [all a1 : a | all b1 : a1.b | never b1.d after b1.c]

In the above test case the assertion fails with the following error message
Error message:

clafer: AlloyLtl.genPExp': No pattern match for IClaferId {_modName = "", _sident = "a1", _isTop = True, _binding = Nothing}
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_after01.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

The following model fails to compile.

a
  xor b ?
    c -->> d
    d

assert[a.b.c => F a.b.d]

The error message:
Compile error at line 8 column 23...
Name resolver: 'd' not found within paths:c0_a.c0_b
in context of '"c0_b"'
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_clafertransitions02.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]

Failed to compile Power Window Total transitions model:
Please take a look at the model which is committed to repository under positive folder or you can also find at: Power Window Total.
The following are the error messages, when it tried to compile using clafer LTL compiler.

1. Error 1: The following let constraint failed to compile:

[no (req, endOfTravel, objectDetected) => (let oldState = State | next (State = oldState))]

Error message:
Parse failed at line 77 column 69...
syntax error at line 77 before | next ( State
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_powerwindow_total.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

1. Error 2: The following constraint fails to compile

[ (req=down || no req) => always (movingDown until endOfTravel) ]

Error message:
Parse failed at line 112 column 39...
syntax error at line 112 before always ( movingDown until
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_powerwindow_total.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

However, the same constraint compiles if we change always to G like the following.

[ (req=down || no req) => G (movingDown until endOfTravel) ]

1. Error 3:
   Finally if we comment the above two error constraints lines, we get the following error:

clafer: AlloyLtl.genPExp': No pattern match for IClaferId {_modName = "", _sident = "ref", _isTop = True, _binding = Nothing}
[Finished in 0.4s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_powerwindow_total.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

The following model fails to compile in alloy instance generator.

final xor a
  initial b
  c
  [eventually c ]

  assert[a.c after a.b]

The following is the error message in alloy analyser 4.2:
Starting the solver...
A type error has occured:
This expression failed to be typechecked

alloy code

With the assertion enabled, the following is the error message:
Parse failed at line 7 column 12...
syntax error at line 7 before after a . b
[Finished in 0.0s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_combinedEGX02.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

The TLS handshake protocol failed to compile.
The model can be found in positive testcase repository or can also be found at: TLS Handshake Protocol

The error message:
All clafers: 69 | Abstract: 10 | Concrete: 59 | References: 6
Constraints: 52
Goals: 0
Global scope: 1..*
Can skip resolver: no

clafer: AlloyLtl.genPExp': No pattern match for IClaferId {_modName = "", _sident = "ref", _isTop = True, _binding = Nothing}
[Finished in 0.5s with exit code 1]
[cmd: ['/home/clafertools/.config/sublime-text-3/Packages/Clafer Tools/clafer', '/home/clafertools/behavioral-0.4.0/clafer/test/positive/tmp_TLSHandshakeProtocol_distributed.cfr', '-m=alloyltl', '-m=clafer', '-m=html', '--add-comments', '--self-contained']]
[dir: /home/clafertools/behavioral-0.4.0/clafer/test/positive]
[path: .:/opt/ghc/7.8.3/bin:/opt/cabal/1.20/bin:/home/clafertools/.cabal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games]

Checks

1. test case is ok ? Yes
2. Compiles ? Yes
3. De-sugars correctly ? yes
4. Compiles in Alloy ? Yes
5. Instances in alloy are ok?

a
  xor b
  c
  d
  [sometime c before d]
assert[never d before c]

The assertion failed and counter example found for the above test case.
The generated Xml instance saved at gist: Sample instance and counter example Xml

test case is ok ? Yes
Compiles ? Yes
De-sugars correctly ? Yes
Compiles in Alloy ? Yes.
Instances in alloy are ok? Yes
The clafer model looks like as follows,

a
 xor b
  c
  d
  e
  [always c after d until e ]
assert [never a.b.c after a.b.d until a.b.e ]

I could not be able to produce any valid instance where at least one occurence of d is present. Some how, it always produce instance with either c or e, but I have not seen in any of the 25-30 instance I have tried.
Moreover, even the assertion failed to find a counter example, which is kind of fishy. Therefore need to be explored further.

The test case compiles correctly by bclafer compiler. But the generated alloy code fails to compile in alloy analyser.

Starting the solver...
A type error occurred.
This expression failed to be typechecked

Here is the alloy code:

/*
All clafers: 3 | Abstract: 0 | Concrete: 3 | References: 0
Constraints: 2
Goals: 0
Global scope: 1..1
Can skip resolver: yes
*/

open util/ordering[Time]
pred show {}
run show for 10

sig Time {loop: lone Time}
fact Loop {loop in last->Time}
fun timeLoop: Time -> Time { Time <: next + loop }

one sig root
{ r_c0_a : one c0_a }

one sig c0_a
{ r_c0_b : c0_b -> Time }
{ all t : Time | lone r_c0_b.t
  let local_next = (this.(root.@r_c0_a)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some t':t.*timeLoop | some (this.@r_c0_b.t').@r_c0_c.t') }

sig c0_b
{ r_c0_c : c0_c -> Time }
{ all t : Time | one r_c0_b.t
  all t : Time | lone r_c0_c.t }

sig c0_c
{}
{ all t : Time | one r_c0_c.t }





assert assertOnLine_8 { one t : first <: Time | (some loop and all t':t.*timeLoop | no (@r_c0_a.@r_c0_b.t').@r_c0_c.t') }
check assertOnLine_8 for 10

Checks

1. test case is ok ? Yes
2. Compiles ? Yes
3. De-sugars correctly ? No
4. Compiles in Alloy ? Yes
5. Instances checked in alloy? Yes.

a
  b ?
    c ?
  [c] // globally c in the context of b
This should desugars to: [globally (this . c0_b => this . c0_b . c0_c) ].
But it actually desugars to: [some this . c0_b . c0_c]

I tested by adding an assertion: assert[never (a.b && not a.b.c) ] and I found a counter example where in the
Time 0: root -> c0_a -> c0_b2 ->c0_c0 plus some orphan instances of b and c.
Time 1: root -> c0_a -> c0_b2 plus some orphan instances of b and c. (which is a voilation)

The following models compiles, desugars correctly, but failed to generate a valid instance.

xor a
   b 
   c 
   [b => eventually c]
assert[always (a.b => eventually a.c)]

I tried many times, but I get the following message and no valid instance can be generated.

Executing "Check assertOnLine_8 for 10"
Solver=minisat(jni) Bitwidth=0 MaxSeq=0 SkolemDepth=1 Symmetry=20
51855 vars. 2230 primary vars. 62664 clauses. 130ms.
No counterexample found. Assertion may be valid. 7ms.

Checks

1. test case is ok ? Yes, added assertion
2. Compiles ? Yes
3. De-sugars correctly ? Yes ( correctly desugars according to property pattern)
4. Compiles in Alloy ? Yes
5. Instances checked in alloy? Yes.

When I checked the compiled alloy code with an assertion, I found counter cases. Two sample alloy counter examples (xml files) are attached to the issue, so that the same instance can run alloy instance visuavaliser.

The sample Xml instances are: instance 1 and instance 2

In the sample instance for afteruntil01 constraint, I found an interesting instance, which is shown in the figure.
When projected over time, c0_a0 is connected to a valid instance, but has no parent.
Is this a bug? I have saved the xml for the instance, which I can not attach here.
The solver used in the Alloy generator is : Minisat with unsat core.

Sample Instance Xml

Alloy Code:

open util/ordering[Time]
pred show {}
run show for 10`

sig Time {loop: lone Time}
fact Loop {loop in last->Time}
fun timeLoop: Time -> Time { Time <: next + loop }

one sig root
{ r_c0_a : c0_a -> Time }
{ all t : Time | one r_c0_a.t }

sig c0_a
{ r_c0_b : c0_b -> Time }
{ all t : Time | one r_c0_b.t }

sig c0_b
{ r_c0_c : c0_c -> Time
, r_c0_d : c0_d -> Time
, r_c0_e : c0_e -> Time }
{ all t : Time | lone r_c0_c.t && lone r_c0_d.t && lone r_c0_e.t
  all t : Time | let children = (r_c0_c.t + r_c0_d.t + r_c0_e.t) | one children
  let local_next = (this.(c0_a.@r_c0_b)) <: next  | one t : Time | one t <: local_next and no local_next :> t and 
  (some loop and all t':t.*timeLoop | ((some this.@r_c0_d.t') && (no this.@r_c0_e.t')) => (((some loop and all t'':t'.*timeLoop | some this.@r_c0_c.t'')) || ((some t'':t'.*timeLoop | some this.@r_c0_e.t'' and ( all t''':t'.*timeLoop & ^timeLoop.t''|some this.@r_c0_c.t'''))))) }

sig c0_c
{}
{ all t : Time | one r_c0_c.t }


sig c0_d
{}
{ all t : Time | one r_c0_d.t }


sig c0_e
{}
{ all t : Time | one r_c0_e.t }

assert assertOnLine_10 { one t : first <: Time | (some loop and all t':t.*timeLoop | ((some (@r_c0_a.t'.@r_c0_b.t').@r_c0_d.t') && (no (@r_c0_a.t'.@r_c0_b.t').@r_c0_e.t')) => (((some loop and all t'':t'.*timeLoop | some (@r_c0_a.t''.@r_c0_b.t'').@r_c0_c.t'')) || ((some t'':t'.*timeLoop | some (@r_c0_a.t''.@r_c0_b.t'').@r_c0_e.t'' and ( all t''':t'.*timeLoop & ^timeLoop.t''|some (@r_c0_a.t'''.@r_c0_b.t''').@r_c0_c.t'''))))) }
check assertOnLine_10 for 10

Update on the power window model.

I am using this power window model below and ran through the compiler. Unfortunately I am not getting any instances. Unsat core tool is not very helpful.

To me, none of the Clafer constraint appear to over-constrain the model.

Clafer Model:
https://gist.github.com/juodaspaulius/a0f2df51e5b49dd24249#file-power_window_full-cfr

Generated Alloy model:
https://gist.github.com/juodaspaulius/542dc6cc2b62514768c7#file-power_window_full_15-als

Running with max scope of 15 takes 76 seconds on my machine.

Executing "Run show for 15"
   Solver=minisatprover(jni) Bitwidth=0 MaxSeq=0 SkolemDepth=1 Symmetry=20
   18410093 vars. 35805 primary vars. 34907246 clauses. 75917ms.
   No instance found. Predicate may be inconsistent. 40975ms.
   . contains 6 top-level formulas. 39ms.

The following part of the traffic light example fails to generate an instance.

xor TrafficLight_4 1
   xor on
      normal
      flashing
   off
   [initially off]
   // a constraint that 'remembers' that we left from normal
   [G (on.normal && X !on) => X(!on W on.normal)]  // (*)
   // a constraint that 'remembers' that we left from flashing
   [G (on.flashing && X !on) => X(!on W on.flashing)] 

It compiles correctly in alloy analyser, but fails to generate an instance.

The following model generates a code that fails in alloy analyser.
Here is the model.

final a
   b ?
   [F b]
assert[never b]

Error message:
Starting the solver...

A type error has occurred.
This expression failed to be type-checked

My guess is that, the error happens when a constraint is nested under a final type.I have seen the same error in other test case (tmp_eventually01.cfr).