Handling names / binders about quotedpatternmatchingproof

[radeusgd]

I have found an interesting alternative to autosubst that seems to be a more flexible framework, namely https://github.com/coq-community/dblib
It was recommended in some INRIA lecture (https://xavierleroy.org/mpri/2-4/mechanization.pdf) and seems to be more actively developed.
I still have to read more about it, but seems like a promising alternative that may be worth trying over just handling this manually.

[radeusgd]

In the next steps I will try to use the library autosubst which helps with proving substitution lemmas for calculi using DeBruijn indices.

As I'm using Coq 8.9:

The Coq Proof Assistant, version 8.9.1 (June 2019)
compiled on Jun 27 2019 16:37:47 with OCaml 4.05.0

I decided to use the coq86-devel branch which adds support for newer Coq releases and seems to be stable enough.
Currently I'm using autosubst from this very commit: coq-community/autosubst@fa6ef30

[radeusgd]

Given #15 and #16 it is likely that autosubst will not be suitable for this project, so we may need to look for other solutions.

First thing I would like to try is to find some calculus with pattern matching that has been formalized mechanically and see how they handle the multiple binders in patterns issue.

Possible next steps:

1. The conceptually simplest solution I think could be to simply use "classical" names for binders and manually write a capture avoiding substitution function. This function is however non-trivial to write and reason about.

2. Another possibility is to use the classical named approach and add an assumption that there is no shadowing, ie. all names are unique, so we don't have to worry about capture. I'm not 100% sure that this assumption is possible to express easily and if it will be easy to reason with that approach.

3. One could investigate using DeBruijn indices (without any helper libraries) and capturing multiple binders in a single term, something that seems to be not possible in autosubst but should be possible in general.

[radeusgd]

This website may be useful but I have to read furhter its content to see if it proposes anything new: https://github.com/coq/coq/wiki/BindingRepresentation

I will most likely read it tomorrow.

[radeusgd]

The dblib library seems quite promising as the user has to manually define a traverse function which defines how the binders are handled so this allows us to define the 2 binders for pattern matching an App.

There is still the issue that it wants the Var constructor similarly to autosubst (see #15), but in this case, the library supports having 2 'sorts' - terms and values so that in a substitution variables inside of a term are replace by values. We could likely define terms to be our typed terms and values to be the untyped terms and achieve what is needed.

[radeusgd]

After being able to prepare an example with Patterns and proving preservation of a simpler calculus with Quotes and Splices, it seems DbLib will be a good choice for this project.

[radeusgd]

I've found a big issue - we have a binder in PVar and we cannot shift it when the term is shifted, because it is not 'touched' by traverse.

First I tried to 'hack' the traverse method for patterns, eg.

Definition traverse_pattern f l pat :=
match pat with
| PVar x => match f l x with
                   | VAR y => PVar y
                   | _ => PVar x
| _ => pat
end.

This however breaks the lemma

Lemma traverse_functorial_value_value:
  forall f g (t : pattrn) l,
    traverse g l (traverse f l t) = traverse (fun l x => traverse g l (f l x)) l t

ie. if we have f = fun x => (App (VAR 0) (VAR 1)); g = fun x => VAR 1; t = PVar 0
then

traverse f l t --> PVar 0 // nothing happened because the result did not match VAR
traverse g l (PVar 0) --> PVar 1
// but
traverse (fun l x => traverse g l (f l x)) l t --> PVar 0
//inside of traverse we call (f l (VAR 0)) --> (App (VAR 0) (VAR 1))
// and then we have g l (App (VAR 0) (VAR 1)) which yields (App (VAR 1) (VAR 1))
// finally this is matched agains VAR and as it doesn't match, the original value, PVar 0 is kept

[radeusgd]

The second approach was to just redefine PVar to take a term.

So we'd get:

Inductive pattrn :=
| PNatLit (n : nat)
| PVar (x : term)
(* | PBindVar (T : type) - we exclude this pattern as it is useless in non-nested pattern matching *)
| PBindApp (T1 T2 : type)
| PBindUnlift
| PBindLam (T : type).

and make sure in typing that only PVar (VAR x) can ever typecheck...
This is a bit ugly but seems to solve the issue.
Until it doesn't... I couldn't prove the basic traverse lemmas (they usually are proven automatically by the library, but the library's tactics failed and I tried helping it manually to no avail).

[radeusgd]

Right now I'm still verifying if we even need this shift (I'm 90% we do, but I want to make sure).
I wrote a separate test.v file where I defined a complex expression in the calculus (a function taking a block of code, checking if it matches sum $x $y and if so, returning sum $y $x - swapping the arguments).

If I verify this requirement positively, I will create separate syntactic categories for each pattern, possibly merging PNat and PVar into one, so the terms would be extended with following cases:

| PMatchConst (e: typedterm) (pat: term) (s: typedterm) (f: typedterm)
| PMatchApp (e: typedterm) (T1 T2 : type) (s: typedterm) (f: typedterm)
| PMatchUnlift (e: typedterm) (s: typedterm) (f: typedterm)
| PMatchLam (e: typedterm) (T : type) (s: typedterm) (f: typedterm)

then PMatchConst would have two typing rules:

| T_Pat_Nat : forall G t1 ts tf T n,
    G ⊢(L0) t1 ∈ □TNat ->
    G ⊢(L0) ts ∈ T ->
    G ⊢(L0) tf ∈ T ->
    G ⊢(L0) (PMatchConst t1 (Nat n) ts tf : T) ∈ T
| T_Pat_Var : forall G T1 t1 ts tf T x,
    G ⊢(L0) t1 ∈ □T1 ->
    lookup x G = Some (L1, T1) ->
    G ⊢(L0) ts ∈ T ->
    G ⊢(L0) tf ∈ T ->
    G ⊢(L0) (PMatchConst t1 (VAR x) ts tf : T) ∈ T

Other cases would have analogous typing rules.

The downside of this approach is that our syntax allows syntactically incorrect programs (like PMatchConst ? (App (Nat 0) (Nat 1)) ? ?). However such programs will fail to typecheck anyway.

If necessary we can also define a Prop - is_syntactically_valid and prove that only syntactically valid programs typecheck.

[radeusgd]

I thought that possibly instead of shifting PVar, we may shift up the variables that are checked in the pattern match, this sounded like a nice and simple potential solution, although my intuition told me there's something wrong with it.

As seen in 6c99782 it is indeed wrong - we absolutely have to shift PVar as not doing so breaks Preservation.

[liufengyun]

I think the solution you suggested to flatten/uniform the syntactic structure makes sense to me.

Ensuring well-formedness in typing rules is common, it's even used in STLC. So it's not a problem.

Conceptually & technically, I think it might be easier to keep PNat and PVar separate. But let your intuition guides you and adapt from experiments.

Don't worry much about the zigzag, it's part of the research process. If you can extract some lessons from this experiment that would be great, which will be valuable to all people doing mechanization in PL.

[radeusgd]

I have rewritten the syntactic structure to the flat version (TODO: update types and semantics, and obviously, proofs).

It didn't come without problems however - for some strange reason, the lemmas required by the library weren't automatically proven by the tactics provided for this use by the library, it seems that the MatchVar case which has 4 nested terms (3 terms for the matched expression, success and failure + 1 term for the dummy var term) was too much and likely some tactics reached their depth limit.
I was trying to redesign the structure to make it more approachable for the tactics, but to no avail. Finally I just tried to manually finish-up what the tactic failed with, basing on the tactic's definition as a hint of how I should proceed, fortunately the missing parts of the proof were quite easy to finish.

Unfortunately the extended syntax makes the 'traverse relative' lemma take a lot of time to prove (up to almost 2 minutes!), likely because the complexity has grown too much. I think this is tolerable though.

[radeusgd]

Having managed to encode (with some issues noted in the previous comment) the calculus in the new way and to prove Progress for it I think for now the naming handling issue is mostly solved.

[liufengyun]

Great to hear that, congrats for the progress !