project-oak / silveroak Goto Github PK
View Code? Open in Web Editor NEWFormal specification and verification of hardware, especially for security and privacy.
License: Apache License 2.0
Formal specification and verification of hardware, especially for security and privacy.
License: Apache License 2.0
In the declaration:
Inductive Kind : Type :=
| Void : Kind (* An empty type *)
| Bit : Kind (* A single wire *)
| BitVec : Kind -> nat -> Kind (* Vectors, possibly nested *)
| ExternalType : string -> Kind. (* An uninterpreted type *)
the constructor BitVec
was so named because originally it was a flat bit-vector with no nesting. Now it can be a nested vector of vectors. Rename it to Vec
to avoid confusion.
Work out how to define an adder-tree circuit recursively and show that we can evaluate it under simulation in Coq and also extract a netlist that can be simulated and implemented with the Xilinx Vivado FPGA tools.
This currently prevents Eval compute in (@some_kappa_arrow Combinatorial) (true)
style evaluation for Kappa built Arrows. Extraction and term manipulation doesn't have this issue.
Rename Cava.v
to CavaMonad.v
to allow CavaArrow.v
to exsit.
Allow for the import of both CavaMonad
and CavaArrow
in the same file.
Port the current code to use Coq modules.
Possible add a level of nesting in case we need things like Cava.Monad.Combinators
and Cava.Arrow.Combinators
.
Let's adopt a "coding style" for the generated SystemVerilog code where all vectors have decreasing indexes i.e. logic[7:0]
and never logic[0:7]
. This should help to avoid terrible mistakes and errors during the SystemVerilog production process in Haskell, and also conforms to the OpenTitan SystemVerilog style rules.
The current directory structure for Cava has grown organically but now I think we should move to something more organized. Currently Cava files are split across three directories: cava (the core system but this also has some examples that serve as light regression tests), cava-examples (which has examples) and a nandgame directory (a Cava version of the nand game done by Ben Laurie in the nand directory). Here's a proposal for a better directory structure. Let's discuss and iterate until we come to something that we can agree upon. Here's the current state of the proposal (which we can edit as we
discuss).
Key points:
Move to having everything in a single cava directory, with examples nested underneath and a top level README.md that has a good overall explanation of the Cava project and which is well linked to examples, documentation etc.
To build the system should just require installation of pre-requisites and then typing make clean all
or just make
. Regression tests. could be run by make test
(and perhaps automatically run as part of make all
).
The CI system should make sure to track enough of the code under Cava.
Right now it is just a few us doing experimental evaluation but at some point we should have a way to incorporating contributed circuits and proofs by others e.g. into a contrib
directory.
We need to think of where to put the code for the OpenTitan components and a process for using our generated circuits being incorporated into the OpenTitan flow.
A first cut at directory structure inside the cava
directory:
cava
with contains a subdirectory Cava: the directory that contains the core Cava system and "standard" library circuits and proofs. Similar to a cleaned up version of the current cava directory, but with examples etc. removed. Also, this is where the current arrow code will end up (hopefully replacing the monadic version).cava-examples
plus any examples we want to pick from the current cava
directory. Include a process for generating SystemVerilog for simulation and synthesis.Makefile
at the top level which builds everything with rules for all
and clean
. Each sub-directory should also have a Makefile
with all
and clean
rules for its components.So from the top-level cava directory:
$ ls
cava contrib examples Makefile opentitan tests
The story for building this project is a little jumbled right now -- lots of different _CoqProject
files, for instance, and some directories require you to run make
in other directories first to build dependencies. I think it would be smoother if we had:
_CoqProject
file, no other _CoqProject
smake
target that makes it possible to automatically populate the _CoqProject
file (maybe using git ls-files
or similar)make
targets for different logical subparts of the project (e.g. make arrow-lib
, make monad-examples
)This is not something I'm intending on doing at this exact moment, and it should probably be prioritized behind some other things we're all busy with. But I figured I'd open an issue so that whenever one of us does have a moment, we'll have already discussed it and figured out what makes the most sense!
Originally I had wanted to use negative net indices to temporarily borrow indices for looped back wires. I don't think we need to do this anymore, so it is better to change the net index type from Z
to N
since negative net numbers no longer make any sense.
Currently when building a constant vector e.g. VecLit [Gnd, Vcc, Gnd]
the emitted systemverilog outputs the value directly:
assign v1 = '{zero,one,zero};
We could instead output a literal:
assign v1 = 3'h2;
Prove:
Lemma bits_of_nat_sized: forall n bv, nat_to_bitvec_sized n (bitvec_to_nat bv) = bv.
Lemma nat_of_bits: forall v, bitvec_to_nat (nat_to_bitvec v) = v.
Lemma nat_of_bits_sized: forall (v : nat),
bitvec_to_nat (nat_to_bitvec_sized (N.size_nat (N.of_nat v)) v) = v.
Arrow conversion could be done either via another higherorder transform (arrow PHOAS representation) or following the current method but using an object suffix instead of an explicit environment. But the latter might prevent removing of free variables.
Use the Coq semantics of a circuit to computed the expected outputs and then generate a Verilator SystemVerilog test to ensure that the SystemVerilog semantics match the Coq semantics for the test cases. This should help to catch bugs where we inadvertently break the semantic model in Coq or the extraction process to SytemVerilog.
Currently, unless a blackbox primitive is explicitly instantiated, extraction is to a single module. This will lead to a lot of duplication in e.g. unrolled_forward_cipher_flat
from arrow-examples/Aes/unrolled_opentitan_cipher.v
where there is a deep hierarchy of modules. Haskell execution of writing a netlist file for unrolled_forward_cipher_flat
currently times out, this might be unrelated, but also could be due to large duplication of sub-expressions.
Revert to using Vector.t
for bit-vectors in the combinational and sequential interpretations, and our own Vec
type supporting indexing and slicing in the netlist representation.
The definition of Primitive has become overly restrictive for primitives that wouldn't naturally be written as input_shape -> output_shape
. We could
Nix has been more painful than first anticipated due to issues with MacOS (Catalina), Coq/Ocaml libraries (potentially installing themselves into the OCaml dir which for nix should be read only), and other misc issues.
Let's keep Nix around for CI (if we have a better solution please open an issue and link), but reduce its prominence otherwise.
A special case for muxing an array of buses as a stepping stone towards multi-level muxing of arrays with an arbitrary degree of indexing.
Device a stripped down simplified monad kernel to experiment with simplifying proofs.
Please can you ensure all the files you have added contain the license header?
For example, for Coq files:
(****************************************************************************)
(* Copyright 2020 The Project Oak Authors *)
(* *)
(* Licensed under the Apache License, Version 2.0 (the "License") *)
(* you may not use this file except in compliance with the License. *)
(* You may obtain a copy of the License at *)
(* *)
(* http://www.apache.org/licenses/LICENSE-2.0 *)
(* *)
(* Unless required by applicable law or agreed to in writing, software *)
(* distributed under the License is distributed on an "AS IS" BASIS, *)
(* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. *)
(* See the License for the specific language governing permissions and *)
(* limitations under the License. *)
(****************************************************************************)
This issue tracks transient build errors on CI. Whenever I notice one I'll add a note here.
Based on previous conversation with @blaxill, we need something like the monad combinational
for the arrow-based pipeline. This doesn't have to be performant because it doesn't need to ever get evaluated. It just has to exist so that we can prove that build_netlist
is correct, since "correctness" for `build_netlist" means "evaluating in the arrow representation = evaluating in the netlist representation", and we need some way to express the right-hand-side. Importantly, it will be part of the trusted code base (unless/until we do #142 and make another layer below it), so it needs to be as simple and obvious as possible.
Requires #234
This is a necessary link in the chain for arrow-based AES proofs; in order to bridge the gap between the arrow representation and the netlist representation, we need to prove that the translation between these two layers (done by the build_netlist
routine) preserves semantics.
This means verifying that the AES block obeys a functional specification (at the level of abstraction of the FIPS documentation). The functional specification will assume functional implementations for the main subroutines (mix_columns
, shift_rows
, sub_bytes
, and add_round_key
). Key expansion routines will also be assumed. The proof should state that if the subcircuits corresponding to the main subroutines match the functional implementations, then the cipher block matches the top-level functional specification.
A lot of the work on this involves developing the proof automation for arrows, since the circuit has too much structure for a manual proof. I've been working on this for a little while now, mostly performance engineering on the tactics. The lemmas in #229 were in service of this goal; they make it easy to compute the arrow structure only a little bit at a time. PRs #223, #226, and #232 were all work that emerged as part of progress towards the top-level cipher proof.
So far I've been working with the unrolled_cipher_naive
definition since the other top-level cipher options have been in flux, but since what I've been doing is general proof automation it should be possible to change that relatively easily. It's likely to be easier to work with an unrolled loop at first, though, so I'll probably continue to push through this proof before updating to a stateful or not-unrolled version.
Add a new component to the netlist data-structure which corresponds to an array lookup (of an array of bits) which models a multiplexor. A different bug will track the more general requirementof multiplexing between inputs of buses rather than single wires.
Before I produced a special _CoqProject
at the root solely to be consumed by VS Code so all the code could be navigated in one session and code could be stepped through (including proofs) in any Coq. The recent organization #213 has broken this.
Following up on conversation with @satnam6502 :
We've discussed me trying to verify a monad-based version of the AES block, both to have a point of comparison for proof effort and to have a "backup option" in case the performance issues I'm currently running into with arrows aren't solvable. For the first pass at this, I'd want to work on a similar proof to what I'm currently working on in arrow-land, which means some version of the whole AES block defined in terms of its sub-components. It can be specialized to AES-256, and the loop can be either unrolled or not. It's okay to leave the subroutines (key expansion, mix_columns
, sub_bytes
, shift_rows
, add_round_key
) as Axiom
s; I don't need to unfold them or inspect them for this proof. It's also okay to leave things at the level of arrow-examples/Combinators.v
(for instance, vector reversal) as axioms for now. I've been looking at unrolled_cipher_naive
for the current proof, so something similar to that will provide the best point of comparison in the short-term. However, if it's a lot of effort to implement, it might be better to go with the more updated, stateful version that we'd need eventually anyway.
We already have a specification for the top-level AES cipher (see silveroak-opentitan/aes/Spec/Aes.v
). Now, we need similar specifications, based on the FIPS documentation, for the subroutines that it composes:
sub_bytes
shift_rows
mix_columns
And their inverses:
inv_sub_bytes
(I think this is actually the same as sub_bytes
)inv_shift_rows
inv_mix_columns
add_round_key
is just xor, so we don't need to separately define it.
Related to and partially dependent on #294
Once we have a specification for mix_columns
and inv_mix_columns
, we need to prove the usual inverse property: forall x, inv_mix_columns (mix_columns x) = x
. This is potentially a complicated proof, because mix_columns
has the most mathematically advanced reasoning of all the AES subroutines.
Instead of rolling our own code for converting between unsigned and two's complement bit level representations with Bvector
and nat
or N
use instead the library Coq.NArith.Ndigits
as a basis and exploit its lemmas for our proofs.
I think it is better to explicitly specify the size of the sum result vector and define the semantics of unsigned addition in terms of addition and modulus.
Currently the colV
combinator has the type:
Fixpoint colV `{Monad m} {A B C} (n: nat)
(circuit : C -> A -> m (B * C)%type)
(c: C) (a: Vector.t A (S n)) :
m (Vector.t B (S n) * C)%type
However, it really needs to have the type:
Fixpoint colV `{Monad m} {A B C} (n: nat)
(circuit : C -> A -> m (B * C)%type)
(c: C) (a: Vector.t A n) :
m (Vector.t B n * C)%type
i.e. the n
should reflect the size of the col
i.e. the size of the input and output vector. col 0
is a valid circuit.
Once the type of col
is fixed all the dependent examples need to be de-hacked e.g.
Definition xilinxAdder {m bit vec} `{Cava m bit vec} {n: nat}
(a: vec Bit (S n)) (b: vec Bit (S n))
: m (vec Bit (S n)) :=
z <- zero ;;
'(sum, carry) <- xilinxAdderWithCarry (z, (a, b)) ;;
ret sum.
needs to be changed to:
Definition xilinxAdder {m bit vec} `{Cava m bit vec} {n: nat}
(a: vec Bit n) (b: vec Bit n)
: m (vec Bit n) :=
z <- zero ;;
'(sum, carry) <- xilinxAdderWithCarry (z, (a, b)) ;;
ret sum.
and likewise for the other ripple-carry adder examples and adder-tree examples.
The `Kind type describes the kind of values that that can be associated with a named port of a cirucit:
Inductive Kind : Type :=
| Void : Kind (* An empty type *)
| Bit : Kind (* A single wire *)
| Vec : Kind -> nat -> Kind (* Vectors, possibly nested *)
| ExternalType : string -> Kind. (* An uninterpreted type *)
The Vec
type let's us represent possible nested vectors. For example Vec Bit 8
is a type for an 8-bit vector and Vec (Vec 8 Bit) 2
is a type for a vector of two elements where each element is an 8-bit vector.
These types are used to specify the interfaces of circuits e.g. here is an interface for an adder circuit that takes two 4-bit inputs and returns a 5-bit result:
Definition adder4Interface
:= combinationalInterface "adder4"
(mkPort "a" (Vec Bit 4), mkPort "b" (Vec Bit 4))
(mkPort "sum" (Vec Bit 5))
[].
We can implement an adder for this interface as follows:
Definition adderGrowth {m bit} `{Cava m bit} {aSize bSize: nat}
(ab: Vector.t bit aSize * Vector.t bit bSize) :
m (Vector.t bit (1 + max aSize bSize)) :=
let (a, b) := ab in
unsignedAdd a b.
which uses the unsignedAdd
from the CavaClass
:
unsignedAdd : forall {a b : nat}, Vector.t bit a -> Vector.t bit b ->
m (Vector.t bit (1 + max a b));
Another example is an adder tree:
Definition adder_tree_Interface name nrInputs bitSize
:= combinationalInterface name
(mkPort "inputs" (Vec (Vec Bit bitSize) nrInputs))
(mkPort "sum" (Vec Bit bitSize))
[].
Definition adderTree {m bit} `{Cava m bit} {sz: nat}
(default : bit)
(n: nat) (v: Vector.t (Vector.t bit sz) (2^(S n))) :
m (Vector.t bit sz) :=
tree (Vector.const default _) n addN v.
Note how the Vec
expressions in the interface corresponds to Cava's Vector.t
type in the overloaded definitions. This works out well for the combinational interpretation because the type Vector.t
directly represents our intention with Vec
. For example Vec Bit 8
maps to Vector.t bool 8
and Vec (Vec Bit 8) 2
maps to Vector.t (Vector.t 8 bool) 2
. This is capture by:
Fixpoint denoteKindWith (k : Kind) (T : Type) : Type :=
match k with
| Void => unit
| Bit => T
| Vec k2 s => Vector.t (denoteKindWith k2 T) s
| ExternalType t => T
end.
However, this is not such a great match directly for the netlist interpretation where we have Signal
values flowing over the wires:
Inductive Signal : Kind -> Type :=
| UndefinedSignal : Signal Void
| UninterpretedSignal: forall {t: string}, string -> Signal (ExternalType t)
| UninterpretedSignalIndex: forall (t: string), N -> Signal (ExternalType t)
| SelectField: forall {k1: Kind} (k2: Kind), Signal k1 -> string -> Signal k2
| Gnd: Signal Bit
| Vcc: Signal Bit
| Wire: N -> Signal Bit
| NamedWire: string -> Signal Bit
| NamedVector: forall k s, string -> Signal (Vec k s)
| LocalVec: forall k s, N -> Signal (Vec k s)
| VecLit: forall {k s}, Vector.t (Signal k) s -> Signal (Vec k s)
(* Dynamic index *)
| IndexAt: forall {k sz isz}, Signal (Vec k sz) ->
Signal (Vec Bit isz) -> Signal k
(* Static indexing *)
| IndexConst: forall {k sz}, Signal (Vec k sz) -> nat -> Signal k
(* Static slice *)
| Slice: forall {k sz} (start len: nat), Signal (Vec k sz) ->
Signal (Vec k len).
A wire carrying a single bit is represented by Signal Bit
. A bus carrying an 8-bit vector is represented by Signal (Vec Bit 8)
. A bus carrying two 8-bit vectors is represented by Signal (Vec (Vec Bit 8) 2)
.
It would be nice to define overloaded combinators that work over vectors in overloaded circuit description e.g. the col
combinator:
Fixpoint colV `{Monad m} {A B C} (n: nat)
(circuit : C -> A -> m (B * C)%type)
(c: C) (a: Vector.t A n) :
m (Vector.t B n * C)%type :=
However, this Signal
type does not work out here because what we really want is Vector.t
of a signal so it fits the type. Currently we do this by smashing Vec
arguments into their elements where are represented as a Vector.t
value. This means that for the netlist interpretation the Vec Bit 4
type is not mapped to a Signal (Vec Bit 4)
value but instead is smashed to a value of type Vector.t (Signal Bit) 4
. Likewise Vec (Vec Bit 4) 2
is smashed to Vector.t (Vector.t (Signal Bit) 4) 2)
i.e. smashing a bit-vector always results in vectors of vectors of vectors etc. where the innermost vector is a vector of bits. Now we have the right type for writing overloaded circuit description using things like the col
combinator where we can instantiate either a combinational circuit (for simulation, which acts as the semantics) or we can extract a circuit netlist.
The smashing operation is described as:
Fixpoint smashTy (T: Type) (k: Kind) : Type :=
match k with
| Void => Signal Void
| Bit => T
| Vec k2 s => Vector.t (smashTy T k2) s
| ExternalType t => Signal (ExternalType t)
end.
A Signal
value is smashed just by taking it apart by statically indexing it with all its legal index values:
(* The function takes a signal and smashes the vectors in it. *)
Fixpoint smash {k: Kind} (v: Signal k) : smashTy (Signal Bit) k :=
match k, v return smashTy (Signal Bit) k with
| Void, vv => UndefinedSignal
| Bit, vv => vv
| Vec k2 s, vv => Vector.map (fun i => smash (IndexConst vv i)) (vseq 0 s)
| ExternalType t, vv => UninterpretedSignal "smash-error"
end.
This is why we have these smash-related types and operations at the interfaces of our circuits. However, it does introduce quite a bit of complexity to the types of functions like makeNetlist
.
Is there a better way of denoting the vector values represented by the Vec
constructors that plays well for both semantics (combinational, sequential simulation) as well as netlist generation (the Signal
type)?
One thing I did explore a while ago was to add a vec
element to my typeclass along with other operations for indexing. Then for each interpretation this can be instantiated to the appropriate type e.g. Vector.t
for combinational semantics and Signal (Vec k s)
for the netlist interpretation. But then how would you sensibly define a map
operation?
Fixpoint mapSignal {k1 k2: Kind} {s: nat}
(f : k1 -> k2)
(v : vec k1 s) : vec k2 s.
Does this involve replicating the functionality of Vector.t
using our own overloaded vector type?
The SystemVerilog layer should not have any Cava kernel specific features in it i.e. it should be a fairly direct and neutral representation of the subset of SystemVerilog that we generate.
Please can you add a "clean" rule that ultimately gets invoked from the clean
rule in cava/Makefile
to remove auto-generated files, perhaps by adding them to a .gitignore
that is used in a clean
rule? Thank you kindly.
Untracked files:
(use "git add <file>..." to include in what will be committed)
ArrowExamples.vok
ArrowExamples.vos
ArrowPHOAS.vok
ArrowPHOAS.vos
ArrowStream.vok
ArrowStream.vos
We can generate circuits with the monad-kernel loop combinators but we should also attribute semantics to them so we can perform proofs about them.
See project-oak/oak#1151 for more information
This change will mean the Coq side does not need to concern itself with how fresh SystemVerilog vectors are made for use with SystemVerilog operations requiring vectors (like arithmetic). This means the Coq netlist representation can now capture the size relationship between the inputs and outputs of arithmetic operations e.g.
| UnsignedAdd : forall m n, Vector.t N m -> Vector.t N n ->
Vector.t N (max m n + 1) -> Primitive
The pre-submit process for PRs takes very long. Can we do anything to speed it up?
I assume some of this is due to the number of git submodules we use?
Roadmap
Kind
back into one, and re-enable extern/uninterpreted type for arrows.object
be Kind
in Arrow.Cava class. Currently this isn't a requirement and I hesitated on making this change to allow more generality but currently there is no use for this generality and it hinders work on proofs which would benefit from some sane props about object such as it has inhabitants or that it is inductively created.Optional
nat
from N
. Native Coq Vector expects nat. Motivation for keeping N
is that extraction should be size log2 n
whereas nat
extraction will be linear.Does:
$ cd third_party
$ make clean
cause coqutil
to get "cleaned" or "uninstalled"?
Using Extraction Library
works but we have little control over how the extraction happens, which means the extracted code is not readable (due to lack of type mapping and also module hierachy), and also causes a lot of files to be generated that we need to make sure are hidden from version control (.gitignore
).
This might be the best way for us, but another way is to manually extract terms with giving us more control. For an example Kami uses constant extract to control exactly what and how terms are extracted:
This adds an example for a three input adder plus verilator lint wavier to allow the automatic extension of vectors for the +
operation.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.