galoisinc / pycryptol

pycryptol: Use Cryptol with Python

pycryptol: Use Cryptol with Python

This package offers deep integration between Python and Cryptol. Its goal is to allow a Python program to use all of the features available at a standalone Cryptol console. Where possible, the bindings attempt to provide deeper integration so that Cryptol values may be used transparently as if they were Python values.

For full documentation, see http://pycryptol.readthedocs.org/en/latest/

pycryptol's Issues

Add a Python representation of Cryptol types

Currently _CryptolModule.typeof returns the pretty-printed type of the expression as a string. This is not amenable to programmatic manipulation. We should define some sort of consistent representation of Cryptol types on the Python side. This could be incremental:

1. monomorphic types
2. unconstrained polymorphic types
3. all types

Perhaps with some representation of type synonyms and newtypes as well, though the utility of these on the Python side is questionable.

Prevent Cryptol names from overwriting Python built-in names

Add a filter for names to assign to new CryptolModules so that things like the class hierarchy and built-in methods don't get overwritten.

Can we override IPython's psource magic to return Cryptol source?

In IPython, you can inspect the source of a value with ?? or %psource. Let's see if we can override that behavior to instead return Cryptol source.

Subclass BitVector for more Cryptol-like defaults

At the very least having str and repr output valid Cryptol syntax.

better error message when pycryptol can't find cryptol-server

(e.g., when it's not on your path).

Right now I get the rather unfriendly:

Traceback (most recent call last):
  File "testInstall.py", line 2, in <module>
    cry = cryptol.Cryptol()
  File "/usr/local/lib/python2.7/site-packages/cryptol/cryptol.py", line 64, in __init__
    stderr=null)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Handle polymorphic declarations

Currently all polymorphic top-level declarations are ignored when populating the object returned by Cryptol.load_module. We should be able to make incremental improvements to this situation:

1. Treat polymorphic values as we treat monomorphic functions. Type arguments would just be another Python value to pass in to a Python function. We could even make keyword arguments that correspond to the type variable names on the Cryptol side. In order to do this, we'll need to define a representation of Cryptol types in Python (#2). The downside here is that the user must provide all of the type arguments; Cryptol can't do any inference at this point.
2. If the polymorphic value is a function, try to infer the type of its application with the types of its arguments. We might be able to special-case this without having to fall back to Cryptol's type inference, but a general solution will mean we'll have to construct an expression on the Cryptol side so that the typechecker can do its work. We can avoid keeping the full expression of the function around for this, though, so long as we remember its type: create an application with a variable in function position with the correct type in an annotation, then drop the arguments in and typecheck the resulting AST.

Gracefully handle exceptions on the Python side

Keyboard interrupts (and presumably other exceptions) can cause pycryptol and the server to get out of sync. Let's figure out how to intercept these calls and propagate interrupts to the server to halt runaway computations.

Add more type checks for Python-originated values

The Cryptol evaluators depend on the invariants guaranteed by the typechecker. However when we're making Cryptol values out of Python in order to apply arguments to functions, we're just manually creating a Value without heed to the types. We should probably address this on the cryptol-server side by keeping track of the types of function handles (already needed for #3) and writing a typechecker for incoming values.

When evaling a function, don't return a method

Right now the same code runs for eval and load_module when evaluating functions, and it's currently specialized to produce methods for load_module. This means that while you can use functions through eval, it's confusing because you manually have to provide the enclosing module as the self argument. We should have different code paths for these cases, and return a value from eval with the module context baked in.

Cryptol structures/records cause crashes in `__from_value`

Minimal reproducer (with current HEAD):
test.py:

import os, cryptol
test = cryptol.Cryptol().load_module(os.path.abspath('test.cry'))

test.cry:

somestruct = {
    x = 0x12,
    y = 0x34
}

Output:

$ python test.py 
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    test = cryptol.Cryptol().load_module(os.path.abspath('test.cry'))
  File "/usr/local/lib/python2.7/site-packages/cryptol/cryptol.py", line 294, in load_module
    mod = cls(port, req, self.__main_req, filepath)
  File "/usr/local/lib/python2.7/site-packages/cryptol/cryptol.py", line 360, in __init__
    val = self.__from_value(val_resp['value'])
  File "/usr/local/lib/python2.7/site-packages/cryptol/cryptol.py", line 478, in __from_value
    fname = field[0]['Name']
TypeError: list indices must be integers, not str

This patch seems to fix it for me, but I haven't tested it other than on my own use-cases:

diff --git a/cryptol/cryptol.py b/cryptol/cryptol.py
index 825a368..8f3a697 100644
--- a/cryptol/cryptol.py
+++ b/cryptol/cryptol.py
@@ -475,7 +475,7 @@ class _CryptolModule(object):
         if 'record' in val:
             rec = {}
             for field in val['record']:
-                fname = field[0]['Name']
+                fname = field[0][1]
                 fval = self.__from_value(field[1])
                 rec[fname] = fval
             return rec

Smart templating for Cryptol syntax strings

Instead of:

m.eval("%s + %s" % (m.to_expr(x), m.to_expr(y)))

We should make eval and similar calls into varargs methods with automatic to_expr calls around the splices:

m.eval("? + ?", x, y)

Handle functions in nested data

The code for building a closure on the Python side out of a Cryptol value currently only checks the top-level type of the expression to see whether it's a function. This means that if a function is wrapped up in, say, a tuple, it does not get converted into a callable Python function.

Unfortunately this will require a fair amount of refactoring on the cryptol-server side given how we're generating JSON.

Better support for infix operators and invalid Python identifiers

Python doesn't allow user-defined infix operators, and Cryptol's lexical syntax for identifiers is more permissive than Python's. Many Cryptol definitions, including most of the Cryptol Prelude, therefore don't appear as methods on objects returned from Cryptol.load_module.

These identifiers are still accessible through calls to _CryptolModule.eval, since one can specify any string as the argument, but the inconsistency is unfortunate.

We could do some name mangling to fit both these identifiers into Python's syntax; are there any other possibilities?

Allow richer Python values for property arguments

Currently, _CryptolModule.{check, exhaust, sat, prove} all take a string argument representing the property to check. It should be possible to pass in Python values to these functions, as long as those Python values originated from Cryptol in the first place. So if you have:

module Foo where

bar : [32] -> Bit
property bar x = x == x

On the Python side you ought to be able to have something like:

Foo = Cryptol().load_module("Foo.cry")
Foo.prove(Foo.bar)

Richer return values for check and exhaust

The Cryptol implementation of :check and :exhaust bakes in a lot of console-specific assumptions. We need to refactor those bits so that those commands can return more meaningful output including counterexamples.

Automatically convert more types of Python values

Once we have actionable type information on the Python side (#2) we should be able to do smarter things about coercing Python values into Cryptol. For example, a Python int in argument position could be coerced to a Cryptol bitvector once we know the width expected by the applied Cryptol function.

We could also extend this to ASCII, unicode, etc: check for sequences of bytes and do the conversion as long as the length works out.

Improve return types for sat and prove

Right now the API for _CryptolModule.prove is pretty ugly. It returns None when a property is valid, and a tuple of counterexample arguments if not.

But even worse than ugly is _CryptolModule.sat, which returns an empty list for unsat; the problem is that if your property is not very interesting and doesn't take any arguments, sat will return an empty list whether it evaluates to True or False.

We should probably have some sort of solver result API that might address #7 as well.