ucsd-progsys / liquidhaskell

Liquid Types For Haskell

This is the development site of the LiquidHaskell formal verification tool.

If you're a LiquidHaskell user (or just curious), you probably want to go to the documentation website instead.

Contributing

This is an open-source project, and we love getting feedback (and patches)!

Reporting a Bug

If something doesn't work as it should, please consider opening a github issue to let us know. If possible, try to:

• Try to use a descriptive title;
• State as clearly as possible what is the problem you are facing;
• Provide a small Haskell file producing the issue;
• Write down the expected behaviour vs the actual behaviour;
• Please, let us know which liquidhaskell version you are using.

Your first Pull Request

We are thrilled to get PRs! Please follow these guidelines, as doing so will increase the chances of having your PR accepted:

• The main LH repo lives here
• Please create pull requests against the develop branch.
• Please be sure to include test cases that illustrate the effect of the PR
  • e.g. show new features that that are supported or how it fixes some previous issue
• If you're making user-visible changes, please also add documentation
  • e.g. options.md, specifications.md, the main tutorial (as relevant)

Pull requests don't just have to be about code: documentation can often be improved too!

Ask for Help

If you have further questions or you just need help, you can always reach out on our slack channel, google groups mailing list, GitHub issue tracker, or by emailing Ranjit Jhala, Niki Vazou.

General Development Guide

For those diving into the implementation of LiquidHaskell, here are a few tips:

Running the pluging on individual files

stack build liquidhaskell
stack exec ghc -- -fplugin=LiquidHaskell FILE.hs

cabal build liquidhaskell
cabal exec ghc -- -fplugin=LiquidHaskell FILE.hs

Building

Stack

stack build

If on NixOS

stack --no-nix-pure build

With the above, stack will unregister and re-register the libraries, but hopefully it won't rebuild any modules.

Cabal

cabal v2-build

Faster recompilation

When changing the liquidhaskell-boot library, sometimes we don't want to rebuild liquidhaskell or liquid-vector when testing the changes. In these cases we can set the environment variable LIQUID_DEV_MODE=true when running stack or cabal to skip rebuilding those packages.

DANGER: Note that this can give an invalid result if the changes to liquidhaskell-boot do require rebuilding other liquid* packages.

How To Run Regression Tests

For documentation on the test-driver executable itself, please refer to the README.md in tests/ or run cabal run tests:test-driver -- --help or stack run test-driver -- --help

You can run all the tests by

$ ./scripts/test/test_plugin.sh

You can run a bunch of particular test-groups instead by

$ ./scripts/test/test_plugin.sh <test-group-name1> <test-group-name2> ...

and you can list all the possible test options with

$ ./scripts/test/test_plugin.sh --help

or get a list of just the test groups, one per line, with

$ ./scripts/test/test_plugin.sh --show-all

To pass in specific parameters, you can invoke cabal directly with

$ cabal build tests:<test-group-name> --ghc-options=-fplugin-opt=LiquidHaskell:--no-termination

For example:

$ cabal build tests:unit-neg --ghc-options=-fplugin-opt=LiquidHaskell:--no-termination

Or your favorite number of threads, depending on cores etc.

You can directly extend and run the tests by modifying the files in

tests/harness/

Parallelism in Tests

Tests run in parallel, unless the flag --measure-timings is specified to test_plugin.sh.

How to create performance comparison charts

When liquidhaskell tests run, we can collect timing information with

$ ./scripts/test/test_plugin.sh --measure-timings

Measures will be collected in .dump-timings files under dist-newstyle directory. These can be converted to json data with

cabal v2-build ghc-timings
cabal v2-exec ghc-timings dist-newstyle

which will produce tmp/*.json files.

Then a csv report can be generated from this json files with

cabal v2-run benchmark-timings -- tmp/*.json --phase LiquidHaskell -o summary.csv

On each line, the report will contain the time taken by each test.

Comparison charts in svg format can be generated by invoking

cabal v2-run plot-performance -- -b path_to_before_summary.csv -a path_to_after_summary.csv -s 50 -f "benchmark"

This will generate three files filtered.svg (a subset of tests with a benchmark prefix, enabled by the -f option), top.svg and bot.svg (top 50 speedups and slowdowns over the entire test set, both enabled by the -s option) in the current directory. The -f and -s options can be used/omitted independently. If both are omitted, a single perf.svg will be produced covering the full input test set. Additionally, their effects can be combined by providing a third -c option (this will produce 2 files filtered-top.svg and filtered-bot.svg instead of 3). An optional key -o can be supplied to specify an output directory for the generated files.

There is also a legacy script scripts/plot-performance/chart_perf.sh that can be used to generate comparison charts in both svg and png formats. It requires gnuplot to run and assumes both files contain the same test set. The following command will produce two files perf.svg and perf.png in the current directory.

$ scripts/plot-performance/chart_perf.sh path_to_before_summary.csv path_to_after_summary.csv

The current formatting is optimized for comparing some subsets of the full test run, typically just the benchmarks alone. If one wishes to save time or is not interested in top speedups/slowdowns, the benchmark subset can be obtained by running

$ scripts/test/test_plugin.sh \
    benchmark-stitch-lh \
    benchmark-bytestring \
    benchmark-vector-algorithms \
    benchmark-cse230 \
    benchmark-esop2013 \
    benchmark-icfp15-pos \
    benchmark-icfp15-neg

Miscelaneous tasks

• Profiling See the instructions in scripts/profiling-driver/ProfilingDriver.hs.
• Getting stack traces on exceptions See -xc flag in the GHC user's guide.
• Working with submodules See man gitsubmodules or the git documentation site.

Releasing on Hackage

NOTE: The following section is relevant only for few developers, i.e. the ones which are directly involved in the release process. Most contributors can skip this section.

We provide a convenience script to upload all the liquid-* packages (including liquid-fixpoint) on Hackage, in a lockstep fashion. To do so, it's possible to simply run the scripts/release_to_hackage.sh Bash script. The script doesn't accept any argument and it tries to determine the packages to upload by scanning the $PWD for packages named appropriately. It will ask the user for confirmation before proceeding, and stack upload will be used under the hood.

GHC support policy

LH supports only one version of GHC at any given time. This is because LH depends heavily on the ghc library and there is currently no distinction between public API's and API's internal to GHC. There are currently no release notes for the ghc library and breaking changes happen without notice and without deprecation periods. Supporting only one GHC version saves developer time because it obviates the need for #ifdef's throughout the codebase, or for an compatibility layer that becomes increasingly difficult to write as we attempt to support more GHC versions. Porting to newer GHC versions takes less time, the code is easier to read and there is less code duplication.

Users of older versions of GHC can still use older versions of LH.

The GHC.API module

In order to minimize the effort in porting LH to new releases of GHC, we need a way to abstract over breaking changes in the ghc library, which might change substantially with every major GHC release. This is accomplished by the GHC.API module. The idea is that rather than importing multiple ghc modules, LH developers must import this single module in order to write future-proof code. This is especially important for versions of the compiler greater than 9, where the module hierarchy changed substantially, and using the GHC.API makes it easier to support new versions of GHC when they are released.

Fragile import strategy

import Predicate
import TyCoRep

...

-- This will break if 'isEqPrimPred' is (re)moved or the import hierarchy changes.
foo :: Type -> Bool
foo = isEqPrimPred

Recommended import strategy

import qualified Language.Haskell.Liquid.GHC.API as GHC

...

foo :: GHC.Type -> Bool
foo = GHC.isEqPrimPred -- OK.

GHC Plugin Development Guide

This code commentary describes the current architecture for the GHC Plugin that enables LiquidHaskell to check files as part of the normal compilation process. For the sake of this commentary, we refer to the code provided as part of the release/0.8.10.2 branch, commit 9a2f8284c5fe5b18ed0410e842acd3329a629a6b.

GHC.Interface vs GHC.Plugin

The module GHC.Plugin is the main entrypoint for all the plugin functionalities. Whenever possible, this module is reusing common functionalities from the GHC.Interface, which is the original module used to interface LH with the old executable. Generally speaking, the GHC.Interface module is considered "legacy" and it's rarely what one wants to modify. It will probably be removed at some point.

Plugin architecture

Broadly speaking, the Plugin is organised this way: In the typechecking phase, we typecheck and desugar each module via the GHC API in order to extract the unoptimised core binds that are needed by LH to work correctly. This is due to a tension in the design space; from one side LH needs access to the "raw" core binds (binds where primitives types are not unboxed in the presence of a PRAGMA annotation, for example) but yet the user can specify any arbitrary optimisation settings during compilation and we do not want to betray the principle of least expectation by silently compiling the code with -O0. Practically speaking, this introduces some overhead and is far from ideal, but for now it allows us to iterate quickly. This phase is also responsible for:

• Extracting the [BareSpec][]s associated to any of the dependent modules;
• Producing the LiftedSpec for the currently-compiled module;
• Storing the LiftedSpec into an interface annotation for later retrieval;
• Checking and verifying the module using LH's existing API.

The reason why we do everything in the typechecking phase is also to allow integrations with tools like ghcide. There are a number of differences between the plugin and the operations performed as part of the GHC.Interface, which we are going to outline in the next section.

Differences with the GHC.Interface

• The GHC.Interface pre-processes the input files and calls into configureGhcTargets trying to build a dependency graph by discovering dependencies the target files might require. Then, from this list any file in the include directory is filtered out, as well as any module which has a "fresh" .bspec file on disk, to save time during checking. In the GHC.Plugin module though we don't do this and for us, essentially, each input file is considered a target, where we exploit the fact GHC will skip recompilation if unnecessary. This also implies that while the GHC.Interface calls into processTargetModule only for target files, the GHC.Plugin has a single, flat function simply called processModule that essentially does the same as GHC.Interface.processModule and GHC.Interface.processTargetModule fused together.

• While the GHC.Interface sometimes "assembles" a [BareSpec][] by mappending the commSpec (i.e. comment spec) with the LiftedSpec fetched from disk, if any, the Plugin doesn't do this but rather piggybacks on the SpecFinder (described later) to fetch dependencies' specs.

• There is a difference in how we process LIQUID pragmas. In particular, for the executable they seems to be accumulated "in bulk" i.e. if we are refining a target module A that depends on B, B seems to inherit whichever flags we were using in the target module A. Conversely, the source plugin is "stateless" when it comes to LIQUID options, i.e. it doesn't have memory of past options, what it counts when compiling a module B is the global options and any option this module defines. The analogy is exactly the same as with GHC language extensions, they have either global scope (i.e. default-extensions in the cabal manifest) or local scope (i.e. {-# LANGUAGE ... #-}).

Finding specs for existing modules

This is all done by a specialised module called the SpecFinder. The main exported function is findRelevantSpecs which, given a list of Modules, tries to retrieve the LiftedSpecs associated with them. Typically this is done by looking into the interface files of the input modules, trying to deserialise any LiftedSpec from the interface file's annotations.

General Development FAQs

A new version of GHC is out. How do I support it?

Typically the first thing you might want to do is to run a "clean" cabal v2-build or stack build using the latest compiler and "check the damage". If you are lucky, everything works out of the box, otherwise compilation might fail with an error, typically because some ghc API function has been removed/moved/renamed. The way to fix it is to modify the GHC.API shim module and perform any required change, likely by conditionally compiling some code in a CPP block. For minor changes, it's usually enough to perform small changes, but for more tricky migrations it might be necessary to backport some GHC code, or create some patter synonym to deal with changes in type constructors.

Is there a way to run the testsuite for different versions of GHC?

Currently, no. Only one version of GHC is supported and that is the one that can be tested with ./scripts/test/test_plugin.sh.

liquidhaskell's Issues

Allow local `assume` without resorting to `undefined`

We should allow the user to write something like:

{-@ assume foo :: t_foo @-}
foo = real-definition-of-foo

while then in the rest of the module just use the type t_foo for foo
without checking the real-definition actually has the type.

Right now, this works if foo is imported from elsewhere (modular reasoning
etc. we don't check imported definitions) OR if we use undefined as the real
definition.

Would be nice to just allow assume without raising warnings...

Make `RType` equality take aliases into account

See tests/todo/TypeAlias.hs, which crashes because Bare.tyCompat checks for syntactic equality without expanding type synonyms, so we get

Bar Int /= Foo Int Int

even though Bar is defined as type Bar = Foo Int

Simply adding calls to expand[R]TypeSynonyms in tyCompat only delays the crash, instead manifesting as an out-of-bounds index in Constraint.splitCIndexed.

@nikivazou any thoughts on what might be the simplest fix? It looks to me that rewriting expandRTypeSynonyms to not drop refinements would work, but would also be quite painful..

make termination analysis code readable

This usually takes time, so I just add it in the after deadline todo list

exception thrown in tests/pos/ListElem.hs

Fixpoint is throwing the following exception on tests/pos/ListElem.hs using the current master of liquid-fixpoint and liquidhaskell..

z3Pred: error converting (Prop(VV) <=> Set_mem(~A0, listElts([~A1])))
Fatal error: exception Failure("Ast.sortcheck_app: type args not fully instantiated Set_mem([~A0; listElts([~A1])])")

This might actually belong in ucsd-progsys/liquid-fixpoint, but the failing test is in liquidhaskell...

Unhandled case in: Bare.mapTyVars

Hi Niki --

can you (do a pull on master) and then run

liquid tests/pos/mapTvCrash.hs

It throws an error:

****************************** ERROR *****************************
Bare.mapTyVars: cannot handle lq_tmp_db0:(GHC.Ptr.FunPtr ((GHC.Ptr.Ptr a) -> (GHC.Types.IO ())))
-> a
****************************** ERROR *****************************

I think it is some unhandled case in mapTyVars (in Bare.hs).

Can you take a look, since I don't know what that function is doing.

RJ.

Bug in nested recursion termination checker

Multiple recursive functions can be alive the same time. See

radicals :: Int -> [a]
radicals n = [ foo (radicals n) i | i <- [1..]]

foo = undefine

Now, the checker assumes only one recursive function

Termination checker doesn't handle weird type created by TransformRec

see tests/pos/fixme.hs for a runnable version.

insert :: Ord k => k -> a -> Map k a -> Map k a
insert = go
  where
    go :: Ord k => k -> a -> Map k a -> Map k a
    go kx x Tip = singleton kx x
    go kx x (Bin sz ky y l r) =
        case compare kx ky of
                  -- Bin ky y (go kx x l) r 
            LT -> balanceL ky y (go kx x l) r
            GT -> balanceR ky y l (go kx x r)
            EQ -> Bin sz kx x l r

go is given a duplicated class constraint (forall k a. Ord k => forall k a. Ord k => ...) which prevents the termination checker from finding the decreasing parameter (Map k a).

There are two things to do here:

1. Fix the TranformRec pass so it no longer makes these strange types.
2. Fix the termination checker to be more robust against strange types.

Error spans for type class instances is wrong

Failed constraints coming from instances of type class methods highlight have bogus associated code-spans, namely they always seem to highlight the instance declaration instead of the method (see tests/neg/Class2.hs).

I suspect that this is due to the auto-generated nature of the actual core-binds we get from GHC...

remove all dependencies on LIQUIDHS env variable

the relevant files should be tucked inside the cabal directory. (cf. nanojs prelude.spec)

`argExpr` crashes on functions with >1 dictionaries

The code (without user specified type signature)

isEven 0 = True
isEven n = isEven  $ n - 1

crashes with

argExpr: isEven @ a $dEq $dNum

Add support for type-indexed measures

We're currently limited in what we can say about a method of a type-class because we don't have a way of abstracting measures, e.g. consider a type-class for computing the size of a container.

class Sized (s :: * -> *) where
  size :: s a -> Int

How would we annotate size? We don't have a generic notion of size in liquidhaskell, nor should we since not all types have a sensible definition for size. I propose to add measures to type-classes, e.g.

class Sized (s :: * -> *) where
  {-@ class measure size :: s a -> Int @-}

  {-@ size :: Sized s => x:s a -> {v:Int | v = (size x)} @-}
  size :: s a -> Int

instance Sized [] where
  {-@ instance measure size [] = len @-}
  size = length

I have a more complete example that also includes client code using the type-class in tests/todo/Class.hs.

Refactor out `CGMonad.hs` from `Constraint.hs`

Constraint.hs has gotten huge. We should split it into the CG monad API, in a separate CGMonad.hs and the actual constraint generation code that traverses the CoreExpr and does the splitting.)

Exit with proper warning with unbound spec

See:

tests/todo/err8.hs
tests/todo/err12.hs

Should gracefully exit (with the line number etc. as with ghc/parse/etc. errors), i.e. the "CRASH" errors, as with

tests/todo/err9.hs

not the ERROR ERROR ERROR business.

Allow expressions as type/pred alias parameters

see:

tests/todo/aliasConst.hs
tests/todo/err9.hs

Qualifier not propagating in Dependent Tuple?

@nikivazou why is tests/pos/tup0.hs failing (the 2nd signature fails, not the first...)

any clue?

Allow refinements in the output type of measures

Only base types are allowed to appear in the type of a measure, but this is less accurate than it could be, e.g. when dealing with measures that denote the size of a value.

measure len :: [a] -> Int
len []     = 0
len (x:xs) = 1 + len xs

Since the type only specifies that len returns an Int, we need to additionally define an invariant on Int.

invariant {v:[a] | (len v) >= 0}

This is a bit tedious when what we really want to write is just

measure len :: [a] -> Nat
len []     = 0
len (x:xs) = 1 + len xs

Perhaps there's a good reason to disallow refinements in the return type of a measure, but it isn't apparent to me.

More opportunities for simplification of inferred refinements

A couple patterns I noticed while working on text which are just noise and make it harder to parse out the minimal inferred type:

1. Some mathematical simplification is possible, e.g. (v <= ((tlen x) - (tlen empty))) could be simplified to (v <= (tlen x)) because empty :: {v:Text | (tlen v) = 0}. Or perhaps you have (v < x + y) && (v < y + x) when we only need one of the conjuncts (this has a nasty interaction with pattern 3).
2. Some logical simplification is possible, e.g. ((tlen empty) > 0) => FOO can be removed entirely because the antecedent is false. I can't actually find an instance of this pattern at the moment so perhaps it's already been fixed, but I'm leaving it here just in case.
3. Some refinements are duplicated due to the eta* vars, e.g. (x = 0) && (eta_B1 = 0) where x == eta_B1. It would be nice to track these aliases somehow and prefer the name that is actually used in the source.

These things may seem minor but they add up. I have seen inferred types that span half of my screen, which are often full of this kind of noise.

crash instead of type mismatch error

The following code crashes with error

panic! (the 'impossible' happened)

data F a = F a

-- give F two parameters instead of one
{-@ foo :: a -> F a b @-} 
foo :: a -> F a
foo = undefined

instead of giving a type mismatch error

Abstract Refinements Being Dropped?

See:

• tests/neg/StrictPair0.hs
• tests/neg/StrictPair1.hs

The first uses "built-in" tuples, the second a new "hand-rolled" Pair constructor.

BOTH are obviously unsafe, but somehow the second one is marked SAFE!

Similarly, the "pos" tests have the wrong behavior. (Works for regular tuples, not for defined type)

• tests/pos/StrictPair0.hs
• tests/pos/StrictPair1.hs

(In general, looks like those refinements are being ignored...)

Any idea what's going on?

add flag to suppress `specification for unknown variable` errors

they can be really annoying during development..

Can we remove `scc` annotations from dumped GHC Core?

Sometimes I want to look at the Core that GHC generates and the scc annotations are really noisy.

Sanity checks with generic measures

Right now we let generic measures apply to any a, but we probably want to have some constraints, e.g. instead of

class measure size :: a -> Int

we might want to say

class measure size :: Sized s => s a -> Int

and then check that the instance definitions have a valid instantiation of s.

We'll probably also want to add some notion of constraint to fixpoint at some point to prevent a lot of pointless qualifier instantiations too.

Malformed Measure: refinement pruned silently

See tests/todo/Eval.hs

In the definition of measure free the last line has a sort-error as it is not a well-formed Prop.

If you change the last line of the above to:

free (Let x e1 e2) = {v | (Set_cup (free e1) (Set_dif (free e2) (Set_sng x)))}

then the program is safe. Otherwise the measure is malformed, and oddness
ensues, but liquidhaskell does not crash, it silently drops the measure.
We should report any malformed measure as an error, and stop...

[hsenv]rjhala@ubuntu:~/research/liquid/liquidhaskell (master)$ liquid --notermination tests/todo/Eval.hs > log
WARNING: prune unsorted reft:
(? Set_cup([free([e1#a1eW]);
        Set_dif([free([e2#a1eX]); Set_sng([x#a1eV])])]))
BExp Set_cup([free([e1#a1eW]);
     Set_dif([free([e2#a1eX]); Set_sng([x#a1eV])])]) with non-propositional type  Set_Set ([GHC.Types.Char])

Please fix strange error message

@gridaphobe if you run tests/pos/fixme0.hs you will see there is a bizarre type error:

Specified Liquid Type Does Not Match Haskell Type
Haskell: Goo.cnt :: forall a a. (GHC.Classes.Eq a, GHC.Num.Num a, GHC.Num.Num a) => a -> a
Liquid : Goo.cnt :: forall a a. dummy.tests.pos.fixme0.hs.4.12:(GHC.Types.Int) -> (GHC.Types.Int)

The bizarre thing is the

forall a a. Int -> Int

shown for the Liquid. Why is the forall a a. there? Why ANY forall? Why the duplicates? At any rate, can you fix it? Thanks! RJ.

Proper Error Message on Malformed Alias Application

See tests/todo/aliasError.hs

Instead of barfing:

safeZip called on non-eq-sized lists (nxs = 3, nys = 2) : expandRPApp: EApp Rng [ECon (I 0),ECon (I 10)]

we should produce, at the very least something like:

malformed alias application (Rng 0 10)` on line XXX column YYY

Failing Test with Abstract Refinements

http://goto.ucsd.edu:8090/index.html#?demo=permalink%2F1380410092.hs

TWO mysteries:

1. why is 'goon' not type checking? (I suppose I can download an older version to see if this is something new).
2. why is 'rev' raising a type error inside the recursive call to 'go' ?

Looks like we've introduced some bugs somewhere...?

exported binder not getting TRUE-d out signature

see: [tests/neg/Eval.hs]

type variable substitution

Type variable substitution is too fragile

tests/pos/deepmeas0.hs is failing with another one of these type mismatch errors where the tyvars are reversed between liquidhaskell and ghc.. We can prevent the error by providing an explicit type signature for klookup, but this is getting to be rather annoying.

We need a more general solution for determining the equality of GHC and liquidhaskell types.

allow signatures for local definitions and "weak" type signatures

Eg,

foo = ...
  where
    bar :: [a] -> Int -> a
    {-@ bar :: xs:[a] -> {v:Int | v = (len xs)} -> a @-}
    bar = ...

With no need to specify the inferred properties of xs.

---

Subtyping in user signatures is more expensive that checking, so maybe we should let checking be the default and annotate where subtyping is expected.

Bug in Termination Checker

@nikivazou why is the tests/neg/qsloop.hs proved "safe" without the --notermination?

That is, why is the function proved terminating? It patently does not terminate, e.g. if you run it with the input [1,2,3] ?

Typecheck measure definitions BEFORE converting to DataCon

Problem:
tests/pos/deepmeas0.hs fails because we merge all measures into a single datacons for [] and : which is malformed here due to the deep-measure definition.

Solution:

• Typecheck measure definitions BEFORE converting into DataCon.
• Typecheck raw DATACON definitions in ISOLATION

Earlier we commenting the DATACON typechecks out to enable deepmeasures but that allows glitches in the datacon specifications to silently creep in and be ignored.

clean `Language.Haskell.Liquid.Prelude.hs`

In Language.Haskell.Liquid.Prelude.hs there are many pointer-related functions,
eg

{-@ isNullPtr :: p:(Ptr a) -> {v:Bool | ((Prop v) <=> (isNullPtr p)) } @-}
isNullPtr :: Ptr a -> Bool
isNullPtr p = (p == nullPtr)

So, when I import Prelude I get the isNullPtr qualifier which most probably won't need.

Can I move these functions to another place, maybe Language.Haskell.Liquid.Foreign.hs?

z3 crashing

The following code

{-@ listElem :: y:a -> zs:[a] -> {v:Bool | (Prop(v) <=> Set_mem(y, (listElts(zs))))} @-}
listElem ::  a -> [a] -> Bool
listElem = undefined

Crashes with a z3 error:

z3Pred: error converting (Prop(VV) <=> Set_mem(~A0, listElts([~A1])))
Fatal error: exception Failure("Ast.sortcheck_app: type args not fully instantiated Set_mem([~A0; listElts([~A1])])")

type check error in measure definitions

When typechecking the measure definitions, we don't check the result type,

see tests/todo/Measure.hs

Need to handle Coercions

The following snippet makes liquidhaskell crash

module Encoding where

import Data.ByteString as B
import Data.ByteString.Internal as B
import Data.Text.Internal (Text(..), safe, textP)
import Data.Word (Word8)
import Foreign.C.Types (CSize)
import Foreign.ForeignPtr (withForeignPtr, ForeignPtr)
import Foreign.Ptr (Ptr, minusPtr, plusPtr)


-- | Encode text using UTF-8 encoding.
encodeUtf8 :: Text -> ByteString
encodeUtf8 (Text arr off len) = inlinePerformIO $ do
  let size0 = max len 4
  mallocByteString size0 >>= start size0 off 0
 where
--  start :: Int -> Int -> Int -> ForeignPtr Word8 -> IO ByteString
  start size n0 m0 fp = withForeignPtr fp $ go n0 m0
   where
      offLen = off + len
      go n m ptr
        | n == offLen = return (PS fp 0 m)
        | otherwise = do
            let ensure k act
                  | size-m >= k = act
                  | otherwise = do
                      let newSize = size
                      fp' <- mallocByteString newSize
                      start newSize n m fp'
                {-# INLINE ensure #-}
            ensure 4 $ return $ PS fp 0 m

The issue seems to be that we don't have a case for handling Coercions in Language.Haskell.Liquid.ANFTransform.normalize. For this case, we can work around it by uncommenting the type or removing the INLINE annotation.

Tests fail with "liquid: mkTopLevEnv: not interpreted..."

I have used GHC 7.4.2 and ocaml 4.00.1 on an x86_64 Fedora 18 system to install liquid-fixedpoint (3817defc18ef6413627fe7ec897f90960060bf2b) and liquid haskell (0593c52). But am unable to get the system to cleanly build any examples - perhaps this is due to my own error?

For example:

$ liquid LiquidArray.hs
© Copyright 2009-12 Regents of the University of California.
All Rights Reserved.
liquid ["LiquidArray.hs"]

paths = ["./","/home/tommd/dev/liquidhaskell/include"]
parseSpec: LiquidArray.hs for module LiquidArray
getSpecs: [("Prelude","/home/tommd/dev/liquidhaskell/include/Prelude.spec")]
parseSpec: /home/tommd/dev/liquidhaskell/include/Prelude.spec for module Prelude
parseSpec: /home/tommd/dev/liquidhaskell/include/GHC/Base.spec for module GHC.Base
parseSpec: /home/tommd/dev/liquidhaskell/include/GHC/List.lhs for module GHC.List
parseSpec: /home/tommd/dev/liquidhaskell/include/GHC/Classes.spec for module GHC.Classes
parseSpec: /home/tommd/dev/liquidhaskell/include/GHC/Prim.spec for module GHC.Prim
parseSpec: /home/tommd/dev/liquidhaskell/include/GHC/Types.spec for module GHC.Types
liquid: mkTopLevEnv: not interpreted main:LiquidArray

and this means the tests fail:

...
exec: liquid ../web/demos/lenMapReduce.hs
liquid: mkTopLevEnv: not interpreted main:ListLengths
0.208691 seconds
FAILURE :( (../web/demos/lenMapReduce.hs)

Let me know if I can provide any other useful information. Feel free to close/pbkc if that's the case.

Remove `import` directive

Since we have both .spec and .hs files in include/, the import directive can give the impression that you can import a .hs from a .spec. This is not the case, .hs imports are all handled by GHC in the depanal phase. Furthermore, the only reason to put a .hs file into the include directory is to add new Haskell functions, which would then have to be imported by the original source file anyway.

I suggest we scrap the import directive since it serves no purpose other than to let spec files include other spec files, but the Haskell module system already solves this problem. We should instead grab the entire module graph from GHC (not the local modules that are exposed by getModuleGraph) and parse the specs that are part of the graph. It's not immediately clear to me how to get the whole graph, but it must be possible somehow..

As an alternative, I would suggest making it an explicit error to import a .hs file from a .spec file.

Error with `exists` in refinement for `(.)`

@nikivazou when you have time can you take a look at Data.Text.Lazy.isInfixOf in my text branch? It (and a few other functions) are crashing with the following type of error:

checkTycon cconsCaselq_anf__d4R0type: exists [z:{VV#2976 : (Data.Text.Fusion.Internal.Stream {VV#2974 : (GHC.Types.Char) | k_2975[f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX] && k_2983[lq_tmp_x2986:=needle#a4nX][VV#2982:=VV#2974][f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX]}) | k_2977[f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX] && k_2985[lq_tmp_x2986:=needle#a4nX][VV#2984:=VV#2976][f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX]}].{VV#2978 : (GHC.Types.Bool) | k_2979[f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX] && k_2989[lq_tmp_x2990:=z][VV#2988:=VV#2978][f:=isSingleton#r226][g:=stream#r3zS][x:=needle#a4nX]}

This appears to be due to the recent addition of the existential in the refinement for (.). I've commented out the refinement for the time being, so I'm not blocked, but it would be nice to know why this is happening.

deriving measures

I would like to write

{-@ data List a = N | C (head:a) (tail:(List a)) 
                deriving (Len, Cmp, Fields) 
  @-}

and get the measure definitions

• for Len derivation

measure dlenList :: (List a) -> Int
  dlenList N        = 0
  dlenList (C x xs) = 1 + dlenList xs

• for Cmp derivation

measure isN :: (List a) -> Prop
  isN N = true
  isN _ = false

measure isC :: (List a) -> Prop
  isN (C _ _) = true
  isN _       = false

• for Fields derivation

measure head :: (List a) -> a
  head (C x xs) = x

measure tail :: (List a) -> (List a)
  tail (C x xs) = xs

If you find more interesting "measure Classes" please add them

Some notes:

• Both Cmp and Fields depend on the names of data constructors,
  but Len is not

  so the user should write

  {v : List a | len v = 12}` and `{v : Tree a | len v = 12}
  

  and get it translated to

{v : List a | dlenList v = 12}` and `{v : Tree a | dlenTree v = 12

(and make the inverse translation while printing the types)

• This is not only for usability but also for soundness.
  There is the question of what happens if the user defines the measures in a wrong way.
  Ie.,

  measure isN :: (List a) -> Prop
   isNull N = false
   isNull _ = true

Now this questions goes down to what happens if a user defines a monadic instance that does
not satisfy monadic laws, which seems more reasonable.

• With this we also have a size measure for termination
  so, we don't have to explicitly provide it the definition

Duplicate Type Aliases Not Reported

type alias duplications not reported, see:

tests/pos/typeAliasDup.hs

GHC Panic due to type family?

If you run

liquid tests/pos/tyfam0.hs

from here https://github.com/ucsd-progsys/liquidhaskell/blob/master/tests/pos/tyfam0.hs

then we get a GHC panic. Perhaps has to do with this call:

https://github.com/ucsd-progsys/liquidhaskell/blob/master/Language/Haskell/Liquid/RefType.hs#L704

but I'm not sure, am building a prof version to get a backtrace. Any clues?

More precision with `LAZYVAR`

Consider the following (in tests/todo/LazyVar.hs)

{-@ foo :: a -> Bool @-}
{-@ bar :: [a] -> Nat -> a @-}
bar :: [a] -> Int -> a
bar xs i
  | i < l && foo x = x
  | otherwise      = undefined
  where
    l = length xs
    {-@ LAZYVAR x @-}
    x = xs !! i

This should be deemed safe because x wil not be demanded unless i < l evaluates to True.

Parse error

This type has bad syntax, but it appears to cause the LiquidHaskell to crash:

{-@ one :: (Num a) => {v:a | (1:a) <= v} @-}                                    
one :: Num a => a                                                               
one = 1

output:

liquid: Printf.printf: bad formatting char t

Finite lists

1. [1..100] has { len xs > 0 }, maybe add { len xs <= 100 }
2. add MAX_INT constant, so I can define Int predicate (n < MAX_INT) and make "length" safe on infinite lists

malformed printing in .html

see the type signature of empty at benchmarks / esop2013-submission / Array.hs

where

{-@ empty :: forall <p :: Int -> a -> Prop>. Vec <{\v -> 0=1}, p> a @-}

is printed as

<{\v -> 0=1}, p> a @-}

refsymbols?

What is refsymbols used for?
Why is it in the CG State monad (global) and not in an environment (local)?

Also (in bsplitC) it is a terrible idea to call variables map... (bit like redefining +)
Please pick a more suitable name. I would suggest one but I'm not sure what's going
on with refsymbols.

More name resolution woes

In Data/Vector/Algorithms/Radix.hs the class method

size :: (Radix e) => e -> Int

clashes with the parameter

size :: Int

of the function sort and causes a haskell-liquid type mismatch.

currently hacked to rename the parameter, but should
obviously fix, probably, by not using ALL bound variables
in a module, just the ones corresponding to function names
during name resolution.

(Having a bit of trouble generating a minimal test)

Generalized Termination Metrics

Generalize the termination specifications to allow sequences of Refinement Expressions.

For example:

foo :: n:Nat -> m:Nat -> Nat/[n+m]
foo n m  
  | cond1 = 0
  | cond2 = foo (n-1) m
  | cond3 = foo (n+1) (m-2)

Note that this generalizes the current setup nicely,
because instead of specifying the NUMBER (position)
of the decreasing argument, you can just NAME it:

foo :: n:Nat -> m:Nat -> Nat/[m]

Furthermore, we could support the lexicographic
business by simply allowing sequences

foo :: n:Nat -> m:Nat -> Nat/[n, m, n+m]

etc.

So the deal would be that the 'termination list' is
a sequence of LiquidHaskell Expr. The trick would
be to generalize the termination checker to accept
the above which should be possible...

@nikivazou says: Do you want to constraint this list to integers? Should we say

foo :: ls : [a] -> a/[ls]

OR

foo :: ls : [a] -> a/[len ls]

Btw, this is exactly what Xi is doing, for each rec function he defines a size that should be decreasing.

@gridaphobe says:

Yes, it seems like a good idea to accept any expression whose type has an associated termination measure. The benefit is that this approach allows both signatures, they would simply be equivalent. You could also write more interesting sizes like

foo :: xs:[a] -> ys:[a] -> a/[xs ++ ys]

Consistent naming of liquid tokens

SUGGESTION: Capitalizing all tokens and allow the old ones to be lower case for compatibility ( @ranjitjhala , @gridaphobe what do you think? ):

• "assume" -> "assume" | "ASSUME"
• "assert" -> "assert" | "ASSERT"
• "measure" -> "measure" | "MEASURE"
• "import" -> "import" | "IMPORT"
• "data" -> "data" | "DATA"
• "include" -> "include" | "INCLUDE"
• "invariant" -> "invariant" | "INVARIANT"
• "type" -> "type" | "TYPE"
• "predicate" -> "predicate" | "PREDICATE"
• "embed" -> "embed" | "EMBED"
• "qualif" -> "qualif" | "QUALIF"
• "Decrease" -> "DECREASE"
• "LAZYVAR"
• "Strict" [Replaced by "Lazy"]
• "Lazy" -> "LAZY"
• "LIQUID"