commercialhaskell / stackage Goto Github PK

Should be trivial to implement: just have a queue and spawn off some workers (worker count depends on number of cores).

highlighting-kate-0.5.3.6 (Brent Yorgey [email protected] @jgm) cannot use:

• blaze-html-0.6.0.0 -- >=0.4.2 && <0.6

hoogle-4.2.15 (Neil Mitchell) cannot use:

• conduit-1.0.0 -- >=0.2 && <0.6
• wai-1.4.0 -- >=1.1 && <1.4

pandoc-1.10.1 (Brent Yorgey [email protected]) cannot use:

• blaze-html-0.6.0.0 -- ==0.5.*

tagstream-conduit-0.5.3 ([email protected] @yihuang) cannot use:

• blaze-builder-conduit-1.0.0 -- ==0.5.*
• conduit-1.0.0 -- ==0.5.*

hoogle-4.2.14 (Neil Mitchell @ndmitchell) cannot use:

• case-insensitive-1.0 -- >=0.2 && <0.5

http-types-0.7.3.0.1 ([email protected] @aristidb) cannot use:

• case-insensitive-1.0 -- >=0.2 && <0.5

The test suite for hspec-1.4.2.2 fails; it appears that the test suite is trying to manipulate environment variables under GHCi, which will probably never work.

1) Test.Hspec.FailureReport.GHCi keeps environment variables on :reload FAILED
expected: "\"bar\"\n"
 but got: "*** Exception: FOO: getEnv: does not exist (no environment variable)\n"

When I build stackage on Mac OS X, I routinely get the following build failures.

localhost:stackage ekmett$ stackage build
Creating a build plan
Printing build plan to build-plan.log
Wiping out old sandbox folder
No mismatches, starting the sandboxed build.
Sandbox built, beginning individual test suites
Test suite failed: http-conduit-1.8.4.5([email protected])
Test suite failed: http-reverse-proxy-0.1.0.6([email protected])
Test suite failed: network-conduit-0.6.1.1(Neil Mitchell)
stackage: There were failures, please see the logs in runtests

I created separate issues on the http-conduit (snoyberg/http-conduit#80) and http-reverse-proxy (fpco/http-reverse-proxy#1) repositories directly with the relevant log fragments, but interestingly there seemed to be no log from the network-conduit failure.

recursion-schemes-3.0.0.1 (Edward Kmett [email protected] @ekmett) cannot use:

• free-3.4 -- >=3.2 && <3.4

Note that the following items are only failures when building against GHC HEAD. There is no urgency in addressing them. Relevant mailing list discussion: https://groups.google.com/forum/?fromgroups=#!topic/stackage/6P5d9KD0yDU

BlogLiterately-0.5.4 (Brent Yorgey <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.0 && <4.7

BlogLiterately-diagrams-0.1.1.2 (Brent Yorgey <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.3 && <4.7

ChasingBottoms-1.3.0.5 (Edward Kmett <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.0 && <4.7

active-0.1.0.3 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.0 && <4.7

ad-3.4 (Edward Kmett <[email protected]> @ekmett) cannot use: 
- template-haskell-2.9.0.0 -- >=2.5 && <2.9

case-insensitive-1.0 ([email protected] @basvandijk) cannot use: 
- base-4.7.0.0 -- >=3 && <4.7

cmdargs-0.10.1 (Vincent Hanquez) cannot use: 
- process-1.2.0.0 -- >=1.0 && <1.2

diagrams-builder-0.3 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

diagrams-cairo-0.6 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

diagrams-contrib-0.6.0.3 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

diagrams-core-0.6.0.1 (Brent Yorgey <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

diagrams-lib-0.6.0.1 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

diagrams-svg-0.6.0.1 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.3 && <4.7

dual-tree-0.1.0.1 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.3 && <4.7

force-layout-0.2 (Brent Yorgey <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

generic-deriving-1.4.0 (Brent Yorgey <[email protected]> @dreixel) cannot use: 
- template-haskell-2.9.0.0 -- >=2.4 && <2.9

hamlet-1.1.6.1 ([email protected]) cannot use: 
- process-1.2.0.0 -- >=1.0 && <1.2

hoogle-4.2.15 (Neil Mitchell) cannot use: 
- Cabal-1.17.0 -- >=1.8 && <1.17

hspec-expectations-0.3.0.3 ([email protected] @sol) cannot use: 
- base-4.7.0.0 -- <4.7

hyphenation-0.2.1.7 (Edward Kmett <[email protected]> @ekmett) cannot use: 
- directory-1.2.0.1 -- >=1.0 && <1.2

lifted-base-0.2.0.2 ([email protected] @basvandijk) cannot use: 
- base-4.7.0.0 -- >=3 && <4.7

monad-control-0.3.1.4 ([email protected] @basvandijk) cannot use: 
- base-4.7.0.0 -- >=3 && <4.7

monoid-extras-0.2.2.2 (Brent Yorgey <[email protected]> @diagrams) cannot use: 
- base-4.7.0.0 -- >=4.3 && <4.7

optparse-applicative-0.5.2.1 ([email protected] @pcapriotti) cannot use: 
- process-1.2.0.0 -- ==1.1.*

pandoc-1.10.1 (Brent Yorgey <[email protected]>) cannot use: 
- process-1.2.0.0 -- >=1 && <1.2

shakespeare-1.0.3 ([email protected]) cannot use: 
- process-1.2.0.0 -- >=1.0 && <1.2

shakespeare-css-1.0.2 ([email protected]) cannot use: 
- process-1.2.0.0 -- >=1.0 && <1.2

split-0.2.1.2 ([email protected]) cannot use: 
- base-4.7.0.0 -- <4.7

vector-space-points-0.1.2.0 (Brent Yorgey <[email protected]>) cannot use: 
- base-4.7.0.0 -- >=4.0 && <4.7

xmlgen-0.4.0.3 (Stefan Wehr <[email protected]> @skogsbaer) cannot use: 
- base-4.7.0.0 -- >=4.2 && <4.7

Today, I took some time to study the code and got a better picture about what it is doing, and I have ideas that might make it even more stable and predictable.

It seems that stackage does three things:

1. Provides a selection of packages with some „stable“ attribute attached.
2. Provides a way to verify this combination of packages is a good one.
3. Provides a repository that can be used with cabal-install by users who do not want to see build features.

At the moment, these three steps are combined in one “build“ command. Also, there is not much one can do to influence step 1 besides listing root packages. So here is my suggestion:

Step 1 produces a file explicitly listing all selected packages with their version, similar to build-plan.log. The format could even be a .cabal file, with the packages listed in the dependencies list.

Step 2 and step 3 the work on this file.

This would allow for a change in workflow: We centrally keep track of this stackage.cabal file, possibly in this repository. There would be a stackage update command that would, based on the previous stackage.cabal file, try to upgrade as many packages as possible to their newest version on hackage so that all package relations are still fulfilled. After verifying that this combination still builds and passes the tests (stackage verify), the updated stackage.cabal can be checked in.

Advantages:

• stackage update can verify that version numbers are monotonously increasing compared to the previous stackage.cabal file. This is important if distributions are to base their work on stackage.
• One can observe the change in allowed packages in the GitHub changelog.
• There is always the latest known-to-work state reported. So even when the latest packages on hackage are temporarily broken, one can still build stackage.
• Given a stackage.cabal file and its previous version, stackage verify could only build and test changed packages (and their reverse dependencies), making it less work to submit a new or update packages.
• We have queryable data e.g. for the Distributions field on hackage or pages like http://people.debian.org/~nomeata/platform.html

certificate-1.3.6 (Vincent Hanquez @vincenthz) cannot use:

• crypto-pubkey-types-0.3.0 -- ==0.2.*

cryptocipher-0.4.0 (Vincent Hanquez @vincenthz) cannot use:

• crypto-pubkey-types-0.3.0 -- ==0.2.*

Stackage breaks on a mac, after a fresh clean install :
It looks like the latest platform does do use the same places to store file, especially :

stackage.hs: /Users/luc/.cabal/packages/hackage.haskell.org/00-index.tar: openBinaryFile: does not exist (No such file or directory)

context:
I had a fresh re install of the HP to go 64, and cleared all my .cabal and hopefully all GHC and lib instance around

MAC OS 10.8, Haskell Platform 2012.4.0.0 for Mac OS X, 64 bit

cd /Users/luc/fasthaskell
Last login: Wed Mar 20 10:39:56 on ttys004
You have mail.
mbp2-de-luc:~ luc$ cd /Users/luc/fasthaskell
mbp2-de-luc:fasthaskell luc$ cabal update
Downloading the latest package list from hackage.haskell.org
mbp2-de-luc:fasthaskell luc$ cabal install cabal-dev
Resolving dependencies...
Downloading setenv-0.1.0...
Configuring setenv-0.1.0...
Building setenv-0.1.0...
Preprocessing library setenv-0.1.0...
[1 of 1] Compiling System.SetEnv ( src/System/SetEnv.hs, dist/build/System/SetEnv.o )
[1 of 1] Compiling System.SetEnv ( src/System/SetEnv.hs, dist/build/System/SetEnv.p_o )
In-place registering setenv-0.1.0...
Running Haddock for setenv-0.1.0...
Preprocessing library setenv-0.1.0...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: rts-1.0
Haddock coverage:
67% ( 2 / 3) in 'System.SetEnv'
Documentation created: dist/doc/html/setenv/index.html
Installing library in
/Users/luc/Library/Haskell/ghc-7.4.2/lib/setenv-0.1.0/lib
Registering setenv-0.1.0...
Installed setenv-0.1.0
Downloading cabal-dev-0.9.2...
[1 of 1] Compiling Main ( /var/folders/wv/r_3xq7s12573p1s44c_9v43c0000gx/T/cabal-dev-0.9.2-1336/cabal-dev-0.9.2/Setup.hs, /var/folders/wv/r_3xq7s12573p1s44c_9v43c0000gx/T/cabal-dev-0.9.2-1336/cabal-dev-0.9.2/dist/setup/Main.o )
Linking /var/folders/wv/r_3xq7s12573p1s44c_9v43c0000gx/T/cabal-dev-0.9.2-1336/cabal-dev-0.9.2/dist/setup/setup ...
Configuring cabal-dev-0.9.2...
Building cabal-dev-0.9.2...
Preprocessing executable 'ghc-pkg-6_8-compat' for cabal-dev-0.9.2...
[1 of 1] Compiling Main ( src/GhcPkgCompat.hs, dist/build/ghc-pkg-6_8-compat/ghc-pkg-6_8-compat-tmp/Main.o )
Linking dist/build/ghc-pkg-6_8-compat/ghc-pkg-6_8-compat ...
Preprocessing executable 'fake-ghc-cabal-dev' for cabal-dev-0.9.2...
[1 of 2] Compiling Distribution.Dev.GhcArgs ( src/Distribution/Dev/GhcArgs.hs, dist/build/fake-ghc-cabal-dev/fake-ghc-cabal-dev-tmp/Distribution/Dev/GhcArgs.o )
[2 of 2] Compiling Main ( src/FakeGhc.hs, dist/build/fake-ghc-cabal-dev/fake-ghc-cabal-dev-tmp/Main.o )
Linking dist/build/fake-ghc-cabal-dev/fake-ghc-cabal-dev ...
Preprocessing executable 'cabal-dev' for cabal-dev-0.9.2...
[ 1 of 19] Compiling Distribution.Dev.MergeCabalConfig ( src/Distribution/Dev/MergeCabalConfig.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/MergeCabalConfig.o )
[ 2 of 19] Compiling Distribution.Dev.RewriteCabalConfig ( src/Distribution/Dev/RewriteCabalConfig.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/RewriteCabalConfig.o )
[ 3 of 19] Compiling Distribution.Dev.GhcArgs ( src/Distribution/Dev/GhcArgs.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/GhcArgs.o )
[ 4 of 19] Compiling Distribution.Dev.Utilities ( src/Distribution/Dev/Utilities.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/Utilities.o )
[ 5 of 19] Compiling Distribution.Dev.InterrogateCabalInstall ( src/Distribution/Dev/InterrogateCabalInstall.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/InterrogateCabalInstall.o )
[ 6 of 19] Compiling Distribution.Dev.TH.DeriveCabalCommands ( src/Distribution/Dev/TH/DeriveCabalCommands.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/TH/DeriveCabalCommands.o )
[ 7 of 19] Compiling Paths_cabal_dev ( dist/build/autogen/Paths_cabal_dev.hs, dist/build/cabal-dev/cabal-dev-tmp/Paths_cabal_dev.o )
[ 8 of 19] Compiling Distribution.Dev.CabalInstall ( src/Distribution/Dev/CabalInstall.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/CabalInstall.o )
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Loading package bytestring-0.9.2.1 ... linking ... done.
Loading package zlib-0.5.4.0 ... linking ... done.
Loading package array-0.4.0.0 ... linking ... done.
Loading package deepseq-1.3.0.0 ... linking ... done.
Loading package containers-0.4.2.1 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package filepath-1.3.0.0 ... linking ... done.
Loading package old-locale-1.0.0.4 ... linking ... done.
Loading package old-time-1.1.0.0 ... linking ... done.
Loading package unix-2.5.1.1 ... linking ... done.
Loading package directory-1.1.0.2 ... linking ... done.
Loading package time-1.4 ... linking ... done.
Loading package tar-0.4.0.1 ... linking ... done.
Loading package setenv-0.1.0 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package text-0.11.2.3 ... linking ... done.
Loading package parsec-3.1.3 ... linking ... done.
Loading package network-2.3.1.0 ... linking ... done.
Loading package HTTP-4000.2.5 ... linking ... done.
Loading package process-1.1.0.1 ... linking ... done.
Loading package Cabal-1.16.0.3 ... linking ... done.
cabal --help
Interrogating cabal-install executable:
cabal install --help
cabal install --help
cabal update --help
cabal list --help
cabal info --help
cabal fetch --help
cabal unpack --help
cabal check --help
cabal sdist --help
cabal upload --help
cabal report --help
cabal init --help
cabal configure --help
cabal build --help
cabal copy --help
cabal haddock --help
cabal clean --help
cabal hscolour --help
cabal register --help
cabal test --help
cabal bench --help
cabal upgrade --help
cabal help --help
[ 9 of 19] Compiling Distribution.Dev.Flags ( src/Distribution/Dev/Flags.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/Flags.o )
[10 of 19] Compiling Distribution.Dev.Sandbox ( src/Distribution/Dev/Sandbox.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/Sandbox.o )
[11 of 19] Compiling Distribution.Dev.InitPkgDb ( src/Distribution/Dev/InitPkgDb.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/InitPkgDb.o )
[12 of 19] Compiling Distribution.Dev.Command ( src/Distribution/Dev/Command.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/Command.o )
[13 of 19] Compiling Distribution.Dev.AddSource ( src/Distribution/Dev/AddSource.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/AddSource.o )
[14 of 19] Compiling Distribution.Dev.InvokeCabal ( src/Distribution/Dev/InvokeCabal.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/InvokeCabal.o )
[15 of 19] Compiling Distribution.Dev.InstallDependencies ( src/Distribution/Dev/InstallDependencies.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/InstallDependencies.o )
[16 of 19] Compiling Distribution.Dev.BuildOpts ( src/Distribution/Dev/BuildOpts.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/BuildOpts.o )
[17 of 19] Compiling Distribution.Dev.Ghci ( src/Distribution/Dev/Ghci.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/Ghci.o )
[18 of 19] Compiling Distribution.Dev.GhcPkg ( src/Distribution/Dev/GhcPkg.hs, dist/build/cabal-dev/cabal-dev-tmp/Distribution/Dev/GhcPkg.o )
[19 of 19] Compiling Main ( src/Main.hs, dist/build/cabal-dev/cabal-dev-tmp/Main.o )
Linking dist/build/cabal-dev/cabal-dev ...
Warning: No documentation was generated as this package does not contain a
library. Perhaps you want to use the --executables flag.
Installing executable(s) in
/Users/luc/Library/Haskell/ghc-7.4.2/lib/cabal-dev-0.9.2/bin
Installed cabal-dev-0.9.2
Updating documentation index /Users/luc/Library/Haskell/doc/index.html
mbp2-de-luc:fasthaskell luc$ git clone https://github.com/fpco/stackage
Cloning into 'stackage'...
remote: Counting objects: 816, done.
remote: Compressing objects: 100% (347/347), done.
remote: Total 816 (delta 587), reused 677 (delta 449)
Receiving objects: 100% (816/816), 105.01 KiB, done.
Resolving deltas: 100% (587/587), done.
mbp2-de-luc:fasthaskell luc$ cd stackage
mbp2-de-luc:stackage luc$ git submodule update --init # get the Haskell Platform files
Submodule 'haskell-platform' (https://github.com/haskell/haskell-platform.git) registered for path 'haskell-platform'
Cloning into 'haskell-platform'...
remote: Counting objects: 2213, done.
remote: Compressing objects: 100% (761/761), done.
remote: Total 2213 (delta 1455), reused 2192 (delta 1434)
Receiving objects: 100% (2213/2213), 1.95 MiB | 1.04 MiB/s, done.
Resolving deltas: 100% (1455/1455), done.
Submodule path 'haskell-platform': checked out '73a58050d86cef941fc82a82d52e70c906785b7f'
mbp2-de-luc:stackage luc$ runghc app/stackage.hs select
Loading Haskell Platform
Loading package database
stackage.hs: /Users/luc/.cabal/packages/hackage.haskell.org/00-index.tar: openBinaryFile: does not exist (No such file or directory)
mbp2-de-luc:stackage luc$

hlint-1.8.41 (Neil Mitchell) cannot use: 
- cpphs-1.16 -- >=1.11 && <1.16

Pinging @ndmitchell

BlogLiterately-0.6 (Brent Yorgey [email protected] @byorgey) cannot use:

• lens-3.9.0.1 -- ==3.8.*

hashable-extras-0.1 (Edward Kmett [email protected] @analytics) cannot use:

• generic-deriving-1.5.0 -- ==1.4.*

lens-3.9.0.1 (Brent Yorgey [email protected] @ekmett) cannot use:

• generic-deriving-1.5.0 -- ==1.4.*

Perhaps I am missing something.
Today I tried to run:

[stackage]$ ~/.cabal/bin/stackage build
Creating a build plan
Printing build plan to build-plan.log
Wiping out old sandbox folder
cabal: Could not resolve dependencies:
next goal: fsnotify (user goal)
rejecting: fsnotify-0.0.4, 0.0.3 (global constraint requires ==0.0.2)
trying: fsnotify-0.0.2
next goal: hinotify (user goal)
rejecting: hinotify-0.3.5 (conflict: fsnotify => hinotify==0.3.2)
rejecting: hinotify-0.3.4, 0.3.3, 0.3.2, 0.3.1, 0.3, 0.2, 0.1 (global
constraint requires ==0.3.5)
Resolving dependencies...
cabal returned a bad result, exiting

active-0.1.0.2 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- >=0.1 && <0.9

adjunctions-3.0.0.1 (Edward Kmett <[email protected]>) cannot use: 
- distributive-0.3 -- >=0.2.2 && <0.3

diagrams-core-0.6 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- >=0.3.4 && <0.9

diagrams-lib-0.6 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- >=0.3.4 && <0.9

dual-tree-0.1.0.0 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- ==0.8.*

lens-3.7.2 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- >=0.8.4 && <0.9

linear-0.4.2.1 (Edward Kmett <[email protected]>) cannot use: 
- distributive-0.3 -- >=0.2.2 && <0.3

monoid-extras-0.2.2.1 (Brent Yorgey <[email protected]>) cannot use: 
- semigroups-0.9 -- ==0.8.*

Paging @byorgey @ekmett

Instituting upper bounds.

pandoc-1.10.1 (Brent Yorgey [email protected]) cannot use:

• Diff-0.3.0 -- ==0.2.*

Pinging @jgm

Tried following instructions to build stackage just now, but was informed that I needed to install tar from hackage first. Did so, and things seem to be building fine now. NBD, but probably should fix either the instructions or the dependencies or something?

This isn't technically a problem yet, since the Haskell Platform specifies older versions of these packages, but it will cause problems for testing on 7.6.2 and could cause user issues.

ChasingBottoms-1.3.0.5 (Neil Mitchell) cannot use:

• QuickCheck-2.6 -- >=2.1 && <2.6
• syb-0.4.0 -- >=0.1.0.2 && <0.4

active-0.1.0.3 (Brent Yorgey [email protected] @diagrams) cannot use:

• QuickCheck-2.6 -- >=2.4.2 && <2.6

blaze-html-0.6.1.0 (Jasper Van der Jeugt @jaspervdj) cannot use:

• QuickCheck-2.6 -- >=2.4 && <2.6

blaze-markup-0.5.1.4 (Jasper Van der Jeugt @jaspervdj) cannot use:

• QuickCheck-2.6 -- >=2.4 && <2.6

diagrams-contrib-0.6.0.3 (Brent Yorgey [email protected] @diagrams) cannot use:

• QuickCheck-2.6 -- >=2.4 && <2.6

doctest-0.9.5 (Edward Kmett [email protected] @sol) cannot use:

• syb-0.4.0 -- ==0.3.*

stylish-haskell-0.5.6.0 (Jasper Van der Jeugt @jaspervdj) cannot use:

• syb-0.4.0 -- ==0.3.*

test-framework-quickcheck2-0.3.0.1 (Stefan Wehr [email protected]) cannot use:

• QuickCheck-2.6 -- >=2.4 && <2.6

uuid-1.2.9 (Antoine Latter) cannot use:

• QuickCheck-2.6 -- >=2.4 && <2.6

README.md still says to just runghc app/stackage.hs build. Can you update the build steps documentation so I can configure my Jenkins build to match?

lens-3.6 in stackage fails to pass its tests due to compiler errors. The relevant portion of a log file is below: I'd attach the whole log if I could figure out how to attach files here. The problem appears to be due to version mismatch in the containers package. Sorry for no patch; couldn't quite figure out how to get there.

Preprocessing test suite 'doctests' for lens-3.6...
[1 of 1] Compiling Main             ( tests/doctests.hs, dist/build/doctests/doctests-tmp/Main.o )
Linking dist/build/doctests/doctests ...
Running 4 test suites...
Test suite doctests: RUNNING...

src/Control/Lens/IndexedTraversal.hs:240:17:
    Couldn't match type `v' with `Maybe v'
      `v' is a rigid type variable bound by
          the type signature for
            traverseMin :: (Indexed Int k, Applicative f) =>
                           k (v -> f v) (IntMap v -> f (IntMap v))
          at src/Control/Lens/IndexedTraversal.hs:240:3
    Expected type: k (v -> f v) (IntMap v -> f (IntMap v))
      Actual type: k (v -> f (Maybe v)) (IntMap v -> f (IntMap v))
    In the expression:
      index
      $ \ f m
          -> case IntMap.minViewWithKey m of {
               Just ((k, a), _) -> (\ v -> IntMap.updateMin (const v) m) <$> f k a
               Nothing -> pure m }
    In an equation for `traverseMin':
        traverseMin
          = index
            $ \ f m
                -> case IntMap.minViewWithKey m of {
                     Just ((k, a), _) -> (\ v -> ...) <$> f k a
                     Nothing -> pure m }

src/Control/Lens/IndexedTraversal.hs:261:17:
    Couldn't match type `v' with `Maybe v'
      `v' is a rigid type variable bound by
          the type signature for
            traverseMax :: (Indexed Int k, Applicative f) =>
                           k (v -> f v) (IntMap v -> f (IntMap v))
          at src/Control/Lens/IndexedTraversal.hs:261:3
    Expected type: k (v -> f v) (IntMap v -> f (IntMap v))
      Actual type: k (v -> f (Maybe v)) (IntMap v -> f (IntMap v))
    In the expression:
      index
      $ \ f m
          -> case IntMap.maxViewWithKey m of {
               Just ((k, a), _) -> (\ v -> IntMap.updateMax (const v) m) <$> f k a
               Nothing -> pure m }
    In an equation for `traverseMax':
        traverseMax
          = index
            $ \ f m
                -> case IntMap.maxViewWithKey m of {
                     Just ((k, a), _) -> (\ v -> ...) <$> f k a
                     Nothing -> pure m }
Test suite doctests: FAIL

esqueleto-1.0.6 (Felipe Lessa [email protected] @meteficha) cannot use:

• hspec-1.5.1 -- >=1.3 && <1.5

BlogLiterately-diagrams-0.1.1.1 (Brent Yorgey [email protected]) cannot use:

• diagrams-builder-0.3 -- ==0.2.*

@byorgey

We probably won't be able to extract what the failure message is, but the user could be directed to the log file.

hspec-1.4.3 ([email protected] @hspec) cannot use:

• ansi-terminal-0.6 -- ==0.5.*

hspec-meta-1.4.3 ([email protected] @hspec) cannot use:

• ansi-terminal-0.6 -- ==0.5.*

pandoc-1.10.0.5 (Brent Yorgey [email protected]) cannot use:

• ansi-terminal-0.6 -- ==0.5.*

Pinging @jgm and @sol

Hi,

it seems that both "stackage check" and "stackage build" require a specific version of the cabal binary, but only "stackage build" checks this.

Also, the error message should be clearer and indicate that it is the binary cabal in the path whose version is insufficient (and not the locally installed Cabal library, or the Cabal library in the build plan).

Thanks,
Joachim

The following packages are affected:

• blaze-html
• blaze-markup
• cipher-aes
• cryptocipher
• cryptohash
• lens
• lifted-base
• optparse-applicative
• pem
• stylish-haskell

Paging @jaspervdj @vincenthz @ekmett @basvandijk @pcapriotti

I'm implementing a temporary upper bound on test-framework.

This is not my library. I just thought that since you already have base-unicode-symbols, it was a bit silly to ignore containers-unicode-symbols.

If it passes the second time, flag it as a problem to be reported, but don't fail the entire build.

diagrams-lib-0.6.0.1 (Brent Yorgey [email protected] @diagrams) cannot use:

• NumInstances-1.2 -- ==1.0.*

A few stackage packages failed to build for me because my version of happy was too old, causing a cascade of errors. Not sure who or what is at fault here, but thought it worth reporting.

$ schroot -c haskell -- dist/build/stackage/stackage build --no-platform
Loading Haskell Platform
Loading package database
Narrowing package database
Printing build plan to build-plan.log
machines-0.2.1.1 (Edward Kmett <[email protected]>) cannot use: 
- profunctors-3.1 -- ==3.0.*

stackage: Conflicting build plan, exiting

(Are we supposed to file such bugs here? Can I notify @ekmett this ways?)

Use case: gtk2hs-buildtools package contains gtk2hsC2hs executable.

Stackage fails to build:

$ schroot -c haskell -- dist/build/stackage/stackage build --no-platform
Loading Haskell Platform
Loading package database
Narrowing package database
Printing build plan to build-plan.log
yesod-1.1.7 ([email protected]) cannot use: 
- optparse-applicative-0.5.0 -- ==0.4.*

stackage: Conflicting build plan, exiting

@snoyberg, this would be yours, it seems.

We should probably prevent QuickCheck from printing anything to stderr, if it is not a terminal.

$  runghc app/stackage.hs check
Checking Cabal version
Checking build plan
cabal: Could not resolve dependencies:
trying: BlogLiterately-0.6.0.1
trying: directory-1.2.0.1/installed-b1a...
rejecting: cabal-install-1.16.0.2, 1.16.0.1, 1.16.0 (global constraint
requires ==0.14.0)
rejecting: cabal-install-0.14.0 (conflict: directory =>
unix==2.6.0.1/installed-feb..., cabal-install => unix>=1.0 && <2.6)
rejecting: cabal-install-0.10.2, 0.10.0, 0.8.2, 0.8.0, 0.6.4, 0.6.2, 0.6.0,
0.5.2, 0.5.1, 0.5.0, 0.4.0 (global constraint requires ==0.14.0)
Resolving dependencies...
cabal returned a bad result, exiting

Here is additonal information:

$ ghc-pkg list
/usr/lib/ghc-7.6.2/package.conf.d
   Cabal-1.16.0
   array-0.4.0.1
   base-4.6.0.1
   bin-package-db-0.0.0.0
   binary-0.5.1.1
   bytestring-0.10.0.2
   containers-0.5.0.0
   deepseq-1.3.0.1
   directory-1.2.0.1
   filepath-1.3.0.1
   ghc-7.6.2
   ghc-prim-0.3.0.0
   haskell2010-1.1.1.0
   haskell98-2.0.0.2
   hoopl-3.9.0.0
   hpc-0.6.0.0
   integer-gmp-0.5.0.0
   old-locale-1.0.0.5
   old-time-1.1.0.1
   pretty-1.1.1.0
   process-1.1.0.2
   rts-1.0
   template-haskell-2.8.0.0
   time-1.4.0.1
   unix-2.6.0.1
~/.ghc/x86_64-linux-7.6.2/package.conf.d
   tar-0.4.0.1
   transformers-0.3.0.0

authenticate-1.3.2 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- >=0.6 && <0.8

hoogle-4.2.14 (Neil Mitchell @ndmitchell) cannot use:

• http-types-0.8.0 -- ==0.7.*

wai-1.3.0.1 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

wai-extra-1.3.2 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

wai-test-1.3.0 ([email protected]) cannot use:

• http-types-0.8.0 -- ==0.7.*

warp-1.3.7.1 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

yesod-1.1.7.2 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

yesod-core-1.1.7.2 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

yesod-static-1.1.1.2 ([email protected] @yesodweb) cannot use:

• http-types-0.8.0 -- ==0.7.*

yesod-test-0.3.3 ([email protected]) cannot use:

• http-types-0.8.0 -- ==0.7.*

Right now all packages must come from Hackage. For local testing and for distributions, it would be convenient to somehow override a package from Hackage, such as to apply patches which are not yet available upstream.

Currently stackage check (and stackage build) fails for me, and it seems that this is due to the existing package in the global data base:

$ dist/build/stackage/stackage check 
Checking build plan
cabal: Could not resolve dependencies:
trying: crypto-api-0.10.2/installed-0b3...
rejecting: largeword-1.0.4 (conflict: crypto-api =>
largeword==1.0.3/installed-8a7...)
rejecting: largeword-1.0.3/installed-8a7..., 1.0.3, 1.0.2, 1.0.1, 1.0.0
(global constraint requires ==1.0.4)
Resolving dependencies...
cabal returned a bad result, exiting

Shoudn’t the result of stackage be isolated from the remaining system (maybe up to core libraries)?

diagrams-lib-0.6.0.2 (Brent Yorgey [email protected] @diagrams) cannot use:

• NumInstances-1.3 -- >=1.0 && <1.3

skein-1.0.1 ([email protected] @meteficha) cannot use:

• crypto-api-0.12 -- >=0.6 && <0.12

uname -a yields

Linux LVN513-9 3.1.10-1.16-desktop #1 SMP PREEMPT Wed Jun 27 05:21:40 UTC 2012 (d016078) x86_64 x86_64 x86_64 GNU/Linux

I am running OpenSUSE, I think 12.1. (I don't actually manage this machine.)

The log of the failure is as follows:

Unpacking to unix-time-0.1.4/
Resolving dependencies...
Configuring unix-time-0.1.4...
Warning: Packages using 'cabal-version: >= 1.10' must specify the
'default-language' field for each component (e.g. Haskell98 or Haskell2010).
If a component uses different languages in different modules then list the
other ones in the 'other-languages' field.
configure: loading site script /usr/share/site/x86_64-unknown-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for strptime_l... yes
configure: creating ./config.status
config.status: creating cbits/config.h
Building unix-time-0.1.4...
Preprocessing library unix-time-0.1.4...
[1 of 5] Compiling Data.UnixTime.Types ( Data/UnixTime/Types.hs, dist/build/Data/UnixTime/Types.o )
[2 of 5] Compiling Data.UnixTime.Diff ( Data/UnixTime/Diff.hs, dist/build/Data/UnixTime/Diff.o )
[3 of 5] Compiling Data.UnixTime.Sys ( dist/build/Data/UnixTime/Sys.hs, dist/build/Data/UnixTime/Sys.o )
[4 of 5] Compiling Data.UnixTime.Conv ( Data/UnixTime/Conv.hs, dist/build/Data/UnixTime/Conv.o )
[5 of 5] Compiling Data.UnixTime    ( Data/UnixTime.hs, dist/build/Data/UnixTime.o )
In-place registering unix-time-0.1.4...
Preprocessing test suite 'spec' for unix-time-0.1.4...
[1 of 2] Compiling UnixTimeSpec     ( test/UnixTimeSpec.hs, dist/build/spec/spec-tmp/UnixTimeSpec.o )
[2 of 2] Compiling Main             ( test/Spec.hs, dist/build/spec/spec-tmp/Main.o )
Linking dist/build/spec/spec ...
Preprocessing test suite 'doctests' for unix-time-0.1.4...
[1 of 1] Compiling Main             ( test/doctests.hs, dist/build/doctests/doctests-tmp/Main.o )
Linking dist/build/doctests/doctests ...
Running 2 test suites...
Test suite spec: RUNNING...

UnixTime
  formatUnixTime
  ���������Falsifiable (after 1 test)...�����������������������������                             �����������������������������    - behaves like the model FAILED [1]

  parseUnixTimeGMT & formatUnixTimeGMT
    - inverses the result

  addUnixDiffTime & diffUnixTime
    - invrses the result

1) UnixTime.formatUnixTime behaves like the model FAILED
*** Failed! Falsifiable (after 1 test): 
UnixTime {utSeconds = 1571502693, utMicroSeconds = 548856}


Finished in 0.0375 seconds
3 examples, 1 failure
Test suite spec: FAIL
Test suite logged to: dist/test/unix-time-0.1.4-spec.log
Test suite doctests: RUNNING...
Test suite doctests: PASS
Test suite logged to: dist/test/unix-time-0.1.4-doctests.log
1 of 2 test suites (1 of 2 test cases) passed.

Happy to provide any further info that might be useful in tracking this down.

crypto-api-0.11 (Felipe Lessa [email protected] @TomMD) cannot use:

• tagged-0.5 -- >=0.1 && <0.5

skein-1.0.0 ([email protected] @meteficha) cannot use:

• tagged-0.5 -- >=0.2 && <0.5

From @nomeata:

The result from version 2 can also be integrated into hackage as another
Distribution; the format required can be seen on
http://people.debian.org/~nomeata/cabalDebianMap.txt.

A couple of packages in current stackage (specifically GLUT and hscurses) failed to build for me, because they FFI-wrap OS libraries and I did not have the dev version of these libraries installed, so the C code wouldn't compile. Not sure what can or should be done about this problem, but it seems worth pursuing. In my fantasy world, the build of these packages would automatically install Debian packages for me; more realistically, it would be nice if the missing stuff was picked up at config type rather than build time, although that is still hard.

E.g.

$ runghc app/stackage.hs build --exclude GLUT --exclude OpenGL

should not build GLUT and OpenGL unless an other package depends on them.

My specific use case: I have access to a machine with many CPUs, but I can't install GLUT/OpenGL dev dependencies there.

Pinging @sol on this.

I'm trying to automate a bunch of tests of Hackage. The installation of packages always takes place in a sandbox. Unfortunately, this means that the doctest-based tests do not run:

Test suite doctests: RUNNING...

Data/Conduit.hs:79:8:
    Could not find module `Data.Void'
    Perhaps you meant
      Data.Word (from base)
      Data.Word (needs flag -package haskell2010-1.1.0.1)
    Use -v to see a list of the files searched for.
Test suite doctests: FAIL
Test suite logged to: dist/test/conduit-0.5.5-doctests.log
Test suite test: RUNNING...
Test suite test: PASS
Test suite logged to: dist/test/conduit-0.5.5-test.log
1 of 2 test suites (1 of 2 test cases) passed.

Is there some way to specify the location of the sandbox to doctest so that it can use it for testing?

The following packages have restrictive upper bounds due to the new hashable 1.2 release (plus test-framework for siphash):

case-insensitive-0.4.0.3 ([email protected]) cannot use: 
- hashable-1.2.0.0 -- >=1.0 && <1.2

compressed-3.0 (Edward Kmett <[email protected]>) cannot use: 
- hashable-1.2.0.0 -- >=1.1.2.1 && <1.2

concurrent-supply-0.1.3 (Edward Kmett <[email protected]>) cannot use: 
- hashable-1.2.0.0 -- ==1.1.*

lens-3.7.0.2 (Edward Kmett <[email protected]>) cannot use: 
- hashable-1.2.0.0 -- >=1.1.2.3 && <1.2

reducers-3.0.0.1 (Edward Kmett <[email protected]>) cannot use: 
- hashable-1.2.0.0 -- >=1.1.2.1 && <1.2

siphash-1.0.2 (Edward Kmett <[email protected]>) cannot use: 
- test-framework-0.8 -- >=0.3.3 && <0.7
- test-framework-quickcheck2-0.3.0.1 -- >=0.2.9 && <0.3

uniplate-1.6.9 (Neil Mitchell) cannot use: 
- hashable-1.2.0.0 -- >=1.1.2.3 && <1.2

vault-0.2.0.1 (Neil Mitchell) cannot use: 
- hashable-1.2.0.0 -- ==1.1.*

Paging @basvandijk @ekmett @vincenthz @ndmitchell @HeinrichApfelmus

I'm going to institute temporary upper bounds.

BlogLiterately-diagrams-0.1.1 (Brent Yorgey <[email protected]>) cannot use:
- pandoc-1.10 -- ==1.9.*

pandoc-1.10 (Brent Yorgey <[email protected]>) cannot use:
- test-framework-0.8 -- >=0.3 && <0.7
- test-framework-hunit-0.3.0 -- ==0.2.*
- test-framework-quickcheck2-0.3.0.1 -- >=0.2.9 && <0.3

Pinging @byorgey and @jgm

diagrams-builder-0.3 (Brent Yorgey [email protected] @diagrams) cannot use:

• cryptohash-0.9.0 -- >=0.7.4 && <0.9

uuid-1.2.9 (Antoine Latter) cannot use:

• cryptohash-0.9.0 -- >=0.7 && <0.9

Paging @byorgey and @aslatter