mitchty / alpine-linux-ghc-bootstrap Goto Github PK

Did this a while back, but it was smaller as a static binary than dynamic. If that is still the case, why not build it as a fully static binary anyway.

So turns out the profiling archive libraries are... huge. Example:

# find /usr -name "*_p.a" -type f -exec du -ks {} \; | awk '{s+=$1} END {print s/1024}'
375.348

This is about 40% of the size of the ghc package just for profile libraries. But not having them is even worse.

That is a lot of space just to make debugging possible. I'm also guessing most people don't care/need profiling libraries. I however have needed them like crazy for this port to debug things and when you need them, you need them now. Waiting to recompile ghc and all its submodules is not what I'd wish on my worst enemy.

So a thought on how this might play out. The only way to have ghc with profiled libraries is to compile it that way. Basically this means ghc will have to be compiled twice (ugg this already takes forever to bootstrap and its only getting worse with llvm).

So maybe we have ghc which is non profiled, and have ghc-full which has the profiled things replace regular ghc if it is there.

Need to test this out and figure out how it effects stack/cabal.

It's not necessarily a bug, however I think it's worth sharing here…

I'm trying to install ghcjs on the image using stack setup – it fails, as stack requires GNU tar (corresponding issue commercialhaskell/stack#2146). The workaround for now is to install GNU tar (apk add tar) and the build will start.

I'm trying to use the new alpine package using docker with a Dockerfile as:

FROM alpine:edge
RUN apk add --no-cache ghc
CMD ["sh"]

The result is:

Sending build context to Docker daemon 2.048 kB
Step 1/3 : FROM alpine:edge
 ---> 9655ad68a6c9
Step 2/3 : RUN apk add --no-cache ghc
 ---> Running in 2fdff1fee961
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  ghc (missing):
    required by: world[ghc]
The command '/bin/sh -c apk add --no-cache ghc' returned a non-zero code: 1

Can anyone point me in the right direction to use this?

Just as a note, need to update the bootstrap compiler to 8.0.2. I have this mostly working, but am waiting on arm to build things.

Once tested I'll update the repo and docker images/apks. It drastically simplifies this port since ghc bug 9007 is fixed and gcc supports --no-pie in alpine linux 3.5. Makes my job way easier.

I'm seeing this error

Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2: unable to decommit memory: Invalid argument

when using the 8.0.2 container as a base. Seems to be related to https://ghc.haskell.org/trac/ghc/ticket/12495. Not sure if theres a patch that needs to be applied to ghc 8.0.2 to fix it.

Need to validate if the gold linker is a better idea to use with ghc.

So this was a bit of an experiment, but it seems to work. In either case, the next release of ghc 7.10 will be using llvm 3.6 in the package.

This is more fallout of getting the 8.0 release candidate building, it makes it infinitely easier to have 7.10 use the same llvm backend as 8.0. Also made building the bootstrap compiler easier and should help with the arm port.

Current progress here:
https://github.com/mitchty/alpine-linux-ghc-bootstrap/compare/llvm-3.6-experiment2

Semi related to the 8.0.1 rc, should have a way to have builds off the master branch in git.

Have to do the following:

• Get llvm 3.6 building like I did 3.5 to ensure we don't have issues with say 3.7 or 3.8 or whatever fancy llvm version breaks building via ghc
• Figure out what else I don't know needs to happen.

I'm not sure if the last bootstrapped ghc was built with the latest 7.10.2 source, but I figured I'd mention that it was released just in case: https://mail.haskell.org/pipermail/haskell/2015-July/024634.html

Right so this is to put myself on record committing to getting this done and hopefully finding out how long it takes me to complete.

So currently the ghc apks use gcc and technically if you pass -fllvm to ghc to compile, the llvm backend would attempt to be used. But, it will never work as alpine linux is shipping llvm 3.6.1 if I remember right. The last version to ship llvm 3.5.2 was 3.1, and well it has a bug I'd need to patch anyway so work is needed.

So to simplify things, switch to using clang as the c compiler/preprocessor away from gcc. I'm not totally sold on this but clang is way faster to compile things generally. Which for testing out things on arm under qemu has been extremely helpful. Will see how this pans out though, it may not end up in the final spiel

But the biggest change will be to build llvm 3.5.2 for 7.10.3 for llc and opt. I have this seemingly working now (waiting for stage 2 compiler to finish building.

To make this whole thing a bit simpler however, the packages will be laid out a bit different.

Current:

• ghc
• ghc-doc
• cabal-install
• cabal-install-doc
• stack
• stack-doc

Biggest change will be that the cabal-install* packages will go the way of the dodo entirely and will become a sub package of ghc. Also ghc will no longer include any *-dev packages but only the bare minimum to run. A new package ghc-dev will include the llvm I build and any reasonable dev packages related to ghc development on alpine linux. Currently, this means: linux-headers, gmp-dev, ncurses-dev, zlib-dev, and musl-dev. Why zlib/ncurses/gmp-dev, well ghc is build with all three packages, linux-headers is needed for network related packages, and finally musl-dev has everything you'd need for c includes. Any others is up to the user to sort out/debug.

So layout after this change the packages will be:

• ghc
• ghc-dev
• ghc-doc
• cabal
• stack
• stack-doc

The primary reason for the changes are mostly to enable porting to arm where llvm is really the only option for building, and to prepare for ghc 8.0 which will similarly have llvm binaries for opt/llc included in the tier 1 binary distributions. And finally to decouple llvm away from the system packages which may be too new to use.

This should also reduce install size a skosh if you just wanted to run ghc/ghci. But the primary cause of the ghc install being huge is installing/compiling all of the profiling libraries. I could remove that, but then I couldn't build profiled versions of things via cabal for debugging (which is really annoying not to be able to do). I think an issue for that itself is in order. Not building profiling libraries if you don't need them would greatly reduce the install size. Just the base/cabal/ghc libraries with profiling alone are 45/72/174 mebibytes in size respectively. Anyway, just rambling now will open a new issue for profiling on its own.

Making this an issue to compel me to get this upstream by hook or by crook.

Hi,

It seems that calling a wrapped callback to haskell code from C causes a segfault. GHC 8.0.1 on other linux platforms doesn't seem to have this issue. The following program should reproduce the problem:

Save as proba.hs:

{-# LANGUAGE ForeignFunctionInterface #-}                                                        

import Foreign.C.Types
import Foreign.Ptr

main :: IO () 
main = do                                                                                        
  f <- wrapIt (\c -> print "hola" >> return (c*c))                                               
  print f 
  c_callMe f
  freeHaskellFunPtr f

foreign import ccall safe "callMe"                                                               
  c_callMe :: FunPtr (CDouble -> IO CDouble) -> IO ()                                            

foreign import stdcall safe "wrapper"                                                            
  wrapIt :: (CDouble -> IO CDouble) -> IO (FunPtr (CDouble -> IO CDouble))   

Save as `c_proba.c``

#include <stdio.h>

typedef double (*callback)(double n);                                                            

void callMe(callback fun)
{   
  printf("%lx\n", fun);
  double n= 34.2;
  printf("The number is: %f", fun(n));
}   

Build:

ghc -o proba proba.hs c_proba.c

Disable PAX protection (else GRSEC will kill it since the callback is created in the heap by the RTS):

paxctl -Cpemrxs proba

And execute:

./proba

Output on my system:

0x0000000000b1bf48
b1bf48
Segmentation fault (core dumped)

Note that the pointer address is correctly passed to C, interestingly the stack trace shows this address is being called twice (the last call causing the segfault)

gdb -c core
(.....)
(gdb) bt
#0  0x0000000000b1bf48 in ?? ()
#1  0x000000000040a229 in ?? ()
#2  0x404119999999999a in ?? ()
#3  0x0000000000b1bf48 in ?? ()
#4  0x0000000000b17bd8 in ?? ()
#5  0x404119999999999a in ?? ()
#6  0x0000000200005350 in ?? ()
#7  0x0000000000409b9e in ?? ()
#8  0x000000000086b1c0 in ?? ()
#9  0x0000000000000000 in ?? ()

Hello,

When using the ghc 8 image and doing apk update && apk upgrade.
It complains that APKINDEX.tar.gz does not exist at the path
"https://s3-us-west-2.amazonaws.com/alpine-ghc/next/8.0/"

Is the image outdated ?

I tried to compile a hello world program with latest mitchty/alpine-ghc docker image (and libc-dev package installed for math.h), and encountered the following error.

<no location info>: error:
    Warning: Couldn't figure out linker information!
             Make sure you're using GNU ld, GNU gold or the built in OS X linker, etc.
gcc: error: unrecognized command line option '-nopie'; did you mean '-no-pie'?
gcc: error: unrecognized command line option '-nopie'; did you mean '-no-pie'?
`gcc' failed in phase `Linker'. (Exit code: 1)

I had this almost working with 7.10.2 but ran into problems with the shared libraries not having been built right due to missing symbols in the haskell runtime shared library.

Figure out a way to get an arm port of 7.10 at least up. Been working on this off and on the past 6 months. Not in a huge rush. Help appreciated if anyone is ambitious, will push up a branch once I figure out what version kinda/sorta worked.

GHC sometimes trips up with PAX when compiling since (I hypothesize) it needs to JIT code. I haven't copied the exact error I got (can do if necessary) but dmesg showed something along the lines of `grsec: prevented mmap of ... by ghc ... blah blah blah). It doesn't occur always but it can be reproduced on a system running Alpine's GRSEC kernel and running "stack install yesod".

I fixed it with a big sledgehammer by adding the following lines at the end of the package function in GHC's APKBUILd:

  paxctl -cpemrxs $pkgdir/usr/lib/ghc-$pkgver/bin/ghc
  paxctl -cpemrxs $pkgdir/usr/lib/ghc-$pkgver/bin/ghc-iserv
  paxctl -cpemrxs $pkgdir/usr/lib/ghc-$pkgver/bin/ghc-iserv-dyn
  paxctl -cpemrxs $pkgdir/usr/lib/ghc-$pkgver/bin/ghc-iserv-profsh

I'm pretty sure not all flags need to be disabled but I think at least MPROTECT (m) needs to.
I'll reproduce and paste the logs.

The README seems to imply that these are part of the docker image:

 ghc ghc-dev cabal alpine-sdk linux-headers musl-dev gmp-dev zlib-dev

but according to the Dockerfile only ghc cabal stack are installed.

Why not include them? It would be nice if all the build packages were part of the base image.

to create static binaries w/ out GPL hangups