Message-Id: <1365010513.18734.140661212891825.28A1A367@webmail.messagingengine.com>
From: Dubiousjim <lists+alpine-devel@jimpryor.net>
To: Alpine <alpine-devel@lists.alpinelinux.org>
Precedence: list
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain
Subject: [alpine-devel] Porting ghc to Alpine, troubles with PaX?
Date: Wed, 03 Apr 2013 13:35:13 -0400

Hi, I've been working on cross-compiling GHC (Glasgow Haskell Compiler)
to Alpine. It's been real struggle, because of the massive complexity of
the GHC build system. You have to have a working GHC somewhere to either
generate C files from the GHC sources, or to cross-compile for your
target system. I haven't been able to get the port-from-C-files setup to
work. After a lot of fiddling, I do think I've gotten the
cross-compiling to work. However, now I'm hitting some new problems that
I think stem not from GHC's build system but rather from its interaction
with PaX and so on on Alpine. I could really use some help from those of
you who understand the PaX setup and Alpine's toolchain better.

So what I've done is install Arch Linux in a chroot in my target Alpine
system. Then I used buildroot (v 2013.02) to install a cross-compiler
toolchain and a few libraries that need to be there to build ghc. I used
pretty recent versions of the kernel headers, gcc, binutils, uClibc, and
so on here. If it's important I can lookup the details. I used the
uClibc config file that Alpine uses when building uClibc for the
cross-compiler. I didn't do anything to customize the build of gcc for
the cross-compiler setup. When I see where I end up, I suspect this may
have been a mistake. We'll come back to this.

Anyway, the cross-compiler seems to work alright.

    $ echo 'int main(void) {return 0;}' > test.c
    $ i686-buildroot-linux-uclibc-gcc -o test test.c # inside my host
    chroot
    $ ls test
    test
    $ ./test
   -bash: ./test: No such file or directory

That's the funny error message I get when trying to run an executable
that can't find some library it needs. This is as expected, since we
compiled test to run on the target system, not the host. It does run
fine from the target. (I would think that if I set LD_LIBRARY_PATH to
the right directory in the cross-compiler setup, I should also be able
to run this executable on the host, but I haven't been able to get that
to work.)

(I'm saying "on the target" and "on the host" for brevity, but these are
really the same machine, it's just that "from the host" means running
inside the ArchLinux chroot on that machine.)

Ok, so as I said I'm able to build a cross-compiling GHC on the host
(using ArchLinux's binary GHC to do so). It builds a "ghc-stage2"
compiler which ought to run on the target. One hitch is that when I try
to run this on the target, I get:

>     $ inplace/lib/ghc-stage2 -B./inplace/lib --interactive
>     GHCi, version 7.6.2: http://www.haskell.org/ghc/  :? for help
>     Loading package ghc-prim ... ghc-stage2: mmap 442368 bytes at (nil):
>     Operation not permitted
>     ghc-stage2: Try specifying an address with +RTS -xm<addr> -RTS

I've seen this kind of error before, with a compiler I built natively in
Alpine for a different language (namely, Pure). I learned that I had to
install the paxctl utility on Alpine and disable some of the PaX
protections to get the binary to run. If I do to GHC what I did to Pure:

    $ paxctl -c inplace/lib/ghc-stage2
    $ paxctl -m inplace/lib/ghc-stage2

then the GHC interpreter will in fact run properly on Alpine:

    $ inplace/bin/ghc-stage2 --interactive
    GHCi, version 7.6.2: http://www.haskell.org/ghc/  :? for help
    Loading package ghc-prim ... linking ... done.
    Loading package integer-gmp ... linking ... done.
    Loading package base ... linking ... done.
    Prelude>

Yay! This is after a lot of work. The next step should be to use this
GHC to do a native build of GHC on the target system. If that works,
then we could make an APKBUILD and others could build GHC themselves, so
long as they downloaded my initial GHC binary to bootstrap.

However, we can't get there yet. This is because although the GHC
interpreter we just cross-compiled works OK, something is funny about
the GHC compiler. And of course one needs the compiler to work right in
order to natively *compile* GHC on the target.

What is funny with the compiler? Well, it builds binaries ok but then
they can't execute:
    $ cat ~/hello.hs
    main = putStrLn "Hello World!\n"
    $ rm -f hello
    $ inplace/bin/ghc-stage2 -B./inplace/lib ~/hello.hs -o hello
    [1 of 1] Compiling Main             ( /home/jim/hello.hs,
    /home/jim/hello.o ) [flags changed]
    Linking hello ...
    $ ./hello
    Can't modify application's text section; use the GCC option -fPIE
    for position-independent executables.

I think what's happening here is that GHC is building position-dependent
binaries, but they can't run on Alpine because of the PaX protections.
Using paxctl doesn't help here, one has to make sure that GHC generates
position-independent binaries. GHC doesn't understand the -fPIE flag,
but it does allegedly honor -fPIC on x86 and x86_64. However, my first
attempts to supply that flag to GHC still ended up producing
position-dependent code. (If you did "readelf -d hello" you'd still see
TEXTREL sections.)

What I think the issue was here was that some flag has to be set when
configuring and building GHC to enable the ability to generate PIC.
(Shame there's no warning if you supply the -fPIC flag to a GHC that
wasn't so built.) I haven't found good documentation of this, but I did
find some posts which suggested this at least used to be true. And it
would explain what was happening here.

So I started building GHC again (for the twentieth or thirtieth time),
this time specifying at the beginning that I wanted to build all the
libraries and the compiler and so on with -fPIC.

However, this time around, before the build finishes there's a point
where two libraries can't be built because the ghc-stage2 that we've
built is dying. (The build process which is mainly running on the host
has to farm a few tasks, like this one, out to the target system. So we
want to be executing ghc-stage2 on the target system. But it's not
working.)

Just doing the "paxctl -c ...; paxctl -m ..." as before doesn't fix
this. Fiddling with various paxctl options, I found that I'll always get
a "Killed" message when executing ghc-stage2 unless I've done "paxctl
... -s ..." on ghc-stage2. In that case, I'll get a segfault instead;
except if I've done "paxctl ... -sr ..." then I won't get a segfault;
instead my CPU will freeze up. I can still switch to other terminals and
so on, I guess using a different CPU, but I can't kill the frozen
process or even reboot my machine. I have to press and hold the power
button to forcefully power down.

Ok, so obviously something is going funny with the interaction between
ghc and the PaX stuff here. But my understanding of all this is very
thin. One way forward might be that I've somehow misconfigured the -fPIC
capacity in GHC, and that I've got to use different configuration
settings to enable that. Another thing that might be going on here is
that the gcc in my cross-compiler setup doesn't have all the same bells
and configuration whistles as the gcc on my target Alpine system. When I
look inside the main/gcc folder of the aports tree, I see a lot of
patches and config options supplied to gcc when building it that I don't
think are also be applied to my cross-compiling gcc. Maybe when setting
up the cross-compiler to target my Alpine system, I have to instead do
more to properly configure gcc so that it generates position-independent
code to run properly under Alpine's PaX stuff. My simple test binary
built ok on the cross-compiler and ran on Alpine, but more complex code
might not do so.

If either of these hypotheses is right, I'm going to need help from
others, because I've reached the limits of how much I understand here.
The GHC folks aren't going to know much about PaX and configuring gcc to
cross-compile for Alpine and so on. But maybe someone here can help me
figure that out? Independently of ghc, does one have to do anything
special when building a cross-compiling gcc on another (glibc-based,
same architecture) distro to target Alpine?



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---