Message-Id: <1365218189.31993.140661214031209.2343358C@webmail.messagingengine.com>
From: Dubiousjim <lists+alpine-devel@jimpryor.net>
To: glasgow-haskell-users@haskell.org,
 Alpine <alpine-devel@lists.alpinelinux.org>
Cc: gtener@gmail.com
Precedence: list
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain
In-Reply-To: <1365217958.31433.140661214023117.2CAF5030@webmail.messagingengine.com>
References: <1365217958.31433.140661214023117.2CAF5030@webmail.messagingengine.com>
Subject: [alpine-devel] Re: porting to uClibc-based 686 Linux
Date: Fri, 05 Apr 2013 23:16:29 -0400

Am posting this follow-up to two mailing lists, because I've inquired
about it on both. Previous messages are at:
  *
  http://www.haskell.org/pipermail/glasgow-haskell-users/2013-April/023912.html
  * http://lists.alpinelinux.org/alpine-devel/1699.html

I have seemed to successfully cross-compile a stage2 ghc for my target
system, an i686 Linux based on uClibc. One issue I don't think I've
mentioned before is that the target system uses a kernel with grsecurity
and PaX, and that involves patching the toolchain as well. However my
cross-compiling toolchain doesn't have any such tweaks applied. This
will be relevant below. Another potential issue is that my host system,
a different i686 Linux based on glibc, is running inside a chroot on my
target system. I don't think this is in fact responsible for any
problems, but I thought I should mention it.

So I last wrote on the glasgow-haskell mailing list:

On Wed, Apr 03, 2013 at 10:55:14AM -0400, Dubiousjim wrote:
> But there's still something funny with the binaries generated by the
> compiler. That means I can't yet use this ghc to directly recompile ghc
> on my target system.
> 
>     $ cat ~/hello.hs
>     main = putStrLn "Hello World!\n"
>     $ rm -f hello
>     $ inplace/bin/ghc-stage2 -B./inplace/lib ~/hello.hs -o hello
>     [1 of 1] Compiling Main             ( /home/jim/hello.hs,
>     /home/jim/hello.o ) [flags changed]
>     Linking hello ...
>     $ ./hello
>     Can't modify application's text section; use the GCC option -fPIE
>     for
>     position-independent executables.
> 
> Seems like I'm close, but not quite there yet.
> 
> Supplying the flag -fPIE to ghc when building hello says:
> 
>     ghc-stage2: unrecognised flags: -fPIE
> 
> The flag -fPIC on the other hand is accepted, but then the generated
> binary hello still fails with the same error.

Krzysztof replied to me offline (thanks Krzysztof!), but I still haven't
managed to get this to work.

I started thinking the difficulty came from the grsecurity or PaX
features of my target system. Maybe the toolchain on the target system
had been configured in a way to build binaries that would work ok on
that system, but ghc wasn't yet so configured, and that's why the
binaries compiled by ghc don't work. I wrote to my target system's
mailing list (alpine-devel@lists.alpinelinux.org) expressing this fear.

There was also some evidence against this hypothesis. (Test binaries I
built using the cross-compiling toolchain, which hadn't been tweaked in
the way the toolchain on the target system had, did build binaries that
executed fine on the target; but perhaps this was just because the test
programs were too simple.) But since I didn't know what was going on,
this seemed worth exploring.

However, I'm pretty confident I've now ruled this explanation out. Even
if I build and boot from a vanilla kernel, binaries generated by the
ghc-stage2 in question still fail with the error message I reported.

So then I started trying to track down the error message in any sources
or binaries on my system. Couldn't find it kernel or ghc sources. I did
find it though in the source code for the uClibc library. It's from
lines 614-681 of the file uClibc-0.9.33.2/ldso/ldso/ldso.c:

----------------------------- start -----------------------------------
    /* At this point we are now free to examine the user application,
     * and figure out which libraries are supposed to be called.  Until
     * we have this list, we will not be completely ready for dynamic
     * linking.
     */

    /* Find the runtime load address of the main executable.  This may
    be
     * different from what the ELF header says for ET_DYN/PIE
     executables.
     */
    {
        unsigned int idx;
        ElfW(Phdr) *phdr = (ElfW(Phdr) *) auxvt[AT_PHDR].a_un.a_val;

        for (idx = 0; idx < auxvt[AT_PHNUM].a_un.a_val; idx++, phdr++)
            if (phdr->p_type == PT_PHDR) {
                DL_INIT_LOADADDR_PROG(app_tpnt->loadaddr,
                auxvt[AT_PHDR].a_un.a_val - phdr->p_vaddr);
                break;
            }

        if (DL_LOADADDR_BASE(app_tpnt->loadaddr))
            _dl_debug_early("Position Independent Executable: "
                    "app_tpnt->loadaddr=%x\n",
                    DL_LOADADDR_BASE(app_tpnt->loadaddr));
    }

    /*
     * This is used by gdb to locate the chain of shared libraries that
     are
     * currently loaded.
     */
    debug_addr = _dl_zalloc(sizeof(struct r_debug));

    ppnt = (ElfW(Phdr) *) auxvt[AT_PHDR].a_un.a_val;
    for (i = 0; i < auxvt[AT_PHNUM].a_un.a_val; i++, ppnt++) {
        if (ppnt->p_type == PT_GNU_RELRO) {
            relro_addr = ppnt->p_vaddr;
            relro_size = ppnt->p_memsz;
        }
        if (!app_mapaddr && (ppnt->p_type == PT_LOAD)) {
            app_mapaddr = DL_RELOC_ADDR (app_tpnt->loadaddr,
            ppnt->p_vaddr);
        }
        if (ppnt->p_type == PT_DYNAMIC) {
            dpnt = (ElfW(Dyn) *) DL_RELOC_ADDR(app_tpnt->loadaddr,
            ppnt->p_vaddr);
            _dl_parse_dynamic_info(dpnt, app_tpnt->dynamic_info,
            debug_addr, app_tpnt->loadaddr);
#ifndef __FORCE_SHAREABLE_TEXT_SEGMENTS__
            /* Ugly, ugly.  We need to call mprotect to change the
             * protection of the text pages so that we can do the
             * dynamic linking.  We can set the protection back
             * again once we are done.
             */
            _dl_debug_early("calling mprotect on the application
            program\n");
            /* Now cover the application program. */
            if (app_tpnt->dynamic_info[DT_TEXTREL]) {
                ElfW(Phdr) *ppnt_outer = ppnt;
                ppnt = (ElfW(Phdr) *) auxvt[AT_PHDR].a_un.a_val;
                for (i = 0; i < auxvt[AT_PHNUM].a_un.a_val; i++, ppnt++)
                {
                    if (ppnt->p_type == PT_LOAD && !(ppnt->p_flags &
                    PF_W))
                        _dl_mprotect((void *)
                        (DL_RELOC_ADDR(app_tpnt->loadaddr,
                        ppnt->p_vaddr) & PAGE_ALIGN),
                                 (DL_RELOC_ADDR(app_tpnt->loadaddr,
                                 ppnt->p_vaddr) & ADDR_ALIGN) +
                                 (unsigned long) ppnt->p_filesz,
                                 PROT_READ | PROT_WRITE | PROT_EXEC);
                }
                ppnt = ppnt_outer;
            }
#else
            if (app_tpnt->dynamic_info[DT_TEXTREL]) {
                _dl_dprintf(_dl_debug_file, "Can't modify application's
                text section; use the GCC option -fPIE for
                position-independent executables.\n");
                _dl_exit(1);
            }
#endif
-----------------------------  end  -----------------------------------

and indeed, the ghc-stage2 I built does seem to be generating
position-dependent code, EVEN WHEN I TELL IT -fPIC. Whereas the native
ghc on my host system doesn't do so, even when -fPIC is omitted.
Witness:

    host$ which ghc
    /usr/bin/ghc
    host$ cat hello.hs
    main = putStrLn "Hello World!\n"
    host$ /usr/bin/ghc -o hello-host hello.hs
    [1 of 1] Compiling Main             ( hello.hs, hello.o )
    Linking hello-host ...

There we've compiled hello-host using the host's native ghc.

Now we switch over to the target system (sharing the same filesystem,
and cding to the same directory):

    target$ rm -f hello.o hello.hi
    target$ inplace/bin/ghc-stage2 -o hello-cross hello.hs
    [1 of 1] Compiling Main             ( hello.hs, hello.o )
    Linking hello-cross ...
    target$ ./hello-cross
    Can't modify application's text section; use the GCC option -fPIE
    for position-independent executables.
    target$ rm -f hello.o hello.hi hello-cross
    target$ inplace/bin/ghc-stage2 -fPIC -o hello-cross hello.hs
    [1 of 1] Compiling Main             ( hello.hs, hello.o )
    Linking hello-cross ...
    target$ ./hello-cross
    Can't modify application's text section; use the GCC option -fPIE
    for position-independent executables.
    target$ readelf -d hello-host | fgrep TEXTREL
    target$ readelf -d hello-cross | fgrep TEXTREL
     0x00000016 (TEXTREL)                    0x0

So something has gone wrong with the way I cross-compiled or have
configured this ghc-stage2. It won't build position-independent code
whereas the native ghc running on my host system will. So I come back to
the ghc folks for advice. I hope some readers might make it this far and
have suggestions or advice about things I might try.

-- 
dubiousjim@gmail.com



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---