Subject: [alpine-devel] Bunch of questions about Alpine's gcc package

From: Dubiousjim <[email-masked]>
Date: Fri, 17 May 2013 13:07:20 -0400

These are questions in the first place for Natanael, but I'm sending to
list because others may also have insight, or wish to hear the answers.

I've been wrestling for a couple of weeks getting ghc (Glasgow Haskell
Compiler) to build on Alpine. I think I've succeeded, and I will write
up the directions and put them on the wiki. If we can put the binaries I
generated somewhere, then they can be used in an APKBUILD to bootstrap a
build of newer versions of ghc. (Ghc needs a relatively-recent binary of
ghc to build itself, else the compilation process is the more complex
and arduous process that I went through.)

Anyways, as part of this write-up, I'm writing up an explanation of how
I setup a cross-compiler inside an ArchLinux install (which, as it
happens, was in a chroot on my Alpine box) to build binaries targeting
Alpine. That's what the rest of this email is about; you can just ignore
the background about Haskell/ghc.

I set up my cross-compiler using recent versions of buildroot (both the
2013.02 and 2013.05-rc2 releases). There were a couple of frustrating
snags here, that I am trying to smooth out in my writeup.

One snag is that I had to do something like this on my Alpine box, to be
able to execute the binaries that the cross-compiler generated:

  $ sudo ln -s libc.so.0.9.32 /lib/libc.so
  $ sudo ln -s libuClibc-0.9.33.2.so /lib/libc.so.0
  $ sudo ln -s libmpc.so.3 /usr/lib/libmpc.so
  $ for f in ld-uClibc libcrypt libdl libm libpthread libresolv librt
  libubacktrace libutil; do sudo ln -s $f-0.9.33.2.so /lib/$f.so.0; done

Anyone have ideas about how to eliminate the need to do this?

I examined the different configuration options that buildroot supplies
when making its cross-compiling gcc, and the options that Alpine's
APKBUILD uses. I thought it'd be a good idea to make buildroot
approximate our native setup as closely as possible. However, to keep
things from getting too complicated, I'm going to try to avoid applying
the patches at
http://distfiles.gentoo.org/distfiles/gcc-4.7.1-piepatches-v${_piepatchver}.tar.bz2.

With that in mind, here are my questions about our present APKBUILD (I'm
looking at the APKBUILD for gcc-4.7.3-r6).

prepare() has the following:
        #PR33200
        sed -i 's/use_fixproto=yes/:/' gcc/config.gcc || return 1

This seems to be obsolete; see bottom of
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33200, also I've verified
that this is a noop on the gcc/config.gcc in the gcc-4.7.3 sources.

Our sources has:
        ...        
        11_all_default-warn-format-security.patch
        12_all_default-warn-trampolines.patch
        15_all_libgomp-Werror.patch
        16_all_libgo-Werror-pr53679.patch
        29_all_arm_armv4t-default.patch
        49_all_gcc-4.7-x86-libitm-pr52695.patch
        51_all_libiberty-pic.patch
        67_all_gcc-poison-system-directories.patch
        74_all_gcc47_cloog-dl.patch

        gcc-spec-env.patch
        pt_gnu_eh_frame.patch
        uclibc-getipinfo.patch
        gcc-4.7-dynamic-linker.patch
        gcc-4.6-pr32219.patch
        boehem-gc-uclibc.patch
        gcc-pure64.patch
        gcc-go.patch
        "

The first block of patches all seem to come from
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/src/patchsets/gcc/4.7.3/gentoo/,
however this doesn't include all of the patches available at that site.
Should I assume that the rest of the patches available there have been
reviewed and specifically rejected as not being relevant to Alpine, or
introducing undesired changes?

What about the patches available at
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/src/patchsets/gcc/4.7.3/uclibc/?
Buildroot includes all of those patches, but at a first glance, it seems
like we don't include them? Are first appearances misleading here?

What about the rest of the patches in our sources list: where do those
come from? Were they generated by Alpine devs, or do they come from
miscellaneous places? If anyone can venture a recommendation as to
whether I should apply those patches (and also the earlier block of
patches) to my cross-compiling gcc, I'd welcome the advice.

Finally, here are the configuration options we use (on x86 and x86_64):

        "$_gccdir"/configure --prefix=/usr \
                --mandir=/usr/share/man \
                --infodir=/usr/share/info \
                --build=${CBUILD} \
                --host=${CHOST} \
                --target=${CTARGET} \
                --with-pkgversion="Alpine ${pkgver}" \
                --disable-altivec \
                --disable-build-with-cxx \
                --disable-checking \
                --disable-fixed-point \
                --disable-libssp \
                --disable-libstdcxx-pch \
                --disable-multilib \
                --disable-nls \
                --disable-werror \
                --enable-__cxa_atexit \
                --enable-cld \
                --enable-esp \
                --enable-cloog-backend \
                --enable-languages=$_languages \
                --enable-shared \
                --enable-target-optspace \
                --enable-tls \
                --enable-threads \
                $(case "$CHOST" in
                  x86_64-*-uclibc) echo
                  "--with-dynamic-linker=ld64-uClibc.so.$_uclibc_abiver";;
                  *-uclibc) echo
                  "--with-dynamic-linker=ld-uClibc.so.$_uclibc_abiver";;
                esac) \
                --with-dynamic-linker-prefix=/lib \
                --with-system-zlib \
                --without-system-libunwind

Some of these configuration options are also used by buildroot, but a
few are different. I'm trying to understand the differences, and whether
I should leave them be, or try to get buildroot to more closely follow
Alpine's configuration choices.

One thing I notice is that Alpine configures uClibc without largefile
support:

    $ fgrep LARGEFILE main/libc0.9.32/uclibcconfig.x86*
    main/libc0.9.32/uclibcconfig.x86:# UCLIBC_HAS_FOPEN_LARGEFILE_MODE
    is not set
    main/libc0.9.32/uclibcconfig.x86_64:#
    UCLIBC_HAS_FOPEN_LARGEFILE_MODE is not set

Hence I did not select the BR2_TOOLCHAIN_BUILDROOT_LARGEFILE option in
buildroot's config, and so buildroot configures its gcc with
--disable-largefile. However, we don't specify that in Alpine's gcc
APKBUILD. Should we do so, given that we don't have largefile support
enabled in uClibc?

Another thing I notice is that buildroot doesn't set the
--with-dynamic-linker flags, and as a result the ldd output of binaries
it generates look a bit different from those generated by Alpine's
native gcc. This may be connected to the issue I mentioned at the
beginning, about needing to make symlinks for some library files on
Alpine, to be able to execute the binaries generated by the
cross-compiler.

These are options that Alpine uses when configuring gcc that I wasn't
able to find documentation on (for instance, at
http://gcc.gnu.org/install/configure.html). Might some of these options
now be obsolete? Or could anyone offer any explanation of these, and
suggestions about whether I should also apply them to the
cross-compiler. (I will continue to research these, as well.)

    --disable-altivec           # from what I could see, gcc's -maltivec
    option is only relevant for RS/6000 and PowerPC
    --disable-build-with-cxx
    --disable-libstdcxx-pch
    --enable-esp
    --without-system-libunwind

Alpine used the option --enable-__cxa_atexit but buildroot, at least
with the configuration options I selected, uses --disable-__cxa_atexit.
Any insight into this conflict?

Alpine uses all of these options (which I could find documentation
about) but buildroot currently does not:
    --disable-werror
    --disable-checking
    --disable-fixed-point
    --enable-cld
    --with-system-zlib

Alpine uses --enable-cloog-backend. I couldn't find info specifically
about this option, but I did find info about the options 
...
buildroot currently doesn't use any of these options.

buildroot currently uses these options (which I could find documentation
about) but Alpine's gcc does not:

    --disable-libquadmath
    --disable-libgomp
    --disable-decimal-float

Again, any insight into these conflicts? Should I make buildroot conform
to Alpine in these respects?

I know there's a lot of info/questions here. I'd be glad for any partial
answers, too.
-- 
[email-masked]

---
Unsubscribe:  [email-masked]
Help:         [email-masked]
---

---

From: Dubiousjim <[email-masked]>
Date: Fri, 17 May 2013 13:50:46 -0400

I can answer two of my questions:

> patches) to my cross-compiling gcc, I'd welcome the advice.

The boehem-gc-uclibc.patch turns out to be the same as
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/src/patchsets/gcc/4.7.3/uclibc/90_all_301-missing-execinfo_h.patch.

But the rest of the patches in that folder on sources.gentoo.org seem
not to be used by our APKBUILD, and I'm not sure where the rest of the
patches in our APKBUILD are coming from.

>     --without-system-libunwind

I found info on one of these, --enable-esp. This is used for the PIE
patchset, which I'm not using. See
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/src/patchsets/gcc/4.7.3/pie/README?revision=1.1&view=markup.

---

---

From: Natanael Copa <[email-masked]>
Date: Fri, 17 May 2013 20:47:04 +0200

On Fri, 17 May 2013 13:07:20 -0400
Dubiousjim <[email-masked]> wrote:

> These are questions in the first place for Natanael, but I'm sending
> to list because others may also have insight, or wish to hear the
> answers.

Hi, sorry for ignoring your previous emails. They came when i was short
on time and the they simply were to long and deep. :-/

It might help if you split up your questions in shorter emails (eg
part 1, part 2, etc)

> up the directions and put them on the wiki.

It would be nice.

>  If we can put the
> binaries I generated somewhere, then they can be used in an APKBUILD
> to bootstrap a build of newer versions of ghc. (Ghc needs a
> relatively-recent binary of ghc to build itself, else the compilation
> process is the more complex and arduous process that I went through.)

ugh, apps that needs bootstrapping are often problematic (gcc, and
openjdk)

> Anyways, as part of this write-up, I'm writing up an explanation of
> how I setup a cross-compiler inside an ArchLinux install (which, as it
> Alpine.

Sounds complicated. You cannot build a native compiler in alpine?

Crosscompiling is a pretty complicated chapter, as you noticed. Last
time we did it was when we enabled NPTL in uclibc iirc.

> That's what the rest of this email is about; you can just
> ignore the background about Haskell/ghc.
> 
> I set up my cross-compiler using recent versions of buildroot (both
> the 2013.02 and 2013.05-rc2 releases). There were a couple of
> frustrating snags here, that I am trying to smooth out in my writeup.
> 
> One snag is that I had to do something like this on my Alpine box, to
> be able to execute the binaries that the cross-compiler generated:
> 

what does readelf -d say that SONAME is set to? Compare that with what
SONAME is in a native alpine linux.

> things from getting too complicated, I'm going to try to avoid
> applying the patches at

Ok. That is for our hardened toolchain.

> With that in mind, here are my questions about our present APKBUILD
> (I'm looking at the APKBUILD for gcc-4.7.3-r6).
> 

Probably leftovers. Could you please create an issu for this on
bugs.a.o?
 
> however this doesn't include all of the patches available at that
> site. Should I assume that the rest of the patches available there
> have been reviewed and specifically rejected as not being relevant to
> Alpine, or introducing undesired changes?

Yes, I think I included some patches that I thought was not yet
relevant but probably soon would be (like the arm patches).

> Buildroot includes all of those patches, but at a first glance, it
> seems like we don't include them? Are first appearances misleading
> here?

I haven't looked over those, so I cannot really comment on those. I
think our patches and the gentoo uclibc patches were closer when we
separated from gentoo (alpine 1.9). I think we didn't need them. We
might have same patches or corresponding patches with different names.

> miscellaneous places? 

both I think.

> If anyone can venture a recommendation as to

>         gcc-spec-env.patch

for the piepatches. It allow us to disable the enforced hardened flags
with GCC_SPECS env var.

>         uclibc-getipinfo.patch

Those I don't remember.

>         gcc-4.7-dynamic-linker.patch

I think this will be critical for you. It enables the
--with-dynamic-linker flag.

We added this when uclibc broke ABI. uClibc does not even try to be ABI
compatible sine the target embeded. We want to be able to upgrade, not
only reinstall so we need some kind of ABI stability. the first
releases had soname libc.so.0. This is hardcoded in gcc sources. when
ABI broke we decided to set soname to libc.so.0.9.32 and add an
additional package named libc0.9.32 so both uclibc and libc0.9.32
packges could be installed in parallel while upgrading.

since then the ABI have not broken again.

>         gcc-4.6-pr32219.patch

I don't remember. might be something uclibc devs found and reported. It
probably bite us at some point.

>         boehem-gc-uclibc.patch

I think this was to be able to compile stdc++ or gcc java.

>         gcc-pure64.patch

This patch dropped the /lib64 and /lib32 dirs for multiarch.

>         gcc-go.patch

Something for go-lang.

...

> I know there's a lot of info/questions here. I'd be glad for any
> partial answers, too.

I'll stop here for now. I'll try comment the rest in other email.

The --with-dynamic-linker is important as explained above. You will
also set a variable when building uclibc, ABI something.

I think it will be a good time for me to look over the toolchain soon.
Will upgrade gcc to 4.8.

Crosscompiling for ARM is also a hot topic, and so is musl instead of
uclibc.

Thanks!

-nc

---

---

From: Natanael Copa <[email-masked]>
Date: Sun, 19 May 2013 13:08:29 +0200

I'll try answer some of the more questions you had...

>     UCLIBC_HAS_FOPEN_LARGEFILE_MODE is not set

UCLIBC_HAS_FOPEN_LARGEFILE_MODE:

Answer Y to enable a uClibc-specific extension to allow passingan
additional 'F' flag in the mode string for fopen() to specify that the
file should be open()ed with the O_LARGEFILE flag set.

Most people will answer N.

> APKBUILD. Should we do so, given that we don't have largefile support

We have large file support.
UCLIBC_HAS_LFS=y

> --with-dynamic-linker flags, and as a result the ldd output of
> binaries it generates look a bit different from those generated by
> Alpine's native gcc. This may be connected to the issue I mentioned
> at the beginning, about needing to make symlinks for some library
> files on Alpine, to be able to execute the binaries generated by the
> cross-compiler.

Yes. Also this is relevant to get the Alpine Linux uclibc

        # set abi version and remove unsupported warnings c flag
        sed -i -e "s/^ABI_VERSION.*/ABI_VERSION := $_abiver/" \
                -e "s/-Wold-style-declaration//g" \
                Rules.mak

You need the gcc-4.7-dynamic-linker.patch and you need to set the
--with-dynamic-linker=0.9.32 which is our current ABI version.

> http://gcc.gnu.org/install/configure.html). Might some of these
> options now be obsolete? Or could anyone offer any explanation of
> these, and suggestions about whether I should also apply them to the
>     --disable-altivec           # from what I could see, gcc's
> -maltivec option is only relevant for RS/6000 and PowerPC

I think its unrelevant yes.

>     --disable-build-with-cxx

I think this was to disable the c++ version of gcc. I think it broke
the gcc plugins for our grsecurity kernel. I am not sure if grsecurity
has resolved those issues or what the status is there.

>     --disable-libstdcxx-pch

not sure.

>     --enable-esp

to enable the genetoo hardened patches. 

> with the configuration options I selected, uses
> --disable-__cxa_atexit. Any insight into this conflict?

I don't remember. Looks like it was introduced with 7302771c, the
initial gcc commit?

> about) but buildroot currently does not:

This is to disable the exit with error when hit a gcc warning.

>     --disable-checking

dont rememember.

>     --disable-fixed-point

I don't think this is relevant to us and could probably be removed.
--enable-fixed-point
--disable-fixed-point
    Enable (or disable) support for C fixed-point arithmetic. This
    option is enabled by default for some targets (such as MIPS) which
    have hardware-support for fixed-point operations. On other targets,
    you may enable this option manually. 

>     --enable-cld

http://en.wikipedia.org/wiki/Direction_flag
I think this was the reason:
http://lwn.net/Articles/272048/

>     --with-system-zlib

link against the system libz rather than use the embedded version in
gcc. It makes our gcc slightly smaller but makes it depend on zlib.

> buildroot currently uses these options (which I could find
> documentation about) but Alpine's gcc does not:
> 
>     --disable-libquadmath
>     --disable-libgomp
>     --disable-decimal-float
> 
> Again, any insight into these conflicts? Should I make buildroot
> conform to Alpine in these respects?

I'm not sure, but remeber hat buildroot targets many more targets than
us.
 
> I know there's a lot of info/questions here. I'd be glad for any

I could mention that when we did the NPTL bootstrapping we used
crosstool-ng rather than buildroot. Main reason was that buildroot does
(did?) not provide us with a native gcc.

-nc

---