~alpine/devel

For discussion of Alpine Linux development and developer support

8 3

[alpine-devel] musl and ARM in 2.7 (and an introduction)

Leslie P. Polzer | PORT ZERO
Details
Message ID
<5260F6C9.9000505@port-zero.com>
Sender timestamp
1382086345
DKIM signature
missing
Download raw message
Hi,

I'm new to Alpine. I've been around GNU/Linux for about 15 years
now as user and developer. During this time I have been working
on various open source projects and have been a community contributor
to Linux From Scratch and ArchLinux.

First let me say that you've done an incredible job! I especially
like the package management system, but there's many more excellent
features that I wouldn't really know where to start. :)

I've come to understand that Alpine 2.7 will be the first version
to feature an ARM port, based on musl as libc. Timo Teräs' efforts
in the past weeks (as per Redmine) also show this. Is this official?

In that case I would like to ask why musl is being used. It certainly
is a great alternative to uclibc, but it appears that the ARM port
is linked to it. I'm seeking to understand the reasoning behind this.

Also, is there an established way already of building Alpine based
on musl and/or cross-compiling for ARM? It doesn't seem to be a big
deal, but I'd like to know the efforts that have already been put
into this so I'm not duplicating any work.

Thanks!

  Leslie

-- 
Dipl.Ing.(BA) Leslie P. Polzer | CTO - PORT ZERO
UG (haftungsbeschränkt)
Softwareentwicklung & IT Engineering
Adalbertstraße 7/8
10999 Berlin
Tel.: +49 (0)30 - 69 200 907 - 0
Fax: +49 (0)30 - 69 200 907 - 9
http://www.port-zero.com


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20131018131615.5286bb75@vostro>
In-Reply-To
<5260F6C9.9000505@port-zero.com> (view parent)
Sender timestamp
1382091375
DKIM signature
missing
Download raw message
On Fri, 18 Oct 2013 10:52:25 +0200
"Leslie P. Polzer | PORT ZERO" <polzer@port-zero.com> wrote:

> I'm new to Alpine. I've been around GNU/Linux for about 15 years
> now as user and developer. During this time I have been working
> on various open source projects and have been a community contributor
> to Linux From Scratch and ArchLinux.

Nice. Welcome!

> First let me say that you've done an incredible job! I especially
> like the package management system, but there's many more excellent
> features that I wouldn't really know where to start. :)

Thanks :)

> I've come to understand that Alpine 2.7 will be the first version
> to feature an ARM port, based on musl as libc. Timo Teräs' efforts
> in the past weeks (as per Redmine) also show this. Is this official?

Yes, I am currently doing ARM port. It is undecided if we want to do a
release of it, since it's quite experimental at this point.

> In that case I would like to ask why musl is being used. It certainly
> is a great alternative to uclibc, but it appears that the ARM port
> is linked to it. I'm seeking to understand the reasoning behind this.

musl is a lot cleaner, smaller faster and more standards compliant in
many cases. It is also easier to bootstrap to new targets compared to
uclibc/glibc.

Though, uclibc is likely more glibc compatible. We have also quite some
uclibc specific bugs open that are not fixable without doing invasive
uclibc patching. E.g. openjdk7 will not run on standard uclibc. You can
also note that we carry about 40 patches to uclibc already because it
is not maintained as well as glibc/musl. We also have had to fix
ourselves numerous bugs in it, and many of them have been upstreamed.

In the long run we are currently planning to drop uclibc, and replace
it with musl in the primary builds for Alpine 3.0. In case we do not
get everything done for next release date, we might still do
Alpine 2.8 based on uclibc.

> Also, is there an established way already of building Alpine based
> on musl and/or cross-compiling for ARM? It doesn't seem to be a big
> deal, but I'd like to know the efforts that have already been put
> into this so I'm not duplicating any work.

We've patched aports so that a minimal bootstrap system can be cross
compiled. All changes are in git already. And some minor notes + links
to the scripts I used to bootstrap musl builders are at my wiki wall:
http://wiki.alpinelinux.org/wiki/User:Fabled

Currently, about 85-90% of edge/main are built against musl for x86 and
armhf (armv6+vfp target in toolchain). x86_64 is not done yet, but I
expect to do it soon. The remaining 10% of main and testing aports
probably need some patching to build.

The repository for these binary packages is at:
http://nl.alpinelinux.org/alpine/edge-musl/main/
http://nl.alpinelinux.org/alpine/edge-musl/testing/

They are updated sporadically (not synchronized to git like the primary
uclibc builds).

x86 musl build does not have any boot images, but you can do chroot apk
install of it.

For armhf we don't really support any boards yet. But for testing, I've
created a Raspberry Pi image. It's available at:
http://dev.alpinelinux.org/~tteras/alpine-edge-131010-armhf.rpi.tar.gz

I also have Wandboard Quad images but those I have not yet published. I
use Wandboard Quad to native compile the arm binaries.

- Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Leslie P. Polzer | PORT ZERO
Details
Message ID
<5264EE11.5080504@port-zero.com>
In-Reply-To
<20131018131615.5286bb75@vostro> (view parent)
Sender timestamp
1382346257
DKIM signature
missing
Download raw message
On 10/18/2013 12:16 PM, Timo Teras wrote:

> In the long run we are currently planning to drop uclibc, and replace
> it with musl in the primary builds for Alpine 3.0. In case we do not
> get everything done for next release date, we might still do
> Alpine 2.8 based on uclibc.

Ah, so it's not intimately linked with the ARM port but more of
a general improvement. Good to know.

Thanks a lot for all the information on cross compiling Alpine for ARM!
That's very helpful. I will spend the next days getting acquainted
with the toolchain. I'm also interested in building packages with
clang.

  Leslie


-- 
Dipl.Ing.(BA) Leslie P. Polzer | CTO - PORT ZERO
UG (haftungsbeschränkt)
Softwareentwicklung & IT Engineering
Adalbertstraße 7/8
10999 Berlin
Tel.: +49 (0)30 - 69 200 907 - 0
Fax: +49 (0)30 - 69 200 907 - 9
http://www.port-zero.com


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20131022081626.5c52f0f0@vostro>
In-Reply-To
<5264EE11.5080504@port-zero.com> (view parent)
Sender timestamp
1382418986
DKIM signature
missing
Download raw message
On Mon, 21 Oct 2013 11:04:17 +0200
"Leslie P. Polzer | PORT ZERO" <polzer@port-zero.com> wrote:

> On 10/18/2013 12:16 PM, Timo Teras wrote:
> 
> > In the long run we are currently planning to drop uclibc, and
> > replace it with musl in the primary builds for Alpine 3.0. In case
> > we do not get everything done for next release date, we might still
> > do Alpine 2.8 based on uclibc.
> 
> Ah, so it's not intimately linked with the ARM port but more of
> a general improvement. Good to know.

Correct.

> Thanks a lot for all the information on cross compiling Alpine for
> ARM! That's very helpful. I will spend the next days getting
> acquainted with the toolchain. I'm also interested in building
> packages with clang.

clang is there and built for ARM already. It works since mesa needs it
to build.

However, it's not fully independent build - it uses libgcc, etc. We
have not seen yet the need to build "self hosting" clang. However, this
is an interesting option, and we would appreciate if someone does it.

We are planning to take a look at it at some point too. We might be also
considering switching to clang as main toolchain later when it has
matured a little bit more still.

- Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Cross-building (was: Re: [alpine-devel] musl and ARM in 2.7 (and an introduction))

Leslie P. Polzer | PORT ZERO
Details
Message ID
<526A6478.3080007@port-zero.com>
In-Reply-To
<20131018131615.5286bb75@vostro> (view parent)
Sender timestamp
1382704248
DKIM signature
missing
Download raw message
On 10/18/2013 12:16 PM, Timo Teras wrote:

>> Also, is there an established way already of building Alpine based
>> on musl and/or cross-compiling for ARM? It doesn't seem to be a big
>> deal, but I'd like to know the efforts that have already been put
>> into this so I'm not duplicating any work.
> 
> We've patched aports so that a minimal bootstrap system can be cross
> compiled. All changes are in git already. And some minor notes + links
> to the scripts I used to bootstrap musl builders are at my wiki wall:
> http://wiki.alpinelinux.org/wiki/User:Fabled

That helped! I had to clear out some roadblocks, after which I now have
a successful and reproducible bootstrap crossbuild of i486-alpine-linux-musl
(without KERNEL_PKG and DEBUG_PKG so far; I simply haven't tried).

The problems I encountered, working with your scripts on current edge:

In createcross-toolchain.sh:

* I instructed abuild to clean up the musl build before installing
  headers. This is necessary to prevent the patches from being applied
  multiple times by the "prepare" step if you call the script more
  than once.

In abuild-crossbuild-x86.conf:

* CBUILD is no longer being set in /etc/abuild.conf (this was changed
  pretty recently). This prevented gmp from being built correctly
  (I think it just used the host setting for the build).
  More packages might be affected by this.

In crossbuild-alpine-bootstrap.sh:

* Set up the proper pkgconfig environment for the cross root.
  Without this the relevant .pc files cannot be found. The first
  package that broke in the ordered crossbuild because of this
  was apk-tools.

* Added installation of all packages after they have been built.
  This is a necessary step since later packages require files from
  earlier ones. Also attempt to install $PKG-dev. Some packages need
  special casing here since their -dev packages have a different
  name. Maybe this should be fixed in the packages themselves.
  The packages affected are gmp5 (gmp-dev) and mpfr3 (mpfr-dev).

* Non-essential: also added per-package logging to cut down on the output.

I'm attaching a rough diff against your x86 conf files and the two
scripts. Please let me know how you want to proceed with integrating
these changes.

I wonder how we can test packages for their sanity properly after
we have built them. The usual way is running the testsuite (make check)
after compilation. Obviously this won't work when cross-compiling to
a different machine architecture. Any ideas?

  Leslie

-- 
Dipl.Ing.(BA) Leslie P. Polzer | CTO - PORT ZERO
UG (haftungsbeschränkt)
Softwareentwicklung & IT Engineering
Adalbertstraße 7/8
10999 Berlin
Tel.: +49 (0)30 - 69 200 907 - 0
Fax: +49 (0)30 - 69 200 907 - 9
http://www.port-zero.com

[alpine-devel] Re: Cross-building

Leslie P. Polzer | PORT ZERO
Details
Message ID
<526A6EFE.30902@port-zero.com>
In-Reply-To
<526A6478.3080007@port-zero.com> (view parent)
Sender timestamp
1382706942
DKIM signature
missing
Download raw message
Another question I forgot to ask:

Why do the scripts install gcc-gnat and build gcc with support
for Ada?

  Leslie

-- 
Dipl.Ing.(BA) Leslie P. Polzer | CTO - PORT ZERO
UG (haftungsbeschränkt)
Softwareentwicklung & IT Engineering
Adalbertstraße 7/8
10999 Berlin
Tel.: +49 (0)30 - 69 200 907 - 0
Fax: +49 (0)30 - 69 200 907 - 9
http://www.port-zero.com


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Re: [alpine-devel] Re: Cross-building

Timo Teräs
Details
Message ID
<1382709250.9899.2.camel@Nokia-N900>
In-Reply-To
<526A6EFE.30902@port-zero.com> (view parent)
Sender timestamp
1382709250
DKIM signature
missing
Download raw message
On Fri Oct 25 2013 04:15:42 PM EEST, Leslie P. Polzer | PORT ZERO <polzer@port-zero.com> wrote:

> Another question I forgot to ask:
> 
> Why do the scripts install gcc-gnat and build gcc with support
> for Ada?

You need gcc-gnat = ada. You need ada to compile ada. So to port full alpine, ada needs to be cross compiled to bootstrap it.

-Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Re: Cross-building (was: Re: [alpine-devel] musl and ARM in 2.7 (and an introduction))

Timo Teras
Details
Message ID
<20131030111416.3a993b49@vostro>
In-Reply-To
<526A6478.3080007@port-zero.com> (view parent)
Sender timestamp
1383124456
DKIM signature
missing
Download raw message
On Fri, 25 Oct 2013 14:30:48 +0200
"Leslie P. Polzer | PORT ZERO" <polzer@port-zero.com> wrote:

> On 10/18/2013 12:16 PM, Timo Teras wrote:
> 
> >> Also, is there an established way already of building Alpine based
> >> on musl and/or cross-compiling for ARM? It doesn't seem to be a big
> >> deal, but I'd like to know the efforts that have already been put
> >> into this so I'm not duplicating any work.
> > 
> > We've patched aports so that a minimal bootstrap system can be cross
> > compiled. All changes are in git already. And some minor notes +
> > links to the scripts I used to bootstrap musl builders are at my
> > wiki wall: http://wiki.alpinelinux.org/wiki/User:Fabled
> 
> That helped! I had to clear out some roadblocks, after which I now
> have a successful and reproducible bootstrap crossbuild of
> i486-alpine-linux-musl (without KERNEL_PKG and DEBUG_PKG so far; I
> simply haven't tried).

Very good!

> The problems I encountered, working with your scripts on current edge:
> 
> In createcross-toolchain.sh:
> 
> * I instructed abuild to clean up the musl build before installing
>   headers. This is necessary to prevent the patches from being applied
>   multiple times by the "prepare" step if you call the script more
>   than once.

Ah, right. I think this can be affected in abuild.conf - but it's good
to have it explicit.

> In abuild-crossbuild-x86.conf:
> 
> * CBUILD is no longer being set in /etc/abuild.conf (this was changed
>   pretty recently). This prevented gmp from being built correctly
>   (I think it just used the host setting for the build).
>   More packages might be affected by this.

Yes, I noticed this too.

> In crossbuild-alpine-bootstrap.sh:
> 
> * Set up the proper pkgconfig environment for the cross root.
>   Without this the relevant .pc files cannot be found. The first
>   package that broke in the ordered crossbuild because of this
>   was apk-tools.

And this too - I added this to x86_64.conf's earlier.

> * Added installation of all packages after they have been built.
>   This is a necessary step since later packages require files from
>   earlier ones. Also attempt to install $PKG-dev. Some packages need
>   special casing here since their -dev packages have a different
>   name. Maybe this should be fixed in the packages themselves.
>   The packages affected are gmp5 (gmp-dev) and mpfr3 (mpfr-dev).

Hmm - this worked for me without this. Though, I did have to fix some
makedepends in the aports. This was done not long ago.

> * Non-essential: also added per-package logging to cut down on the
> output.

Ok.

> I'm attaching a rough diff against your x86 conf files and the two
> scripts. Please let me know how you want to proceed with integrating
> these changes.

I need think what to do with the scripts. Might be an idea to bundle
them in abuild git tree. Even if they are not shipped in abuild.
Ultimately I would abuild to have "--target" and "--host" flags that
can be used to trigger the cross build so no additional "kludge" config
files would be needed.

> I wonder how we can test packages for their sanity properly after
> we have built them. The usual way is running the testsuite (make
> check) after compilation. Obviously this won't work when
> cross-compiling to a different machine architecture. Any ideas?

I don't think we plan to support cross-compiling all of aports. So this
would not be a goal for us.

I believe scratchbox etc. use statically built qemu-user, or some
binfmt_misc magic to launch the files on remote target. In either case
both will be "kludgy" and probably not doing 100% testing.

- Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: Cross-building

Leslie P. Polzer | PORT ZERO
Details
Message ID
<52716A8F.9060604@port-zero.com>
In-Reply-To
<20131030111416.3a993b49@vostro> (view parent)
Sender timestamp
1383164559
DKIM signature
missing
Download raw message
On 10/30/2013 10:14 AM, Timo Teras wrote:

>> * Added installation of all packages after they have been built.
>>   This is a necessary step since later packages require files from
>>   earlier ones. Also attempt to install $PKG-dev. Some packages need
>>   special casing here since their -dev packages have a different
>>   name. Maybe this should be fixed in the packages themselves.
>>   The packages affected are gmp5 (gmp-dev) and mpfr3 (mpfr-dev).
> 
> Hmm - this worked for me without this. Though, I did have to fix some
> makedepends in the aports. This was done not long ago.

I see, abuild will install the necessary deps automatically so
there's no need to manually install them if the makedeps are
correct.


> I need think what to do with the scripts. Might be an idea to bundle
> them in abuild git tree. Even if they are not shipped in abuild.

Good, please let me know when you've decided what to do.

In the meantime I will test some more and maybe try to automate building
a new cross-compiled image.


> Ultimately I would abuild to have "--target" and "--host" flags that
> can be used to trigger the cross build so no additional "kludge" config
> files would be needed.

Yes, that'd be very nice indeed.

Thanks!

  Leslie


-- 
Dipl.Ing.(BA) Leslie P. Polzer | CTO - PORT ZERO
UG (haftungsbeschränkt)
Softwareentwicklung & IT Engineering
Adalbertstraße 7/8
10999 Berlin
Tel.: +49 (0)30 - 69 200 907 - 0
Fax: +49 (0)30 - 69 200 907 - 9
http://www.port-zero.com


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---