Subject: [alpine-devel] A few questions about Alpine

From: "A. Wilcox" <[email-masked]>
Date: Sat, 10 Jun 2017 05:27:38 -0500

Hello Alpine developers,

I’m A. Wilcox, the leader of the Adélie Linux distribution.  We are a
distribution which, like Alpine, use the musl libc.  We are presently
using Gentoo’s Portage system for package building, but unfortunately
we feel that the Gentoo ecosystem is no longer suitable for use.

Our goals for Adélie are as follows:

- - A lightweight desktop distribution that is user-friendly while still
remaining a powerful Linux system underneath, built on technologies
like the musl libc.

- - Support for many CPU architectures including some that Alpine does
not yet support (we already have PPC32 and experimental MIPS support).

- - The ability for users to build some packages the way they want while
still remaining a binary-first distribution (like FreeBSD’s ports vs
pkg).

As such, we are considering “rebasing” our distribution on Alpine
Linux.  We have a few questions and ideas to think about before we
would go ahead.

1. If we had either a different set of patches for a package, or our
patches allowed different features in a package to be built that are
currently unavailable in Alpine, would Alpine be willing to evaluate
including them in the Alpine APKBUILD?

2. Does the abuild utility support multiple repositories of packages?
 How are build-time dependencies resolved in abuild?  If we had our
own mesa package, for an example, would abuild be able to build
against ours when we build packages from the Alpine package tree?  Or
would we need to maintain a fork of the Alpine tree and maintain our
changes in the same tree?

3. Would support for build-time configuration settings (like FreeBSD’s
ports OPTIONS or Gentoo’s portage USE) be welcomed in abuild?  This
would allow us (and our users) to build packages how we want while
still having the defaults remain the same in Alpine upstream.

4. Is Alpine interested in ports to other architectures supported by
musl including PPC32 and MIPS?

5. Would support for alternative shells in abuild be a welcome
addition (allowing abuild to run in shells including Busybox, bash,
zsh, etc)?  These changes would only be to more firmly align abuild
with the POSIX shell standard where applicable.

6. Would there be any interest in jointly creating some user-facing
utilities to make the abuild experience a more pleasant one for users?

Thank you for your time.

Best to you and yours,
—arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org

---
Unsubscribe:  [email-masked]
Help:         [email-masked]
---

---

From: William Pitcock <[email-masked]>
Date: Sat, 10 Jun 2017 18:17:58 -0500

Hello,

On Sat, Jun 10, 2017 at 5:27 AM, A. Wilcox <[email-masked]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hello Alpine developers,
>

Yes, absolutely.

> changes in the same tree?

abuild has no repository awareness at all.  It just looks at APKBUILD files.
We use aports-build to run the buildservers which generate the actual repos.

> still having the defaults remain the same in Alpine upstream.

Yes, this is a planned feature already.  We just haven't gotten around
to making a spec for how it will look in an APKBUILD.

> musl including PPC32 and MIPS?

Yes.  We would like to have all architectures supported by musl as
release architectures.
I am planning on doing MIPS64 in the 3.7 release series.

> with the POSIX shell standard where applicable.

This is probably OK, as long as it retains compatibility with Alpine's /bin/sh.

> utilities to make the abuild experience a more pleasant one for users?

We are open to proposals to do so, as the point was to enable such
things in the first place.

William

---

---

From: Natanael Copa <[email-masked]>
Date: Mon, 12 Jun 2017 12:47:19 +0200

On Sat, 10 Jun 2017 05:27:38 -0500
"A. Wilcox" <[email-masked]> wrote:

> I*m A. Wilcox, the leader of the Adélie Linux distribution.  We are a
> using Gentoo*s Portage system for package building, but unfortunately

Versions 1.0 - 1.8 was built with Gentoo's portage. From alpine 1.9 we
switched to abuild.

So yeah, we've been there :)

> still remaining a binary-first distribution (like FreeBSD*s ports vs
> pkg).
> 
> 
> As such, we are considering *rebasing* our distribution on Alpine
> Linux.

I think this makes sense. I am very positive to this.

>  We have a few questions and ideas to think about before we
> would go ahead.
> 

Yes.

We try to avoid patching the packages if possible so patches should go
upstream if possible.

But we will evaluate any suggestions for different patches and
different compile time features.

> 2. Does the abuild utility support multiple repositories of packages?

we have "main", "community" and "testing" package repositories. 

>  How are build-time dependencies resolved in abuild?  

abuild will call `apk add $makdepends $depends` (more or less), so the
dependencies needs to be found by apk.

> If we had our

Depends a bit if its ABI compatible or not. If its not ABI compatible
then you will have to rebuild all the packages that depends on your
version of mesa and maintain a fork of those.
 
> 3. Would support for build-time configuration settings (like FreeBSD*s
> ports OPTIONS or Gentoo*s portage USE) be welcomed in abuild?  This

Yes. This would be welcomed.
 
> musl including PPC32 and MIPS?

Yes, as long someone steps up and help us do the work, provide the
needed hardware and help maintain the port.
 
> with the POSIX shell standard where applicable.

Yes. We currently use some bashism that is supported in busybox ash
which is handy. For example ${variable/search/replace}. We'd need
figure out which extensions works on which shells and see if we can
allow them or not.

> utilities to make the abuild experience a more pleasant one for users?

Yes.

We have talked about replacing APKBUILD format with something else, but
not really been able to come up with anything. The problem is that it
is difficult (if not impossible) to make a total replace, so changes
will need to go incrementally.

The major issue with APKBUILD is that it is not possible to specify the
metadata for subpackages (pkgdesc, depends, arch etc) in global scope,
and regardless of how we define it in shell, it will be clumsy at best.

I also sort of like the template format that void linux uses, or at
least i like parts of it, but I don't like the idea of hard depndency
on bash.

I suppose this is a different discussion, but I think its good that you
know that we are not 100% happy with APKBUILD format, and that we are
open to suggestions and ideas.

-nc

> *arw
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQIcBAEBCAAGBQJZO8mSAAoJEMspy1GSK50UAJgP/RH+LaNmnuCQm/tyeH3YJ67n
> hWU7XBWeSy7NSfkujUTJ6suBA55BQAmkVrrLFuFddi6hLvPLfWFjfOtmwe7GPV7u
> Wn2Sim2dBnANGJeB1W7SqloY84Xdvn9UCBF1ISb3fUzJrADjxWArVPh6y0m+X3IY
> 84iUksE13zRKWlL7jtJ246jnwpf2rrJsMGLACLL4tLh5bM3sF+QUDerVvSo9FAk3
> 9ybo+F9giiGI9qydo+7Xjncz+I5lWgC1JhoTcEoInwiN8AsrCWXWsEAQkZpLjR+z
> qVUNzerZnPsooqH3pNzOpMVuXiGLVGn9+oLUh0l5rI+0s1F5cf1oLYdISscVnfxq
> Mz3EoygBM1jyzGlV4v6i9uwSvfw8sTRFbdT543thXPfQ43DAxHlIFE7xQnIoYNCP
> kvzRWY8csfwwM+rdHMIqIH3fx5+dLasLoQIij6doxTLJVfAQAtqqO3AMRimI82wL
> lb+bPOraWB9y1rvEV5I0V9VeUd+4yrzueroisOuX7vrAAgqMg7jrsPNW+qoLa0hi
> sKy4sJy+tsiFJF8U/7b6qp2w/5m1tZY8sgrYPBfcQi7W0ZSRFug/mqTnJ91LhdDK
> jMxxPe2VF/ofoD/nh2/A02RQ4hAlzOHLTEwIeG7OvABVZxazT3HNc3q0Ls5JIi+P
> xhh30los4Ztq8V3UhNXe
> =blXf
> -----END PGP SIGNATURE-----
> 
> 
> ---

---

---

From: Breno Leitao <[email-masked]>
Date: Mon, 12 Jun 2017 09:17:37 -0300

> "A. Wilcox" <[email-masked]> wrote:

<snip>

> needed hardware and help maintain the port.

The current ppc64le machine can run ppc32 virtual machines, and that is how
we do with Debian. The Debian build machines are VMs on top of a ppc64le big box.

Unfortunately we cannot use containers since the kernel use different ABIs
with different byte endianess, that is why we need VMs, with different kernel.

---

---

From: Natanael Copa <[email-masked]>
Date: Mon, 12 Jun 2017 15:58:08 +0200

On Mon, 12 Jun 2017 09:17:37 -0300

> we do with Debian. The Debian build machines are VMs on top of a ppc64le big box.

Very good. We have the needed hardware then for ppc32.

-nc

---

---

From: "A. Wilcox" <[email-masked]>
Date: Mon, 12 Jun 2017 17:11:54 -0500

On 12/06/17 07:17, Breno Leitao wrote:
> 
> The current ppc64le machine can run ppc32 virtual machines, and
> that is how we do with Debian. The Debian build machines are VMs on
> top of a ppc64le big box.
> 
> Unfortunately we cannot use containers since the kernel use
> different ABIs with different byte endianess, that is why we need
> VMs, with different kernel.

One thing to accomplish two things with one VM is to run a ppc64 BE
kernel in the VM, and then use a chroot for ppc32.  This is what we
did and the built packages seem to work fine on real ppc32 hardware
(testing on a few 750s and a few 74x7).

- --arw

- -- 
A. Wilcox (awilfox)

---

---

From: "A. Wilcox" <[email-masked]>
Date: Mon, 12 Jun 2017 17:38:00 -0500

On 12/06/17 05:47, Natanael Copa wrote:
> On Sat, 10 Jun 2017 05:27:38 -0500 "A. Wilcox"
> <[email-masked]> wrote:
> 
>> 1. If we had either a different set of patches for a package, or
>> our patches allowed different features in a package to be built
>> that are currently unavailable in Alpine, would Alpine be willing
>> to evaluate including them in the Alpine APKBUILD?
> 
> Yes.
> 
> We try to avoid patching the packages if possible so patches should
> go upstream if possible.
> 

That is our policy in Adélie as well.  I don't like us to carry
patches and am in contact with various upstreams.

But there are some upstreams that seem to no longer exist (CVS is
one), and some upstreams that are hostile to standards conformance :(
 That is when we start doing package patches.

> abuild will call `apk add $makdepends $depends` (more or less), so
> the dependencies needs to be found by apk.

That makes sense.  Much simpler than the overlay situation with ebuild.

>> If we had our own mesa package, for an example, would abuild be
>> able to build against ours when we build packages from the Alpine
>> package tree?  Or would we need to maintain a fork of the Alpine
>> tree and maintain our changes in the same tree?
> 
> Depends a bit if its ABI compatible or not. If its not ABI
> compatible then you will have to rebuild all the packages that
> depends on your version of mesa and maintain a fork of those.

Yeah, that makes sense.  So we would just put our repository first in
the /etc/apk/repositories file and then it seems abuild will do the
right thing.  Great to hear, makes life much easier.

>> 3. Would support for build-time configuration settings (like
>> FreeBSD*s ports OPTIONS or Gentoo*s portage USE) be welcomed in
>> abuild?  This would allow us (and our users) to build packages
>> how we want while still having the defaults remain the same in
>> Alpine upstream.
> 

I suppose this needs to be in its own thread :)  I have a few
different ideas but I want to continue working with abuild more to get
a better feel for how it works so that my proposal is nice and clean.

>> 4. Is Alpine interested in ports to other architectures supported
>> by musl including PPC32 and MIPS?
> 

Hardware is not an issue for me, my office-turned-build lab has:

3x ppc32
2x mips64
1x mips32
2x sparc64 (not musl ported yet)
2x arm32
2x arm64/aarch64
1x ppc64 (Power4 based)

What kind of requirements do you have for the hardware?  Do I need to
rack some of it at a datacentre for high bandwidth Internet?  Or is an
office-based cable line (~150 mbit) good enough?

>> addition (allowing abuild to run in shells including Busybox,
>> bash, zsh, etc)?  These changes would only be to more firmly
>> align abuild with the POSIX shell standard where applicable.
> 
> Yes. We currently use some bashism that is supported in busybox
> ash which is handy. For example ${variable/search/replace}. We'd
> need figure out which extensions works on which shells and see if
> we can allow them or not.

Sure, my main concern is being able to run abuild on computers that
don't have ash.  I suppose since most other build systems have a hard
requirement on bash, having a hard requirement on ash isn't unusual.
But I think Alpine can be even better and not have a hard requirement.

>> 6. Would there be any interest in jointly creating some
>> user-facing utilities to make the abuild experience a more
>> pleasant one for users?
> 
> Yes.
> 
> We have talked about replacing APKBUILD format with something else,
> but not really been able to come up with anything. The problem is
> that it is difficult (if not impossible) to make a total replace,
> so changes will need to go incrementally.
> 
> The major issue with APKBUILD is that it is not possible to specify
> the metadata for subpackages (pkgdesc, depends, arch etc) in global
> scope, and regardless of how we define it in shell, it will be
> clumsy at best.
> 
> I also sort of like the template format that void linux uses, or
> at least i like parts of it, but I don't like the idea of hard
> depndency on bash.
> 
> I suppose this is a different discussion, but I think its good that
> you know that we are not 100% happy with APKBUILD format, and that
> we are open to suggestions and ideas.
> 
> -nc

Replacing a format is pretty hard as I am finding out converting all
our custom ebuilds to APKBUILD.  Hehe.

It is good to know that the door is open to further improvements to
the format.  As I work with it more I will continue to think about
ways to incrementally make it a better format.

One thought that I did have was to have some common things abstracted
into things that could be sourced from the APKBUILD.  Almost like
'inherit' eclasses in Gentoo or '.include' templates in FreeBSD.  (Can
you tell yet which systems I am familiar with?)  It might make things
like Python and Ruby easier to maintain, since you would for instance
just do:

package() {
   . "$templatedir"/python/package
}

check() {
   . "$templatedir"/python/check_pytest
}

But I don't know if that really would save a lot of maintenance
burden, or if it would just shift it to another place.  Will need to
think more and start a new thread for that as well.

Best,
- --arw

- -- 
A. Wilcox (awilfox)

---