Subject: Re: [alpine-devel] rethinking the building infra
To: alpine-devel@lists.alpinelinux.org
References: <20180206013944.7fa393b6@ncopa-macbook.copa.dup.pw>
From: "A. Wilcox" <awilfox@adelielinux.org>
Organization: =?UTF-8?Q?Ad=c3=a9lie_Linux?=
Message-ID: <192872b3-068a-0d7a-d618-e00e543fff3d@adelielinux.org>
Date: Tue, 6 Feb 2018 12:31:22 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.2.0
Precedence: list
MIME-Version: 1.0
In-Reply-To: <20180206013944.7fa393b6@ncopa-macbook.copa.dup.pw>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="5qemSsk7dSb2HoVMRdVVBQmU0T0ptUp6T"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--5qemSsk7dSb2HoVMRdVVBQmU0T0ptUp6T
Content-Type: multipart/mixed; boundary="frhveAuVxHpumXXHUngTN5F0RGgi9XWa1";
 protected-headers="v1"
From: "A. Wilcox" <awilfox@adelielinux.org>
To: alpine-devel@lists.alpinelinux.org
Message-ID: <192872b3-068a-0d7a-d618-e00e543fff3d@adelielinux.org>
Subject: Re: [alpine-devel] rethinking the building infra
References: <20180206013944.7fa393b6@ncopa-macbook.copa.dup.pw>
In-Reply-To: <20180206013944.7fa393b6@ncopa-macbook.copa.dup.pw>

--frhveAuVxHpumXXHUngTN5F0RGgi9XWa1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi there Natanael,

I thought I could go through and say what we (Ad=C3=A9lie) would like to =
see too.

On 02/05/18 18:39, Natanael Copa wrote:
> I'd like to discuss what we need from the build infra and why, before w=
e
> start talk about how to do it and what implementation etc.
>=20
> Here are some things I want a new building infra should be able to
> do:
>=20
> - automatic build on git push
>=20
>   There should not be needed to do anything more than do a git push to
>   get the package built and uploaded. Like we have today, but with
>   better error reporting.


That would be good.  One important thing is that it may be desireable to
have a "[build skip]" or such for commits that deal with non-build
things (like git hooks or such) or a series with interdependencies.


> - isolated environment for each build
>=20
>   each build should set up an isolated enviroment and destroy it when
>   build is done. This could be a container but it would be nice to be
>   able to set up a disposable build env in a vm in case we want hook it=

>   into github PRs or similar. It should also kill everything after
>   build and test is done so we dont get any remains of test suites that=

>   does not clean up after themselves (like redis and epmd)


It should not kill everything in cases of error.  I can't tell you how
much time abuild has saved me by leaving src/ alone when it fails to
compile or test.


> - support multi architectures
>=20
>   need to support x86_64, x86, armhf, aarch64, ppc64le and s390x. Would=

>   be nice it its not too complicated to add new architectures.


+1


> - support parallel building
>=20
>   would be nice to be able to distribute the workload over available
>   build servers. Should be possible and relatively easy to add new
>   hardware to the pool or remove or replace old without taking
>   everything down.


That would be a very nice thing.  It would also be nice for it to track
what builders are idle so it can pick one.  I am just thinking of a
scenario:

Two builders: ppc64-1, ppc64-2
Four packages are modified close together: llvm4, which, unzip, acl

If it just goes in sequence, 'unzip' will be held up on llvm4, when
ppc64-2 could easily build it.

Maybe that is too much of a "implementation detail", but I think it is
an important requirement.


> - support cross compile
>=20
>   would be nice to cross compile packages that (easily) can be cross
>   compiled. For example, we could let a big x86_64 or ppc64le machine
>   build linux kernel for armhf instead doing that on the slow armhf
>   server. Packages that cannot be cross compiled should be built on
>   native hardware.


The APKBUILD format doesn't do very well with cross.

Sure, some packages can cross easily, but many cannot.  Please see
coreutils mess in Ad=C3=A9lie repo for an example:

https://code.foxkit.us/adelie/aports/blob/master/main/coreutils/APKBUILD

I don't really recommend cross to 'speed up' builds, especially since it
isn't possible to run tests then.  Much better to wait for real
hardware.  Only use it for things in bootstrap.sh.


> - separate out signing process of packages and index
>=20
>   Would be nice if we could give access to build servers to more
>   people without giving those people access to the private signing keys=
=2E


Yes, when a builder is done, it should give the built package to an
internal, secure system for signing.  I have always agreed with this,
and the fact that the abuild signing key has to be present and readable
by abuild has prevented me from giving people access to our 32-bit x86
builder.


> - build infra should be able to be used as CI
>=20
>   We need do automatic compile checks of contributions, for example via=

>   github pull requests or something corresponding.


I note here that GitLab has CI too, and Jenkins is open source, so
Alpine doesn't always need to be tied to GitHub to still have CI :)


> - efficient caching
>=20
>   Would be nice to not need to git clone the entire repo for every
>   build for every server. Would be nice if we checkout a shared git
>   repo or do something so data does not goes over the wire more than
>   necessary. Same goes for source and apk packages.


Well, you could use NFS or such if the build servers are on the same
local network.  That is what we did at Galapagos Linux.  But I'm not
sure if Alpine's build systems are all located in the same network.


> Anything else we need from the building infra?


Ad=C3=A9lie would strongly benefit from having build logs kept in a
centralised place.  Something like logrotate could be used to keep it
from overflowing, so log maintenance is not something that needs to be
thought about.


The ability to have hooks (like git) for "build started", "build
success", and "build failure" would be useful.  Preferrably it would
allow custom ones written in shell or script, not web hoooks.  Then we
could have IRC poking if builds fail, and something like #alpine-commits
for all build statuses.


Best to you and yours,
--arw


>=20
> -nc

--=20
A. Wilcox (awilfox)
Project Lead, Ad=C3=A9lie Linux
http://adelielinux.org


--frhveAuVxHpumXXHUngTN5F0RGgi9XWa1--

--5qemSsk7dSb2HoVMRdVVBQmU0T0ptUp6T
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJMBAEBCAA2FiEEjNyWOYPU1SaTSMHHyynLUZIrnRQFAlp59HoYHGF3aWxmb3hA
YWRlbGllbGludXgub3JnAAoJEMspy1GSK50UZYcQAIMIate1dDfgGqAgAmXfExuM
eJpXrjZXwJiPtzsz9cld6CrJOfEPjPbD3m6emcDtTyg2fhP8l6WUqKRW2Bl9XMZY
E17DfQsALXxOVz1W7ESIrPJJs9ErjUzYClUt1h5+fj8WjZGIQ2OFIG72ZQCzfgeQ
fpbcNViA0wdGKI6yPLPEOLxNTNC827Qd1YHcCfb4IIH1waXsF4/01JZyMXllTOvQ
VXYwF3HDhKEd8RM1UC2oQ+nb1SkHNFDoflgevSPvQMQig9pVX5f0iFCv0nR/XbVw
mzg0RyHYxORVyEZZUzw3At1bcgyGzC3vLvWoph1RyqZcxwA1Vd8AyEpNsOj0LqAP
MX9XRLdPxxRaihn82kOpoMWF4i9zYtMJ+GsvWOETcoj7jxPRPIfzyI1REeTRnqOV
0NOo6xeBHhw9wtr7nA18G0Hn/IT5wIJl9G1AeX54Zl+sKbJ5yArdxBaunhisqQaU
UG2YLKP/7t2PfKfL5f1wbgvv2QTjH78/Q2Ugzmai/E+Z7Ez7OF5rMnfHD6ARCC50
xR5SRevJOIBgljznaU4SzFZcwpDT+dJvLx5fFkz5WAdxnI6+2lAjZm0lw/cu+KPv
np7PDp2+RKMxa9BUZbxyJw0juIjMwpKAe4TIPRTYOmqFSPu0qLsS184Szm0ledoA
jnCSfnpql9K+eiPsMesI
=gyDu
-----END PGP SIGNATURE-----

--5qemSsk7dSb2HoVMRdVVBQmU0T0ptUp6T--


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---