I think we need to rethink the building infrastructure. The current
build scripts were written as a quick and dirt way to get started and
have lived longer way longer than originally planned. It simply does
not scale and is very fragile.
I guess I don't need to go any deeper into why we need replace it...
I'd like to discuss what we need from the build infra and why, before we
start talk about how to do it and what implementation etc.
Here are some things I want a new building infra should be able to
- automatic build on git push
There should not be needed to do anything more than do a git push to
get the package built and uploaded. Like we have today, but with
better error reporting.
- isolated environment for each build
each build should set up an isolated enviroment and destroy it when
build is done. This could be a container but it would be nice to be
able to set up a disposable build env in a vm in case we want hook it
into github PRs or similar. It should also kill everything after
build and test is done so we dont get any remains of test suites that
does not clean up after themselves (like redis and epmd)
- support multi architectures
need to support x86_64, x86, armhf, aarch64, ppc64le and s390x. Would
be nice it its not too complicated to add new architectures.
- support parallel building
would be nice to be able to distribute the workload over available
build servers. Should be possible and relatively easy to add new
hardware to the pool or remove or replace old without taking
- support cross compile
would be nice to cross compile packages that (easily) can be cross
compiled. For example, we could let a big x86_64 or ppc64le machine
build linux kernel for armhf instead doing that on the slow armhf
server. Packages that cannot be cross compiled should be built on
- separate out signing process of packages and index
Would be nice if we could give access to build servers to more
people without giving those people access to the private signing keys.
- build infra should be able to be used as CI
We need do automatic compile checks of contributions, for example via
github pull requests or something corresponding.
- efficient caching
Would be nice to not need to git clone the entire repo for every
build for every server. Would be nice if we checkout a shared git
repo or do something so data does not goes over the wire more than
necessary. Same goes for source and apk packages.
Anything else we need from the building infra?
Received on Tue Feb 06 2018 - 01:39:44 GMT