Sender: =?UTF-8?Q?Timo_Ter=C3=A4s?= <timo.teras@gmail.com>
Date: Wed, 27 Jul 2016 15:59:46 +0300
From: Timo Teras <timo.teras@iki.fi>
To: "alpine-devel" <alpine-devel@lists.alpinelinux.org>
Subject: [alpine-devel] Alpine (cross) building
Message-ID: <20160727155946.4aa60480@vostro.util.wtbts.net>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Hi,

We've recently had several people working on different new ports e.g.
aarch64, s390x and sh4. There's also feature request about mips on bug
tracker.

This means we need proper cross-compile support for bootstrapping new
architectures. And preferably even pure cross-compile for those arches
that are embedded slow machines.

While for the occational bootstrap my earlier scripts were enough. They
don't really scale that well, and it was a slightly hacky way to do it.

As result, I took the effort to do proper cross-building support to
abuild itself. This is now available in abuild.git on the cross-build
branch (hopefully to be in git master soon) at:
  http://git.alpinelinux.org/cgit/abuild/log/?h=cross-build

This allowed to clean up the bootstrap process, and all of aports
bootstrap packages have been fixed to work again. The bootstrap script
itself now lies in aports.git/scripts/bootstrap.sh. If all goes well,
it'll do a complete bootstrap (create cross compiler, and use that to
cross compile new base system for target architecture).
  http://git.alpinelinux.org/cgit/aports/tree/scripts/bootstrap.sh

I tested these to successfully bootstrap aarch64 and armv7 (thumb2)
arches. At least aarch64 we hope to have as a supported arch in near
future. We are still working out some issues in the native toolchain,
and checking that everything builds.

While this is a major improvement, I still have more ideas on what to
improve.

As next step I'm planning to rewrite alpine-iso (the Makefile we use to
create bootable images) as set of scripts in aports.git/scripts/ and
have them support cross-building boot media and reuse existing scripts
such as update-kernel.

The ultimate goal is to redo the builder setup to more CI like setup.
I'm currently looking at buildbot, if we can write a plugin that would
allow buildbot to handle build dependencies, and distribution of jobs.
This would allow several improvements:
 - would simplify us supporting contributed ppa type aports trees our
   infra would build; and simplify setting up your own build instance
 - one builder slave would build for all branches, greatly simplifying
   the steps when a stable version is branched
 - for embedded cpu types: cross-build packages on fast machines if
   package supports that, otherwise native build it 
 - allow build master to coordinate which builder's noarch package
   becomes official, and start sharing the noarch packages
 - make command that test builds given commit and finally emails the
   build results
 - rework the change process to go via buildbot. developers send pull
   request to buildbot which builds it everywhere, and only after that
   buildbot would commit it to our master aports.git. greatly improving
   stability of edge.

There is still quite a way to get there. But I wanted to write about
what I have in mind, and to also get some feedback if there's something
I missed, or better alternatives. Or even better if someone is working
on something similar and can contribute. :)

Thanks,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---