Mail archive
alpine-devel

Re: [alpine-devel] rethinking the building infra

From: Oliver Smith <ollieparanoid_at_bitmessage.ch>
Date: Wed, 07 Feb 2018 22:53:00 +0000

A. Wilcox:
> On 02/07/18 13:58, Oliver Smith wrote:
>> Hey Natanael,
>>
>> the list sounds good to me, especially the increased security
>> features!
>>
>> While I agree with A. Wilcox that cross-compiling has the
>> disadvantage of not running tests (or not properly if done through
>> QEMU), I think it *does* make sense at least for kernels (there
>> aren't any tests to execute for them). In my opinion, this is rather
>> a detail, and the important thing would be getting everything else
>> implemented first. With that being said: We have cross-compiling
>> binary packages in the postmarketOS repo since we do lots of
>> cross-compiling: gcc-armhf, binutils-armhf, musl-armhf etc. The
>> aports for these are automatically generated from the upstream ones
>> in Alpine (basically hardcoded variables from the bootstrap script on
>> top). Unless the new build system would approach cross-compiling in a
>> completely different way, you would also need such binary packages to
>> do it, so maybe this feature can be upstreamed properly (that is the
>> long-term plan anyway, if you guys want to have this - maybe with
>> subpackages?).
>
>
> I'm not sure what feature you are talking about. There is already
> bootstrap.sh to generate a working cross environment. I suppose you
> could make a bootstrap-lite.sh or such that only makes the toolchain and
> none of the other components, which would work for a kernel build.

What I mean is having packages directly in the binary repository, so one can do:
$ apk add gcc-armhf

Just like it's possible to install gcc-avr right now. But gcc-avr package needs to be manually synced with the gcc aport. Instead of that it would be nice, if we had cross-compilers automatically built for all architectures, without running bootstrap.sh. Possibly as subpackage of gcc (but that's probably not the desired solution, since that will blow up the build time of GCC drastically). I think that this might be relevant to cross-compiling in the binary repository, because once such packages in the binary repo exist, it would be a clean way to install the cross-compiler in the compiling VM/container with apk.

>
>
>> Regarding caching: Maybe ccache makes sense, if it was separated for
>> each package, and incoming pull-requests could only read from that
>> cache, not write to it.
>
>
> I think the cache point is more for the actual git data and distfiles,
> not the compilation.
>

I understood that, this was just me throwing a different idea in to reduce compilation times for commits which only have small changes (configure flag changed etc.).

>
>> Finally, I am not sure if the idea is to replace abuild entirely, or
>> just abuild-rootbld.
>
>
> Who said abuild was being replaced? This is about `buildrepo` and
> friends, not abuild.

Thanks for clearing that up.

>
>
>> In case abuild should be replaced, I hope to see
>> the following features preserved (since we make use of them in
>> pmbootstrap, which wraps abuild):
>>
>> - (passing through environment variables)
>
> This isn't a wise idea. It's better to export them in the build() or
> package() functions.
>

For package specific variables, we export them in the APKBUILD. We only pass them through abuild for package independent variables, such as:
CARCH, CROSS_COMPILE, CC, CCACHE_PREFIX, CCACHE_PATH, CCACHE_COMPILERCHECK, DISTCC_HOSTS

>> - abuild -r (install depends with abuild)
>
> This is currently how all Adélie packages are built, but it is a bit
> buggy. It is useful for end users making single contributions, but
> honestly, I think that this new system is going to be much better than
> `abuild -r`.
>
>> - abuild undeps
>
> Due to the way abuild works right now, `apk del .makedepends-$pkgname`
> has the same effect, if you ever need to do that without abuild. That's
> an implementation detail and is not guaranteed to always work, but it
> does work right now if you need it.

Right, thanks!

>
>> - abuild menuconfig
>
> ??? This isn't even an abuild feature. It (ab)uses the fact that
> abuild phases are shell functions by calling a menuconfig() function in
> the kernel APKBUILD. Honestly, I would be much in favour of /not/
> abusing that fact, and making abuild more hardened against that, but
> that is just me.
>

Alpine's linux-vanilla APKBUILD used to have a menuconfig() function with a comment on top saying something like "# This is so we can use 'abuild menuconfig'". But I just realized that this was removed. Well, we still use that feature for that purpose. But if that did not work anymore in abuild, we could call menuconfig directly.

Thanks,
Oliver

>
> Best,
> --arw
>



---
Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
Help:         alpine-devel+help_at_lists.alpinelinux.org
---
Received on Wed Feb 07 2018 - 22:53:00 GMT