~alpine/devel

6 4

Let's discuss cross compiler packaging again

Details
Message ID
<4cfb2f21-ab4b-91eb-f548-02a52c0d6e54@postmarketos.org>
DKIM signature
missing
Download raw message
Dear Alpine developers,

it would be nice if one could just install cross compilers that target
all of Alpine's supported architectures. For example:

$ apk add gcc-aarch64

This isn't a new idea and has been somewhat discussed on this mailing
list before - but without result. The problem appears to be that it's
unclear how the APKBUILDs of these gcc-* aports should look like.
Ideally, they would not duplicate the existing gcc aport, but still be
able to use the same patch files.

I've recently written down how it looks like from the postmarketOS
perspective, and how we are working around it (TL;DR: we generate the
gcc-*, binutils-*, musl-* APKBUILDs from Alpine's APKBUILDs with some
cross compiler setup variables hardcoded). The issue also has some
approaches for upstreaming, but I'm not really happy with any of them.

https://gitlab.com/postmarketOS/pmaports/-/issues/551

Bart pointed out that I should point this ML to that issue, and he's
right with that, so here we are. I'd like to upstream this properly into
Alpine, but don't see a good way to do it. I'm curious if anybody has an
idea how to solve it elegantly.

(Maybe this could be solved in a better way by extending abuild?)

Thanks,
Oliver
Details
Message ID
<3302711.7VNl89jVkd@localhost>
In-Reply-To
<4cfb2f21-ab4b-91eb-f548-02a52c0d6e54@postmarketos.org> (view parent)
DKIM signature
missing
Download raw message
Hi,

On Tuesday, July 7, 2020 3:58:35 PM MDT Oliver Smith wrote:
> Dear Alpine developers,
> 
> it would be nice if one could just install cross compilers that target
> all of Alpine's supported architectures. For example:
> 
> $ apk add gcc-aarch64
> 
> This isn't a new idea and has been somewhat discussed on this mailing
> list before - but without result. The problem appears to be that it's
> unclear how the APKBUILDs of these gcc-* aports should look like.
> Ideally, they would not duplicate the existing gcc aport, but still be
> able to use the same patch files.

I would prefer to add proper multi-arch support.  From there the path to 
having cross-compilers becomes more obvious.  Technically, it is possible to 
build them already using bootstrap.sh, but without real multi-arch support 
it's not so useful.

> I've recently written down how it looks like from the postmarketOS
> perspective, and how we are working around it (TL;DR: we generate the
> gcc-*, binutils-*, musl-* APKBUILDs from Alpine's APKBUILDs with some
> cross compiler setup variables hardcoded). The issue also has some
> approaches for upstreaming, but I'm not really happy with any of them.
> 
> https://gitlab.com/postmarketOS/pmaports/-/issues/551
> 
> Bart pointed out that I should point this ML to that issue, and he's
> right with that, so here we are. I'd like to upstream this properly into
> Alpine, but don't see a good way to do it. I'm curious if anybody has an
> idea how to solve it elegantly.
> 
> (Maybe this could be solved in a better way by extending abuild?)

Both abuild and apk need to be extended, as well as making APKBUILDs cross-
aware by switching from $makedepends to $makedepends_build and 
$makedepends_host.

Ariadne
Details
Message ID
<3UJHRZ95NB7Q8.2XQ87FF99F53L@8pit.net>
In-Reply-To
<4cfb2f21-ab4b-91eb-f548-02a52c0d6e54@postmarketos.org> (view parent)
DKIM signature
missing
Download raw message
Hi,

I would also be highly interested in having more cross compilers
directly packaged in aports. I previously used clang successfully for
cross compiling to bare metal targets on Alpine. Haven't yet tried
linking against musl with clang cross compilers, but it seems to me that
cross compilation is easier with clang. Maybe that's also something that
could be investigated further (i.e. in addition to gcc cross compilers)?

Cheers,
Sören

Oliver Smith <ollieparanoid@postmarketos.org> wrote:
> Dear Alpine developers,
> 
> it would be nice if one could just install cross compilers that target
> all of Alpine's supported architectures. For example:
> 
> $ apk add gcc-aarch64
> 
> This isn't a new idea and has been somewhat discussed on this mailing
> list before - but without result. The problem appears to be that it's
> unclear how the APKBUILDs of these gcc-* aports should look like.
> Ideally, they would not duplicate the existing gcc aport, but still be
> able to use the same patch files.
> 
> I've recently written down how it looks like from the postmarketOS
> perspective, and how we are working around it (TL;DR: we generate the
> gcc-*, binutils-*, musl-* APKBUILDs from Alpine's APKBUILDs with some
> cross compiler setup variables hardcoded). The issue also has some
> approaches for upstreaming, but I'm not really happy with any of them.
> 
> https://gitlab.com/postmarketOS/pmaports/-/issues/551
> 
> Bart pointed out that I should point this ML to that issue, and he's
> right with that, so here we are. I'd like to upstream this properly into
> Alpine, but don't see a good way to do it. I'm curious if anybody has an
> idea how to solve it elegantly.
> 
> (Maybe this could be solved in a better way by extending abuild?)
> 
> Thanks,
> Oliver
Rasmus Thomsen <oss@cogitri.dev>
Details
Message ID
<0FAA80EC-A493-4F60-A141-72AF068DAAD6@cogitri.dev>
In-Reply-To
<3302711.7VNl89jVkd@localhost> (view parent)
DKIM signature
missing
Download raw message

On July 10, 2020 4:07:34 PM UTC, Ariadne Conill <ariadne@dereferenced.org> wrote:
>Hi,
>
>On Tuesday, July 7, 2020 3:58:35 PM MDT Oliver Smith wrote:
>> Dear Alpine developers,
>> 
>> it would be nice if one could just install cross compilers that
>target
>> all of Alpine's supported architectures. For example:
>> 
>> $ apk add gcc-aarch64
>> 
>> This isn't a new idea and has been somewhat discussed on this mailing
>> list before - but without result. The problem appears to be that it's
>> unclear how the APKBUILDs of these gcc-* aports should look like.
>> Ideally, they would not duplicate the existing gcc aport, but still
>be
>> able to use the same patch files.
>
>I would prefer to add proper multi-arch support.  From there the path
>to 
>having cross-compilers becomes more obvious.  Technically, it is
>possible to 
>build them already using bootstrap.sh, but without real multi-arch
>support 
>it's not so useful.
>
>> I've recently written down how it looks like from the postmarketOS
>> perspective, and how we are working around it (TL;DR: we generate the
>> gcc-*, binutils-*, musl-* APKBUILDs from Alpine's APKBUILDs with some
>> cross compiler setup variables hardcoded). The issue also has some
>> approaches for upstreaming, but I'm not really happy with any of
>them.
>> 
>> https://gitlab.com/postmarketOS/pmaports/-/issues/551
>> 
>> Bart pointed out that I should point this ML to that issue, and he's
>> right with that, so here we are. I'd like to upstream this properly
>into
>> Alpine, but don't see a good way to do it. I'm curious if anybody has
>an
>> idea how to solve it elegantly.
>> 
>> (Maybe this could be solved in a better way by extending abuild?)
>
>Both abuild and apk need to be extended, as well as making APKBUILDs
>cross-
>aware by switching from $makedepends to $makedepends_build and 
>$makedepends_host.

Are we going to have cross only for core packages or for all packages? To be frank I'd rather not have it for all packages, as that's a lot of work - many upstreams don't test cross compilation and as such are pretty broken. I'd rather not invest the effort of marking all packages !cross that are broken, especially when we have native builders anyway.

Regards,

Rasmus

>
>Ariadne
Details
Message ID
<2775783.pU9fvGE0RX@localhost>
In-Reply-To
<0FAA80EC-A493-4F60-A141-72AF068DAAD6@cogitri.dev> (view parent)
DKIM signature
missing
Download raw message
On Friday, July 10, 2020 12:42:20 PM MDT you wrote:
> On July 10, 2020 4:07:34 PM UTC, Ariadne Conill <ariadne@dereferenced.org> 
wrote:
> >Hi,
> >
> >On Tuesday, July 7, 2020 3:58:35 PM MDT Oliver Smith wrote:
> >> Dear Alpine developers,
> >> 
> >> it would be nice if one could just install cross compilers that
> >
> >target
> >
> >> all of Alpine's supported architectures. For example:
> >> 
> >> $ apk add gcc-aarch64
> >> 
> >> This isn't a new idea and has been somewhat discussed on this mailing
> >> list before - but without result. The problem appears to be that it's
> >> unclear how the APKBUILDs of these gcc-* aports should look like.
> >> Ideally, they would not duplicate the existing gcc aport, but still
> >
> >be
> >
> >> able to use the same patch files.
> >
> >I would prefer to add proper multi-arch support.  From there the path
> >to
> >having cross-compilers becomes more obvious.  Technically, it is
> >possible to
> >build them already using bootstrap.sh, but without real multi-arch
> >support
> >it's not so useful.
> >
> >> I've recently written down how it looks like from the postmarketOS
> >> perspective, and how we are working around it (TL;DR: we generate the
> >> gcc-*, binutils-*, musl-* APKBUILDs from Alpine's APKBUILDs with some
> >> cross compiler setup variables hardcoded). The issue also has some
> >> approaches for upstreaming, but I'm not really happy with any of
> >
> >them.
> >
> >> https://gitlab.com/postmarketOS/pmaports/-/issues/551
> >> 
> >> Bart pointed out that I should point this ML to that issue, and he's
> >> right with that, so here we are. I'd like to upstream this properly
> >
> >into
> >
> >> Alpine, but don't see a good way to do it. I'm curious if anybody has
> >
> >an
> >
> >> idea how to solve it elegantly.
> >> 
> >> (Maybe this could be solved in a better way by extending abuild?)
> >
> >Both abuild and apk need to be extended, as well as making APKBUILDs
> >cross-
> >aware by switching from $makedepends to $makedepends_build and
> >$makedepends_host.
> 
> Are we going to have cross only for core packages or for all packages? To be
> frank I'd rather not have it for all packages, as that's a lot of work -
> many upstreams don't test cross compilation and as such are pretty broken.
> I'd rather not invest the effort of marking all packages !cross that are
> broken, especially when we have native builders anyway.

The idea is to enable you to add aarch64 repos to your x86_64 machine and then 
cross with those prebuilt binaries.  This is mostly for end-users who want to 
cross, we would still be building packages outside the bootstrap path 
natively.

Ariadne
Timo Teras <timo.teras@iki.fi>
Details
Message ID
<20200730131203.27a68f16@vostro.wlan>
In-Reply-To
<2775783.pU9fvGE0RX@localhost> (view parent)
DKIM signature
missing
Download raw message
On Fri, 10 Jul 2020 12:44:35 -0600
Ariadne Conill <ariadne@dereferenced.org> wrote:

> On Friday, July 10, 2020 12:42:20 PM MDT you wrote:
> > On July 10, 2020 4:07:34 PM UTC, Ariadne Conill
> > <ariadne@dereferenced.org>   
> wrote:
> > >Hi,
> > >
> > >On Tuesday, July 7, 2020 3:58:35 PM MDT Oliver Smith wrote:  
> > >> Dear Alpine developers,
> > >> 
> > >> it would be nice if one could just install cross compilers that  
> > >
> > >target
> > >  
> > >> all of Alpine's supported architectures. For example:
> > >> 
> > >> $ apk add gcc-aarch64
> > >> 
> > >> This isn't a new idea and has been somewhat discussed on this
> > >> mailing list before - but without result. The problem appears to
> > >> be that it's unclear how the APKBUILDs of these gcc-* aports
> > >> should look like. Ideally, they would not duplicate the existing
> > >> gcc aport, but still  
> > >
> > >be
> > >  
> > >> able to use the same patch files.  
> > >
> > >I would prefer to add proper multi-arch support.  From there the
> > >path to
> > >having cross-compilers becomes more obvious.  Technically, it is
> > >possible to
> > >build them already using bootstrap.sh, but without real multi-arch
> > >support
> > >it's not so useful.
> > >  
> > >> I've recently written down how it looks like from the
> > >> postmarketOS perspective, and how we are working around it
> > >> (TL;DR: we generate the gcc-*, binutils-*, musl-* APKBUILDs from
> > >> Alpine's APKBUILDs with some cross compiler setup variables
> > >> hardcoded). The issue also has some approaches for upstreaming,
> > >> but I'm not really happy with any of  
> > >
> > >them.
> > >  
> > >> https://gitlab.com/postmarketOS/pmaports/-/issues/551
> > >> 
> > >> Bart pointed out that I should point this ML to that issue, and
> > >> he's right with that, so here we are. I'd like to upstream this
> > >> properly  
> > >
> > >into
> > >  
> > >> Alpine, but don't see a good way to do it. I'm curious if
> > >> anybody has  
> > >
> > >an
> > >  
> > >> idea how to solve it elegantly.
> > >> 
> > >> (Maybe this could be solved in a better way by extending
> > >> abuild?)  
> > >
> > >Both abuild and apk need to be extended, as well as making
> > >APKBUILDs cross-
> > >aware by switching from $makedepends to $makedepends_build and
> > >$makedepends_host.  
> > 
> > Are we going to have cross only for core packages or for all
> > packages? To be frank I'd rather not have it for all packages, as
> > that's a lot of work - many upstreams don't test cross compilation
> > and as such are pretty broken. I'd rather not invest the effort of
> > marking all packages !cross that are broken, especially when we
> > have native builders anyway.  
> 
> The idea is to enable you to add aarch64 repos to your x86_64 machine
> and then cross with those prebuilt binaries.  This is mostly for
> end-users who want to cross, we would still be building packages
> outside the bootstrap path natively.

The problem is still that ./configure often tries to build and execute
code, and that goes often wrong with cross unless given a bunch of
configure options.

I think yes, everything towards this goal is welcome. But often
cross-built is still not as reliable / well tested as native built.

But agreeably the tooling should better support this.

Enhancing the APKBUILDs more cross-aware and adding the cross compilers
are first steps. But definitely not the full solution.

Timo
Details
Message ID
<1814201.JbZtjmkz0I@localhost>
In-Reply-To
<20200730131203.27a68f16@vostro.wlan> (view parent)
DKIM signature
missing
Download raw message
Hello,

On Thursday, July 30, 2020 4:12:03 AM MDT Timo Teras wrote:
> On Fri, 10 Jul 2020 12:44:35 -0600
> 
> Ariadne Conill <ariadne@dereferenced.org> wrote:
> > On Friday, July 10, 2020 12:42:20 PM MDT you wrote:
> > > On July 10, 2020 4:07:34 PM UTC, Ariadne Conill
> > > <ariadne@dereferenced.org>
> > 
> > wrote:
> > > >Hi,
> > > >
> > > >On Tuesday, July 7, 2020 3:58:35 PM MDT Oliver Smith wrote:
> > > >> Dear Alpine developers,
> > > >> 
> > > >> it would be nice if one could just install cross compilers that
> > > >
> > > >target
> > > >
> > > >> all of Alpine's supported architectures. For example:
> > > >> 
> > > >> $ apk add gcc-aarch64
> > > >> 
> > > >> This isn't a new idea and has been somewhat discussed on this
> > > >> mailing list before - but without result. The problem appears to
> > > >> be that it's unclear how the APKBUILDs of these gcc-* aports
> > > >> should look like. Ideally, they would not duplicate the existing
> > > >> gcc aport, but still
> > > >
> > > >be
> > > >
> > > >> able to use the same patch files.
> > > >
> > > >I would prefer to add proper multi-arch support.  From there the
> > > >path to
> > > >having cross-compilers becomes more obvious.  Technically, it is
> > > >possible to
> > > >build them already using bootstrap.sh, but without real multi-arch
> > > >support
> > > >it's not so useful.
> > > >
> > > >> I've recently written down how it looks like from the
> > > >> postmarketOS perspective, and how we are working around it
> > > >> (TL;DR: we generate the gcc-*, binutils-*, musl-* APKBUILDs from
> > > >> Alpine's APKBUILDs with some cross compiler setup variables
> > > >> hardcoded). The issue also has some approaches for upstreaming,
> > > >> but I'm not really happy with any of
> > > >
> > > >them.
> > > >
> > > >> https://gitlab.com/postmarketOS/pmaports/-/issues/551
> > > >> 
> > > >> Bart pointed out that I should point this ML to that issue, and
> > > >> he's right with that, so here we are. I'd like to upstream this
> > > >> properly
> > > >
> > > >into
> > > >
> > > >> Alpine, but don't see a good way to do it. I'm curious if
> > > >> anybody has
> > > >
> > > >an
> > > >
> > > >> idea how to solve it elegantly.
> > > >> 
> > > >> (Maybe this could be solved in a better way by extending
> > > >> abuild?)
> > > >
> > > >Both abuild and apk need to be extended, as well as making
> > > >APKBUILDs cross-
> > > >aware by switching from $makedepends to $makedepends_build and
> > > >$makedepends_host.
> > > 
> > > Are we going to have cross only for core packages or for all
> > > packages? To be frank I'd rather not have it for all packages, as
> > > that's a lot of work - many upstreams don't test cross compilation
> > > and as such are pretty broken. I'd rather not invest the effort of
> > > marking all packages !cross that are broken, especially when we
> > > have native builders anyway.
> > 
> > The idea is to enable you to add aarch64 repos to your x86_64 machine
> > and then cross with those prebuilt binaries.  This is mostly for
> > end-users who want to cross, we would still be building packages
> > outside the bootstrap path natively.
> 
> The problem is still that ./configure often tries to build and execute
> code, and that goes often wrong with cross unless given a bunch of
> configure options.

Some of that I am addressing with autoconf-policy but I haven't had much time 
to work on that lately.
 
> I think yes, everything towards this goal is welcome. But often
> cross-built is still not as reliable / well tested as native built.
> 
> But agreeably the tooling should better support this.

+1, it will make life a lot easier for the pmOS project amongst others if we 
can improve cross support I think.

> Enhancing the APKBUILDs more cross-aware and adding the cross compilers
> are first steps. But definitely not the full solution.

Agreed, and in Alpine ecosystem there is also dabuild which can cross using 
QEMU.  But that is of course slow and not foolproof either.

Ariadne
Reply to thread Export thread (mbox)