Date: Sat, 20 Nov 2021 21:05:38 +0100
To: ~alpine/devel@lists.alpinelinux.org
Subject: Re: Thoughts on self-hosting compilers in Alpine
From: =?UTF-8?Q?S=C3=B6ren?= Tempel <soeren@soeren-tempel.net>
References: <33KG0XO61I4IL.2Z7RTAZ5J3SY6@8pit.net>
 <9c3ab565-3489-62e4-c15e-a7af3c2ef569@dereferenced.org>
In-Reply-To: <9c3ab565-3489-62e4-c15e-a7af3c2ef569@dereferenced.org>
Message-Id: <2DGGIA6U0PY5D.3SO8BVCBQLL47@8pit.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Ariadne Conill <ariadne@dereferenced.org> wrote:
> Hi,

Hello,

Thanks for your input, comments below.

> Also gcc and clang.

Yes. I would also like to stress that the goal of the proposal (and the
goal of the Bootstrapple Builds project) is minimizing the amount of
bootstrap binaries and making use of bootstrap binaries transparent, not
eliminating bootstrap binaries entirely.

> The solution here is to version key system libraries like libffi3.3 and s=
o=20
> on.  This isn't done as much as it needs to be done.  Every time I bring=20=

> this up, I am told that we don't need to do something we need to be doing=
.=20
> This proposal would be far more expensive (rebootstrapping) than just=20
> doing the right thing.

Sure, that works as well for this particular problem but doesn't solve
the other two problems I outlined in the original post.

> > 3. Builder setup: When setting up builders for a new Alpine release
> >   (e.g. as we did a few weeks ago for 3.15), manual intervention is
> >   necessary to install an initial version for every packaged self-hosti=
ng
> >   compiler from Alpine Edge.
>=20
> An alternative would be to provide a bootstrap repo providing only the=20=

> bootstrap compilers and their dependencies, and no other packages from=20=

> edge.

Could you elaborate on that? Because self-hosted compilers we are
shipping presently have dependencies on other packages in Edge (e.g. GHC
depending on libffi).

> What happens for cases where architectures were added *late* into the=20
> bootstrap path, such as ppc64le?  A stage0 would not exist for these.

Depends on the self-hosting compiler. Regarding Go, there are two Go
implementations: The Google implementation written in Go and the GCC
implementation written in C/C++ (which we package as gcc-go). A stage0
for ppc64le could use the GCC implementation (instead of using Go 1.4).

Generally speaking, the stage0 should use whatever upstream recommends
for bootstrapping their compiler on a given architecture. I personally
wouldn't mind if the stage0 is cross-compiled from Ubuntu using a Docker
file in an APKBUILD with a snapshot() rule (as we did with
ghc-bootstrap). The goal of the stage0 is simply making the initial
bootstrap transparent.

> As noted above, rebootstrapping the whole chain of compilers would be=20
> highly expensive for languages like Rust.  Each step would take hours.

I am not suggesting to rebootstrap the entire chain. Ideally
${self_hosting_compiler}-stage0 should always be capable of building
${self_hosting_compiler} at any point in time. As such, the chain would
be:

	${self_hosting_compiler}-stage0 -> ${self_hosting_compiler}

which shouldn't be expensive. I don't know Rust so I don't know what
upstream suggests regarding bootstrap, but as an example it would be
possible for the Rust stage0 to use (static?) binaries provided by Rust
upstream.

> This proposal is not possible for Rust: mrustc is not capable of serving=20=

> as stage0 for several architectures, at least riscv64, s390x, as 1.29 did=
=20
> not have musl support for them.

I am not proposing use of mrustc *right now*. This proposal isn't about
any self-hosting compiler in particular. I am just saying that using
mrustc may become feasible in the future, thereby eliminating the need
to trust pre-existing Rust binaries for bootstrapping rustc entirely.

BTW: Just as a case-study I started implemented this proposal for the
Cyclone C compiler were upstream-provided autogenerated C files are used
for the -stage0 <https://github.com/justinethier/cyclone-bootstrap>:

	https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/27636

Greetings,
S=C3=B6ren