Subject: Implement versioned names for major libraries to ease upgrade process

From: maxice8 <[email-masked]>
Date: Sat, 12 Dec 2020 06:36:16 -0300

Hello,

Distros like Debian and Void Linux have packages for major version of 
libraries include the major version in their pkgname (example: 
libboost1.75).

I'd like to implement that for Alpine Linux.

## Benefits to Alpine

Ease the upgrade process of major libraries with lots of consumers (like 
boost that has 127-ish consumers) by allowing 2 or more versions of a 
library to be installed in parallel.

Consumers will no longer need to be rebuilt all at once as users can 
have multiple versions of the library installed and will be slowly 
switched over to the new version as consumers are rebuilt to use the new 
soname.

This is already done for llvm which breaks API and ABI every MAJOR 
release so we keep llvmN and llvmN+1 and remove llvmN once all packages 
are switched over to use llvmN+1.

## Drawbacks

More APKBUILDs will need to be maintained until the whole process is done.

If we depend on foo-dev with foo-dev being claimed by the latest version 
of foo, it becomes possible for some packages to be missed during the 
upgrade process and never processed until the older foo is removed.

Otherwise if we decide on depending on fooX-dev directly then it 
increases work on upgrading packages but guarantees no package will be 
missed in the way mentioned above.

People might start manually installing fooX instead of installing foo 
and just getting the latest version and then their system **could 
possibly** break during an upgrade.

## Workflow

First time:

1. Upstream releases a new major version
2. New aport is created that has pkgname that includes the major version
    and claims common subpackages (-dev, -doc, -openrc, -*sh-completion,
    etc) via provides="foo-dev=$pkgver-r$pkgrel".
3. Consumers are rebuilt all at once (for the last time)
4. the old aport is removed

All following times:

1. Upstream release a new major version
2. New aport is created that has pkgname that includes the major version
    and claims common subpackages (-dev, -doc, -openrc, -*sh-completion,
    etc) via provides="foo-dev=$pkgver-r$pkgrel".
3. Old aport is rebuilt without the provides=
4. Consumers are rebuilt as possible/expedient
5. old aport is removed

## Timeline

This will be implemented as soon as 3.13 is released and consensus is 
reached (assuming it is positive)

Regards
Leo

---

From: Ariadne Conill <[email-masked]>
Date: Sat, 12 Dec 2020 10:36:46 GMT

> libraries include the major version in their pkgname (example:

We already do this with some libraries.  I think it would be good to make it 
standard practice.

> missed in the way mentioned above.

One solution here would be for fooX-dev to also provide foo-dev=X.  This will 
cause abuild to pull in the newest package if possible.

> possibly** break during an upgrade.

It should be fine, as the dependencies are against so:libfoo.so.X and not 
actually against fooX.

Ariadne

---

From: maxice8 <[email-masked]>
Date: Sat, 12 Dec 2020 08:12:00 -0300

Hello

> We already do this with some libraries.  I think it would be good to make it

Can be as long as there aren't multiple versions laying around, the 
express purpose is to ease the burden of upgrading major libraries with 
lots of consumers like boost.

> cause abuild to pull in the newest package if possible.

That is the way of the paragraph above, if we decide on foo-dev we can 
miss a package, if we pick fooX-dev then we will never miss a package 
because we just have to move all fooX-dev to fooY-dev.

Another problem is the dependency ordering if we don't queue the builds 
carefully some packages won't be rebuilt properly (talking from 
experience on Void Linux).

> It should be fine, as the dependencies are against so:libfoo.so.X and not

It is more about the possibility of something funky happening when 
upgrading with deleted packages, I have no clue if there is any actual 
problem but I'm staying on the safe side.

Regards,
Leo

---

From: Nico Schottelius <[email-masked]>
Date: Sat, 12 Dec 2020 11:16:01 +0100

Good morning,

maxice8 <[email-masked]> writes:
> [...]
> release so we keep llvmN and llvmN+1 and remove llvmN once all
> packages are switched over to use llvmN+1.
> [...]

if we are already doing that, why do we not do it for the linux kernel?

Cheers,

Nico

--
Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch

---

From: Rasmus Thomsen <[email-masked]>
Date: Sat, 12 Dec 2020 11:30:23 +0100

Hello,
> if we are already doing that, why do we not do it for the linux

Because maintaining multiple kernels is a lot of work because they're
released relatively often and we have to rebuild all modules (e.g. zfs)
against new kernel versions.

Regards,

Rasmus Thomsen

---

From: Ariadne Conill <[email-masked]>
Date: Sat, 12 Dec 2020 10:33:34 GMT

Hello,

> against new kernel versions.

I don't think they propose maintaining multiple kernels in parallel per se, 
but instead versioning the package (with a provides linux-lts=${pkgver}) so 
that older kernels can stick around if a newer kernel has a regression.

Ariadne

---

From: Nico Schottelius <[email-masked]>
Date: Sat, 12 Dec 2020 12:16:47 +0100

> that older kernels can stick around if a newer kernel has a regression.

Correct - it's not about maintaining them, but to `apk upgrade` without
being forced to reboot, because some modules are missing after the
upgrade.

I'd imagine something on the lines of:

a)
    "on kernel upgrade, do not remove linux-lts-$(uname -r)"

    However all other older versions can be replaced.

    This would already solve a major pain point.

or b)
   Keep all previous kernel versions and let the user delete old
   versions manually

   I think this is +/- what Debian is doing. It results in a very long
   grub list and is annoying to cleanup, but it also give a lot of
   security, being able to switch back to an older version.

I think (a) is cleaner, but probably more complicated to implement, as
(b) could just mean that the package *name* differs and they all provide
the linux-kernel.

Cheers,

Nico

---

From: Natanael Copa <[email-masked]>
Date: Wed, 20 Jan 2021 10:23:09 +0100

On Sat, 12 Dec 2020 12:16:47 +0100

> >> > if we are already doing that, why do we not do it for the linux

Kernel is a bit different animal.

Currently, we don't version the /boot/vmlinuz-lts which means that we
shouldn't need to regenerate the syslinux config. It also means that
multiple versions of the kernel cannot be installed at the same time.

But it is also a question of how to implement it in a clean way for
package manager.

> (b) could just mean that the package *name* differs and they all provide

I added a few comments here:
https://gitlab.alpinelinux.org/alpine/aports/-/issues/12157#note_134555

After some thought I think what would be nice is:

- mark the kernel package with some flag, "keep-previous" or similar,
  which tells apk that on upgrade, files should not be deleted, but
  moved to a virtual package .$pkgname-previous or similar.

- new kernel gets installed, while previous remains on the system, so
  modprobe still works for running kernel. User can manually delete
  .$pkgname-previous to uninstall the previous kernel if wanted.

- during (re)boot, as last step, we have an openrc script that checks for
  .$pkgname-previous, and deletes it unless it is the running kernel.
  That way, if kernel fails to boot, the old kernel is still around as
  fallback. If boot of new kernel is successful, it will clean up the
  previous kernel and user does not need to manually remove old kernels.

To implement this we need to:

- add a version to /boot/vmlinuz-$flavor and initramfs
- add the move-ownership-of-files-if-keep-previous feature to apk-tools.
- figure out what to do if .$pkgname-previous already exists during upgrade.

---

From: Rasmus Thomsen <[email-masked]>
Date: Sat, 12 Dec 2020 11:38:44 +0100

Hello,

> library to be installed in parallel.

Hm, I think the only time users interact with major libraries upgrades
are on edge or if they upgrade between major versions. As such I don't
think the effort/maintaince overhead versioned libraries introduce is
worth it just to not break edge during rebuilds of big packages like
boost.

One thing that might be interesting about having versioned names for
libraries would be that it would make using edge packages on stable
releases less broken, but if we only do it for major libraries it'd
still be pretty broken for packages which use un-versioned libraries. 

> are switched over to use llvmN+1.

I think we only do this for LLVM because packages tend to take a long
time before they switch over to new LLVM versions because it's so much
effort to stay on top of LLVM effort.

---

From: Ariadne Conill <[email-masked]>
Date: Sat, 12 Dec 2020 10:52:13 GMT

> still be pretty broken for packages which use un-versioned libraries.

Adopting versioning practice for libraries we choose to not have multiple 
versions of is still valuable because we can immediately provide new versions 
of the library in the futue without breaking anything.

> effort to stay on top of LLVM effort.

We also do this with GTK+ because some programs want GTK+2, others want GTK+3 
and others want GTK+4.  We also did this with Qt 4 vs Qt 5 for a long time, 
and we will probably do this with Qt 5 vs Qt 6 for some time too.

Ariadne

---

From: maxice8 <[email-masked]>
Date: Sat, 12 Dec 2020 08:25:13 -0300

Hello,

> worth it just to not break edge during rebuilds of big packages like

I don't think the maintenance effort is that big, there is none in a 
stable release because there is only one major version of the library 
present, the only thing that changes is when you install *foo* it brings 
*lib$VERSION* instead of *lib*.

In edge the old aports would be removed as soon as the upgrade process 
is complete.

> still be pretty broken for packages which use un-versioned libraries.

I don't have any plans for mixing edge and stable releases.

> effort to stay on top of LLVM effort.

Yes, I meant that we already have basic knowledge in how to implement 
packages with versioning embedded in the pkgname.

Regards
Leo

---

From: maxice8 <[email-masked]>
Date: Sun, 17 Jan 2021 02:56:22 -0300

Hello

I have done the boost upgrade to 1.75.0, and the subsequent rebuild of
all its reverse dependencies, with the help from J0WI:

1. Rebasing J0WI's Merge request was a chore on its own as things get
outdated by other changes, when rebasing 107! commits more than half
had conflicts and some were lost.
2. CI failed to build the whole thing so we had to go with knowledge
there would be failures, and there were lots of failures.
3. The builders were clogged for 3-4 hours at minimum.
4. Thankfully other developers/contributors stepped up to bump packages
that were missed.

So I have made a merge request[1] to make boost into boost1.75 to ease
the upgrade process.

I want to do this for other packages that have **huge** amount of
reverse dependencies and packages where upstream supports parallel
installing (by having headers in namespaced directories
(/usr/include/foo-1.0) and libraries namespaced themselves
(/usr/lib/libfoo-1.0.so.1).

Regards
Leo

[1]:
https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/16894

---

From: Natanael Copa <[email-masked]>
Date: Wed, 20 Jan 2021 09:59:56 +0100

On Sat, 12 Dec 2020 06:36:16 -0300

> I'd like to implement that for Alpine Linux.

This is requirement for all the libs in build-base, otherwise the old
version in use will prevent the new -dev package to be installed.

> switched over to the new version as consumers are rebuilt to use the new 

This may also create problems. What I am afraid of that we end up with
3 different version of boost in stable branches. And a single CVE needs
to be fixed in 3 * 5 branches (4 stable + edge). It means that every
single CVE needs to fixed in 15 packages.

> are switched over to use llvmN+1.

The reason for llvm was that some packages requires a specific, often
older, version of llvm. For example crystal.

We actually have a problem now because chromium 87 seems to require
llvm-11, which we dont have for 3.13-stable.

> If we depend on foo-dev with foo-dev being claimed by the latest
> version of foo, it becomes possible for some packages to be missed
> during the upgrade process and never processed until the older foo is
> removed.
> 
> increases work on upgrading packages but guarantees no package will
> be missed in the way mentioned above.
> 
> 2. New aport is created that has pkgname that includes the major
> version and claims common subpackages (-dev, -doc, -openrc,
> -*sh-completion, etc) via provides="foo-dev=$pkgver-r$pkgrel".

provides="foo-dev=$pkgver-r$pkgrel" will not work.

You need subpackages="foo-dev". The reason is that otherwise it will be
impossible to calculate the build order until after the build is done.

The build order calculator needs to know how to find foo-dev, and it
cannot do that if it is a provides in a split function.

Have a look at mpfr/mpfr3/mpfr4 git history for an example.

> 2. New aport is created that has pkgname that includes the major
> version and claims common subpackages (-dev, -doc, -openrc,
> -*sh-completion, etc) via provides="foo-dev=$pkgver-r$pkgrel".

---

From: maxice8 <[email-masked]>
Date: Wed, 20 Jan 2021 06:52:17 -0300

> version in use will prevent the new -dev package to be installed.

Ok

> single CVE needs to fixed in 15 packages.

I don't expect that to happen, the creation of versioned names isn't
meant for the user, it is explicitly meant for us (maintainers) to have
some time (a few days to weeks, maybe up to a month if we find a
particularly uncooperative upstream) to properly rebuild everything.

The last rebuild of boost 1.69.0->1.75.0 kept me awake all night fixing
stuff and it still technically isn't done because mkvtoolnix is failing
in one arch.

I am drafting an E-Mail about expected downtime while I upgrade python3
to 3.9.1 (which breaks modules with 3.8.1).

> llvm-11, which we dont have for 3.13-stable.

The reason is not of much relevance, what is relevant is that we have
some basic experience in versioned packages.

> > 
> > ## Drawbacks
> > 
> > More APKBUILDs will need to be maintained until the whole process
> > is
> > done.
> > 

That is good to know and it means boost1.75 is busted, I'll make a fix
merge request.