Date: Mon, 30 Dec 2019 14:55:42 +0200
From: Timo Teras <timo.teras@iki.fi>
To: ~alpine/devel@lists.alpinelinux.org
Subject: new package format and repository layout changes
Message-ID: <20191230145542.1a7ca9cf@vostro>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Hi all,

I am currently going through the list of data that goes to a package
and a repository index, as well as the installed db. And trying to
draft the first schema version of what data goes where. So I am now
having some issues I'd like to discuss here.

1. Repository pinning

There's one fundamental issue in the current installed-db that causes
pain - especially when we do strong signing. This is how the package
pinning is done (the "@edge" tagging to enable specific repositories
for specific dependencies only).

Main problem is that the package origin repository needs to be tracked
for detecting pinning changes and it's not in the package meta data
currently. The current workaround is to have the origin tag in
installed-db which means there's data that cannot be signed ahead of
time. There's also some other subtle issues.

My thinking is to start putting the repository meta data (distribution
name, branch, component etc.) in the package. This way the package
origin is known and signed.

2. repositories list format

If the above happens, we might need to do some changes how the tags are
specified.

There was some discussion earlier if we should support more debian
style definition of listing the distro repositories. E.g.
  http://dl-cdn.alpinelinux.org/alpine edge main community

Where the first word is the base URL (or perhaps even some $MIRROR
variable). The second word the distribution branch. And remaining words
would be the list of enabled repositories.

I think the package naming could then be:
  $base_url/$branch/$repo/$arch/$pkgname-$pkgrel.$uniqueid.$arch.apk
and automatically constructed from the package metadata.

(Also wondering if the $uniqueid should be just random generated uuid,
or some sort of hash calculated from the package metadata and contents.
The requirement is that it can be used to identify if two packages are
the same or not.)

3. 'noarch' handling

When implementing the above, I would finally like to properly implement
the 'noarch'. Currently the sources set 'noarch' and build subpackage
properly. But they are put to the target architecture's storage and
when creating index the arch is rewritten to the target arch always.
The plan is to start creating real 'noarch' repository and put the
built package there. I'm wonder if we'd put separate index there, or
include the noarch packages also in the target arch index.

4. version handling

Sort of unrelated, but something I'd like to also bring up once again.
Since now that if we do proper distribution / branch tracking. And the
package downgrades happen at times. I'm wondering if we should make the
package version "informative" only. And use the build_time to decide
which package is the "preferred version". In most cases it is the
latest built package from the repository we want to be using.

Alternative would be introduce some sort of concept similar to
debian/pacman package "epoch".

Though another way to look at it that the buildtime is the
automatically generated epoch number. :)

--

Any thoughts, comments or concerns regarding the above planned changes?

Thanks,
Timo