Received: from mail.cmpwn.com (mail.cmpwn.com [45.56.77.53])
	by nld3-dev1.alpinelinux.org (Postfix) with ESMTPS id AF0E0782BDE
	for <~alpine/devel@lists.alpinelinux.org>; Thu, 23 Jan 2020 15:45:51 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cmpwn.com; s=cmpwn;
	t=1579794349; bh=5qoJLBgDd1vTfGcGi5PIH41bVxDfzwiQAQXHxMW97Gw=;
	h=In-Reply-To:Date:Subject:From:To;
	b=n2WIhHT1weuFd8NTd6KC3y2Q5n7vhusmeCShL6FjEXJ4ghqpStt7dhJhylwEM9Raw
	 iLcND6mqtoSzCLWp/W+O/IrWzNfQ9zQqQUe90J3QVIPWhi6p3/AikDISwaQREPawSV
	 LahBCgURcztvFK3qYwJCFzqa9B2k20PUAMmwS6Mc=
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <1c4796e0cda2248c2de159d4d467421c@dereferenced.org>
Date: Thu, 23 Jan 2020 10:36:10 -0500
Subject: Re: Lets talk about apk-tools 3, and apk-tools in 2020 in general
From: "Drew DeVault" <sir@cmpwn.com>
To: "Ariadne Conill" <ariadne@dereferenced.org>,
 <~alpine/devel@lists.alpinelinux.org>
Message-Id: <C03A5J4BZ2TN.2VO1P2UDDOI6Q@homura>

Thanks for writing this up, Ariadne. I agree with everything you said.
I'd like to call attention to a few points:

On Thu Jan 23, 2020 at 3:13 PM, Ariadne Conill wrote:
> * Changing the format in such a radical way brings significant
> risk. The tar streams code has already been audited and a
> few CVEs have been fixed over the years. Throwing that out
> means we start over again, possibly reintroducing variations
> of bugs we have already fixed. Many stakeholders have said
> privately that they would rather not have exposure to this
> risk and would prefer a more conservative approach.

We ought to be careful to avoid fixing things which aren't broken,
noting that unoptimal !=3D broken. I do NOT see APKv3 as an opportunity to
scratch a bunch of itches. Let's keep the changes as small as possible
and avoid throwing out lots of code, or introducing lots of new code.

We should structure these changes as slow, incremental improvements,
where one change can be built and shipped and then fully battle tested
before the next one lands. apk is used in many production systems today
and has production-tier expectations of stability. The importance of a
conservative approach is probably the most relevant take-away from this
discussion.

> * Usage of a unified container format for package data and
> database data removes transparency from the current package
> format. Right now, an APK package can be manipulated with
> the tar command if a user wishes to know its contents. Using
> the package manager is not even required.

Just to lend validation this point: I use /bin/tar to study apk files
all the time. Tar beats homegrown 10 times out of 10.

> There are other changes that people are concerned about, such
> as being able to compose new repositories from pre-existing
> ones. While those are important discussions to have as well,
> we are not really discussing them here, as those concerns can
> easily be overcome. Overall governance of the apk-tools
> project itself is also not necessarily being discussed here,
> while we need to have that discussion as there are many
> non-Alpine stakeholders at this point, we can do that later.

I was one of the people bringing these concerns forward. I think that
tightening the scope and focusing on reuse of existing designs and code
makes this easier - if we can avoid overhauling the container format
then we don't have to bikeshed about what goes into it.

Finally, in terms of general principles, I want to emphasize that we
shouldn't turn ourselves into paperclip maximizers. Performance isn't
the singluar metric we need to optimize - maintainability, reliability,
and usability are all _more_ important. Any optimizations which cannot
be proven to directly improve known bottlenecks in apk are a NACK from
me. apk is already one of the fastest, if not _the_ fastest, package
managers around.