Date: Sun, 30 Dec 2018 22:17:56 -0500
From: Max Rees <maxcrees@me.com>
To: alpine-devel@lists.alpinelinux.org
Subject: Re: [alpine-devel] Report from Reproducible builds summit 2018
Message-ID: <20181231031754.GC9101@sachiel>
Mail-Followup-To: alpine-devel@lists.alpinelinux.org
References: <20181217133328.4dd1ef26@ncopa-desktop.copa.dup.pw>
 <b8bbde6c-90b0-4c96-59e5-8475ae655bb7@bitmessage.ch>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b8bbde6c-90b0-4c96-59e5-8475ae655bb7@bitmessage.ch>
User-Agent: Mutt/1.10.1 (2018-07-13)

On Dec 18 08:27 AM, Oliver Smith wrote:
> Hello Natanel and ML,
> 
> I'm glad to read about this, thank you for this writeup!
> 
> I've looked into reproducible builds myself last year, even had a proof
> of concept with a few packages. The tooling can't be re-used, as it was
> based on pmbootstrap from postmarketOS, not Alpine's abuild directly.
> But maybe I can help with some insights or contribute otherwise.
> 
> Natanael Copa:
> > * we may need to store the exact versions and/or hashes of the
> >   dependencies used when a package was built. I am not sure where we
> >   want store this. Maybe in the APKINDEX?
> 
> I had created a .buildinfo.json file, where I placed all dependencies
> that were installed at the build time, with their versions. That file
> was placed next to the main apk (so no extra buildinfo file for
> subpackages) in the binary repository directory. Storing the hashes
> would be even better. I chose JSON, as it's trivial to parse that with
> Python, but since Alpine's build tools are lightweight and do not depend
> on Python, using another format probably makes moer sense. The idea for
> this file was based on Debian's buildinfo file, that is described here:
> 
> https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles

Debian buildinfo seems like a source of good ideas. However I think
that, as you stated, the format should remain simple in order to remain
compatible with abuild. I've been working on abuildd and I think my
current rudimentary strategy would be to dump the contents of
`apk info -v` or similar to a file (probably including the APKINDEX "C:"
field as well) for each package origin (thus not subpackages). This'll
be possible since abuildd is based on rootbld and thus only build-base +
the (recursive) dependencies of the package will be installed for each
build. This information will naturally be quite verbose and thus it
probably shouldn't be put in the APKINDEX - perhaps in a separate git
repo?

> > * we embed the signature in the .apk, which means its not possible to
> >   re-create the exact same .apk without having access to the private
> >   key. I'm not sure how to deal with that.
> 
> My cheap workaround for that was: just make all files inside the .apk
> file reproducible, not the apk itself. It would be better to have the
> entire apk reproducible of course, but to do that, we would need to
> store the signature elsewhere (e.g. create a .sig file for each .apk).

This is also the solution I arrived at - the data.tar.gz should be
reproducible exactly (same datahash), and then we can compare most of
the .PKGINFO modulo the builddate, datetime comment, and packager.

Max


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---