MIME-Version: 1.0
References: <D194B900-6A60-4A2C-B520-513D8F1263D6@msys.ch>
In-Reply-To: <D194B900-6A60-4A2C-B520-513D8F1263D6@msys.ch>
From: Carl Chave <online@chave.us>
Date: Sun, 8 May 2022 07:43:42 -0400
Message-ID: <CAGP1gyPfhGBi2BjAq7D1fCfG0RyLo4zyyw_=Uhx69pfECmB9XQ@mail.gmail.com>
Subject: Re: Creating an .apk package from local files, without downloading
 and building sources
To: Marc Balmer <marc@msys.ch>
Cc: ~alpine/apk-tools@lists.alpinelinux.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Marc,

Not entirely the same use case as you, but a couple years ago I wanted
to resign some already built packages with a different key without
rebuilding from scratch and came up with the below (the main snippet).
It's not elegant and there's probably a better way etc. but caveats
aside, it does work. Maybe there's something in there that jumps out
at you:

for apk in ${apks}
do
  tmp=3D$(mktemp -dp .)
  chmod 0755 ${tmp}
  printf "=3D=3D=3D> Processing package ${apk}\n"
  tar -xzpf ${apk} -C ${tmp}
  rm ${apk}
  rm ${tmp}/.SIGN*.pub
  mv ${tmp}/.PKGINFO .
  cd ${tmp}
  tar --xattrs -f - -c $(ls -A1) | abuild-tar --hash | gzip -9 > ../data.ta=
r.gz
  cd ..
  sha256=3D$(sha256sum data.tar.gz | cut -d' ' -f1)
  sed -i '$d' .PKGINFO
  echo "datahash =3D ${sha256}" >> .PKGINFO
  tar -f - -c .PKGINFO | abuild-tar --cut | gzip -9 > control.tar.gz
  abuild-sign -k ${priv_key} control.tar.gz
  cat control.tar.gz data.tar.gz > ${apk}
  rm -rf control.tar.gz data.tar.gz .PKGINFO ${tmp}
  i=3D$(( ${i} +1 ))
done

On Sun, May 8, 2022 at 3:22 AM Marc Balmer <marc@msys.ch> wrote:
>
> G'day!
>
> TL;DR:
>
> We are looking for a way to build an .apk package from locally installed =
files, without going through the download and build stages that abuild norm=
ally does. I.e without an APKBUILD file.
>
> A caveat:  My first post to this list... I write this E-Mail using HTML f=
or better readability of the code.  If that is a no-go here, please tell me=
 and I will resend as plain text.
>
>
> Our setting
>
> We have a large git repo with the source code for many libraries and bina=
ries, much of them open source.  We build and package them for various linu=
x distributions and architectures.  We use docker containers to build for t=
he various systems.   We think that our binaries, packaged for AlpineLinux,=
 would be a good fit for container use.  We thus did setup a build containe=
r for Alpine Linux, which builds all of our stuff just fine.
>
> We are now trying to create .apk packages using the already compiled libr=
aries and binaries.  Our approach is to create a directory for each package=
 we want to build and to describe the package using a Makefile.  The packag=
es Makefile then includes a Makefile fragment that drives the actual packag=
e building.
>
> PACKAGE=3D lua
> PKGVER=3D 5.4.4-r1
> DESCRIPTION=3D "Lua interpreter and compiler"
> PACKAGER=3D micro systems <info@msys.ch>
> URL=3D https://lua.msys.ch
> MAINTAINER=3D micro systems <info@msys.ch>
> LICENSE=3D MIT
>
> FILES=3D usr/bin/lua \
> usr/bin/luac
>
> MKDIR?=3D ../../../mk/
>
> include $(MKDIR)alpine.package.mk
>
> The actual binaries are installed in the docker build container under /tm=
p/alpine.  We copy the files from there to a local staging directory "stagi=
ng" and build the package from there. The Makefile Fragment alpine.package.=
mk looks like this:
>
> LICENSE?=3D custom
> PACKAGER?=3D unknown
> DESCRIPTION?=3D unknwon
> PACKAGER?=3D unknown
> URL?=3D unknown
> MAINTAINER?=3D unknown
>
> INSTSCRIPTS!=3D find . -name .p\*-\*install -exec basename {} \;
> DATE_EPOCH!=3D date -u "+%s"
>
> .DEFAULT_GOAL=3D package
>
> package:
> # Create the data part
> @-mkdir -p staging
> @tar -C /tmp/alpine -f - -c ${FILES} | tar -C staging -x
>
> find staging -exec touch -h -d @${DATE_EPOCH} {} +
>
> @(cd staging; tar --xattrs -f - -c *) | abuild-tar --hash | \
> gzip > data.tar.gz
>
> (cd staging; find . -print0) | LC_ALL=3DC sort -z | tar -C staging --xatt=
rs \
> --xattrs-exclude=3Dsecurity.selinux \
> --format=3Dposix \
> --pax-option=3Dexthdr.name=3D%d/PaxHeaders/%f,atime:=3D0,ctime:=3D0 \
> --mtime=3D"@${DATE_EPOCH}" \
> --no-recursion --null -T - \
> -f - -c | abuild-tar --hash | gzip -n -9 >data.tar.gz
>
> # Create .PKGINFO
> @echo "# Generated by alpine.package.mk" > .PKGINFO
> @echo -n "# " >> .PKGINFO
> @date >> .PKGINFO
> @echo "package =3D ${PACKAGE}" >> .PKGINFO
> @echo "pkgver =3D ${PKGVER}" >> .PKGINFO
> @echo "pkgdesc =3D ${DESCRIPTION}" >> .PKGINFO
> @echo "url =3D ${URL}" >> .PKGINFO
> @echo "builddate =3D ${DATE_EPOCH}" >> .PKGINFO
> @echo "commit =3D 5ff2c19564c162690aa5abbfb46fd6d3960c1c24" >> .PKGINFO
> @echo "packager =3D ${PACKAGER}" >> .PKGINFO
> @echo -n "size =3D " >> .PKGINFO
> @(cd staging; du -b -c ${FILES}) | tail -n 1 | cut -f 1 >> .PKGINFO
> @echo "arch =3D x86_64" >> .PKGINFO
> @echo "origin =3D ${PACKAGE}" >> .PKGINFO
> @echo "maintainer =3D ${MAINTAINER}" >> .PKGINFO
> @echo "license =3D ${LICENSE}" >> .PKGINFO
> @for dep in ${DEPENDS}; do echo "depend =3D $${dep}" >> .PKGINFO; done
> @echo "# automatically detected:" >> .PKGINFO
> @for f in ${FILES}; \
> do \
> find /tmp/alpine -name $${f} -type f -perm +111; \
> echo -n "provides =3D cmd:" >> .PKGINFO; \
> basename $${f} | tr -d '\n' >> .PKGINFO; \
> echo "=3D${PKGVER}" >> .PKGINFO; \
> done
> @echo -n "datahash =3D " >>.PKGINFO
> @sha256sum data.tar.gz | cut -d " " -f 1 >> .PKGINFO
>
> # Create the control part (index)
> tar --format=3Dposix \
> --pax-option=3Dexthdr.name=3D%d/PaxHeaders/%f,atime:=3D0,ctime:=3D0 \
> --mtime=3D"@${DATE_EPOCH}" \
> -f - -c .PKGINFO ${INSTSCRIPTS} | abuild-tar --cut \
> | gzip -n -9 > control.tar.gz
>
>
> #@tar --owner=3D0 --group=3D0 --numeric-owner -c .PKGINFO ${INSTSCRIPTS} =
| \
> # abuild-tar --cut | gzip > control.tar.gz
> @abuild-sign -q control.tar.gz
>
> # Produce the final apk
> @cat control.tar.gz data.tar.gz > ${PACKAGE}-${PKGVER}.apk
> @cat control.tar.gz data.tar.gz > /var/apk/${PACKAGE}-${PKGVER}.apk
> -(cd /var/apk; apk index ${PACKAGE}-${PKGVER}.apk -o APKINDEX.tar.gz)
> abuild-sign -q /var/apk/APKINDEX.tar.gz
>
> # Show info and clean up
> @cat .PKGINFO
> @rm -rf data.tar.gz control.tar.gz .PKGINFO staging
>
> Essentially, we try to run the packaging steps in the same way as abuild =
does it (we used the abuild source
> code as a reference). Of course our approach is not as elegant as abuild.=
 Moreover, it does not work, the
> apk package being produced can not be installed.
>
> While the resulting .apk package looks like any other .apk package, and t=
he content looks correct, apk exits
> with exit code 99 when we try to add or index the package. apk add prints=
 "ERROR: <pkg name> Not supported."
>
> (Not supported seems to be the result from musl libc for an unknow error =
code).
>
> Do you have any ideas what we are doing wrong?
>
> Thanks,
> Marc
>
>