MIME-Version: 1.0
References: <CAD+eXGRiTnrSnkE5iyS5cYB1bTR8JXOLzvYyW0-eEc0VtusZKA@mail.gmail.com>
In-Reply-To: <CAD+eXGRiTnrSnkE5iyS5cYB1bTR8JXOLzvYyW0-eEc0VtusZKA@mail.gmail.com>
From: Konstantin Kulikov <k.kulikov2@gmail.com>
Date: Wed, 11 Sep 2019 19:14:50 +0000
Message-ID: <CAD+eXGQmWT+2ASHj+423TyosrHFRF974ogY+SqSCxYP99JQrdA@mail.gmail.com>
Subject: Re: Packaging go projects
To: ~alpine/devel@lists.alpinelinux.org
Content-Type: text/plain; charset="UTF-8"

Working with abuild a bit more I've came up with a simpler way.
Instead of redefining fetch, making it depend on go and putting stuff
in SRCDEST,
we can instead define a few env vars in APKBUILD like this:

export GOPROXY=${GOPROXY:-https://proxy.golang.org}  # Default in
go1.13 and can be removed.
export GOPATH=${GOPATH:-$srcdir/go}
export GOCACHE=${GOCACHE:-$srcdir/go-build}
export GOTMPDIR=${GOTMPDIR:-$srcdir}

This way "go build" will download modules from GOPROXY and place all
build files inside srcdir.
Hoverwer if user has those vars in their environment (ie local dev
machine) or in abuild.conf (ie ci checks or build server),
they will be used instead. This way devs work as they usually do (no
difference between running "go build" manually and from abuild)
and CI can opt in into module and/or build caches if they want to
remove calls to external network and speed up builds.

Build cache has one case when it doesn't always rebuild packages when
it should so enabling it on build servers now might be problematic.
https://github.com/golang/go/issues/24355

Module cache should work well.

I've updated my example:
https://github.com/alpinelinux/aports/commit/ac34f80cdadd78523fde160e1bbc032dd04bfbca
Previous version is here:
https://github.com/alpinelinux/aports/commit/7892cf94cd917c080a0c381caf5ac5e90aa84c97

On Wed, Jul 31, 2019 at 7:59 PM Konstantin Kulikov <k.kulikov2@gmail.com> wrote:
>
> Hey everyone.
>
> I'm trying to package a go project which uses modules. This is what I
> came up with.
>
> https://github.com/alpinelinux/aports/pull/9800
>
> Notes below:
> 1. fetch() depends on go command which means that abuild checksum will
> fail if go is not installed.
> 2. $SRCDEST will contain "go" directory with module archives using
> layout defined by goproxy protocol (see
> https://godoc.org/cmd/go#hdr-Module_proxy_protocol). The good thing is
> that directory can be served over http and used as real GOPROXY,
> avoiding contacting google or other proxies unnecessarily. Current go
> projects in aports run "glide install" or "dep ensure" inside
> prepare() or build() and do not cache sources at all.
> 3. GOPROXY should be defined in abuild.conf and default to
> https://proxy.golang.org or any other proxy to avoid depending on
> git,svn,bzr and others and allow users to use their own proxies
> (googles proxy is unavailable in China for example).
> 4. sha512sums doesn't record hashes of module files, instead they are
> recorded in go.sum file and go tool verifies them itself.
> 5. chmod line in fetch() is unfortunate, without it cleanup step will
> fail. Go devs previously refused to add option to make unpacked module
> files writable.
> 6. fetch() will be the same for every go project and can be defined in
> abuild and used with options="gomodule" for example, same goes for
> GOPATH var which should be set to the same value for all steps.
> 7. To build a project we create dummy module which I called "apk" (but
> can be any name). It allows to record exact version of every used
> dependency, selectively upgrade or replace those dependencies and
> record that info in binaries (it can be inspected using
> rsc.io/goversion or, since go 1.13, "go version").
> 8. Patching is undoable in a way it's done in alpine (using .patch
> files) because checksums are recorded in binaries and can be verified
> against google's sumdb. Instead maintainer will need to fork project,
> apply patches, tag release and use their fork in APKBUILD.
> 9. To create new package contibutor would need to create apk.go and
> "blank import" all binaries in it, run "go mod init apk && go mod
> tidy", then write APKBUILD as usual.
> 10. To update a package `go get $(go list -f '{{ join .Imports " "
> }}')` can be used. If specific version is wanted instead of @latest,
> run manually "go get github.com/9seconds/mtg@v0.16"
>
> Does this look sane to you? Should we package it like this? Any other ideas?