Date: Thu, 21 May 2020 17:33:46 +0200
From: Natanael Copa <ncopa@alpinelinux.org>
To: Rasmus Thomsen <oss@cogitri.dev>
Cc: Alpine develmopment  <~alpine/devel@lists.alpinelinux.org>
Subject: Re: RFC: tool to scan aports tree
Message-ID: <20200521173346.139a0101@ncopa-desktop.copa.dup.pw>
In-Reply-To: <a668aa6389c0d4bb4798ed01d46f87620d6fd61a.camel@cogitri.dev>
References: <20200521154157.6df2d118@ncopa-desktop.copa.dup.pw>
	<a668aa6389c0d4bb4798ed01d46f87620d6fd61a.camel@cogitri.dev>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, 21 May 2020 16:56:31 +0200
Rasmus Thomsen <oss@cogitri.dev> wrote:

> On Thu, 2020-05-21 at 15:41 +0200, Natanael Copa wrote:
> > I would like to have some feedback on how you would like that the
> > interface should be?  
> 
> One thing I'd really love that tool to have would be more usability for
> developers, see below.
> 
> > I am not sure those are needed:
> > 
> > [...]
> > - print the reverse build time dependencies for a given package (ap
> >   revdep)  
> 
> That one might be useful for developers to rebuild things on soname
> bumps. Right now it's kind of a pain to rebuild everything in the right
> order when soname bumps occur, it'd be lovely if the new tool had
> something git-bisect-ish. That is if I could tell it "package X
> changed, run script Y for all packages that (directly) depend on it".

I use `apk list --depends so:libname.so.$old` to generate a list of
packages that needs to be rebuilt, which I feed to a script
'rebuild-since' which will bump pkgrel, rebuild and git commit unless
there are commits since specified commit hash (the commit which has the
ABI breaking library update). `ap builddirs` is used to calculate the
build order.

not sure if there is any value print one level of reverse dependencies.
 
> > Things that the tools needs to be able to help us to do:
> > 
> > - detect if there are any invalid dependencies (eg. packages in main
> >   that depends on packages in community or testing)  
> 
> I think that is something that CI catches anyway these days - but it
> might be useful when demoting a packge from main to community (or
> community -> testing).

May be nice to have the answer of: "if I move this from testing to
community, what other packages needs to also move?"

> > - detect missing build time dependencies (due to arch being
> > disabled)  
> 
> That sounds a bit redundant, you'd notice that when running abuild.

No. You don't. When qt5-qtwebkit was disabled for armhf there was a lot
of packages that broke. It was not detected by abuild until next
rebuild of those. Would be nice to solve that at the same time as it
gets disabled rather than wait til the builders fails during next
upgrade.

> > - detect circular build time dependencies  
> 
> That would be something that'd be really nice to have for CI so we
> don't run into these in the first place.

exactly.

> 
> > - clean source cache (delete downloaded sources older than X that is
> >   not currently used by any aports)  
> 
> That sounds nice, I usually do `rm -rf */*/src` in my aports tree
> every month or so when my disk starts filling up once again. It'd be
> nice if this could be run automatically somehow so I..err users don't
> forget to run this before things start to blow up because there's no
> space left in / once again ;)

This is not for the */*/src. Its for /var/cache/distfiles. If you have those files in /var/cache/distfiles, which ones are candidates to delete?

strongswan-5.8.1.tar.bz2
strongswan-5.8.4.tar.bz2
subversion-1.12.2.tar.bz2
subversion-1.13.0.tar.bz2
sudo-1.8.23.tar.gz
sudo-1.8.31.tar.gz
sweeper-20.04.0.tar.xz
swig-3.0.12.tar.gz
swig-4.0.1.tar.gz
...


You could either look for duplicates and have some nasty algorithm that
finds the latest version and delete the oldest, or you could whitelist
all the remote sources specified in the aports and delete the rest.

> > - implement relgroups (check that all aports with same relgroup has
> >   same pkgver - this is not in use in aports yet)  
> 
> Ah, that'd for things like sqlite-tcl and sqlite to make sure they're
> the same version? That'd be very nice to have for CI.

exactly. also for kernel 3rd party modules, poppler and poppler-qt and
similar. There are various packages that needs this.
> 
> > - print reverse build deps. If you for example disable/enable a
> > library
> >   for an architecture, what other packages needs also to be
> >   disabled/enabled.  
> 
> I think it'd be nice if this also had the ability to run a script for
> each affected package. Maybe functionality that enables/disables the
> affected packages on all arches the affected package is disabled on
> would be good to have too (and very helpful when bootstrapping new
> arches), or when enabling core packages on a certain arch (e.g. when
> Rust was made available for more arches).

Good idea. I guess that is what you meant with git-bisec-ish, that you
can pass a script that is executed for each. That is something we may
want have to most of the operations. Alternatively you can `... | xargs
./myscript.sh ` so you get the full list and can loop over the args.

> 
> Thanks for working on this,
> 
> Rasmus
>