Date: Mon, 3 Jan 2011 14:20:59 +0100
From: Natanael Copa <ncopa@alpinelinux.org>
To: "Matt Smith" <mcs@darkregion.net>
Cc: "Jeff Bilyk" <jbilyk@gmail.com>, alpine-devel@lists.alpinelinux.org
Subject: Re: [alpine-devel] Managing source packages
Message-ID: <20110103142059.5bff7030@ncopa-desktop.nor.wtbts.net>
In-Reply-To: <1293939379.793825469@192.168.2.227>
References: <20101229111233.00776ec7@ncopa-desktop.nor.wtbts.net>
	<60E8FCA5-7D2E-453A-834D-91129FC0E53F@gmail.com>
	<1293939379.793825469@192.168.2.227>
Precedence: list
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Sat, 1 Jan 2011 21:36:19 -0600 (CST)
"Matt Smith" <mcs@darkregion.net> wrote:

> On Saturday, January 1, 2011 11:52am, "Jeff Bilyk" <jbilyk@gmail.com>
> said:
> > Personally, I like the simplicity of not having to redo the build
> > infrastructure to accommodate source packages.  If abuild srcpkg
> > was working and a script existed to parse aports tree and alert
> > when a source package was no longer available, I think that the
> > current setup is ok.
> > 
> > As far as cleaning out the distfiles cache, a script that runs on
> > distfiles.alpinelinux.org and clears out any pre 1.10 source
> > packages would be good enough imho.  When 1.10 becomes deprecated,
> > the script could then be run to clear out pre2.0 files.
> > 
> > Thoughts?
> > 
> > Jeff
> > 
> > On Dec 29, 2010, at 5:12 AM, Natanael Copa <ncopa@alpinelinux.org>
> > wrote:
> ...snip...
> >> I still wonder though, how and when do we delete files from
> >> distfiles? I dont think we should delete source files that are
> >> used for v1.10 releases, so how do we do that? git checkout v1.10,
> >> make a list of all sourcefiles and mark them as "protected"? then
> >> finally delete all files that are not marked as "protected"?
> >>
> >> Or do we switch to debian style src packages or srpms like?
> >>
> >> I have also some ideas of a script that parses current aports and
> >> downloads headers only, and then reports all 404 or non-responding
> >> servers and notifies the maintainer.
> >>
> >> -nc
> 
> I want to point out the potential for filename clashes, especially
> with patch files, etc., all going into one directory like that.

I dont think that is a big deal. It is only the http:// and ftp://
files we keep in distfiles. The patches are kept in git.

>  It
> might be better to have the distfiles be saved to
> $distfiles_dir/$package/<package files>.

This might actually be a good idea anyways. It will avoid one directory
with tons of files (check how long time it takes to open
http://distfiles.gentoo.org/distfiles and you'll see whta I mean)

It will also make cleaning maintenence easier since it will be clear
what apkbuild that pulled in that particular source.

The downside is that aports cannot share source package so things like
linuk kernel source will be downloaded * N times (where N is number of
flavours). Same with for example poppler.

One thing I thought was that we could create snapshot dirs for
each git tag (v1.10.0, v1.10.1 -> v2.1.x) where we create
hardlinks to the distfiles dir. So then you can just delete all files
in distfiles that are not referenced to in current git master. The
hardlink in snapshot dir will make sure its kept as long as needed.

If you would like to rebuild v1.10.x you could just checkout that git
tag, point your DISTFILES_MIRROR to that specific dir and you had
everythin you needed for rebuilding it.

The only drawback I can see with hardlinks is if we want rsync the
distfiles to mirrors. That is ofcourse aslo  possible to solve
different ways and we could wait with it til there actually are more
distfiles mirrors.

> Aside from that, what we if did like Ubuntu and had Long-Term Support
> (LTS) releases, where bug-fixes are backported and
> distfiles+APKBUILDs are kept for the life of the release (1+ years,
> for instance).  Then keep the dist-files+APKBUILDs for the previous
> two releases; the next release, be it an LTS release or not, would
> mark the End-of-Life (EOL) for the oldest release still kept.

I have got requests for this before and I do think its a good idea.
The problem is that it would require someone to actually take the
responsability do the job. I don't have capacity for that.

That said, I try maintain older releases as good as possible and I will
backport fixes fro 1.10 (and even 1.9) on request.

> So at any time, we're supporting one LTS release and 2 update
> releases.
> 
> Something like that anyway.

I like it. Anyone willing to do the real job backporting bugfixes?
Actually, only keeping an eye on the changes and decide what should be
backported would be helpful. We could start with bugfixes only.

> At any rate, I think it'd be a good idea to keep older
> distfiles+APKBUILDs around for a little while, so in case someone
> should need to downgrade a package, they atleast have the option in
> one form or another.  Perhaps even keeping the APKs.

Yes. We could create source packages (with 'abuild srcpkg') for
archival purposes. But this won't be very useful when rebuilding from
scratch, which was Andrew's initial frustration.

> Unfortunately, that will eat up a good bit of disk space, I presume.
> And although source packages sound like a good idea, people using
> embedded systems or otherwise wouldn't necessarily have the
> alpine-sdk installed.

Yeah. tradeoffs...

> 
> Matt
> 
> 
> 
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
> 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---