Date: Tue, 19 Mar 2019 10:31:53 +0100
From: Natanael Copa <ncopa@alpinelinux.org>
To: Drew DeVault <sir@cmpwn.com>
Cc: alpine-devel@lists.alpinelinux.org
Subject: Re: [alpine-devel] Normalizing Python packages in aports
Message-ID: <20190319103153.3964ded5@ncopa-desktop.copa.dup.pw>
In-Reply-To: <20190314191241.GF1544@homura.localdomain>
References: <20190306212501.GD2800@cirno.localdomain>
	<20190314112413.29dd9816@ncopa-desktop.copa.dup.pw>
	<20190314191241.GF1544@homura.localdomain>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, 14 Mar 2019 15:12:41 -0400
Drew DeVault <sir@cmpwn.com> wrote:

> Thanks for the feedback!
> 
> On 2019-03-14 11:24 AM, Natanael Copa wrote:
> > > # Maintainer: Joe Bloe <joe@example.org>
> > > pkgname=py-alpine-name  
> > 
> > What do we for python2 only modules?  
> 
> Delete them.

Ok if I redirect affected people who needs support to you, and let you
respond on opened issues?

I agree that we should try move away from python2, but I don't we we
can remove it yet. And even if we decided to do so, it is a relatively
big project. Should we sacrifice llvm 6 upgrade? should we sacrifice
openjdk upgrade? Should we delay or skip alpine 3.10 release?
 
> > What do we when python4 arrives? should we start plan for that
> > already now? Does it make sense to use pkgname=py3-alpine-name?
> > -%<-
> > I think we should already now start think how to deal with python 4,
> > whenever that comes.  
> 
> Nack. I will eat my hat (and volunteer for another overhaul effort) if
> another backwards-incompatible Python shitstorm happens in the next 25
> years.

Well. Sorry if I am skeptic here, but it is not first time a
contributor show with strong opinions on how we should do things,
claiming that the consequences does not matter. Years later, when the
contributor moved on to something else, I am still the one who will do
the extra work maintaining it.

Python 4 is already mentioned:
https://www.python.org/dev/peps/pep-0563/#implementation

and i highly doubt that /usr/bin/python3 symlink will point
to /usr/bin/python4.0. So I kind of expect it to be "incompatible".

So I think we already now should think about what will happen when
python4 shows up (probably after python3.9), and I don't know if
you will still be around when that happens.

> > > makedepends="py-setuptools"  
> > 
> > I think this is somewhat tricky, because the real dependency here is
> > py3-setuptools which gets pulled in indirectly. I would prefer that we
> > use the direct dependency instead of rely that the indirect
> > dependencies will be done correctly.
> > -%<-
> > I would prefer that we use the direct dependency instead of rely that
> > the indirect dependencies will be done correctly.  
> 
> Using `makedepends="py3-setuptools"` presents a problem as well, for
> Python 2 deprecation/removal. If we rename py3-* to py-*, we will have
> to go through and fix all of these later. If we use meta-packages, this
> problem with fix itself when Python 2 is removed, and I'd err on the
> side of spot-fixing weirdness until then.

But it also require that build time dependency resolver is aware of it,
which it is not. I can tell you that it is far easier to keep the build
time dependency resolver stupid and simple and manually set direct
dependencies in APKBUILDs than fix the build time dependency resolver.

Also errors caused by direct dependencies (eg missed rename of py3- to
py-) are easy to spot and trivial to fix, while errors caused by broken
indirect dependencies are often tricky to understand why it happens. In
practice what happens is people tell me "builder is broken, i don't know
why, please fix asap!"

So I would prefer to have direct dependencies and go over and replace
py3-* with py-* in the future.

> > > prepare() {
> > > 	cp -r "$builddir" "$builddir"-py3  
> > 
> > setuptools does not support out-of-tree builds?  
> 
> Not without hacks, I'm afraid. I'm not too torn up about it since we'll
> want to be leaving Python 2 behind anyway.

ok.
 
> > > _py() {
> > >	python="$1"  
> > 
> > we need 'local' here.  
> 
> local is also a bash extension. It's undefined behavior in POSIX shell.

At a minimum it should be prefixed with _ (eg _python) to make sure
that it does not conflict with any internal variable in abuild itself.

We still have accepted use of 'local' in both abuild and the APKBUILDs.
There are a few extension we have accepted:

local
${var//search/replace}
${var:0:1}

If we should accept those or go for strict POSIX compliance is another
discussion which I don't think we should mix with the python packaging
policy.

> > There is an optional way to do it, without the _py2 and _py3 functions:
> > -%<-
> > 
> > But I suppose using _py2 and _py3 functions makes things more readable.  
> 
> Is doing it without the _py2 and _py3 functions desirable if it makes
> things less readable?

I think not. I
 
> > > Another issue we'll have to face soon enough is Python 2 end-of-life. Do
> > > we want to address this at the same time? I think the best balance of
> > > conservatism and speedy removal of python 2 would be:
> > > 
> > > - Stop adding new python 2 packages to aports
> > > - As we normalize existing packages, drop python 2 if it's easy, or make
> > >   a note if not
> > > - Make case-by-case judgement calls on the more difficult packages,
> > >   based on the upstream's progress/amicability towards a python 3 port.
> > >   Upstreams which are unwilling to port to python 3 should probably just
> > >   be dropped.  
> > 
> > I think those are good suggestions.
> > 
> > I think removing python2 completely may be tricky and I support doing
> > it in small steps. We should work in that direction.  
> 
> I've written this up into the wiki page, along with incorporating the
> rest of your feedback.
> 
> https://wiki.alpinelinux.org/w/index.php?title=Python_package_policies&type=revision&diff=15799&oldid=15755
> 
> Another question: how should we structure this work? I would appreciate
> volunteers (I think there already were some on IRC) to participate and
> help with code reviews. We probably ought to work out-of-tree and
> periodically request-pull our work into aports.

Once we decided on how we want it, we should probably start with the
current PRs so new additions is done correctly and contributors are
trained to the new standard. Then I think we should take things in
chunks. For example we could do py-django-* in one go, py-flask-* on
another.

Earler, before python3 was supported, all the py-* packages was
python2. We should probably also look over all py-* packages and make
sure that all python2 packages are renamed to py2-*.

Thanks!

-nc

> 
> 
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
> 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---