Date: Tue, 19 Mar 2019 19:57:50 -0400
From: Drew DeVault <sir@cmpwn.com>
To: u-3qpt@aetey.se
Cc: Natanael Copa <ncopa@alpinelinux.org>,
	alpine-devel@lists.alpinelinux.org,
	Fabian Affolter <fabian@affolter-engineering.ch>
Subject: Re: [alpine-devel] Normalizing Python packages in aports
Message-ID: <20190319235750.GA20521@homura.localdomain>
References: <20190306212501.GD2800@cirno.localdomain>
 <20190314112413.29dd9816@ncopa-desktop.copa.dup.pw>
 <20190314191241.GF1544@homura.localdomain>
 <20190319103153.3964ded5@ncopa-desktop.copa.dup.pw>
 <20190319113230.GM23721@example.net>
 <20190319125738.73f31222@ncopa-desktop.copa.dup.pw>
 <20190319130824.GN23721@example.net>
 <20190319135227.GO23721@example.net>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190319135227.GO23721@example.net>


On 2019-03-19 10:31 AM, Natanael Copa wrote:
> > > What do we for python2 only modules?  
> > 
> > Delete them.
> 
> Ok if I redirect affected people who needs support to you, and let you
> respond on opened issues?
> 
> I agree that we should try move away from python2, but I don't we we
> can remove it yet. And even if we decided to do so, it is a relatively
> big project. Should we sacrifice llvm 6 upgrade? should we sacrifice
> openjdk upgrade? Should we delay or skip alpine 3.10 release?

Sorry, in retrospect this was a snarky and unrealistic answer. Instead
I'd like to make case-by-case judgements on python2 packages, and I
don't think that establishing a standard is necessary for packages which
will be short lived by nature - since I think we very much ought to
ditch them when Python 2 is EoL.

> Well. Sorry if I am skeptic here, but it is not first time a
> contributor show with strong opinions on how we should do things,
> claiming that the consequences does not matter. Years later, when the
> contributor moved on to something else, I am still the one who will do
> the extra work maintaining it.
> 
> Python 4 is already mentioned:
> https://www.python.org/dev/peps/pep-0563/#implementation
> 
> and i highly doubt that /usr/bin/python3 symlink will point
> to /usr/bin/python4.0. So I kind of expect it to be "incompatible".
> 
> So I think we already now should think about what will happen when
> python4 shows up (probably after python3.9), and I don't know if
> you will still be around when that happens.

The Python devs have already committed to making Python 4, if it
happens at all, as compatible with Python 3 as possible, with only
conservative changes announced well in advance. They aren't going to
change how strings work again - those kinds of sweeping changes that
break large swaths of the ecosystem are extremely unlikely. I would, in
fact, be surprised if symlinking /usr/bin/python3 to /usr/bin/python4
wouldn't work fine for all but a few programs. I'm certain the
disruption to Alpine will be smaller than that of, for example, the
OpenSSL vs. LibreSSL issues.

> > Using `makedepends="py3-setuptools"` presents a problem as well, for
> > Python 2 deprecation/removal. If we rename py3-* to py-*, we will have
> > to go through and fix all of these later. If we use meta-packages, this
> > problem with fix itself when Python 2 is removed, and I'd err on the
> > side of spot-fixing weirdness until then.
> 
> But it also require that build time dependency resolver is aware of it,
> which it is not. I can tell you that it is far easier to keep the build
> time dependency resolver stupid and simple and manually set direct
> dependencies in APKBUILDs than fix the build time dependency resolver.
> 
> Also errors caused by direct dependencies (eg missed rename of py3- to
> py-) are easy to spot and trivial to fix, while errors caused by broken
> indirect dependencies are often tricky to understand why it happens. In
> practice what happens is people tell me "builder is broken, i don't know
> why, please fix asap!"
> 
> So I would prefer to have direct dependencies and go over and replace
> py3-* with py-* in the future.

Alright, +1. Let's go with moving to py3-* for all Python packages.

> > local is also a bash extension. It's undefined behavior in POSIX
> > shell.
> 
> At a minimum it should be prefixed with _ (eg _python) to make sure
> that it does not conflict with any internal variable in abuild itself.
> 
> We still have accepted use of 'local' in both abuild and the APKBUILDs.
> There are a few extension we have accepted:
> 
> local
> ${var//search/replace}
> ${var:0:1}
> 
> If we should accept those or go for strict POSIX compliance is another
> discussion which I don't think we should mix with the python packaging
> policy.

I don't like this mixed-leniency policy, it seems kind of arbitrary.
However, I agree that this isn't the venue for this discussion. Let's
call it _python - without the local keyword - for the time being. I do
have some thoughts around POSIX compliance that I'll mention when the
time comes.

> > I've written this up into the wiki page, along with incorporating the
> > rest of your feedback.
> > 
> > https://wiki.alpinelinux.org/w/index.php?title=Python_package_policies&type=revision&diff=15799&oldid=15755
> > 
> > Another question: how should we structure this work? I would appreciate
> > volunteers (I think there already were some on IRC) to participate and
> > help with code reviews. We probably ought to work out-of-tree and
> > periodically request-pull our work into aports.
> 
> Once we decided on how we want it, we should probably start with the
> current PRs so new additions is done correctly and contributors are
> trained to the new standard. Then I think we should take things in
> chunks. For example we could do py-django-* in one go, py-flask-* on
> another.

Starting with the existing PRs is a good idea, as well as requiring new
patches to follow these conventions. As far as planning the broader
normalization project goes, my plan is to collect a list of all Python
packages, assign them a status (something like [unreviewed, reviewed,
normalized]), and do some light scripting to help us keep up with this
work. This should make it easy for someone to pick up work, and give us
a place to record any issues that come up for addressing later (like
Python 2-only upstreams). From there, I think doing it one chunk at a
time seems wise indeed.

Updated the wiki page to incorporate feedback since my last reply.


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---