Dear Alpinists,
at least Timo Teras[1] and William Pitcock[2] have proposed, that the
Alpine package building infrastructure could be used for unofficial
Alpine packages, when "the new build infrastructure [is] in place".
So postmarketOS[3] is a thin layer on top of Alpine, that provides
packages to make it work on mobile devices. Currently every user is
compiling these from source, but we would be very grateful if we could
use Alpine's infrastructure for building binary packages. That way we
could focus more on actual development and giving back to Alpine (e.g.
together with Adélie, we're currently upstreaming KDE), instead of
duplicating the effort.
For reference, here[4] are our current aports. Especially the device
folder makes no sense to be upstreamed. We will not build packages that
contain closed source blobs (our firmware aports will be refactored to
download these files at installation time[5]).
Thoughts?
Best regards,
Oliver Smith
[1]: Timo Teras: "This would allow several improvements: - would
simplify us supporting contributed ppa type aports trees our infra would
build" <https://lists.alpinelinux.org/alpine-devel/5427.html>
[2]: William Pitcock: "Once we have the new build infrastructure in
place, I am sure that we could arrange for derivatives to make use of
it. But I will need to talk with the infra team before committing us to
that." <https://github.com/postmarketOS/pmbootstrap/issues/663#issuecomment-333352865>
[3]: <https://postmarketOS.org>
[4]: <https://github.com/postmarketOS/pmbootstrap/tree/master/aports>
[5]: <https://github.com/postmarketOS/pmbootstrap/issues/797>
Hi,
On Tue, Oct 24, 2017 at 5:47 PM, Oliver Smith
<ollieparanoid@bitmessage.ch> wrote:
> Dear Alpinists,>>> at least Timo Teras[1] and William Pitcock[2] have proposed, that the> Alpine package building infrastructure could be used for unofficial> Alpine packages, when "the new build infrastructure [is] in place".
Yes, I have indeed proposed this.
But the new build infrastructure is not yet in place.
> So postmarketOS[3] is a thin layer on top of Alpine, that provides> packages to make it work on mobile devices. Currently every user is> compiling these from source, but we would be very grateful if we could> use Alpine's infrastructure for building binary packages. That way we> could focus more on actual development and giving back to Alpine (e.g.> together with Adélie, we're currently upstreaming KDE), instead of> duplicating the effort.
I proposed that we (Adelie and pmOS) might work together, but
unfortunately our requirements are incompatible (LTS vs. non-LTS
KDE).[1]
> For reference, here[4] are our current aports. Especially the device> folder makes no sense to be upstreamed. We will not build packages that> contain closed source blobs (our firmware aports will be refactored to> download these files at installation time[5]).>> Thoughts?
Well, I mean, I don't want to tell you what to do, but it seems
foolish to use a package manager which can cryptographically verify
package contents just to download a script which downloads the real
files.
At least, any firmware I would be loading into my phone, I would want
to be delivered to me in a signed package, not downloaded at install
time.
Not to mention that packages which download files in their
post-install scripts are a violation of Alpine policy.
William
[1]: Since we're using footnotes, I might ponder out loud why a phone
OS distribution is concentrating on KDE when they haven't even managed
to check off the "makes phone calls" tickbox yet.[2]
[2]: While I am not involved in sysadmin tasks around here, I am
pretty sure that any such collaboration on buildserver usage would be
dependent on the derivative proving it's viability first. See also
checking off the "makes phone calls" tickbox.
On Tue, 24 Oct 2017 22:47:00 +0000
Oliver Smith <ollieparanoid@bitmessage.ch> wrote:
> Dear Alpinists,> > > at least Timo Teras[1] and William Pitcock[2] have proposed, that the> Alpine package building infrastructure could be used for unofficial> Alpine packages, when "the new build infrastructure [is] in place".
replacing the building infra is a big project. I don't know how much
time it will take, and so far the progress have been slow.
> So postmarketOS[3] is a thin layer on top of Alpine, that provides> packages to make it work on mobile devices. Currently every user is> compiling these from source, but we would be very grateful if we could> use Alpine's infrastructure for building binary packages. That way we> could focus more on actual development and giving back to Alpine (e.g.> together with Adélie, we're currently upstreaming KDE), instead of> duplicating the effort.> > For reference, here[4] are our current aports. Especially the device> folder makes no sense to be upstreamed. We will not build packages that> contain closed source blobs (our firmware aports will be refactored to> download these files at installation time[5]).> > Thoughts?
I am glad to have you around I want help if we can. I have already have
use for the apk string version compare implementation that you wrote in
python.
What are your exact needs? Build servers on what architecture(s)?
Where do you want upload the built packages?
Do you mind if we follow this up after the v3.7 release?
Thanks!
> > > Best regards,> Oliver Smith> > > [1]: Timo Teras: "This would allow several improvements: - would> simplify us supporting contributed ppa type aports trees our infra would> build" <https://lists.alpinelinux.org/alpine-devel/5427.html>> [2]: William Pitcock: "Once we have the new build infrastructure in> place, I am sure that we could arrange for derivatives to make use of> it. But I will need to talk with the infra team before committing us to> that." <https://github.com/postmarketOS/pmbootstrap/issues/663#issuecomment-333352865>> [3]: <https://postmarketOS.org>> [4]: <https://github.com/postmarketOS/pmbootstrap/tree/master/aports>> [5]: <https://github.com/postmarketOS/pmbootstrap/issues/797>> >
> I am glad to have you around I want help if we can. I have already have> use for the apk string version compare implementation that you wrote in> python.
I am happy to read that and a bit curious what you have used for :)
> What are your exact needs? Build servers on what architecture(s)?
x86_64, armhf, aarch64
> Where do you want upload the built packages?
We have two mirrors set up (thanks to Martijn Braam!):
http://postmarketos1.brixit.nlhttp://postmarketos2.brixit.nl
Right now they host some outdated test packages we don't use, which were from an experiment with providing packages, where the files inside the apk packages are reproducible. Lots of code has been written for that to "challenge" package contents by rebuilding them and what not, but all in all it didn't seem feasible to continue this effort at this point and would only drive away from how Alpine handles things instead of being closer to upstream. (I can provide more details if someone is interested, there's a lot of theory already written down and it involves every package being rebuilt and challenged by a third party (e.g. Travis) before it gets accepted into the binary repo. Maybe at some far point in the future I can pick this up again and help with Alpine getting reproducible builds upstream.)
When I realized that this path was not working out, and with the quotes from Alpine developers I have put in the first mail of this thread I figured directly building the packages with Alpine's infra would be the next best thing. So users didn't have to trust another entity for building the packages, it would be all from Alpine.
> Do you mind if we follow this up after the v3.7 release?
Not at all!
In fact I realize that the timing for my post wasn't that great because of the upcoming 3.7 release, and it would have been nicer of me if I asked kaniini directly (sorry for that!).
From the answers you have given me, I think it a better short term solution for postmarketOS would be building the packages on our own. This is really good to know, we can move forward with this information.
Thank you very much,
Oliver
Natanael Copa:
> On Tue, 24 Oct 2017 22:47:00 +0000> Oliver Smith <ollieparanoid@bitmessage.ch> wrote:> >> Dear Alpinists,>>>>>> at least Timo Teras[1] and William Pitcock[2] have proposed, that the>> Alpine package building infrastructure could be used for unofficial>> Alpine packages, when "the new build infrastructure [is] in place".> > replacing the building infra is a big project. I don't know how much> time it will take, and so far the progress have been slow.> >> So postmarketOS[3] is a thin layer on top of Alpine, that provides>> packages to make it work on mobile devices. Currently every user is>> compiling these from source, but we would be very grateful if we could>> use Alpine's infrastructure for building binary packages. That way we>> could focus more on actual development and giving back to Alpine (e.g.>> together with Adélie, we're currently upstreaming KDE), instead of>> duplicating the effort.>>>> For reference, here[4] are our current aports. Especially the device>> folder makes no sense to be upstreamed. We will not build packages that>> contain closed source blobs (our firmware aports will be refactored to>> download these files at installation time[5]).>>>> Thoughts?> > I am glad to have you around I want help if we can. I have already have> use for the apk string version compare implementation that you wrote in> python.> > What are your exact needs? Build servers on what architecture(s)?> > Where do you want upload the built packages?> > Do you mind if we follow this up after the v3.7 release?> > Thanks!> >>>>>> Best regards,>> Oliver Smith>>>>>> [1]: Timo Teras: "This would allow several improvements: - would>> simplify us supporting contributed ppa type aports trees our infra would>> build" <https://lists.alpinelinux.org/alpine-devel/5427.html>>> [2]: William Pitcock: "Once we have the new build infrastructure in>> place, I am sure that we could arrange for derivatives to make use of>> it. But I will need to talk with the infra team before committing us to>> that." <https://github.com/postmarketOS/pmbootstrap/issues/663#issuecomment-333352865>>> [3]: <https://postmarketOS.org>>> [4]: <https://github.com/postmarketOS/pmbootstrap/tree/master/aports>>> [5]: <https://github.com/postmarketOS/pmbootstrap/issues/797>>>>>> >
> why a phone OS distribution is concentrating on KDE when they haven't even managed to check off the "makes phone calls" tickbox yet.
From what I have learned from various comments on our project is, that the people who say "but it does not make calls yet" would be the same ones who say "but it doesn't have a UI yet" and then go on with jokes about making phone calls with emacs and what not (which I do find pretty funny). postmarketOS is far from ready for being a daily driver and we're trying to be upfront with that basically everywhere (on the homepage, in every blog post, in github, in the wiki, ... - suggestions on how we can improve that are welcome).
There are people working on the plasma stack, because that is what interests them. I don't see any point in saying: "No, everyone must work on the telephony stack first." Personally I try support all efforts that bring the project forward, in one direction or another (tablets and using old phones as raspberry pis with sensors built in are fine use cases as well, that can also save the devices from being useless and thrown away). And I'm trying to see where we are blocked and push in that direction so we can move forward.
A binary repository is one of them, it would make the life of all contributors much easier (some contributors don't have access to fast CPUs and compiling kernels already takes hours for them), that's why I'd like to have a solution for that rather sooner than later.
So I've tried to politely ask with this post if Alpine would like to build postmarketOS packages, or not (which is answered now). And also only because making this possible was suggested by Alpine developers in the first place, I would not have asked otherwise.
> At least, any firmware I would be loading into my phone, I would want> to be delivered to me in a signed package, not downloaded at install> time.
The idea is to use abuild *on the phone* to download and verify the blobs and build the firmware package, then install that cleanly with apk. If you are interested in this topic and have the time to spare, I would be happy if you reviewed the idea in the GitHub issue (also regarding Alpine policy and how we could make it conform to that).
Best regards,
Oliver
William Pitcock:
> Hi,> > On Tue, Oct 24, 2017 at 5:47 PM, Oliver Smith> <ollieparanoid@bitmessage.ch> wrote:>> Dear Alpinists,>>>>>> at least Timo Teras[1] and William Pitcock[2] have proposed, that the>> Alpine package building infrastructure could be used for unofficial>> Alpine packages, when "the new build infrastructure [is] in place".> > Yes, I have indeed proposed this.> But the new build infrastructure is not yet in place.> >> So postmarketOS[3] is a thin layer on top of Alpine, that provides>> packages to make it work on mobile devices. Currently every user is>> compiling these from source, but we would be very grateful if we could>> use Alpine's infrastructure for building binary packages. That way we>> could focus more on actual development and giving back to Alpine (e.g.>> together with Adélie, we're currently upstreaming KDE), instead of>> duplicating the effort.> > I proposed that we (Adelie and pmOS) might work together, but> unfortunately our requirements are incompatible (LTS vs. non-LTS> KDE).[1]> >> For reference, here[4] are our current aports. Especially the device>> folder makes no sense to be upstreamed. We will not build packages that>> contain closed source blobs (our firmware aports will be refactored to>> download these files at installation time[5]).>>>> Thoughts?> > Well, I mean, I don't want to tell you what to do, but it seems> foolish to use a package manager which can cryptographically verify> package contents just to download a script which downloads the real> files.> At least, any firmware I would be loading into my phone, I would want> to be delivered to me in a signed package, not downloaded at install> time.> Not to mention that packages which download files in their> post-install scripts are a violation of Alpine policy.> > William> > [1]: Since we're using footnotes, I might ponder out loud why a phone> OS distribution is concentrating on KDE when they haven't even managed> to check off the "makes phone calls" tickbox yet.[2]> [2]: While I am not involved in sysadmin tasks around here, I am> pretty sure that any such collaboration on buildserver usage would be> dependent on the derivative proving it's viability first. See also> checking off the "makes phone calls" tickbox.>
Hi
> From what I have learned from various comments on our project is, that the people who say „but it does not make calls yet“ would be the same ones who say "but it doesn't have a UI yet" and then go on with jokes about making phone calls with emacs and what not…
Exactly my thoughts! I think that I’m not the only one who consider this comment quite unfair from William. Actually most of his response was more or less about questioning postmarketOS… But I don’t have much information about it, so maybe it was valid, I can’t judge. What changed my mind was not his response, but only Awilfox’s; she works for William, but unlike him she provided very comprehensive technical arguments.
Anyway, I’d like to add this: It may seem that “making a phone call” is the most important feature on mobile phones, but that’s not exactly right nowadays, is it? Many people, including me, would be very happy to have mobile device running Alpine with working network stack just to use it as PDA and don’t mind carrying good old so-called “stupid” phone just for calling (or maybe use some app for calls). We can’t say the same about missing GUI. Since CLI is unusable on todays so-called “smart” phones due to missing keyboard, mobile device without GUI would be totally useless for users. (Note: Of course I know that software should be built from bottom to up, that’s not the point here.)
Do you really need the latest KDE version, i.e. LTS version is not usable for postmarketOS? In that case I’d suggest to take abuilds from Adélie Linux, update them to the latest version / merge with yours, and maintain your own aports repository with these packages. Running aports repository itself is quite simple.
> The idea is to use abuild *on the phone* to download and verify the blobs and build the firmware package, then install that cleanly with apk.
Why? Alpine is a binary distribution, packages should be built on builders, not end-user machines. It looks quite odd for me to build packages right on the *phone* (and I used Gentoo for 5 years, still building pkgs on a phone doesn’t look like a good idea for me). Why you can’t download and verify these blobs on builders, package them as signed apk packages and use just apk on a phone to install them?
> So I’ve tried to politely ask with this post if Alpine would like to build postmarketOS packages, or not (which is answered now).
I have some x86_64 24-cores 64 GiB RAM machine which I can possible provide you for building packages. Unfortunately I don’t have any physical aarch64 and armhf machines. But we might try to run it with QEMU emulation; it will be slow, but better than nothing.
I’ll ask some people if I can get some aarch64 machine.
Jakub
> On 29. Oct 2017, at 13:27, Oliver Smith <ollieparanoid@bitmessage.ch> wrote:> >> why a phone OS distribution is concentrating on KDE when they haven't even managed to check off the "makes phone calls" tickbox yet.> > From what I have learned from various comments on our project is, that the people who say "but it does not make calls yet" would be the same ones who say "but it doesn't have a UI yet" and then go on with jokes about making phone calls with emacs and what not (which I do find pretty funny). postmarketOS is far from ready for being a daily driver and we're trying to be upfront with that basically everywhere (on the homepage, in every blog post, in github, in the wiki, ... - suggestions on how we can improve that are welcome).> > There are people working on the plasma stack, because that is what interests them. I don't see any point in saying: "No, everyone must work on the telephony stack first." Personally I try support all efforts that bring the project forward, in one direction or another (tablets and using old phones as raspberry pis with sensors built in are fine use cases as well, that can also save the devices from being useless and thrown away). And I'm trying to see where we are blocked and push in that direction so we can move forward.> > A binary repository is one of them, it would make the life of all contributors much easier (some contributors don't have access to fast CPUs and compiling kernels already takes hours for them), that's why I'd like to have a solution for that rather sooner than later.> > So I've tried to politely ask with this post if Alpine would like to build postmarketOS packages, or not (which is answered now). And also only because making this possible was suggested by Alpine developers in the first place, I would not have asked otherwise.> >> At least, any firmware I would be loading into my phone, I would want>> to be delivered to me in a signed package, not downloaded at install>> time.> > The idea is to use abuild *on the phone* to download and verify the blobs and build the firmware package, then install that cleanly with apk. If you are interested in this topic and have the time to spare, I would be happy if you reviewed the idea in the GitHub issue (also regarding Alpine policy and how we could make it conform to that).> > Best regards,> Oliver> > > William Pitcock:>> Hi,>> >> On Tue, Oct 24, 2017 at 5:47 PM, Oliver Smith>> <ollieparanoid@bitmessage.ch> wrote:>>> Dear Alpinists,>>> >>> >>> at least Timo Teras[1] and William Pitcock[2] have proposed, that the>>> Alpine package building infrastructure could be used for unofficial>>> Alpine packages, when "the new build infrastructure [is] in place".>> >> Yes, I have indeed proposed this.>> But the new build infrastructure is not yet in place.>> >>> So postmarketOS[3] is a thin layer on top of Alpine, that provides>>> packages to make it work on mobile devices. Currently every user is>>> compiling these from source, but we would be very grateful if we could>>> use Alpine's infrastructure for building binary packages. That way we>>> could focus more on actual development and giving back to Alpine (e.g.>>> together with Adélie, we're currently upstreaming KDE), instead of>>> duplicating the effort.>> >> I proposed that we (Adelie and pmOS) might work together, but>> unfortunately our requirements are incompatible (LTS vs. non-LTS>> KDE).[1]>> >>> For reference, here[4] are our current aports. Especially the device>>> folder makes no sense to be upstreamed. We will not build packages that>>> contain closed source blobs (our firmware aports will be refactored to>>> download these files at installation time[5]).>>> >>> Thoughts?>> >> Well, I mean, I don't want to tell you what to do, but it seems>> foolish to use a package manager which can cryptographically verify>> package contents just to download a script which downloads the real>> files.>> At least, any firmware I would be loading into my phone, I would want>> to be delivered to me in a signed package, not downloaded at install>> time.>> Not to mention that packages which download files in their>> post-install scripts are a violation of Alpine policy.>> >> William>> >> [1]: Since we're using footnotes, I might ponder out loud why a phone>> OS distribution is concentrating on KDE when they haven't even managed>> to check off the "makes phone calls" tickbox yet.[2]>> [2]: While I am not involved in sysadmin tasks around here, I am>> pretty sure that any such collaboration on buildserver usage would be>> dependent on the derivative proving it's viability first. See also>> checking off the "makes phone calls" tickbox.>> >
Hi,
On Sun, Oct 29, 2017 at 10:49 AM, Jakub Jirutka <jakub@jirutka.cz> wrote:
> Hi>>> From what I have learned from various comments on our project is, that the people who say „but it does not make calls yet“ would be the same ones who say "but it doesn't have a UI yet" and then go on with jokes about making phone calls with emacs and what not…>> Exactly my thoughts! I think that I’m not the only one who consider this comment quite unfair from William. Actually most of his response was more or less about questioning postmarketOS… But I don’t have much information about it, so maybe it was valid, I can’t judge. What changed my mind was not his response, but only Awilfox’s; she works for William, but unlike him she provided very comprehensive technical arguments.
Well, to be honest, I only responded to that thread because you
misrepresented how Adelie is structured, for reasons I still do not
understand.
Maybe you would like to explain to us why you decided to bring Adelie
(the company) into that discussion, because to me, it looked like you
were trying to sabotage our efforts.
Anyway.
My doubt is in providing technical resources and personnel to a
project that may very well not be around 6 months from now.
Doing hacks to quickly implement infrastructure for a project that has
an unknown future (due to not actually being usable for daily use on
any device) takes away opportunity to properly solve these problems.
Surely you would agree with this?
To be clear, I don't really have any position on pmOS either way.
I just doubt the longterm viability of the project given what I have
seen so far.
And since I doubt the longterm viability, I am not in favour of giving
them special treatment.
Once rootbld and the other infrastructure changes are done *and the
infra team evaluates their requirements* then there is maybe something
we can do.
Which is what I said about it to begin with -- at no time did I imply
we could do that immediately.
> Anyway, I’d like to add this: It may seem that “making a phone call” is the most important feature on mobile phones, but that’s not exactly right nowadays, is it? Many people, including me, would be very happy to have mobile device running Alpine with working network stack just to use it as PDA and don’t mind carrying good old so-called “stupid” phone just for calling (or maybe use some app for calls). We can’t say the same about missing GUI. Since CLI is unusable on todays so-called “smart” phones due to missing keyboard, mobile device without GUI would be totally useless for users. (Note: Of course I know that software should be built from bottom to up, that’s not the point here.)>> Do you really need the latest KDE version, i.e. LTS version is not usable for postmarketOS? In that case I’d suggest to take abuilds from Adélie Linux, update them to the latest version / merge with yours, and maintain your own aports repository with these packages. Running aports repository itself is quite simple.>>> The idea is to use abuild *on the phone* to download and verify the blobs and build the firmware package, then install that cleanly with apk.>> Why? Alpine is a binary distribution, packages should be built on builders, not end-user machines. It looks quite odd for me to build packages right on the *phone* (and I used Gentoo for 5 years, still building pkgs on a phone doesn’t look like a good idea for me). Why you can’t download and verify these blobs on builders, package them as signed apk packages and use just apk on a phone to install them?
They can't do that because the binary firmware blobs do not meet
Alpine's licensing guidelines.
Notably, Qualcomm explicitly forbid redistribution of their firmware.
Even if we were building packages for them, I do not believe that the
infrastructure team is interested in hosting repositories that contain
packages, or building packages which violate Alpine's licensing
guidelines.
>> So I’ve tried to politely ask with this post if Alpine would like to build postmarketOS packages, or not (which is answered now).>> I have some x86_64 24-cores 64 GiB RAM machine which I can possible provide you for building packages. Unfortunately I don’t have any physical aarch64 and armhf machines. But we might try to run it with QEMU emulation; it will be slow, but better than nothing.> I’ll ask some people if I can get some aarch64 machine.
The builds need to run on real hardware so testsuites run on real hardware.
They have contributed qemu-based builds in the past where the
testsuite and package itself failed to run on real hardware, but
worked on qemu.
Qemu is too permissive to run testsuites on, due to playing loose with
what causes segfaults, etc.
William
On Sun, 29 Oct 2017 21:36:44 -0500
William Pitcock <nenolod@dereferenced.org> wrote:
> My doubt is in providing technical resources and personnel to a> project that may very well not be around 6 months from now.> Doing hacks to quickly implement infrastructure for a project that has> an unknown future (due to not actually being usable for daily use on> any device) takes away opportunity to properly solve these problems.> Surely you would agree with this?> > To be clear, I don't really have any position on pmOS either way.> I just doubt the longterm viability of the project given what I have> seen so far.> And since I doubt the longterm viability, I am not in favour of giving> them special treatment.> Once rootbld and the other infrastructure changes are done *and the> infra team evaluates their requirements* then there is maybe something> we can do.> Which is what I said about it to begin with -- at no time did I imply> we could do that immediately.
I think if we try help them they might still be around 6 months from
now. But in any case, I think we should at least try help. Having two
different downstream derivatives are better than one.
But lets bring this up again after the v3.7 release.
-nc