Subject: Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

From: Jakub Jirutka <[email-masked]>
Date: Tue, 20 Jun 2017 18:06:28 +0200

Hi,

seding APKBUILDs is not a very good approach how to gather this information. That said, it can quite work in this case, for “arch”, because this is rarely (if ever) dynamic (e.g. adding archs based on some conditions) or variously formatted, but not for all variables.

We already have library for sort of programmatic reading of APKBUILDs: https://github.com/alpinelinux/lua-aports <https://github.com/alpinelinux/lua-aports>. It basically just evaluates every APKBUILD, prints and parsers the selected variables. This is used e.g. in pkgs.a.o.

I agree that such reports would be useful for maintainers for recently added platforms. However, there are much more reports that would be useful, so it’d be better to separate data and presentation. I mean, create a simple RESTful API for aports or just some JSON export, so anyone can make whatever report needs, without parsing APKBUILDs over and over again. This is yet another thing on my very long TODO list… Hm, maybe this week. It’s pity that we are so dislocated over the planet, some local hackathons may be very productive for such tasks. :)

Jakub

> On 20. Jun 2017, at 16:21, Tmp File <[email-masked]> wrote:
> 
> Good morning Alpiners.
>  
> I wrote a shell script that reads all APKBUILDS on a repository, parses the arch= setting, and output a csv table with detailed information about which package is supported by which architecture.
>  
> https://github.com/tmpfile/table
>  
> It outputs a complete table and also a table of only packages that aren't supported for all archs.
>  
> Here's the final result: https://github.com/tmpfile/table/blob/master/missing_support_table.csv
>  
> I think this can be of great help to arch maintainers so they can know with more ease which packages they must work on.
>  
> My goal sharing this work is that someone implements it as a cron job (for example) on alpinelinux.org/table.html or some other address.
>  
> In the testing I parsed only main/ but it should also work for community/ and /testing.
>  
> Let me know your opinion.
>  
> tmpfile
>  
> --- Unsubscribe: [email-masked] Help: [email-masked] ---

---

From: Jiri Horner <[email-masked]>
Date: Fri, 23 Jun 2017 10:41:56 +0200

>I agree that such reports would be useful for maintainers for recently added platforms. However, there are much more reports that would be useful, so it’d be better to separate data and presentation…

Maybe we could do a mini hackathon in Prague this summer. I have a list of 
packages I'd like to bring to alpine. As always the only problem is the free 
time.

Cheers,
Jiri

---
Unsubscribe:  [email-masked]
Help:         [email-masked]
---

---

From: Jakub Jirutka <[email-masked]>
Date: Fri, 23 Jun 2017 22:47:23 +0200

> Maybe we could do a mini hackathon in Prague this summer.

Yeah, that would be great! I’m very glad that I finally know someone else Alpinist from Czech Republic! :) 

Jakub

> On 23. Jun 2017, at 10:41, Jiri Horner <[email-masked]> wrote:
> 
> Maybe we could do a mini hackathon in Prague this summer. I have a list of packages I'd like to bring to alpine. As always the only problem is the free time.
> 
> Cheers,
> Jiri

---

---

From: Natanael Copa <[email-masked]>
Date: Fri, 30 Jun 2017 08:14:54 +0200

On Tue, 20 Jun 2017 18:06:28 +0200
Jakub Jirutka <[email-masked]> wrote:

> Hi,
> 
> seding APKBUILDs is not a very good approach how to gather this
> information. That said, it can quite work in this case, for *arch*,
> because this is rarely (if ever) dynamic (e.g. adding archs based on
> some conditions) or variously formatted, but not for all variables.
> 

I agree. The way to parse the APKBUILDs is to execute them.

> We already have library for sort of programmatic reading of
> APKBUILDs: https://github.com/alpinelinux/lua-aports
> <https://github.com/alpinelinux/lua-aports>. It basically just
> evaluates every APKBUILD, prints and parsers the selected variables.
> This is used e.g. in pkgs.a.o.

I started on a aports-cache project while on the plane. Its a small C
program (soon library) that parses all the APKBUILDs and writes a yaml
output file. It compares the timepstams on the APKBUILDs and update the
cache if needed.

I haven't decided if we want keep the yaml format or if we should use
toml or json for this.

I think the way forward is to have a common library that parses the
APKBUILDs and let the tools use the cache where all shell variables are
expanded.

The idea is to make a lua-aports-cache binding and let lua-aports use
this. We could also make bindings to python, ruby etc.

> I agree that such reports would be useful for maintainers for
> recently added platforms. However, there are much more reports that
> would be useful, so it*d be better to separate data and presentation.

Agree.

When thinking of it, we might need a separate json/yaml/toml for each
arch. There might be conditionals, for example different set of
subpackages for each arch.

> I mean, create a simple RESTful API for aports or just some JSON
> export, so anyone can make whatever report needs, without parsing
> APKBUILDs over and over again. This is yet another thing on my very
> long TODO list* Hm, maybe this week. It*s pity that we are so
> dislocated over the planet, some local hackathons may be very
> productive for such tasks. :)

-nc

---

---

From: Natanael Copa <[email-masked]>
Date: Fri, 30 Jun 2017 08:21:36 +0200

On Fri, 30 Jun 2017 08:14:54 +0200

> output file. It compares the timepstams on the APKBUILDs and update the

https://git.alpinelinux.org/cgit/user/ncopa/aports-cache/

-nc

---