4 3

Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

Jakub Jirutka
Details
Message ID
<315008DD-D5EB-4369-8A80-F46F10DC528A@jirutka.cz>
Sender timestamp
1497974788
DKIM signature
missing
Download raw message
Hi,

seding APKBUILDs is not a very good approach how to gather this information. That said, it can quite work in this case, for “arch”, because this is rarely (if ever) dynamic (e.g. adding archs based on some conditions) or variously formatted, but not for all variables.

We already have library for sort of programmatic reading of APKBUILDs: https://github.com/alpinelinux/lua-aports <https://github.com/alpinelinux/lua-aports>. It basically just evaluates every APKBUILD, prints and parsers the selected variables. This is used e.g. in pkgs.a.o.

I agree that such reports would be useful for maintainers for recently added platforms. However, there are much more reports that would be useful, so it’d be better to separate data and presentation. I mean, create a simple RESTful API for aports or just some JSON export, so anyone can make whatever report needs, without parsing APKBUILDs over and over again. This is yet another thing on my very long TODO list… Hm, maybe this week. It’s pity that we are so dislocated over the planet, some local hackathons may be very productive for such tasks. :)

Jakub

> On 20. Jun 2017, at 16:21, Tmp File <tmpfile@mail.com> wrote:
> 
> Good morning Alpiners.
>  
> I wrote a shell script that reads all APKBUILDS on a repository, parses the arch= setting, and output a csv table with detailed information about which package is supported by which architecture.
>  
> https://github.com/tmpfile/table
>  
> It outputs a complete table and also a table of only packages that aren't supported for all archs.
>  
> Here's the final result: https://github.com/tmpfile/table/blob/master/missing_support_table.csv
>  
> I think this can be of great help to arch maintainers so they can know with more ease which packages they must work on.
>  
> My goal sharing this work is that someone implements it as a cron job (for example) on alpinelinux.org/table.html or some other address.
>  
> In the testing I parsed only main/ but it should also work for community/ and /testing.
>  
> Let me know your opinion.
>  
> tmpfile
>  
> --- Unsubscribe: alpine-devel+unsubscribe@lists.alpinelinux.org Help: alpine-devel+help@lists.alpinelinux.org ---

Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

Jiri Horner
Details
Message ID
<20170623084156.wt74h3nv2yvi26d2@eucalyptus>
In-Reply-To
<315008DD-D5EB-4369-8A80-F46F10DC528A@jirutka.cz> (view parent)
Sender timestamp
1498207316
DKIM signature
missing
Download raw message
On Tue, Jun 20, 2017 at 06:06:28PM +0200, Jakub Jirutka wrote:
>
>I agree that such reports would be useful for maintainers for recently added platforms. However, there are much more reports that would be useful, so it’d be better to separate data and presentation. I mean, create a simple RESTful API for aports or just some JSON export, so anyone can make whatever report needs, without parsing APKBUILDs over and over again. This is yet another thing on my very long TODO list… Hm, maybe this week. It’s pity that we are so dislocated over the planet, some local hackathons may be very productive for such tasks. :)
>

Maybe we could do a mini hackathon in Prague this summer. I have a list of 
packages I'd like to bring to alpine. As always the only problem is the free 
time.

Cheers,
Jiri


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

Jakub Jirutka
Details
Message ID
<9503F776-768E-44B4-B33B-C6CE33C1217E@jirutka.cz>
In-Reply-To
<20170623084156.wt74h3nv2yvi26d2@eucalyptus> (view parent)
Sender timestamp
1498250843
DKIM signature
missing
Download raw message
> Maybe we could do a mini hackathon in Prague this summer.

Yeah, that would be great! I’m very glad that I finally know someone else Alpinist from Czech Republic! :) 

Jakub

> On 23. Jun 2017, at 10:41, Jiri Horner <laeqten@gmail.com> wrote:
> 
> On Tue, Jun 20, 2017 at 06:06:28PM +0200, Jakub Jirutka wrote:
>> 
>> I agree that such reports would be useful for maintainers for recently added platforms. However, there are much more reports that would be useful, so it’d be better to separate data and presentation. I mean, create a simple RESTful API for aports or just some JSON export, so anyone can make whatever report needs, without parsing APKBUILDs over and over again. This is yet another thing on my very long TODO list… Hm, maybe this week. It’s pity that we are so dislocated over the planet, some local hackathons may be very productive for such tasks. :)
>> 
> 
> Maybe we could do a mini hackathon in Prague this summer. I have a list of packages I'd like to bring to alpine. As always the only problem is the free time.
> 
> Cheers,
> Jiri


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

Natanael Copa
Details
Message ID
<20170630081454.019fa6e4@ncopa-desktop.copa.dup.pw>
In-Reply-To
<315008DD-D5EB-4369-8A80-F46F10DC528A@jirutka.cz> (view parent)
Sender timestamp
1498803294
DKIM signature
missing
Download raw message
On Tue, 20 Jun 2017 18:06:28 +0200
Jakub Jirutka <jakub@jirutka.cz> wrote:

> Hi,
> 
> seding APKBUILDs is not a very good approach how to gather this
> information. That said, it can quite work in this case, for *arch*,
> because this is rarely (if ever) dynamic (e.g. adding archs based on
> some conditions) or variously formatted, but not for all variables.
> 

I agree. The way to parse the APKBUILDs is to execute them.

> We already have library for sort of programmatic reading of
> APKBUILDs: https://github.com/alpinelinux/lua-aports
> <https://github.com/alpinelinux/lua-aports>. It basically just
> evaluates every APKBUILD, prints and parsers the selected variables.
> This is used e.g. in pkgs.a.o.

I started on a aports-cache project while on the plane. Its a small C
program (soon library) that parses all the APKBUILDs and writes a yaml
output file. It compares the timepstams on the APKBUILDs and update the
cache if needed.

I haven't decided if we want keep the yaml format or if we should use
toml or json for this.

I think the way forward is to have a common library that parses the
APKBUILDs and let the tools use the cache where all shell variables are
expanded.

The idea is to make a lua-aports-cache binding and let lua-aports use
this. We could also make bindings to python, ruby etc.

> I agree that such reports would be useful for maintainers for
> recently added platforms. However, there are much more reports that
> would be useful, so it*d be better to separate data and presentation.

Agree.

When thinking of it, we might need a separate json/yaml/toml for each
arch. There might be conditionals, for example different set of
subpackages for each arch.

> I mean, create a simple RESTful API for aports or just some JSON
> export, so anyone can make whatever report needs, without parsing
> APKBUILDs over and over again. This is yet another thing on my very
> long TODO list* Hm, maybe this week. It*s pity that we are so
> dislocated over the planet, some local hackathons may be very
> productive for such tasks. :)


-nc

> Jakub
> 
> > On 20. Jun 2017, at 16:21, Tmp File <tmpfile@mail.com> wrote:
> > 
> > Good morning Alpiners.
> >  
> > I wrote a shell script that reads all APKBUILDS on a repository, parses the arch= setting, and output a csv table with detailed information about which package is supported by which architecture.
> >  
> > https://github.com/tmpfile/table
> >  
> > It outputs a complete table and also a table of only packages that aren't supported for all archs.
> >  
> > Here's the final result: https://github.com/tmpfile/table/blob/master/missing_support_table.csv
> >  
> > I think this can be of great help to arch maintainers so they can know with more ease which packages they must work on.
> >  
> > My goal sharing this work is that someone implements it as a cron job (for example) on alpinelinux.org/table.html or some other address.
> >  
> > In the testing I parsed only main/ but it should also work for community/ and /testing.
> >  
> > Let me know your opinion.
> >  
> > tmpfile
> >  
> > --- Unsubscribe: alpine-devel+unsubscribe@lists.alpinelinux.org Help: alpine-devel+help@lists.alpinelinux.org ---  
> 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

Re: [alpine-devel] Script to parse APKBUILDS and output table of arch support

Natanael Copa
Details
Message ID
<20170630082136.4399de3a@ncopa-desktop.copa.dup.pw>
In-Reply-To
<20170630081454.019fa6e4@ncopa-desktop.copa.dup.pw> (view parent)
Sender timestamp
1498803696
DKIM signature
missing
Download raw message
On Fri, 30 Jun 2017 08:14:54 +0200
Natanael Copa <ncopa@alpinelinux.org> wrote:

 
> > We already have library for sort of programmatic reading of
> > APKBUILDs: https://github.com/alpinelinux/lua-aports
> > <https://github.com/alpinelinux/lua-aports>. It basically just
> > evaluates every APKBUILD, prints and parsers the selected variables.
> > This is used e.g. in pkgs.a.o.  
> 
> I started on a aports-cache project while on the plane. Its a small C
> program (soon library) that parses all the APKBUILDs and writes a yaml
> output file. It compares the timepstams on the APKBUILDs and update the
> cache if needed.


https://git.alpinelinux.org/cgit/user/ncopa/aports-cache/

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---