~alpine/devel

5 2

[alpine-devel] apk-tools - In need of a minimal spec

Orion
Details
Message ID
<20150202003218.6209a4a2@twinpeaks.my.domain>
Sender timestamp
1422865938
DKIM signature
missing
Download raw message
I have began reading through the apk-tools source code with the goal of
teaching myself more about how package managers work by porting
it into go-lang. I would appreciate help compiling some minimally
sufficient specification for this.

I ask for your help because I have had little luck finding any kind of
documentation, design thoughts, comments, or self describing
functions/variables. Let me know what the easiest means of discussion
would be.

# Stuff I Found Confusing
 * blob.c
	* memspn()
	* memcspn()
	* apk_blob_spn()
	* apk_blob_cspn()
	* digitdecode & dx()
		- is data taken, encoded with your own custom codec and
		  then put into a blow?
 * archive.c & gunzip.c
	* is the tar and gzip code used custom or pulled from existing
	  implementations?

Thank you for your time.

-- 
keybase.io/systmkor


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: apk-tools - In need of a minimal spec

Timo Teras
Details
Message ID
<20150202111718.66140f1d@vostro>
In-Reply-To
<20150202003218.6209a4a2@twinpeaks.my.domain> (view parent)
Sender timestamp
1422868638
DKIM signature
missing
Download raw message
On Mon, 2 Feb 2015 00:32:18 -0800
Orion <systmkor@gmail.com> wrote:

> I have began reading through the apk-tools source code with the goal
> of teaching myself more about how package managers work by porting
> it into go-lang. I would appreciate help compiling some minimally
> sufficient specification for this.
> 
> I ask for your help because I have had little luck finding any kind of
> documentation, design thoughts, comments, or self describing
> functions/variables. Let me know what the easiest means of discussion
> would be.

Yes, we should add some overview documentation. And specs of how the
package format is handled.

> # Stuff I Found Confusing
>  * blob.c
> 	* memspn()
> 	* memcspn()
> 	* apk_blob_spn()
> 	* apk_blob_cspn()

It's basically similar to strspn() but with two differences:
 - it works with blobs, so the source string is not zero-terminated
 - the list of characters to accept or reject is a bitmap instead of
   zero-terminated string (see the header for apk_spn_match_def typedef)

memspn() would probably be the libc name, but there is no such
function. Use of that name is probably misleading.

I wrote this earlier when I had tendency to micro-optimize things. If
I'd write it now, I'd leave the memspn() stuff out, and keep only the
C-versions.

> 	* digitdecode & dx()
> 		- is data taken, encoded with your own custom codec

This is just a lookup-table based ASCII hexnumber to number conversion.
Basically atoi() and strtol() equivalent, but it's hand written again,
so we can work with "blobs" (length limited, not zero-terminated).

> and then put into a blow?
>  * archive.c & gunzip.c
> 	* is the tar and gzip code used custom or pulled from existing
> 	  implementations?

gunzip.c ends up using libz. It's just glue for the internal
abstractionlayer.

archive.c contains full tar decoder and encoder. It's mostly custom
code. And currently it is required due to how the .apk file is
constructed.

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: apk-tools - In need of a minimal spec

Orion
Details
Message ID
<20150202191547.4a07f782@twinpeaks.my.domain>
In-Reply-To
<20150202111718.66140f1d@vostro> (view parent)
Sender timestamp
1422933347
DKIM signature
missing
Download raw message
Thanks. That starts to clear up a few things.

I've created a wiki page for the apk-tools specification. 
http://wiki.alpinelinux.org/wiki/Apk_spec

I think this place would make sense for us to start. Let me know if you
think there is a better way of going about this. Thank you for your
time.

-- 
keybase.io/systmkor


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: apk-tools - In need of a minimal spec

Orion
Details
Message ID
<20150204014231.091729cf@twinpeaks.my.domain>
In-Reply-To
<20150202111718.66140f1d@vostro> (view parent)
Sender timestamp
1423042951
DKIM signature
missing
Download raw message
I've began to try define the syntax/keywords used inside
the /lib/apk/db/installed . Let me know if I'm on the right track.
http://wiki.alpinelinux.org/wiki/Apk_spec

-- 
keybase.io/systmkor


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: apk-tools - In need of a minimal spec

Timo Teras
Details
Message ID
<20150204120112.7a159181@vostro>
In-Reply-To
<20150204014231.091729cf@twinpeaks.my.domain> (view parent)
Sender timestamp
1423044072
DKIM signature
missing
Download raw message
On Wed, 4 Feb 2015 01:42:31 -0800
Orion <systmkor@gmail.com> wrote:

> I've began to try define the syntax/keywords used inside
> the /lib/apk/db/installed . Let me know if I'm on the right track.
> http://wiki.alpinelinux.org/wiki/Apk_spec

Yes, mostly. I could probably fix / update some details too.

It might be an idea to explain that the same file format is used for
both: installed database, and the package index file. The index file
contains a subset, and installed database has some additional fields.

Please remove the 'blob' term there. Blobs are internal implementation
detail. apk_blob is basically a pointer + length structure. On the
index/database file it is just a specifically formatted text string.

Also, there's a convention that capital letter fields are mandatory.
Apk refuses to install the package if it recognize it. Lower case
letters are optional support, and apk will give warning but accept to
install it.

Maybe it'd be good as a table with columns:
 - Field letter
 - Description
 - Field data format
 - Allowed in installed-db / package-index

For APKBUILD we already have some other wiki pages, and it's not read
by apk-tools so that can be left out.

Over all, for the fileformats. It'd need also a section for index file
and it's structure, the installed db file structure, and the package
file structure.

I do have some notes on these. But it's not in any formatted way. I
should take time to write it to the wiki page.

Thanks for the effort so far :)

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---

[alpine-devel] Re: apk-tools - In need of a minimal spec

Orion
Details
Message ID
<20150206185816.36037002@twinpeaks.my.domain>
In-Reply-To
<20150204120112.7a159181@vostro> (view parent)
Sender timestamp
1423277896
DKIM signature
missing
Download raw message
On Wed, 4 Feb 2015 12:01:12 +0200
Timo Teras <timo.teras@iki.fi> wrote:

> Please remove the 'blob' term there.

'blob' terms has been removed.


> Also, there's a convention that capital letter fields are mandatory.
> Apk refuses to install the package if it recognize it. Lower case
> letters are optional support, and apk will give warning but accept to
> install it.

One line noting this but leaving out the warning until I know more
about what causes it etc.


> Maybe it'd be good as a table with columns:
>  - Field letter
>  - Description
>  - Field data format
>  - Allowed in installed-db / package-index

Table with these fields have been created. That said I haven't filled
the entire table out yet. I think the table should probably be a quick
reference and then if needed we have sections for each one going into
depth about that field.


> For APKBUILD we already have some other wiki pages, and it's not read
> by apk-tools so that can be left out.

Section removed.


> Over all, for the fileformats. It'd need also a section for index file
> and it's structure, the installed db file structure, and the package
> file structure.
> 
> I do have some notes on these. But it's not in any formatted way. I
> should take time to write it to the wiki page.

Maybe add some of your notes to the wiki page this weekend? :3 :P


> Thanks for the effort so far :)

Your welcome. Thanks for responding and not leaving me stranded to
reverse engineer everything. 

-- 
keybase.io/systmkor


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---