5 3

[alpine-devel] dealing with kernel module apkbuilds

Natanael Copa
Details
Message ID
<1239110538.15720.63.camel@nc>
Sender timestamp
1239110538
DKIM signature
missing
Download raw message
Hi,

I have an issue with building kernel module packages. Basicly, whenever
you rebuild the kernel you need to rebuild the kernel modules packages
(i.e dahdi-linux and xtables-addons, iscsi will come soon too)

The goals are:
* you update or rebuild the kernel, and all separate kernel module
packages gets rebuilt automatically.
* be simple. newbies should be able to read the apkbuild and understand
whats going on
* the kernel module packages should increase the pkgrel when rebuilt so
that you dont end up upgrading the kernel but not the kernel module
packages.

Currently, if kernel is updated I need to remember to manually go to all
kernel packages an bump the pkgrel. Ideally this should be done
automatically.

As I can see there are 3 ways to solve this:

1. build all kernel packages in the kernel APKBUILD.

The benefit is: if you rebuild kernel, you rebuild all kernel module
packages. The kernel module ackages inherits the kernel version so for
example dahdi-linux will have the version 2.6.28.9-r4 or similar.

The drawback is that you will rebuild the kernel if you update for
example dadhi-linux. If you want to add more non-standard kernel modules
you will need to modify the kernel APKBUILD.

2. we include the kernel version in the kernel module package pkgver.
This is doe by sourcing the kernel APKBUILD. For example:

# source kernel version
. ../linux-grsec/APKBUILD
pkgname=dahdi-linux-grsec
_dahdiver=2.1.0.4
pkgver=$pkgver.$pkgrel.$_dahdiver
pkgrel=1
...

If kernel is 2.6.28.9-r4, the dadhdi-linux-grsec would here end up as
2.6.28.9.4.2.1.0.4-r1

Alternatively we could just add the pkgrels so reduce one digit:
# source kernel version
. ../linux-grsec/APKBUILD
pkgname=dahdi-linux-grsec
_dahdiver=2.1.0.4
_rel=1
pkgver=$pkgver.$_dahdiver
pkgrel=$(( $_rel + $pkgrel ))
...

In this example we would end up with
dahdi-linux-grsec-2.6.28.9.2.1.0.4-r4

If we now upgrade the kernel to 2.6.28.9-r5 then will 'abuild up2date'
detect that the package is missing and will trigger a rebuild and things
just works. However, it is somewhat hackish, creates ugly version
numbers and is not so simple.

3. the third option is to do more or less what we do today, but have
some kind of sanity check scripts that keeps track on the kernel
versions and kerlen package versions and will make buildrepo fail if the
kernel module package is out of sync with the kernel package. This
requires that maintainer manually bump the pkgrel in the kernel module
APKBUILD. we could have some helper scripts to make that job easier.
Benefit here would be tat its less hackish (no need for the ugly
sourcing of other APKBUILDS) and we have nice version numbers,
dahdi-linux-grsec-2.1.0.4-r2 instead of
dahdi-linux-grsec-2.6.28.9.4.2.1.0.4-r2 or
dahdi-linux-grsec-2.6.28.9.2.1.0.4-r5.

But the drawback is that it will not auto rebuild if kernel is updated.

What do you prefer?

-nc



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Cameron Banta
Details
Message ID
<531b02ea0904070725u7318f7bfu40eb9d4f758b2ef9@mail.gmail.com>
In-Reply-To
<49DB58C3.4080505@iki.fi> (view parent)
Sender timestamp
1239114351
DKIM signature
missing
Download raw message
2009/4/7 Timo Teräs <timo.teras@iki.fi>

> Natanael Copa wrote:
> > 2. we include the kernel version in the kernel module package pkgver.
> > This is doe by sourcing the kernel APKBUILD. For example:
> >
> > # source kernel version
> > . ../linux-grsec/APKBUILD
> > pkgname=dahdi-linux-grsec
> > _dahdiver=2.1.0.4
> > pkgver=$pkgver.$pkgrel.$_dahdiver
> > pkgrel=1
> > ...
> >
> > If kernel is 2.6.28.9-r4, the dadhdi-linux-grsec would here end up as
> > 2.6.28.9.4.2.1.0.4-r1
> >
> > Alternatively we could just add the pkgrels so reduce one digit:
> > # source kernel version
> > . ../linux-grsec/APKBUILD
> > pkgname=dahdi-linux-grsec
> > _dahdiver=2.1.0.4
> > _rel=1
> > pkgver=$pkgver.$_dahdiver
> > pkgrel=$(( $_rel + $pkgrel ))
> > ...
> >
> > In this example we would end up with
> > dahdi-linux-grsec-2.6.28.9.2.1.0.4-r4
> >
> > If we now upgrade the kernel to 2.6.28.9-r5 then will 'abuild up2date'
> > detect that the package is missing and will trigger a rebuild and things
> > just works. However, it is somewhat hackish, creates ugly version
> > numbers and is not so simple.
> > But the drawback is that it will not auto rebuild if kernel is updated.
> >
> > What do you prefer?
>
> I'd prefer something like option 2.
>
> But I would not include the kernel version in the package version.
> Instead apk-tools should do versioned dependencies. Then during build
> you could just rebuild packages that have no longer valid dependencies
> after kernel package upgrade.
>
> Ideally you should not need to specify the kernel flavor in apkbuild.
> Instead the build system should know all available kernel variants,
> and rebuild the modules for each variant.
>
> - Timo
>
>
Speaking of ideally, I think it would be nice to have a parameter in
APKBUILD that specified this package was a kernel or a kernel module. Then
abuild would have a couple extra options for building kernels and kernel
modules.

For example, if you were building a module, you would need to specify to
abuild which kernel you wanted to build it for.  e.g:
abuild -K ../linux-grsec

If you were building a kernel, you could specify to build all the modules
with it and it would would find them dynamically and build them.  e.g:
abuild -m

Any scripts to build all the packages could simply ignore the kernel
modules, and tell each kernel it builds to build the modules with it.

Perhaps the extra work to abuild is not worth it, though.

-Cameron
Cameron Banta
Details
Message ID
<531b02ea0904071354r3e287d02n2713f4f84b25a084@mail.gmail.com>
In-Reply-To
<1239116482.15720.95.camel@nc> (view parent)
Sender timestamp
1239137679
DKIM signature
missing
Download raw message
2009/4/7 Natanael Copa <natanael.copa@gmail.com>

> ok, now we are talking. What I need is some help with ideas how to
> implement it and I think I got some ideas here.
>
> basicly, this would mean that you would need to tag a certain apkbuilds
> to be either a normal apkbuild (no tagging?), to be a kernel apkbuild,
> or to be a kernel module apkbuild.
>
> examples:
>
> cd dahdi-linux
> abuild          # builds packages with suffix for all kernel flavors
> abuild -K grsec # builds only dahdi-linux-grsec
>
> cd linux-grsec
> abuild -m       # builds kernel + it scans the aports tree for packages
>                # tagged as module apkbuild, increases pkgrel and
>                # rebuilds.
>

I would suggest that abuild (or abuild-mod) appends the kernel version and
pkgrel to the pkgrel on any kernel modules to create a dynamic version,
instead of updating the version every time you build it, whether you've
changed it or not.

Also, it'd be nice if abuild automatically set the dependancy to the kernel,
too.

>
> Maintainer would still need to commit those modified apkbuilds.
>
> I wonder if we could have a separate tool rather than include that in
> abuild script. maybe abuild-mod/abuild-kernel.
>

What would happen if someone ran abuild instead of abuild-kernel/abuild-mod?
And, I'm not familiar with the inner working of the abuild script, but I
would guess there would be a lot of similar code shared between abuild and
abuild-mod/abuild-kernel.

What about installing a kernel module? Does the user have to figure out the
right version himself?  e.g:
apk add my-kernel-mod-gresec-2.6.27.19

I think apk-tools might need to be modified to automatically search for the
modules for the current kernel. e.g:
apk add my-kernel-mod
would add the version of my-kernel-mod for the currently running kernel -
with the option of using the first example if needed. (Like on a system with
multiple kernels installed.)

-Cameron
Natanael Copa
Details
Message ID
<1239115391.15720.79.camel@nc>
In-Reply-To
<49DB58C3.4080505@iki.fi> (view parent)
Sender timestamp
1239115391
DKIM signature
missing
Download raw message
On Tue, 2009-04-07 at 16:44 +0300, Timo Teräs wrote:
> Natanael Copa wrote:
> > 2. we include the kernel version in the kernel module package pkgver.
> > This is doe by sourcing the kernel APKBUILD. For example:
> > 
> > # source kernel version
> > . ../linux-grsec/APKBUILD
> > pkgname=dahdi-linux-grsec
> > _dahdiver=2.1.0.4
> > pkgver=$pkgver.$pkgrel.$_dahdiver
> > pkgrel=1
> > ...
> > 
> > If kernel is 2.6.28.9-r4, the dadhdi-linux-grsec would here end up as
> > 2.6.28.9.4.2.1.0.4-r1
> > 
> > Alternatively we could just add the pkgrels so reduce one digit:
> > # source kernel version
> > . ../linux-grsec/APKBUILD
> > pkgname=dahdi-linux-grsec
> > _dahdiver=2.1.0.4
> > _rel=1
> > pkgver=$pkgver.$_dahdiver
> > pkgrel=$(( $_rel + $pkgrel ))
> > ...
> > 
> > In this example we would end up with
> > dahdi-linux-grsec-2.6.28.9.2.1.0.4-r4
> > 
> > If we now upgrade the kernel to 2.6.28.9-r5 then will 'abuild up2date'
> > detect that the package is missing and will trigger a rebuild and things
> > just works. However, it is somewhat hackish, creates ugly version
> > numbers and is not so simple.
> > But the drawback is that it will not auto rebuild if kernel is updated.
> > 
> > What do you prefer?
> 
> I'd prefer something like option 2.
> 
> But I would not include the kernel version in the package version.

I'm afraid you cannot have both without adding some speacial treatment
for kernel module packages in apk-tools.

> Instead apk-tools should do versioned dependencies. Then during build
> you could just rebuild packages that have no longer valid dependencies
> after kernel package upgrade.

...and bump the pkgrel? or make apk-tools try to reinstall packages that
no longer have valid dependencies?

I'm thinking, you have dahdi-linux-grsec-2.1.0.4-r0 installed which
depends on linux-grsec=2.6.28.9-r0, then kernel gets recompiled and is
2.6.28.9-r0. The build system detects that and rebuilds the package. but
unless the pkgrel is increased, how would apk-tools know that
dahdi-linux also should be upgraded?


> Ideally you should not need to specify the kernel flavor in apkbuild.
> Instead the build system should know all available kernel variants,
> and rebuild the modules for each variant.

would require some smarts, and some pleace to specify which apkbuilds
are kernels and some special variables telling where to find the kernel
config etc. There might alos be situations where you dont want to build
module packages for certain kernels.

I'm afraid we would end up with a complicated build system like gentoo
with lots of special vars etc. I think we have too much of that already.

need to think about that though...

> - Timo



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teräs
Details
Message ID
<49DB58C3.4080505@iki.fi>
In-Reply-To
<1239110538.15720.63.camel@nc> (view parent)
Sender timestamp
1239111875
DKIM signature
missing
Download raw message
Natanael Copa wrote:
> 2. we include the kernel version in the kernel module package pkgver.
> This is doe by sourcing the kernel APKBUILD. For example:
> 
> # source kernel version
> . ../linux-grsec/APKBUILD
> pkgname=dahdi-linux-grsec
> _dahdiver=2.1.0.4
> pkgver=$pkgver.$pkgrel.$_dahdiver
> pkgrel=1
> ...
> 
> If kernel is 2.6.28.9-r4, the dadhdi-linux-grsec would here end up as
> 2.6.28.9.4.2.1.0.4-r1
> 
> Alternatively we could just add the pkgrels so reduce one digit:
> # source kernel version
> . ../linux-grsec/APKBUILD
> pkgname=dahdi-linux-grsec
> _dahdiver=2.1.0.4
> _rel=1
> pkgver=$pkgver.$_dahdiver
> pkgrel=$(( $_rel + $pkgrel ))
> ...
> 
> In this example we would end up with
> dahdi-linux-grsec-2.6.28.9.2.1.0.4-r4
> 
> If we now upgrade the kernel to 2.6.28.9-r5 then will 'abuild up2date'
> detect that the package is missing and will trigger a rebuild and things
> just works. However, it is somewhat hackish, creates ugly version
> numbers and is not so simple.
> But the drawback is that it will not auto rebuild if kernel is updated.
> 
> What do you prefer?

I'd prefer something like option 2.

But I would not include the kernel version in the package version.
Instead apk-tools should do versioned dependencies. Then during build
you could just rebuild packages that have no longer valid dependencies
after kernel package upgrade.

Ideally you should not need to specify the kernel flavor in apkbuild.
Instead the build system should know all available kernel variants,
and rebuild the modules for each variant.

- Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Natanael Copa
Details
Message ID
<1239116482.15720.95.camel@nc>
In-Reply-To
<531b02ea0904070725u7318f7bfu40eb9d4f758b2ef9@mail.gmail.com> (view parent)
Sender timestamp
1239116482
DKIM signature
missing
Download raw message
On Tue, 2009-04-07 at 09:25 -0500, Cameron Banta wrote:
> 2009/4/7 Timo Teräs <timo.teras@iki.fi>
>         
>         Natanael Copa wrote:
...

>         Ideally you should not need to specify the kernel flavor in
>         apkbuild.
>         Instead the build system should know all available kernel
>         variants,
>         and rebuild the modules for each variant.
>         
>         - Timo
>         
>         
>         
> 
> Speaking of ideally, I think it would be nice to have a parameter in
> APKBUILD that specified this package was a kernel or a kernel module.
> Then abuild would have a couple extra options for building kernels and
> kernel modules. 
> 
> For example, if you were building a module, you would need to specify
> to abuild which kernel you wanted to build it for.  e.g:
> abuild -K ../linux-grsec
> 
> If you were building a kernel, you could specify to build all the
> modules with it and it would would find them dynamically and build
> them.  e.g:
> abuild -m
> 
> Any scripts to build all the packages could simply ignore the kernel
> modules, and tell each kernel it builds to build the modules with it.
> 
> Perhaps the extra work to abuild is not worth it, though.

ok, now we are talking. What I need is some help with ideas how to
implement it and I think I got some ideas here.

basicly, this would mean that you would need to tag a certain apkbuilds
to be either a normal apkbuild (no tagging?), to be a kernel apkbuild,
or to be a kernel module apkbuild.

examples:

cd dahdi-linux
abuild		# builds packages with suffix for all kernel flavors
abuild -K grsec	# builds only dahdi-linux-grsec

cd linux-grsec
abuild -m	# builds kernel + it scans the aports tree for packages
		# tagged as module apkbuild, increases pkgrel and
		# rebuilds.

Maintainer would still need to commit those modified apkbuilds.

I wonder if we could have a separate tool rather than include that in
abuild script. maybe abuild-mod/abuild-kernel.

What scares me most with all this, is that things get complicated. I'd
like to have the build system stupid simple if possible.


-nc

> 
> -Cameron



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---