~alpine/aports

3 2

[alpine-aports] [RFC] Splitting linux-firmware

Isaac Dunham
Details
Message ID
<20160403052116.GA22996@newbook>
Sender timestamp
1459660878
DKIM signature
missing
Download raw message
Hello,
I recently checked the size of installed packages on my system, and found
that linux-firmware was the largest (~120 megabytes).
I need the Realtek firmware, and perhaps could use the i915 firmware.
So I figured it would be a significant cutdown in installed size,
as well as a major reduction in downloads, if it were possible to just
select firmware from a couple of manufacturers.

While I was in the process of doing this, I noticed a few smaller issues:
-the non-firmware files that the APKBUILD tries to delete are actually
located in /lib/firmware, not /usr/lib/firmware
-the source for carl9170 firmware is included in the final package

Here's a tentative patch that fixes those issues and tries to split
things up.

Does this look like a good idea?
Should any further splitting be done? 
For example:
-splitting iwlwifi firmware off because it's the bulk of the intel
firmware, but not the bulk of the users;
-splitting qlogic, 3com, and chelsio firmware from -misc


Thanks,
Isaac Dunham
Isaac Dunham
Details
Message ID
<20160403205501.GA13799@newbook>
In-Reply-To
<20160403162518.213e9151@vostro> (view parent)
Sender timestamp
1459716902
DKIM signature
missing
Download raw message
On Sun, Apr 03, 2016 at 04:25:18PM +0300, Timo Teras wrote:
> On Sat, 2 Apr 2016 22:21:18 -0700
> Isaac Dunham <ibid.ag@gmail.com> wrote:
> 
> > I recently checked the size of installed packages on my system, and
> > found that linux-firmware was the largest (~120 megabytes).
> > I need the Realtek firmware, and perhaps could use the i915 firmware.
> > So I figured it would be a significant cutdown in installed size,
> > as well as a major reduction in downloads, if it were possible to just
> > select firmware from a couple of manufacturers.
> > 
> > While I was in the process of doing this, I noticed a few smaller
> > issues: -the non-firmware files that the APKBUILD tries to delete are
> > actually located in /lib/firmware, not /usr/lib/firmware
> > -the source for carl9170 firmware is included in the final package
> > 
> > Here's a tentative patch that fixes those issues and tries to split
> > things up.
> > 
> > Does this look like a good idea?
> > Should any further splitting be done? 
> > For example:
> > -splitting iwlwifi firmware off because it's the bulk of the intel
> > firmware, but not the bulk of the users;
> > -splitting qlogic, 3com, and chelsio firmware from -misc
> 
> Before doing this, a first step would be to delete the unneeded
> firmwares.
> 
> We already do this for modloop. But to do it for the package, would
> need a list of files needed to be maintained. I believe about half of
> the package can be just deleted based on our module configuration and
> specific kernel versions.
> 
> See http://git.alpinelinux.org/cgit/alpine-iso/tree/Makefile#n92 for
> how to get info on what firmware is required by the kernel modules.

Before doing that, there's a question of whether we're going to support
'noarch' "properly" (ie, with a single build/package upload for all
arches).
If so, it would be necessary to compile a single list of files for 
all arches.

Checking it against linux-firmware I get results along the lines of
- one third of the firmware is 'missing' (ie, not in linux-firmware)
A large chunk of this is in alsa-firmware and other packages.

- the remainder runs just over 40 megabytes

- there's lots of firmware that seems to be 'unneeded' even though the
relevant drivers are enabled (eg, only 2 files from brcm/ of 24 are
needed), which makes me wonder if something's wrong with modinfo...

> If the package is still too large after this, doing splitting might be
> an idea.

*If* the results are correct, it wouldn't be *needed*.

Thanks,
Isaac Dunham


---
Unsubscribe:  alpine-aports+unsubscribe@lists.alpinelinux.org
Help:         alpine-aports+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160403162518.213e9151@vostro>
In-Reply-To
<20160403052116.GA22996@newbook> (view parent)
Sender timestamp
1459689918
DKIM signature
missing
Download raw message
On Sat, 2 Apr 2016 22:21:18 -0700
Isaac Dunham <ibid.ag@gmail.com> wrote:

> I recently checked the size of installed packages on my system, and
> found that linux-firmware was the largest (~120 megabytes).
> I need the Realtek firmware, and perhaps could use the i915 firmware.
> So I figured it would be a significant cutdown in installed size,
> as well as a major reduction in downloads, if it were possible to just
> select firmware from a couple of manufacturers.
> 
> While I was in the process of doing this, I noticed a few smaller
> issues: -the non-firmware files that the APKBUILD tries to delete are
> actually located in /lib/firmware, not /usr/lib/firmware
> -the source for carl9170 firmware is included in the final package
> 
> Here's a tentative patch that fixes those issues and tries to split
> things up.
> 
> Does this look like a good idea?
> Should any further splitting be done? 
> For example:
> -splitting iwlwifi firmware off because it's the bulk of the intel
> firmware, but not the bulk of the users;
> -splitting qlogic, 3com, and chelsio firmware from -misc

Before doing this, a first step would be to delete the unneeded
firmwares.

We already do this for modloop. But to do it for the package, would
need a list of files needed to be maintained. I believe about half of
the package can be just deleted based on our module configuration and
specific kernel versions.

See http://git.alpinelinux.org/cgit/alpine-iso/tree/Makefile#n92 for
how to get info on what firmware is required by the kernel modules.

If the package is still too large after this, doing splitting might be
an idea.

Thanks,
Timo


---
Unsubscribe:  alpine-aports+unsubscribe@lists.alpinelinux.org
Help:         alpine-aports+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160404084305.3c82183e@vostro.util.wtbts.net>
In-Reply-To
<20160403205501.GA13799@newbook> (view parent)
Sender timestamp
1459748585
DKIM signature
missing
Download raw message
On Sun, 3 Apr 2016 13:55:02 -0700
Isaac Dunham <ibid.ag@gmail.com> wrote:

> On Sun, Apr 03, 2016 at 04:25:18PM +0300, Timo Teras wrote:
> > On Sat, 2 Apr 2016 22:21:18 -0700
> > Isaac Dunham <ibid.ag@gmail.com> wrote:
> >   
> > > I recently checked the size of installed packages on my system,
> > > and found that linux-firmware was the largest (~120 megabytes).
> > > I need the Realtek firmware, and perhaps could use the i915
> > > firmware. So I figured it would be a significant cutdown in
> > > installed size, as well as a major reduction in downloads, if it
> > > were possible to just select firmware from a couple of
> > > manufacturers.
> > > 
> > > While I was in the process of doing this, I noticed a few smaller
> > > issues: -the non-firmware files that the APKBUILD tries to delete
> > > are actually located in /lib/firmware, not /usr/lib/firmware
> > > -the source for carl9170 firmware is included in the final package
> > > 
> > > Here's a tentative patch that fixes those issues and tries to
> > > split things up.
> > > 
> > > Does this look like a good idea?
> > > Should any further splitting be done? 
> > > For example:
> > > -splitting iwlwifi firmware off because it's the bulk of the intel
> > > firmware, but not the bulk of the users;
> > > -splitting qlogic, 3com, and chelsio firmware from -misc  
> > 
> > Before doing this, a first step would be to delete the unneeded
> > firmwares.
> > 
> > We already do this for modloop. But to do it for the package, would
> > need a list of files needed to be maintained. I believe about half
> > of the package can be just deleted based on our module
> > configuration and specific kernel versions.
> > 
> > See http://git.alpinelinux.org/cgit/alpine-iso/tree/Makefile#n92 for
> > how to get info on what firmware is required by the kernel
> > modules.  
> 
> Before doing that, there's a question of whether we're going to
> support 'noarch' "properly" (ie, with a single build/package upload
> for all arches).
> If so, it would be necessary to compile a single list of files for 
> all arches.

We could make linux-firmware 'all' package and keep only the modules
relevant for that architectures kernel. Quite a bit of the hardware
is architecture specific. Well technically it would work, but nobody
really uses specific combinations and they are turned off in the
relevant configs.

> Checking it against linux-firmware I get results along the lines of
> - one third of the firmware is 'missing' (ie, not in linux-firmware)
> A large chunk of this is in alsa-firmware and other packages.
> 
> - the remainder runs just over 40 megabytes
> 
> - there's lots of firmware that seems to be 'unneeded' even though the
> relevant drivers are enabled (eg, only 2 files from brcm/ of 24 are
> needed), which makes me wonder if something's wrong with modinfo...

I believe it's just a result of having firmware for all supported
kernel versions in the linux-firmware package. E.g. many of the intel
wifi drivers duplicates. The firmware version used really depends on the
kernel version you run. Often the module version and the
required firmware version go hand in hand.

Each module can (and do) list several firmware file names -- all
potential names they can request. So the list of modules names should
be accurate.

> > If the package is still too large after this, doing splitting might
> > be an idea.  
> 
> *If* the results are correct, it wouldn't be *needed*.

There was request also for some other firmwares not in linux-firmware.
Such as the rpi3 wifi firmware. So we might need to split the firmware
package in any case.

Thanks,
Timo


---
Unsubscribe:  alpine-aports+unsubscribe@lists.alpinelinux.org
Help:         alpine-aports+help@lists.alpinelinux.org
---