Subject: Installing package with kernel module fails on diskless mode? (wireguard-rpi)

From: Davy Landman <[email-masked]>
Date: Mon, 10 Feb 2020 10:44:16 +0100

I'm loving alpine, but I'm running into a problem that I do not know how to
solve, and no amount of googling has helped me.

I'm on v3.11.3 on a rpi1, just downloaded the release and unpacked it onto
a fresh fat32 sd card. It boots-up and stuff just works. Regular package
installation works, but when I wanted to add wireguard, it fails and I
think I understand what is happening, but I cannot seem to figure out to
solve it.

Here is the full log:

(1/1) Installing wireguard-rpi (5.4.12-r0)
ERROR: Failed to create lib/modules/5.4.12-0-rpi/extra/wireguard.ko: No
such file or directory
ERROR: wireguard-rpi-5.4.12-r0: failed to rename
lib/modules/5.4.12-0-rpi/extra/.apk.6b9f5d1b725356e84556fbab4f1ce19495578693bf92d01e
to lib/modules/5.4.12-0-rpi/extra/wireguard.ko.
Executing busybox-1.31.1-r9.trigger
Executing kmod-26-r0.trigger
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.dep.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.dep.bin.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.alias.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.alias.bin.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.softdep.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.symbols.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.symbols.bin.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.builtin.bin.2636.32472.1581276993, 301, 644): Read-only file system
depmod: ERROR: openat(/lib/modules/5.4.12-0-rpi,
modules.devname.2636.32472.1581276993, 301, 644): Read-only file system
1 error; 135 MiB in 57 packages

if I understand correctly, it wants to add a kernel module but fails since
the kernel is on a read-only side of the system. I've tried remounting the
whole disk as rw, but that didn't solve it.

I'm hoping someone can help me or maybe there is a wiki page I missed, but
it looks like this should work, is the plain alpine-rpi install and the
plain precompiled wireguard-rpi package.

Thanks,
Davy

---

From: Marco Dickert <[email-masked]>
Date: Tue, 11 Feb 2020 08:47:45 +0100

> whole disk as rw, but that didn't solve it.

Can you provide us with the output of "cat /proc/mounts"? Are you able
to install other packages than kernel modules?

Cheers,
-- 
Marco Dickert
[email-masked]
https://misterunknown.de

---

From: Davy Landman <[email-masked]>
Date: Tue, 11 Feb 2020 09:29:40 +0100

indeed, other packages are no problem. just to be clear, I followed these
instructions: https://wiki.alpinelinux.org/wiki/Raspberry_Pi#Installation

$ cat /proc/mounts
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
devtmpfs /dev devtmpfs
rw,nosuid,relatime,size=10240k,nr_inodes=53900,mode=755 0 0
devpts /dev/pts devpts
rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
/dev/mmcblk0p1 /media/mmcblk0p1 vfat
ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro
0 0
tmpfs / tmpfs rw,relatime,mode=755 0 0
tmpfs /run tmpfs rw,nodev,relatime,size=44376k,mode=755 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
/dev/loop0 /.modloop squashfs ro,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0
tracefs /sys/kernel/debug/tracing tracefs rw,nosuid,nodev,noexec,relatime 0
0

Cheers,
Davy

---

From: Marco Dickert <[email-masked]>
Date: Tue, 11 Feb 2020 10:18:27 +0100

Disclaimer: I neither used alpine on a Raspi, nor a diskless setup at
all. I'm just trying to understand what happens.

Apparently Alpine uses a mechanism called "modloop", which specifies a
squashfs filesystem that is mounted on "/.modloop". This mountpoint is
in turn the target of the /lib/modules symlink – that explains why apk
grouses about missing write permissions.

However, I'm not sure how you're supposed to add kernel modules. There
is this [1] wiki page which indicates, that you can set modloop= to an
URL which contains an image with additional kernel modules, but I'm not
sure if that is the solution of choice for your use case.

-- 
Marco Dickert
[email-masked]
https://misterunknown.de

---

From: Christian Haschek <[email-masked]>
Date: Mon, 24 Feb 2020 18:14:08 +0100

I'm having the same problems and I have tested so far with

- PXE Booted Alpine (using 
https://blog.haschek.at/2019/build-your-own-datacenter-with-pxe-and-alpine.html
)
- RasPi 3 and 4 (using 
https://wiki.alpinelinux.org/wiki/Raspberry_Pi#Installation)
- USB booted (using 
https://wiki.alpinelinux.org/wiki/Create_a_Bootable_USB)
- Full disk install on a Server

Only the last method allowed me to install the wireguard packages. I
went to the IRC a few days ago and someone said they're preparing a
method that will allow diskless setups to install these modules but I
didn't get a hold of the guy in charge.

Does anyone have any updates on this matter?

Am Montag, den 10.02.2020, 10:44 +0100 schrieb Davy Landman:
> I'm loving alpine, but I'm running into a problem that I do not know
> how to solve, and no amount of googling has helped me.
> 
> I'm on v3.11.3 on a rpi1, just downloaded the release and unpacked it
> onto a fresh fat32 sd card. It boots-up and stuff just works. Regular
> package installation works, but when I wanted to add wireguard, it
> fails and I think I understand what is happening, but I cannot seem
> to figure out to solve it.
> 
> Here is the full log:
> 
> ERROR: Failed to create lib/modules/5.4.12-0-rpi/extra/wireguard.ko:
> No such file or directory
> ERROR: wireguard-rpi-5.4.12-r0: failed to rename lib/modules/5.4.12-
> 0-rpi/extra/.apk.6b9f5d1b725356e84556fbab4f1ce19495578693bf92d01e to
> lib/modules/5.4.12-0-rpi/extra/wireguard.ko.
> if I understand correctly, it wants to add a kernel module but fails
> since the kernel is on a read-only side of the system. I've tried
> remounting the whole disk as rw, but that didn't solve it.
> 
> I'm hoping someone can help me or maybe there is a wiki page I
> missed, but it looks like this should work, is the plain alpine-rpi
> install and the plain precompiled wireguard-rpi package.
> 
> Thanks,
> Davy

---

From: Allan Riordan Boll <[email-masked]>
Date: Thu, 18 Jun 2020 19:50:44 -0700

Had this problem too, and came up with a (hacky) solution. Hopefully, 
now that Wireguard is officially part of the kernel since a few months 
ago, this will not be needed in the future. But for now, the following 
does the trick:

# Remount kernel module dir /.modloop as an overlay, to allow writing
modprobe overlay
mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
mount /dev/loop0 /.modloop.lower
umount /.modloop/
mount -t overlay -o 
lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir 
none /.modloop
lbu include /.modloop.upper
lbu commit -d

# Manually get the Wireguard kernel module to avoid installing
# the wireguard-rpi2 which does not work with diskless systems
cd /tmp
pkgname=$(apk list | grep wireguard-rpi2 | cut -d " " -f 1)
wget http://dl-cdn.alpinelinux.org/alpine/v3.12/community/armhf/$pkgname.apk
mkdir /tmp/$pkgname
tar -xzf $pkgname.apk -C /tmp/$pkgname
mkdir -p /lib/modules/$(uname -r)/extra/
cp /tmp/$pkgname/lib/modules/$(uname -r)/extra/wireguard.ko \
   /lib/modules/$(uname -r)/extra/
rm -fr /tmp/$pkgname /tmp/$pkgname.apk
depmod

# Create an init script to remount the /.modloop overlay on next boot
cat > /etc/init.d/modloopoverlay <<EOF
#!/sbin/openrc-run

depend() {
     before networking
     need modules
}

start() {
     ebegin "Starting modloop overlay"
     modprobe overlay
     mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
     if [ ! -d /.modloop.lower/modules ]; then
         mount /dev/loop0 /.modloop.lower
     fi
     umount /.modloop
     mount -t overlay -o 
lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir 
none /.modloop
     eend 0
}
EOF
chmod +x /etc/init.d/modloopoverlay
/etc/init.d/modloopoverlay restart
rc-update add modloopoverlay boot
lbu include /etc/init.d/modloopoverlay
lbu commit -d