Subject: ZFS ROOT fails with mount: mounting zroot/ROOT/default on /sysroot failed: No such file or directory

From: Greg Hall <[email-masked]>
Date: Sun, 17 May 2020 00:24:23 -0300

I have two scripts for partitioning a hard drive and setting up Alpine
Linux 3.11.

One script uses GPT with an EFI system partition (doesn't actually install
an EFI bootloader, just a BIOS boot loader), and an ext4 root partition. It
works perfectly.

The second script is a copy of the first, where the only changes are
related to ZFS. It works, except for one minor irritating quirk: at boot
time, I get the following message:

mount: mounting zroot/ROOT/default on /sysroot failed: No such file or
directory
Mounting root failed.
initramfs emergency recovery shell launched. Type 'exit' to continue boot
sh: can't access tty; job control turned off

I simply type in the following two commands, and it boots up just fine:
/ # mount -t zfs zroot/ROOT/default /sysroot
/ # exit

I don't know why this manual mounting is necessary. I don't have this
problem with ext4. It seems odd to me that I can simply go ahead and mount
the exact thing it says doesn't exist - especially since the mount command
requires that the zpool has already been imported.

After booting up, the /boot/extlinux.conf has the following lines in it:

LABEL lts
  MENU default
  MENU LABEL Linux lts
  LINUX vmlinuz-lts
  INITRD initramfs-lts
  APPEND ROOT=zpool/ROOT/default modules=sd-mod,usb-storage,zfs quiet

The lines of my install script that perform ZFS operations are as follows:

########
# The installation system is a plain old ext4 system installed by using
setup-alpine
# This is necessary because the boot media can't have ZFS added to it
# sda is the ext4 system installed with setup-alpine
# sdb is the drive to install zfs to
# Modify installation system on sda to ensure it has ZFS and other needed
things
apk add parted zfs zfs-`uname -r | rev | cut -d- -f 1 | rev` util-linux udev
echo zfs > /etc/modules-load.d/zfs.conf
modprobe zfs

# Partition sdb - I find sgdisk is unreliable at creating device entries,
so I use parted
# Parted doesn't understand the zfs partition code, so I lie and use ext4
echo 'mklabel gpt
mkpart BOOT fat32 2048s 100
set 1 esp on
set 1 legacy_boot on
mkpart ROOT ext4 100 -100
print
' | parted /dev/sdb

# Determine ashift from physical sector size, should be 9 (sector size =
512) or 12 (sector size = 4096)
phy_sec="`lsblk -nldo PHY-SEC /dev/sdb`"
ashift="0"
while [ "$phy_sec" -ne 1 ]
do
    phy_sec=$((phy_sec / 2))
    ashift=$((ashift + 1))
done

# Create zfs filesystem
pool="zroot"
vdev="/dev/disk/by-partuuid/`lsblk -nlo PARTUUID /dev/sdb2`"
zpool create -f -o ashift=$ashift -O normalization=formD -m none $pool $vdev
zfs create -o mountpoint=none -o canmount=off $pool/ROOT
zfs create -o mountpoint=legacy $pool/ROOT/default
zpool set bootfs=$pool/ROOT/default $pool

# Mount filesystems
mount -t zfs $pool/ROOT/default /mnt
mkdir /mnt/boot
mount /dev/sdb1 /mnt/boot

# Install base system
apk --arch x86_64 \
   -X "`grep '^http' /etc/apk/repositories`" \
   -U \
   --allow-untrusted \
   --root /mnt \
   --initdb \
   add alpine-base tzdata alpine-mirrors linux-lts zfs zfs-lts syslinux
parted util-linux udev chrony

# Copy zfs info
cp /etc/zfs/zpool.cache /mnt/etc/zfs

# Setup bootloader (in chroot)
dd bs=440 count=1 if=/usr/share/syslinux/gptmbr.bin of=/dev/sdb
sed -i -r "s/^(default_kernel_opts=.*)/\1 rootfstype=zfs/"
/etc/update-extlinux.conf
sed -i "s,^root=.*,root=$pool/ROOT/default," /etc/update-extlinux.conf
sed -i -r "s/^(modules=.*)ext4(.*)/\1zfs\2/" /etc/update-extlinux.conf
update-extlinux
extlinux --install /boot

# Run mkinitfs (in chroot)
sed -i 's/"$/ zfs"/' /etc/mkinitfs/mkinitfs.conf
mkinitfs $(ls /lib/modules)

# Ensure zfs module loads at boot (in chroot)
echo zfs > /etc/modules-load.d/zfs.conf

# Load zfs rc scripts (in chroot)
rc-update add zfs-import sysinit
rc-update add zfs-mount sysinit

# Set up fstab (in chroot)
echo -n '/dev/sda1    /boot    vfat   defaults   0 0
$pool/ROOT/default    /    zfs   defaults   0 0
' > /etc/fstab

# After exiting chroot, unmount filesystems
umount /mnt/boot
zfs umount -a
zpool export $pool
########

I don't know if can provide attachments to a mailing list, but would be
happy to provide the complete install script.

Thanks !

---

From: Konstantin Kulikov <[email-masked]>
Date: Sun, 17 May 2020 15:39:24 +0300

>   APPEND ROOT=zpool/ROOT/default modules=sd-mod,usb-storage,zfs quiet

I think if you change to APPEND root=zpool/ROOT/default" (root lowercased)
and add "zfs_force=1" it'll start working.

Another thing you can try is to set
# zfs set mountpoint=/ zpool/ROOT/default
and in bootloader:
APPEND root=ZFS=zpool/ROOT/default
rootfstype and zfs_force are not neccessary in this case.

It's difficult for me to say exactly what the problem is
because interaction between initramfs-init and nlplug-findfs
is complicated in zfs case and I have troubles reading C code.

Relevant links:
https://git.alpinelinux.org/mkinitfs/tree/initramfs-init.in#n506
https://git.alpinelinux.org/mkinitfs/tree/nlplug-findfs.c#n1015