Subject: [alpine-user] Boot Issue: How do I setup Xen with dom 0 Alpine Linux, LUKS LVM and GRUB on a UEFI platform?

From: Marco Boom <[email-masked]>
Date: Mon, 28 Jan 2019 21:08:00 +0000

Hi,

I would like to have the following setup: a system in UEFI mode with a hard disk with GPT partitions. The disk should contain an (unencrypted) EFI System Partition, encrypted boot partition and encrypted lvm partition. GRUB should be the bootloader and on top of it I want the Xen kernel and Alpine Linux as dom 0.

In order to install Alpine Linux without Xen I downloaded the ISO image and burned it on a USB drive with Rufus (GPT, iso mode). Then I I boot from USB drive in UEFI mode and I can install Alpine successfully.

To install Alpine Linux with the customized partitions I run the following commands (I’m sorry for the massive amount of code that follows. It is also attached as shell files. If you don’t have much time: scroll down to the second to last paragraph.):

Setting up Alpine Linux:

setup-keymap us us-intl

setup-hostname -n localhost

hostname=$(cat $ROOT/etc/hostname 2>/dev/null)

setup-interfaces -i <<EOF

auto lo

iface lo inet loopback

auto eth0

iface eth0 inet dhcp

    hostname $hostname

auto eth1

iface eth1 inet dhcp

    hostname $hostname

EOF

/etc/init.d/networking --quiet start >/dev/null

passwd

setup-timezone -z Europe/Amsterdam

setup-proxy none

setup-apkrepos -f

setup-sshd -c none

setup-ntp -c chrony

Install tools:

apk update

apk add cryptsetup e2fsprogs grub-efi haveged lvm2 parted

rc-service haveged start # optionally: only needed to wipe disks

Creating disk partitions:

parted --script /dev/sda mklabel gpt

parted --script --align=optimal /dev/sda mkpart fat32 0% 538MB

parted --script /dev/sda set 1 esp on

parted --script --align=optimal /dev/sda mkpart non-fs 538MB 748MB

parted --script --align=optimal /dev/sda mkpart non-fs 748MB 100%

parted --script /dev/sda set 3 LVM on

# optionally: wiping disks, but this takes too much time for test setups

haveged -n 0 | dd of=/dev/sda1

haveged -n 0 | dd of=/dev/sda2

haveged -n 0 | dd of=/dev/sda3

Creating file systems:

mkfs.vfat /dev/sda1 # fat32 for ESP

cryptsetup luksFormat --type luks /dev/sda2

cryptsetup open --type luks /dev/sda2 bootcrypt

mkfs.ext4 /dev/mapper/bootcrypt # encrypted boot partition with ext4

cryptsetup luksFormat --type luks2 /dev/sda3

cryptsetup open --type luks2 /dev/sda3 lvmcrypt

pvcreate /dev/mapper/lvmcrypt # encrypted lvm partition

vgcreate vg0 /dev/mapper/lvmcrypt

lvcreate -L 512M vg0 -n swap

lvcreate -l 100%FREE vg0 -n root

lvscan # check lvm partitions

mkfs.ext4 /dev/vg0/root # ext4 on lvm root partition (alias /dev/mapper/vg0-root)

mkswap /dev/vg0/swap # swap lvm partition (alias /dev/mapper/vg0-swap)

Creating mounts and folders, installing Alpine Linux:

mount -t ext4 /dev/vg0/root /mnt/

mkdir -p /mnt/boot/

mount -t ext4 /dev/mapper/bootcrypt /mnt/boot/

mkdir -p /mnt/boot/efi/

mount -t vfat /dev/sda1 /mnt/boot/efi/

USE_EFI=1 # seems to be ignored by the setup-disk script, can be removed

setup-disk -m sys /mnt/

Update configuration:

boot_UUID=$(blkid | awk "\$1 == \"/dev/sda2:\" { print \$2 }" | cut -d'"' -f2)

lvm_UUID=$(blkid | awk "\$1 == \"/dev/sda3:\" { print \$2 }" | cut -d'"' -f2)

root_UUID=$(blkid | awk "\$1 == \"/dev/mapper/vg0-root:\" { print \$2 }" | cut -d'"' -f2)

swap_UUID=$(blkid | awk "\$1 == \"/dev/mapper/vg0-swap:\" { print \$2 }" | cut -d'"' -f2)

printf "target='bootcrypt'\n" >> /mnt/etc/conf.d/dmcrypt

printf "source=UUID=\"$boot_UUID\"\n" >> /mnt/etc/conf.d/dmcrypt

#chroot /mnt rc-update add dmcrypt boot (there seems to be a bug in openrc: https://github.com/OpenRC/openrc/issues/243)

chroot /mnt ln -s /etc/init.d/dmcrypt /etc/runlevels/boot/dmcrypt # temporary workaround

printf "UUID=$swap_UUID\tswap\tswap\tdefault\t0 0\n" >> /mnt/etc/fstab

printf 'features="ata base ide scsi usb virtio ext4 lvm cryptsetup"\n' > /mnt/etc/mkinitfs/mkinitfs.conf

mkinitfs -c /mnt/etc/mkinitfs/mkinitfs.conf -b /mnt/ $(ls /mnt/lib/modules/)

mkdir -p /mnt/boot/grub/

mkdir -p /etc/default/

cat > /mnt/boot/grub/grub.cfg <<EOF

set timeout=2

insmod all_video

menuentry "Alpine Linux" {

    linux   /boot/vmlinuz-vanilla modules=sd-mod,usb-storage,ext4 cryptroot=UUID=$lvm_UUID cryptdm=lvmcrypt root=UUID=$root_UUID nomodeset quiet rootfstype=ext4

    initrd  /boot/initramfs-vanilla

}

EOF

cat >> /etc/default/grub <<EOF

GRUB_ENABLE_CRYPTODISK=y

EOF

grub-install --target=x86_64-efi --bootloader-id=alpine --boot-directory=/mnt/boot --efi-directory=/mnt/boot/efi --recheck --no-nvram

install -D /mnt/boot/efi/EFI/alpine/grubx64.efi /mnt/boot/efi/EFI/boot/bootx64.efi

In this way GRUB asks for the boot partition password, initramfs (or kernel or something else?) asks for the lvm partition password and finally OpenRC asks for the boot partition password (internet provides enough sources why the boot partition needs to be decrypted twice).

Finish setup:

umount /mnt/boot/efi/

umount /mnt/boot/

umount /mnt/

swapoff -a

vgchange -a n

cryptsetup luksClose lvmcrypt

cryptsetup luksClose bootcrypt

reboot

So at this point I have the system in UEFI mode with GPT partitions, LUKS, LVM, GRUB and Alpine Linux. I can use Alpine Linux as expected and no issues seems to be here.

Now I want to install Xen and run the following commands:

for mod in xen_netback xen_blkback xenfs xen_pciback xen_wdt tun; do

    if modprobe $mod; then

        grep -q -q $mod /etc/modules || echo $mod >> /etc/modules

    fi

done

apk add xen xen-hypervisor

for svc in xenstored xenconsoled xendomains xenqemu; do

    rc-update add $svc default

done

grubcfg=$(cat /boot/grub/grub.cfg)

cat > /boot/grub/grub.cfg <<EOF

menuentry "Xen Alpine Linux" {

    multiboot2  /boot/xen.gz placeholder smt=1

    module2 /boot/vmlinuz-vanilla placeholder modules=sd-mod,usb-storage,ext4 cryptroot=UUID=$lvm_UUID cryptdm=lvmcrypt root=UUID=$root_UUID nomodeset quiet rootfstype=ext4

    module2 /boot/initramfs-vanilla

}

$grubcfg

EOF

When I choose Xen Alpine Linux from the boot options Xen seems to start without errors, but after it relinquishes the console I got a black screen and the keyboard does not respond anymore. The same happens when I remove the quiet kernel option from grub.cfg. It does not output any additional information. How can I fix this issue or is this setup not supported?

I've noticed that when I use multiboot and module instead of multiboot2 and module2 Xen throws the error "(XEN) ACPI Error (tbxfroot-8217): A valid RSDP was not found [20070126]" and turns ACPI off but this time after it relinquishes the console, the system asks for the password of the lvm partition. But the problem here is that the keyboard does not respond, so I cannot fill in the password and continue the boot process.

With kind regards,

Marco Boom