Apparently there were multiple issues with my setup, so I have updated the scripts.


  *   Using as much as the same tools during normal setup (eg replacing parted with sfdisk)
  *   Different disk layout. The boot partition is gone. The physical disk /dev/sda contains an esp and luks partition. On the luks partition is an lvm group containing a swap and root partition. The boot folder is inside the root partition and mounts the esp partition on /boot/efi. This disk layout asks for a decryption password twice (once by GRUB2 and once by the ramdisk).
  *   UEFI and BIOS mode simultanously installed when the OS setup will be run in UEFI mode. This makes it possible to start the system in UEFI or BIOS mode, which can be chosen during each boot cycle.
  *   Using GRUB2 configuration scripts to generate the GRUB2 config file instead of manually building the config file.
  *   Fixing some issues with the GRUB2 Xen configuration script. (I didn’t find the repository where this script is stored. A pull request could be made in order to fix these issues in the package).

So, now the system supports UEFI mode and BIOS mode, I have tested the Xen setup in both modes. The system works in BIOS mode but when I start in UEFI mode I still get a black screen after the Xen kernel is loaded. Maybe this is an GRUB2 EFI issue since I have found some older messages indicating a similar problem. But according to Xen documentation it should be able to start Xen in UEFI mode by now. Does someone knows the details about EFI, GRUB2 and Xen?

With kind regards,

Marco Boom


Van: Marco Boom <marco.boom@outlook.com>
Verzonden: Monday, January 28, 2019 10:08:00 PM
Aan: alpine-user@lists.alpinelinux.org
Onderwerp: Boot Issue: How do I setup Xen with dom 0 Alpine Linux, LUKS LVM and GRUB on a UEFI platform?


Hi,

I would like to have the following setup: a system in UEFI mode with a hard disk with GPT partitions. The disk should contain an (unencrypted) EFI System Partition, encrypted boot partition and encrypted lvm partition. GRUB should be the bootloader and on top of it I want the Xen kernel and Alpine Linux as dom 0.

In order to install Alpine Linux without Xen I downloaded the ISO image and burned it on a USB drive with Rufus (GPT, iso mode). Then I I boot from USB drive in UEFI mode and I can install Alpine successfully.

To install Alpine Linux with the customized partitions I run the following commands (I’m sorry for the massive amount of code that follows. It is also attached as shell files. If you don’t have much time: scroll down to the second to last paragraph.):

Setting up Alpine Linux:

setup-keymap us us-intl

setup-hostname -n localhost

hostname=$(cat $ROOT/etc/hostname 2>/dev/null)

setup-interfaces -i <<EOF

auto lo

iface lo inet loopback



auto eth0

iface eth0 inet dhcp

    hostname $hostname



auto eth1

iface eth1 inet dhcp

    hostname $hostname

EOF

/etc/init.d/networking --quiet start >/dev/null

passwd

setup-timezone -z Europe/Amsterdam

setup-proxy none

setup-apkrepos -f

setup-sshd -c none

setup-ntp -c chrony



Install tools:

apk update

apk add cryptsetup e2fsprogs grub-efi haveged lvm2 parted

rc-service haveged start # optionally: only needed to wipe disks



Creating disk partitions:

parted --script /dev/sda mklabel gpt

parted --script --align=optimal /dev/sda mkpart fat32 0% 538MB

parted --script /dev/sda set 1 esp on

parted --script --align=optimal /dev/sda mkpart non-fs 538MB 748MB

parted --script --align=optimal /dev/sda mkpart non-fs 748MB 100%

parted --script /dev/sda set 3 LVM on



# optionally: wiping disks, but this takes too much time for test setups

haveged -n 0 | dd of=/dev/sda1

haveged -n 0 | dd of=/dev/sda2

haveged -n 0 | dd of=/dev/sda3



Creating file systems:

mkfs.vfat /dev/sda1 # fat32 for ESP



cryptsetup luksFormat --type luks /dev/sda2

cryptsetup open --type luks /dev/sda2 bootcrypt

mkfs.ext4 /dev/mapper/bootcrypt # encrypted boot partition with ext4



cryptsetup luksFormat --type luks2 /dev/sda3

cryptsetup open --type luks2 /dev/sda3 lvmcrypt

pvcreate /dev/mapper/lvmcrypt # encrypted lvm partition

vgcreate vg0 /dev/mapper/lvmcrypt

lvcreate -L 512M vg0 -n swap

lvcreate -l 100%FREE vg0 -n root

lvscan # check lvm partitions

mkfs.ext4 /dev/vg0/root # ext4 on lvm root partition (alias /dev/mapper/vg0-root)

mkswap /dev/vg0/swap # swap lvm partition (alias /dev/mapper/vg0-swap)



Creating mounts and folders, installing Alpine Linux:

mount -t ext4 /dev/vg0/root /mnt/

mkdir -p /mnt/boot/

mount -t ext4 /dev/mapper/bootcrypt /mnt/boot/

mkdir -p /mnt/boot/efi/

mount -t vfat /dev/sda1 /mnt/boot/efi/

USE_EFI=1 # seems to be ignored by the setup-disk script, can be removed

setup-disk -m sys /mnt/



Update configuration:

boot_UUID=$(blkid | awk "\$1 == \"/dev/sda2:\" { print \$2 }" | cut -d'"' -f2)

lvm_UUID=$(blkid | awk "\$1 == \"/dev/sda3:\" { print \$2 }" | cut -d'"' -f2)

root_UUID=$(blkid | awk "\$1 == \"/dev/mapper/vg0-root:\" { print \$2 }" | cut -d'"' -f2)

swap_UUID=$(blkid | awk "\$1 == \"/dev/mapper/vg0-swap:\" { print \$2 }" | cut -d'"' -f2)



printf "target='bootcrypt'\n" >> /mnt/etc/conf.d/dmcrypt

printf "source=UUID=\"$boot_UUID\"\n" >> /mnt/etc/conf.d/dmcrypt

#chroot /mnt rc-update add dmcrypt boot (there seems to be a bug in openrc: https://github.com/OpenRC/openrc/issues/243)

chroot /mnt ln -s /etc/init.d/dmcrypt /etc/runlevels/boot/dmcrypt # temporary workaround

printf "UUID=$swap_UUID\tswap\tswap\tdefault\t0 0\n" >> /mnt/etc/fstab

printf 'features="ata base ide scsi usb virtio ext4 lvm cryptsetup"\n' > /mnt/etc/mkinitfs/mkinitfs.conf

mkinitfs -c /mnt/etc/mkinitfs/mkinitfs.conf -b /mnt/ $(ls /mnt/lib/modules/)



mkdir -p /mnt/boot/grub/

mkdir -p /etc/default/

cat > /mnt/boot/grub/grub.cfg <<EOF

set timeout=2

insmod all_video

menuentry "Alpine Linux" {

    linux   /boot/vmlinuz-vanilla modules=sd-mod,usb-storage,ext4 cryptroot=UUID=$lvm_UUID cryptdm=lvmcrypt root=UUID=$root_UUID nomodeset quiet rootfstype=ext4

    initrd  /boot/initramfs-vanilla

}

EOF

cat >> /etc/default/grub <<EOF

GRUB_ENABLE_CRYPTODISK=y

EOF

grub-install --target=x86_64-efi --bootloader-id=alpine --boot-directory=/mnt/boot --efi-directory=/mnt/boot/efi --recheck --no-nvram

install -D /mnt/boot/efi/EFI/alpine/grubx64.efi /mnt/boot/efi/EFI/boot/bootx64.efi



In this way GRUB asks for the boot partition password, initramfs (or kernel or something else?) asks for the lvm partition password and finally OpenRC asks for the boot partition password (internet provides enough sources why the boot partition needs to be decrypted twice).

Finish setup:

umount /mnt/boot/efi/

umount /mnt/boot/

umount /mnt/

swapoff -a

vgchange -a n

cryptsetup luksClose lvmcrypt

cryptsetup luksClose bootcrypt



reboot



So at this point I have the system in UEFI mode with GPT partitions, LUKS, LVM, GRUB and Alpine Linux. I can use Alpine Linux as expected and no issues seems to be here.

Now I want to install Xen and run the following commands:

for mod in xen_netback xen_blkback xenfs xen_pciback xen_wdt tun; do

    if modprobe $mod; then

        grep -q -q $mod /etc/modules || echo $mod >> /etc/modules

    fi

done



apk add xen xen-hypervisor



for svc in xenstored xenconsoled xendomains xenqemu; do

    rc-update add $svc default

done



grubcfg=$(cat /boot/grub/grub.cfg)

cat > /boot/grub/grub.cfg <<EOF

menuentry "Xen Alpine Linux" {

    multiboot2  /boot/xen.gz placeholder smt=1

    module2 /boot/vmlinuz-vanilla placeholder modules=sd-mod,usb-storage,ext4 cryptroot=UUID=$lvm_UUID cryptdm=lvmcrypt root=UUID=$root_UUID nomodeset quiet rootfstype=ext4

    module2 /boot/initramfs-vanilla

}



$grubcfg

EOF



When I choose Xen Alpine Linux from the boot options Xen seems to start without errors, but after it relinquishes the console I got a black screen and the keyboard does not respond anymore. The same happens when I remove the quiet kernel option from grub.cfg. It does not output any additional information. How can I fix this issue or is this setup not supported?

I've noticed that when I use multiboot and module instead of multiboot2 and module2 Xen throws the error "(XEN) ACPI Error (tbxfroot-8217): A valid RSDP was not found [20070126]" and turns ACPI off but this time after it relinquishes the console, the system asks for the password of the lvm partition. But the problem here is that the keyboard does not respond, so I cannot fill in the password and continue the boot process.


With kind regards,

Marco Boom