Subject: [alpine-user] Raspberry Pi Zero can't boot rpi armhf version

From: East <[email-masked]>
Date: Tue, 21 May 2019 01:00:31 +0000

I am starting a new thread here, seeing as my earlier question was answered, (thanks Paul).

Based on [this](https://wiki.alpinelinux.org/wiki/Raspberry_Pi_Zero_W_-_Installation) page, I am lead to believe that someone got Alpine working on the Pi Zero W. The Zero W is basically the same as the RPi Zero, with the only difference being the addition of wifi and bluetooth. In that page, it sounded like most of the author's problems had to do with wifi, and the install worked normally otherwise. So, there is the possibility that something changed between 3.9.2 and 3.9.4 that broke the install on RPi Zero.

I have been referencing these three pages:
[1] https://wiki.alpinelinux.org/wiki/Raspberry_Pi_Zero_W_-_Installation (linked above)
[2] https://wiki.alpinelinux.org/wiki/Raspberry_Pi
[3] https://wiki.alpinelinux.org/wiki/Classic_install_or_sys_mode_on_Raspberry_Pi

The installation process is basically the same for the first part: simply create a FAT32 partition (in [1] and [2]) or FAT16 (in [3]) and extract the tar to it.

[1] recommends setting the following in usercfg.txt:
gpu_mem=16
dtparam=audio=off
dtoverlay=pi3-disable-bt
dtoverlay=w1-gpio
enable_uart=1

It also has a section: "Create cache folder and add rng-tools packages". I do not believe this section is relevant for this problem.

[2] recommends setting the following in usercfg.txt:
dtparam=audio=on
disable_overscan=1

[3] recommends this in usercfg.txt:
enable_uart=1

With the exception of perhaps "enable_uart", I don't think any of these settings would prevent the Pi Zero from booting.

I have extracted the tar to both FAT32 and FAT16 partitions. I have tried setting the boot and lba flags on the partition in gparted. No matter what I do, the Pi Zero will not boot.

My theory is that this has something to do with the Pi Zero being unable to find the correct kernel. The LED blinks 7 times, pauses, and repeats. From [here](https://elinux.org/R-Pi_Troubleshooting#Green_LED_blinks_in_a_specific_pattern): "7 flashes: kernel.img not found"

I have tried moving "boot/vmlinuz-rpi" up a directory (to what would be the / directory if it were booted instead of mounted on my PC) and renaming it "kernel.img". When I try booting it that way, I get a kernel panic:

"Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

This is an improvement, because at least it gives me something other than the rainbow screen (which I enabled in config.txt).

Beyond this, I am not sure what I should do. In theory, the Pi Zero should parse the config.txt file and locate the correct kernel to load that way. I have tried setting "boot_delay=1" to no avail. I have also theorized that it might refuse to boot because I extracted the files from the tar using my regular user, and the Pi won't read config.txt unless it is owned by root. A recursive chgrp/chown has debunked that theory, because even after that it still refuses to boot.

I am going to try moving/renaming initramfs as well as the kernel and see if that changes anything. At the very least, it might be a workaround... but if I do that, it will refuse to boot on RPi 2/3. In my case I don't care, because I only intend to use this on a Zero, but would be unusable as a long term fix.

---

From: Paul Zillmann <[email-masked]>
Date: Tue, 21 May 2019 15:35:25 +0200

Hello East,

> "Kernel panic - not syncing: VFS: Unable to mount root fs on 
suggest that the kernel doesn't know what rootfs to mount nor has any 
modules for that.
An initramfs would be usefull in that situation - at least to get an 
emergency shell.
In the config.txt look out for the kernel= line (you could also set it 
so your kernel don't have to be on kernel.img).
kernel=path/to/kernel
initramfs path/to/initramfs

The missing = sign is there on purpose!

You could also try ramfsfile= and ramfsaddr= but I've found no memory 
address for the Pi Zero W. Some use 0x00a0000, other use 0x00800000, 
"followkernel" is also a valid address.

You could also create an cmdline.txt which should contain kernel cmd 
paremeters, eg. root=/dev/mmcblk0xyz

Please check out this resource and see if it helps: 
https://pi3g.com/2019/01/10/alpine-boot-process-on-the-raspberry-pi/

Here are some boot parameters for the Pi bootloader: 
https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md

- Paul

Am 21.05.19 um 03:00 schrieb East:
> Based on this 
> <https://wiki.alpinelinux.org/wiki/Raspberry_Pi_Zero_W_-_Installation> 
> page, I am lead to believe that someone got Alpine working on the Pi 
> Zero W. The Zero W is basically the same as the RPi Zero, with the 
> only difference being the addition of wifi and bluetooth. In that 
> page, it sounded like most of the author's problems had to do with 
> wifi, and the install worked normally otherwise. So, there is the 
> possibility that something changed between 3.9.2 and 3.9.4 that broke 
> the install on RPi Zero.
>
> [1] 
> https://wiki.alpinelinux.org/wiki/Raspberry_Pi_Zero_W_-_Installation 
> (linked above)
> [3] 
> https://wiki.alpinelinux.org/wiki/Classic_install_or_sys_mode_on_Raspberry_Pi
>
> The installation process is basically the same for the first part: 
> simply create a FAT32 partition (in [1] and [2]) or FAT16 (in [3]) and 
> extract the tar to it.
>
> It also has a section: "Create cache folder and add rng-tools 
> packages". I do not believe this section is relevant for this problem.
>
> With the exception of perhaps "enable_uart", I don't think any of 
> these settings would prevent the Pi Zero from booting.
>
> I have extracted the tar to both FAT32 and FAT16 partitions. I have 
> tried setting the boot and lba flags on the partition in gparted. No 
> matter what I do, the Pi Zero will not boot.
>
> My theory is that this has something to do with the Pi Zero being 
> unable to find the correct kernel. The LED blinks 7 times, pauses, and 
> repeats. From here 
> <https://elinux.org/R-Pi_Troubleshooting#Green_LED_blinks_in_a_specific_pattern>: 
> "7 flashes: kernel.img not found"
>
> I have tried moving "boot/vmlinuz-rpi" up a directory (to what would 
> be the / directory if it were booted instead of mounted on my PC) and 
> renaming it "kernel.img". When I try booting it that way, I get a 
> kernel panic:
>
> This is an improvement, because at least it gives me something other 
> than the rainbow screen (which I enabled in config.txt).
>
> Beyond this, I am not sure what I should do. In theory, the Pi Zero 
> should parse the config.txt file and locate the correct kernel to load 
> that way. I have tried setting "boot_delay=1" to no avail. I have also 
> theorized that it might refuse to boot because I extracted the files 
> from the tar using my regular user, and the Pi won't read config.txt 
> unless it is owned by root. A recursive chgrp/chown has debunked that 
> theory, because even after that it still refuses to boot.
>
> I am going to try moving/renaming initramfs as well as the kernel and 
> see if that changes anything. At the very least, it might be a 
> workaround... but if I do that, it will refuse to boot on RPi 2/3. In 
> my case I don't care, because I only intend to use this on a Zero, but 
> would be unusable as a long term fix.

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 00:33:04 +0000

I have read both of those links thoroughly. Neither tell me anything new.

As I said in one of my other emails, the issue seems to be that it isn't reading config.txt. I have tried taking out the model specific conditionals and moving the file to ./boot, neither of which have changed anything. I have tried making the kernel line an absolute path, I have removing boot/ from the front of it (after moving config.txt to /boot), neither of which have done anything. If it could read config.txt properly, I wouldn't have this problem in the first place. By default, the Alpine image HAS a kernel line in config.txt that points to the correct kernel, it just isn't using config.txt for whatever reason. No matter what I change in config.txt, it seems to have no effect. I know that the file is changing SOME settings, because I can enable the rainbow screen and set boot_delay... But the kernel lines aren't correctly loading the kernel.

I have also tried doing this with Alpine 3.9.2, because that is the version used in [1]. Same problem.

I have also tried moving the kernel to /kernel.img again, and specifying the root filesystem in cmdline.txt. It still kernel panics. I have also tried specifying the rootfstype as fat32, and rootwait. Neither of them do anything. With only rootfstype=fat32 and root=/dev/mmcblk0p1, I also get a register dump and "mmc0: timeout waiting for hardware interrupt". I don't believe this is relevant.

Because config.txt is being correctly read for SOME of the settings, that implies there is some kind of error with the kernel line.

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, May 21, 2019 1:35 PM, Paul Zillmann <[email-masked]> wrote:

> suggest that the kernel doesn't know what rootfs to mount nor has any modules for that.
> An initramfs would be usefull in that situation - at least to get an emergency shell.
> In the config.txt look out for the kernel= line (you could also set it so your kernel don't have to be on kernel.img).
> kernel=path/to/kernel
> You could also try ramfsfile= and ramfsaddr= but I've found no memory address for the Pi Zero W. Some use 0x00a0000, other use 0x00800000, "followkernel" is also a valid address.
>
> You could also create an cmdline.txt which should contain kernel cmd paremeters, eg. root=/dev/mmcblk0xyz
>
> Please check out this resource and see if it helps: https://pi3g.com/2019/01/10/alpine-boot-process-on-the-raspberry-pi/
>
> Here are some boot parameters for the Pi bootloader: https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md
>
> - Paul
>

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 07:25:08 +0000

I apologize if my tone earlier was abrasive. I was frustrated at the time.

I am going to try to figure out how "kernel=" should be formatted. I have compared it to raspbian, and by default, raspbian does not use that line at all. They simply place the kernel(s) in the / directory, named kernel.img and kernel7.img. IIRC, the latter is used for the pi 2/3, while the former is used for 0/1. I have no idea where they keep initramfs.

I have also tested this on a Pi 1/B, (2012 model) with the same result. I do not have any other Pi's laying around on which I can test it, so I can't be sure if this issue affects models other than 1B and Zero.

Another theory I had was that this was failing because it had multiple partitions. Because I intended to do a sys install, I had one FAT32 and one ext4 partition. I thought there was an off chance it was failing because it couldin't find the right partition. This theory is wrong, because it also fails with only the FAT32 partition.

---

From: Gee <[email-masked]>
Date: Wed, 22 May 2019 09:04:44 +0100

For what it's worth, I am seeing this behaviour on a 3B+ as well, but not
with raspbian on the same hardware.

I'll try some older builds later to see if we're dealing with a regression
or not...

On Wed, 22 May 2019, 08:25 East, <[email-masked]> wrote:

> I am going to try to figure out how "kernel=" should be formatted. I have
> compared it to raspbian, and by default, raspbian does not use that line at
> all. They simply place the kernel(s) in the / directory, named kernel.img
> and kernel7.img. IIRC, the latter is used for the pi 2/3, while the former
> is used for 0/1. I have no idea where they keep initramfs.
>
> I have also tested this on a Pi 1/B, (2012 model) with the same result. I
> do not have any other Pi's laying around on which I can test it, so I can't
> be sure if this issue affects models other than 1B and Zero.
>
> Another theory I had was that this was failing because it had multiple
> partitions. Because I intended to do a sys install, I had one FAT32 and one
> ext4 partition. I thought there was an off chance it was failing because it
> couldin't find the right partition. This theory is wrong, because it also
> fails with only the FAT32 partition.
>
>
> Sent with ProtonMail <https://protonmail.com> Secure Email.
>
> As I said in one of my other emails, the issue seems to be that it isn't
> reading config.txt. I have tried taking out the model specific conditionals
> and moving the file to ./boot, neither of which have changed anything. I
> have tried making the kernel line an absolute path, I have removing boot/
> from the front of it (after moving config.txt to /boot), neither of which
> have done anything. If it could read config.txt properly, I wouldn't have
> this problem in the first place. By default, the Alpine image HAS a kernel
> line in config.txt that points to the correct kernel, it just isn't using
> config.txt for whatever reason. No matter what I change in config.txt, it
> seems to have no effect. I know that the file is changing SOME settings,
> because I can enable the rainbow screen and set boot_delay... But the
> kernel lines aren't correctly loading the kernel.
>
> I have also tried moving the kernel to /kernel.img again, and specifying
> the root filesystem in cmdline.txt. It still kernel panics. I have also
> tried specifying the rootfstype as fat32, and rootwait. Neither of them do
> anything. With only rootfstype=fat32 and root=/dev/mmcblk0p1, I also get a
> register dump and "mmc0: timeout waiting for hardware interrupt". I don't
> believe this is relevant.
>
> Because config.txt is being correctly read for SOME of the settings, that
> implies there is some kind of error with the kernel line.
>
>
> Sent with ProtonMail <https://protonmail.com> Secure Email.
>
> In the config.txt look out for the kernel= line (you could also set it so
> your kernel don't have to be on kernel.img).
> kernel=path/to/kernel
> page, I am lead to believe that someone got Alpine working on the Pi Zero
> W. The Zero W is basically the same as the RPi Zero, with the only
> difference being the addition of wifi and bluetooth. In that page, it
> sounded like most of the author's problems had to do with wifi, and the
> install worked normally otherwise. So, there is the possibility that
> something changed between 3.9.2 and 3.9.4 that broke the install on RPi
> Zero.
>
> The installation process is basically the same for the first part: simply
> create a FAT32 partition (in [1] and [2]) or FAT16 (in [3]) and extract the
> tar to it.
>
> It also has a section: "Create cache folder and add rng-tools packages".
> I do not believe this section is relevant for this problem.
>
> With the exception of perhaps "enable_uart", I don't think any of these
> settings would prevent the Pi Zero from booting.
>
> I have extracted the tar to both FAT32 and FAT16 partitions. I have tried
> setting the boot and lba flags on the partition in gparted. No matter what
> I do, the Pi Zero will not boot.
>
> My theory is that this has something to do with the Pi Zero being unable
> to find the correct kernel. The LED blinks 7 times, pauses, and repeats.
> From here
> I have tried moving "boot/vmlinuz-rpi" up a directory (to what would be
> the / directory if it were booted instead of mounted on my PC) and renaming
> it "kernel.img". When I try booting it that way, I get a kernel panic:
>
> This is an improvement, because at least it gives me something other than
> the rainbow screen (which I enabled in config.txt).
>
> Beyond this, I am not sure what I should do. In theory, the Pi Zero should
> parse the config.txt file and locate the correct kernel to load that way. I
> have tried setting "boot_delay=1" to no avail. I have also theorized that
> it might refuse to boot because I extracted the files from the tar using my
> regular user, and the Pi won't read config.txt unless it is owned by root.
> A recursive chgrp/chown has debunked that theory, because even after that
> it still refuses to boot.
>
> I am going to try moving/renaming initramfs as well as the kernel and see
> if that changes anything. At the very least, it might be a workaround...
> but if I do that, it will refuse to boot on RPi 2/3. In my case I don't
> care, because I only intend to use this on a Zero, but would be unusable as
> a long term fix.
>
>
>
>

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 09:12:29 +0000

I have made a breakthrough. I am not quite sure how yet, but I have gotten Alpine 3.9.2 to boot. I assume that 3.9.4 will also work with this fix, but I have not tested it yet.

Because I needed a reference that is relatively similar to Alpine (Raspbian is not) I downloaded the Arch ARM image for the RPi (I think they use the same one for all models, I am not sure). Much like Raspbian, they have the kernel in / as "kernel.img". However, they also store initramfs in / as "initramfs-linux.img". The contents of config.txt is as follows:

gpu_mem=64
initramfs initramfs-linux.img followkernel

After moving vmlinuz-rpi to ../kernel.img, and moving initramfs-rpi to ../initramfs-linux.img, I copied and pasted the Arch config.txt contents into the Alpine one. Magically, it just booted. I have tested this on the Pi 1/B and Zero.

There are two possibilities: 1) The raspberry pi "bootloader" (or perhaps the kernel) expects there to be files with certain names. This is at least the case for kernel.img. 2) There is some conflict in config.txt that makes everything refuse to boot.

Maybe it is a combination of these things, maybe it is neither. I am going to play around with it later today as see what combinations of filenames and config options result in a bootable system.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, May 22, 2019 8:04 AM, Gee <[email-masked]> wrote:

> For what it's worth, I am seeing this behaviour on a 3B+ as well, but not with raspbian on the same hardware.
>
> I'll try some older builds later to see if we're dealing with a regression or not...
>

---

From: Gee <[email-masked]>
Date: Wed, 22 May 2019 11:19:12 +0100

That's great news! Would love to hear what you end up with :-)

On Wed, 22 May 2019, 10:12 East, <[email-masked]> wrote:

> I have made a breakthrough. I am not quite sure how yet, but I have gotten
> Alpine 3.9.2 to boot. I assume that 3.9.4 will also work with this fix, but
> I have not tested it yet.
>
> Because I needed a reference that is relatively similar to Alpine
> (Raspbian is not) I downloaded the Arch ARM image for the RPi (I think they
> use the same one for all models, I am not sure). Much like Raspbian, they
> have the kernel in / as "kernel.img". However, they also store initramfs in
> / as "initramfs-linux.img". The contents of config.txt is as follows:
>
> gpu_mem=64
> After moving vmlinuz-rpi to ../kernel.img, and moving initramfs-rpi to
> ../initramfs-linux.img, I copied and pasted the Arch config.txt contents
> into the Alpine one. Magically, it just booted. I have tested this on the
> Pi 1/B and Zero.
>
> There are two possibilities: 1) The raspberry pi "bootloader" (or perhaps
> the kernel) expects there to be files with certain names. This is at least
> the case for kernel.img. 2) There is some conflict in config.txt that makes
> everything refuse to boot.
>
> Maybe it is a combination of these things, maybe it is neither. I am going
> to play around with it later today as see what combinations of filenames
> and config options result in a bootable system.
>
>
>> should parse the config.txt file and locate the correct kernel to load that
>> way. I have tried setting "boot_delay=1" to no avail. I have also theorized
>> that it might refuse to boot because I extracted the files from the tar
>> using my regular user, and the Pi won't read config.txt unless it is owned
>> by root. A recursive chgrp/chown has debunked that theory, because even
>> after that it still refuses to boot.
>>

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 21:17:16 +0000

Note: this is a running tally of everything I have tried since the last email. Read everything here like it occurred in chronological order.

After more testing with the Pi 1/B (easier than the Zero, because it saves me the extra step of unplugging the microSD card from the converter) I don't think there is a conflict in config.txt. This is what I have tried, with the following being the total contents of the file:

gpu_mem=64
kernel=boot/vmlinuz-rpi
initramfs boot/initramfs-rpi followkernel

I have also tried the same parameter without followkernel. The issue still seems to be that it isn't parsing the kernel line correctly.

When I remove the kernel line and move kernel to /kernel.img, it kernel panics. This indicates that it isn't correctly interpreting the path for initramfs either.
---
I have also tried this, where I move initramfs to / but keep the same filename:

gpu_mem=64
initramfs initramfs-rpi followkernel

This is the exact same line as the Arch config.txt, with the only difference being the name of initramfs. With this setup, it boots. This tells me that it doesn't expect a specific filename for initramfs, but it can only find it when it is in / and specified in config.txt. The kernel must be in a file called "kernel.img" in /.
---
I have also tried playing with the path to both initramfs and the kernel in config.txt. Normally, it is specified as follows:

kernel=boot/vmlinuz-rpi
initramfs boot/initramfs-rpi

When I change boot/<file> to /boot/<file> (for initramfs only), it kernel panics. The same behavior occurs with the relative path ./boot/<file>. This still have the kernel in /. Only initramfs is in /boot.

What I know for sure is that neither kernel.img or initramfs can be in the boot subdirectory. They both must be in /. As long as initramfs is specified in config.txt, it can have an arbitrary name.
---
After testing with a custom name for the kernel, the following allows Alpine to boot:

gpu_mem=64
kernel=vmlinuz-rpi
initramfs initramfs-rpi followkernel

In this case, I have moved both the kernel and initramfs to / but kept the original names. This means that the kernel can have an arbitrary name as long as it is specified in config.txt and it is not in a subdirectory.
---
I have also tried deleting everything from config.txt except the gpu_mem line, moving kernel to /kernel.img, and moving initramfs to /initramfs-linux.img. This does not boot. This indicates that there is no default location for initramfs, or if there is, then it isn't that filename.
---
My conclusion so far: config.txt can't handle subdirectories for the kernel and initramfs arguments, and may not be able to for any arguments.

This still does not explain how the guy in https://wiki.alpinelinux.org/wiki/Raspberry_Pi_Zero_W_-_Installation got it to boot. Unless he found this problem, fixed it, and didn't include it in the wiki... which seems unlikely. I am going to assume--for the sake of argument--that Alpine 3.9.2 worked with his particular configuration on his Pi Zero W. How he did that, I have no idea.

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 21:28:30 +0000

One other thing: I have tested the Zero with this configuration:

gpu_mem=64
kernel=vmlinuz-rpi

Quoe: "In this case, I have moved both the kernel and initramfs to / but kept the original names. This means that the kernel can have an arbitrary name as long as it is specified in config.txt and it is not in a subdirectory."

I did this to verify that it works on the Zero as well. Note that I tried this with and without followkernel. Either way, it works.

---

From: East <[email-masked]>
Date: Wed, 22 May 2019 23:51:33 +0000

I can confirm that the above fix also works with 3.9.4 on the Pi 1/B and Zero.

As a proper fix, one might move the contents of /boot to / and tweak the existing config as follows:

disable_splash=1
boot_delay=0
gpu_mem=256
gpu_mem_256=64
[pi0]
kernel=vmlinuz-rpi
initramfs initramfs-rpi
[pi1]
kernel=vmlinuz-rpi
initramfs initramfs-rpi
[all]
include usercfg.txt

The only real difference is that boot/ is removed from the kernel and initramfs lines.

On an unrelated note, I have discovered two other minor issues:
1. The full keyboard I have plugged into the Pi Zero via an adapter does not work on boot, and it needs to be unplugged and replugged to work. This might be on my end, because I have a USB to microUSB adapter. This is not an issue on the Pi 1/B
2. There is a second initramfs and kernel for the Pi 2/3, it looks like. The files are named "initramfs-rpi2" and "vmlinuz-rpi2" respectively. I assume this is meant to be used with the models [2B 1.2, 3B, 3B+ and 3A+](https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications), all of which use an ARMv8 64 bit processor. I have no evidence to back this up, just a hunch. I am assuming this kernel has 64 bit and/or multicore support, but that is also a hunch. Either way, there is no conditional that loads this kernel/initramfs in config.txt, so even if Alpine is booting on one of these systems, this kernel is never used. This is not a huge issue, as ARMv7/8 are backward compatible with ARMv6, but there will be no multicore or 64bit support on platforms that can actually use it. To fix this (for multicore support), simply add https://www.raspberrypi.org/documentation/configuration/config-txt/conditional.md conditionals to config.txt and point them at the right kernel/initramfs. Not sure how to do it for 64bit platforms though, as the Pi 2B is ARMv7 but not 64bit, and is included under the [pi2] conditional. I am not even sure that Alpine maintains 64bit ARM versions, so this may not be relevant.

---

From: East <[email-masked]>
Date: Thu, 23 May 2019 04:22:58 +0000

I have moved on from this issue, and am currently working around it. It would be nice to know what actually caused it though, because the workaround interferes with bind mounting the boot folder on partition one. This is from the walkthrough in https://wiki.alpinelinux.org/wiki/Raspberry_Pi#Traditional_disk-based_.28sys.29_installation. I suppose one could bind mount the entire partition or something, but that is just kludgy.

I am also unsure if Alpine will put the new kernel in the first place after apk upgrades. Will it put it in /boot, or will it properly put it under "/"?

---

From: East <[email-masked]>
Date: Tue, 21 May 2019 01:08:14 +0000

As an aside, I am still unclear on whether the RPi Zero supports the armhf image. I am assuming it does, because again, the Pi Zero and Zero W have the same processor... But hard float support is something that the processor used in the original Pi and Pi Zero does not have. As I understand it, that is why Raspbian exists: it was originally a build of armhf that was modified to work with the processor in the original Pi 1/B. So, although [1] suggests otherwise, should I even be expecting Alpine to boot on this hardware?