Kernel Panic on VIA Nano (x86_64) for 3.15 and above
Just FYI, for the archives...
I get a kernel panic on VIA Nano (x86_64) motherboards from Alpine
v3.15 and above. I know 3.12 *works*, but haven't checked v3.13 or
v3.14.
It was a "well that stinks" moment when my internet facing firewall
refused to work, and just reverted. Had a few moments to look at it,
and here's what I know.
On 3.12, everything boots, and things continue:
* Mounting /dev/mqueue ... [ ok ]
* Mounting modloop ... * Verifying modloop
[ ok ]
* Mounting security filesystem ... [ ok ]
on 3.15+:
* Mounting /dev/mqueue ... [ ok ]
* Mounting modloop ... * Verifying modloop
[ 15.024965] general protection fault, maybe for address 0x18: 0000
[#1] SMP PTI
[ 15.024984] CPU: 0 PID: 1227 Comm: openssl Not tainted 5.15.41-0-lts
#1-Alpine
[ 15.024998] Hardware name: VIA Technologies Ltd. VX800 /VX800 , BIOS
6.00 PG 11/09/2009
[ 15.025004] RIP: 0010:native_read_pmc+0x4/0x50
[ 15.025028] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89 c7
c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41 54 89
f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c 31 d2 89
[ 15.025038] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
[ 15.025050] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
0000000040000001
[ 15.025057] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
0000000040000001
[ 15.025064] RBP: ffff9ad181224950 R08: 000000000000030a R09:
0000000000000000
[ 15.025070] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9ad181224b30
[ 15.025077] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
ffff9ad181224950
[ 15.025084] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
knlGS:0000000000000000
[ 15.025093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 15.025100] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
00000000000006f0
[ 15.025109] Call Trace:
[ 15.025116] <TASK>
[ 15.025123] x86_perf_event_update+0x4b/0xc0
[ 15.025151] zhaoxin_pmu_handle_irq+0x1ac/0x280
[ 15.025202] perf_event_nmi_handler+0x28/0x50
[ 15.025216] nmi_handle+0x58/0x100
[ 15.025235] default_do_nmi+0x40/0x130
[ 15.025256] exc_nmi+0x168/0x190
[ 15.025268] asm_exc_nmi+0x8e/0xd7
[ 15.025289] RIP: 0033:0x7f8f0250ed31
[ 15.025299] Code: 24 38 66 0f 72 f3 02 31 c6 89 d7 66 41 0f 72 d0 1e
c1 c2 05 01 f1 31 c7 c1 cd 07 66 41 0f eb d8 01 d1 03 5c 24 3c 31 ef 89
ce <c1> c1 05 01 fb 31 ee c1 ca 07 01 cb 03 04 24 31 d6 89 df c1 c3 05
[ 15.025307] RSP: 002b:00007ffe5bdce5c0 EFLAGS: 00000206
[ 15.025315] RAX: 000000008997d0af RBX: 0000000057d97dce RCX:
00000000cc3c5aa5
[ 15.025321] RDX: 00000000275408e9 RSI: 00000000cc3c5aa5 RDI:
000000000d2f8ac6
[ 15.025328] RBP: 00000000cd82fa2e R08: 00007f8f02371030 R09:
00007f8f0235be20
[ 15.025334] R10: 00007f8f0235d1a0 R11: 00007ffe5bdce648 R12:
00007f8f0235b1a0
[ 15.025340] R13: 00007f8f0235b1a0 R14: 00007f8f02511780 R15:
00007ffe5bdcfddc
[ 15.025356] </TASK>
[ 15.025360] Modules linked in: nls_utf8 nls_cp437 vfat fat
hid_generic usbhid hid uas ata_generic pata_acpi pata_via libata
uhci_hcd ehci_pci ehci_hcd simpledrm drm_kms_helper cfbfillrect
syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea cec
drm i2c_core drm_panel_orientation_quirks agpgart usb_storage usbcore
usb_common sd_mod t10_pi scsi_mod squashfs loop
[ 15.262695] ---[ end trace 5c8c48eee1b7891d ]---
[ 15.262700] RIP: 0010:native_read_pmc+0x4/0x50
[ 15.262712] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89 c7
c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41 54 89
f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c 31 d2 89
[ 15.262720] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
[ 15.262729] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
0000000040000001
[ 15.262735] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
0000000040000001
[ 15.262741] RBP: ffff9ad181224950 R08: 000000000000030a R09:
0000000000000000
[ 15.262748] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9ad181224b30
[ 15.262754] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
ffff9ad181224950
[ 15.262760] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
knlGS:0000000000000000
[ 15.262769] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 15.262776] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
00000000000006f0
[ 15.262784] Kernel panic - not syncing: Fatal exception in interrupt
[ 15.262795] Kernel Offset: 0x36000000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 15.369416] ---[ end Kernel panic - not syncing: Fatal exception in
interrupt ]---
Looks like something changed in the "verifying modloop" step.
Don't have time to dig deeper. The VIA Nano chips are old. The fast
way to solve this is to upgrade to some other chipset.
Just wanted to report, for the records.
On Wed, Jun 08, 2022 at 05:56:20PM -0400, Nathan Angelacos wrote:
> Just FYI, for the archives...
>
> I get a kernel panic on VIA Nano (x86_64) motherboards from Alpine
> v3.15 and above. I know 3.12 *works*, but haven't checked v3.13 or
> v3.14.
>
> It was a "well that stinks" moment when my internet facing firewall
> refused to work, and just reverted. Had a few moments to look at it,
> and here's what I know.
>
> On 3.12, everything boots, and things continue:
>
> * Mounting /dev/mqueue ... [ ok ]
> * Mounting modloop ... * Verifying modloop
> [ ok ]
> * Mounting security filesystem ... [ ok ]
>
> on 3.15+:
>
>
> * Mounting /dev/mqueue ... [ ok ]
> * Mounting modloop ... * Verifying modloop
> [ 15.024965] general protection fault, maybe for address 0x18: 0000
> [#1] SMP PTI
> [ 15.024984] CPU: 0 PID: 1227 Comm: openssl Not tainted 5.15.41-0-lts
> #1-Alpine
> [ 15.024998] Hardware name: VIA Technologies Ltd. VX800 /VX800 , BIOS
> 6.00 PG 11/09/2009
> [ 15.025004] RIP: 0010:native_read_pmc+0x4/0x50
> [ 15.025028] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89 c7
> c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41 54 89
> f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c 31 d2 89
> [ 15.025038] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
> [ 15.025050] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
> 0000000040000001
> [ 15.025057] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 0000000040000001
> [ 15.025064] RBP: ffff9ad181224950 R08: 000000000000030a R09:
> 0000000000000000
> [ 15.025070] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9ad181224b30
> [ 15.025077] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
> ffff9ad181224950
> [ 15.025084] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
> knlGS:0000000000000000
> [ 15.025093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 15.025100] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
> 00000000000006f0
> [ 15.025109] Call Trace:
> [ 15.025116] <TASK>
> [ 15.025123] x86_perf_event_update+0x4b/0xc0
> [ 15.025151] zhaoxin_pmu_handle_irq+0x1ac/0x280
> [ 15.025202] perf_event_nmi_handler+0x28/0x50
> [ 15.025216] nmi_handle+0x58/0x100
> [ 15.025235] default_do_nmi+0x40/0x130
> [ 15.025256] exc_nmi+0x168/0x190
> [ 15.025268] asm_exc_nmi+0x8e/0xd7
> [ 15.025289] RIP: 0033:0x7f8f0250ed31
> [ 15.025299] Code: 24 38 66 0f 72 f3 02 31 c6 89 d7 66 41 0f 72 d0 1e
> c1 c2 05 01 f1 31 c7 c1 cd 07 66 41 0f eb d8 01 d1 03 5c 24 3c 31 ef 89
> ce <c1> c1 05 01 fb 31 ee c1 ca 07 01 cb 03 04 24 31 d6 89 df c1 c3 05
> [ 15.025307] RSP: 002b:00007ffe5bdce5c0 EFLAGS: 00000206
> [ 15.025315] RAX: 000000008997d0af RBX: 0000000057d97dce RCX:
> 00000000cc3c5aa5
> [ 15.025321] RDX: 00000000275408e9 RSI: 00000000cc3c5aa5 RDI:
> 000000000d2f8ac6
> [ 15.025328] RBP: 00000000cd82fa2e R08: 00007f8f02371030 R09:
> 00007f8f0235be20
> [ 15.025334] R10: 00007f8f0235d1a0 R11: 00007ffe5bdce648 R12:
> 00007f8f0235b1a0
> [ 15.025340] R13: 00007f8f0235b1a0 R14: 00007f8f02511780 R15:
> 00007ffe5bdcfddc
> [ 15.025356] </TASK>
> [ 15.025360] Modules linked in: nls_utf8 nls_cp437 vfat fat
> hid_generic usbhid hid uas ata_generic pata_acpi pata_via libata
> uhci_hcd ehci_pci ehci_hcd simpledrm drm_kms_helper cfbfillrect
> syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea cec
> drm i2c_core drm_panel_orientation_quirks agpgart usb_storage usbcore
> usb_common sd_mod t10_pi scsi_mod squashfs loop
> [ 15.262695] ---[ end trace 5c8c48eee1b7891d ]---
> [ 15.262700] RIP: 0010:native_read_pmc+0x4/0x50
> [ 15.262712] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89 c7
> c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41 54 89
> f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c 31 d2 89
> [ 15.262720] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
> [ 15.262729] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
> 0000000040000001
> [ 15.262735] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 0000000040000001
> [ 15.262741] RBP: ffff9ad181224950 R08: 000000000000030a R09:
> 0000000000000000
> [ 15.262748] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9ad181224b30
> [ 15.262754] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
> ffff9ad181224950
> [ 15.262760] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
> knlGS:0000000000000000
> [ 15.262769] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 15.262776] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
> 00000000000006f0
> [ 15.262784] Kernel panic - not syncing: Fatal exception in interrupt
> [ 15.262795] Kernel Offset: 0x36000000 from 0xffffffff81000000
> (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 15.369416] ---[ end Kernel panic - not syncing: Fatal exception in
> interrupt ]---
>
>
> Looks like something changed in the "verifying modloop" step.
>
> Don't have time to dig deeper. The VIA Nano chips are old. The fast
> way to solve this is to upgrade to some other chipset.
>
> Just wanted to report, for the records.
Hello Nathan,
That message comes from the [modloop services][0]. It verifies with
`openssl dgst` the contents of the modloop, so it might be any change to
openssl or it's dependencies.
If it's indeed the `openssl dgst` that is failing, you could bypass that
by adding `modloop_verify=no` to the kernel cmdline parameters.
Kevin
[0]:https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/main/openrc/modloop.initd#L56
Hi
Sounds like:
https://bugzilla.kernel.org/show_bug.cgi?id=212389
Seems there's a patch, but it's not merged :/ Should probably ping on
kernel folks on this. Possible cherry-pick the patch if needed? Nicely
there is a boot time cmdline workaround:
"initcall_blacklist=init_hw_perf_events"
Per https://gitlab.alpinelinux.org/alpine/tsc/-/issues/20 i586 should
be the baseline for most things. So I'd assume VIA Nano should work as
it's full i586 (instead of those cheap something in between i486 and
i586 things).
Timo
On Wed, 08 Jun 2022 17:56:20 -0400
Nathan Angelacos <nangel@alpinelinux.org > wrote:
> Just FYI, for the archives...
>
> I get a kernel panic on VIA Nano (x86_64) motherboards from Alpine
> v3.15 and above. I know 3.12 *works*, but haven't checked v3.13 or
> v3.14.
>
> It was a "well that stinks" moment when my internet facing firewall
> refused to work, and just reverted. Had a few moments to look at it,
> and here's what I know.
>
> On 3.12, everything boots, and things continue:
>
> * Mounting /dev/mqueue ... [ ok ]
> * Mounting modloop ... * Verifying modloop
> [ ok ]
> * Mounting security filesystem ... [ ok ]
>
> on 3.15+:
>
>
> * Mounting /dev/mqueue ... [ ok ]
> * Mounting modloop ... * Verifying modloop
> [ 15.024965] general protection fault, maybe for address 0x18: 0000
> [#1] SMP PTI
> [ 15.024984] CPU: 0 PID: 1227 Comm: openssl Not tainted
> 5.15.41-0-lts #1-Alpine
> [ 15.024998] Hardware name: VIA Technologies Ltd. VX800 /VX800 ,
> BIOS 6.00 PG 11/09/2009
> [ 15.025004] RIP: 0010:native_read_pmc+0x4/0x50
> [ 15.025028] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89
> c7 c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41
> 54 89 f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c
> 31 d2 89 [ 15.025038] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
> [ 15.025050] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
> 0000000040000001
> [ 15.025057] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 0000000040000001
> [ 15.025064] RBP: ffff9ad181224950 R08: 000000000000030a R09:
> 0000000000000000
> [ 15.025070] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9ad181224b30
> [ 15.025077] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
> ffff9ad181224950
> [ 15.025084] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
> knlGS:0000000000000000
> [ 15.025093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 15.025100] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
> 00000000000006f0
> [ 15.025109] Call Trace:
> [ 15.025116] <TASK>
> [ 15.025123] x86_perf_event_update+0x4b/0xc0
> [ 15.025151] zhaoxin_pmu_handle_irq+0x1ac/0x280
> [ 15.025202] perf_event_nmi_handler+0x28/0x50
> [ 15.025216] nmi_handle+0x58/0x100
> [ 15.025235] default_do_nmi+0x40/0x130
> [ 15.025256] exc_nmi+0x168/0x190
> [ 15.025268] asm_exc_nmi+0x8e/0xd7
> [ 15.025289] RIP: 0033:0x7f8f0250ed31
> [ 15.025299] Code: 24 38 66 0f 72 f3 02 31 c6 89 d7 66 41 0f 72 d0
> 1e c1 c2 05 01 f1 31 c7 c1 cd 07 66 41 0f eb d8 01 d1 03 5c 24 3c 31
> ef 89 ce <c1> c1 05 01 fb 31 ee c1 ca 07 01 cb 03 04 24 31 d6 89 df
> c1 c3 05 [ 15.025307] RSP: 002b:00007ffe5bdce5c0 EFLAGS: 00000206
> [ 15.025315] RAX: 000000008997d0af RBX: 0000000057d97dce RCX:
> 00000000cc3c5aa5
> [ 15.025321] RDX: 00000000275408e9 RSI: 00000000cc3c5aa5 RDI:
> 000000000d2f8ac6
> [ 15.025328] RBP: 00000000cd82fa2e R08: 00007f8f02371030 R09:
> 00007f8f0235be20
> [ 15.025334] R10: 00007f8f0235d1a0 R11: 00007ffe5bdce648 R12:
> 00007f8f0235b1a0
> [ 15.025340] R13: 00007f8f0235b1a0 R14: 00007f8f02511780 R15:
> 00007ffe5bdcfddc
> [ 15.025356] </TASK>
> [ 15.025360] Modules linked in: nls_utf8 nls_cp437 vfat fat
> hid_generic usbhid hid uas ata_generic pata_acpi pata_via libata
> uhci_hcd ehci_pci ehci_hcd simpledrm drm_kms_helper cfbfillrect
> syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea
> cec drm i2c_core drm_panel_orientation_quirks agpgart usb_storage
> usbcore usb_common sd_mod t10_pi scsi_mod squashfs loop
> [ 15.262695] ---[ end trace 5c8c48eee1b7891d ]---
> [ 15.262700] RIP: 0010:native_read_pmc+0x4/0x50
> [ 15.262712] Code: 89 df e8 3f 14 01 00 48 89 03 5b 31 c0 89 c6 89
> c7 c3 48 89 f0 48 89 03 5b 31 c0 89 c6 89 c7 c3 66 0f 1f 44 00 00 41
> 54 89 f9 <0f> 33 66 90 48 c1 e2 20 48 09 c2 49 89 d4 4c 89 e0 41 5c
> 31 d2 89 [ 15.262720] RSP: 0000:ffffaeaf0063bc88 EFLAGS: 00010046
> [ 15.262729] RAX: 0000000000000018 RBX: fffffffdad714620 RCX:
> 0000000040000001
> [ 15.262735] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 0000000040000001
> [ 15.262741] RBP: ffff9ad181224950 R08: 000000000000030a R09:
> 0000000000000000
> [ 15.262748] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9ad181224b30
> [ 15.262754] R13: 0000000000000018 R14: ffff9ad1b5a15820 R15:
> ffff9ad181224950
> [ 15.262760] FS: 00007f8f02720b48(0000) GS:ffff9ad1b5a00000(0000)
> knlGS:0000000000000000
> [ 15.262769] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 15.262776] CR2: 00007f8f0235c000 CR3: 0000000002728000 CR4:
> 00000000000006f0
> [ 15.262784] Kernel panic - not syncing: Fatal exception in
> interrupt [ 15.262795] Kernel Offset: 0x36000000 from
> 0xffffffff81000000 (relocation range:
> 0xffffffff80000000-0xffffffffbfffffff) [ 15.369416] ---[ end Kernel
> panic - not syncing: Fatal exception in interrupt ]---
>
>
> Looks like something changed in the "verifying modloop" step.
>
> Don't have time to dig deeper. The VIA Nano chips are old. The fast
> way to solve this is to upgrade to some other chipset.
>
> Just wanted to report, for the records.
On Thu, 2022-06-09 at 21:48 +0300, Timo Teras wrote:
> Hi
>
> Sounds like:
> https://bugzilla.kernel.org/show_bug.cgi?id=212389
>
> Seems there's a patch, but it's not merged :/ Should probably ping on
> kernel folks on this. Possible cherry-pick the patch if needed?
> Nicely
> there is a boot time cmdline workaround:
> "initcall_blacklist=init_hw_perf_events"
>
> Per https://gitlab.alpinelinux.org/alpine/tsc/-/issues/20 i586 should
> be the baseline for most things. So I'd assume VIA Nano should work
> as
> it's full i586 (instead of those cheap something in between i486 and
> i586 things).
>
> Timo
>
Thanks Timo,
That worked. (Although I can't believe you top-posted! :D)