Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by gbr-app-1.alpinelinux.org (Postfix) with ESMTPS id 57ACA225B2E for <~alpine/users@lists.alpinelinux.org>; Sat, 8 Feb 2025 08:25:45 +0000 (UTC) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-38dd2f309deso23873f8f.0 for <~alpine/users@lists.alpinelinux.org>; Sat, 08 Feb 2025 00:25:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739003142; x=1739607942; darn=lists.alpinelinux.org; h=content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=duQNQf0B7JNmt2GKmBgRpuaKOLm2bLtlQQOCr4Dx+f8=; b=SlzPpboGy1TgnpRwxG/iuDaYPd/yPBSg81I+4puBdf42pDczcqPJayg2+JWXdsNYyr GUogmevpboCegD4R51nwPAn9Q0WZFktpUbEoPmvg7a96FG2IxIw4SoMBlvCdNZ1JRyPO vk8wXzVrbP1UiKIErn0HqhSr5lTW1EwS/XqfO1hiiV3XFxUu8voF1o6xSzmAsms1rfIZ M5CaltYd/orpGsVxR+5CtIQEg/MF+wfaTvmCmA8R7BmdWRGONnF0M2mAdNaOnmSLqjWI vAEoTrpJQ3a6lqYSHzXGON/S9llvSbrLIPoBdqjCF7CisYFvZiQu96XqRHW3Ces9CWVL RjkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739003142; x=1739607942; h=content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=duQNQf0B7JNmt2GKmBgRpuaKOLm2bLtlQQOCr4Dx+f8=; b=whOlBmepl82qf1n8JrUrJkAM3Lqd9jIQ8qyMZkn1WFTe0F9yjU6HnATY9GrOsnZ4vo yrbt7flmWE6j4WGgcxYkzoaeH+GbsIg0joeP8YrbcytEzuzC/4v58WxYIZEku5Exf1oB AN5nH+OhKR2rHEk3NhAFgh4hq+zFj4RVAKVk9xUbDxtAHFiArA1JS1piPmvTQuuMOXkR TcTP9pvMcjY8cQ5lab6qrxI3aWU6cpjYzgqWtXkdU4hyKKbpJ8rOLYJFGX5AmHwiqTwF rQjWFbBQjH0xk1POisEIwRxQfgeLKSSTZiVnr+dvZhdIrfpBACI8Jw6JocfTqU3MlJYn JHeg== X-Gm-Message-State: AOJu0YyBduHTWFD7Ea1NmmWgupTPdKUnQ0hddPKeZ2xOxmPSXWlC/Vpa n3bfyy78ZoO0+pLVQr3GohlVa69WOQsffMxyvPLT8PBlYLCdHcGIjjm8tO2L X-Gm-Gg: ASbGnctTHpGB90C7t6BZE89me+U41eadW2TzCtMVdaM8+AnOirR9mCIATaaQj0gkfqN nsyfP+ZEUBq62yQ/nt+kZP0Jjue+XBPGEXGGVa7sdeBtp1gAtKzKlEu0wHXegkpOqiGNjrDDy9l 24N6NQyM7kLWMyIKcUt0jAxdIjnzvRoqrTMi0tZPdf2HSQUMuCuf8auUzZRHLvxWlhyqYiidhXF gqWJQl6OKuXpHs+JtoeSZ/u29t9N4i/jlG5Rcxjwr3FEf1m4Ny1qJw0540CFO/DAxdcWdtfbXd3 nJ43wKYi X-Google-Smtp-Source: AGHT+IEhQvk6jUV5YK6kgGGRrtIcWFSvJkOfWx2asZ7bDFOJ9cJW4PFvY2WgMzQ6t+T57kQrKo5ubw== X-Received: by 2002:a05:600c:1553:b0:434:ff85:dd77 with SMTP id 5b1f17b1804b1-439249bd3e8mr20714515e9.3.1739003141882; Sat, 08 Feb 2025 00:25:41 -0800 (PST) Received: from cloud.fritzweb.at (cloud.fritzweb.at. [2a00:63c1:e:50::2]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38dce18d6f9sm3043224f8f.29.2025.02.08.00.25.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Feb 2025 00:25:41 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by cloud.fritzweb.at (Postfix) with ESMTP id 5949C800885; Sat, 8 Feb 2025 09:25:40 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at cloud.fritzweb.at Received: from cloud.fritzweb.at ([IPv6:::1]) by localhost (cloud.fritzweb.at [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id fsU1EOCkK8An; Sat, 8 Feb 2025 09:25:38 +0100 (CET) Received: from ntb-norbert.localnet (unknown [62.93.118.77]) by cloud.fritzweb.at (Postfix) with ESMTPSA id 90AE68008AA; Sat, 8 Feb 2025 09:25:37 +0100 (CET) From: MBB To: "~alpine/users@lists.alpinelinux.org" <~alpine/users@lists.alpinelinux.org>, Sertonix , Kevin Garman Subject: Re: Diskless systems do not boot with latest apk versions from cache Date: Sat, 08 Feb 2025 09:25:37 +0100 Message-ID: <23824424.6Emhk5qWAg@ntb-norbert> Organization: Private In-Reply-To: References: <9410103.CDJkKcVGEf@ntb-norbert> <2408933.iZASKD2KPV@ntb-norbert> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="nextPart2641315.Lt9SDvczpP" Content-Transfer-Encoding: 7Bit This is a multi-part message in MIME format. --nextPart2641315.Lt9SDvczpP Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Am Montag, 3. Februar 2025, 16:07:18 Mitteleurop=C3=A4ische Normalzeit schr= ieb Kevin Garman: > I have long been struggling with similar issues on my diskless installs. = I run.. >=20 > apk cache -v sync >=20 > ...and sometimes it breaks my system(s) and sometimes it doesn't. I've r= ead where cache sync should be used with care, but I believe I'm using it a= s intended. It seems like it should be straight forward as all it's suppos= e to do (as I understand it) is make the cache match what the system needs = during boot. It seems like running cache sync should be the solution to pr= oblem it's causing! >=20 > I've been working to figure out a simple/repeatable set of steps to repro= duce the problem, but so far I haven't been able to get there. >=20 >=20 >=20 >=20 > ________________________________ > From: ~alpine/users <~alpine/users@lists.alpinelinux.org> on behalf of MB= B > Sent: Sunday, February 2, 2025 09:16 > To: ~alpine/users@lists.alpinelinux.org <~alpine/users@lists.alpinelinux.= org>; Sertonix > Subject: Re: Diskless systems do not boot with latest apk versions from c= ache >=20 >=20 > Am Sonntag, 2. Februar 2025, 09:11:22 Mitteleurop=C3=A4ische Normalzeit s= chrieb MBB: > > Am Samstag, 1. Februar 2025, 14:39:34 Mitteleurop=C3=A4ische Normalzeit= schrieb MBB: > > > Am Samstag, 1. Februar 2025, 13:12:29 Mitteleurop=C3=A4ische Normalze= it schrieb Sertonix: > > > > On Sat Feb 1, 2025 at 9:24 AM CET, MBB wrote: > > > > > Hi all! > > > > > > > > > > I have 3 Raspberry Pis, different PI versions and different Alpin= e versions (3.18.9, 3.19.1, 3.19.4). > > > > > > > > > > All have the same issue. > > > > > > > > > > I'm doing: > > > > > apk update; apk upgrade > > > > > > > > > > Checking the version of an upgraded binary before reboot, e.g. na= med, looks ok. > > > > > > > > > > After reboot, it loads an old version even though a newer package= is available in cache. > > > > > > > > > > gate [/etc/apk]$ ls -al cache > > > > > lrwxrwxrwx 1 root root 22 Feb 1 08:50 cache ->= /media/mmcblk0p1/cache > > > > > > > > > > gate [/etc/apk/cache]$ ls -altr bind-9* > > > > > -rwxr-xr-x 1 root root 207297 Mar 15 2024 bind-9.1= 8.24-r1.5aca84cc.apk > > > > > -rwxr-xr-x 1 root root 208996 Dec 17 16:37 bind-9.1= 8.32-r0.1254446d.apk > > > > > -rwxr-xr-x 1 root root 209078 Jan 29 19:44 bind-9.1= 8.33-r0.b6697cb2.apk > > > > > > > > > > gate [/etc/apk/cache]$ named -V > > > > > BIND 9.18.32 (Extended Support Version) > > > > > running on Linux armv7l 6.6.14-0-rpi #1-Alpine SMP Fri Jan 26 13:= 40:47 UTC 2024 > > > > > > > > > > Is there something I'm doing wrong? > > > > > > > > It looks correct what you are doing. Could you try if running this > > > > commands returns any error? > > > > > > > > $ apk --simulate add bind=3D9.18.33-r0 --no-network > > > > > > No errors with this command. > > > > > > *gate [*~*]$ *doas apk --simulate add bind=3D9.18.33-r0 --no-network > > > doas (norbert@gate) password: > > > (1/5) Upgrading bind-libs (9.18.32-r0 -> 9.18.33-r0) > > > (2/5) Upgrading bind-tools (9.18.32-r0 -> 9.18.33-r0) > > > (3/5) Upgrading bind-dnssec-root (9.18.32-r0 -> 9.18.33-r0) > > > (4/5) Upgrading bind (9.18.32-r0 -> 9.18.33-r0) > > > (5/5) Upgrading bind-openrc (9.18.32-r0 -> 9.18.33-r0) > > > OK: 157 MiB in 215 packages > > > > I removed all outdated bind packages from the cache with the result, th= at bind is no longer available after reboot. > > After moving the bind*9.18.32-r0*.apk files back into the cache directo= ry, the service gets restored and starts up without issues after reboot. > > > > It seems that something's pointing to a specific version. > > >=20 > `apk cache sync` breaks the entire system. >=20 > This cleans the cache from outdated APKINDEX* and *.apk files. > But after reboot nothing is loaded and the boot process ends with a `(non= e) local:` prompt. >=20 > Restoring the old cache directory, brings the system to life again. >=20 > Same issue when I manually delete only the APKINDEX* files, and update th= e local DB with `apk update` to download fresh APKINDEX. >=20 > Restoring the old APKINDEX* files from a backup fixes this. >=20 I fixed it for all 3 of my systems. Key is to update the /media/mmcblk0p1/boot/initramfs-rpi I created a new one following this guide: https://wiki.alpinelinux.org/wiki= /Initramfs_init[1] I activated these features to get an initramfs which matches the original=20 features=3D"base dhcp keymap kms mmc squashfs usb" When I compared the old an new initramfs, only these binaries showed differ= ences: =46iles initramfs/bin/busybox and mkinitfs/bin/busybox differ=20 =46iles initramfs/lib/libapk.so.2.14.0 and mkinitfs/lib/libapk.so.2.14.0 di= ffer=20 =46iles initramfs/lib/libcrypto.so.3 and mkinitfs/lib/libcrypto.so.3 differ= =20 =46iles initramfs/lib/libssl.so.3 and mkinitfs/lib/libssl.so.3 differ=20 =46iles initramfs/sbin/apk and mkinitfs/sbin/apk differ Because creating a new initramfs is part of the update-kernel script, updat= ing the kernel fixes the issue as well. https://wiki.alpinelinux.org/wiki/Diskless_Mode#Upgrading_a_Diskless_System= [2] > > > > > > > > > BR > > > > > Norbert > > > > > > > > > > > > > > > > >=20 >=20 >=20 >=20 =2D------- [1] https://wiki.alpinelinux.org/wiki/Initramfs_init [2] https://wiki.alpinelinux.org/wiki/Diskless_Mode#Upgrading_a_Diskless_Sy= stem --nextPart2641315.Lt9SDvczpP Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8"

Am Montag, 3. Februar 2025, 16:07:18 Mitteleurop=C3=A4ische Normalzeit s= chrieb Kevin Garman:

>= ; I have long been struggling with similar issues on my diskless installs.= =C2=A0 I run..

>= ;

>= ;=C2=A0 apk cache -v sync

>= ;

>= ; ...and sometimes it breaks my system(s) and sometimes it doesn't.=C2=A0 I= 've read where cache sync should be used with care, but I believe I'm using= it as intended.=C2=A0 It seems like it should be straight forward as all i= t's suppose to do (as I understand it) is make the cache match what the sys= tem needs during boot.=C2=A0 It seems like running cache sync should be the= solution to problem it's causing!

>= ;

>= ; I've been working to figure out a simple/repeatable set of steps to repro= duce the problem, but so far I haven't been able to get there.

>= ;

>= ;

>= ;

>= ;

>= ; ________________________________

>= ; From: ~alpine/users <~alpine/users@lists.alpinelinux.org> on behalf= of MBB <mbbert@gmail.com>

>= ; Sent: Sunday, February 2, 2025 09:16

>= ; To: ~alpine/users@lists.alpinelinux.org <~alpine/users@lists.alpinelin= ux.org>; Sertonix <sertonix@posteo.net>

>= ; Subject: Re: Diskless systems do not boot with latest apk versions from c= ache

>= ;

>= ;

>= ; Am Sonntag, 2. Februar 2025, 09:11:22 Mitteleurop=C3=A4ische Normalzeit s= chrieb MBB:

>= ; > Am Samstag, 1. Februar 2025, 14:39:34 Mitteleurop=C3=A4ische Normalz= eit schrieb MBB:

>= ; > > Am Samstag, 1. Februar 2025, 13:12:29 Mitteleurop=C3=A4ische No= rmalzeit schrieb Sertonix:

>= ; > > > On Sat Feb 1, 2025 at 9:24 AM CET, MBB wrote:

>= ; > > > > Hi all!

>= ; > > > >

>= ; > > > > I have 3 Raspberry Pis, different PI versions and dif= ferent Alpine versions (3.18.9, 3.19.1, 3.19.4).

>= ; > > > >

>= ; > > > > All have the same issue.

>= ; > > > >

>= ; > > > > I'm doing:

>= ; > > > > apk update; apk upgrade

>= ; > > > >

>= ; > > > > Checking the version of an upgraded binary before reb= oot, e.g. named, looks ok.

>= ; > > > >

>= ; > > > > After reboot, it loads an old version even though a n= ewer package is available in cache.

>= ; > > > >

>= ; > > > > gate [/etc/apk]$ ls -al cache

>= ; > > > > lrwxrwxrwx=C2=A0=C2=A0=C2=A0 1 root=C2=A0=C2=A0=C2=A0= =C2=A0 root=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 22 Feb=C2=A0 1 08:50 cache -> /media/mmcblk0p1/cache

>= ; > > > >

>= ; > > > > gate [/etc/apk/cache]$ ls -altr bind-9*

>= ; > > > > -rwxr-xr-x=C2=A0=C2=A0=C2=A0 1 root=C2=A0=C2=A0=C2=A0= =C2=A0 root=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 207297 Mar 15=C2=A0 2= 024 bind-9.18.24-r1.5aca84cc.apk

>= ; > > > > -rwxr-xr-x=C2=A0=C2=A0=C2=A0 1 root=C2=A0=C2=A0=C2=A0= =C2=A0 root=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 208996 Dec 17 16:37 b= ind-9.18.32-r0.1254446d.apk

>= ; > > > > -rwxr-xr-x=C2=A0=C2=A0=C2=A0 1 root=C2=A0=C2=A0=C2=A0= =C2=A0 root=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 209078 Jan 29 19:44 b= ind-9.18.33-r0.b6697cb2.apk

>= ; > > > >

>= ; > > > > gate [/etc/apk/cache]$ named -V

>= ; > > > > BIND 9.18.32 (Extended Support Version) <id:d1f139= 2>

>= ; > > > > running on Linux armv7l 6.6.14-0-rpi #1-Alpine SMP Fr= i Jan 26 13:40:47 UTC 2024

>= ; > > > >

>= ; > > > > Is there something I'm doing wrong?

>= ; > > >

>= ; > > > It looks correct what you are doing. Could you try if runn= ing this

>= ; > > > commands returns any error?

>= ; > > >

>= ; > > > $ apk --simulate add bind=3D9.18.33-r0 --no-network

>= ; > >

>= ; > > No errors with this command.

>= ; > >

>= ; > > *gate [*~*]$ *doas apk --simulate add bind=3D9.18.33-r0 --no-ne= twork

>= ; > > doas (norbert@gate) password:

>= ; > > (1/5) Upgrading bind-libs (9.18.32-r0 -> 9.18.33-r0)

>= ; > > (2/5) Upgrading bind-tools (9.18.32-r0 -> 9.18.33-r0)

>= ; > > (3/5) Upgrading bind-dnssec-root (9.18.32-r0 -> 9.18.33-r0)<= /p>

>= ; > > (4/5) Upgrading bind (9.18.32-r0 -> 9.18.33-r0)

>= ; > > (5/5) Upgrading bind-openrc (9.18.32-r0 -> 9.18.33-r0)

>= ; > > OK: 157 MiB in 215 packages

>= ; >

>= ; > I removed all outdated bind packages from the cache with the result,= that bind is no longer available after reboot.

>= ; > After moving the bind*9.18.32-r0*.apk files back into the cache dire= ctory, the service gets restored and starts up without issues after reboot.=

>= ; >

>= ; > It seems that something's pointing to a specific version.

>= ; >

>= ;

>= ; `apk cache sync` breaks the entire system.

>= ;

>= ; This cleans the cache from outdated APKINDEX* and *.apk files.

>= ; But after reboot nothing is loaded and the boot process ends with a `(non= e) local:` prompt.

>= ;

>= ; Restoring the old cache directory, brings the system to life again.

>= ;

>= ; Same issue when I manually delete only the APKINDEX* files, and update th= e local DB with `apk update` to download fresh APKINDEX.

>= ;

>= ; Restoring the old APKINDEX* files from a backup fixes this.

>= ;

I fixed it for all 3 of my systems.

Key i= s to update the /media/mmcblk0p1/boot/initramfs-rpi


I created a new one following this guide: https://wiki.alpinelinux.org/wiki/Initr= amfs_init

I act= ivated these features to get an initramfs which matches the original=C2=A0<= /span>


features=3D"base dhcp keymap kms mmc squashfs usb"

 

Whe= n I compared the old an new initramfs, only these binaries showed differenc= es:


Files initramfs/bin/busybox and mkinitfs/bin/busybox differ
Files in= itramfs/lib/libapk.so.2.14.0 and mkinitfs/lib/libapk.so.2.14.0 differ
Files initramfs/lib/libcrypto.so.3 and mkinitfs/lib/libcrypto.so.3 differ =
Files initramfs/lib/libssl.so.3 and mkinitfs/lib/libssl.so.3 differ <= br />Files initramfs/sbin/apk and mkinitfs/sbin/apk differ

 

Because creating a new initramfs is part of the update-kernel script,= updating the kernel fixes the issue as well.

https://wiki.alpinelinux.org/wiki/Diskless_Mode#Upgrading_a_Diskl= ess_System

>= ; > > >

>= ; > > > > BR

>= ; > > > > Norbert

>= ; > > >

>= ; > > >

>= ; > >

>= ; > >

>= ; >

>= ;

>= ;

>= ;

>= ;

--nextPart2641315.Lt9SDvczpP--