MIME-Version: 1.0
References: <9410103.CDJkKcVGEf@ntb-norbert> <2408933.iZASKD2KPV@ntb-norbert>
 <MW5PR22MB3060F6ABB3D979F10878C9F7D6F52@MW5PR22MB3060.namprd22.prod.outlook.com>
 <23824424.6Emhk5qWAg@ntb-norbert>
In-Reply-To: <23824424.6Emhk5qWAg@ntb-norbert>
From: Prabu Anand Kalivaradhan <kprabuanand@gmail.com>
Date: Sat, 8 Feb 2025 14:31:44 +0530
Message-ID: <CAAt6f0aTgdLSr+QC-gbZeEB8u_8H41wqtc-qa5jFoQ+q0_Y0nw@mail.gmail.com>
Subject: Re: Diskless systems do not boot with latest apk versions from cache
To: MBB <mbbert@gmail.com>
Cc: "~alpine/users@lists.alpinelinux.org" <~alpine/users@lists.alpinelinux.org>, Sertonix <sertonix@posteo.net>, 
	Kevin Garman <Garman@scadaware.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi MBB,

Please confirm if the steps given in the page
https://wiki.alpinelinux.org/wiki/Diskless_Mode#Upgrading_a_Diskless_System
is complete.

If something is missing, please highlight here or in the wiki. Most of
the content on the above page has been consolidated from other parts
of wiki. Since i don't use diskless i'm not 100% sure about the
correctness of the content.

I am still hesitating to touch those wiki pages related to Raspberry.
Anyone willing to offering to help test content changes, please reply
me here in the list or personally.

Regards,
Prabu

On Sat, 8 Feb 2025 at 13:56, MBB <mbbert@gmail.com> wrote:
>
> Am Montag, 3. Februar 2025, 16:07:18 Mitteleurop=C3=A4ische Normalzeit sc=
hrieb Kevin Garman:
>
> > I have long been struggling with similar issues on my diskless installs=
.  I run..
>
> >
>
> >  apk cache -v sync
>
> >
>
> > ...and sometimes it breaks my system(s) and sometimes it doesn't.  I've=
 read where cache sync should be used with care, but I believe I'm using it=
 as intended.  It seems like it should be straight forward as all it's supp=
ose to do (as I understand it) is make the cache match what the system need=
s during boot.  It seems like running cache sync should be the solution to =
problem it's causing!
>
> >
>
> > I've been working to figure out a simple/repeatable set of steps to rep=
roduce the problem, but so far I haven't been able to get there.
>
> >
>
> >
>
> >
>
> >
>
> > ________________________________
>
> > From: ~alpine/users <~alpine/users@lists.alpinelinux.org> on behalf of =
MBB <mbbert@gmail.com>
>
> > Sent: Sunday, February 2, 2025 09:16
>
> > To: ~alpine/users@lists.alpinelinux.org <~alpine/users@lists.alpinelinu=
x.org>; Sertonix <sertonix@posteo.net>
>
> > Subject: Re: Diskless systems do not boot with latest apk versions from=
 cache
>
> >
>
> >
>
> > Am Sonntag, 2. Februar 2025, 09:11:22 Mitteleurop=C3=A4ische Normalzeit=
 schrieb MBB:
>
> > > Am Samstag, 1. Februar 2025, 14:39:34 Mitteleurop=C3=A4ische Normalze=
it schrieb MBB:
>
> > > > Am Samstag, 1. Februar 2025, 13:12:29 Mitteleurop=C3=A4ische Normal=
zeit schrieb Sertonix:
>
> > > > > On Sat Feb 1, 2025 at 9:24 AM CET, MBB wrote:
>
> > > > > > Hi all!
>
> > > > > >
>
> > > > > > I have 3 Raspberry Pis, different PI versions and different Alp=
ine versions (3.18.9, 3.19.1, 3.19.4).
>
> > > > > >
>
> > > > > > All have the same issue.
>
> > > > > >
>
> > > > > > I'm doing:
>
> > > > > > apk update; apk upgrade
>
> > > > > >
>
> > > > > > Checking the version of an upgraded binary before reboot, e.g. =
named, looks ok.
>
> > > > > >
>
> > > > > > After reboot, it loads an old version even though a newer packa=
ge is available in cache.
>
> > > > > >
>
> > > > > > gate [/etc/apk]$ ls -al cache
>
> > > > > > lrwxrwxrwx    1 root     root            22 Feb  1 08:50 cache =
-> /media/mmcblk0p1/cache
>
> > > > > >
>
> > > > > > gate [/etc/apk/cache]$ ls -altr bind-9*
>
> > > > > > -rwxr-xr-x    1 root     root        207297 Mar 15  2024 bind-9=
.18.24-r1.5aca84cc.apk
>
> > > > > > -rwxr-xr-x    1 root     root        208996 Dec 17 16:37 bind-9=
.18.32-r0.1254446d.apk
>
> > > > > > -rwxr-xr-x    1 root     root        209078 Jan 29 19:44 bind-9=
.18.33-r0.b6697cb2.apk
>
> > > > > >
>
> > > > > > gate [/etc/apk/cache]$ named -V
>
> > > > > > BIND 9.18.32 (Extended Support Version) <id:d1f1392>
>
> > > > > > running on Linux armv7l 6.6.14-0-rpi #1-Alpine SMP Fri Jan 26 1=
3:40:47 UTC 2024
>
> > > > > >
>
> > > > > > Is there something I'm doing wrong?
>
> > > > >
>
> > > > > It looks correct what you are doing. Could you try if running thi=
s
>
> > > > > commands returns any error?
>
> > > > >
>
> > > > > $ apk --simulate add bind=3D9.18.33-r0 --no-network
>
> > > >
>
> > > > No errors with this command.
>
> > > >
>
> > > > *gate [*~*]$ *doas apk --simulate add bind=3D9.18.33-r0 --no-networ=
k
>
> > > > doas (norbert@gate) password:
>
> > > > (1/5) Upgrading bind-libs (9.18.32-r0 -> 9.18.33-r0)
>
> > > > (2/5) Upgrading bind-tools (9.18.32-r0 -> 9.18.33-r0)
>
> > > > (3/5) Upgrading bind-dnssec-root (9.18.32-r0 -> 9.18.33-r0)
>
> > > > (4/5) Upgrading bind (9.18.32-r0 -> 9.18.33-r0)
>
> > > > (5/5) Upgrading bind-openrc (9.18.32-r0 -> 9.18.33-r0)
>
> > > > OK: 157 MiB in 215 packages
>
> > >
>
> > > I removed all outdated bind packages from the cache with the result, =
that bind is no longer available after reboot.
>
> > > After moving the bind*9.18.32-r0*.apk files back into the cache direc=
tory, the service gets restored and starts up without issues after reboot.
>
> > >
>
> > > It seems that something's pointing to a specific version.
>
> > >
>
> >
>
> > `apk cache sync` breaks the entire system.
>
> >
>
> > This cleans the cache from outdated APKINDEX* and *.apk files.
>
> > But after reboot nothing is loaded and the boot process ends with a `(n=
one) local:` prompt.
>
> >
>
> > Restoring the old cache directory, brings the system to life again.
>
> >
>
> > Same issue when I manually delete only the APKINDEX* files, and update =
the local DB with `apk update` to download fresh APKINDEX.
>
> >
>
> > Restoring the old APKINDEX* files from a backup fixes this.
>
> >
>
>
> I fixed it for all 3 of my systems.
>
> Key is to update the /media/mmcblk0p1/boot/initramfs-rpi
>
>
> I created a new one following this guide: https://wiki.alpinelinux.org/wi=
ki/Initramfs_init
>
> I activated these features to get an initramfs which matches the original
>
>
> features=3D"base dhcp keymap kms mmc squashfs usb"
>
>
>
> When I compared the old an new initramfs, only these binaries showed diff=
erences:
>
>
> Files initramfs/bin/busybox and mkinitfs/bin/busybox differ
> Files initramfs/lib/libapk.so.2.14.0 and mkinitfs/lib/libapk.so.2.14.0 di=
ffer
> Files initramfs/lib/libcrypto.so.3 and mkinitfs/lib/libcrypto.so.3 differ
> Files initramfs/lib/libssl.so.3 and mkinitfs/lib/libssl.so.3 differ
> Files initramfs/sbin/apk and mkinitfs/sbin/apk differ
>
>
>
> Because creating a new initramfs is part of the update-kernel script, upd=
ating the kernel fixes the issue as well.
>
> https://wiki.alpinelinux.org/wiki/Diskless_Mode#Upgrading_a_Diskless_Syst=
em
>
>
> > > > >
>
> > > > > > BR
>
> > > > > > Norbert
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>