For discussion of Alpine Linux development and developer support

6 3

[alpine-devel] How to rescue your edge box after the mkinitfs incident

Natanael Copa
Details
Message ID
<20140327194522.5a4d04b8@ncopa-laptop>
Sender timestamp
1395949522
DKIM signature
missing
Download raw message
Hi,

It turned out that there were 2 bugs in mkinitfs, one that only
affected uClibc and one that affected both uClibc and musl.

I thought I'd write how I rescued my laptop after it became unbootable
after upgrading to broken mkinitfs.

* Boot a rescue boot USB, with alpine v2.7

* Configure network (run setup-alpine -q for setting up the live
  session)

* make sure you have new apk-tools:

    apk upgrade -U

* I have my root on lvm so:
  
    apk add lvm2
    lvchange -a y vg0

* mount the root:

    mount -t /dev/vg0/lv_root /mnt

* mount the boot partition. In my case this is /dev/sda2:

    mount /dev/sda2 /mnt/boot

* I had to mount /mnt/proc otherwise readlink -f wouldn't work in the
  chroot. Not sure if it is needed for uclibc:

    mount --bind /proc /mnt/proc

* Upgrade the broken system:

    apk upgrade -U --root /mnt

* Verify that you get mkinitfs-2.6.0-r3 and check that mkinitfs trigger
  don't give any cpio errors.

* cleanup and reboot.

    umount /mnt/proc /mnt/boot /mnt
    reboot

That should be it.

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Alan Messias
Details
Message ID
<CAFaTuVZav3cssQK6qy3Mm4OV8e2dx3aTHNA1PmWHpk0o1gfSsg@mail.gmail.com>
In-Reply-To
<20140327194522.5a4d04b8@ncopa-laptop> (view parent)
Sender timestamp
1395949612
DKIM signature
missing
Download raw message
Is it useful to be placed on wiki.al.o ?


On Thu, Mar 27, 2014 at 4:45 PM, Natanael Copa <ncopa@alpinelinux.org>wrote:

> Hi,
>
> It turned out that there were 2 bugs in mkinitfs, one that only
> affected uClibc and one that affected both uClibc and musl.
>
> I thought I'd write how I rescued my laptop after it became unbootable
> after upgrading to broken mkinitfs.
>
> * Boot a rescue boot USB, with alpine v2.7
>
> * Configure network (run setup-alpine -q for setting up the live
>   session)
>
> * make sure you have new apk-tools:
>
>     apk upgrade -U
>
> * I have my root on lvm so:
>
>     apk add lvm2
>     lvchange -a y vg0
>
> * mount the root:
>
>     mount -t /dev/vg0/lv_root /mnt
>
> * mount the boot partition. In my case this is /dev/sda2:
>
>     mount /dev/sda2 /mnt/boot
>
> * I had to mount /mnt/proc otherwise readlink -f wouldn't work in the
>   chroot. Not sure if it is needed for uclibc:
>
>     mount --bind /proc /mnt/proc
>
> * Upgrade the broken system:
>
>     apk upgrade -U --root /mnt
>
> * Verify that you get mkinitfs-2.6.0-r3 and check that mkinitfs trigger
>   don't give any cpio errors.
>
> * cleanup and reboot.
>
>     umount /mnt/proc /mnt/boot /mnt
>     reboot
>
> That should be it.
>
> -nc
>
>
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
>
>


-- 
Atenciosamente,
Alan Messias Cordeiro

*The nice thing about standards is that there are so many to choose from.
And if you really don't like all the standards you just have to wait
another year until the one arises you are looking for. -- A. Tanenbaum,
"Introduction to Computer Networks"*
Natanael Copa
Details
Message ID
<20140328170507.44eb116d@ncopa-desktop.alpinelinux.org>
In-Reply-To
<CAFaTuVZav3cssQK6qy3Mm4OV8e2dx3aTHNA1PmWHpk0o1gfSsg@mail.gmail.com> (view parent)
Sender timestamp
1396022707
DKIM signature
missing
Download raw message
On Thu, 27 Mar 2014 16:46:52 -0300
Alan Messias <alancordeiro@gmail.com> wrote:

> Is it useful to be placed on wiki.al.o ?

I first though no, because this should not happen, but on the other
hand, it might be useful to have a general rescue wiki page.

-nc

 
> On Thu, Mar 27, 2014 at 4:45 PM, Natanael Copa <ncopa@alpinelinux.org>wrote:
> 
> > Hi,
> >
> > It turned out that there were 2 bugs in mkinitfs, one that only
> > affected uClibc and one that affected both uClibc and musl.
> >
> > I thought I'd write how I rescued my laptop after it became unbootable
> > after upgrading to broken mkinitfs.
> >
> > * Boot a rescue boot USB, with alpine v2.7
> >
> > * Configure network (run setup-alpine -q for setting up the live
> >   session)
> >
> > * make sure you have new apk-tools:
> >
> >     apk upgrade -U
> >
> > * I have my root on lvm so:
> >
> >     apk add lvm2
> >     lvchange -a y vg0
> >
> > * mount the root:
> >
> >     mount -t /dev/vg0/lv_root /mnt
> >
> > * mount the boot partition. In my case this is /dev/sda2:
> >
> >     mount /dev/sda2 /mnt/boot
> >
> > * I had to mount /mnt/proc otherwise readlink -f wouldn't work in the
> >   chroot. Not sure if it is needed for uclibc:
> >
> >     mount --bind /proc /mnt/proc
> >
> > * Upgrade the broken system:
> >
> >     apk upgrade -U --root /mnt
> >
> > * Verify that you get mkinitfs-2.6.0-r3 and check that mkinitfs trigger
> >   don't give any cpio errors.
> >
> > * cleanup and reboot.
> >
> >     umount /mnt/proc /mnt/boot /mnt
> >     reboot
> >
> > That should be it.
> >
> > -nc
> >
> >
> > ---
> > Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> > Help:         alpine-devel+help@lists.alpinelinux.org
> > ---
> >
> >
> 
> 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Jeff Pohlmeyer
Details
Message ID
<CAEuRe+2t=Lzz5fcJsD2eKBjEHnC2xJ66ayVXHgBQGGAk0QUXtQ@mail.gmail.com>
In-Reply-To
<20140327194522.5a4d04b8@ncopa-laptop> (view parent)
Sender timestamp
1396376824
DKIM signature
missing
Download raw message
On Thu, Mar 27, 2014 at 2:45 PM, Natanael Copa wrote:

> It turned out that there were 2 bugs in mkinitfs, one that only
> affected uClibc and one that affected both uClibc and musl.

I'm running into another problem, or maybe it's just something wrong with
my installation...

The mkinitfs upgrade creates /etc/mkinitfs/features.d/base.files.apk-new
This new file no longer lists the dependent libraries in /lib/.

I assume this is because "lddtree" is supposed to take care of that now.

But, "lddtree -l /bin/busybox" lists "/lib/ld-uClibc.so.0.9.32" which is
a symbolic link, but for some reason it never resolves that link, so the
newly-created initramfs fails to run /init because of the missing loader.

Seems to me like that is a bug in lddtree, but maybe the "base.files.apk-new"
should still at least list "/lib/ld-uClibc-0.9.33.2.so" until the issue can
be resolved?

 - Jeff


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Jeff Pohlmeyer
Details
Message ID
<CAEuRe+3+7=sop9Db3M6WUgs_jby_kR9WUDBvisZOFbM5Ed=GhA@mail.gmail.com>
In-Reply-To
<20140402090204.59c9c99e@ncopa-desktop.alpinelinux.org> (view parent)
Sender timestamp
1396444547
DKIM signature
missing
Download raw message
On Apr 2, 2014, Natanael Copa wrote:
> On Tue, 1 Apr 2014 Jeff Pohlmeyer wrote:

>> Seems to me like that is a bug in lddtree

> Whoops. this is definitively a bug in lddtree yes. It doesnt resolve
> the interp symlinks. I think I fixed it with lddtree-1.25-r1.

Ah, I didn't realize you were maintaining this. I glanced at the lddtree file
and saw the word "gentoo" so I assumed it was an upstream project.



> You could manually add the file to base.file as a temp workaround.

Yes, that's what I did.



> but it should be fixed now

1.25-r1 looks good from here.



> Sorry about this.

No problem, just another "learning experience" for me ;-)


Thanks for the quick fix!

- Jeff


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Natanael Copa
Details
Message ID
<20140402090204.59c9c99e@ncopa-desktop.alpinelinux.org>
In-Reply-To
<CAEuRe+2t=Lzz5fcJsD2eKBjEHnC2xJ66ayVXHgBQGGAk0QUXtQ@mail.gmail.com> (view parent)
Sender timestamp
1396422124
DKIM signature
missing
Download raw message
On Tue, 1 Apr 2014 13:27:04 -0500
Jeff Pohlmeyer <yetanothergeek@gmail.com> wrote:

> On Thu, Mar 27, 2014 at 2:45 PM, Natanael Copa wrote:
> 
> > It turned out that there were 2 bugs in mkinitfs, one that only
> > affected uClibc and one that affected both uClibc and musl.
> 
> I'm running into another problem, or maybe it's just something wrong with
> my installation...
> 
> The mkinitfs upgrade creates /etc/mkinitfs/features.d/base.files.apk-new
> This new file no longer lists the dependent libraries in /lib/.
> 
> I assume this is because "lddtree" is supposed to take care of that now.
> 
> But, "lddtree -l /bin/busybox" lists "/lib/ld-uClibc.so.0.9.32" which is
> a symbolic link, but for some reason it never resolves that link, so the
> newly-created initramfs fails to run /init because of the missing loader.

Whoops. this is definitively a bug in lddtree yes. It doesnt resolve
the interp symlinks.

I think I fixed it with lddtree-1.25-r1.

> Seems to me like that is a bug in lddtree, but maybe the "base.files.apk-new"
> should still at least list "/lib/ld-uClibc-0.9.33.2.so" until the issue can
> be resolved?

You could manually add the file to base.file as a temp workaround. (but
it should be fixed now)

Sorry about this.

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Natanael Copa
Details
Message ID
<20140402153247.36f00072@ncopa-desktop.alpinelinux.org>
In-Reply-To
<CAEuRe+3+7=sop9Db3M6WUgs_jby_kR9WUDBvisZOFbM5Ed=GhA@mail.gmail.com> (view parent)
Sender timestamp
1396445567
DKIM signature
missing
Download raw message
On Wed, 2 Apr 2014 08:15:47 -0500
Jeff Pohlmeyer <yetanothergeek@gmail.com> wrote:

> On Apr 2, 2014, Natanael Copa wrote:
> > On Tue, 1 Apr 2014 Jeff Pohlmeyer wrote:
> 
> >> Seems to me like that is a bug in lddtree
> 
> > Whoops. this is definitively a bug in lddtree yes. It doesnt resolve
> > the interp symlinks. I think I fixed it with lddtree-1.25-r1.
> 
> Ah, I didn't realize you were maintaining this. I glanced at the lddtree file
> and saw the word "gentoo" so I assumed it was an upstream project.

I have forked it since gentoo upstream was not interested in shell version:
https://bugs.gentoo.org/show_bug.cgi?id=504954

I ported it to posix shell and fixed some issues, like resolving
symlinks. I missed the interp.

...

> Thanks for the quick fix!

Thanks for you patience :)

 
> - Jeff



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---