Subject: [alpine-devel] Some questions about apk-tools

From: Dubiousjim <[email-masked]>
Date: Wed, 28 Nov 2012 18:44:06 -0500

Here are a few things I don't understand about apk-tools. Can anyone
help fill in the picture?

What exactly is the difference between `apk add ...` and `apk fix ...`?

What is the difference between `apk add -u ...` and `apk fix -u ...` and
`apk fix -r ...` and `apk upgrade` (except for the fact that the last
operates on a broader set of packages)?

What exactly is the effect of `apk add --initdb`?

What exactly is the effect of the "generic options" --clean-protected
and --purge? I see that they have to do with .apk-new files, but would
appreciate hearing them explained more fully.

What exactly is the behavior of `apk audit`? Does it work only in
cooperation with lbu? What is the exact effect of the --backup and
--recursive options for this command?

If I do get to understand these things better, I'll try to help make
sure they're better documented.

-- 
Dubiousjim
[email-masked]

---
Unsubscribe:  [email-masked]
Help:         [email-masked]
---

---

From: Timo Teras <[email-masked]>
Date: Thu, 29 Nov 2012 08:40:44 +0200

> What exactly is the difference between `apk add ...` and `apk

"add" will record in the '/etc/apk/world' that the package is something
user wanted explicitly to be installed.

"fix" will just re-install it only if it is already installed. "fix"
will always also re-install the package, "add" will only do things if
it was not installed.

> What is the difference between `apk add -u ...` and `apk fix -u ...`
> and `apk fix -r ...` and `apk upgrade` (except for the fact that the
> last operates on a broader set of packages)?

"add -u" will record the package in world. "fix -u" will do almost same
thing except not record the package in world.

With "fix -r" will operate on the specified package list, and can
re-install most of the package. "upgrade" works with world - it
upgrades all packages. "upgrade -a" is also notable, and in addition to
regular "upgrade" it also resets all versioned world dependencies, and
tries to make sure all packages come from some active repository.

> What exactly is the effect of `apk add --initdb`?

It creates the apk databases in /lib/apk/db/. Basically, this is needed
when you are creating a new rootfs, or chroot install.

> appreciate hearing them explained more fully.

In /etc/apk/protected_paths.d/*.list you can define which system
directories are treated specially. Generally this is /etc. On these
"protected directories" apk never overwrites modified files. Normally,
if file is modified, the new file will be extracted with .apk-new
extension (e.g. updated sample or default config file).

--clean-protected will make apk not create those .apk-new files. The
updated config files are just silently discarded and not made available.

--purge affects only "del" command. If package is deleted, and it has
modified config file, it is normally left intact. With --purge all
config files will be deleted regardless if they are modified or not.

> --recursive options for this command?

You can use "audit" independently and it can be useful. It was created
for lbu, though. Basically, lbu uses "audit" to figure out which files
need to go into the overlay tarball.

"audit" requires you to specify --backup or --system to determine mode
of operation. --backup makes it only consider "protected files", that
is the special directories marked as to contain configuration files.
Where as --system makes apk consider all installed files - this can be
used to do security audit that no system binary is changed.

--recursive option changes audit output. Normally if there is a new
directory inside /etc, apk would just report the directory entry.
--recursive makes apk scan the new directory for all files/subdirs and
report them individually. This was created to help 'lbu' pass the full
filelist to tar. I believe we needed this to properly store minimal set
of changes with tar in case someone modified permissions of
protected directory. Basically it makes output to something which can
be just passed on to "tar --no-recursion".

> sure they're better documented.

Thanks!

Please do ask if there's additional things you want to know. Many of
apk's features are not documented well enough. So thank you for helping
in this!

- Timo

---

---

From: Dubiousjim <[email-masked]>
Date: Thu, 29 Nov 2012 07:40:28 -0500

Hi Timo, thanks for your replies!

> thing except not record the package in world.

Ok, but I'm still unclear on the difference between the without "-u" and
the with "-u" version of each of these.

Suppose I've got these packages installed:

alpha-1.0
beta-1.0

And that these packages are available in my repositories:

alpha-1.0
beta-2.0
gamma-2.0

The handling of /etc/apk/world aside, will there be any difference in
the behavior of "apk add _", "apk add -u _", "apk fix _" and "apk fix -u
_", when each of alpha,beta,gamma are substituted for _:

Trying to get a handle on why the "-u" option is there.

> re-install most of the package.

So "fix -r foo" reinstalls foo and all of foo's dependencies? I'm not
sure if I'm understanding rightly?

> "upgrade" works with world - it

Right, I understand the broad outlines of upgrade and upgrade -a. But
let's say that beta is the only installed package I have that's out of
date (as in the above examples). In that situation, would "apk upgrade"
have the same net effect as "apk add beta"? Or "apk add -u beta"? or
some form of "apk fix"?

> when you are creating a new rootfs, or chroot install.

Ok, thanks.

> config files will be deleted regardless if they are modified or not.

Ok, thanks.

> apk's features are not documented well enough. So thank you for helping

Thanks, all of the rest is clear.

-- 
Dubiousjim
[email-masked]

---

---

From: Timo Teras <[email-masked]>
Date: Fri, 30 Nov 2012 15:44:04 +0200

On Thu, 29 Nov 2012 07:40:28 -0500 Dubiousjim
<[email-masked]> wrote:

> > "add" will record in the '/etc/apk/world' that the package is
> > something user wanted explicitly to be installed.
> > 
> > will always also re-install the package, "add" will only do things
> > if it was not installed.
> > 
> > > What is the difference between `apk add -u ...` and `apk fix
> > > -u ...` and `apk fix -r ...` and `apk upgrade` (except for the
> > > fact that the last operates on a broader set of packages)?
> > 
> > "add -u" will record the package in world. "fix -u" will do almost
> > same thing except not record the package in world.
> 
> Ok, but I'm still unclear on the difference between the without "-u"
> and the with "-u" version of each of these.
> 
> the behavior of "apk add _", "apk add -u _", "apk fix _" and "apk fix
> -u _", when each of alpha,beta,gamma are substituted for _:

If gamma-2.0 depends on "beta" (without any version requirement).

"apk add gamma" would just install "gamma-2.0" and do nothing else.
"apk add -u gamma" would install "gamma-2.0" and also upgrade the
dependency "beta" from 1.0 to 2.0.

Basically on add, the -u means "prefer upgraded versions of all
mentioned packages and their dependencies". Without -u minimum changes
to installed packages are attempted, that is the installed package is
always preferred.

Same applies to fix. "fix" alone will re-install the same package
currently installed if available. With -u it will be instead upgraded
if such upgrade is available.

> sure if I'm understanding rightly?

with "fix" -r is actually reinstall, the default behaviour. with -d you
get the fix action propagated to all dependencies. With -d it would
reinstall the named package and all it's dependencies.

> > upgrades all packages. "upgrade -a" is also notable, and in
> > addition to regular "upgrade" it also resets all versioned world
> > dependencies, and tries to make sure all packages come from some
> > active repository.
> 
> date (as in the above examples). In that situation, would "apk
> upgrade" have the same net effect as "apk add beta"? Or "apk add -u
> beta"? or some form of "apk fix"?

"apk upgrade" is roughly equivalent to "apk fix -u -d
`cat /etc/apk/world`"

"apk add" always adds the package names to 'world'. If you know the
named package is already there, then "fix -u" and "add -u" do not
really have any functional difference.

So yes, the functionality is slightly overlapping.

-Timo

---

---

From: Dubiousjim <[email-masked]>
Date: Fri, 30 Nov 2012 10:14:17 -0500

Thanks so much for these explanations. Things are a lot clearer now.

> reinstall the named package and all it's dependencies.

So the -r flag on fix is a noop?

-- 
Dubiousjim
[email-masked]

---

---

From: Dubiousjim <[email-masked]>
Date: Fri, 30 Nov 2012 10:36:33 -0500

>From quickly peeking at the source, it looks like:

    * `apk fix foo` and `apk fix -r foo` behave the same. Can I assume
      that it's always the current version that's re-installed, even if an
      upgrade is available? And that if foo isn't currently installed,
      these commands do nothing?

    * `apk fix -u foo` won't reinstall foo, but will install an upgrade
       for foo if one is available.

    * `apk fix -r -u foo` will install an upgrade for foo if one is
       available, else will reinstall the current version.

Is that right?

-- 
Dubiousjim
[email-masked]

---

---

From: Timo Teras <[email-masked]>
Date: Mon, 3 Dec 2012 15:42:14 +0200

On Fri, 30 Nov 2012 10:36:33 -0500 Dubiousjim
<[email-masked]> wrote:

> > > with "fix" -r is actually reinstall, the default behaviour. with
> > > -d you get the fix action propagated to all dependencies. With -d
> > > it would reinstall the named package and all it's dependencies.
> > 
>       that it's always the current version that's re-installed, even
> if an upgrade is available? And that if foo isn't currently installed,

Correct. This is a legacy thing, on previous versions (alpine 2.3 and
earlier), the you had to specify -r or -u. That was kinda lame to
most of the time need to write the extra -r, so now it is assumed if -u
is not specified.

>     * `apk fix -u foo` won't reinstall foo, but will install an
> upgrade for foo if one is available.

Correct.

>        available, else will reinstall the current version.

Correct.

> Is that right?

Yes. :)

- Timo

---

---

From: Dubiousjim <[email-masked]>
Date: Mon, 3 Dec 2012 10:20:17 -0500

> >        available, else will reinstall the current version.

Thanks Timo, except I was experimenting with these right now (before
reading your email), to verify my hypotheses, and this is not the
behavior I see.

    # apk version
    Installed:                                Available:
    linux-grsec-3.6.7-r0                    < 3.6.8-r1
    cups-libs-1.6.1-r0                      < 1.6.1-r1
    ghostscript-9.06-r2                     < 9.06-r3
    imagemagick-6.8.0.4-r0                  < 6.8.0.7-r0
    liblcms-1.19-r3                         < 1.19-r4
    poppler-0.20.3-r1                       < 0.20.5-r0
    poppler-gtk-0.20.3-r1                   < 0.20.5-r0
    epdfview-0.1.8-r5                       < 0.1.8-r6
    evince-2.32.0-r11                       < 2.32.0-r12

Ok, so given what we've said, I'd have expected
`apk fix _` or `apk fix -r _` for one of the packages with an
upgrade available to only install the current version, not the available
upgrade. However:

    # apk fix cups-libs
    (1/1) Upgrading cups-libs (1.6.1-r0 -> 1.6.1-r1)
    Executing uclibc-utils-0.9.33.2-r14.trigger
    OK: 831 MiB in 317 packages

When the package is up-to-date, then `apk fix _` does indeed re-install:

    # apk fix cups-libs
    (1/1) Re-installing cups-libs (1.6.1-r1)
    Executing uclibc-utils-0.9.33.2-r14.trigger
    OK: 831 MiB in 317 packages

But it seems like `apk fix _` is interpreted as `apk fix -u -r _`, not
as just `apk fix -r _`.

Verified that `apk fix -u _` is a noop if no upgrade is available:

    # apk fix -u cups-libs
    OK: 831 MiB in 317 packages

I thought perhaps `apk fix -r _` might suppress any available upgrades,
even if `apk fix _` didn't. Nope, it installed the upgrade and perhaps
also assumed a `-d`:

    # apk fix -r evince
    (1/4) Upgrading liblcms (1.19-r3 -> 1.19-r4)
    (2/4) Upgrading poppler (0.20.3-r1 -> 0.20.5-r0)
    (3/4) Upgrading poppler-gtk (0.20.3-r1 -> 0.20.5-r0)
    (4/4) Upgrading evince (2.32.0-r11 -> 2.32.0-r12)
    Executing busybox-1.20.2-r3.trigger
    Executing uclibc-utils-0.9.33.2-r14.trigger
    Executing glib-2.34.2-r0.trigger
    Executing gtk+2.0-2.24.13-r2.trigger
    Executing desktop-file-utils-0.20-r0.trigger
    OK: 831 MiB in 317 packages

Hmm, so now I'm confused. It looks like the absence of a `-u` in
`apk fix _` and `apk fix -r _` is not being honored. And maybe a `-d` is
sometimes being tacitly assumed, too... unless the upgraded dependencies of
evince were tagged with versions because they were auto-detected library
dependencies. That wouldn't explain why the liblcms upgrade was applied,
though.

-- 
Dubiousjim
[email-masked]

---

---

From: Timo Teras <[email-masked]>
Date: Wed, 5 Dec 2012 11:53:44 +0200

On Mon, 3 Dec 2012 10:20:17 -0500 Dubiousjim
<[email-masked]> wrote:

> upgrade available to only install the current version, not the
> available upgrade. However:
> 
>     # apk fix cups-libs

This happens because the older version is no longer available in the
index. Thus it can't reinstall the same version. So it prefers to
intall the new version compared to doing nothing.

If you want to test the regular fix, you might take some cdrom image
and add it as one source - install package from it. Then add some
online repository and see if it has updated version. And then try
regular fix - it should reinstall the old version from cdrom.

> But it seems like `apk fix _` is interpreted as `apk fix -u -r _`, not

Not exactly. It is more like: I want this package reinstalled. Please
reinstall the one currently installed, but if not available pick best
available option. (Which usually is the latest version).

> I thought perhaps `apk fix -r _` might suppress any available
> upgrades, even if `apk fix _` didn't. Nope, it installed the upgrade
> and perhaps also assumed a `-d`:
> 
>     # apk fix -r evince
> `apk fix _` and `apk fix -r _` is not being honored. And maybe a `-d`
> is sometimes being tacitly assumed, too... unless the upgraded
> dependencies of evince were tagged with versions because they were
> auto-detected library dependencies. That wouldn't explain why the
> liblcms upgrade was applied, though.

I'm not sure about why this would happen. This could be a bug.

---

---

From: Dubiousjim <[email-masked]>
Date: Wed, 5 Dec 2012 07:34:07 -0500

> regular fix - it should reinstall the old version from cdrom.

Ok, that makes sense at least. I *think* the older version was still
available in my apk cache though---can't check anymore since I've
synched the cache recently. If the old version were available in the
cache but not any index, and `-u` were omitted, is the intended behavior for it
to reinstall or upgrade?

-- 
Dubiousjim
[email-masked]

---