For discussion of Alpine Linux development and developer support

11 7

[alpine-devel] apk-tools ideas

Timo Teras
Details
Message ID
<20151028181756.4715b37c@vostro>
Sender timestamp
1446049076
DKIM signature
missing
Download raw message
Hi,

Another ideas collection mail for post-3.3 time; this time for
apk-tools.

My ideas box has following items:

- Support 'bare' root. The idea is to have boot-media managed by
  apk-tools, and be upgradeable using apk. It'd technically be one .apk
  that can be upgraded from fixed URL. Something like
  "apk --root /media/boot upgrade" or so. Of course alpine-iso / abuild
  would need to support creating boot images in compatible format. They
  should also support creating boot-media / firmware package by giving
  in an apkovl for easy roll out of appliances. But that's another
  story.

- Potentially update the package format to use pax-headers instead
  of .PKGINFO and .SIGN.* meta-data files. This would allow extraction
  with tar and there would be no extra files extracted.

- Rework "repositories" file so that it apk config manages $MIRROR and
  $BRANCH. So upgrading branch/changing mirror changes config file, not
  the repository config. Similar to apt sources.list format.

- Elliptic Curve signature support

- Tracking what services to restart after upgrade, see:
  http://bugs.alpinelinux.org/issues/2251
  Though, migrating to s6-rc was also in the discussion so that might
  affect things if it happens.

- Merging of pre/post and install/upgrade scripts to single script

- Have apk add users/groups from .PKGINFO meta data instead of
  requiring scripts for that

and few other more of minor bug fix issues.

Any other ideas? Or comments on the priorities?

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Andy Shinn
Details
Message ID
<CAGF8=FygQSsGbRomvy9Y6qNxXv3JqRSqWMrCRyt2ftzp3V4WAQ@mail.gmail.com>
In-Reply-To
<20151028181756.4715b37c@vostro> (view parent)
Sender timestamp
1446050476
DKIM signature
missing
Download raw message
This might already be implied by the bare root idea, but support for
installing .apk directly from http/https would be neat. I build a
couple one-off package and currently install using a wget, apk, rm
combo. Would be cool to have this contained as one remote file install
option.


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20151028185024.7419b157@vostro>
In-Reply-To
<CAGF8=FygQSsGbRomvy9Y6qNxXv3JqRSqWMrCRyt2ftzp3V4WAQ@mail.gmail.com> (view parent)
Sender timestamp
1446051024
DKIM signature
missing
Download raw message
On Wed, 28 Oct 2015 09:41:16 -0700
Andy Shinn <andys@andyshinn.as> wrote:

> This might already be implied by the bare root idea, but support for
> installing .apk directly from http/https would be neat. I build a
> couple one-off package and currently install using a wget, apk, rm
> combo. Would be cool to have this contained as one remote file install
> option.

Yes, it's on my todo list too. The only problem is that the way the
code currently works, it'd need to GET the .apk twice; or stash it
to /tmp or something. The first pass reads the meta data for the
package solver, and then second time when we actually want to install
it.

But it's on the list.


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Christian Kampka
Details
Message ID
<CADq4isTmVgVYXyXcDD=YZ5ASLA1TvUNBDpnVP2Z+jP1+th2f0g@mail.gmail.com>
In-Reply-To
<20151028181756.4715b37c@vostro> (view parent)
Sender timestamp
1446117275
DKIM signature
missing
Download raw message
Hi,

since you are opening a wishlist, I'd like to see a feature that allows to
divert binaries through apk.
For reference, Debian offers such a feature through dpkg-divert[1].

What diversion basically provide is a way to tell apk that binary foo is
not to be installed in /usr/bin/foo but /usr/bin/foo.orig instead. Any
update of the package providing foo will then update the binary located at
/usr/bin/foo.orig instead of /usr/bin/foo.
I find this useful for replacing binaries with wrappers that provide
additional functionality. This is typically a problem if you use tooling
like ansible for your deployment that expects a binary in a certain place,
eg. the apk binary.
Tooling aside, may binaries are expected to be in a certain location, eg.
/usr/sbin/sendmail, which is currently provided by postfix. An admin might
want to place another sendmail in that path to provide the functionality
without causing a potential conflict with the apk system.

It would be great if apk would allow this.

 Cheers,
Christian

[1] http://manpages.debian.org/cgi-bin/man.cgi?query=dpkg-divert
systmkor
Details
Message ID
<B51684EC-D4D1-4BEC-BBB5-289DFC2D71F8@gmail.com>
In-Reply-To
<20151029160957.0a077f63@ncopa-desktop.alpinelinux.org> (view parent)
Sender timestamp
1446150920
DKIM signature
missing
Download raw message
A feature that would be nice is that apk would check to make sure all the paths it’s going to upgrade/update are not read-only. This would be nice for systems where you want the /boot and /usr partitions usually read-only

--
keybase.io/systmkor
Natanael Copa
Details
Message ID
<20151029160957.0a077f63@ncopa-desktop.alpinelinux.org>
In-Reply-To
<20151028181756.4715b37c@vostro> (view parent)
Sender timestamp
1446131397
DKIM signature
missing
Download raw message
On Wed, 28 Oct 2015 18:17:56 +0200
Timo Teras <timo.teras@iki.fi> wrote:

> Hi,
> 
> Another ideas collection mail for post-3.3 time; this time for
> apk-tools.

...

> Any other ideas? Or comments on the priorities?

There were patches for apk logfile some time ago.

I have also missed the possibility to apk add files from sftp/ssh
repository.

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20151031081108.7a21ddbb@vostro>
In-Reply-To
<CADq4isTmVgVYXyXcDD=YZ5ASLA1TvUNBDpnVP2Z+jP1+th2f0g@mail.gmail.com> (view parent)
Sender timestamp
1446271868
DKIM signature
missing
Download raw message
hi,

On Thu, 29 Oct 2015 11:14:35 +0000
Christian Kampka <christian@kampka.net> wrote:

> since you are opening a wishlist, I'd like to see a feature that
> allows to divert binaries through apk.
> For reference, Debian offers such a feature through dpkg-divert[1].
>>
> What diversion basically provide is a way to tell apk that binary foo
> is not to be installed in /usr/bin/foo but /usr/bin/foo.orig instead.
> Any update of the package providing foo will then update the binary
> located at /usr/bin/foo.orig instead of /usr/bin/foo.
> I find this useful for replacing binaries with wrappers that provide
> additional functionality. This is typically a problem if you use
> tooling like ansible for your deployment that expects a binary in a
> certain place, eg. the apk binary.
> Tooling aside, may binaries are expected to be in a certain location,
> eg. /usr/sbin/sendmail, which is currently provided by postfix. An
> admin might want to place another sendmail in that path to provide
> the functionality without causing a potential conflict with the apk
> system.

Yes. I remember wanting to have this too. There's two current hacks:

1. busybox

Busybox has trigger and it executes always some package has
touched /usr/bin or /bin and runs busybox --install -s. Effectively
giving the possibility for packages to override bb symlinks with real
files, and if that package gets removed then bb symlink is restored.

2. java-common

Has trigger that updates the 'default-jvm' path symlink whenever java
provider dir is touched.

The down side is that bb symlinks, and the default-jvm symlink is not
"owned" by anything. So we modified "apk info -W" to show who owns the
symlink target if the symlink is not owned.

---

apk might be able to do the above by itself. It could also
automatically setup correct target for the symlink based on package
priority field. Though, if a way is needed to custom select the package
to use, it'd need some more design and addition to the database format.

I don't like dpkg-divert's way of touching installed system, and
having pkg A modify pkg B's files. Seems dpkg-divert is sort of
deprecated because of that. update-alternatives[1] is another similar
system and closer to what I'd prefer. Though it seems slightly
overkill. It'd need to be something a bit simpler still, but the
underlying idea is what I'm ok with.

Comments?

[1] http://manpages.debian.org/cgi-bin/man.cgi?query=update-alternatives



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20151031081341.71c544b0@vostro>
In-Reply-To
<B51684EC-D4D1-4BEC-BBB5-289DFC2D71F8@gmail.com> (view parent)
Sender timestamp
1446272021
DKIM signature
missing
Download raw message
On Thu, 29 Oct 2015 13:35:20 -0700
systmkor <systmkor@gmail.com> wrote:

> A feature that would be nice is that apk would check to make sure all
> the paths it’s going to upgrade/update are not read-only. This would
> be nice for systems where you want the /boot and /usr partitions
> usually read-only

The cannot be done pre-emptively. Apk does not know *all* directories
it's going to write until a package is being extracted.

Though, we do get EROFS, and can do things there. What would be the
preferrer option? To remount rw? Abort and rollback and let admin do
that?
systmkor
Details
Message ID
<0FBE45E2-AF60-4650-BD21-D06C2F0C48B0@gmail.com>
In-Reply-To
<563488D0.5040400@skarnet.org> (view parent)
Sender timestamp
1446308409
DKIM signature
missing
Download raw message
>> Though, we do get EROFS, and can do things there. What would be the
>> preferrer option? To remount rw? Abort and rollback and let admin do
>> that?
> 
> Abort and rollback. No magic, please - you can't be sure that's what
> the admin wanted.

Abort and rollback would be plenty enough. I just figure that when apk
is installing a package and a file fails to be written to it should at
least return an error.

--
keybase.io/systmkor
Laurent Bercot
Details
Message ID
<563488D0.5040400@skarnet.org>
In-Reply-To
<20151031081341.71c544b0@vostro> (view parent)
Sender timestamp
1446283472
DKIM signature
missing
Download raw message
On 31/10/2015 07:13, Timo Teras wrote:
> Though, we do get EROFS, and can do things there. What would be the
> preferrer option? To remount rw? Abort and rollback and let admin do
> that?

  Abort and rollback. No magic, please - you can't be sure that's what
the admin wanted.

-- 
  Laurent



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
systmkor
Details
Message ID
<811F2041-D735-4AAA-A0AC-149533AD7AD2@gmail.com>
In-Reply-To
<0FBE45E2-AF60-4650-BD21-D06C2F0C48B0@gmail.com> (view parent)
Sender timestamp
1446593840
DKIM signature
missing
Download raw message
# Automatic Static Code Anaylsis
I was wondering if there has been any thoughts of using the coverity scanner for some automated bug detection. It’s free for open source projects. https://scan.coverity.com/

# Reproducible Builds
I’m for this but I think it’s farther down on a desired feature list. To add that may be more coupled with abuild as apposed to apk.

#apk API
I would argue that it would be important to build a local & remote API for apk/abuild for easy automated building and deployment (e.g. ansible,chef, etc.).

--
keybase.io/systmkor
Thomas Harning Jr.
Details
Message ID
<CAAuvu7cLbvC_+LEZ_2at=3w9f1yxHGbkbBG_b0cuoT0y18Q_Kw@mail.gmail.com>
In-Reply-To
<811F2041-D735-4AAA-A0AC-149533AD7AD2@gmail.com> (view parent)
Sender timestamp
1446594847
DKIM signature
missing
Download raw message
Another useful addition I think would be hooks to run at various stages.
The main feature that I would want to use this for is to hook in etckeeper
to track my etc configuration and updates automatically performed by apk
task execution.