For discussion of Alpine Linux development and developer support

3 3

[alpine-devel] udev replacement in Alpine Linux

Christoph Lohmann
Details
Message ID
<20150726103211.3A0FF1025EE9D@r-36.net>
Sender timestamp
1437905426
DKIM signature
missing
Download raw message
Greetings.

Alan Pillay was asking me to join in on the discussion about Alpine Lin‐
ux needing a udev replacement to strive more towards a  KISS  future  of
this distribution.

I  am  the  author of nldev and nlmon, which I am using personally on my
Laptop.  During this development my goal was to have a KISS solution  to
the udev mess and its abstraction layers. Well, the idea never complete‐
ly finished due to me being satisfied with what nldev and nlmon  do  and
no  reason to write the biggest threat to motivation: a libudev replace‐
ment.

I’m just describing how device management in Linux works now:

1.) mount devtmpfs onto /tmp on your own or let the kernel do it
    * this will have the kernel create all devices on its own but without any
      permissions

Now there are two options:
1.1.) Give the kernel an application to run on each new device event.
1.2.) Listen on a netlink socket for events and handle the device events.

The  first  option  is  what  mdev does and the second one is what udev,
smdev and nldev do.

What’s the problem with the first approach and why udev exists? There is
no concurrency. How does udev solve this? By parallel execution  but  in
the end waiting / settling for all events to have happened. Of course it
was extended to even more methodologies and is now the abscess  of  sys‐
temd.

The  KISS  way  I  imagined  would be this way: nldev not just executing
mdev, but it caching the configuration file for the speedup. That’s what
smdev tries to do but it does not implement all of the busybox mdev.conf
syntax. For compatibility reasons the libudev‐compat needs to be  ripped
off into a separate project so you don’t infect smdev with systemd code.
The libudev compatibility is especially needed  for  X.org  and  dynamic
uevent device discovery.

What needs to be done for solving this udev problem once and for all and
have a modular replacement:
* Send in patches to smdev to have the complete busybox syntax mdev.conf
  implemented so all the mdev.conf examples out there can be reused.
* Create a repository with all kind of mdev scripts so they can be reused.
* Rip off libudev-compat from vdev and have it a separate package, if needed.

Please  don’t  use vdev. It does not follow the KISS principles and uses
C++.  The namespace for gid/uid is a big hack which  should  be  in  the
kernel, as it is in Plan 9.

I  invite all interested developers to join suckless in its strive for a
better KISS future and send  patches  to  hackers@suckless.org  for  the
smdev[0]  project or to implement the libudev‐compat or join discussions
on dev@suckless.org.


Sincerely,

Christoph Lohmann

[0] http://git.suckless.org/smdev/



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Natanael Copa
Details
Message ID
<20150727120744.4ffdfa0b@ncopa-desktop.alpinelinux.org>
In-Reply-To
<20150726103211.3A0FF1025EE9D@r-36.net> (view parent)
Sender timestamp
1437991664
DKIM signature
missing
Download raw message
On Sun, 26 Jul 2015 12:10:26 +0200
Christoph Lohmann <20h@r-36.net> wrote:

> Greetings.
> 
> Alan Pillay was asking me to join in on the discussion about Alpine Lin*
> ux needing a udev replacement to strive more towards a  KISS  future  of
> this distribution.

Hi,

Thanks for joing the discussion.
 
> I  am  the  author of nldev and nlmon, which I am using personally on my
> Laptop.  During this development my goal was to have a KISS solution  to
> the udev mess and its abstraction layers. Well, the idea never complete*
> ly finished due to me being satisfied with what nldev and nlmon  do  and
> no  reason to write the biggest threat to motivation: a libudev replace*
> ment.

Thanks for joining. I have been doing some experiments and are using
nldev code as base.

> I*m just describing how device management in Linux works now:

[snip]...

My conclusion is that /proc/sys/kernel/hotplug is a dead end for two
reasons:

1) serialization. You can get race conditions. busybox mdev has some
workarounds for those but I think it is ugly. It is better solved with
netlink.

2) performance. You can get a lot of forks during boot. Yes I know some
people does not care if the box take 60 seconds to boot. I do care, and
I believe it can be solved and still keep things simple.

However, using netlink has the drawback that it requires a forever
running daemon, and normally you only need it during bootup, and after
that, you need it seldomly, when you plug in an usb disk, a mouse or
similar.

> The  KISS  way  I  imagined  would be this way: nldev not just
> executing mdev, but it caching the configuration file for the
> speedup. That*s what smdev tries to do but it does not implement all
> of the busybox mdev.conf syntax. 

Here is what I have been thinking, and what came out of a long
discussion on busybox mailing list.

Implement the netlink daemon as a socket activator. When there is a new
event, fork/exec the handler and pass over the socket.

The handler reads the events from the socket and handles them. Set a
timout on the socket and exit when there are no events withing one or two
seconds. The socket activator takes over again.

I have this implemented, based on nldev code:
http://git.alpinelinux.org/cgit/user/ncopa/nlplug/tree/

> For compatibility reasons the
> libudev*compat needs to be  ripped off into a separate project so you
> don*t infect smdev with systemd code. The libudev compatibility is
> especially needed  for  X.org  and  dynamic uevent device discovery.

This makes sense yes.

I am interested in libudev comaptibility, but have not looked too much
into it. I don't need full ABI compatibility, but it would be very
helpful to make it easy to port current applictions who uses libudev.

> What needs to be done for solving this udev problem once and for all
> and have a modular replacement:
> * Send in patches to smdev to have the complete busybox syntax
> mdev.conf implemented so all the mdev.conf examples out there can be
> reused.
> * Create a repository with all kind of mdev scripts so they can be
> reused.
> * Rip off libudev-compat from vdev and have it a separate package, if
> needed.

Alternatively, we could make an alternative xorg config backend. It
does not look too difficult:
http://cgit.freedesktop.org/xorg/xserver/tree/config/udev.c

> Please  don*t  use vdev. It does not follow the KISS principles and
> uses C++.  The namespace for gid/uid is a big hack which  should  be
> in  the kernel, as it is in Plan 9.

C++ is a no-go.

> I  invite all interested developers to join suckless in its strive
> for a better KISS future and send  patches  to  hackers@suckless.org
> for  the smdev[0]  project or to implement the libudev*compat or join
> discussions on dev@suckless.org.

I think it might make sense to do the mdev.conf parsing and event
handlig from either busybox or toybox. Those toolboxes also have things
like modprobe and blkid, which should make it possible to load
modaliases without fork/exec and find root=... on /proc/cmdline. (very
useful from initramfs)

 
> Sincerely,
> 
> Christoph Lohmann
> 
> [0] http://git.suckless.org/smdev/


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Christoph Lohmann
Details
Message ID
<20150727105017.2DB701025EEDC@r-36.net>
In-Reply-To
<20150727120744.4ffdfa0b@ncopa-desktop.alpinelinux.org> (view parent)
Sender timestamp
1437993699
DKIM signature
missing
Download raw message
Greetings.

On Mon, 27 Jul 2015 12:41:39 +0200 Natanael Copa <ncopa@alpinelinux.org> wrote:
> On Sun, 26 Jul 2015 12:10:26 +0200
> Christoph Lohmann <20h@r-36.net> wrote:
>
> However, using netlink has the drawback that it requires a forever
> running daemon, and normally you only need it during bootup, and after
> that, you need it seldomly, when you plug in an usb disk, a mouse or
> similar.

But  the  interactive part is where the arguments of the systemd disease
comes from. There simple scripts have to adapt and make it  possible  to
use Linux in this too fast changing world.

> > The  KISS  way  I  imagined  would be this way: nldev not just
> > executing mdev, but it caching the configuration file for the
> > speedup. That*s what smdev tries to do but it does not implement all
> > of the busybox mdev.conf syntax. 
> 
> Here is what I have been thinking, and what came out of a long
> discussion on busybox mailing list.
> 
> Implement the netlink daemon as a socket activator. When there is a new
> event, fork/exec the handler and pass over the socket.
> 
> The handler reads the events from the socket and handles them. Set a
> timout on the socket and exit when there are no events withing one or two
> seconds. The socket activator takes over again.

Processes and sockets are cheap in Linux. There is no need for this com‐
plexity. Keep it simple.

You  will  have  more problems by adapting to the dbus herd. There long‐
running daemons are the norm. But that’s a different topic  which  might
get  addressed  in  the future either by the demise of the developers of
that crappy services or by having yet another important  part  of  Linux
(Bluetooth anyone?) depending on such a false design.

> I have this implemented, based on nldev code:
> http://git.alpinelinux.org/cgit/user/ncopa/nlplug/tree/

These are simple changes. You could have sent me the patch for nldev in‐
stead of splitting it up again and creating  a  blackhole  in  juridical
questions  by  adding  a  half‐baked  license  line. Now there’s another
project. As I recommended, all of this should be now done under smdev at
suckless.org.

> > What needs to be done for solving this udev problem once and for all
> > and have a modular replacement:
> > * Send in patches to smdev to have the complete busybox syntax
> > mdev.conf implemented so all the mdev.conf examples out there can be
> > reused.
> > * Create a repository with all kind of mdev scripts so they can be
> > reused.
> > * Rip off libudev-compat from vdev and have it a separate package, if
> > needed.
> 
> Alternatively, we could make an alternative xorg config backend. It
> does not look too difficult:
> http://cgit.freedesktop.org/xorg/xserver/tree/config/udev.c

Wrong.  Be  liberal in what you produce and conservative in what you ac‐
cept.  And, with the work of vdev and the rip off  nearly  all  work  is
done. Just add all the pieces and you are done in one productive day.

> > I  invite all interested developers to join suckless in its strive
> > for a better KISS future and send  patches  to  hackers@suckless.org
> > for  the smdev[0]  project or to implement the libudev*compat or join
> > discussions on dev@suckless.org.
> 
> I think it might make sense to do the mdev.conf parsing and event
> handlig from either busybox or toybox. Those toolboxes also have things
> like modprobe and blkid, which should make it possible to load
> modaliases without fork/exec and find root=... on /proc/cmdline. (very
> useful from initramfs)

There is sbase in suckless.org, which does much of that too. Keep every‐
thing simple and don’t add dependencies you can just assume from wherev‐
er they come from. You will join the complexity spiral.


Sincerely,

Christoph Lohmann



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Jude Nelson
Details
Message ID
<CAFsQEP0pvphGo6uNjPCF0fn3RZKNduMThfVgr9bCOVn-RNLtkA@mail.gmail.com>
In-Reply-To
<20150727120744.4ffdfa0b@ncopa-desktop.alpinelinux.org> (view parent)
Sender timestamp
1438268611
DKIM signature
missing
Download raw message
Hi Natanael,

On Mon, Jul 27, 2015 at 6:07 AM, Natanael Copa <ncopa@alpinelinux.org>
wrote:

> On Sun, 26 Jul 2015 12:10:26 +0200
> Christoph Lohmann <20h@r-36.net> wrote:
>
> However, using netlink has the drawback that it requires a forever
> running daemon, and normally you only need it during bootup, and after
> that, you need it seldomly, when you plug in an usb disk, a mouse or
> similar.
>

You may be interested to know that vdevd offers a command-line switch to
have it synchronously "patch" /dev and exit.  It finds the set of devices
that the kernel knows about but are not represented in /dev and creates
device nodes and metadata for them, and it removes the set of device nodes
and metadata from /dev that are no longer recognized by the kernel.  The
motivating use-case is to help server admins maintain a /dev that
represents the set of available hardware without having to run a hotplug
daemon in the background.


> I am interested in libudev comaptibility, but have not looked too much
> into it. I don't need full ABI compatibility, but it would be very
> helpful to make it easy to port current applictions who uses libudev.
>

In my experience creating vdev, I have found that the key challenge in
preserving libudev compatibility happy isn't API or ABI compatibility, but
is in generating the data they expect to pull from /run/udev and device
events.  This is because the metadata in /run/udev and the device packets
sent by udev contain considerably more information than is found in the
kernel's uevent (for example, data returned by privileged ioctls on
devices, data queried from the hardware database, etc.).  This information
needs to be gathered and propagated to achieve full compatibility--either
by the device manager or by some other agent.  Unfortunately, the
dependence on privileged ioctls precludes most libudev clients from loading
the requisite information themselves; a sufficiently privileged program
needs to load it and make it available on their behalf.  Currently, that
role is filled by udevd (and vdevd), but I could imagine an alternative
design where libudev-compat proactively gathered and cached such
information by periodically running a setuid helper built for the purpose.

Thanks,
Jude