Subject: [alpine-user] liblxc segfaults when trying to start unprivileged container

From: Daniel Kulesz <[email-masked]>
Date: Mon, 4 Mar 2019 23:56:59 +0100

Hi folks,

I tried setting up lxc in unprivileged mode on Alpine 3.9.2 (amd64), but every time I try to start any newly created container liblxc just segfaults like this (taken from dmesg):

[   41.711333] 3[2590]: segfault at 0 ip 00007f20c35d9812 sp 00007ffd82b61740 error 4 in liblxc.so.1.5.0[7f20c35c4000+71000]
[   41.711346] Code: c7 44 24 08 00 00 00 00 48 89 c3 4a 8d 04 20 48 89 04 24 c6 00 00 45 31 ed 48 8b 45 30 44 89 6c 24 0c 4e 8d 34 ed 00 00 00 00 <4e> 8b 24 e8 4d 85 e4 0f 84 84 01 00 00 48 89 de 4c 89 e7 e8 d8 f3

I've setup cgroups and uid/gid mappings using shadow-uidmap and I don't have any networking configured yet.

Here is the user's lxc configuration file:

localhost:~$ cat .config/lxc/default.conf 
lxc.include = /etc/lxc/default.conf
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

And here's the output of lxc-checkconfig:

localhost:~$ lxc-checkconfig 
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled

--- Control groups ---
Cgroups: enabled

Cgroup v1 mount points: 
/sys/fs/cgroup/openrc
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/net_prio
/sys/fs/cgroup/pids

Cgroup v2 mount points: 
/sys/fs/cgroup/unified

Cgroup v1 systemd controller: missing
Cgroup v1 clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled, not loaded
Macvlan: enabled, not loaded
Vlan: enabled, not loaded
Bridges: enabled, loaded
Advanced netfilter: enabled, not loaded
CONFIG_NF_NAT_IPV4: enabled, not loaded
CONFIG_NF_NAT_IPV6: enabled, not loaded
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, not loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded
FUSE (for use with lxcfs): enabled, not loaded

--- Checkpoint/Restore ---
checkpoint restore: missing
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities: 

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

Privileged containers work just fine.

Any ideas?

Cheers, Daniel

---
Unsubscribe:  [email-masked]
Help:         [email-masked]
---

---

From: paul gauret <[email-masked]>
Date: Fri, 29 Mar 2019 08:43:27 +0000 (UTC)

Hmm I have LXC running on 3.9.2 with unprivileged containers without seeing these messages.
A couple of differences in my setup though:- I'm using a ZFS backend- I'm not mounting cgroups fs, only cgroups2
Not sure which user you're using to start your containers, in my case doing everything with root.

    On Monday, March 4, 2019, 11:57:26 PM GMT+1, Daniel Kulesz <[email-masked]> wrote:  
 
 Hi folks,

--- Namespaces ---
Namespaces: enabled

--- Control groups ---
Cgroups: enabled

Cgroup v1 mount points: 
/sys/fs/cgroup/openrc
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/net_prio
/sys/fs/cgroup/pids

Cgroup v2 mount points: 
/sys/fs/cgroup/unified

--- Checkpoint/Restore ---

Any ideas?

Cheers, Daniel

---

---

From: Chloe Kudryavtsev <[email-masked]>
Date: Fri, 29 Mar 2019 13:21:21 -0400

On 3/29/2019 4:43 AM, paul gauret wrote:
> in my case doing everything with root.

We're missing kernel.unprivileged_userns_clone for whatever reason.
You have to enable that to run things as non-root (which I suspect 
you're trying to do).

Spun up a VM to test: unprivileged containers are just fine as root, but 
not as a user (in the latter case you get a segfault - likely because an 
unprivileged user is trying to userns clone without having the right to :) )

I suppose the question now becomes "why are we missing that option".

In the interim, feel free to have root-owned unprivileged containers 
(you can give root subuids just like everywhere else, and everything 
ends up running as UID 100000 or whatever you use).

---

---

From: Daniel Kulesz <[email-masked]>
Date: Sun, 16 Jun 2019 17:34:09 +0200

Hi Chloe,

thank you for pointing this out (and sorry for taking so long to thank and answer you)! It seems there are various opinions whether enabling this option is insecure or not:

https://security.stackexchange.com/questions/209529/what-does-enabling-kernel-unprivileged-userns-clone-do

Would you argue that running unprivileged containers as root is more secure than enabling this option and running unprivileged containers as non-root?

Cheers, Daniel

P.S.
I found a promising tutorial for running unprivileged containers as root:

https://blog.benoitblanchon.fr/lxc-unprivileged-container/

On Fri, 29 Mar 2019 13:21:21 -0400

---

---

From: Chloe Kudryavtsev <[email-masked]>
Date: Sun, 16 Jun 2019 12:44:16 -0400

> Hi Chloe,

Heya.

> Would you argue that running unprivileged containers as root is more secure than enabling this option and running unprivileged containers as non-root?

That kind of depends - these are different things with different concerns:
1. Running subuid (unprivileged) containers as root.
2. Having unprivileged user namespaces enabled.

It's hard to actually claim having one is "more secure" than the other, 
since it'd be like comparing apples and oranges.
I personally prefer doing the former, through lxd, but that shouldn't 
necessarily dictate what you do.

However, to be pedantic, I'm going to answer your question as-asked.
Running unprivileged containers as a user is undeniably more secure (or, 
at least, of equivalent security) to running unprivileged containers as 
root.
The concern is not with the containers, but with the unprivileged userns 
option being enabled at all.

> https://blog.benoitblanchon.fr/lxc-unprivileged-container/

You run unprivileged containers as root the same way you run them as 
non-root, just while being root.
I'd also recommend following upstream's documentation [1] rather than 
arbitrary tutorials.

P.S.
In case any of the above is unclear, do feel free to ask for further 
explanations.
Having a general understanding of linux namespaces and suid mechanics 
should be enough, though.

Recommended reading (just in case it's sufficient without requiring any 
further explanation):
- setuid(2)
- namespaces(7), including "see also"
- https://lwn.net/Articles/531114/#series_index (especially part 5)
- https://lwn.net/Articles/543273/

[1]: 
https://linuxcontainers.org/lxc/getting-started/#creating-unprivileged-containers-as-root

---