MIME-Version: 1.0
Message-ID: <trinity-34aabe98-ab3d-44df-8787-6973091d0be3-1720185124721@3c-app-mailcom-bs03>
From: daggs <daggs@gmx.com>
To: Natanael Copa <ncopa@alpinelinux.org>
Cc: ~alpine/users@lists.alpinelinux.org
Subject: Re: unable to write to dev node
Content-Type: text/plain; charset=UTF-8
Date: Fri, 5 Jul 2024 15:12:04 +0200
Importance: normal
Sensitivity: Normal
In-Reply-To: <20240705092544.73dcb8c3@ncopa-desktop>
References: <trinity-ca06feea-8538-4eff-a897-19e97bc49431-1720102545939@3c-app-mailcom-bs08>
 <20240704174637.78fb7052@ncopa-desktop>
 <trinity-ca8c4581-d670-44eb-a37f-2e45b91b7561-1720109874963@3c-app-mailcom-bs04>
 <20240705092544.73dcb8c3@ncopa-desktop>
UI-OutboundReport: notjunk:1;M01:P0:EmqXUPP0C18=;fW6i0WD4DnlMkyEtLtCXup5S2LO
 wlSvbJ6e9ebK7Ig7CVG7r81VRRUCUnE48W0pbRO2eUPD8gWzDdSN9JZOEBLyAw8j0SzpnFYL0
 W/oEuLF0Ip2ybTyy2YZMg1+DXKBW9TBMjV2AzYKzygxNVM/R6Lhfgr3NkFfAMIHQmz8P96HyE
 gdKI/v2gpPTyuxUFX8LdKs8A/1Bodxa9kATJgaXhrFO7cS44vBSaK0+3QZwa3ac+Sw/blu9B7
 /0tePPYo0d+YqlwVbvfY08sdnu0ZWb+GvHsmFzJijGuk8CqxdVcass5AHzIZb1Vrh9khHfDzp
 hkN3DGLUBa7pA1vY8K1ekD0IasHOf2s9VdpKhAYqAzNGqNzk1iwVp4N+yXQbnhJr+25z12kh0
 Wvtpe4RpHPd8ltI2ktlza4NMtxMCi/x0eGMZoxeVqntm3ssZJdIksmInKKavVi2wRF+gD2i7t
 0MvHgMH4h1cSGpnWFSRC+C5fdhIyfsH/r8rfpVa/ElyqvYBh1+QYANa+hQ20Y5LGGj3x00Bbk
 iDl3/k+VpTyJWrjtBB+xNe/9TLY1krkq5/bIDhoQ7lxEXqjJmqVqYpGQBwm+8uviNQHhsGmJG
 RHqs31F8eWT+mU5AnhkKaoG86Bs7n4t81AlZPntFHIJEAAzksDRzi3Be2gZ29lCsVHRzQmpga
 E624aJ73mPNF+Qt1MItPjQOYpjhZNWbUIFjjmJNnyt1vGs+Jr+X0SuV9zyYHDu4=
Content-Transfer-Encoding: quoted-printable

> > > > Greeting,
> > > >
> > > > I'm trying to debug a bug under alpine linux when it comes to sess=
ion based libvirt vms.
> > > > when starting a vm that has a virt nic binded to a bridge, I get t=
his error: Unable to create tap device vnet0: Operation not permitted
> > > > I've looked into the code of libvirt and narrowed it down to this =
func: virNetDevTapCreate
> > > > I've taken the relevant code to a side file for testing, there is =
the code I use:
> > > > # include <linux/if.h>
> > > > # include <linux/if_tun.h>    /* IFF_TUN, IFF_NO_PI */
> > > > #include <stddef.h>
> > > > #include <stdlib.h>
> > > > # include <sys/ioctl.h>
> > > > #include <fcntl.h>
> > > > #include <errno.h>
> > > > #include <stdio.h>
> > > >
> > > >
> > > > enum {
> > > >    VIR_NETDEV_TAP_CREATE_NONE =3D 0,
> > > >    /* Bring the interface up */
> > > >    VIR_NETDEV_TAP_CREATE_IFUP               =3D 1 << 0,
> > > >    /* Enable IFF_VNET_HDR on the tap device */
> > > >    VIR_NETDEV_TAP_CREATE_VNET_HDR           =3D 1 << 1,
> > > >    /* Set this interface's MAC as the bridge's MAC address */
> > > >    VIR_NETDEV_TAP_CREATE_USE_MAC_FOR_BRIDGE =3D 1 << 2,
> > > >    /* The device will persist after the file descriptor is closed =
*/
> > > >    VIR_NETDEV_TAP_CREATE_PERSIST            =3D 1 << 3,
> > > >    /* The device is allowed to exist before creation */
> > > >    VIR_NETDEV_TAP_CREATE_ALLOW_EXISTING     =3D 1 << 4,
> > > > };
> > > >
> > > > int main()
> > > > {
> > > > 	int fd;
> > > >         char *tunpath =3D "/dev/net/tun";
> > > > 	size_t tapfdSize =3D 1;
> > > >         struct ifreq ifr =3D { 0 };
> > > > 	unsigned int flags =3D VIR_NETDEV_TAP_CREATE_IFUP;
> > > > 	if (1)
> > > > 		flags |=3D VIR_NETDEV_TAP_CREATE_VNET_HDR;
> > > >
> > > >         if ((fd =3D open(tunpath, O_RDWR)) < 0) {
> > > >             perror("Unable to open, is tun module loaded?");
> > > >             exit(1);
> > > >         }
> > > >
> > > >         snprintf(ifr.ifr_name, 5, "vnet%d", 0);
> > > >         ifr.ifr_flags =3D IFF_TAP | IFF_NO_PI;
> > > >         /* If tapfdSize is greater than one, request multiqueue */
> > > >         if (tapfdSize > 1)
> > > >             ifr.ifr_flags |=3D IFF_MULTI_QUEUE;
> > > >
> > > >         if (flags &  VIR_NETDEV_TAP_CREATE_VNET_HDR)
> > > >             ifr.ifr_flags |=3D IFF_VNET_HDR;
> > > >
> > > >         if (ioctl(fd, TUNSETIFF, &ifr) < 0) {
> > > >             perror("Unable to create tap device");
> > > >         }
> > > >
> > > >         return 0;
> > > > }
> > > >
> > > > it compiles fine and works under user root.
> > > > I have a user named foo which I use for the sessioned vm, looking =
at /dev/net/tun's permissions, I see this:
> > > > $ ll /dev/net/tun
> > > > crw-rw-rw- 1 root netdev 10, 200 Jul  4 15:52 /dev/net/tun
> > > >
> > > > so I added foo to netdev group, now it has the following id output=
: uid=3D1002(foo) gid=3D1002(foo) groups=3D1002(foo),28(netdev),34(kvm),36=
(qemu),102(libvirt)
> > > > and ran the code again, I'm getting the same error.
> > > > I went to the libvirt community and one of the devs tried to help =
me with it, he concluded that there is something wrong in the alpine becau=
se it works in fedora.
> > > > in contrast, /dev/null has the same permissions as /dev/net/tun bu=
t the group is root and I can write to it as user foo.
> > > >
> > > > any ideas what I am missing?
> > >
> > > Can you run your app under strace? To show which syscall that fails.
> > >
> > $ strace ./test
> > execve("./test", ["./test"], 0x7ffd948179e0 /* 17 vars */) =3D 0
> > arch_prctl(ARCH_SET_FS, 0x7f9eff1a1b28) =3D 0
> > set_tid_address(0x7f9eff1a1f90)         =3D 3922
> > brk(NULL)                               =3D 0x559659075000
> > brk(0x559659077000)                     =3D 0x559659077000
> > mmap(0x559659075000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYM=
OUS, -1, 0) =3D 0x559659075000
> > mprotect(0x7f9eff19e000, 4096, PROT_READ) =3D 0
> > mprotect(0x5596579ff000, 4096, PROT_READ) =3D 0
> > open("/dev/net/tun", O_RDWR|O_LARGEFILE) =3D 3
>
> This means the permissions to /dev/net/tun and the group etc works.
>
> > ioctl(3, TUNSETIFF, 0x7ffd72891cf0)     =3D -1 EPERM (Operation not pe=
rmitted)
>
> So it is the ioctl that fails, for some reason.
>
> Is the kernel module `tun` loaded`?
yes it is.
is there a way to compile the kernel faster so I can debug it from within?

>
> > writev(2, [{iov_base=3D"", iov_len=3D0}, {iov_base=3D"Unable to create=
 tap device", iov_len=3D27}], 2Unable to create tap device) =3D 27
> > writev(2, [{iov_base=3D"", iov_len=3D0}, {iov_base=3D":", iov_len=3D1}=
], 2:) =3D 1
> > writev(2, [{iov_base=3D"", iov_len=3D0}, {iov_base=3D" ", iov_len=3D1}=
], 2 ) =3D 1
> > writev(2, [{iov_base=3D"", iov_len=3D0}, {iov_base=3D"Operation not pe=
rmitted", iov_len=3D23}], 2Operation not permitted) =3D 23
> > writev(2, [{iov_base=3D"", iov_len=3D0}, {iov_base=3D"\n", iov_len=3D1=
}], 2
> > ) =3D 1
> > exit_group(0)                           =3D ?
> > +++ exited with 0 +++
> >
> >
> > > Do you run this under docker? if so it might be libseccomp that is
> > > causing problems.
> > no, I'm running inside a libvirt qemu vm. e.g. nested vm
> >
> > >
> > > -nc
> > >
> > > >
> > > > Thanks,
> > > >
> > > > Dagg
> > >
> > >
>
>