Subject: Alpine 3.18 in aarch64 QEMU/KVM VM, has ethernet interface, but no communication results?

From: Jibun no Kage <[email-masked]>
Date: Wed, 28 Jun 2023 17:38:53 -0700

Alpine 3.18 in aarch64 QEMU/KVM VM, has ethernet interface, but no
communication results?  The eth0 interface exists, I can ping it if I
assign a static IP address via /etc/network/interfaces.  And the route
table has a default gateway as expected.  But no communication results?

On the same aarch64 system, VMs running say Debian 11, Debian 12, and
Ubuntu 22.04 Jammy Jelly fish work as expected, in reference to network
communication.  So I believe I have QEM/KVM installed correctly, I happen
to be using a bridge interface to the physical NIC on the SBC that is
aarch64 based.  The SBC is an IndieDroid Nova.

Any help is greatly appreciated.  Detail information...

Core OS...
# cat /etc/os-release
PRETTY_NAME="Armbian 23.05.0-trunk jammy"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="
https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

# uname -a
Linux indiedroid-nova 5.10.110-rockchip-rk3588-indiedroid #1 SMP PREEMPT
Thu Mar 9 02:22:35 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

QEMU/KVM
# /usr/bin/qemu-system-aarch64 --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.11)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
# kvm --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.11)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.42c8c028b51b       no              enP4p65s0
                                                        vnet5
docker0         8000.024258701c19       no

# virsh list
 Id   Name          State
-----------------------------
 6    Alpine-3.18   running

# virsh dumpxml Alpine-3.18
<domain type='kvm' id='6'>
  <name>Alpine-3.18</name>
  <uuid>bbb0252d-062f-4e05-bb14-90f8355730aa</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="
http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://alpinelinux.org/alpinelinux/3.18"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='aarch64' machine='virt-6.2'>hvm</type>
    <loader readonly='yes'
type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/Alpine-3.18_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <gic version='3'/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-aarch64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/Alpine-3.18.qcow2' index='2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00'
function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/ttt/Downloads/alpine-standard-3.18.2-aarch64.iso'
index='1'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <readonly/>
      <boot order='1'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00'
function='0x0'/>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <alias name='pci.6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <alias name='pci.7'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0xf'/>
      <alias name='pci.8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x10'/>
      <alias name='pci.9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x11'/>
      <alias name='pci.10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x1'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x12'/>
      <alias name='pci.11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x2'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x13'/>
      <alias name='pci.12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x3'/>
    </controller>
    <controller type='pci' index='13' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='13' port='0x14'/>
      <alias name='pci.13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x4'/>
    </controller>
    <controller type='pci' index='14' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='14' port='0x15'/>
      <alias name='pci.14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00'
function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:62:b1:4b'/>
      <source bridge='br0'/>
      <target dev='vnet5'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='system-serial' port='0'>
        <model name='pl011'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind'
path='/var/lib/libvirt/qemu/channel/target/domain-6-Alpine-3.18/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0'
state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <audio id='1' type='none'/>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00'
function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+64055:+109</label>
    <imagelabel>+64055:+109</imagelabel>
  </seclabel>
</domain>

# ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:62:B1:4B
          inet addr:192.168.1.211  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe62:b14b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:35010 errors:0 dropped:10 overruns:0 frame:0
          TX packets:983 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2100600 (2.0 MiB)  TX bytes:296646 (289.6 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1432 (1.3 KiB)  TX bytes:1432 (1.3 KiB)

# ip route
default via 192.168.1.2 dev eth0  metric 1 onlink
192.168.1.0/24 dev eth0 scope link  src 192.168.1.211

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
0.0.0.0         192.168.1.2     0.0.0.0         UG    1      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen
1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP qlen 1000
    link/ether 52:54:00:62:b1:4b brd ff:ff:ff:ff:ff:ff

# ping 192.168.1.211 -c 4
PING 192.168.1.211 (192.168.1.211): 56 data bytes
64 bytes from 192.168.1.211: seq=0 ttl=64 time=0.130 ms
64 bytes from 192.168.1.211: seq=1 ttl=64 time=0.508 ms
64 bytes from 192.168.1.211: seq=2 ttl=64 time=0.235 ms
64 bytes from 192.168.1.211: seq=3 ttl=64 time=0.198 ms

--- 192.168.1.211 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.130/0.267/0.508 ms

# ping 192.168.1.2 -c 4
PING 192.168.1.2 (192.168.1.2): 56 data bytes

--- 192.168.1.2 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

# cat /etc/resolv.conf
nameserver 192.168.1.3

# cat /var/log/dmesg | grep INET
[    0.191443] NET: Registered PF_INET protocol family
[    3.245211] NET: Registered PF_INET6 protocol family

# cat /var/log/dmesg | grep net
localhost:/etc/network# cat /etc/network/interfaces
auto eth0
#iface eth0 inet dhcp
iface eth0 inet static
        address 192.168.1.211/24
        gateway 192.168.1.2

# service networking restart
 * Stopping networking ...
 *   lo ...
 [ ok ]
 *   eth0 ...
 [ ok ]
 * Starting networking ...
 *   lo ...
 [ ok ]
 *   eth0 ...
 [ ok ]

---

From: Natanael Copa <[email-masked]>
Date: Thu, 29 Jun 2023 12:18:30 +0200

On Wed, 28 Jun 2023 17:38:53 -0700

> br0             8000.42c8c028b51b       no              enP4p65s0

What ip address has br0?

Can you ping 192.168.1.2 from the host?

Would it be possibele that there are other hosts that has the address
192.168.1.211 (eg ip address conflict)?

Could it be other machines connected to br0 that has mac
address='52:54:00:62:b1:4b'?

Can you share the dumpxml from one of the working debian or ubuntu
machines?

---

From: "Jibun no Kage ." <[email-masked]>
Date: Thu, 29 Jun 2023 10:04:25 -0700

Thanks for the reply, I did some more testing, and it appears the 
direct-bridge configuration is now broken.  Something has broken the br0 
configuration as it is now, it was working, otherwise the Debian 11 and 
12 NetInst installs would have failed.

When I happen to install alpine in a VM, and it could not find the DHCP 
server, so I assigned a static IP, and it still was not communicating.

Recreated a typical NAT based virtual network under QEMU/KVM, it works, 
all VMs, including the alpine VM, which communicates as expected.  So I 
am going to have to back track to find what has broken the br0 bridge on 
my SBC device.  There have been several updates, etc. between the Debian 
VMs built and the Alpine VM.  Really odd.

-JnK

---

From: "Jibun no Kage ." <[email-masked]>
Date: Sun, 2 Jul 2023 10:26:10 -0700

Just to close out this thread...

I rebuilt the system, re-installed QEMU/KVM, recreated the bridge 
configuration.  Both Debian 11/12 based VMs work as expected, and now 
Alpine works as expected in reference to the odd connectivity issue as 
noted below.  As some point the bridge configuration was broken in 
between the time I created the Debian 11/12 VMs and then the Alpine VM. 
There was other work done on the host system, system updates, etc. 
Docker was installed at one point.  I am now retracing the steps to see 
if I can pin point the specific step that broke the bridge configuration.

To all those that replied, etc.  Thanks.  Notice to see, or read rather, 
there is active support on a mailing list these days!

-JnK