Photonicat Debian + Virtual Machine

Follow this guide to install and configure libvirt on Debian, add a user to the libvirt group, and set up an OpenWRT VM using libvirtd. which allows both debain and openwrt use simultaneously.

The graph shows the idea of how it is running and may have some difference between this wiki and the graph.

Debian will have full USB Wifi and 5G USB Modem Access.

Install libvirtd in Debian

Install libvirtd using APT (require root privilege):

apt-get install libvirt-daemon libvirt-clients bridge-utils

Add user to libvirtd

This operation require root privilege.

#Replace photonicat to your user name if you changed user name.
usermod -a -G libvirt photonicat

Then reboot to apply changes.

Example: Run OpenWRT on libvirtd

The operations below require root privilege.

Setup Network

Add bridge configurations below to /etc/network/interfaces:

iface eth0 inet manual
iface eth1 inet manual

auto br-wan
iface br-wan inet dhcp
  bridge_ports eth0
  bridge_fd 0

auto br-lan
iface br-lan inet static
  bridge_ports eth1
  address 172.16.0.2
  netmask 255.255.255.0
  bridge_fd 0

Then reboot to apply changes.

Prepare kernel & rootfs image

mkdir -p /var/lib/libvirt/boot/openwrt
cd /var/lib/libvirt/boot/openwrt
wget https://downloads.openwrt.org/releases/23.05.5/targets/armsr/armv8/openwrt-23.05.5-armsr-armv8-generic-kernel.bin
wget https://downloads.openwrt.org/releases/23.05.5/targets/armsr/armv8/openwrt-23.05.5-armsr-armv8-generic-initramfs-kernel.bin

cd /var/lib/libvirt/images
wget https://downloads.openwrt.org/releases/23.05.5/targets/armsr/armv8/openwrt-23.05.5-armsr-armv8-generic-ext4-rootfs.img.gz
gunzip openwrt-23.05.5-armsr-armv8-generic-ext4-rootfs.img.gz

Setup VM

Create file /tmp/openwrt.xml with contents below:

<domain type='kvm' id='1'>
  <name>OpenWRT</name>
  <uuid>8392d747-f6ae-469c-8024-647949f59ece</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://libosinfo.org/linux/2022"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='aarch64' machine='virt-7.2'>hvm</type>
    <kernel>/var/lib/libvirt/boot/openwrt/openwrt-23.05.5-armsr-armv8-generic-kernel.bin</kernel>
    <initrd>/var/lib/libvirt/boot/openwrt/openwrt-23.05.5-armsr-armv8-generic-initramfs-kernel.bin</initrd>
    <cmdline>root=fe00</cmdline>
    <boot dev='hd'/>
  </os>
  <features>
    <gic version='3'/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/openwrt-23.05.5-armsr-armv8-generic-ext4-rootfs.img' index='1'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
     <target chassis='3' port='0xa'/>
      <alias name='ci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <mode name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <alias name='pci.6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <alias name='pci.7'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0xf'/>
      <alias name='pci.8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x10'/>
      <alias name='pci.9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x11'/>
      <alias name='pi.10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x12'/>
      <alias name='pci.11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x13'/>
      <alias name='pci.12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='13' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='13' port='0x14'/>
      <alias name='pci.13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='14' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='14' port='0x15'/>
      <alias name='pci.14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:54:c1:e7'/>
      <source bridge='br-wan'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:2c:86:f2'/>
      <source bridge='br-lan'/>
      <target dev='vnet1'/>
      <model type='virtio'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </interface>
    <interface type="network">
      <mac address="52:54:00:2c:18:cc"/>
      <source network="default"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='system-serial' port='0'>
        <model name='pl011'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-OpenWRT/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <audio id='1' type='none'/>
   <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x2c7c'/>
        <product id='0x0900'/>
        <address bus='7' device='2'/>
      </source>
      <alias name='hostdev0'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x0e8d'/>
        <product id='0x7961'/>
        <address bus='2' device='3'/>
      </source>
      <alias name='hostdev1'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+64055:+64055</label>
    <imagelabel>+64055:+64055</imagelabel>
  </seclabel>
</domain>

Import VM:

virsh
connect qemu:///system
create /tmp/openwrt.xml

Setup Guest

Use virt-manager or other frontend to login VM, set eth1 to 172.16.0.1. Access OpenWRT with 172.16.0.1 by SSH or browser.

Change VM configuration

virsh
connect qemu:///system
edit OpenWRT

Change system disk

If you want to change system disk image, an ext4 filesystem with or without partition image:

<source file='/var/lib/libvirt/images/openwrt-23.05.5-armsr-armv8-generic-ext4-rootfs.img' index='1'/>

Change file property to the system disk image you want.

If your system disk image is just a filesystem image (no partition table), set cmdline section to root=fe00: (like the one you downloaded: https://downloads.openwrt.org/releases/23.05.5/targets/armsr/armv8/openwrt-23.05.5-armsr-armv8-generic-ext4-rootfs.img.gz)

<cmdline>root=fe00</cmdline>

If your system disk image has a partition table, you should set cmdline section to setup rootfs device (e.g: /dev/vda1):

<cmdline>root=/dev/vda1</cmdline>

Use Quectel RM500U in VM

Install kernel modules:

opkg update
opkg install kmod-usb-net-cdc-eem kmod-usb-net-cdc-ether kmod-usb-net-cdc-mbim kmod-usb-net-cdc-ncm kmod-usb-net-rndis kmod-usb-serial-option

Modem Manager cannot work well on Quectel RM500U. Remove it if installed.

opkg remove luci-proto-modemmanager
opkg remove modemmanager

Setup modem with AT commands:

echo -en 'AT+QCFG="usbnet",1\r\n' >/dev/ttyUSB2
echo -en 'AT+QCFG="nat",0\r\n' >/dev/ttyUSB2
echo -en 'AT+QCFG="ethernet",1\r\n' >/dev/ttyUSB2
echo -en 'AT+QNETDEVCTL=1,3,1\r\n' >/dev/ttyUSB2
echo -en 'AT+CFUN=1,1\r\n' >/dev/ttyUSB2
poweroff

Start VM again to apply changes.

Set interface usb0 to DHCP client, it will connect to internet automatically if your SIM card do not need APN, username or password.

Sample Image

OpenWRT:
LAN IP: 172.16.0.1
User: root
Password: photonicat

Potential Issues and fixes

• USB device path may change in each reboot. User may need to detect USB device path and update VM configuration by itself.
• Some LTE modems like Quectel RM500U may not work if VM reboots. Need to stop VM completely (go inside openwrt to run:

  poweroff

  ) then start VM again.