Storage and Disk Management

MicroVM uses virtio-blk-device for block storage. This article covers disk image management, multiple drives, and I/O optimization.

Storage Types

Raw Disk Image

The simplest format with the best performance:

# Create a 10GB raw image
qemu-img create -f raw data.img 10G

# Check the info
qemu-img info data.img

QCOW2 Image

A format with snapshot and thin provisioning features:

# Create a QCOW2 image
qemu-img create -f qcow2 data.qcow2 10G

# With compression
qemu-img create -f qcow2 -o compression_type=zstd data.qcow2 10G

Format Comparison

Multiple Drives

MicroVM supports multiple virtio-blk-devices (limited to a maximum of 8 virtio devices in total):

# Rootfs + Data disk
qemu-system-x86_64 \
  -M microvm -enable-kvm -cpu host -m 256m -smp 2 \
  -kernel vmlinuz \
  -append "console=ttyS0 root=/dev/vda rw" \
  -drive id=rootfs,file=rootfs.img,format=raw,if=none \
  -device virtio-blk-device,drive=rootfs \
  -drive id=data,file=data.img,format=raw,if=none \
  -device virtio-blk-device,drive=data \
  -nographic -serial stdio

Inside the Guest

# Check the block devices
lsblk

# Output:
# NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
# vda    252:0    0   2G  0 disk
# └─vda1 252:1    0   2G  0 part /
# vdb    252:16   0  10G  0 disk

# Mount the data disk
mkdir -p /data
mount /dev/vdb /data

# or with a fresh filesystem
mkfs.ext4 /dev/vdb
mount /dev/vdb /data

# Add to fstab
echo "/dev/vdb /data ext4 defaults,noatime 0 2" >> /etc/fstab

Host Disk Passthrough

Access a physical host disk directly from the MicroVM:

# Pass through the whole disk
qemu-system-x86_64 \
  -M microvm -enable-kvm -cpu host -m 512m \
  -kernel vmlinuz \
  -append "console=ttyS0 root=/dev/vda rw" \
  -drive id=rootfs,file=rootfs.img,format=raw,if=none \
  -device virtio-blk-device,drive=rootfs \
  -drive id=hostdisk,file=/dev/sdb,format=raw,if=none \
  -device virtio-blk-device,drive=hostdisk \
  -nographic -serial stdio

Passthrough via LVM

# Create an LVM volume for the VM
sudo lvcreate -L 10G -n vm-data vg0

# Pass through to QEMU
-drive id=lvmdata,file=/dev/vg0/vm-data,format=raw,if=none \
-device virtio-blk-device,drive=lvmdata

NFS Mount (Network Storage)

Mount an NFS share from the host into the guest:

# Set up an NFS server on the host
sudo apt install nfs-kernel-server
echo "/srv/nfs 10.0.0.0/24(rw,sync,no_subtree_check)" | sudo tee -a /etc/exports
sudo exportfs -ra

# In the guest, mount the NFS share
apt install nfs-common  # Debian
mount -t nfs 10.0.0.1:/srv/nfs /mnt/nfs

Overlay Filesystem

Use an overlay for an efficient rootfs (read-only base image):

# Create the base image (read-only)
qemu-img create -f raw base-rootfs.img 1G
mkfs.ext4 base-rootfs.img
# ... install the OS into base-rootfs.img

# Create the overlay image
qemu-img create -f raw overlay-rootfs.img 2G

# In the guest, mount with overlay
mount -t overlay overlay -o \
  lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work \
  /merged

I/O Optimization

Cache Mode

# None (direct I/O) - best performance
-drive id=rootfs,file=rootfs.img,format=raw,if=none,cache=none

# Writeback - buffering on the host
-drive id=rootfs,file=rootfs.img,format=raw,if=none,cache=writeback

# Writethrough - safe, sync writes
-drive id=rootfs,file=rootfs.img,format=raw,if=none,cache=writethrough

AIO Mode

# Linux AIO for async I/O
-drive id=rootfs,file=rootfs.img,format=raw,if=none,aio=linux

# io_uring (QEMU >= 6.0, Linux >= 5.1)
-drive id=rootfs,file=rootfs.img,format=raw,if=none,aio=io_uring

Discard/TRIM

# Enable discard for SSD thin provisioning
-drive id=rootfs,file=rootfs.img,format=raw,if=none,discard=unmap

# Or QCOW2
-drive id=rootfs,file=rootfs.qcow2,format=qcow2,if=none,discard=unmap

Optimization Script

#!/bin/bash
# Script: run-microvm-optimized.sh

KERNEL="./vmlinuz"
ROOTFS="./microvm-rootfs.img"

exec qemu-system-x86_64 \
  -M microvm,pit=off,pic=off,rtc=off \
  -enable-kvm \
  -cpu host \
  -m 512m \
  -smp 4 \
  -kernel "$KERNEL" \
  -append "console=ttyS0 root=/dev/vda rw elevator=noop" \
  -drive id=rootfs,file="$ROOTFS",format=raw,if=none,cache=none,aio=linux,discard=unmap \
  -device virtio-blk-device,drive=rootfs \
  -netdev tap,id=net0,script=no,downscript=no \
  -device virtio-net-device,netdev=net0 \
  -nographic -serial stdio

Backup and Restore

Backup Raw Image

# Live backup with dd
dd if=/dev/vda of=backup.img bs=4M status=progress

# Or with qemu-img
qemu-img convert -f raw -O raw rootfs.img backup.img

# Compressed backup
dd if=rootfs.img bs=4M | gzip > rootfs.img.gz

Backup QCOW2

# Export to raw for backup
qemu-img convert -f qcow2 -O raw rootfs.qcow2 backup.img

# or back up directly
cp rootfs.qcow2 rootfs.backup.qcow2