Using QCOW2 backing files & overlays for creating libvirt/kvm vm base images.

I'm fascinated by and thankful for storage solutions supporting checkpoint technologies such as snapshotting. In essence when snapshot is created, storage state is saved and future changes do not alter the state. Snapshots can save time and associated storage costs in containerization, virtualization and plain OS workflows. QCOW2 backing files & overlays can also be used for creating virtual machine disk base images.

Below is kickstart configuration for Fedora installation. It is losely based on minimum kickstart from try-fedora article, with customizations to storage and user access.

#version=DEVEL

# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'

# Use text mode install
text

# System language
lang en_US.UTF-8

# System timezone
timezone UTC --isUtc

# Network information
network --bootproto=dhcp --device=link --activate

# Firewall configuration
firewall --enabled --service=ssh

# Do not configure the X Window System
skipx

# System services
services --enabled=sshd,chronyd

# Lock root account
rootpw --lock

# Create user with authorized key for ssh access
user --groups=wheel,users --name="ansible"
# Contents of ansible.pub from 'ssh-keygen -b 2048 -t rsa -f ~/.ssh/ansible -q -N ""'
sshkey --username="ansible" "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCKnhHiwSGuV65LWLG3D6JubYpBzQyUlAXh1bo5VDgfhU1ixGnkX6ucm/aEKNwZr3DTMZSbW+pmuDq/7plsxa8BDvZWFMHUc+w990EikwNDRizYwywUdmc7C4v81FO++j92SGnclX8lC+V9N/4FzFZz8otT6z8xHyNzFtmACCjBiHhb0/KvQZ7NyPC2y0GY5pA2152qSujZ1+Vx5DhAJBJAw8WhiGu+T6DVHQ29D1KTZ3hTy+bNP8o499N79eQSoX2RMoW8+B93cFAbQqw/tknSX4wi3vatyLvnw6jjeYVkWT+WzktcCZ8Yb0ql3OTQWxCLTJloAmbjDsmL9rWRZuv tomas@uola"

# System bootloader configuration
bootloader --location=mbr --boot-drive=vda --append="net.ifnames=0 biosdevname=0"

# Format partitions created in %pre
part /boot --fstype=ext4 --label=boot --onpart=vda1
# create physical volume for LVM
part pv.1 --onpart=vda2
# create vg volume group
volgroup vg --pesize=4096 pv.1
# create root volume
logvol swap --label=swap --name=swap --vgname=vg --size=1000
logvol / --fstype=xfs --label=root --name=root --vgname=vg --grow --percent 100

# Use net install
url --url="https://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/"

# Enable repos
repo --name=os --baseurl="https://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/"
repo --name=updates --baseurl="https://download.fedoraproject.org/pub/fedora/linux/updates/$releasever/Everything/$basearch/"

# Packages
%packages
@^custom-environment
%end

# Pre install script
%pre --logfile=/root/ks-pre.log

# Setup disk partitions
echo "Creating partitions on /dev/vda"
parted --script /dev/vda \
    mklabel msdos \
    mkpart primary ext4 1MiB 300MiB \
    mkpart primary ext4 300MiB 100% \
    set 2 lvm on

kpartx /dev/vda

%end

# Post install script
%post --logfile=/root/ks-post.log

# Allow users in wheel group execute sudo commands without password
echo "Creating /etc/sudoers.d/wheel"
echo "%wheel ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/wheel

# Remove hostname
echo "" > /etc/hostname

%end

# Halt after installation
halt

Fedora base os image is created using network installation with a kickstart file.

🌈[base-image]$ virt-install \
--name f32-base \
--ram 2048 \
--os-type linux \
--os-variant fedora31 \
--graphics none \
--disk=./f32-base.vda.x86_64.qcow2,bus=virtio,format=qcow2,size=7 \
--location=https://download.fedoraproject.org/pub/fedora/linux/releases/32/Everything/x86_64/os \
--initrd-inject ./ks-base.cfg \
--extra-args="inst.ks=file:/ks-base.cfg console=ttyS0 net.ifnames=0 biosdevname=0"

f32-base vm should not be started after the installation as this will prevent storage file from being used as backing file.

Qemu-img command is used to create virtual disks with various configurations, including backed storage.

🌈[base-image]$ qemu-img create -b f32-base.vda.x86_64.qcow2 -f qcow2 saturn.vda.x86_64.qcow2 30G

Newly created backed images almost take no disk space with the benefit of having contents of the parent disk image.

🌈[base-image]$ qemu-img info saturn.vda.x86_64.qcow2 --backing-chain

image: saturn.vda.x86_64.qcow2
file format: qcow2
virtual size: 30 GiB (32212254720 bytes)
disk size: 196 KiB
cluster_size: 65536
backing file: f32-base.vda.x86_64.qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: f32-base.vda.x86_64.qcow2
file format: qcow2
virtual size: 7 GiB (7516192768 bytes)
disk size: 1.37 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

Virtual machine creation using backed storage.

🌈[base-image]$ virt-install \
--name saturn \
--ram 1024 \
--os-type linux \
--os-variant generic \
--graphics none \
--disk=./saturn.vda.x86_64.qcow2,bus=virtio,format=qcow2 \
--qemu-commandline="-netdev user,id=net0,hostname=saturn,hostfwd=tcp::22001-:22 -device pcnet,netdev=net0" \
--import

Virt-install by default connects to qemu:///session which uses user networking (slirp) networking backend. Extra qemu arguments are specified to enable tcp port forwarding from host 22001 to guest 22.

Base image is setup to prevent console access, instead ansible user has authorized ssh key, which will allow ssh access via forwarded tcp port 22001

🌈[base-image]$ ssh ansible@127.0.0.1 -p 22001 -i ~/.ssh/ansible
The authenticity of host '[127.0.0.1]:22001 ([127.0.0.1]:22001)' can't be established.
ECDSA key fingerprint is SHA256:x4weSw+f9CFHpTQ1Rm/O4ynwXPLnowF+qGc6Vo3zodc.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[127.0.0.1]:22001' (ECDSA) to the list of known hosts.

[ansible@saturn ~]$ uname -a
Linux saturn 5.6.8-300.fc32.x86_64 #1 SMP Wed Apr 29 19:01:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[ansible@saturn ~]$ sudo lvs -a
LV   VG Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
root vg -wi-ao----   <5.73g                                                    
swap vg -wi-ao---- 1000.00m                                                    

[ansible@saturn ~]$ sudo parted /dev/vda unit GB print free
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
        0.00GB  0.00GB  0.00GB           Free Space
1      0.00GB  0.31GB  0.31GB  primary  ext4         boot
2      0.31GB  7.52GB  7.20GB  primary               lvm
        7.52GB  32.2GB  24.7GB           Free Space


[ansible@saturn ~]$ free -h
            total        used        free      shared  buff/cache   available
Mem:          973Mi       121Mi       678Mi       0.0Ki       173Mi       713Mi
Swap:         999Mi          0B       999Mi

[ansible@saturn ~]$ nmcli -p device
=====================
Status of devices
=====================
DEVICE  TYPE      STATE      CONNECTION         
----------------------------------------------------------------------
eth0    ethernet  connected  eth0               
eth1    ethernet  connected  Wired connection 1 
lo      loopback  unmanaged  --                 

[ansible@saturn ~]$ sudo dnf update
Fedora 32 openh264 (From Cisco) - x86_64                     2.8 kB/s | 5.1 kB     00:01    
Fedora Modular 32 - x86_64                                   2.9 MB/s | 4.9 MB     00:01    
Fedora Modular 32 - x86_64 - Updates                         1.1 MB/s | 1.4 MB     00:01    
Fedora 32 - x86_64 - Updates                                 2.4 MB/s | 7.8 MB     00:03    
Fedora 32 - x86_64                                           4.9 MB/s |  70 MB     00:14    
Dependencies resolved.
Nothing to do.
Complete!
[ansible@saturn ~]$ exit
logout
Connection to 127.0.0.1 closed.

As seen from parted command vda disk has 24.7GB free space that can be used to create new partitions or extend existing ones. Multiple VM's can share same backing storage.