Locked in: the inextricable dependency on VMWare

Introduction

The recent acquisition of VMware by Broadcom and the subsequent announcement of their new subscription model have raised significant concerns in the IT industry regarding dependency on software vendors.

While most customers are dealing with an exorbitant cost increase, some are exploring alternatives, often based on open-source software, to mitigate the impact on their operations. Despite the advocacy from open-source specialists for their preferred solutions, VMware has, over the past decades, established an unmatched ecosystem comprising hardware vendors, software integrators and resellers.

Replicating such a comprehensive portfolio, either from scratch or using open-source software, demands substantial investment and carries a high risk of failure.

For learning purposes, I challenged myself to recreate some of VMware's key products using open-source software. This article is part of a series of articles that aim to contribute to the community by sharing the results of my exploration.

Disclaimer: These explorations were conducted in a lab environment. Using this in a production environment would require a thoroughly developed open-source strategy.

Hypervisors

ESXi is VMWare hypervisor. The role of the hypervisor or VMM (virtual machine manager) is to create and manage virtual machines (VM). By abstracting the underlying hardware and allocating physical resources to virtual machines, it enables multiple operating systems to run concurrently on the same physical host.

Hypervisors serve various purposes, ranging from general uses like web hosting and business applications to specialized uses such as safety-critical and security-critical systems. Their implementation varies based on their intended purpose.

Bare metal hypervisors

Bare metal hypervisors, also known as Type 1 hypervisors, are built directly on top of dedicated operating systems (OS) designed specifically for virtualization. These operating systems usually feature a microkernel architecture and expose a minimal set of features to the VMs through a low-level API consisting in privileged functions called Hypercalls. Additionally, it provides the applications and tools that constitute the hypervisor control plane with higher level APIs to control and monitor the hypervisor.

This approach offers several advantages, such as improved security and reliability by reducing the amount of code executed with high CPU privileges. It also facilitates the formal verification of the hypervisor software for critical use cases. For instance, such verification may be necessary to achieve the highest levels of safety certification in airborne systems (DO-178). One drawback may be limited hardware compatibility.

ESXi is a type 1 hypervisor that uses a custom OS called VMkernel. Other examples include Xen, PikeOS, QNX, L4Re and more.

Hosted hypervisors

In hosted hypervisors also known as Type 2 hypervisors, the hypervisor runs as a user-space application within the host operating system. When a VM running on a Type 2 hypervisor needs to perform an operation, the hypervisor typically issues system calls (syscalls) to the host operating system, which in turn manages access to hardware resources and executes the requested operations on behalf of the VM.

Their portability and ease of use make them ideal for local virtualization environments. However, they must share hardware resources with other applications which can impact performance, stability and security. Examples include Oracle VirtualBox and VMware Workstation.

Hybrid hypervisors

There are also hybrid implementations like KVM and Bhyve, integrated into the Linux and FreeBSD kernels, respectively. The syscalls from the guest VMs, are translated into hypercalls by the kernel module for interaction with the hardware virtualization layer. They blur the lines between type 1 and type 2 hypervisors, and are often referred to as type 1.5 hypervisors. By combining the advantages of type 1 and type 2 hypervisors, it enables great flexibility and a familiar environment by supporting a broad range of hardware and tools. The separation of concerns is however less strict, increasing the potential blast radius in case of failure or compromise.

Software stack

To build an equivalent to VMware ESXi, it would make sense to leverage a similar approach. ESXi and Xen are both bare metal hypervisors that run directly on the hardware and leverage direct calls for communication between the VMs and hypervisor layer. It provides secure and efficient means of communication between the VMs and the hypervisor, ensuring robust virtualization capabilities while maintaining isolation and security boundaries between different VMs. However, there are some subtle differences in their implementation. As an example, in Xen, the control plane runs on top of the hypervisor as a privileged virtual machine (VM), known as Dom0. In contrast, ESXi services run as processes within the VMkernel operating system.

Xen is a general purpose type 1 open source hypervisor published under the terms of the GPLv2 license and sponsored by the Linux Foundation. It is a mature piece of software issued from research at the university of Cambridge and has been widely used by many companies including famous public cloud providers such as Amazon to support their AWS infrastructure. Its longevity has fostered a large ecosystem of hardware vendors and integrators. Additionally, it is well-documented and benefits from strong community support and tools.
Xen require at least a privileged virtual machine or domain, often referred to as dom0. Dom0 provides the APIs and tools that constitute the hypervisor control plane. It is responsible for managing domUs (VMs), devices, security, networking, and storage.
FreeBSD is an excellent fit for this purpose. It’s a Unix/POSIX operating system issued from the University of Berkeley and distributed under the terms of the permissive BSD license. Known for its stability and security, FreeBSD has been used for decades in education, research, civil and military applications. It has first-class support for Xen as both dom0 and domU.

libvirt is a C library developped by RedHat under the terms of the GNU LGPL license, that provides a generic API that abstracts the bits and bytes of the underlying hypervisor, networking and storage technologies. Using libvirt generic API permits to benefit from a large ecosystem of open source software and tools. As an example, it is possible to use the virt-manager client to create and manage virtual machines.
It supports a wide range of hypervisors including Xen via the libxl driver, KVM, BHyve, Hyper-V and storage backends, including ZFS.

ZFS is a fully featured file system and volume manager first developped by Sun Microsystems under the terms of the CDDL license. Now property of Oracle, ongoing open source ZFS development has moved to the OpenZFS Project. It provides efficient software RAID, snapshots, compression, de-duplication, encryption and other advanced features. ZVols are very practical for VM primary storage. Known for its reliability, it is integrated in FreeBSD and many storage appliances.

Server side

Xen

In order to install FreeBSD as a Xen host, follow the steps described in the FreeBSD handbook.
Below is a summary of the key steps.

First install xen packages:

# pkg update
# pkg install -y xen-kernel xen-tools

Then, adjust some system parameters required for xen dom0:

# echo "vm.max_wired=-1" >> /etc/sysctl.conf
# sed -i '' -e 's/memorylocked=64K/memorylocked=unlimited/' /etc/login.conf
# cap_mkdb /etc/login.conf
# echo 'xc0     "/usr/libexec/getty Pc"         xterm   onifconsole  secure' >> /etc/ttys

Configure the kernel to load the tap module, enable Xen by specifying the path to the Xen kernel, and provide the command line required to boot FreeBSD as the dom0:

# sysrc -f /boot/loader.conf if_tap_load="YES"
# sysrc -f /boot/loader.conf xen_kernel="/boot/xen"
# sysrc -f /boot/loader.conf xen_cmdline="dom0_mem=8192M dom0_max_vcpus=4 dom0=pvh console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all"

Enable the xencommons service:

# sysrc xencommons_enable=yes

Configure the network bridge and add a network interface to the bridge (replace em0 with the name of your network interface):

# sysrc cloned_interfaces="bridge0"
# sysrc ifconfig_bridge0="addm em0 SYNCDHCP"
# sysrc ifconfig_em0="up"

Reboot the system

# reboot

libvirt

Although it is possible to install a pre-compiled libvirt binary with FreeBSD package manager, it is not built with Xen support. To overcome this limitation, it is however possible to compile a custom version using the ports system.

First update your ports tree from FreeBSD git repository:

# pkg install -y git
# git clone https://git.freebsd.org/ports.git /usr/ports

Ensure that your ports tree version is consistent with your package manager configured in /etc/pkg/FreeBSD.conf. By default pkg is configured to use the quarterly repository:

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

Back in your ports tree, browse all available branches:

# cd /usr/ports
# git branch -a

And checkout the latest quarterly branch:

# git checkout 2024Q2

In order to compile and install libvirt port, first install dependencies using the package manager:

# cd /usr/ports/devel/libvirt/
# make install-missing-packages

Edit /etc/make.conf to enable xen support:

OPTIONS_SET+=XEN

Compile and install libvirt:

# export BATCH="yes"
# make clean install

Set xen as the default libvirt hypervisor backend:

cat <<EOF | tee /usr/local/etc/libvirt/libvirt.conf
uri_default = "xen:///system"
EOF

Configure libvirtd to allow the libvirt group members to read and write to the libvirt socket:

# cat <<EOF | tee /usr/local/etc/libvirt/libvirtd.conf
unix_sock_group = "libvirt"
unix_sock_ro_perms = "0770"
unix_sock_rw_perms = "0770"
EOF

Ensure the libvirt group exists and add the required users as group members:

# pw groupadd libvirt
# pw groupmod libvirt -m username

More libxl parameters can be configured in /usr/local/etc/libvirt/libxl.conf.

Finally enable libvirtd service.

# sysrc libvirtd_enable="YES"
# service libvirtd start

Client side

One of the advanges of using libvirt which abstracts the underlying hypervisor, storage, and network backends by exposing a generic API is to benefit from a large ecosystem of software and tools, including command line, graphical clients including, web clients, and provisioning tools.

Graphical interface

Using libvirt as a generic API permits to use the virt-manager graphical client to manage the VMs.

Following are a few screenshots of a FreeBSD guest installation:

Command line

Virsh is the command line client that is packaged with libvirt. It provides an intuitive command line interface (CLI) that permits to manage the life cycle of virtual machines and their associated network and storage resources.

It is for example possible to use virsh to check that our installation has been successful and that dom0 is up and running:

root@lab-01:~ # virsh list
 Id   Name       State
--------------------------
 0    Domain-0   running

Conclusion

This experimentation demonstrates that it is possible to reproduce ESXi key features by relying on a mature ecosystem of Open Source technologies such as Xen and FreeBSD.
In the next article, I would like to explore alternatives to VMWare VCenter, including clustering, high availability and monitoring.