Using cloud-init to customize a VM while provisioning with terraform on vSphere

We are going to perform below steps on a newly installed ubuntu VM and convert it to a template

1) Check if VMware tools is installed,

root@linux:~# dpkg -l | grep open-vm-tools
ii  open-vm-tools                          2:11.3.0-2ubuntu0~ubuntu20.04.4   amd64        Open VMware Tools for virtual machines hosted on VMware (CLI)
root@linux:~#

2) Update VMware tools,

root@linux:~# apt install open-vm-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
open-vm-tools is already the newest version (2:11.3.0-2ubuntu0~ubuntu20.04.4).
The following packages were automatically installed and are no longer required:
  linux-headers-5.15.0-46-generic linux-headers-5.4.0-137 linux-headers-5.4.0-137-generic linux-hwe-5.15-headers-5.15.0-46 linux-image-5.15.0-46-generic linux-image-5.4.0-137-generic
  linux-modules-5.15.0-46-generic linux-modules-5.4.0-137-generic linux-modules-extra-5.15.0-46-generic linux-modules-extra-5.4.0-137-generic
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root@linux:~#

3) Check the version installed,

root@linux:~# vmtoolsd -v
VMware Tools daemon, version 11.3.0.29534 (build-18090558)
root@linux:~#

4) Ensure the open-vm-tools service is enabled and started,

root@linux:~# systemctl enable open-vm-tools.service
Synchronizing state of open-vm-tools.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable open-vm-tools
root@linux:~#
root@linux:~# systemctl start open-vm-tools.service
root@linux:~#
root@linux:~# systemctl is-enabled open-vm-tools.service
enabled
root@linux:~#
root@linux:~# systemctl status open-vm-tools.service
● open-vm-tools.service - Service for virtual machines hosted on VMware
     Loaded: loaded (/lib/systemd/system/open-vm-tools.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-03-02 06:08:57 +04; 9min ago
       Docs: http://open-vm-tools.sourceforge.net/about.php
   Main PID: 710 (vmtoolsd)
      Tasks: 3 (limit: 9350)
     Memory: 4.3M
     CGroup: /system.slice/open-vm-tools.service
             └─710 /usr/bin/vmtoolsd

Mar 02 06:08:57 linux systemd[1]: Started Service for virtual machines hosted on VMware.
root@linux:~#

5) Install cloud-init package,

root@linux:~# dpkg -l | grep -w cloud-init
root@linux:~#
root@linux:~# apt install cloud-init
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-5.15.0-46-generic linux-headers-5.4.0-137 linux-headers-5.4.0-137-generic linux-hwe-5.15-headers-5.15.0-46 linux-image-5.15.0-46-generic linux-image-5.4.0-137-generic
  linux-modules-5.15.0-46-generic linux-modules-5.4.0-137-generic linux-modules-extra-5.15.0-46-generic linux-modules-extra-5.4.0-137-generic
Use 'apt autoremove' to remove them.
The following additional packages will be installed:
  eatmydata libeatmydata1 python3-distutils python3-importlib-metadata python3-json-pointer python3-jsonpatch python3-jsonschema python3-lib2to3 python3-more-itertools python3-pyrsistent
  python3-setuptools python3-zipp
Suggested packages:
  python-jsonschema-doc python-setuptools-doc
The following NEW packages will be installed:
  cloud-init eatmydata libeatmydata1 python3-distutils python3-importlib-metadata python3-json-pointer python3-jsonpatch python3-jsonschema python3-lib2to3 python3-more-itertools
  python3-pyrsistent python3-setuptools python3-zipp
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,260 kB of archives.
After this operation, 7,347 kB of additional disk space will be used.
Do you want to continue? [Y/n] y

root@linux:~# dpkg -l | grep -w cloud-init
ii  cloud-init                             22.4.2-0ubuntu0~20.04.2           all          initialization and customization tool for cloud instances
root@linux:~#

6) enable the cloud-init service,
it doesn't need to be started, but ensure it is enabled,

root@linux:~# systemctl status cloud-init.service
● cloud-init.service - Initial cloud-init job (metadata service crawler)
     Loaded: loaded (/lib/systemd/system/cloud-init.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
root@linux:~#
root@linux:~# systemctl enable cloud-init.service
root@linux:~#
root@linux:~# sudo systemctl is-enabled cloud-init.service
enabled
root@linux:~#

7) Ensure no datasources are enabled in /etc/cloud/cloud.cfg like below,

root@linux:~# cat /etc/cloud/cloud.cfg | grep datasource
# If you use datasource_list array, keep array items in a single line.
# Example datasource config
# datasource:
root@linux:~#

8) Check the current activated datasources,

root@linux:/etc/cloud/cloud.cfg.d# cat /etc/cloud/cloud.cfg.d/90_dpkg.cfg
# to update this file, run dpkg-reconfigure cloud-init
datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, None ]
root@linux:/etc/cloud/cloud.cfg.d#

9) Run the command "dpkg-reconfigure cloud-init" and select only VMware (this is not mandatory , but deselecting other datasources will reduce our boot time, so cloud-init doesn't need to look for the userdata/metadata from all the datasources in the list)

root@linux:/etc/cloud/cloud.cfg.d# dpkg-reconfigure cloud-init
root@linux:/etc/cloud/cloud.cfg.d#
root@linux:/etc/cloud/cloud.cfg.d# cat /etc/cloud/cloud.cfg.d/90_dpkg.cfg
# to update this file, run dpkg-reconfigure cloud-init
datasource_list: [ VMware ]
root@linux:/etc/cloud/cloud.cfg.d#

10) ensure /etc/cloud/cloud.cfg.d has only below files,

root@linux:/etc/cloud/cloud.cfg.d# ls
05_logging.cfg  90_dpkg.cfg  README
root@linux:/etc/cloud/cloud.cfg.d#

11) disable network, so cloud-init doesn't configure networking,

root@linux:/etc/cloud/cloud.cfg.d# vi /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
root@linux:/etc/cloud/cloud.cfg.d#
root@linux:/etc/cloud/cloud.cfg.d# cat /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
network: {config: disabled}
root@linux:/etc/cloud/cloud.cfg.d#

12) As a final step, run the clean command to ensure that cloud-init will all the modules in the userdata and metadata.

root@linux:/# cloud-init clean --logs
root@linux:/#

13) below we are verifying that niginx/apache2 are not installed on the vm before converting it to a template,

root@linux:~# dpkg -l | grep nginx
root@linux:~#
root@linux:~# dpkg -l | grep apache2
root@linux:~#

14) Convert the VM to a template from vCenter

15) Provision a VM using terraform code which can be obtained from below repo,

https://github.com/balajivedagiri/terraform_cloud-init.git

userdata we defined it,

cloud-config

runcmd:

• [ sh, -c, echo "=========Hello There from Terraform and Cloud-init automation=========" > /root/testing-01] packages:
  • nginx
  • apache2

16) Once VM is provisioned, login to the new VM and verify if cloud-init ran successfully.

root@cloud-init-testing02:~# cloud-init status
status: running
root@cloud-init-testing02:~#

# Above it is still running, wait for sometime and check
# Below it completed successfully

root@cloud-init-testing02:~# cloud-init status
status: done
root@cloud-init-testing02:~#

17) Check if packages are installed successfully,

root@cloud-init-testing02:~# dpkg -l | grep nginx
ii  libnginx-mod-http-image-filter         1.18.0-0ubuntu1.4                 amd64        HTTP image filter module for Nginx
ii  libnginx-mod-http-xslt-filter          1.18.0-0ubuntu1.4                 amd64        XSLT Transformation module for Nginx
ii  libnginx-mod-mail                      1.18.0-0ubuntu1.4                 amd64        Mail module for Nginx
ii  libnginx-mod-stream                    1.18.0-0ubuntu1.4                 amd64        Stream module for Nginx
ii  nginx                                  1.18.0-0ubuntu1.4                 all          small, powerful, scalable web/proxy server
ii  nginx-common                           1.18.0-0ubuntu1.4                 all          small, powerful, scalable web/proxy server - common files
ii  nginx-core                             1.18.0-0ubuntu1.4                 amd64        nginx web/proxy server (standard version)
root@cloud-init-testing02:~#
root@cloud-init-testing02:~# dpkg -l | grep apache2
ii  apache2                                2.4.41-4ubuntu3.13                amd64        Apache HTTP Server
ii  apache2-bin                            2.4.41-4ubuntu3.13                amd64        Apache HTTP Server (modules and other binary files)
ii  apache2-data                           2.4.41-4ubuntu3.13                all          Apache HTTP Server (common files)
ii  apache2-utils                          2.4.41-4ubuntu3.13                amd64        Apache HTTP Server (utility programs for web servers)
root@cloud-init-testing02:~#
root@cloud-init-testing02:~# cat /root/testing-01
=========Hello There from Terraform and Cloud-init automation=========
root@cloud-init-testing02:~#
root@cloud-init-testing02:~# ls -ltr /root/testing-01
-rw-r--r-- 1 root root 71 Mar  2 07:15 /root/testing-01
root@cloud-init-testing02:~#

18) to check what userdata we have passed,

root@cloud-init-testing02:~# cloud-init query userdata
#cloud-config
runcmd:
  - [ sh, -c, echo "=========Hello There from Terraform and Cloud-init automation=========" > /root/testing-01]
packages:
- nginx
- apache2
root@cloud-init-testing02:~#
root@cloud-init-testing02:~# vmware-rpctool "info-get guestinfo.userdata" | base64 -d
#cloud-config
runcmd:
  - [ sh, -c, echo "=========Hello There from Terraform and Cloud-init automation=========" > /root/testing-01]
packages:
- nginx
- apache2

root@cloud-init-testing02:~#