Installing Vagrant + libvrt plugin in Centos 8

Has been a while since I have used vagrant and when I did, I just used the default virtualbox to test oracle-base RAC installations

Now that I'm thinking about how to create a Oracle RAC cluster using Ansible (there are few places you can find how to do it, but the fun part is to figure it out by yourself, right?) I thought is good idea to use Vagrant to make few things easier.

The original Idea was to create a post to get Vagrant boxes deployed using Ansible, but just getting Vagrant working with the libvrt plugin in my Centos 8 server was so painful, that I decided to create this post where I only install Vagrant + libvrt plugin so I can follow the same steps next time I need to reinstall the system.

Let's start installing Vagrant. I just followed this Process which also mentions how to get libvrt plugin

|=| server in ~ ○ → sudo dnf groupinstall "Development Tools" -y 
[....]

Completed!
|=| server in ~ ○ → sudo dnf -y install rsync gcc zlib-devel libvirt-devel
[....]

Completed!
|=| server in ~ ○ → sudo dnf install -y ruby ruby-devel
[....]

Completed!

|=| server in ~ ○ → sudo dnf install -y ruby ruby-devel
Last metadata expiration check: 0:37:12 ago on Sat 02 Jan 2021 01:59:57 PM GMT.
Dependencies resolved.


|=| server in ~ ○ → VERSION="2.2.14"

|=| server in ~ ○ → wget https://releases.hashicorp.com/vagrant/${VERSION}/vagrant_${VERSION}_x86_64.rpm

--2021-01-02 14:41:11--  https://releases.hashicorp.com/vagrant/2.2.14/vagrant_2.2.14_x86_64.rpm
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.1.183, 151.101.65.183, 151.101.129.183, ...
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.1.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 40313007 (38M) [application/x-redhat-package-manager]
Saving to: ‘vagrant_2.2.14_x86_64.rpm’

vagrant_2.2.14_x86_64.rpm            100%[====================================================================>]  38.45M  7.94MB/s    in 5.8s

2021-01-02 14:41:17 (6.60 MB/s) - ‘vagrant_2.2.14_x86_64.rpm’ saved [40313007/40313007]


|=| server in ~ ○ → sudo dnf localinstall vagrant_${VERSION}_x86_64.rpm

Last metadata expiration check: 0:41:24 ago on Sat 02 Jan 2021 01:59:57 PM GMT.
Dependencies resolved.
[....]

Installed:
  vagrant-1:2.2.14-1.x86_64

Complete!

|=| server in ~ ○ →


|=| server in ~ ○ → CONFIGURE_ARGS="with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib64" vagrant plugin install vagrant-libvirt
Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Building native extensions. This could take a while...
Building native extensions. This could take a while...
Installed the plugin 'vagrant-libvirt (0.3.0)'!

|=| server in ~ ○ → gem install nokogiri
Fetching: mini_portile2-2.4.0.gem (100%)
Successfully installed mini_portile2-2.4.0
Fetching: nokogiri-1.10.10.gem (100%)
Building native extensions. This could take a while...
Successfully installed nokogiri-1.10.10
Parsing documentation for mini_portile2-2.4.0
Installing ri documentation for mini_portile2-2.4.0
Parsing documentation for nokogiri-1.10.10
Installing ri documentation for nokogiri-1.10.10
Done installing documentation for mini_portile2, nokogiri after 3 seconds
2 gems installed

|=| server in ~ ○ →

Let's test it out using vagrant centos8 boxes (spoiler alert, will fail!)

|=| server in ~ ○ → mkdir vagrant_test

|=| server in ~ ○ → cd vagrant_test/

|=| server in ~/vagrant_test ○ →

|=| server in ~/vagrant_test ○ → vagrant box add centos/8 --provider=libvirt
==> box: Loading metadata for box 'centos/8'
    box: URL: https://vagrantcloud.com/centos/8
==> box: Adding box 'centos/8' (v2011.0) for provider: libvirt
    box: Downloading: https://vagrantcloud.com/centos/boxes/8/versions/2011.0/providers/libvirt.box
Download redirected to host: cloud.centos.org
    box: Calculating and comparing box checksum...
==> box: Successfully added box 'centos/8' (v2011.0) for 'libvirt'!


|=| server in ~/vagrant_test ○ → vagrant box list
centos/8 (libvirt, 2011.0)

|=| server in ~/vagrant_test ○ → vagrant init centos/8
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

|=| server in ~/vagrant_test ○ → vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'centos/8' version '2011.0' is up to date...
Traceback (most recent call last):
        35: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/batch_action.rb:86:in `block (2 levels) in run'
        34: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:201:in `action

[......]
         2: from /home/solifugo/.vagrant.d/gems/2.6.6/gems/ruby-libvirt-0.7.1/lib/libvirt.rb:11:in `<top (required)>'
         1: from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/solifugo/.vagrant.d/gems/2.6.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)

|=| server in ~/vagrant_test ○ →

Unfortunately, got some errors, but found the solution on this github discussion in order to get the libk5crypto.so.3 file into vagrant libraries

|=| server in ~/vagrant_test ○ → sudo dnf install gcc  libguestfs-tools-c libvirt libvirt-devel libxml2-devel libxslt-devel make ruby-devel
Last metadata expiration check: 1:01:29 ago on Sat 02 Jan 2021 01:59:57 PM GMT.
Package gcc-8.3.1-5.1.el8.x86_64 is already installed.
[....]


|=| server in ~/vagrant_test ○ → cd

|=| server in ~ ○ → cd /tmp/; wget http://vault.centos.org/8.2.2004/BaseOS/Source/SPackages/krb5-1.17-18.el8.src.rpm

[....]

krb5-1.17-18.el8.src.rpm             100%[====================================================================>]   9.77M  1.52MB/s    in 6.6s

2021-01-02 15:02:01 (1.48 MB/s) - ‘krb5-1.17-18.el8.src.rpm’ saved [10241920/10241920]


|=| server in /tmp ○ → rpm2cpio krb5-1.17-18.el8.src.rpm | cpio -imdV
................................................
20516 blocks

|=| server in /tmp ○ → tar xf krb5-1.17.tar.gz

|=| server in /tmp ○ → cd krb5-1.17/src

|=| server in /tmp/krb5-1.17/src ○ → LDFLAGS='-L/opt/vagrant/embedded/' ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...

[....]


make[1]: Leaving directory '/tmp/krb5-1.17/src/doc'
making all in po...
make[1]: Entering directory '/tmp/krb5-1.17/src/po'
msgfmt -o en_US.mo en_US.po
msgfmt -o de.mo de.po
make[1]: Leaving directory '/tmp/krb5-1.17/src/po'

|=| server in /tmp/krb5-1.17/src ○ →
|=| server in /tmp/krb5-1.17/src ○ → sudo cp lib/libk5crypto.so.3 /opt/vagrant/embedded/lib64/

Now, second issue I found was with libssh library

|=| server in ~/vagrant_test ○ → vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'centos/8' version '2011.0' is up to date...
Traceback (most recent call last):

[....]

2: from /home/solifugo/.vagrant.d/gems/2.6.6/gems/ruby-libvirt-0.7.1/lib/libvirt.rb:11:in `<top (required)>'
         1: from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': /lib64/libssh.so.4: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/solifugo/.vagrant.d/gems/2.6.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)

|=| server in ~/vagrant_test ○ →

The solution is similar to this github post
More info of both errors can be found here

The main issue is that both describe the issue for fedora and I needed to manually locate the libssh rpm source file libssh-0.9.4-2.el8.src.rpm

|=| server in ~ ○ → sudo dnf install cmake
Last metadata expiration check: 2:16:10 ago on Sat 02 Jan 2021 01:59:57 PM GMT.

 [....]
Complete!

|=| server in ~ ○ →



|=| server in ~ ○ → wget https://vault.centos.org/8.3.2011/BaseOS/Source/SPackages/libssh-0.9.4-2.el8.src.rpm

--2021-01-02 16:14:02--  https://vault.centos.org/8.3.2011/BaseOS/Source/SPackages/libssh-0.9.4-2.el8.src.rpm
Resolving vault.centos.org (vault.centos.org)... 18.159.124.96, 2001:4de0:aaae::194
Connecting to vault.centos.org (vault.centos.org)|18.159.124.96|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 534122 (522K) [application/x-rpm]
Saving to: ‘libssh-0.9.4-2.el8.src.rpm’

libssh-0.9.4-2.el8.src.rpm           100%[====================================================================>] 521.60K  1.76MB/s    in 0.3s

2021-01-02 16:14:03 (1.76 MB/s) - ‘libssh-0.9.4-2.el8.src.rpm’ saved [534122/534122]


|=| server in ~ ○ →


|=| server in ~ ○ → rpm2cpio libssh-0.9.4-2.el8.src.rpm | cpio -imdV
...........
1038 blocks

|=| server in ~ ○ → tar xf libssh-0.9.4.tar.xz

|=| server in ~ ○ → mkdir build

|=| server in ~ ○ → cd build/

|=| server in ~/build ○ → cmake ../libssh-0.9.4 -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
-- The C compiler identification is GNU 8.3.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
[....]
-- Generating done
-- Build files have been written to: /home/solifugo/build

|=| server in ~/build ○ →

|=| server in ~/build ○ → make
Scanning dependencies of target ssh
[  1%] Building C object src/CMakeFiles/ssh.dir/agent.c.o
[  1%] Building C object src/CMakeFiles/ssh.dir/auth.c.o
[....]
Scanning dependencies of target libsshpp
[ 99%] Building CXX object examples/CMakeFiles/libsshpp.dir/libsshpp.cpp.o
[100%] Linking CXX executable libsshpp
[100%] Built target libsshpp

|=| server in ~/build ○ → sudo cp lib/libssh* /opt/vagrant/embedded/lib64

And voilà!

|=| server in ~/build ○ → cd ~/vagrant_test/

|=| server in ~/vagrant_test ○ → vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'centos/8' version '2011.0' is up to date...
==> default: Uploading base box image as volume into Libvirt storage...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default:  -- Name:              vagrant_test_default
[....]

    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Rsyncing folder: /home/solifugo/vagrant_test/ => /vagrant

|=| server in ~/vagrant_test ○ →

Finally, we can connect to the test box and confirm is running using virsh:

|=| server in ~/vagrant_test ○ → vagrant ssh
[vagrant@localhost ~]$ uname -a ; head -1 /etc/*rel*
Linux localhost.localdomain 4.18.0-240.1.1.el8_3.x86_64 #1 SMP Thu Nov 19 17:20:08 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
==> /etc/centos-release <==
CentOS Linux release 8.3.2011

==> /etc/centos-release-upstream <==
Derived from Red Hat Enterprise Linux 8.3

==> /etc/os-release <==
NAME="CentOS Linux"

==> /etc/prelink.conf.d <==
head: error reading '/etc/prelink.conf.d': Is a directory

==> /etc/redhat-release <==
CentOS Linux release 8.3.2011

==> /etc/system-release <==
CentOS Linux release 8.3.2011

==> /etc/system-release-cpe <==
cpe:/o:centos:centos:8
[vagrant@localhost ~]$ exit

|=| server in ~/vagrant_test ○ → virsh list
 Id   Name                   State
--------------------------------------
 1    vagrant_test_default   running