Running FreeIPA on Ubuntu Using Podman

There was this consideration to replace OpenLDAP with FreeIPA in the university authentication infrastructure. The main motivation for this was to improve the security in the system and to give more identity management features which the OpenLDAP lacks.

Some of these features are

• Ticket based authentication instead of sending the passwords over the network constantly.
• Centralized user and authentication management.
• Certificate for verifying systems and services etc.

The issue was that the existing university infrastructure is mainly based on Ubuntu Linux systems, but FreeIPA is mainly designed for Red Hat-based systems like AlmaLinux , Centos etc. Therefore direct installation of FreeIPA was not a choice here.
Therefore, this system needed to be deployed in a containerized manner.

So this article mainly focuses on the decisions made, issues that came across with and what solutions were used to fix them.

Introduction to FreeIPA
FreeIPA is an integrated identity management system used in Linux environments but was mainly developed in Red Hat based systems. It is integrated of set of services as,

1. 389 Directory Server (LDAP): Stores the identity data.
2. MIT Kerberos (KDC):Handles the tickets for SSO authentication
3. Apache (HTTPD): The administrative web dashboard and API.
4. Dogtag Certificate System (PKI):The Certification Authority (CA) for the secure communication between systems.

Service Startup Order
These services need to be started in the specific order given above. As the LDAP has the initial configuration settings for other services on it, it needs to start first, then KDC as it needs LDAP data for authentication, next the dashboard and finally the PKI (Dogtag CA) service as it depends on other main services being ready.

If any of the services did not start with this order the whole system crashes. For this order to be preserved it uses the systemd in here.

Platform Compatibility Issues
As said above direct installation of FreeIPA on to the existing ubuntu systems was not a solution.

The Centos Stream 9 container image lacked the required binaries, so it was unable to even start.

But Almalinux was compatible with FreeIPA. However, changing all the existing systems to Almalinux was not a solution also. That is why it was thought to use a containerization appproach to run an Almalinux environment while still keeping the existing Ubuntu infrastructure unchanged.

_{CentOS 9 image does not start properly}

Failure 1: Docker and Cgroups v2 Incompatibility
systemd and cgroups - When systemd runs as the PID1 inside a container it is responsible for starting and managing the other services there. It uses the cgroups to control resources like memory and CPU. To do that systemd needs read, write access to cgroup file system at /sys/fs/cgroup .

Modern Ubuntu use cgroups v2, which is stricter. This causes Docker container to not get proper access to system resources causing systemd to fail.

So as the systemd is the PID1, failing it prevents the other services to start, causing the container to run but without running any FreeIPA services in it.

_{Docker blocks system access needed for systemd to run properly}

There were many attempts to fix this issue by changing cgroup isolation settings using flags like --cgroupns=host etc ,but nothing worked.
Similar outcomes were also obtained by other attempts like these. https://github.com/moby/moby/issues/16238

Transition to Podman
To solve these issues, Podman was thought to be used considering 2 main reasons.

1. systemd support - As podman supports systemd-based containers directly, it automatically configures the container environment to run systemd correctly.
2. Compatibility - As Podman fits better with the same ecosystem use by FreeIPA and AlmaLinux as it was also made by Red Hat ecosystem. This reduced the runtime issues.

Failure 2: CA Initialization in Pre-packaged Images
Then when tried to get an official container image of FreeIPA to be run in this, it failed during the Certificate Authority(CA) setup.

This was because the official container images are built to be minimal. So some parts of the operating system environments were removed and simplified.

Specially, the default directory folders and the services that auto-create them were missing. These couldn't be created manually during the installation as FreeIPA is a tightly coupled package.(To be installed as a complete system)

Due to those missing directories, CA setup failed, causing no certificates to be created. As LDAP needs certificates to have secure connections, LDAP fails, failing the whole system.

_{FreeIPA CA setup failed because the container was incomplete}

Controlled Setup Process
Because important directories, security settings, and LDAP schema files needed for installation were missing, the setup failed as said above.
As a solution then a clean AlmaLinux 9 base image was obtained and it was prepared for required conditions before running the FreeIPA installer inside it.

Main preparations done was:

1. Creating the necessary filesystems
2. Temporarily allowing security policies for CA initialization
3. Restoring the missing LDAP schema files

After doing these steps, the FreeIPA installation was completed successfully and the system ran successfully after that.

_{All services in the FreeIPA are running properly now}

As this is a series of articles, in the next part a full guide on starting FreeIPA in a Podman container, including all the preparation steps will be provided.