DEV Community: Chanuth Abeynayake

Agentic Pipeline to convert AWS infrastructure to Terraform

Chanuth Abeynayake — Mon, 13 Jul 2026 15:58:35 +0000

AWS architecture of the TF-Agentic-Engine

Technical Analysis of the TF-Agentic-Engine

The TF-Agentic-Engine converts an existing AWS environment into Terraform code. Its main purpose is to support two tasks: bringing existing AWS resources under Terraform management and creating a reusable copy of an existing environment.

In Import Mode, the system creates Terraform resource blocks and import definitions using the real IDs collected from AWS. It should only generate resources that were found during the scan.

Import Mode output with fixed AWS resource values from the scanned environment.

In Clone Mode, the system converts the same infrastructure into reusable Terraform by replacing account-specific values, names, CIDR ranges and other fixed settings with variables. This allows the environment to be recreated in another AWS account, region or project.

Clone Mode output using Terraform variables to create a reusable infrastructure configuration.

Resource Discovery

The process starts in main.py, which controls the full execution. It starts the AWS scan, creates the initial system state and runs the LangGraph workflow.

The AWS discovery logic is mainly located in src/aws_client.py. It uses boto3 to collect details about resources such as:

VPCs and subnets
Route tables and gateways
EC2 instances
Security groups
RDS databases and DB subnet groups
S3 buckets and DynamoDB tables

For local testing, the same discovery layer can use moto instead of connecting to a real AWS account. This makes it possible to test the pipeline without creating real cloud resources.

AWS API responses contain many values that are not required in Terraform. The system removes runtime details and keeps configuration values that are needed to describe the infrastructure.

Infrastructure Graph

After discovery, the system converts the resource data into a graph.

Each AWS resource is stored as a node. The connection between two resources is stored as an edge. For example, an edge can show that a subnet belongs to a VPC, an EC2 instance uses a subnet or an RDS database uses a DB subnet group.

The compile_infrastructure_graph() function creates this graph from the scanned AWS data. This is important because the resource relationships come from AWS IDs rather than being guessed by the language model.

The graph also helps with incomplete AWS data. For example, when an RDS instance exists but its subnet-group information is missing, the system can create a DB subnet-group entry and connect it to the discovered private subnets.

Generated architecture diagram showing the discovered AWS resources and their dependency relationships.

LangGraph Workflow

The Terraform generation process is controlled by src/agent.py.

LangGraph runs the system in a fixed order:

Network generation
Security generation
Compute generation
Data generation
Terraform validation

Each stage is implemented inside src/nodes.py.

The Network Node creates Terraform for VPCs, subnets, gateways and route tables.

The Security Node creates security groups, rules, IAM roles and policies.

The Compute Node creates EC2 and related compute resources.

The Data Node creates RDS, S3 and DynamoDB resources.

Separating the resources this way keeps each generation request smaller and reduces the chance of networking, security and database code being mixed together. The output is normally separated into files such as network.tf, security.tf, compute.tf and data.tf.

GraphState

The nodes share information through a structure called GraphState, defined in src/state.py.

GraphState stores the selected mode, scanned AWS data, infrastructure graph, generated Terraform sections, current phase, validation errors, retry count and final validation result.

This allows every node to work with the latest version of the infrastructure and generated code. It also allows the validation stage to send errors back into the generation process.

Terraform Validation and Repair

After all Terraform files are generated, the Validation Node runs the real Terraform CLI.

It checks the files for invalid syntax, unsupported provider arguments, missing references and incorrect resource structures. A temporary provider file can be added so that the Terraform configuration can be checked locally without using real AWS credentials.

When validation fails, the error is saved in GraphState and the generation workflow runs again. The retry count prevents the system from entering an endless loop.

Execution log showing a failed Terraform validation, automatic retry, and successful validation on the second run (As an example).

The project also includes fixed Python-based corrections for common language-model errors.

The S3 correction logic removes arguments that are no longer valid inside the main S3 bucket resource.

The RDS correction logic removes incorrectly placed subnet IDs, creates a proper DB subnet group and links it to the database.

The security-group correction logic fixes invalid ingress and egress block formats.

The template-protection logic escapes braces found in Terraform and JSON error messages so that LangChain does not treat them as prompt variables.

These corrections are handled by normal code because common Terraform mistakes are easier and safer to fix with fixed rules than by repeatedly asking the model to rewrite everything.

Final Outputs

When Terraform validation succeeds, the system produces the final Terraform files and architecture documents.

The same infrastructure graph is used to generate a PNG image and an editable Draw.io file. This keeps the Terraform output and architecture diagram based on the same resource relationships.

The final output includes:

Terraform files
Infrastructure graph data
Validation results
PNG architecture image
Editable Draw.io file

AWS Infrastructure Used by the System

In the AWS architecture, a Discovery Lambda can start an on-demand scan. AWS Config and EventBridge can also detect infrastructure changes and start the process automatically.

Amazon S3 stores the scanned graph, generated Terraform files, diagrams and logs.

Amazon Neptune Serverless is used temporarily to store and query the infrastructure graph. It can check multi-level relationships, such as the connection from an EC2 instance to its subnet, VPC and security group. Neptune does not create the relationships; the Python graph compiler creates them first.

Amazon Bedrock provides the language model used for Terraform generation and repair in the AWS version of the system. LangGraph still controls the order of the generation and validation steps.

After validation finishes and the outputs are stored, the temporary Neptune environment is removed. This prevents the graph database from remaining active when the system is not running.

The system therefore separates the work clearly: Python reads and organizes the AWS environment, LangGraph controls the stages, the language model writes Terraform, and Terraform checks whether the generated code is valid.

(The testing was done in an HPC in the university with the local LLM models gemma3:27b and llama3:70b instead of testing using AWS Bedrock for cost efficiency).

Running FreeIPA on Ubuntu Using Podman – Part 2: Step-by-Step Deployment

Chanuth Abeynayake — Thu, 28 May 2026 15:56:37 +0000

This is the full guide of step by step implementation to launch, prepare and configure the FreeIPA server inside a Podman container.

Accessing the FreeIPA web dashboard with HTTPS.

Step 1: Allow Web Ports
FreeIPA uses normal web ports such as 80 and 443. Sometimes, Linux does not allow normal container processes to use these ports directly, so allow the system to use ports starting from 80.

sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80

Step 2: Start the Container
FreeIPA needs some system services like LDAP and Kerberos, which systemd manages, so we start the container with systemd enabled.
So when we start the container, we must enable systemd inside it.

podman run -d --name freeipa-server \
  --systemd=always \
  --restart always \
  --cap-add=SYS_ADMIN \
  -p 443:443 -p 80:80 -p 389:389 -p 636:636 \
  -p 88:88 -p 464:464 -p 88:88/udp -p 464:464/udp \
  -h ipa.example.edu\
  almalinux:9 /usr/sbin/init

In here,

--systemd=always allows systemd to run inside the container.
--restart always makes sure the container starts again after a reboot or crash.
The -p values open the ports needed by FreeIPA.

Step 3: Enter the Container
To install and configure FreeIPA it needs to go inside the running container.

podman exec -it freeipa-server /bin/bash

Step 4: Prepare the Container
As said in the previous blog, the AlmaLinux image is very minimal. So for those missing folders and configuration files needed, it should be created those required folders, install packages, restore LDAP schema files, and set some security options

# 1. Install foundational packages
dnf install -y ipa-server ipa-server-dns

# 2. Reconstruct missing directory structures
rm -rf /etc/dirsrv /etc/sysconfig /etc/tmpfiles.d /etc/pkcs11
mkdir -p /etc/sysconfig /etc/tmpfiles.d /etc/pkcs11/modules /etc/dirsrv/config
mkdir -p /var/lib/ipa/sysrestore /var/lib/ipa-client/sysrestore /var/log/dirsrv

# 3. Restore missing LDAP schemas and configure cryptographic policies
dnf reinstall -y 389-ds-base --setopt=tsflags=noscripts --setopt=sslverify=false
echo "module: /usr/lib64/libsofthsm2.so" > /etc/pkcs11/modules/softhsm2.module
update-crypto-policies --set LEGACY

# 4. Generate necessary Kerberos configuration placeholders
touch /etc/sysconfig/krb5kdc /etc/sysconfig/kadmin
chmod 644 /etc/sysconfig/krb5kdc /etc/sysconfig/kadmin

# 5. Map local networking and identity alignment
MY_IP=$(hostname -I | awk '{print $1}')
echo -e "127.0.0.1\tlocalhost\n$MY_IP\tipa.example.edu ipa" > /etc/hosts

# 6. Set Java and Certificate Authority environment variables
export JAVA_OPTS="-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true"
export NSS_SDB_USE_CACHE=yes

Step 5: Install FreeIPA
Then the FreeIPA installer can be run in unattended mode as the container is now ready.

ipa-server-install \
  --unattended \
  --domain=example.edu \
  --realm=EXAMPLE.EDU \
  --ds-password=<YOUR_DS_PASSWORD> \
  --admin-password=<YOUR_ADMIN_PASSWORD> \
  --no-ntp \
  --no-host-dns \
  --no-pkinit \
  --skip-mem-check

Replace YOUR_DS_PASSWORD and YOUR_ADMIN_PASSWORD with your own secure passwords.

Step 6: Access the Web Interface

Local Access

If the container is on a local system, add this line to the hosts file:

127.0.0.1 ipa.example.edu

Then open the browser:

https://ipa.example.edu

Username: admin
Password: YOUR_ADMIN_PASSWORD

Remote Access via SSH Tunnel

If the container is on a remote server, use an SSH tunnel:

ssh -L 443:localhost:443 -L 80:localhost:80 user@remote-server

Then add the same hosts line locally and open the browser.

Step 7: Verify Setup

Inside the container, verify that FreeIPA and Kerberos are working:

kinit admin
ipa user-show admin

Step 8: Make Port Change Permanent

The earlier port change will reset after a reboot. To keep it:

echo "net.ipv4.ip_unprivileged_port_start=80" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Checking FreeIPA Service Status

Testing Kerberos Authentication

Running FreeIPA on Ubuntu Using Podman – Part 1: Overview

Chanuth Abeynayake — Sat, 16 May 2026 19:14:12 +0000

There was this consideration to replace OpenLDAP with FreeIPA in the university authentication infrastructure. The main reason for this was to improve the security in the system and to give more identity management features which the OpenLDAP lacks.

Some of these features are

Ticket based authentication instead of sending the passwords over the network constantly.
Centralized user and authentication management.
Certificate for verifying systems and services etc.

The issue was that the existing university infrastructure is mainly based on Ubuntu Linux systems, but FreeIPA is mainly designed for Red Hat-based systems like AlmaLinux , Centos etc. Therefore direct installation of FreeIPA was not a choice here.
Therefore, this system needed to be deployed in a containerized manner.

So this article mainly focuses on the decisions made, issues that came across with and what solutions were used to fix them.

Introduction to FreeIPA
FreeIPA is an integrated identity management system used in Linux environments but was mainly developed in Red Hat based systems. It is integrated of set of services as,

389 Directory Server (LDAP): Stores the identity data.
MIT Kerberos (KDC):Handles the tickets for SSO authentication
Apache (HTTPD): The administrative web dashboard and API.
Dogtag Certificate System (PKI):The Certification Authority (CA) for the secure communication between systems.

Service Startup Order
These services need to be started in the specific order given above. As the LDAP has the initial configuration settings for other services on it, it needs to start first, then KDC as it needs LDAP data for authentication, next the dashboard and finally the PKI (Dogtag CA) service as it depends on other main services being ready.

If any of the services did not start with this order the whole system crashes. For this order to be preserved it uses the systemd in here.

Platform Compatibility Issues
As said above direct installation of FreeIPA on to the existing ubuntu systems was not a solution.

The Centos Stream 9 container image lacked the required binaries, so it was unable to even start.

But Almalinux was compatible with FreeIPA. However, changing all the existing systems to Almalinux was not a solution also. That is why it was thought to use a containerization appproach to run an Almalinux environment while still keeping the existing Ubuntu infrastructure unchanged.

CentOS 9 image does not start properly

Failure 1: Docker and Cgroups v2 Incompatibility
systemd and cgroups - When systemd runs as the PID1 inside a container it is responsible for starting and managing the other services there. It uses the cgroups to control resources like memory and CPU. To do that systemd needs read, write access to cgroup file system at /sys/fs/cgroup .

Modern Ubuntu use cgroups v2, which is stricter. This causes Docker container to not get proper access to system resources causing systemd to fail.

So as the systemd is the PID1, failing it prevents the other services to start, causing the container to run but without running any FreeIPA services in it.

Docker blocks system access needed for systemd to run properly

There were many attempts to fix this issue by changing cgroup isolation settings using flags like --cgroupns=host etc ,but nothing worked.
Similar outcomes were also obtained by other attempts like these. https://github.com/moby/moby/issues/16238

Transition to Podman
To solve these issues, Podman was thought to be used considering 2 main reasons.

systemd support - As podman supports systemd-based containers directly, it automatically configures the container environment to run systemd correctly.
Compatibility - As Podman fits better with the same ecosystem use by FreeIPA and AlmaLinux as it was also made by Red Hat ecosystem. This reduced the runtime issues.

Failure 2: CA Initialization in Pre-packaged Images
Then when tried to get an official container image of FreeIPA to be run in this, it failed during the Certificate Authority(CA) setup.

This was because the official container images are built to be minimal. So some parts of the operating system environments were removed and simplified.

Specially, the default directory folders and the services that auto-create them were missing. These couldn't be created manually during the installation as FreeIPA is a tightly coupled package.(To be installed as a complete system)

Due to those missing directories, CA setup failed, causing no certificates to be created. As LDAP needs certificates to have secure connections, LDAP fails, failing the whole system.

FreeIPA CA setup failed because the container was incomplete

Controlled Setup Process
Because important directories, security settings, and LDAP schema files needed for installation were missing, the setup failed as said above.
As a solution then a clean AlmaLinux 9 base image was obtained and it was prepared for required conditions before running the FreeIPA installer inside it.

Main preparations done was:

Creating the necessary filesystems
Temporarily allowing security policies for CA initialization
Restoring the missing LDAP schema files

After doing these steps, the FreeIPA installation was completed successfully and the system ran successfully after that.

All services in the FreeIPA are running properly now

As this is a series of articles, in the next part a full guide on starting FreeIPA in a Podman container, including all the preparation steps will be provided.