Using AI for Terraform: flows, prompts, and agents with LangFlow & Docling

In part 1, I learned to deploy a local stack for running an AI agent. Originally, I thought I could generate "good" Terraform configuration based on all my content for secure and scalable infrastructure as code.

Shortly after, I received a message encouraging me to think about the second edition of my book. I have procrastinated on this for a while now since it takes time to revise material and generate new examples. One of the biggest challenges with my book was writing examples. Initial feedback suggested I write everything in Python for greater accessibility and Google Cloud for lower cost, which I did. In retrospect, I should have just written everything in Terraform to run on AWS.

As I reflected on this further, I realized something important. Isn't book writing the perfect use case for an AI agent? If I had agent who knew my writing style help me write new examples in Terraform for my book, maybe I could expedite the process of creating a second edition.

With my regrets in mind, I decided to try to create a "book writing" agent that helps me generate examples to match my text. After all, I had the chapters of the book written. I wanted new examples to reframe some of the principles and practices. This sent me on a major exploration of prompts, agent instructions, and flows in LangFlow.

Process PDFs with Docling

I had editable drafts of the chapters, but only the final PDF version of the book had the proper code annotations and figures. I needed to process the PDF book chapters into text and images before chunking and storing them in my OpenSearch vector database. Enter Docling, a document processing tool for unstructured data.

Fortunately, LangFlow has a Docling component for processing a set of files and chunking them. You do have to install it before you run LangFlow.

uv pip install langflow[docling]

When you start creating a flow in LangFlow, drag-and-drop the Docling component.

There are a few attributes you need to consider with Docling:

• Pipeline - I opted for standard just to process the text. If I wanted to also process the figures, I could select vlm (Visual Language Model).
• OCR Engine - None for now. I wanted to test if I had sufficient resources to run Docling.

My laptop constrains the amount of memory Docling can use to process the documents, which is why I did not upload the entire book or use VLM or OCR.

Next, I need to chunk the text into my vector database. Rather than do fixed-size chunking, I decided to try hybrid chunking which combines fixed-size chunking with semantic chunking. This ensures that the various chapters of my book have chunks with proper context. After chunking three chapters of my book, I stored the chunks in the OpenSearch vector database using Granite embeddings hosted by Ollama.

Besides the PDF chapters of the book, I had a few blogs on best practices for writing Terraform. These had some text and examples that I wanted to include as part of the agent's response. Using the URL component, I added the set of URLs for the blog posts and passed it to the vector database.

Now that I had my expert-level content on infrastructure as code and Terraform practices into my vector database, I could use an agent to reference that context.

Create the Terraform coding agent

I started with what I thought was the easier agent to build - a coding agent that generates "good" Terraform. This agent needed to generate Terraform configuration with the following:

• Proper resource and module declarations - no hallucinations please
• Correct formatting
• All variables and outputs defined

In addition to writing proper working Terraform, I wanted the agent to include a good example. I had a demo repository that I constantly copied and pasted into other repositories, so I wanted the agent to reference that configuration when the prompt matched.

With all these requirements, I realized I need to use two MCP servers:

• Terraform MCP server for the latest resource and module documentation
• GitHub MCP server for getting files in my reference repository

I did not need all the tools available on these MCP servers. From an access control perspective, I used the following for each MCP server:

• Terraform MCP server - get_latest_module_version, get_latest_provider_version, get_module_details, get_provider_details, get_provider_capabilities, search_modules, and search_providers
• GitHub MCP server - get_file_contents and search_code

My agent could retrieve the latest Terraform modules and providers or search GitHub for reference code. I connected the GitHub and Terraform MCP servers with the URL component to my agent using LangFlow.

Finally, the coding agent needed instructions. I started with the official agent skills for Terraform and refined the instructions to better suite Granite and my use case. It took quite a bit of trial-and-error. The full set of instructions are on a GitHub repository. I had two main observations.

First, I had be very specific about which repository and commit the agent should reference for a "good" Terraform example. It turns out the MCP component in LangFlow cannot handle optional parameters at the time of this post so I had to put the exact commit hash and branch the agent should reference in the agent instructions.

Second, the prompt had to include the specific module and resource I wanted the example to include (e.g., Create an example with the aws_opensearchserverless_collection resource.) If I did not include the exact module or resource, Granite would search for the wrong module or resource with the Terraform MCP server. After adjusting my expectations on the prompts, I moved onto the book writing agent.

Create the book writing agent

Why did I separate the book writing task into its own agent? I discovered that combining both Terraform generation and an explanation based on context from OpenSearch led to very poor results. The agent was tasked with doing too much ("generate the explanation AND the example"), which led to some garbled responses that made little sense.

I decided to split the book writing into its own agent so it could properly draft a response that sounds like I wrote the paragraph and not just reiterating the Terraform configuration. This worked much better overall. I also moved the book writing agent first, since it could generate the explanation and the Terraform coding agent could adjust the example.

I connected the output of the writer agent to the input of the coding agent. This ensures that the explanation includes references to the expected resources and examples from user input.

The full set of instructions are on the demo GitHub repository. I wanted it to include the sources so I could check exactly where it found the information.

With both agents, I am going to do more work to refine the prompts. They do a decent job of producing a semi-coherent explanation and example but I think I can improve them with more review.

The result

I passed in a few prompts to test out my agents. While the results needed some edits, they turned out more usable than I expected. For example, I asked the following:

Explain the singleton pattern using the aws_opensearchserverless_collection resource. Include ideas on when to refactor from a singleton to a composite module.

The agent returned a very lengthy explanation, which I'll include a few excerpts below. The first paragraph wasn't completely incorrect but needed to remove the mention of Google project resources since I wanted to use AWS for examples.

The singleton pattern is commonly used in Terraform configurations to manage resources that should exist only once within an environment, such as Google project configurations. In the context of AWS OpenSearch Serverless collections, this pattern ensures that there is a single instance of the collection resource, which typically does not change frequently.

As for the example, it did use the resource I requested but it did not use the right arguments. For example, domain_id doesn't exist for the aws_opensearchserverless_collection.

resource "aws_opensearchserverless_collection" "main" {
  name        = var.collection_name
  domain_id   = aws_opensearch_serverless_domain.main.id
  description = "Primary collection for ${var.environment}"

  tags = merge(
    local.common_tags,
    {
      Environment = var.environment
      Project     = var.project_name
    }
  )
}

When I asked the agent to explain when to refactor from a singleton to composite module, it provided a correct but very generic explanation.

Refactoring from a singleton pattern to a composite module becomes necessary when:

1. Resource Evolution: Resources evolve beyond single instances, requiring multiple configurations or variations.
2. Shared Configurations: Multiple resources need shared configurations that are not strictly unique but require common parameters.
3. Complex Workflows: The infrastructure management involves complex workflows where different components interact and share state.

I asked the book writing agent to apply the other principles when possible, which it did try in later paragraphs. In general, the agent provided a good start for me to edit and iterate on the explanation. I would not use the response as-is, since it has some incorrect points and the explanation is far too generic, but it does write it in the style and tone of voice that I would write in my book.

Conclusion

I learned quite a bit about prompt engineering while trying to build a book writing agent for myself. In some situations, I had to be very specific about how and where an agent should refer to certain tools or data sources. In some of the first iterations, the agent kept bringing up other principles like simplicity, which I do not mention in my book. I had to ask the agent for the source of the principle, which was another book entirely.

In general, the agent did improve over time. The more I asked of it and provided feedback, the better the response it generated. However, I still wouldn't use the response in the book without some editing. I could use the examples, for the most part, but I had to check them correctness and clarity.

Next, I plan on moving these components off my local machine into some cloud infrastructure.