<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Vijaya Rajeev Bollu</title>
    <description>The latest articles on DEV Community by Vijaya Rajeev Bollu (@vijaya_bollu).</description>
    <link>https://dev.to/vijaya_bollu</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3808851%2F28130c4b-5e4a-4938-94b6-370a13c89737.png</url>
      <title>DEV Community: Vijaya Rajeev Bollu</title>
      <link>https://dev.to/vijaya_bollu</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/vijaya_bollu"/>
    <language>en</language>
    <item>
      <title>Why a Program Can Fail Before main() Runs on Linux</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 30 Oct 2026 03:35:16 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/why-a-program-can-fail-before-main-runs-on-linux-2084</link>
      <guid>https://dev.to/vijaya_bollu/why-a-program-can-fail-before-main-runs-on-linux-2084</guid>
      <description>&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;A program works on your laptop. You copy it to another Linux machine, run the same command, and it fails immediately, before any of your own code has executed. This article follows one small imaginary program, &lt;code&gt;orders&lt;/code&gt;, from source code to exit, so you can place any startup failure at the right step. &lt;code&gt;orders&lt;/code&gt; reads a list of orders and prints a total. It is illustrative: nothing here was measured, and no outputs are shown that were not produced.&lt;/p&gt;

&lt;p&gt;Behavior described is for Linux with the glibc dynamic linker unless stated. Other systems and C libraries can differ.&lt;/p&gt;

&lt;h2&gt;
  
  
  Source, machine code and the executable file
&lt;/h2&gt;

&lt;p&gt;These are three different things:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Source code&lt;/strong&gt; is text for people.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Machine code&lt;/strong&gt; is what the processor runs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The executable file&lt;/strong&gt; packages the machine code with what the OS needs to load it. On Linux this is usually an ELF file. It records where each part goes in memory, which dynamic linker to use, and which shared libraries the program needs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Not every language works this way. CPython compiles source to bytecode and runs it on a virtual machine; bytecode is not machine code, and the Python documentation says it is an implementation detail that can change between versions. Java source compiles to class files with bytecode; the JVM loads, links and initializes classes. So the useful question is not "compiled or interpreted?" but "what exists on the machine at the moment the program starts?" The C version arrives as an executable. The Python version needs an interpreter installed. The Java version needs a Java runtime.&lt;/p&gt;

&lt;h2&gt;
  
  
  From the shell to a process
&lt;/h2&gt;

&lt;p&gt;A shell starts a command in two separate steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;fork&lt;/code&gt; creates a new process by duplicating the caller. The child has its own process ID and a memory space with the same content as the parent's at that moment. On Linux, pages are copy-on-write.&lt;/li&gt;
&lt;li&gt;In the child, &lt;code&gt;execve&lt;/code&gt; replaces the program being run with a new one, with fresh stack, heap and data. It does not return on success because the old image is gone. The process ID stays the same.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Fork alone gives you copies of the shell. Exec alone would replace the shell with your program. Starting a command needs both, in that order.&lt;/p&gt;

&lt;p&gt;What survives exec:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Thing&lt;/th&gt;
&lt;th&gt;After execve&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Open file descriptors&lt;/td&gt;
&lt;td&gt;stay open, unless marked close-on-exec&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Environment&lt;/td&gt;
&lt;td&gt;passed to the new program&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Signal handlers&lt;/td&gt;
&lt;td&gt;reset to default (ignored signals stay ignored)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Memory mappings&lt;/td&gt;
&lt;td&gt;not preserved; fresh layout&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Scripts start through exec too: if the file is executable and its first line is &lt;code&gt;#!&lt;/code&gt; followed by an interpreter, the kernel runs that interpreter with the script path.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the process looks like inside
&lt;/h2&gt;

&lt;p&gt;Each process has its own virtual address space. Regions: code (text), data (globals and statics), heap (memory requested at run time), stack (function-call frames), and mappings including shared libraries. On a real Linux system, &lt;code&gt;/proc/&amp;lt;pid&amp;gt;/maps&lt;/code&gt; lists each mapped region with its address range, permissions, whether it is shared or private, and the backing file if any.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Read the map of the current shell process (standard Linux command).&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /proc/self/maps
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If a program touches memory it is not allowed to use, the kernel sends SIGSEGV, and the default action is to terminate the process. Reaching the stack size limit also generates SIGSEGV.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the dynamic linker finds shared libraries
&lt;/h2&gt;

&lt;p&gt;If &lt;code&gt;orders&lt;/code&gt; uses a shared library, that file must exist on the machine where it runs. The ELF file names a program interpreter (&lt;code&gt;PT_INTERP&lt;/code&gt;), which on glibc systems is the dynamic linker &lt;code&gt;ld.so&lt;/code&gt;. It runs before &lt;code&gt;main&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;For a dependency name without a slash, the glibc linker searches in this order:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Order&lt;/th&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;DT_RPATH&lt;/code&gt; in the executable, only if there is no &lt;code&gt;DT_RUNPATH&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;The &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt; environment variable&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;DT_RUNPATH&lt;/code&gt; in the executable&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;The cache file &lt;code&gt;/etc/ld.so.cache&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;Default directories (&lt;code&gt;/lib&lt;/code&gt;, &lt;code&gt;/usr/lib&lt;/code&gt;; &lt;code&gt;lib64&lt;/code&gt; on some 64-bit systems)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;DT_RUNPATH&lt;/code&gt; applies to direct dependencies; &lt;code&gt;DT_RPATH&lt;/code&gt; also applies to the dependencies of those. In secure-execution mode (for example set-user-ID programs), &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt; is ignored, so a user-controlled variable cannot redirect a privileged program.&lt;/p&gt;

&lt;p&gt;If the library is in none of those places, the linker cannot finish preparing the program and the process ends before &lt;code&gt;main&lt;/code&gt;. No file was opened, no log line was written, and none of your tests could have caught it. The program is fine; the machine lacks what the program declared. A container image that omits a needed library behaves the same way.&lt;/p&gt;

&lt;p&gt;To inspect what an executable declares (standard binutils commands; outputs vary by binary):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Which program interpreter does this ELF file name?&lt;/span&gt;
readelf &lt;span class="nt"&gt;-l&lt;/span&gt; ./orders | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; interpreter

&lt;span class="c"&gt;# Which shared libraries does it list as needed?&lt;/span&gt;
readelf &lt;span class="nt"&gt;-d&lt;/span&gt; ./orders | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; NEEDED
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Interpreted languages ask the same question at a different time: Python fails when an import runs for a package that is not installed, and Java fails when the runtime cannot find a class.&lt;/p&gt;

&lt;h2&gt;
  
  
  The startup stages and where each one fails
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stage&lt;/th&gt;
&lt;th&gt;What happens&lt;/th&gt;
&lt;th&gt;Typical failure&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Shell + kernel&lt;/td&gt;
&lt;td&gt;find the file, check permission, build the image&lt;/td&gt;
&lt;td&gt;not found (127), not executable (126)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Dynamic linker&lt;/td&gt;
&lt;td&gt;find and load shared libraries&lt;/td&gt;
&lt;td&gt;missing or incompatible library&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Language runtime&lt;/td&gt;
&lt;td&gt;C library setup; interpreter or JVM starts, imports or loads classes, runs initialization&lt;/td&gt;
&lt;td&gt;missing config or module, import error&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Your code&lt;/td&gt;
&lt;td&gt;first line of your own logic&lt;/td&gt;
&lt;td&gt;bugs in your logic&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A service that never listens on its port might have failed in the linker, the runtime, or its own configuration code. Asking which handoff it reached tells you where to look.&lt;/p&gt;

&lt;h2&gt;
  
  
  How programs end, and what the numbers mean
&lt;/h2&gt;

&lt;p&gt;There are two different ways for a process to end.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Normal exit.&lt;/strong&gt; The program returns from &lt;code&gt;main&lt;/code&gt; or calls &lt;code&gt;exit&lt;/code&gt; with a number. The parent calls &lt;code&gt;wait&lt;/code&gt;, and the status can be decoded: one macro reports normal termination, another returns the exit status (the least significant eight bits, so 0 to 255). By convention 0 is success; non-zero is failure.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Termination by a signal.&lt;/strong&gt; SIGKILL and SIGSTOP cannot be caught, blocked or ignored. The default action for SIGSEGV is to terminate the process and dump core, if the core file size limit is not zero. The parent can tell this apart from a normal exit using a different &lt;code&gt;wait&lt;/code&gt; macro.&lt;/p&gt;

&lt;p&gt;Bash then folds both into one number. When a command dies on a fatal signal numbered N, Bash reports 128 + N:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Documented Bash convention: killed by SIGKILL (signal 9 on x86 and ARM) -&amp;gt; 128 + 9.&lt;/span&gt;
&lt;span class="c"&gt;# Shown as a pattern, not as captured output.&lt;/span&gt;
&lt;span class="nb"&gt;sleep &lt;/span&gt;60 &amp;amp;
&lt;span class="nb"&gt;kill&lt;/span&gt; &lt;span class="nt"&gt;-9&lt;/span&gt; &lt;span class="nv"&gt;$!&lt;/span&gt;
&lt;span class="nb"&gt;wait&lt;/span&gt; &lt;span class="nv"&gt;$!&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$?&lt;/span&gt;     &lt;span class="c"&gt;# expected by the convention: 137&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;127&lt;/td&gt;
&lt;td&gt;Bash could not find the command&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;126&lt;/td&gt;
&lt;td&gt;found, but not executable&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;128+N&lt;/td&gt;
&lt;td&gt;terminated by signal N (137 = SIGKILL)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The Bash manual also warns that the shell may use values above 125 specially, so statuses in that range deserve a second look. Signal numbers vary on some architectures.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;137 does not prove an out-of-memory kill.&lt;/strong&gt; The kernel's out-of-memory killer is one possible cause: a control group that reaches its memory limit and cannot reduce usage has the OOM killer invoked inside that group, which is how a memory-limited container can lose its main process. But &lt;code&gt;kill -9&lt;/code&gt;, a supervisor, or another program sending SIGKILL gives the same 137. To claim an OOM kill you need more evidence, such as the kernel's report of the tasks involved or the OOM kill counter in the cgroup memory events file. The number is a lead, not a verdict.&lt;/p&gt;

&lt;p&gt;Runtimes add their own conventions. In Node.js, an uncaught exception by default prints the stack trace to standard error and exits with code 1.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to take away
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Source, machine code and the executable file are different things; what exists at startup differs by language.&lt;/li&gt;
&lt;li&gt;Starting a command is &lt;code&gt;fork&lt;/code&gt; then &lt;code&gt;execve&lt;/code&gt;: create a process, then replace what it runs.&lt;/li&gt;
&lt;li&gt;The dynamic linker runs before &lt;code&gt;main&lt;/code&gt; and searches a fixed order; a missing library ends the process before your code starts.&lt;/li&gt;
&lt;li&gt;127 is not found, 126 is not executable, 128+N is a signal, and 137 is SIGKILL, not proof of memory exhaustion.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Limits of this article
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;orders&lt;/code&gt; is imaginary. Commands above are standard Linux commands shown as patterns; they were not run for this article, so outputs are not reproduced. The linker order is glibc on Linux. Python bytecode and JVM/HotSpot internals vary by version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;fork(2), execve(2), wait(2), signal(7), getrlimit(2), ld.so(8), proc_pid_maps(5): &lt;a href="https://man7.org/linux/man-pages/" rel="noopener noreferrer"&gt;https://man7.org/linux/man-pages/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Bash Reference Manual, Exit Status: &lt;a href="https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html" rel="noopener noreferrer"&gt;https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Kernel cgroup v2 and vm sysctl documentation: &lt;a href="https://docs.kernel.org/admin-guide/cgroup-v2.html" rel="noopener noreferrer"&gt;https://docs.kernel.org/admin-guide/cgroup-v2.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Python glossary (bytecode): &lt;a href="https://docs.python.org/3/glossary.html" rel="noopener noreferrer"&gt;https://docs.python.org/3/glossary.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;JVM Specification SE 21, chapter 5: &lt;a href="https://docs.oracle.com/javase/specs/jvms/se21/html/jvms-5.html" rel="noopener noreferrer"&gt;https://docs.oracle.com/javase/specs/jvms/se21/html/jvms-5.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Node.js process documentation: &lt;a href="https://nodejs.org/api/process.html" rel="noopener noreferrer"&gt;https://nodejs.org/api/process.html&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;Video version with animations: [&lt;a href="https://youtu.be/FFZaY7Di66c" rel="noopener noreferrer"&gt;https://youtu.be/FFZaY7Di66c&lt;/a&gt;]. Next in the series: CPU, memory and storage for developers.&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>beginners</category>
      <category>debugging</category>
    </item>
    <item>
      <title>ArgoCD Silently Stuck at "Unknown" Sync Status? It Was a Kubernetes Version, Not ArgoCD</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Wed, 16 Sep 2026 13:10:56 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/argocd-silently-stuck-at-unknown-sync-status-it-was-a-kubernetes-version-not-argocd-27gj</link>
      <guid>https://dev.to/vijaya_bollu/argocd-silently-stuck-at-unknown-sync-status-it-was-a-kubernetes-version-not-argocd-27gj</guid>
      <description>&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;Adding a GitOps delivery layer on top of an existing 11-service Online Boutique deployment: ArgoCD ApplicationSet generating dev/staging/prod Applications, Argo Rollouts canary for the frontend service, Sealed Secrets for committing encrypted secrets. Everything ran on a single minikube cluster with three namespaces simulating the three environments — no extra cloud cost.&lt;/p&gt;

&lt;p&gt;Every manifest applied cleanly. Every Application generated correctly. And then every single one of them sat at &lt;code&gt;Sync Status: Unknown&lt;/code&gt; — forever.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Investigation
&lt;/h2&gt;

&lt;p&gt;No error in &lt;code&gt;kubectl describe application online-boutique-dev&lt;/code&gt;. No crash in the ArgoCD application-controller pod. Just &lt;code&gt;Unknown&lt;/code&gt;, indefinitely.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get applications &lt;span class="nt"&gt;-n&lt;/span&gt; argocd
NAME                     SYNC STATUS   HEALTH STATUS
online-boutique-dev      Unknown       Healthy
online-boutique-staging  Unknown       Missing
online-boutique-prod     Unknown       Missing
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Checked the application-controller logs directly — buried among routine warnings:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;W... controller.go:139] slow openapi aggregation of "applicationsets.argoproj.io": 20.808634845s
E... timeout.go:140] "Post-timeout activity" ... path="/openapi/v2" result=null
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Tested the endpoint directly, ruling out a timeout:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get &lt;span class="nt"&gt;--raw&lt;/span&gt; /openapi/v2
&lt;span class="c"&gt;# clean, fast 404 — not hanging, just not there&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The Reveal
&lt;/h2&gt;

&lt;p&gt;Kubernetes 1.35.1 — minikube's default at the time — no longer serves &lt;code&gt;/openapi/v2&lt;/code&gt;. ArgoCD v2.13.2's application-controller depends on that exact endpoint for its schema-based diff between live cluster state and Git. Without it, the controller can generate Applications and even sync them, but it can never actually compute whether they're in sync. It just sits at &lt;code&gt;Unknown&lt;/code&gt;, silently, with nothing in the UI or logs pointing at the real cause.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;Pin a dedicated Kubernetes profile to a version that still serves the endpoint, without touching the original cluster:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;minikube start &lt;span class="nt"&gt;-p&lt;/span&gt; v4-gitops &lt;span class="nt"&gt;--driver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;docker &lt;span class="nt"&gt;--kubernetes-version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;v1.30.0 &lt;span class="nt"&gt;--cpus&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;6 &lt;span class="nt"&gt;--memory&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;6500
kubectl get &lt;span class="nt"&gt;--raw&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;/openapi/v2 | &lt;span class="nb"&gt;head&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; 100
&lt;span class="c"&gt;# {"swagger":"2.0","info":{"title":"Kubernetes","version":"v1.30.0"},...}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The Result
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NAME                  SYNC STATUS   HEALTH STATUS
online-boutique-dev   Synced        Healthy
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;From there, the rest of the GitOps flow proved out live. Manually scaling a deployment outside Git:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NAME           DESIRED   READY
adservice      5         1
SYNC        HEALTH
OutOfSync   Progressing
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;...and self-heal reverting it, unattended:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NAME           DESIRED   READY
adservice      1         1
SYNC     HEALTH    PHASE
Synced   Healthy   Succeeded
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And a genuinely bad image commit going &lt;code&gt;Degraded&lt;/code&gt;, fixed the GitOps way — not with &lt;code&gt;kubectl&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;REVISION      SYNC        HEALTH     PHASE
14151c9a...   OutOfSync   Degraded   Running

$ git revert 14151c9a
[f0e6a50f] Revert "test(gitops): deploy invalid frontend image"

NAME                  SYNC STATUS   HEALTH STATUS   REVISION
online-boutique-dev   Synced        Healthy         f0e6a50f...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. A missing API endpoint can fail silent, not loud.&lt;/strong&gt; No crash, no error surfaced anywhere obvious — just a status field that never resolves. Check the controller's own logs before assuming the manifests are wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Version pinning is a legitimate fix, not a workaround to feel bad about.&lt;/strong&gt; A dedicated profile pinned to a known-good Kubernetes version, kept separate from the primary cluster, cost nothing and solved it cleanly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Self-heal and git-revert are two different rollback tools.&lt;/strong&gt; Self-heal fixes drift — someone changed the cluster without touching Git. &lt;code&gt;git revert&lt;/code&gt; fixes a bad commit — Git said something wrong, and Git is what has to say something right again. Neither substitutes for the other.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. The whole exercise ran on one minikube cluster with three namespaces, not three separate clusters&lt;/strong&gt; — zero cloud cost to prove a real multi-environment GitOps promotion model end to end.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub: &lt;a href="https://github.com/ThinkWithOps/thinkwithops-online-boutique-production" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/thinkwithops-online-boutique-production&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Youtube: &lt;a href="https://youtu.be/2CJqoVwJEqY" rel="noopener noreferrer"&gt;https://youtu.be/2CJqoVwJEqY&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/thinkwithops-online-boutique-production.git
&lt;span class="nb"&gt;cd &lt;/span&gt;thinkwithops-online-boutique-production
git checkout v4.0-gitops-argocd
minikube start &lt;span class="nt"&gt;-p&lt;/span&gt; v4-gitops &lt;span class="nt"&gt;--driver&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;docker &lt;span class="nt"&gt;--kubernetes-version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;v1.30.0 &lt;span class="nt"&gt;--cpus&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;6 &lt;span class="nt"&gt;--memory&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;6500
kubectl apply &lt;span class="nt"&gt;-n&lt;/span&gt; argocd &lt;span class="nt"&gt;-f&lt;/span&gt; https://raw.githubusercontent.com/argoproj/argo-cd/v2.13.2/manifests/install.yaml
kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; argocd/project.yaml &lt;span class="nt"&gt;-f&lt;/span&gt; argocd/applicationset.yaml
kubectl get applications &lt;span class="nt"&gt;-n&lt;/span&gt; argocd
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;Has anyone else hit a silent &lt;code&gt;/openapi/v2&lt;/code&gt; gap after a Kubernetes minor version bump? Curious how many other controllers quietly depend on that same endpoint.&lt;/p&gt;




</description>
      <category>kubernetes</category>
      <category>argocd</category>
      <category>devops</category>
      <category>git</category>
    </item>
    <item>
      <title>Can My AWS EKS Cluster Handle 100,000 Users? Load Testing + Autoscaling, Proven Live</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 04 Sep 2026 02:08:58 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/can-my-aws-eks-cluster-handle-100000-users-load-testing-autoscaling-proven-live-274e</link>
      <guid>https://dev.to/vijaya_bollu/can-my-aws-eks-cluster-handle-100000-users-load-testing-autoscaling-proven-live-274e</guid>
      <description>&lt;p&gt;Part 2 of a 4-part series — this article covers the repo at tag v2.0-load-testing-autoscaling. Part 1 (EKS deployment, tag v1.0-eks-deployment) is &lt;a href="https://youtu.be/qjnJab8mqcI" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;Last build: Google's Online Boutique running on a fixed 2-node AWS EKS cluster — 11 services, one Helm release, no autoscaling. The question this time: what actually happens under load? Not "the YAML looks right" — actually driven, actually watched, actually proven.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Build
&lt;/h2&gt;

&lt;p&gt;Two changes to the cluster from the last video:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;kubernetes-manifests-aws/
├── hpa/              → HorizontalPodAutoscaler for 10 services, CPU + memory targets
└── metrics-server/   → the add-on HPA depends on — verified running first

karpenter/
├── nodepool.yaml     → replaces the fixed-size node group
└── ec2nodeclass.yaml → instance requirements Karpenter provisions against
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;metrics-server is the dependency almost nobody checks first. HPA reads CPU/memory percentages from it — if it's not running, &lt;code&gt;kubectl get hpa&lt;/code&gt; just shows &lt;code&gt;&amp;lt;unknown&amp;gt;&lt;/code&gt; targets forever, silently. First command in this build: &lt;code&gt;kubectl get deployment metrics-server -n kube-system&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Load Test
&lt;/h2&gt;

&lt;p&gt;k6 runs in-cluster, ramping virtual users directly against the storefront's internal service address — no external load balancer bottleneck skewing the numbers. The test script targets 100,000 VUs at full scale; for this recorded run it's scaled down to 3,000 VUs, stated openly rather than glossed over, to keep the run inside what this AWS account's free-tier-eligible node ceiling can actually absorb.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; k6-load-test.yaml
kubectl get pods &lt;span class="nt"&gt;-n&lt;/span&gt; load-testing &lt;span class="nt"&gt;-w&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Watching HPA Respond
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get hpa &lt;span class="nt"&gt;-n&lt;/span&gt; online-boutique &lt;span class="nt"&gt;-w&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;NAME                REFERENCE                      TARGETS           MINPODS   MAXPODS   REPLICAS
frontend            Deployment/frontend             45%/60%, 210Mi    2         10        2
cartservice         Deployment/cartservice          78%/60%, 340Mi    2         10        4
checkoutservice     Deployment/checkoutservice      82%/60%, 290Mi    2         10        5
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;REPLICAS climbs live as CPU/memory cross the target threshold — not a static config, an observed response to the k6 traffic actually landing on these pods.&lt;/p&gt;

&lt;h2&gt;
  
  
  Watching Karpenter Respond
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get nodes &lt;span class="nt"&gt;-w&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Node count climbs from 2 to 8 as HPA's new replicas outgrow what the original 2 nodes can schedule. Karpenter watches for &lt;code&gt;Pending&lt;/code&gt; pods, picks an instance type from the &lt;code&gt;EC2NodeClass&lt;/code&gt; constraints, and provisions a new node — no fixed node group ceiling to hit.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get nodepool,ec2nodeclass
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The moment a &lt;code&gt;Pending&lt;/code&gt; pod's node comes &lt;code&gt;Ready&lt;/code&gt;, the scheduler places it immediately — that transition, pod snapping onto a freshly-provisioned node, is the single clearest proof autoscaling is actually working end to end.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Dashboard
&lt;/h2&gt;

&lt;p&gt;Grafana at &lt;code&gt;/d/online-boutique-autoscaling&lt;/code&gt; plots HPA replica counts, node counts, and CPU/memory per service on one screen — every panel trending upward together during the ramp, and back down together once k6 finishes and Karpenter consolidates idle nodes away.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Result
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get hpa &lt;span class="nt"&gt;-n&lt;/span&gt; online-boutique
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;10 services scaled, &lt;code&gt;REPLICAS&lt;/code&gt; up across the board.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get nodes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;8 &lt;code&gt;Ready&lt;/code&gt; nodes at peak, back to 2 within minutes of load testing stopping — Karpenter's consolidation reclaiming capacity nobody's using anymore.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. Verify metrics-server before trusting HPA at all.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;An HPA with no metrics source doesn't error loudly — it just sits at &lt;code&gt;&amp;lt;unknown&amp;gt;&lt;/code&gt; targets and never scales. Check the dependency first, every time.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Karpenter removes the node-group ceiling, not the need to think about limits.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No fixed max-node count means no silent capacity wall — but &lt;code&gt;EC2NodeClass&lt;/code&gt; constraints (instance types, AZs) still need to match what the account is actually allowed to provision.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. A load test is worth nothing until you've watched it end to end.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;HPA config and a NodePool definition sitting in a repo prove nothing on their own. Watching REPLICAS and node count actually climb under generated traffic is the only real verification.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. State the scale-down honestly.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;3,000 VUs, not the full 100,000-VU script — said on camera, not hidden. Same honesty standard as marking ALB "provisioned, not installed" in the first build: every claim matches what the repo and the run actually did.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub (this video's exact state): &lt;a href="https://github.com/ThinkWithOps/thinkwithops-online-boutique-production/tree/v2.0-load-testing-autoscaling" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/thinkwithops-online-boutique-production/tree/v2.0-load-testing-autoscaling&lt;/a&gt;&lt;br&gt;
Demo walkthrough: &lt;a href="https://youtu.be/mjGCdLFqZ7k" rel="noopener noreferrer"&gt;https://youtu.be/mjGCdLFqZ7k&lt;/a&gt;&lt;br&gt;
Tag: &lt;code&gt;v2.0-load-testing-autoscaling&lt;/code&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git checkout v2.0-load-testing-autoscaling
kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; kubernetes-manifests-aws/hpa/
kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; karpenter/
kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; k6-load-test.yaml
kubectl get hpa,nodes &lt;span class="nt"&gt;-w&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;What's your default — Karpenter or Cluster Autoscaler — for EKS in 2026? Curious what other teams have landed on.&lt;/p&gt;




</description>
      <category>aws</category>
      <category>kubernetes</category>
      <category>karpenter</category>
      <category>devops</category>
    </item>
    <item>
      <title>I Deployed Google's Online Boutique on AWS EKS — Full Production Setup, Zero App Code Changes</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 21 Aug 2026 19:17:52 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/i-deployed-googles-online-boutique-on-aws-eks-full-production-setup-zero-app-code-changes-4aal</link>
      <guid>https://dev.to/vijaya_bollu/i-deployed-googles-online-boutique-on-aws-eks-full-production-setup-zero-app-code-changes-4aal</guid>
      <description>&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;Google's Online Boutique is an open-source 11-service e-commerce demo — Go, Python, Node.js, C#, and Java, all communicating over gRPC. It ships designed for GKE. The task here: build a complete production AWS platform around it — Terraform, EKS, ECR, IAM, Helm, GitHub Actions — without changing a single line of the application source.&lt;/p&gt;

&lt;p&gt;The application layer stays exactly as Google shipped it. Everything else gets built from scratch.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Build
&lt;/h2&gt;

&lt;p&gt;Seven Terraform files, each with one job:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;terraform-aws/
├── vpc.tf         → VPC, 3 AZs, public + private subnets, NAT Gateway
├── eks.tf         → EKS 1.30, cluster "online-boutique-production", 2× t3.small nodes
├── ecr.tf         → 12 ECR repos, one per service, image scan on push
├── iam.tf         → IRSA roles, least-privilege, no node-wide IAM
├── backend.tf     → S3 remote state, DynamoDB locking
├── variables.tf   → region, cluster name, instance types
└── outputs.tf     → cluster endpoint, kubeconfig command, ECR URLs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;VPC applied in 13 seconds. IAM roles, under a second each. EKS control plane, 9m37s — normal for EKS. Total: &lt;strong&gt;110 AWS resources, one &lt;code&gt;terraform apply&lt;/code&gt;.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Node instance type is &lt;code&gt;t3.small&lt;/code&gt; — 2 vCPU, 2GB RAM. Not an arbitrary choice: this AWS account is restricted to free-tier-eligible instance types, and &lt;code&gt;t3.small&lt;/code&gt; is the smallest one that comfortably runs all 11 lightweight services with room to spare.&lt;/p&gt;

&lt;h2&gt;
  
  
  The IAM Design
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;enable_irsa = true&lt;/code&gt; in &lt;code&gt;eks.tf&lt;/code&gt; turns on the cluster's OIDC provider — the mechanism that lets a Kubernetes ServiceAccount assume an AWS IAM role directly, without any node-wide credentials.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# node role — attached to every node, every pod inherits this&lt;/span&gt;
&lt;span class="c1"&gt;# AmazonEC2ContainerRegistryReadOnly — every pod needs to pull images&lt;/span&gt;

&lt;span class="c1"&gt;# ebs_csi_irsa_role — scoped to exactly one ServiceAccount:&lt;/span&gt;
&lt;span class="c1"&gt;# kube-system:ebs-csi-controller-sa — nothing else can assume it&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The distinction matters: the node role carries only what every pod needs regardless of which service it is (image pull). IRSA carries what only one specific pod needs. Getting that split right is most of what production IAM design actually is.&lt;/p&gt;

&lt;h2&gt;
  
  
  The CI/CD Pipeline
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;.github/workflows/aws-eks-deploy.yaml&lt;/code&gt; triggers on push to main, on version tags, or manually via &lt;code&gt;workflow_dispatch&lt;/code&gt;. The permissions block:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;permissions&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;id-token&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;write&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That one line is what lets the workflow request an OIDC token from GitHub at all. The credentials step that follows:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;aws-actions/configure-aws-credentials@v4&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;role-to-assume&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;arn:aws:iam::&amp;lt;account-id&amp;gt;:role/github-actions-deploy&lt;/span&gt;
    &lt;span class="na"&gt;aws-region&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;us-east-1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No &lt;code&gt;aws-access-key-id&lt;/code&gt;. No &lt;code&gt;aws-secret-access-key&lt;/code&gt;. GitHub issues a short-lived token proving exactly which workflow, in which repo, is running. AWS's IAM trust policy — scoped to this specific repository — accepts it and issues temporary credentials for that job only. When the job ends, the credentials are gone. Nothing to rotate, nothing to leak.&lt;/p&gt;

&lt;p&gt;The build job matrix runs all 12 services in parallel (&lt;code&gt;fail-fast: false&lt;/code&gt;), pushing each to its own ECR repo, then &lt;code&gt;helm upgrade --install&lt;/code&gt; deploys the release to EKS.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Helm Layer
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;helm-chart/values-aws-production.yaml&lt;/code&gt; layers on top of the base &lt;code&gt;values.yaml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;helm upgrade &lt;span class="nt"&gt;--install&lt;/span&gt; online-boutique helm-chart/ &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-f&lt;/span&gt; values.yaml &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-f&lt;/span&gt; values-aws-production.yaml &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--namespace&lt;/span&gt; online-boutique
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;images.repository&lt;/code&gt; points at the ECR registry, not Docker Hub. Every service carries its own resource request/limit block — &lt;code&gt;recommendationservice&lt;/code&gt; asks for 220Mi, &lt;code&gt;checkoutservice&lt;/code&gt; only 64Mi — because the scheduler uses the request value to decide which node a pod can fit on. The &lt;code&gt;serviceAccounts&lt;/code&gt; block carries an IRSA annotation wired to the ECR-pull role, applied to every service.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Result
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get nodes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;NAME                          STATUS   ROLES    AGE
&lt;/span&gt;&lt;span class="gp"&gt;ip-10-0-1-23.ec2.internal     Ready    &amp;lt;none&amp;gt;&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;4m
&lt;span class="gp"&gt;ip-10-0-2-45.ec2.internal     Ready    &amp;lt;none&amp;gt;&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;4m
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get pods &lt;span class="nt"&gt;-n&lt;/span&gt; online-boutique
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;NAME                                     READY   STATUS    RESTARTS   AGE
adservice-...                            1/1     Running   0          2m
cartservice-...                          1/1     Running   0          2m
checkoutservice-...                      1/1     Running   0          2m
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;11/11 pods &lt;code&gt;Running&lt;/code&gt;. One Helm release manages all of them.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get svc frontend-external &lt;span class="nt"&gt;-n&lt;/span&gt; online-boutique
&lt;span class="c"&gt;# EXTERNAL-IP: &amp;lt;real-elb-hostname&amp;gt;.us-east-1.elb.amazonaws.com&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Live storefront, real AWS LoadBalancer, real cart flow — add a product to cart, and that request hits &lt;code&gt;frontend&lt;/code&gt;, which calls &lt;code&gt;cartservice&lt;/code&gt;, which writes to Redis.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. IAM should be scoped per pod, not per node.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Node-wide IAM means one compromised pod can reach everything every other pod on that node can reach. IRSA fixes that — a ServiceAccount can assume exactly one role, scoped to exactly what it needs, nothing shared.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Free-tier account restrictions shape real infra decisions.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;t3.small&lt;/code&gt; wasn't picked for cost alone — this account is restricted to free-tier-eligible instance types at all. Checking &lt;code&gt;aws ec2 describe-instance-types --filters Name=free-tier-eligible,Values=true&lt;/code&gt; before writing &lt;code&gt;terraform.tfvars&lt;/code&gt; avoids surprises later.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Keyless CI/CD removes an entire class of leaked-credential incidents.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No static AWS key ever exists in GitHub Secrets. OIDC + a scoped trust policy means there's nothing long-lived to rotate or leak.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Small instance types are enough for lightweight microservices.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;2 vCPU / 2GB per node, 2 nodes, ran all 11 services comfortably. The workload never needed anything bigger.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub: &lt;a href="https://github.com/ThinkWithOps/thinkwithops-online-boutique-production" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/thinkwithops-online-boutique-production&lt;/a&gt;&lt;br&gt;
Video walkthrough: &lt;a href="https://youtu.be/qjnJab8mqcI" rel="noopener noreferrer"&gt;https://youtu.be/qjnJab8mqcI&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;cd &lt;/span&gt;terraform-aws/bootstrap &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform init &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform apply
&lt;span class="nb"&gt;cd&lt;/span&gt; ../terraform-aws
&lt;span class="nb"&gt;cp &lt;/span&gt;terraform.tfvars.example terraform.tfvars
&lt;span class="c"&gt;# fill in your AWS account ID, then:&lt;/span&gt;
terraform init &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform plan &lt;span class="nt"&gt;-out&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;tfplan &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform apply &lt;span class="s2"&gt;"tfplan"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;What's your default IAM boundary in Kubernetes — per node, or per pod? Curious how other teams are actually doing this.&lt;/p&gt;




</description>
      <category>aws</category>
      <category>kubernetes</category>
      <category>terraform</category>
      <category>devops</category>
    </item>
    <item>
      <title>I Benchmarked 3 Local LLMs on My Laptop — Here's What the Numbers Actually Show</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 05 Jun 2026 15:45:23 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/i-benchmarked-3-local-llms-on-my-laptop-heres-what-the-numbers-actually-show-4a00</link>
      <guid>https://dev.to/vijaya_bollu/i-benchmarked-3-local-llms-on-my-laptop-heres-what-the-numbers-actually-show-4a00</guid>
      <description>&lt;h2&gt;
  
  
  The Problem With Choosing a Local Model
&lt;/h2&gt;

&lt;p&gt;Everyone has an opinion on which local LLM is best.&lt;/p&gt;

&lt;p&gt;"Use Llama — it's the most popular." "Mistral 7B has the best quality." "Phi-3 Mini is small and efficient."&lt;/p&gt;

&lt;p&gt;None of these claims come with numbers. Specifically: &lt;em&gt;your&lt;/em&gt; numbers, on &lt;em&gt;your&lt;/em&gt; hardware, for &lt;em&gt;your&lt;/em&gt; workload.&lt;/p&gt;

&lt;p&gt;I built a benchmarking system to change that. Three models, 30 prompts, full latency distribution, memory profiling per inference call, and a JSON validation layer to measure structured output reliability.&lt;/p&gt;

&lt;p&gt;Here's what I found — and why the results matter for anyone deploying local models in production.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Three models tested:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;llama3.2:3b&lt;/code&gt; — 3B parameters, Q4_K_M quantization, 2 GB download&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;phi3:mini&lt;/code&gt; — 3.8B parameters, Q4_K_M, 2.3 GB download&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;mistral:7b&lt;/code&gt; — 7B parameters, Q4_K_M, 4.1 GB download&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Hardware:&lt;/strong&gt; CPU only, no GPU acceleration. This is the worst-case baseline — the scenario that exposes real latency and memory numbers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;30 test prompts across 5 categories:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Short factual (10): "What is the capital of France?"&lt;/li&gt;
&lt;li&gt;Reasoning (8): "Explain why the sky appears blue."&lt;/li&gt;
&lt;li&gt;Code generation (5): "Write a Python function to reverse a string."&lt;/li&gt;
&lt;li&gt;Structured output (5): "List 3 frameworks in JSON format with name and use_case."&lt;/li&gt;
&lt;li&gt;Multi-step (2): Complex chained reasoning tasks.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Architecture
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;POST /query
  → Pydantic validation → Ollama HTTP API → JSON Validator → QueryResponse

POST /benchmark
  → Load test_prompts.json
  → For each prompt: psutil memory before → Ollama → psutil memory after
  → NumPy: P50/P95/P99 latency, avg TPS, peak/avg memory
  → BenchmarkResult JSON
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The benchmark runs prompts sequentially, not in parallel. Parallel would contaminate the per-prompt memory measurements.&lt;/p&gt;




&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Llama 3.2 3B (Q4_K_M)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;avg_tokens_per_second&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;42.3&lt;/span&gt;
&lt;span class="na"&gt;p50_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1203&lt;/span&gt;
&lt;span class="na"&gt;p95_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3847&lt;/span&gt;
&lt;span class="na"&gt;p99_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5120&lt;/span&gt;
&lt;span class="na"&gt;peak_memory_mb&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;6953&lt;/span&gt;
&lt;span class="na"&gt;avg_memory_mb&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;6842&lt;/span&gt;
&lt;span class="na"&gt;total_test_duration_s&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;87.4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Interpretation:&lt;/strong&gt; P50 at 1.2 seconds is excellent. P95 at 3.8 seconds misses a 3-second SLA — the outliers are multi-step tasks and longer code generation. Memory is stable: the model loads once and stays hot between requests (Ollama's KV cache). Delta between peak and average is only 111 MB.&lt;/p&gt;

&lt;h3&gt;
  
  
  Phi-3 Mini (Q4_K_M)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;avg_tokens_per_second&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;4.7&lt;/span&gt;
&lt;span class="na"&gt;p50_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;29554&lt;/span&gt;
&lt;span class="na"&gt;p95_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;34127&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Interpretation:&lt;/strong&gt; 4.7 tok/s on CPU. A simple factual question takes 29 seconds. This is a CPU architecture issue — Phi-3 Mini's attention mechanism is less efficient on CPU-only inference than Llama's. With a GPU, these numbers would look very different. On CPU: not usable for interactive applications.&lt;/p&gt;

&lt;h3&gt;
  
  
  Mistral 7B (Q4_K_M)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;avg_tokens_per_second&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;28.1&lt;/span&gt;
&lt;span class="na"&gt;p50_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;2301&lt;/span&gt;
&lt;span class="na"&gt;p95_latency_ms&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5912&lt;/span&gt;
&lt;span class="na"&gt;peak_memory_mb&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;14413&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Interpretation:&lt;/strong&gt; Best output quality, highest memory. 14 GB peak RSS means this model doesn't fit on machines with 8 GB RAM unless you close everything else. P95 at 5.9 seconds — slower than Llama 3.2 3B across the board, expected for a 7B model on CPU.&lt;/p&gt;




&lt;h2&gt;
  
  
  The JSON Validation Layer
&lt;/h2&gt;

&lt;p&gt;One of the project's core features: send a JSON schema with your query, get validated structured output back.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;POST&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;List 3 programming languages&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json_schema&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;object&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;properties&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;languages&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;array&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;items&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;string&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without retry: 68% of responses matched the schema on the first attempt.&lt;/p&gt;

&lt;p&gt;With retry + error injection:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;retry_prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;original_prompt&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Your previous response was invalid JSON. &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Error: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;validation_error&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;. &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Please respond with valid JSON matching this schema: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;schema&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With retry: &lt;strong&gt;94% success rate&lt;/strong&gt; across all three models.&lt;/p&gt;

&lt;p&gt;The error injection is what matters. Telling the model &lt;em&gt;exactly what went wrong&lt;/em&gt; is significantly more effective than "try again."&lt;/p&gt;




&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. P95 is the production number, not the average.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Average latency for Llama 3.2 3B is ~1.4 seconds. P95 is 3.8 seconds. If you set a 3-second SLA based on the average, you'll miss it 5% of the time. That's 1 in 20 users seeing a timeout. Measure the distribution, not the center.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Phi-3 Mini's CPU performance is misleading from the model card.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The model card advertises strong benchmark scores. Those scores are measured on GPU. On CPU-only inference, 4.7 tok/s makes it unusable for interactive applications. Always benchmark on your actual hardware.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Memory delta tells you more than peak.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Peak RSS includes OS overhead and Ollama itself. The delta between pre-inference and post-inference memory tells you how much the model's KV cache is actually growing per request. For Llama 3.2 3B, this delta was ~111 MB — relatively stable across prompt types.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Q4_K_M is the right default.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Ollama uses Q4_K_M by default. It's 4-bit quantization with K-means clustering, which recovers some quality compared to naive Q4_0. For factual and code tasks, quality degradation from FP16 to Q4_K_M is minimal. For complex reasoning tasks, there's a measurable drop — but at 4x the memory, FP16 isn't practical on consumer hardware anyway.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Sequential benchmarking is the only accurate method.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I tried parallelizing the benchmark for speed. The memory numbers became meaningless — Ollama's memory usage overlapped across concurrent requests and couldn't be attributed per-prompt. Sequential is slower but gives clean, attributable measurements.&lt;/p&gt;




&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;No GPU measurements.&lt;/strong&gt; All results are CPU-only. Phi-3 Mini's poor CPU performance might reverse completely on GPU — it's designed for Apple Silicon and NVIDIA acceleration. If you have a GPU, run your own benchmark.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Single hardware configuration.&lt;/strong&gt; Results are from one machine. RAM speed, CPU generation, and available cores all affect inference speed. These numbers are directional, not universal.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quality scoring is manual.&lt;/strong&gt; The benchmark measures latency and throughput automatically. Output quality is subjective and not automated here — it requires a golden dataset and an LLM judge (a separate project).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;30 prompts is not statistically robust.&lt;/strong&gt; P99 from 30 samples is noisy. A production benchmark should run 200+ prompts to get stable percentile estimates.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub: [&lt;a href="https://github.com/ThinkWithOps/02-local-ai-assistant" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/02-local-ai-assistant&lt;/a&gt;]&lt;br&gt;
Youtube : [&lt;a href="https://youtu.be/SMI-eIn-tuw" rel="noopener noreferrer"&gt;https://youtu.be/SMI-eIn-tuw&lt;/a&gt;]&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/02-local-ai-assistant.git
&lt;span class="nb"&gt;cd &lt;/span&gt;02-local-ai-assistant
bash scripts/install_models.sh  &lt;span class="c"&gt;# pulls llama3.2:3b, phi3:mini, mistral:7b&lt;/span&gt;
pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt
uvicorn src.main:app &lt;span class="nt"&gt;--host&lt;/span&gt; 0.0.0.0 &lt;span class="nt"&gt;--port&lt;/span&gt; 8000

&lt;span class="c"&gt;# Benchmark llama3.2:3b&lt;/span&gt;
python cli/main.py benchmark &lt;span class="nt"&gt;--model&lt;/span&gt; llama3.2:3b

&lt;span class="c"&gt;# Compare all 3 models&lt;/span&gt;
python cli/main.py compare
&lt;span class="c"&gt;# Generates: reports/model_comparison_YYYYMMDD.md&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;Which local model are you running, and what's your P95 latency? Drop it in the comments.&lt;/p&gt;




</description>
      <category>ai</category>
      <category>python</category>
      <category>machinelearning</category>
      <category>ollama</category>
    </item>
    <item>
      <title># I Built a RAG System That Enforces Its Own Citations — And Blocks Its Own Merges</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Thu, 14 May 2026 23:02:16 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/-i-built-a-rag-system-that-enforces-its-own-citations-and-blocks-its-own-merges-16ad</link>
      <guid>https://dev.to/vijaya_bollu/-i-built-a-rag-system-that-enforces-its-own-citations-and-blocks-its-own-merges-16ad</guid>
      <description>&lt;h2&gt;
  
  
  The Problem With Most RAG Tutorials
&lt;/h2&gt;

&lt;p&gt;Every RAG tutorial ends the same way.&lt;/p&gt;

&lt;p&gt;You send a question, the LLM returns an answer, and you ship it. What the tutorial doesn't show you: that answer might be confidently fabricated. The LLM might be citing a source it invented. There's no way to know.&lt;/p&gt;

&lt;p&gt;I spent three days debugging exactly this in a prototype. The system sounded authoritative. It was hallucinating chunk references that didn't exist in the retrieved context.&lt;/p&gt;

&lt;p&gt;The fix wasn't a better prompt. The fix was building actual enforcement into the pipeline — and then automating quality measurement so metric regressions literally cannot ship.&lt;/p&gt;

&lt;p&gt;This is a production-grade RAG system that answers questions from documents with verifiable citations, hybrid search, cross-encoder re-ranking, and CI/CD quality gates.&lt;/p&gt;




&lt;h2&gt;
  
  
  Architecture
&lt;/h2&gt;

&lt;p&gt;The request flow looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight http"&gt;&lt;code&gt;&lt;span class="err"&gt;POST /query
  → HybridRetriever.search()   # BM25 + vector via RRF
  → rerank()                   # Cohere cross-encoder, top 5 from 20
  → generate_answer()          # gpt-4o-mini + citation validation
  → QueryResponse              # cited answer or refusal
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three layers, each with a measurable reason to exist.&lt;/p&gt;




&lt;h2&gt;
  
  
  Layer 1: Hybrid Retrieval (BM25 + Vector)
&lt;/h2&gt;

&lt;p&gt;Pure vector search fails on exact terminology. If a document says "CO2 emissions" and the user asks "carbon dioxide output," a cosine similarity search might miss it. BM25 catches it because it matches exact tokens.&lt;/p&gt;

&lt;p&gt;Reciprocal Rank Fusion (RRF) merges the two ranked lists:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;doc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;rank_vector&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;rank_bm25&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With &lt;code&gt;k=60&lt;/code&gt;, this gives stable fusion without needing to weight or normalize the scores. BM25 index rebuilds in-memory on each query from ChromaDB — always reflects current state, no sync required.&lt;/p&gt;

&lt;p&gt;The tradeoff: rebuild latency. For large corpora this matters. For most document Q&amp;amp;A workloads, it's negligible.&lt;/p&gt;




&lt;h2&gt;
  
  
  Layer 2: Cross-Encoder Re-Ranking
&lt;/h2&gt;

&lt;p&gt;The retriever returns 20 candidates. The re-ranker returns the top 5.&lt;/p&gt;

&lt;p&gt;Bi-encoders (used in vector search) embed query and document independently — fast but imprecise. Cross-encoders (Cohere &lt;code&gt;rerank-english-v3.0&lt;/code&gt;) see the query and document together — slower, but significantly more accurate on relevance.&lt;/p&gt;

&lt;p&gt;The pattern: retrieve broadly (20), re-rank precisely (top 5). You get the recall of broad retrieval with the precision of cross-encoding.&lt;/p&gt;




&lt;h2&gt;
  
  
  Layer 3: Citation Enforcement
&lt;/h2&gt;

&lt;p&gt;Every chunk stored in ChromaDB gets a unique ID: &lt;code&gt;chunk-&lt;/code&gt; followed by 8 hex characters. The LLM is instructed to cite these IDs inline:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Global warming is primarily driven by greenhouse gas emissions [chunk-1a2b3c4d].
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After generation, a regex extracts every cited ID from the answer. Each one is checked against the set of IDs that were actually passed into the prompt. If any cited ID doesn't exist in that set — hallucinated reference — the answer is replaced with a refusal.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;cited_ids&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;findall&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;\[chunk-([0-9a-f]{8})\]&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;answer&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;hallucinated&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cited_ids&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;valid_chunk_ids&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;hallucinated&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;REFUSAL_RESPONSE&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This doesn't prevent the LLM from being wrong about the content of real chunks. But it prevents citation fabrication — a distinct and common failure mode.&lt;/p&gt;




&lt;h2&gt;
  
  
  The CI/CD Quality Gate
&lt;/h2&gt;

&lt;p&gt;This is the part most RAG tutorials skip entirely.&lt;/p&gt;

&lt;p&gt;Ragas measures two metrics:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Faithfulness&lt;/strong&gt;: Is the answer supported by the retrieved context?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context precision@5&lt;/strong&gt;: Are the retrieved chunks actually relevant to the question?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I maintain a golden dataset of 20 hand-verified Q&amp;amp;A pairs. On every PR to &lt;code&gt;main&lt;/code&gt;, GitHub Actions:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Starts ChromaDB&lt;/li&gt;
&lt;li&gt;Ingests demo documents&lt;/li&gt;
&lt;li&gt;Runs all 20 questions through the full pipeline&lt;/li&gt;
&lt;li&gt;Scores faithfulness and context precision@5 via Ragas&lt;/li&gt;
&lt;li&gt;Fails the workflow if &lt;code&gt;faithfulness &amp;lt; 0.85&lt;/code&gt; or &lt;code&gt;context_precision@5 &amp;lt; 0.70&lt;/code&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Metric regressions cannot merge. You find out in CI before any code ships, not after a deploy.&lt;/p&gt;

&lt;p&gt;The check script is a standalone Python file (&lt;code&gt;scripts/check_quality_gate.py&lt;/code&gt;) that exits 1 if thresholds aren't met — easy to wire into any CI system.&lt;/p&gt;




&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. Chunk size is not arbitrary.&lt;/strong&gt;&lt;br&gt;
I tested 500, 700, and 1000 characters. At 500, long paragraphs split mid-sentence and the re-ranker couldn't reconstruct context. At 1000, chunks were too long for the LLM to synthesize cleanly. 700 with 100 overlap hit the right balance for the climate domain documents I was using. This number will be different for your corpus — test it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. The golden dataset quality matters more than its size.&lt;/strong&gt;&lt;br&gt;
My first golden dataset had subjective Q&amp;amp;A pairs — "What is an important source of emissions?" Any answer could be justified. I rebuilt it with binary-verifiable claims: exact figures, named entities, specific relationships. Ragas faithfulness scoring only means something if the ground truth is unambiguous.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Citation format is load-bearing.&lt;/strong&gt;&lt;br&gt;
The LLM initially produced &lt;code&gt;(chunk-042)&lt;/code&gt; and &lt;code&gt;chunk_1a2b3c4d&lt;/code&gt; — close but not matching the regex. The fix was putting the exact format string in the system prompt with an explicit example, not just a description. Format specification in prompts must be concrete.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. BM25 re-index latency is real.&lt;/strong&gt;&lt;br&gt;
Rebuilding the BM25 index on every query adds latency proportional to corpus size. For 500 chunks it's ~5ms. For 50,000 chunks it becomes a problem. The current design is correct for a portfolio-scale corpus; at production scale you'd maintain a persistent BM25 index and update it incrementally on ingest.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Prompt versioning changes how you iterate.&lt;/strong&gt;&lt;br&gt;
Moving prompts to &lt;code&gt;prompts/rag_prompts.yaml&lt;/code&gt; with a &lt;code&gt;version&lt;/code&gt; field meant I could iterate on prompt content without touching Python code and track what changed in git diffs. It also let me hot-reload prompts at startup without redeploying. Small architectural decision, large practical impact.&lt;/p&gt;


&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Synchronous BM25 rebuild.&lt;/strong&gt; Rebuilds from ChromaDB on every query. Fast for small corpora, problematic at scale. A persistent index with delta updates would fix this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Single collection.&lt;/strong&gt; All documents share one ChromaDB collection (&lt;code&gt;rag_documents&lt;/code&gt;). There's no namespace isolation between document sets. If you ingest documents for two different topics, retrieval can bleed across domains.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Golden dataset is climate-domain only.&lt;/strong&gt; The evaluation is tuned for the demo documents. Ragas metrics are meaningful only when the golden dataset matches your actual document domain.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No streaming.&lt;/strong&gt; &lt;code&gt;POST /query&lt;/code&gt; waits for the full answer before returning. For long answers, this adds perceived latency. FastAPI supports streaming responses via &lt;code&gt;StreamingResponse&lt;/code&gt; — not implemented here.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Citation enforcement catches fabricated IDs, not wrong facts.&lt;/strong&gt; If the LLM correctly cites a real chunk but misrepresents what it says, that passes citation enforcement. Ragas faithfulness catches this, but at evaluation time, not at runtime.&lt;/p&gt;


&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub: [&lt;a href="https://github.com/ThinkWithOps/01-rag-from-scratch" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/01-rag-from-scratch&lt;/a&gt;]&lt;br&gt;
Demo: [&lt;a href="https://youtu.be/wRZpmzIexnQ" rel="noopener noreferrer"&gt;https://youtu.be/wRZpmzIexnQ&lt;/a&gt;]&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/01-rag-from-scratch.git
&lt;span class="nb"&gt;cd &lt;/span&gt;01-rag-from-scratch
&lt;span class="nb"&gt;cp&lt;/span&gt; .env.example .env
&lt;span class="c"&gt;# Add OPENAI_API_KEY and COHERE_API_KEY&lt;/span&gt;

docker compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
bash scripts/ingest_demo_docs.sh

curl &lt;span class="nt"&gt;-X&lt;/span&gt; POST http://localhost:8000/query &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"question": "What is the main cause of climate change?", "top_k": 5}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run the full evaluation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;bash scripts/run_evaluation.sh
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;What's your quality bar for RAG before you'd ship it to users? Drop it in the comments.&lt;/p&gt;




</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>python</category>
      <category>devops</category>
    </item>
    <item>
      <title>I Built a Production Food Delivery Platform on AWS EKS — Here's Everything I Learned</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Thu, 30 Apr 2026 14:25:28 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/i-built-a-production-food-delivery-platform-on-aws-eks-heres-everything-i-learned-2cnf</link>
      <guid>https://dev.to/vijaya_bollu/i-built-a-production-food-delivery-platform-on-aws-eks-heres-everything-i-learned-2cnf</guid>
      <description>&lt;h2&gt;
  
  
  Why I Built This
&lt;/h2&gt;

&lt;p&gt;Most Kubernetes tutorials stop at &lt;code&gt;kubectl apply -f deployment.yaml&lt;/code&gt;. They don't show you how a VPC is laid out, why you need two availability zones, what IAM roles EKS nodes actually need, or how to debug a live failure using Prometheus metrics.&lt;/p&gt;

&lt;p&gt;I wanted to build something that forced me to make every decision a senior DevOps engineer would make on a real project. So I built a food delivery platform — four independent microservices, a React frontend, full Terraform infrastructure on AWS, a GitHub Actions pipeline, and a Grafana dashboard — and recorded the whole thing.&lt;/p&gt;

&lt;p&gt;This is what I learned.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;h3&gt;
  
  
  The Application Layer
&lt;/h3&gt;

&lt;p&gt;Four FastAPI microservices, each completely independent with its own SQLite database:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;user-service&lt;/strong&gt; (port 8001): Registration, JWT login, user profiles. Seeds 3 users on startup.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;restaurant-service&lt;/strong&gt; (port 8002): Restaurant listing + full menus. Seeds 5 restaurants with 10 menu items each — real food names, USD prices.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;order-service&lt;/strong&gt; (port 8003): Order placement. Makes a synchronous HTTP call to restaurant-service to validate menu items before placing the order. Has a built-in &lt;code&gt;ORDER_SERVICE_FAILURE_MODE&lt;/code&gt; env var for the observability demo.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;delivery-service&lt;/strong&gt; (port 8004): Agent assignment and delivery tracking. Seeds 5 delivery agents.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each service exposes &lt;code&gt;/health&lt;/code&gt; (returns &lt;code&gt;{"status":"healthy","service":"&amp;lt;name&amp;gt;","version":"1.0.0"}&lt;/code&gt;) and &lt;code&gt;/metrics&lt;/code&gt; (auto-generated by &lt;code&gt;prometheus-fastapi-instrumentator&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;An NGINX gateway (port 8080 locally) routes &lt;code&gt;/api/users&lt;/code&gt;, &lt;code&gt;/api/restaurants&lt;/code&gt;, &lt;code&gt;/api/orders&lt;/code&gt;, &lt;code&gt;/api/delivery&lt;/code&gt; to the right service and serves the React frontend at &lt;code&gt;/&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Infrastructure
&lt;/h3&gt;

&lt;p&gt;Terraform is split into four modules:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;modules/vpc&lt;/strong&gt;: VPC (10.0.0.0/16), 2 public + 2 private subnets across us-east-1a and us-east-1b, Internet Gateway, 1 NAT Gateway (single point of failure — intentional cost trade-off for a demo, documented in comments), route tables.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;modules/eks&lt;/strong&gt;: EKS 1.32 cluster, managed node group with t3.small instances (min=1, desired=2, max=4 in private subnets), cluster IAM role, node IAM role with three AWS-managed policies, launch template to name EC2 instances in the console.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;modules/ecr&lt;/strong&gt;: Five repositories (&lt;code&gt;food-delivery/user-service&lt;/code&gt;, &lt;code&gt;food-delivery/frontend&lt;/code&gt;, etc.), image scan on push, lifecycle policy keeping last 10 images.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;modules/iam&lt;/strong&gt;: GitHub Actions IAM user with an inline policy scoped to ECR push/pull and EKS describe — nothing else.&lt;/p&gt;

&lt;h3&gt;
  
  
  The CI/CD Pipeline
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;deploy.yml&lt;/code&gt; triggers on push to main. It:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Applies Kubernetes manifests, ingress-nginx, and kube-prometheus-stack&lt;/li&gt;
&lt;li&gt;Uses a matrix job for user-service, restaurant-service, order-service, delivery-service, and frontend&lt;/li&gt;
&lt;li&gt;Logs into ECR&lt;/li&gt;
&lt;li&gt;Builds and tags each image with &lt;code&gt;$GITHUB_SHA&lt;/code&gt; and &lt;code&gt;latest&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Runs &lt;code&gt;aws eks update-kubeconfig&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Does &lt;code&gt;kubectl set image&lt;/code&gt; with the SHA tag&lt;/li&gt;
&lt;li&gt;Waits for &lt;code&gt;kubectl rollout status&lt;/code&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;code&gt;pr-checks.yml&lt;/code&gt; runs flake8, pytest, &lt;code&gt;terraform fmt -check&lt;/code&gt;, and &lt;code&gt;terraform validate&lt;/code&gt; on every pull request.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;destroy.yml&lt;/code&gt; is a manual workflow_dispatch with a typed confirmation — safeguard against accidental &lt;code&gt;terraform destroy&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Observability Demo
&lt;/h2&gt;

&lt;p&gt;This is the part that makes the project worth recording.&lt;/p&gt;

&lt;p&gt;Set &lt;code&gt;ORDER_SERVICE_FAILURE_MODE=true&lt;/code&gt; in Docker Compose and restart order-service. Now 50% of &lt;code&gt;POST /orders&lt;/code&gt; requests return HTTP 500. Run &lt;code&gt;scripts/load-test.sh&lt;/code&gt; — it fires 300 requests in 10 concurrent workers over 3 minutes.&lt;/p&gt;

&lt;p&gt;In Grafana, the "Error rate per service" panel spikes immediately from 0% to ~50% for order-service. The &lt;code&gt;failed_orders_total&lt;/code&gt; counter climbs. P95 latency creeps up because failed requests still go through the restaurant-service validation call before failing.&lt;/p&gt;

&lt;p&gt;Meanwhile HPA detects elevated CPU, scales replicas from 2 to 6. More pods, same error rate — the bug is in code, not capacity.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;kubectl logs&lt;/code&gt; on any order-service pod shows the failure mode immediately. Fix: set &lt;code&gt;ORDER_SERVICE_FAILURE_MODE=false&lt;/code&gt;, redeploy. Grafana recovers in under 30 seconds.&lt;/p&gt;

&lt;p&gt;That recovery graph — the spike, the plateau, the drop — is the money shot of the video.&lt;/p&gt;




&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. EKS nodes don't get Name tags by default.&lt;/strong&gt;&lt;br&gt;
The &lt;code&gt;aws_eks_node_group&lt;/code&gt; resource tags the node group, not the individual EC2 instances. You need a &lt;code&gt;launch_template&lt;/code&gt; with &lt;code&gt;tag_specifications { resource_type = "instance" }&lt;/code&gt; to see names in the EC2 console. Lost 20 minutes on this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. One NAT Gateway is a trade-off, not a mistake.&lt;/strong&gt;&lt;br&gt;
The prompt called for cost saving. A single NAT Gateway means if us-east-1a goes down, private subnets in us-east-1b lose internet access. I documented this in a comment on the resource. Production would use one NAT per AZ. That trade-off is worth explaining explicitly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. The IAM roles for EKS are the biggest footgun.&lt;/strong&gt;&lt;br&gt;
You need three separate IAM roles: cluster role (for the control plane), node role (for EC2 instances in the node group), and optionally a IRSA role per service. Mixing them up silently breaks things. The &lt;code&gt;AmazonEKS_CNI_Policy&lt;/code&gt; on the node role is what makes pod networking work — missing it gives you running pods with no network connectivity.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. &lt;code&gt;prometheus-fastapi-instrumentator&lt;/code&gt; is one line of code.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nc"&gt;Instrumentator&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;instrument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;expose&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's it. You get request count, latency histograms, and HTTP status breakdown per endpoint, all at &lt;code&gt;/metrics&lt;/code&gt;. The custom counters (&lt;code&gt;orders_total&lt;/code&gt;, &lt;code&gt;failed_orders_total&lt;/code&gt;, &lt;code&gt;order_processing_seconds&lt;/code&gt;) are 5 more lines.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Service-to-service calls need explicit timeouts.&lt;/strong&gt;&lt;br&gt;
order-service calls restaurant-service with &lt;code&gt;httpx.AsyncClient(timeout=5.0)&lt;/code&gt;. Without the timeout, a slow restaurant-service will hold an order-service worker indefinitely, causing cascade failures that look like order-service bugs in the logs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;6. &lt;code&gt;maxUnavailable=0&lt;/code&gt; in rolling updates protects you more than you think.&lt;/strong&gt;&lt;br&gt;
With &lt;code&gt;maxSurge=1, maxUnavailable=0&lt;/code&gt;, Kubernetes brings up the new pod and passes readiness checks before terminating the old one. The &lt;code&gt;/health&lt;/code&gt; readinessProbe with &lt;code&gt;initialDelaySeconds=15&lt;/code&gt; means the new pod gets 15 seconds to initialize SQLite and seed data before traffic hits it. Without this, users hit 503s during every deploy.&lt;/p&gt;




&lt;h2&gt;
  
  
  Limitations (honest)
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;SQLite is fine for local dev and demos. This would use RDS or Aurora in production.&lt;/li&gt;
&lt;li&gt;Single NAT Gateway is a cost optimization, not production-ready.&lt;/li&gt;
&lt;li&gt;The React frontend hardcodes &lt;code&gt;http://localhost:8080&lt;/code&gt; — a real app would use environment injection at build time.&lt;/li&gt;
&lt;li&gt;No secrets management — passwords and JWT secret are env vars. Production would use AWS Secrets Manager + Kubernetes Secrets.&lt;/li&gt;
&lt;li&gt;The GitHub Actions IAM user uses long-lived access keys. Production would use OIDC federation (no keys at all).&lt;/li&gt;
&lt;li&gt;The Grafana dashboard started as a local Docker Compose dashboard. Kubernetes metrics need their own PromQL queries and dashboard panels.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Local — everything runs in Docker&lt;/span&gt;
git clone https://github.com/vijayb-aiops/devops-production-projects
&lt;span class="nb"&gt;cd &lt;/span&gt;devops-production-projects/projects/01-food-delivery-eks-platform
bash scripts/bootstrap.sh

&lt;span class="c"&gt;# Trigger the observability demo&lt;/span&gt;
&lt;span class="nv"&gt;ORDER_SERVICE_FAILURE_MODE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;true &lt;/span&gt;docker compose up &lt;span class="nt"&gt;-d&lt;/span&gt; order-service
bash scripts/load-test.sh
&lt;span class="c"&gt;# Open Grafana at http://localhost:3000 (admin/foodrush123)&lt;/span&gt;

&lt;span class="c"&gt;# Deploy to AWS&lt;/span&gt;
&lt;span class="nb"&gt;cd &lt;/span&gt;infra/terraform
terraform init
terraform apply
&lt;span class="nb"&gt;cd&lt;/span&gt; ../..
bash scripts/deploy-eks.sh
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Estimated AWS cost while recording: ~$0.19/hr. Run &lt;code&gt;terraform destroy&lt;/code&gt; when done.&lt;/p&gt;

&lt;p&gt;📺 Full build-along: &lt;a href="https://www.youtube.com/watch?v=HDiWR1uVI9s" rel="noopener noreferrer"&gt;https://www.youtube.com/watch?v=HDiWR1uVI9s&lt;/a&gt;&lt;br&gt;
📁 GitHub: &lt;a href="https://github.com/ThinkWithOps/devops-production-projects/tree/main/projects/01-food-delivery-eks-platform" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/devops-production-projects/tree/main/projects/01-food-delivery-eks-platform&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>kubernetes</category>
      <category>aws</category>
      <category>terraform</category>
    </item>
    <item>
      <title>I Built an MCP Server That Lets Claude Control My Kubernetes Cluster</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Thu, 09 Apr 2026 13:52:04 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/i-built-an-mcp-server-that-lets-claude-control-my-kubernetes-cluster-mfp</link>
      <guid>https://dev.to/vijaya_bollu/i-built-an-mcp-server-that-lets-claude-control-my-kubernetes-cluster-mfp</guid>
      <description>&lt;h2&gt;
  
  
  Why I Built This
&lt;/h2&gt;

&lt;p&gt;Every DevOps incident I've dealt with follows the same pattern.&lt;/p&gt;

&lt;p&gt;Something breaks. I open five terminal tabs. I run kubectl, check the AWS console, tail Docker logs, and manually piece together what went wrong — all while the clock is ticking.&lt;/p&gt;

&lt;p&gt;The tools all exist. They're just completely disconnected from each other.&lt;/p&gt;

&lt;p&gt;I wanted one interface that could ask a question, reach into all of them simultaneously, and explain what it found. So I built an MCP server that gives Claude Desktop direct access to my real infrastructure.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;The server runs locally on my laptop. Claude Desktop launches it on startup and communicates over stdio (standard input/output).&lt;/p&gt;

&lt;p&gt;When I type a question, Claude reads the 14 registered tool descriptions, decides which ones to call, and sends a JSON request to my server:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"method"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"call_tool"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"params"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"get_failing_pods"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"arguments"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"namespace"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"default"&lt;/span&gt;&lt;span class="p"&gt;}}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;My server executes against real infrastructure and returns structured text. Claude synthesises everything into a plain English response.&lt;/p&gt;

&lt;p&gt;The key thing: &lt;strong&gt;Claude never connects to AWS or Kubernetes directly.&lt;/strong&gt; It only talks to my Python server. My server uses local credentials (&lt;code&gt;~/.aws/credentials&lt;/code&gt;, &lt;code&gt;~/.kube/config&lt;/code&gt;) to reach the real systems.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;14 tools across 4 categories:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Kubernetes:&lt;/strong&gt; pod status, failing pods, logs, restart deployment, describe pod&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AWS:&lt;/strong&gt; cost report, EC2 instances, CloudWatch alarms, S3 buckets&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Docker:&lt;/strong&gt; list containers, container logs, restart container&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Terraform:&lt;/strong&gt; run plan, check state&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Demo / Results
&lt;/h2&gt;

&lt;p&gt;I typed: &lt;em&gt;"Give me a health report of my infrastructure"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Claude called &lt;code&gt;get_pod_status&lt;/code&gt;, &lt;code&gt;get_aws_cost&lt;/code&gt;, &lt;code&gt;get_cloudwatch_alarms&lt;/code&gt;, and &lt;code&gt;list_containers&lt;/code&gt; simultaneously and returned a unified summary — Kubernetes pods, AWS costs, Docker containers, CloudWatch alarms — all in one response.&lt;/p&gt;

&lt;p&gt;One pod was in CrashLoopBackOff with 14 restarts. I typed:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;"Pull the logs from broken-app and tell me why it's crashing"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Claude diagnosed it instantly — busybox with no long-running process, exits immediately every time, Kubernetes keeps restarting it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Before:&lt;/strong&gt; 45 minutes of manual investigation across 5 tabs.&lt;br&gt;
&lt;strong&gt;After:&lt;/strong&gt; 10 seconds, one chat window.&lt;/p&gt;


&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. The tool description is everything.&lt;/strong&gt;&lt;br&gt;
Claude picks tools based on their description, not their name. A vague description causes Claude to call the wrong tool or skip it entirely. My first version had &lt;code&gt;get_failing_pods&lt;/code&gt; described as &lt;em&gt;"Get pods"&lt;/em&gt;. Claude never called it. I changed it to &lt;em&gt;"List only pods in Failed, CrashLoopBackOff, OOMKilled, or Error state"&lt;/em&gt; and it worked perfectly every time. The description is the API contract between you and Claude.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Parallel tool calls are not guaranteed — they're emergent.&lt;/strong&gt;&lt;br&gt;
I didn't write any code to call tools in parallel. Claude decided to do that on its own when the question was broad enough ("health report"). For narrow questions ("which pods are failing?") Claude called only one tool. The parallelism comes from Claude's reasoning, not from your server. This means your tool descriptions need to be distinct enough that Claude can make that judgment call correctly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Sync SDKs in an async server is the silent performance killer.&lt;/strong&gt;&lt;br&gt;
boto3, the Kubernetes Python client, and the Docker SDK are all blocking. If you call them directly inside an async MCP handler, you stall the entire event loop. Every tool call wraps the SDK call in &lt;code&gt;asyncio.to_thread()&lt;/code&gt;. Without this, Claude would time out waiting for responses when multiple tools were called simultaneously.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Error handling scope matters more than you think.&lt;/strong&gt;&lt;br&gt;
My first version caught SDK import errors at module level. If boto3 failed to connect to AWS at startup, the entire server crashed — Kubernetes and Docker tools went down too. The fix: catch errors inside each tool call, not at import time. One broken service should never take down the whole server.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. The MCP protocol is simpler than it looks.&lt;/strong&gt;&lt;br&gt;
There are really only two things you implement: &lt;code&gt;list_tools()&lt;/code&gt; (return tool names + descriptions + input schema) and &lt;code&gt;call_tool()&lt;/code&gt; (receive tool name + arguments, return text). Everything else is handled by the MCP SDK. Building a tool is 20 lines of Python.&lt;/p&gt;


&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Single EC2 instance.&lt;/strong&gt; The demo infrastructure runs on one t3.small. In a real multi-node Kubernetes cluster, the pod information would be spread across namespaces and nodes. The tools work against any kubeconfig — but the demo is scoped to what's in one cluster.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Read-heavy, not write-heavy.&lt;/strong&gt; Most tools read state — pod status, AWS costs, Docker containers. The only write operation is &lt;code&gt;restart_deployment&lt;/code&gt;. There's no tool to edit a deployment YAML, push a config change, or create resources. Giving Claude write access to production infrastructure is a separate (and more serious) conversation about guardrails.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Claude restarts but can't fix root causes.&lt;/strong&gt; If a pod is in CrashLoopBackOff because of a bad container image, Claude can restart it — but it'll crash again immediately. Claude diagnosed the issue correctly and warned me, but the actual fix (updating the deployment YAML and pushing a correct image) is outside the scope of the tools I gave it. The tools match what I was comfortable giving an AI access to.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No authentication on the MCP server.&lt;/strong&gt; The server runs locally over stdio — there's no network exposure. But if you ever expose this over a network (e.g., run the server on EC2 and connect Claude Desktop remotely), you'd need to add authentication. Don't do that without thinking it through.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AWS credentials are local.&lt;/strong&gt; The server uses whatever credentials are in &lt;code&gt;~/.aws/credentials&lt;/code&gt;. If you're running this on a shared machine, those credentials are accessible to anything running as your user. Use IAM roles with minimal permissions scoped to read-only for Cost Explorer, EC2, CloudWatch, and S3.&lt;/p&gt;


&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;GitHub: [&lt;a href="https://github.com/ThinkWithOps/ai-devops-systems-lab.git" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/ai-devops-systems-lab.git&lt;/a&gt;]&lt;br&gt;
Demo: [&lt;a href="https://youtu.be/2XsNcKSa28s" rel="noopener noreferrer"&gt;https://youtu.be/2XsNcKSa28s&lt;/a&gt;]&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/ai-devops-systems-lab.git
&lt;span class="nb"&gt;cd &lt;/span&gt;03-ai-devops-mcp-server
pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt

&lt;span class="c"&gt;# Test with mock data — no real infra needed&lt;/span&gt;
&lt;span class="nv"&gt;KUBE_MOCK_MODE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;true &lt;/span&gt;python &lt;span class="nt"&gt;-m&lt;/span&gt; pytest tests/ &lt;span class="nt"&gt;-v&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add the MCP server to your Claude Desktop config and restart. Full setup guide in the README.&lt;/p&gt;

&lt;p&gt;This is Project 03 of the ThinkWithOps AI + DevOps series &lt;/p&gt;




&lt;p&gt;What would you control first if Claude had access to your infrastructure? Drop it in the comments.&lt;/p&gt;




</description>
      <category>devops</category>
      <category>kubernetes</category>
      <category>ai</category>
      <category>python</category>
    </item>
    <item>
      <title># I Built an AI That Understands Any GitHub Repo Using LangChain and ChromaDB</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Wed, 01 Apr 2026 16:54:53 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/-i-built-an-ai-that-understands-any-github-repo-using-langchain-and-chromadb-m30</link>
      <guid>https://dev.to/vijaya_bollu/-i-built-an-ai-that-understands-any-github-repo-using-langchain-and-chromadb-m30</guid>
      <description>&lt;h2&gt;
  
  
  Why I Built This
&lt;/h2&gt;

&lt;p&gt;Every time I join a new codebase, the first few days are the same: open the repo, stare at folders, try to figure out which service does what, read half a file, get interrupted, lose context, start over.&lt;/p&gt;

&lt;p&gt;GitHub's built-in search is keyword-only. ChatGPT has never seen your repo. Teammates are busy. Documentation is either missing or out of date.&lt;/p&gt;

&lt;p&gt;I wanted a tool that could answer &lt;em&gt;"how does checkout work?"&lt;/em&gt; from the actual code — not from training data, not from docs, but from the real source files.&lt;/p&gt;

&lt;p&gt;So I built one.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;The system is built around a RAG (Retrieval-Augmented Generation) pipeline. The idea: instead of asking an LLM to answer from memory, you first retrieve the most relevant code chunks, then ask the LLM to answer using only those chunks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ingest flow:&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Clone the GitHub repo locally&lt;/li&gt;
&lt;li&gt;Walk every file and split into overlapping chunks (~500 tokens, 50-token overlap)&lt;/li&gt;
&lt;li&gt;Convert each chunk to a vector embedding using &lt;code&gt;all-MiniLM-L6-v2&lt;/code&gt; (Sentence Transformers — local, free)&lt;/li&gt;
&lt;li&gt;Store embeddings + metadata in ChromaDB&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Query flow:&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Embed the user's question with the same model&lt;/li&gt;
&lt;li&gt;ChromaDB cosine similarity search → top-5 most relevant chunks&lt;/li&gt;
&lt;li&gt;Inject chunks into a LangChain prompt&lt;/li&gt;
&lt;li&gt;LLM generates an answer with source file citations&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The frontend is a Next.js split-pane layout — file tree on the left, chat on the right. The ingest endpoint streams progress via Server-Sent Events so the user sees live updates instead of a spinner.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# LangChain RAG chain (simplified)
&lt;/span&gt;&lt;span class="n"&gt;retriever&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;vectorstore&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;as_retriever&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;search_kwargs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;k&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;span class="n"&gt;qa_chain&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;RetrievalQA&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;from_chain_type&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;llm&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;llm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;chain_type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;stuff&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;retriever&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;retriever&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;return_source_documents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;qa_chain&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;query&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The LLM only ever sees 5 chunks — not the whole repo. That keeps answers fast and grounded.&lt;/p&gt;




&lt;h2&gt;
  
  
  Demo / Results
&lt;/h2&gt;

&lt;p&gt;I indexed the ShopFlow ecommerce demo app — ~847 files across 12 services.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt; "How does the checkout flow work?"&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;The checkout flow works as follows:
1. Cart validation — CartService checks inventory availability (src/services/cart.py:142)
2. Payment processing — PaymentService calls Stripe API with order total (src/services/payment.py:89)
3. Order creation — OrderService writes to PostgreSQL with status PENDING (src/models/order.py:67)
4. Inventory update — InventoryService decrements stock counts (src/services/inventory.py:203)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Index time: ~30 seconds. Answer latency: ~3 seconds. API cost: $0 (self-hosted on AWS EC2).&lt;/p&gt;




&lt;h2&gt;
  
  
  What Surprised Me
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. FastAPI path parameters break on slashes&lt;/strong&gt;&lt;br&gt;
Repo names like &lt;code&gt;owner/repo&lt;/code&gt; contain a &lt;code&gt;/&lt;/code&gt; — FastAPI treats it as a path separator and routes to the wrong endpoint. Fix: declare the parameter as &lt;code&gt;{repo_name:path}&lt;/code&gt;. One character. Found it via a 404 in production after deployment.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. TypeScript types drift silently from backend SSE events&lt;/strong&gt;&lt;br&gt;
The backend was emitting &lt;code&gt;file&lt;/code&gt;, &lt;code&gt;file_path&lt;/code&gt;, and &lt;code&gt;indexed_at&lt;/code&gt; fields in the SSE stream that the frontend TypeScript interface didn't declare. No error in local dev — only failed during &lt;code&gt;next build&lt;/code&gt; inside Docker. A shared OpenAPI-generated type contract would have caught this at development time.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Docker layer caching hides real code&lt;/strong&gt;&lt;br&gt;
After pushing a fix, the Docker build was still serving old code because it cached the &lt;code&gt;COPY . .&lt;/code&gt; layer. Always run &lt;code&gt;docker-compose build --no-cache&lt;/code&gt; when a fix isn't appearing after git pull.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Chunk overlap matters more than chunk size&lt;/strong&gt;&lt;br&gt;
I initially had no overlap between chunks. The AI would give incomplete answers for questions that spanned function boundaries — the relevant context was split across two chunks that were never returned together. Adding 50-token overlap fixed most of these cases.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/ai-devops-systems-lab
&lt;span class="nb"&gt;cd &lt;/span&gt;projects/02-ai-github-repo-explainer
&lt;span class="nb"&gt;cp &lt;/span&gt;backend/.env.example backend/.env
&lt;span class="c"&gt;# Add your GROQ_API_KEY or leave blank to use Ollama&lt;/span&gt;
docker-compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Open &lt;code&gt;http://&amp;lt;your-ec2-ip&amp;gt;:3000&lt;/code&gt;, paste any public GitHub URL, and start asking questions.&lt;/p&gt;

&lt;p&gt;🔗 GitHub: &lt;a href="https://github.com/ThinkWithOps/ai-devops-systems-lab" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/ai-devops-systems-lab&lt;/a&gt;&lt;br&gt;
📺 Full build walkthrough: &lt;a href="https://youtu.be/a6376K9Lm00" rel="noopener noreferrer"&gt;https://youtu.be/a6376K9Lm00&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is Project 02 of a 30-project AI + DevOps series. Each project is a real deployed system — not a tutorial snippet.&lt;/p&gt;




&lt;p&gt;What's the most confusing codebase you've ever had to onboard into? What would you have asked an AI first?&lt;/p&gt;

</description>
      <category>langchain</category>
      <category>chromadb</category>
      <category>devops</category>
      <category>python</category>
    </item>
    <item>
      <title># I Built a DevOps Chatbot That Checks My Live App for Failures — Here's How It Works</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Tue, 31 Mar 2026 16:50:21 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/-i-built-a-devops-chatbot-that-checks-my-live-app-for-failures-heres-how-it-works-h78</link>
      <guid>https://dev.to/vijaya_bollu/-i-built-a-devops-chatbot-that-checks-my-live-app-for-failures-heres-how-it-works-h78</guid>
      <description>&lt;h2&gt;
  
  
  Why I Built This
&lt;/h2&gt;

&lt;p&gt;Every DevOps engineer has had the 2am moment. Something is broken. You don't know what. You SSH in, check logs, Google the error, open five tabs, still nothing clear. Thirty minutes later you find it — a config flag someone changed, a slow query, a dependency timing out.&lt;/p&gt;

&lt;p&gt;I wanted to ask an AI instead. Not a generic ChatGPT that gives you textbook answers, but an AI connected to my actual running system that can check what's broken right now.&lt;/p&gt;

&lt;p&gt;So I built the AI DevOps Copilot — Project 01 of my 30-project AI + DevOps YouTube series.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;The system has four layers:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. LangChain agent (the brain)&lt;/strong&gt;&lt;br&gt;
Uses &lt;code&gt;create_tool_calling_agent&lt;/code&gt; with Llama 3.1 via Groq. When you ask a question, the agent decides whether to answer from knowledge or call a tool. General DevOps questions → instant answer. Questions about the live app → tool call.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. ChromaDB RAG (the knowledge base)&lt;/strong&gt;&lt;br&gt;
Nine runbook documents embedded into a vector database — Docker troubleshooting, AWS debugging, Kubernetes, Terraform, Linux performance, security, and more. The agent searches these for context when answering general questions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Tool layer (the live connection)&lt;/strong&gt;&lt;br&gt;
Four tools: &lt;code&gt;restaurant_monitor&lt;/code&gt; (hits the live restaurant app API), &lt;code&gt;log_search&lt;/code&gt; (searches application logs), &lt;code&gt;github_search&lt;/code&gt; (searches repos), and &lt;code&gt;devops_docs&lt;/code&gt; (searches the runbook vector store).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. FastAPI + SSE streaming (the interface)&lt;/strong&gt;&lt;br&gt;
The agent runs in a thread executor, streaming tokens back through an asyncio Queue with a sentinel pattern. The Next.js frontend connects via Server-Sent Events and renders each token as it arrives.&lt;/p&gt;

&lt;p&gt;Data flow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User question → FastAPI → LangChain agent → tool call or RAG search → streamed response → Next.js UI
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Demo / Results
&lt;/h2&gt;

&lt;p&gt;I deployed a chaos engineering demo app — a restaurant application with injectable failure modes. One failure mode: &lt;code&gt;slow_menu&lt;/code&gt;, which adds a 2-second artificial delay to &lt;code&gt;GET /api/menu&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt; Menu page spinning, customers waiting. I ask the copilot: "What is wrong with the restaurant app right now?"&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What happened:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Tool called: restaurant_monitor → action: failures
API response: slow_menu: ACTIVE (2s delay on /api/menu)

AI answer:
The slow_menu failure mode is currently ACTIVE.
This injects a 2-second delay into the Menu API.

Fix:
1. Operator dashboard → Failures → Disable slow_menu
2. Or: POST /api/admin/failures/slow_menu/disable
3. Verify: menu should load in &amp;lt;100ms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Time to diagnosis: ~8 seconds.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After disabling the toggle, the menu loaded instantly. The entire incident lifecycle — detection, diagnosis, fix, verification — took under 2 minutes.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Surprised Me
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. &lt;code&gt;create_react_agent&lt;/code&gt; kept hitting iteration limits.&lt;/strong&gt;&lt;br&gt;
I started with the ReAct text-parsing agent. Llama 3.1-8b kept failing the "Thought/Action/Observation" format, exhausting 10 iterations on simple questions. Switching to &lt;code&gt;create_tool_calling_agent&lt;/code&gt; — which uses native LLM function calling — fixed this completely. The model knows how to call functions; it doesn't know how to produce exact ReAct formatting.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Tool call chunks were streaming to the user as garbage text.&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;on_llm_new_token&lt;/code&gt; in LangChain fires for every token — including internal tool call encodings like &lt;code&gt;{"name": "restaurant_monitor", "arguments":&lt;/code&gt;. Fixed by checking &lt;code&gt;chunk.tool_call_chunks&lt;/code&gt; and skipping those tokens. Without this, users see raw JSON blobs in the chat.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. &lt;code&gt;host.docker.internal&lt;/code&gt; doesn't resolve on EC2 Linux.&lt;/strong&gt;&lt;br&gt;
Added &lt;code&gt;extra_hosts: - "host.docker.internal:host-gateway"&lt;/code&gt; to Docker Compose — still failed. Fixed by using the EC2 private IP directly: &lt;code&gt;RESTAURANT_API_URL=http://172.31.90.69:8010&lt;/code&gt; in the &lt;code&gt;.env&lt;/code&gt; file. Simple, obvious in hindsight.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Groq free tier is 6,000 tokens per minute.&lt;/strong&gt;&lt;br&gt;
With &lt;code&gt;llama-3.3-70b-versatile&lt;/code&gt;, three questions hit the limit. Switched to &lt;code&gt;llama-3.1-8b-instant&lt;/code&gt; — much faster, lower token usage, still very capable for DevOps Q&amp;amp;A. For a demo or portfolio project, this is the right call.&lt;/p&gt;


&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;📁 GitHub: &lt;a href="https://github.com/ThinkWithOps/ai-devops-systems-lab/tree/main/projects/01-ai-devops-copilot" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/ai-devops-systems-lab/tree/main/projects/01-ai-devops-copilot&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;🎬 Video walkthrough: &lt;a href="https://youtu.be/a50334Szt5g" rel="noopener noreferrer"&gt;https://youtu.be/a50334Szt5g&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Clone and run locally&lt;/span&gt;
git clone https://github.com/ThinkWithOps/ai-devops-systems-lab
&lt;span class="nb"&gt;cd &lt;/span&gt;projects/01-ai-devops-copilot
&lt;span class="nb"&gt;cp &lt;/span&gt;backend/.env.example backend/.env
&lt;span class="c"&gt;# Add your GROQ_API_KEY to .env&lt;/span&gt;
docker compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Open &lt;code&gt;http://localhost:3000&lt;/code&gt; — the copilot is running.&lt;/p&gt;

&lt;p&gt;This is Project 01 of a 30-project series building AI + DevOps systems from scratch.&lt;/p&gt;




&lt;p&gt;What's the most painful part of your on-call experience that you wish an AI could handle? Drop it in the comments — it might become Project 02.&lt;/p&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
`

---
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>devops</category>
      <category>ai</category>
      <category>langchain</category>
      <category>python</category>
    </item>
    <item>
      <title>How I Built an AI That Diagnoses GitHub Actions Failures Automatically</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 20 Mar 2026 19:17:38 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/how-i-built-an-ai-that-diagnoses-github-actions-failures-automatically-2d7j</link>
      <guid>https://dev.to/vijaya_bollu/how-i-built-an-ai-that-diagnoses-github-actions-failures-automatically-2d7j</guid>
      <description>&lt;h2&gt;
  
  
  The Problem
&lt;/h2&gt;

&lt;p&gt;GitHub Actions failure logs are noisy. Finding the actual error in 500 lines of output takes time you don't have during an incident. You get a red X, click through multiple pages, scroll past runner setup noise and dependency install output, land on the real error — and then you have to figure out what it means and what to do about it. I was doing this loop manually too often, so I automated it.&lt;/p&gt;




&lt;h2&gt;
  
  
  What It Does
&lt;/h2&gt;

&lt;p&gt;The tool fetches your repository's failed workflow runs via the GitHub API, extracts the relevant error sections from job logs, pulls in the workflow YAML for context, and sends everything to Ollama running locally. You get back a root cause, an explanation of why it happened, exact YAML changes to make, and steps to prevent it from happening again — in about 15 seconds.&lt;/p&gt;

&lt;p&gt;Zero cloud costs. Your logs and code never leave your machine.&lt;/p&gt;




&lt;h2&gt;
  
  
  The 5 Failure Types It Handles
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Dependency conflicts&lt;/strong&gt; — version mismatches, packages missing from requirements.txt or package.json&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Missing secrets&lt;/strong&gt; — env vars referenced in the workflow but not configured in repository settings&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Permission errors&lt;/strong&gt; — &lt;code&gt;GITHUB_TOKEN&lt;/code&gt; scope issues, OIDC misconfiguration, action not allowed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Docker build failures&lt;/strong&gt; — base image not found, build context issues, registry auth failures&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Flaky tests&lt;/strong&gt; — identifies non-deterministic failures vs real bugs based on error patterns and exit codes&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Architecture
&lt;/h2&gt;

&lt;p&gt;The flow is straightforward: GitHub REST API → Python log parser → Ollama.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;GitHub API (workflow runs + job logs + YAML)
         │
         ▼
  extract_error_from_logs()   ← keyword scan, context windows, dedup
         │
         ▼
  analyze_failure()           ← structured prompt to Ollama
         │
         ▼
  Terminal report             ← root cause + YAML changes + prevention
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The log extraction step is where most of the work happens. Raw GitHub Actions logs are thousands of lines — runner diagnostics, apt output, pip install progress bars, none of which is useful. The parser scans for a list of error keywords (&lt;code&gt;error:&lt;/code&gt;, &lt;code&gt;failed&lt;/code&gt;, &lt;code&gt;Traceback&lt;/code&gt;, &lt;code&gt;exit code&lt;/code&gt;, &lt;code&gt;permission denied&lt;/code&gt;, etc.), then captures 8 lines of context before and 12 after each hit, deduplicates overlapping windows, and caps the result at 1500 characters before sending to the AI.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;extract_error_from_logs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;logs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;lines&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;logs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;error_keywords&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;error:&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;failed&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Traceback&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;fatal:&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;command not found&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;permission denied&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;exit code&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;returned non-zero&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;error_sections&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="n"&gt;seen_lines&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;kw&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;kw&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;error_keywords&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;seen_lines&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;continue&lt;/span&gt;
            &lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="n"&gt;seen_lines&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;error_sections&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;]))&lt;/span&gt;

    &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;error_sections&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="s"&gt;... (truncated)&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;1500&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The AI prompt includes the workflow name, failed job name, the extracted error section, and up to 1000 characters of the workflow YAML. That YAML context is what lets the AI suggest specific YAML fixes rather than generic advice.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;You are a GitHub Actions expert analyzing a failed CI/CD workflow.

Workflow: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;workflow_name&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;
Failed Job: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;job_name&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;

&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;workflow_context&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;

ERROR LOGS:
&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;limited_logs&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;

**ROOT CAUSE:** [each specific error, all of them]
**WHY THIS HAPPENED:** [plain language explanation]
**HOW TO FIX:** [numbered, actionable steps]
**YAML CHANGES:** [exact changes needed]
**PREVENTION:** [how to avoid this next time]
&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Most Useful Feature: Failure Classification
&lt;/h2&gt;

&lt;p&gt;The thing that saves the most time isn't root cause identification — it's the flaky vs real distinction.&lt;/p&gt;

&lt;p&gt;When a test fails with a non-deterministic error (race condition, network timeout, port already in use), re-running the workflow is the right call. When a test fails because you introduced a bug, re-running wastes 5 minutes and doesn't help. Before this tool, I'd often re-run first and only look at logs after the second failure confirmed it wasn't flaky. That's a 5-10 minute delay every time.&lt;/p&gt;

&lt;p&gt;The AI picks this up from patterns in the error output. A &lt;code&gt;ConnectionRefused&lt;/code&gt; or &lt;code&gt;socket timeout&lt;/code&gt; alongside a test failure is a different signal than a clean &lt;code&gt;AssertionError: expected 200, got 404&lt;/code&gt;. The prompt explicitly asks for this classification, and Llama 3.2 handles it reliably — it's the kind of pattern matching that LLMs are genuinely good at.&lt;/p&gt;

&lt;p&gt;The accuracy isn't perfect (the README documents ~85%), but it's good enough that I check the classification before deciding whether to re-run or investigate.&lt;/p&gt;




&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt; — a failed workflow log containing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Run pytest tests/
ERROR: No module named 'requests'
FAILED tests/test_api.py::test_health - ModuleNotFoundError
error: Process completed with exit code 1.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;🔍 GITHUB ACTIONS FAILURE ANALYSIS
Repository: myname/myrepo | Workflow: CI Build | Failed Job: test

🤖 AI DIAGNOSIS:

&lt;span class="gs"&gt;**ROOT CAUSE:**&lt;/span&gt;
pytest is failing because the 'requests' library is not installed.
The test imports it, but it's not listed in requirements.txt, so
pip install -r requirements.txt doesn't include it.

&lt;span class="gs"&gt;**WHY THIS HAPPENED:**&lt;/span&gt;
The package works locally because you have it installed globally on your
machine. In the GitHub Actions runner, only packages in requirements.txt
get installed — nothing else is available.

&lt;span class="gs"&gt;**HOW TO FIX:**&lt;/span&gt;
&lt;span class="p"&gt;1.&lt;/span&gt; Add 'requests' to requirements.txt: requests&amp;gt;=2.32.0
&lt;span class="p"&gt;2.&lt;/span&gt; Commit and push — the workflow will pick it up automatically.
&lt;span class="p"&gt;3.&lt;/span&gt; If requests is only needed for tests, add a requirements-dev.txt
   and install it separately in a dedicated workflow step.

&lt;span class="gs"&gt;**YAML CHANGES:**&lt;/span&gt;
No YAML changes needed. The fix is in requirements.txt.

&lt;span class="gs"&gt;**PREVENTION:**&lt;/span&gt;
Run your CI workflow locally with a clean virtualenv before pushing:
  python -m venv .venv &amp;amp;&amp;amp; source .venv/bin/activate
  pip install -r requirements.txt
  pytest tests/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That took 14 seconds. The manual version of this (click to run page, scroll logs, identify error, Google it, figure out the fix) is 10-15 minutes minimum.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a href="https://github.com/ThinkWithOps/ai-devops-projects" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps/ai-devops-projects&lt;/a&gt;&lt;br&gt;
&lt;strong&gt;Video:&lt;/strong&gt; &lt;a href="https://youtu.be/EwgdZ8KmBJg" rel="noopener noreferrer"&gt;https://youtu.be/EwgdZ8KmBJg&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Prerequisites: Ollama running with llama3.2 pulled&lt;/span&gt;
git clone https://github.com/ThinkWithOps/ai-devops-projects
&lt;span class="nb"&gt;cd &lt;/span&gt;ai-devops-projects/04-ai-github-actions-healer
pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt

&lt;span class="c"&gt;# Set your GitHub token (needs repo + workflow scopes)&lt;/span&gt;
&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;GITHUB_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"your_token_here"&lt;/span&gt;

&lt;span class="c"&gt;# Run against your repo&lt;/span&gt;
python src/github_actions_healer.py &lt;span class="nt"&gt;--repo&lt;/span&gt; your-username/your-repo

&lt;span class="c"&gt;# Save report to file&lt;/span&gt;
python src/github_actions_healer.py &lt;span class="nt"&gt;--repo&lt;/span&gt; owner/repo &lt;span class="nt"&gt;--output&lt;/span&gt; report.json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Project 4 in my AI+DevOps series — all tools run locally with Ollama, zero cloud AI costs. Project 3 was an AI AWS Cost Detective, Project 5 is an AI Terraform Code Generator. Links in my profile.&lt;/p&gt;




&lt;p&gt;What GitHub Actions failure do you dread the most? For me it's the OIDC / GITHUB_TOKEN permission errors — the error message never tells you which specific permission is missing. The AI actually handles those surprisingly well. Drop yours in the comments.&lt;/p&gt;

</description>
      <category>github</category>
      <category>devops</category>
      <category>cicd</category>
      <category>ai</category>
    </item>
    <item>
      <title># I Built a Local AI Terraform Generator and Tested It By Actually Deploying to AWS — Here Are the Results</title>
      <dc:creator>Vijaya Rajeev Bollu</dc:creator>
      <pubDate>Fri, 13 Mar 2026 15:29:11 +0000</pubDate>
      <link>https://dev.to/vijaya_bollu/-i-built-a-local-ai-terraform-generator-and-tested-it-by-actually-deploying-to-aws-here-are-the-n4m</link>
      <guid>https://dev.to/vijaya_bollu/-i-built-a-local-ai-terraform-generator-and-tested-it-by-actually-deploying-to-aws-here-are-the-n4m</guid>
      <description>&lt;h2&gt;
  
  
  The Idea
&lt;/h2&gt;

&lt;p&gt;Every time I needed a new AWS resource, I'd spend 20 minutes reading Terraform docs just to get the syntax right for something I'd done before. I wanted to type plain English and get working HCL back. But I also didn't want to just &lt;em&gt;generate&lt;/em&gt; code — I wanted to know if it actually deploys. So I tested every resource by running &lt;code&gt;terraform apply&lt;/code&gt; against a real AWS account.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;You describe infrastructure in plain English. The tool sends it to a local Llama 3.2 model via Ollama, which returns four Terraform files. Those files get saved to a &lt;code&gt;generated/&lt;/code&gt; folder, ready for &lt;code&gt;terraform init&lt;/code&gt; and &lt;code&gt;terraform apply&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Plain English → Python → Ollama (local) → Parse HCL → main.tf + variables.tf + outputs.tf + tfvars.example
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The key piece is the prompt. Getting consistent, parseable HCL out of an LLM required a very specific structure:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;You are a Terraform/OpenTofu expert. Generate production-ready infrastructure code.

USER REQUEST:
&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;

PROVIDER: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;provider&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;

CRITICAL:
- Generate ONLY valid Terraform/HCL code
- NO markdown formatting or code blocks
- Start each file with a comment showing the filename
- Separate files with: ### FILENAME ###

Format your response like this:

### main.tf ###
terraform {{
  required_version = &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;&amp;gt;= 1.0&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;
  required_providers {{
    &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;provider&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; = {{
      source  = &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;hashicorp/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;provider&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;
      version = &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;~&amp;gt; 5.0&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;
    }}
  }}
}}

[rest of main.tf code]

### variables.tf ###
[variables code]

### outputs.tf ###
[outputs code]

### terraform.tfvars.example ###
[example values]

Generate production-ready code now:&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;### FILENAME ###&lt;/code&gt; markers are what make the response parseable. The script splits on &lt;code&gt;###&lt;/code&gt;, reads the filename, grabs everything after it until the next marker, and writes that to disk. There's also a fallback parser for when the model goes off-script and wraps things in code blocks anyway.&lt;/p&gt;




&lt;h2&gt;
  
  
  Test Results: 10 Resources Tested
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Resource&lt;/th&gt;
&lt;th&gt;Generated&lt;/th&gt;
&lt;th&gt;Validated&lt;/th&gt;
&lt;th&gt;Deployed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;EC2 instance&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;S3 bucket&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;IAM role&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;VPC + subnets&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Security group&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RDS instance&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ALB&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lambda&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ECS task&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;⚠️ needed fix&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Complex module&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;td&gt;⚠️ needed fix&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;8/10 deployed first try. 2/10 needed minor manual fixes.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The ECS task definition had an incorrect &lt;code&gt;network_mode&lt;/code&gt; value for Fargate. The complex multi-resource module had a missing &lt;code&gt;depends_on&lt;/code&gt; for the security group. Both were one-line fixes once &lt;code&gt;terraform validate&lt;/code&gt; pointed at them.&lt;/p&gt;




&lt;h2&gt;
  
  
  What It's Good At vs Where It Struggles
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Good at:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Standard single-resource configs (EC2, S3, IAM, RDS) — near-perfect every time&lt;/li&gt;
&lt;li&gt;Wiring dependencies correctly between resources it knows well&lt;/li&gt;
&lt;li&gt;Generating &lt;code&gt;variables.tf&lt;/code&gt; with descriptions and sensible defaults&lt;/li&gt;
&lt;li&gt;Adding tags and naming conventions without being asked&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Struggles with:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Very new AWS resources where Llama's training data is thin&lt;/li&gt;
&lt;li&gt;Complex modules with many interdependent resources — sometimes misses a &lt;code&gt;depends_on&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Provider version pinning — occasionally suggests a deprecated argument from an older AWS provider version&lt;/li&gt;
&lt;li&gt;ECS/EKS specifics — these configs are dense and the model sometimes gets task definition fields wrong&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Honest assessment: treat it like a junior engineer who's read all the Terraform docs but hasn't deployed much to production. Good first draft, always needs a review.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Prompt Engineering That Made It Work
&lt;/h2&gt;

&lt;p&gt;Three things made the difference between garbage output and deployable HCL:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Kill the markdown.&lt;/strong&gt; LLMs love wrapping code in &lt;code&gt;&lt;/code&gt;&lt;code&gt;hcl&lt;/code&gt;&lt;code&gt;&lt;/code&gt; blocks. That breaks the file parser completely. The explicit instruction &lt;code&gt;NO markdown formatting or code blocks&lt;/code&gt; eliminated this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Show the exact format in the prompt.&lt;/strong&gt; Telling the model to use &lt;code&gt;### main.tf ###&lt;/code&gt; as a separator, and including the &lt;code&gt;terraform {}&lt;/code&gt; block structure directly in the prompt, anchored the output format. Without this, every response looked slightly different.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Demand variables explicitly.&lt;/strong&gt; Early versions hardcoded values like &lt;code&gt;instance_type = "t3.micro"&lt;/code&gt; directly in &lt;code&gt;main.tf&lt;/code&gt;. Adding &lt;code&gt;Use variables for configurable values&lt;/code&gt; to the requirements section fixed this — now everything configurable lands in &lt;code&gt;variables.tf&lt;/code&gt; with proper descriptions.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Local Matters for IaC Generation
&lt;/h2&gt;

&lt;p&gt;Your Terraform descriptions contain your architecture. "Create a VPC with private subnets, an RDS cluster for our auth service, and an ECS task that pulls from our private ECR registry" — that's a roadmap of your production infrastructure. Sending that to a cloud API means it leaves your machine.&lt;/p&gt;

&lt;p&gt;Running Ollama locally means the description, the generated code, and any sensitive context like account IDs or naming patterns stay on your machine. For anything touching production infrastructure, that's not optional.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a href="https://github.com/ThinkWithOps" rel="noopener noreferrer"&gt;https://github.com/ThinkWithOps&lt;/a&gt;&lt;br&gt;
&lt;strong&gt;Live deploy demo:&lt;/strong&gt; &lt;a href="https://youtu.be/nhhZqrCEhOA" rel="noopener noreferrer"&gt;https://youtu.be/nhhZqrCEhOA&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ThinkWithOps/ai-devops-projects
&lt;span class="nb"&gt;cd &lt;/span&gt;ai-devops-projects/05-ai-terraform-generator
pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt

&lt;span class="c"&gt;# Generate code&lt;/span&gt;
python src/terraform_generator.py &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--description&lt;/span&gt; &lt;span class="s2"&gt;"EC2 instance with S3 bucket for logs"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--provider&lt;/span&gt; aws

&lt;span class="c"&gt;# Deploy it&lt;/span&gt;
&lt;span class="nb"&gt;cd &lt;/span&gt;generated/
&lt;span class="nb"&gt;cp &lt;/span&gt;terraform.tfvars.example terraform.tfvars
terraform init &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform plan &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; terraform apply
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Project 5 in my AI+DevOps series — all tools run locally with Ollama, zero cloud API costs.&lt;/p&gt;




&lt;p&gt;What's your current Terraform workflow? I'm curious whether people are using Copilot for this, manually writing it, or something else entirely — drop it in the comments.&lt;/p&gt;

</description>
      <category>terraform</category>
      <category>aws</category>
      <category>devops</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
