DEV Community: Shrestha Pandey

Complete Guide to Context Engineering in LLMs

Shrestha Pandey — Sun, 02 Aug 2026 07:14:02 +0000

Context engineering is one of those terms that you understand when you actually build with LLMs in production. Then it becomes clear that this is the real work of deciding what information the model should see, how that information should be arranged, what should be remembered, what should be retrieved, and what should be kept out of the window entirely. This brings most of the quality improvements in 2026.

For a long time, LLM output quality was treated as a prompt-writing problem, which was true to a certain point when the use cases were simple and the conversations were short. But once LLMs started powering agents, coding assistants, research workflows, and copilots, the old prompt-first mindset began to break. More often, the model had the wrong context, too much context, stale context, or context arranged in a way that made it hard to use. That is the problem context engineering tries to solve.

What context engineering means

Context engineering is the discipline of shaping the full information payload around an LLM at inference time. The payload includes system instructions, user input, retrieved documents, memory, tool outputs, schemas, summaries, and the conversational history. In other words, the model is reasoning inside a temporary workspace, and the quality of that workspace heavily affects the answer it gives.

Prompt engineering changes how you ask the model a question, while context engineering changes what the model knows when it answers. A well-written prompt can still fail if the model is missing the relevant facts, overloaded with noise, or forced to reason over stale memories and irrelevant tool outputs.

Why it matters now

The reason this topic has exploded in 2026 is that LLM applications are now systems, they read files, call tools, search databases, remember prior state, execute multi-step workflows, and sometimes even hand work off to other agents. As soon as you move into that world, the biggest source of failure becomes context management, not prompting.

This is especially visible in long-horizon agents. If an agent is working for many steps, it creates its own history, accumulates its own tool results, and gradually fills the window with decisions, partial outputs, and summaries. Without intentional context control, the model starts to lose track of what’s important. Anthropic’s agent guidance for long-running systems highlights this problem and points to compaction, structured notes, and careful memory handling as practical solutions.

The four pillars

A useful way to organize context engineering is around four connected actions: write, select, compress, and isolate. This framework captures the full lifecycle of context rather than just the initial prompt.

Write means storing durable information outside the active window. If something should survive beyond the current turn, it should not live only in chat history. User preferences, task checkpoints, important decisions, stable project facts, and reusable notes all belong in a more persistent store. That can be a memory system, a database, a file, or any structured state layer.

Select means retrieving only what is relevant for the current step. This is where RAG, semantic search, code search, and memory retrieval matter. Good selection is all about finding the smallest set of evidence that is enough to support the task.

Compress means reducing context size without losing meaning. This usually involves summaries, pruning old tool outputs, shortening long conversations, and representing repeated information in compact form. Compression is what keeps a system usable after the first few steps.

Isolate means separating tasks so they do not corrupt each other. A planning context should not be mixed with execution noise. Untrusted text should not sit in the same place as trusted instructions. Different agents or stages should be kept distinct when the workflow gets complicated.

How a production pipeline works

In production, context engineering usually looks more like pipeline design. The user request enters the system, then the app classifies the task, fetches relevant sources, trims and ranks them, strips out unnecessary content, adds the minimum required instructions and tool definitions, and then makes the model call. After the call, useful outputs are stored externally so they can be reused later without bloating the live context.

That pipeline is easy to describe, but many teams still skip parts of it. They either retrieve too much and drown the model in text, or retrieve too little and leave it guessing. The best systems tend to be aggressively selective. They trust structure over volume.

Where systems go wrong

The most common mistake is assuming more context automatically means better answers. In practice, that often creates the opposite effect. Long windows can degrade when the context is noisy, poorly ordered, stale, or full of redundant material. The model may technically have the information, but it may not use it well.

Another classic failure is weak retrieval. If the right facts exist somewhere but are not brought into the window at the right time, the model will improvise. This can be dangerous in coding, research, support, or agent workflows where correctness matters. Tool output has the same problem, if it is not integrated cleanly, the model may ignore it or overvalue it.

A third issue is context pollution. Once irrelevant text is mixed into the working set, the model can treat it as if it matters. This is why untrusted content, especially retrieved text or user-provided documents, has to be handled carefully. It is not enough to fetch information; you also need to control how that information is presented.

What works

The same techniques appear in most practical 2026 guides. Using clear sections for instructions, background, and expected output helps the model understand what each part is for. Short summaries of earlier messages keep important context without including the entire conversation. Breaking information into smaller, relevant chunks makes retrieval more accurate and avoids unnecessary text. Caching also saves time and cost by reusing prompts or templates that do not change.

The key idea is that more context is not always better. The best context gives the model only the information it needs to complete the task correctly. This may seem simple, but it actually improves performance. Leaving out unnecessary details reduces distractions and helps the model focus on the most relevant information.

For agents and coding assistants

This becomes very clear in agentic systems and coding tools. A coding assistant does not need your entire repository in the window to help you fix one bug. It needs the right files, the relevant symbols, the recent diffs, the related tests, and a bit of project-level convention. Good context engineering is about providing exactly the information needed for the task.

The same idea applies to AI agents that work over long periods. They need ways to save progress, organize memory, compress old information, and separate long-term knowledge from temporary working notes. Recent guidance, including Anthropic's recommendations and other 2026 resources, supports this more structured approach, where the system actively manages context over time instead of treating it as one large block of information.

The 2026 mindset shift

The biggest shift in 2026 is that context engineering is now seen as a complete system design practice. It combines prompt design, retrieval, memory management, tool integration, and state management to help AI models perform reliably. As a result, newer concepts such as retrieval budgeting, context compaction, memory tiering, tool-result pruning, and context isolation have become important parts of modern AI system design.

At the same time, the field is becoming more realistic about the limits of large context windows. While larger context windows are useful, they do not automatically improve results. The real challenge is selecting and managing the right information instead of simply providing more of it. This is why improving LLM performance is increasingly seen as a systems engineering problem during inference, rather than relying only on building larger models.

Final thought

If prompt engineering was about wording, context engineering is about design. It asks a harder question: what should the model know right now, and what is the cleanest possible way to make that knowledge available? Once you start thinking in this way, a lot of LLM failures start to look like ordinary information architecture problems. And that is good news, because information architecture is something that engineers can actually improve.

For more such developer content, visit:
https://vickybytes.com

My First End-to-End Data Pipeline in Databricks

Shrestha Pandey — Fri, 24 Jul 2026 13:40:50 +0000

Whenever I searched for resources on Databricks, I found two extremes. Some explained the concepts without showing how they fit together, while others jumped straight into large projects assuming you already knew the basics, but I wanted something in the middle.

Rather than learning every feature separately, I decided to build a small end-to-end project that covered the fundamentals of data engineering in Databricks. I wanted to understand how data moves through the platform, how different components connect, and why concepts like Delta Lake and the Medallion Architecture are used so often.

For this project, I used three simple retail datasets containing customers, products, and orders. Starting with these CSV files, I built a pipeline that reads the data using PySpark, stores it in Delta Lake, organizes it into Bronze, Silver, and Gold layers, analyzes it with Spark SQL, creates a dashboard, and finally automates the entire workflow using Databricks Jobs.

If you're just getting started with Databricks, this project covers many of the concepts you'll use in real-world workflows while keeping the implementation simple enough to follow.

Why Databricks?

Before starting with code, I explored the Databricks workspace to understand what the platform offers. The interesting part was how everything required for a data engineering workflow is available in one place.

The Workspace is where notebooks live, the Catalog helps organize data assets, the SQL Editor is used for writing analytical queries, and Jobs allows notebooks to run automatically on a schedule. Having these components integrated into a single platform makes it much easier to move from raw data to analytics without switching between multiple tools.

Another concept that appears everywhere in Databricks is the Lakehouse Architecture. Raw data needs to be stored safely, transformed into cleaner datasets, and eventually prepared for reporting or dashboards. The Lakehouse approach supports all these stages while using Delta Lake as the storage layer, which brings features like reliable transactions and version history.

This project follows that same approach from start to finish, which makes it easier to understand why the Lakehouse architecture has become a common choice for modern data engineering.

Setting Up the Environment

For development, I used a Databricks Notebook and uploaded my datasets into a Databricks Volume. I used Python and PySpark for data ingestion and transformations, switched to SQL for analysis, and added Markdown cells to organize different sections of the notebook.

The datasets were uploaded into a Databricks Volume, making them easy to access from the notebook.

The project uses three CSV files:

customers.csv
orders.csv
products.csv

Keeping the dataset small made it much easier to focus on understanding the workflow rather than spending time cleaning complex data.

Building the Data Pipeline with PySpark

With the environment ready, the next step was reading the datasets into Databricks using PySpark. Since the files were already uploaded to a Volume, accessing them from the notebook was simple.

customers = spark.read.csv(
    "/Volumes/workspace/default/retail_data/customers.csv",
    header=True,
    inferSchema=True
)

orders = spark.read.csv(
    "/Volumes/workspace/default/retail_data/orders.csv",
    header=True,
    inferSchema=True
)

products = spark.read.csv(
    "/Volumes/workspace/default/retail_data/products.csv",
    header=True,
    inferSchema=True
)

PySpark automatically inferred the schema, so I didn't have to manually define the data types for every column. After loading each dataset, I used display() to verify that everything had been imported correctly before moving on to transformations.

Storing the Raw Data with Delta Lake

Once the CSV files were loaded, I converted them into Delta tables. This was the beginning of the Bronze layer in the Medallion Architecture.

customers.write.mode("overwrite").format("delta").saveAsTable("bronze_customers")

orders.write.mode("overwrite").format("delta").saveAsTable("bronze_orders")

products.write.mode("overwrite").format("delta").saveAsTable("bronze_products")

The Bronze layer stores the data exactly as it arrives. At this stage, no cleaning or transformations are applied because it's useful to preserve the original data for auditing, debugging, or reprocessing later.

Cleaning the Data in the Silver Layer

To create the Silver layer, I performed some simple transformations on the orders dataset. For this, I removed duplicate records and filled missing values.

silver_orders = (
    bronze_orders
    .dropDuplicates()
    .na.fill({"quantity": 1})
)

The cleaned dataset was then stored as another Delta table. This stage represents a common pattern in data engineering. Cleaning and validating data before using it for analysis helps improve the reliability of downstream reports and dashboards.

Creating the Gold Layer

The final step in the transformation process was creating a dataset that could be used directly for analysis. I joined the customer, product, and order tables into a single DataFrame and calculated the revenue for each order.

from pyspark.sql.functions import col

gold_sales = (
    silver_orders
    .join(bronze_customers, "customer_id")
    .join(bronze_products, "product_id")
    .withColumn("revenue", col("price") * col("quantity"))
)

Finally, I saved the result as a Delta table.

gold_sales.write.mode("overwrite") \
    .format("delta") \
    .saveAsTable("gold_sales")

At this point, raw CSV files gradually turned into a structured dataset that could answer business questions with just a few SQL queries. It also demonstrated how PySpark and Delta Lake work together. PySpark handled the transformations, while Delta Lake provided a reliable storage layer for every stage of the pipeline.

Querying the Data with Spark SQL

Once the Gold table got ready, I switched to SQL to explore the data. It was really feasible to move between PySpark and SQL. Transforming data with PySpark felt simpler, while SQL made it simple to answer business questions without writing additional Python code.

For this project, I used the Databricks SQL Editor and a SQL Warehouse to run analytical queries. SQL Warehouses are optimized for interactive queries and reporting workloads, making them a good choice when exploring datasets or building dashboards.

I started with: Which products generated the highest revenue?

SELECT
product_name,
SUM(revenue) AS total_revenue
FROM gold_sales
GROUP BY product_name
ORDER BY total_revenue DESC;

The result highlighted the products contributing the most revenue.

Creating a Dashboard

After running the query, I created a simple bar chart directly within Databricks. The dashboard visualized revenue by product, making the results much easier to interpret than reading rows in a table.

Databricks lets you create charts from SQL query results in just a few clicks, which is useful for quickly sharing insights with teammates or stakeholders.

Automating the Pipeline with Databricks Jobs

Running a notebook manually is useful during development, though production pipelines usually need to execute on a schedule.

To complete the workflow, I created a Databricks Job for my notebook. The setup involved selecting the notebook, attaching the compute resource, and defining a schedule. Databricks also provides options for retries and notifications, making it easier to monitor automated workloads as projects become more complex.

Exploring Delta Time Travel

One feature I wanted to try before finishing the project was Delta Time Travel. Every change made to a Delta table is recorded, allowing previous versions to be inspected whenever required.

I viewed the history of my Gold table using:

DESCRIBE HISTORY gold_sales;

This feature can be especially useful when debugging pipelines or recovering from accidental updates.

A Quick Look at Unity Catalog

Since all my tables were created inside Databricks, I also explored Unity Catalog, which serves as the central place for managing data assets. It organizes tables, volumes, and other resources, making them easier to discover and manage across projects.

While this project focused on the fundamentals, Unity Catalog also supports governance features such as permissions and data lineage, which become increasingly important in collaborative environments.

What I Learned

Starting with raw CSV files and gradually moving through Bronze, Silver, and Gold layers showed how data evolves before reaching analysts or dashboards. PySpark handled the transformations, Delta Lake provided a reliable storage layer, SQL made analysis simpler, and Databricks Jobs completed the workflow by automating the notebook.

The project is small, but it covers many of the core ideas you'll encounter while working with Databricks. It helped me understand why these concepts exist rather than simply memorizing their definitions.

If you're getting started with Databricks, I'd recommend building a similar end-to-end project. It doesn't require a large dataset, and you'll come away with a much clearer understanding of how the platform works.

GitHub Repository

The complete notebook, datasets, and project files are available here:

🔗 GitHub: Databricks-pipeline

If you build on top of this project or have suggestions for improving the pipeline, I'd love to hear your thoughts.

For more such project ideas, visit:
https://vickybytes.com

The Debugger Is Lying to You Sometimes

Shrestha Pandey — Tue, 21 Jul 2026 20:46:07 +0000

Debugging should feel like the safest part of programming. Sometimes it does, but sometimes the debugger makes everything look fine while the real bug is hiding somewhere else, which is one of the most frustrating parts of development.

Everything looks correct, still the app breaks. This happens because the debugger only shows one moment in time. Software keeps moving, values change, requests come back late, state updates in the background, and the bug may already have shifted by the time you look at it.

When everything looks correct

A lot of confusing bugs start here. The code looks fine in the debugger, but the app still behaves badly. This usually means the problem is not in the line you are looking at. It may be in what happened before that line, or after it, or somewhere completely different.

This is very common in frontend work. A component may show the right props, but the state is already stale. A hook may run with old data. A callback may still be using an earlier value. In backend code, a request may arrive at the right place but with data that was changed by another process. In both cases, the debugger is honest, but only for that second.

Why state causes trouble

State is where many bugs hide. It changes quietly, and sometimes it changes in more than one place. A local variable may look perfect, but the real issue is the state that was copied earlier and never updated. A UI may look correct on screen, while the internal data is out of sync.

The issue is usually the flow around it. That's also the reason why bugs in modern apps can feel harder than they should. The code may be doing exactly what you wrote, but not what you thought it would do.

Logs can confuse you

Logs help a lot, but they can also mislead you if you trust them too much. A log only shows what you decided to print. If the important branch never ran, the log will not tell you that. If a promise resolved later, you may miss the real order of events. If an error happened before your log line, the message can give you the wrong idea.

That is why logs work best when they show movement. I usually find them most useful when they capture input, output, and any point where the program changes direction. A single log line not always tells the full story.

Hot reload and cache make it worse

Sometimes the bug is not even in the code you think you are running. Hot reload can keep old state alive. Browser cache can hold on to old files. Service workers can serve stale assets. A local build can look updated while the browser is still running something older.

These bugs are annoying because they make you doubt yourself. You change the code, refresh the page, and still see the old behavior. It feels like the debugger or the code is broken, when the real issue is often the environment. That is why clearing cache, restarting the dev server, or opening a clean session fixes more problems than people expect.

Local and production are different

A bug on your machine is not always the same bug users see in production. Something that behaves fine locally can fail under real load or with real user actions.

Local debugging is only part of the job. It helps you narrow things down, but it does not always show the whole picture. Production needs its own signals like logs, error tracking, metrics, traces, and good reporting.

What helps

When a bug refuses to show itself, I think it helps to stop looking only at the line in front of you.

A few simple habits make this easier:

Watch the full flow rather than just one line.
Check values before and after async work.
Clear cache when behavior looks stale.
Restart the app when hot reload feels suspicious.
Add temporary logs around the change in state.

These small steps reveal what the debugger is missing.

Final thoughts

The debugger is useful, but it is not the whole truth. It shows you one moment, and sometimes that moment is not the one that matters. Real debugging is more about understanding flow, state, timing, and environment. Once you start thinking that way, hard bugs become less mysterious.

For more such developer content, visit:
https://vickybytes.com

How Beginner Developers Can Find Great Project Ideas

Shrestha Pandey — Mon, 06 Jul 2026 18:12:33 +0000

Every beginner developer hits the same issue at some point. You learn a few basics, finish a tutorial, and then you have no idea what to build next. That gap can feel bigger than learning the code itself, because now the question is not “How do I write this?” but “What should I build at all?”

This article is for that moment. I want to make it simple, practical, and useful, because project ideas do not need to be too advanced to be valuable. A good project is one that teaches you something, keeps you going, and gives you enough confidence to build the next one.

Why project ideas are important

There’s a common thing that I have noticed in most of the beginners, that is, watching too many tutorials. Tutorials are helpful, but actual learning starts when you try to build something on your own. That is when you start facing real decisions, small bugs, unclear logic, and the feeling of connecting different parts into one working product.

That is one of the reasons why project ideas matter so much. The right idea gives you direction, but it also gives you energy. When the project feels too huge, you get stuck. When it feels too small or boring, you stop caring. The sweet spot is a project that feels possible and still a little exciting.

This matters even more today. Tools like ChatGPT or Copilot can help you write code faster, but that doesn't solve the real problem beginners have. Writing the code was never the hard part for long but knowing what to build is.

Start with problems you already know

The easiest project ideas often come from your own life. Think about small things you do every day that feel annoying, repetitive, or messy. A simple to-do list, habit tracker, note saver, expense log, study planner, or meal planner can all become strong beginner projects if you build them well.

This works because the problem is already familiar to you. You do not have to invent a fake use case or force a complicated feature list. You already know what the app should do, what feels useful, and what would make it easier to use.

Borrow ideas from tutorials, then make them yours

Tutorial projects are not bad. In fact, they are one of the best ways to learn. But you need to avoid copying them word for word and calling it done. If you followed a weather app tutorial, try changing the design, adding saved cities, showing alerts, or making the app work for your own city list.

This small change matters a lot. It turns a passive learning exercise into a valuable project. You still get the guidance, but you also start making choices on your own, and that is where confidence starts growing.

Look at everyday tools

Another easy way to find ideas is to look at tools you already use. Think about apps for tasks, reminders, shopping lists, expense tracking, journaling, or learning. These tools are popular because they solve simple problems clearly, and beginner developers can build smaller versions of them without needing a huge team.

You do not need to recreate the full product. A clean, focused version is enough. A mini version of a notes app or a simple budget tracker can teach you a lot more than a random overcomplicated idea that you never finish.

Turn one feature into one project

Beginners often make the mistake of thinking a project needs many features to be impressive. It does not. A tiny, focused project is often better because it is easier to finish and easier to understand. For example, one feature can become one project. A form that saves data. A search bar that filters results. A login page with validation. A dashboard that shows one useful metric. When you build around one clear action, the project feels manageable and still useful.

Try something small with AI in it

You don't need to build a big AI product. A small one still teaches you a lot, like how to call an API, handle a response, and manage what's happening on screen. A few simple ideas:

A tool that shortens long articles or text
Something that turns your notes into flashcards
A small tool that answers questions from a PDF you upload
A journal app that gives you a short reply based on what you wrote

A simple, working version of any of these makes a solid beginner project.

Use communities for inspiration

If your own ideas feel stuck, look at what other beginners are building. GitHub, Dev.to, Reddit, Discord communities, hackathon submissions, project labs and open-source repositories can all give you fresh direction. This way you start to notice patterns, problems, and styles of projects that keep appearing. You will often find that many useful ideas are small variations of the same core concept. Most great beginner projects are not original inventions, but they are thoughtful versions of common ideas with a personal twist.

Think in “versions”

A lot of beginners wait for the perfect idea, that usually delays everything. A better way to think is in versions. Version one can be simple and ugly, as long as it works. Version two can improve the design, and version three can add one or two stronger features. This way of thinking helps you start faster. It also keeps you from quitting because the idea feels too ambitious. You are not building the final version of a startup. You are building something that helps you learn, ship, and improve.

Make the project useful to someone

A project becomes more meaningful when it helps a real person, even in a small way. That person can be you, a friend, a student, or a small community. When you know who it is for, the idea becomes easier to shape.

For example, a revision planner for students, a simple content calendar for creators, or a shared checklist for a small team already has a clear purpose. The moment you know the user, you start building something that makes sense.

A simple way to choose

If you still do not know what to build, use this simple test.

Ask yourself:

Can I explain this idea in one sentence?
Can I build a first version in a reasonable amount of time?
Will I learn something new from it?
Do I care enough to finish it?

If the answer is yes to most of these, you probably have a good project idea.

What to avoid

Try not to start with ideas that are too broad. “Make a social media app” sounds exciting, but it often turns into confusion fast. Huge projects can be motivating at first, then frustrating once the scope starts growing. Also avoid choosing an idea just because it sounds impressive. The best beginner projects are often the ones that teach you core skills clearly. A simple app that you actually finish is far more valuable than a complex one that stays half-done.

Where Creator Labs fits in

If you are someone who learns better with guidance, structured challenge spaces can help a lot. That is one reason places like Creator Labs by VickyBytes can be useful for beginners who want direction, ideas, and a nudge to actually build. It is easier to stay consistent when you have a place that keeps you thinking in terms of projects, not just tutorials.

I like that kind of setup because it helps beginners move from “I know the basics” to “I can build something real.” That transition is where most people get stuck, and that is where the right support can make a difference.

Final thought

Finding a great project idea is all about noticing small problems, starting with something simple, and building in a way that keeps you moving. The best ideas are usually the ones you can explain clearly, start quickly, and finish without losing interest.

If you are a beginner, give yourself permission to build small. A finished small project teaches more than an unfinished big one, and every strong developer starts by making that first real thing

The Best AI Tools for Developers in 2026

Shrestha Pandey — Sat, 04 Jul 2026 12:46:52 +0000

AI tools for developers are everywhere right now. Some are genuinely useful, some are overhyped, and some only fit a very specific kind of workflow. AI has moved from basic autocomplete into something that can help with debugging, refactoring, code review, app scaffolding, and parts of deployment too.

This post looks at the tools that keep coming up in real developer conversations and the kinds of tasks they actually help with. The goal is to know which tool makes your day-to-day work smoother, faster, and less annoying.

Why this topic

A few years ago, most AI coding tools were mostly about completing lines of code. In 2026, the conversation is much broader. Developers are using AI for explanation, planning, code generation, debugging, review, and workflow cleanup.

That matters because in software development, a lot of the job is reading unfamiliar code, making safe changes, reviewing pull requests, and moving through repetitive work without losing focus. The best AI tools are the ones that help with those parts.

What counts as a useful AI dev tool

For this article, I’m focusing on tools that help with real development work. That includes code completion, editing, reasoning through bugs, generating app scaffolds, reviewing code, and speeding up routine tasks. I’m also keeping the definition practical. A product can call itself “AI for developers”, but that does not automatically make it useful. The tools that matter are the ones that save time without getting in the way of quality.

The editor-first options

Cursor is one of the most talked-about AI-first coding environments right now. It gives you an AI-driven editor experience rather than just a plugin layered on top of an existing setup. That makes it appealing if you want the assistant to feel closely connected to the code you’re working on.

GitHub Copilot still makes a lot of sense for developers who want AI help without changing their habits too much. Its strength is that it fits naturally into workflows people already use every day. For many developers, that familiarity matters more than having the newest interface.

The reasoning-heavy helpers

Claude Code comes up often in conversations about harder coding tasks. People tend to reach for it when they want help reading large codebases, untangling bugs, or working through bigger refactors. That kind of tool is useful because a lot of developer time goes into figuring out what a system is doing before writing the fix. When the problem is messy, a tool that helps you think more clearly can be worth a lot.

The terminal-friendly choices

Aider stands out because it fits well into a Git-based workflow. It works for developers who like staying in the terminal and want AI edits tied directly to the repository and change history.

This category matters because not every developer wants a full AI-first IDE. Some people prefer smaller tools that feel close to the command line and less disruptive to the way they already work. For them, a terminal-friendly assistant can feel much more natural.

The newer all-round contenders

Windsurf is another name that keeps showing up in 2026 AI tool conversations. It sits in the same broad category as other AI-first coding environments, so the real decision usually comes down to workflow fit, pricing, and how the tool behaves in practice.

The bigger point here is that the market is no longer about one dominant product. It’s about which tool removes the most friction from your own workflow. That is why people keep comparing these tools by use case rather than treating one of them as the universal answer.

What these tools do well

The most useful AI tools are the ones that remove friction from repetitive parts of development. That usually means generating a first draft faster, explaining unfamiliar code, helping with small edits, accelerating debugging, or handling tedious transformations. That kind of help can make a real difference during a busy week. It gives you more room to focus on the parts that need judgment, like architecture, testing, and code quality.

What to be careful about

AI can make people feel productive very quickly. That can be helpful, and it can also be risky if you move faster than your understanding. Generated code still needs review. Suggestions still need testing. If a tool helps you ship faster without helping you understand the changes, that speed can become a problem later when the code needs maintenance or collaboration.

My honest take

If you are choosing one tool to start with, pick the one that fits how you already work. If you want an AI-first editor, Cursor is a strong place to look. If you want a familiar workflow with low friction, Copilot still makes sense. If you want help thinking through harder code, Claude Code is worth exploring. If you like terminal-based work, Aider is a practical option.

I would not frame this as a one-tool-fits-all situation. Different tools solve different problems, and the best one is the one that helps you ship better work with less friction. The best AI tools for developers in 2026 are not always the loudest ones, they are the ones that make development smoother, clearer, and less repetitive without getting in the way of quality.

How to Deploy Your ML Model to AWS (Step-by-Step Guide)

Shrestha Pandey — Mon, 22 Jun 2026 07:21:40 +0000

I've trained more ML models than I've deployed. There's something comforting about the local loop—model.fit(), model.evaluate(), hitting 94% accuracy, then staring at the screen wondering, "Okay, how do I make this actually useful?"

If you're stuck there right now, this guide will help.

Note: I wrote this based on AWS documentation and standard SageMaker patterns. If you try it, drop a comment about what worked (or broke).

What You Need Before Starting

AWS account with SageMaker enabled
A trained model saved as model.pkl (or .joblib)
requirements.txt with your dependencies
Python 3.8+ installed
AWS CLI configured (aws configure)

Step 1: Save Your Model

import joblib
joblib.dump(model, 'model.pkl')

Create a requirements.txt file:

sklearn==1.2.0
pandas==1.5.0
numpy==1.23.0`

Keep both files in the same folder.

Step 2: Upload to S3

import boto3

s3 = boto3.client('s3')

bucket_name = 'my-unique-ml-bucket-12345'  # Make this unique
s3.create_bucket(Bucket=bucket_name, CreateBucketConfiguration={
    'LocationConstraint': 'us-east-1'
})

s3.upload_file('model.pkl', bucket_name, 'models/model.pkl')
s3.upload_file('requirements.txt', bucket_name, 'models/requirements.txt')

model_s3_path = f's3://{bucket_name}/models/model.pkl'

Step 3: Write Your Inference Script

Save this as inference.py:

import json
import joblib
import numpy as np
import os

model = None

def model_fn(model_dir):
    return joblib.load(os.path.join(model_dir, 'model.pkl'))

def input_fn(input_data, content_type):
    if content_type == 'application/json':
        data = json.loads(input_data)
        return np.array(data['features'])
    raise ValueError(f"Unsupported content type: {content_type}")

def predict_fn(input_data, model):
    return model.predict(input_data)

def output_fn(prediction, content_type):
    return json.dumps({'predictions': prediction.tolist()})

These four functions are what SageMaker calls when someone hits your endpoint.

Step 4: Deploy Using Python SDK

Run this in a Python script:

from sagemaker.sklearn.model import SKLearnModel
from sagemaker import get_execution_role

sklearn_model = SKLearnModel(
    model_data=model_s3_path,
    role=get_execution_role(),
    instance_type='ml.m5.large',
    entry_point='inference.py',
    py_version='py3'
)

sklearn_model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.large',
    endpoint_name='my-model-endpoint'
)

This takes 5–10 minutes. You'll see Creating → In Service.

Step 5: Test Your Endpoint

import boto3
import json

runtime = boto3.client('sagemaker-runtime')

response = runtime.invoke_endpoint(
    EndpointName='my-model-endpoint',
    ContentType='application/json',
    Body=json.dumps({'features': [[5.1, 3.5, 1.4, 0.2]]})
)

result = json.loads(response['Body'].read().decode())
print(result)

If you see {'predictions': [...]}, it worked.

Step 6: Clean Up

Endpoints cost money even when idle:

aws sagemaker delete-endpoint --endpoint-name my-model-endpoint
aws sagemaker delete-endpoint-config --endpoint-config-name my-model-endpoint

Common Errors (And Fixes)

Error	Fix
`NoCredentialsError`	Run `aws configure` again
`InvalidRoleException`	IAM role needs S3 + SageMaker permissions
`ModelError`	Check `inference.py` for missing imports
Endpoint stuck on `Creating`	Wait 5–10 more minutes

Your IAM role needs:

s3:GetObject, s3:PutObject
sagemaker:CreateModel, sagemaker:CreateEndpoint

Cost Breakdown

Resource	Cost
`ml.m5.large`	~$0.20/hour (~$6/month if 24/7)
S3 storage	~$0.02/GB/month

Delete when not using. I've seen $50 surprises from idle endpoints.

Verify This Before You Trust It

If you're following this, check:

AWS SDK version — Run pip show boto3 sagemaker
IAM role permissions — Biggest blocker is usually missing permissions
Region mismatch — S3 bucket region must match SageMaker region
Inference.py imports — Make sure os, joblib, numpy are installed

If something breaks, comment below with the error. I'll update this guide.

Final Thoughts

Deploying ML feels intimidating until you do it once. SageMaker handles most of the complexity. You just upload your model to S3, point SageMaker at it, and deploy.

I've trained models that sat on my laptop for months because I didn't know how to deploy them. Now I tell people: "Just run this script, it's not that hard."

If you're building something with this, drop a comment. I love seeing what people deploy.

70B AI Model Runs on 8GB Laptop

Shrestha Pandey — Tue, 16 Jun 2026 18:29:25 +0000

You needed a $100,000 server to run huge AI models. Now you can do it on a regular laptop. One developer figured out how, and it changes everything for students, developers, and small companies who want to use AI without breaking the bank.

A few years ago, running LLaMA 70B required serious hardware, multiple GPUs, 80GB RAM per GPU., a server rack costing more than a car, due to which most people couldn't touch it. You either worked at a big tech company with a data budget, or you couldn't run these models at all.

In 2026, you can run the same model on a laptop with 8GB RAM. The laptop you bought three years ago. The one on your desk right now and it works.

What Happened

A developer uploaded something to GitHub called AirLLM. The README said: "Run 70B models on 8GB RAM. No GPU required." That's the whole pitch.

Developers downloaded it. They tested it on old laptops and budget computers. Even on machines that should not work, and it worked.

How It Works

A 70B model takes about 140GB of RAM normally. Even compressed to 4-bit, you still need 35GB. Mostly, laptops don't have that.

AirLLM gets it down to 8GB. It loads the model differently. Instead of putting everything in RAM at once, it loads parts. When you ask it something, it loads the layers it needs, answers, then swaps them out for the next layers.

Like reading a book page by page instead of holding all 1,000 pages at once. AirLLM does this with the model. The model is still 70 billion parameters. It's still smart but it never needs all that memory at the same time.

The technique uses memory mapping and layer swapping. Both are old ideas but putting them together in one tool is what made it work.

Is It Fast?

No. Running a 70B model on 8GB RAM from a laptop is slower than running it on a server. You're trading speed for getting it to work at all.

On a 2021 MacBook with 8GB RAM, AirLLM generates about 3-5 tokens per second. That's readable. You can chat with it and ask questions, which was not instant, but still usable.

On a faster laptop with 16GB RAM? Maybe 8-12 tokens per second. Close to real-time.
On a server with a GPU? 50-100 tokens per second. That's the speed people expect.

So AirLLM is slower, but it works on computers that are not expected to work.

Who Can Use This?

Students don't need a $10,000 computer to learn AI. They can run huge models on the laptop their parents gave them, which removes the biggest barrier to learning.

Developers can test AI locally without sending data to the cloud. Their code stays on their machine and their questions stay private.

Small companies don't need to rent GPU servers from AWS or Google Cloud. They can run models on regular computers. That saves thousands of dollars every month.

What Models Work?

AirLLM supports LLaMA 2 70B, Mistral 7B, Gemma 2 27B, and Falcon 180B if you have more RAM.

The 70B models are the sweet spot. They are big enough to be smart and small enough to fit on a laptop when compressed.

You can also run smaller models faster. A 7B model on AirLLM runs at 20-30 tokens per second on a regular laptop, which is instant.

The Tradeoffs

Speed is slower: A 70B model on a server is 10-20x faster. If you need speed for production, AirLLM is not for you.

Quality drops a bit: The model is compressed to 4-bit, which means less precision. But it still answers well and makes sense.

The model takes about 35GB of disk space. So your laptop gets hot and the fan gets loud, maybe after 10 minutes.

How to Run It

You need Python.

bashpip install airllm

Download the model from Hugging Face:

pythonfrom airllm import AirLLM

model = AirLLM("meta-llama/Llama-2-70b-hf")
response = model.generate("What is quantum computing?")
print(response)

Final Thoughts

AI is no longer controlled by companies with money. You don't need to send questions to a cloud server, pay for API calls, or wait for a company to give you access. You can run the model yourself on your computer.

It's not perfect and fast, but it works. And it works on a laptop with 8GB RAM.

A few years ago, running a 70B AI model was fantasy. You needed a data center. But now, you need a laptop. It's a power shift.

AI is no longer just for the rich, it's for anyone with a computer.

Resources

AirLLM GitHub — The main tool
LLaMA 2 70B on Hugging Face — Download the model

Note: Edited with AI Assistance

AI Created Its First Real Cyber Attack And It Bypassed 2FA

Shrestha Pandey — Thu, 11 Jun 2026 21:46:05 +0000

For the first time, AI has been used to exploit a software vulnerability. Google discovered it in May 2026, and the consequences could be serious.

The Moment Everything Changed

In May 2026, Google’s Threat Intelligence Group identified something that security researchers had been warning about for years, and it came sooner than expected.

Hackers used an AI model to create a working zero-day exploit, a cyber attack that targets a vulnerability no one knows about yet. And it wasn’t a simple attack. It bypassed two-factor authentication (2FA), the security layer that millions of people trust every single day.

This is the first case of AI weaponizing a vulnerability for real-world attacks. Before this, zero-days required elite hackers and months of research. But now, AI can find them in hours.

Let’s breakdown what happened and why it’s so terrifying.

What Got Attacked?

The target of attack was an open-source web-based system administration tool. Google didn’t disclose the name, but it’s a popular system that IT companies used to manage servers, websites and computers.

The exploit let attackers log in without entering the second authentication code, even when 2FA was turned on. You could enter the password and the system would log you in directly, skipping the phone code setup completely.

The vulnerability was not just some coding mistake like memory error, it was a logic flaw.

The developers hardcoded something like:

if user_is_admin:
skip_2fa = True

The traditional security scanners missed it, but AI found it.

How the Attack Worked?

The AI used by hackers, wrote some Python script.

The script still needed valid credentials to work, so knowing the correct password was required. But as soon as it had these credentials, it was able to bypass 2FA entirely. There was no phone code or verification required, and it went straight into the account.

The major plan was mass exploitation. The cybercrime group wanted to use this on thousands of users of the tool at once. There goal was going after everyone using the software, not just one company.

Google worked with the vendor to fix the vulnerability so the mass exploitation didn’t happen. But the exploit was real and AI created it.

The AI Tells: How We Know It Was AI

This was not just an assumption, the code had clear AI fingerprints.

Overly Explanatory Comments: The script had comments that explained what every function did, like a teacher walking through code. Human hackers don't write comments like that, but AI does.
A Hallucinated CVSS Score: The code had a CVSS score, that is, a security vulnerability rating, but the score didn't really exist, AI made it up.This was a hallucination, a common AI mistake.
Textbook Python Format: The code was perfectly clean without any hacker shortcuts or any optimizations. It was like a textbook Python from AI training data.
A Junk _C Color Class: The script used a basic ANSI color class named _C. It's a common pattern in AI-generated code. Google found this exact pattern in multiple AI scripts.

A human hacker wouldn't include these points. They're too neat, too educational, too obvious, which indicates AI does this.

Why AI Found This Flaw

Zero-day vulnerabilities require elite security talent. It actually needs someone who can read code, understand intent, and spot contradictions, which is hard.

Modern large language models (LLMs) have something called context reasoning. They can:

Read through thousands of lines of code
Understand what the developer was trying to do
Find contradictions between intent and implementation
Surface logic errors that look correct but are broken

The zero-day was a high-level logic flaw. Security scanners don’t catch these. They look for memory errors, syntax mistakes, known vulnerability patterns, but they don’t understand developer intent.

Google said frontier LLMs (the biggest AI models) are getting better at this. They can spot logic errors the way a senior security engineer would. But they do it faster, and they don’t get tired.

Why This Changes Everything

The scary part is, this will happen more.

Zero-Days are now scalable: Before AI, finding a zero-day was skilled work but now, AI can scan code, find logic flaws, and write exploits in hours.
2FA is not 100% safe: Two-factor authentication is the gold standard for security. It’s what you tell your employees to use. It’s what banking apps require to keep accounts safe. This exploit bypassed it.
AI can find flaws humans miss: Traditional security tools look for patterns, check for known vulnerabilities, but they don’t understand intent. AI understands intent, reads code like a human would and finds contradictions, that are the most dangerous vulnerabilities.

What Developers Should Do Right Now

You can’t stop AI from finding vulnerabilities, but you can make yourself a tougher target.

Update Your Packages Faster: When a vulnerability is announced, patch it as soon as you can. Attackers are moving faster with AI, so waiting weeks to update is a risk. If there’s a fix, apply it quickly.
Audit Your Dependencies: Check what libraries and tools you're using. Are they popular? Do they have security teams? Are they open-source? Audit your entire dependency tree. Each one is a potential attack vector.
Don't Trust AI Code 100%: If you use AI to write code, always review it. AI can get things wrong, add insecure code, or suggest solutions that don’t really work.
Get Ready for More AI-Powered Attacks: AI-driven attacks aren’t going away. Set up monitoring, watch for unusual activity, rotate passwords and keys regularly, and have a plan for handling security incidents.

Final thoughts

The AI-created zero-day is a turning point. It shows that AI is no longer just a future threat, it's already being used in real attacks.

2FA is still a strong layer of security, but it’s not perfect. Logic flaws are still dangerous, and now AI can help find them faster. Zero-days are still uncommon, but creating them is becoming easier.

This is just the beginning.

The next thing could be much bigger.

I Fine-Tuned Llama 3.2 on My Own Writing Style Using LoRA, Unsloth, and a Free Colab GPU

Shrestha Pandey — Sun, 07 Jun 2026 14:38:38 +0000

For a long time, fine-tuning language models felt like something that multiple people talked about, but very few actually tried themselves.

Whenever I saw posts about fine-tuning, I saw things like massive datasets, GPU setups, and much more, which made me realise it’s out of reach. But it got added to my list of things to explore.

Recently, while browsing Creator Labs on VickyBytes, I came across one of those labs which sounded simple yet practical to someone like me, who wants to explore. It was: taking a small language model and fine-tune it on your own writing style. The main goal here was, Could a model learn to write more like me?

As someone who spends a lot of time creating technical content, that question caught my attention.

I assumed fine-tuning still needed expensive hardware, large datasets, and a significant amount of machine learning knowledge. Instead, I tried to lower the barrier and discovered tools like using Google Colab notebook, LoRA, and a small dataset built from my own content.

This article documents the entire process from start to finish, including environment setup, dataset preparation, LoRA fine-tuning, evaluation, GGUF conversion, and the lessons I learned along the way.

What Does "Training on Your Writing Style" Actually Mean?

A language model doesn’t understand who you are. It learns the statistical patterns present in your writing.

For example:

Do you prefer short or long paragraphs?
Do you use analogies?
Do you use emojis in your writing?
Do you end posts with questions?
Do you write formally or conversationally?

When enough examples are provided, the model begins reproducing those patterns.

Thus, fine-tuning on writing style is less about teaching a model who you are and more about teaching it how you tend to communicate.

Architecture Overview

Choosing a Model

The lab suggested working with models between 1B and 7B parameters. Initially, I considered using one of the newer Qwen models. However, after exploring the available Unsloth notebooks, I decided to use:

Llama 3.2 3B Instruct

Three reasons for this decision were:

First, the model was small enough to fine-tune comfortably on free Colab resources. Second, Unsloth provides a mature training notebook for Llama models. Third, the resulting model can easily be exported to GGUF and run locally through Ollama.

At this point, I wanted a model that could actually learn from a relatively small dataset and allow me to complete the entire worlflow on consumer-grade hardware.

Setting Up the Environment

I wanted the entire project to work on free resources. So instead of renting a GPU or using paid cloud infrastructure, I used Google Colab because it provides access to a free NVIDIA T4 GPU, which is sufficient for LoRA fine-tuning small language models such as Llama 3.2 3B.

The first step was GPU acceleration.

From the Colab runtime settings, I selected a GPU runtime as T4 GPU.

Once the runtime was ready, I opened the official Unsloth notebook for Llama 3.2 and executed the setup cells.

Unsloth handles much of the optimization automatically, which means there is very little configuration required from the user.

Building and Preparing the Dataset

This was the most important step in the entire project.

Most of us think focus on discussing models instead of discussing more about the data. I experienced that building the dataset was more challenging as the result depends on the data we provide.

Since the goal was to teach the model my writing style, I built the dataset using content I had already written over time, including LinkedIn posts, Instagram captions, technical explanations, and educational content. I focused on examples that showed how I naturally write and explain technical concepts.

One challenge I encountered was preparing the data in the format expected by the training pipeline. So, I converted each example into an instruction-response pair.

Example:

{
  "instruction":"Write a LinkedIn post about Kubernetes",
  "response":"Kubernetes is one of those technologies..."
}

The notebook expected a JSONL dataset, where each example is stored as a separate JSON object. Initially, I spent some time on dataset loading errors before realizing the issue was with the structure of the data. To simplify, I stored the examples as a Python string inside the notebook and generated the dataset.jsonl file before loading it.

Once the dataset was formatted correctly, it loaded successfully and became the foundation for the rest of the fine-tuning process. It looked simple but it was one of the most important parts of the entire project because the model can only learn the patterns that exist in the data it receives.

Loading the Dataset

After correcting the JSONL structure, loading the dataset became simple.

from datasets import load_dataset

dataset = load_dataset(
"json",
data_files="dataset.jsonl",
split="train"
)

To verify everything loaded correctly:

print("Rows:",len(dataset))

Preparing the Dataset for Llama 3.2

The raw dataset still wasn’t ready for training because Llama expeects conversational data rather than instruction-response pairs.

To solve this, I used Unsloth’s chat template utilities.

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(

tokenizer,

chat_template="llama-3.1"

)

This converts the dataset into the same format used by Llama during instruction tuning.

A single training example now looks something like this:

<|start_header_id|>user
Write a LinkedIn post about Docker

<|start_header_id|>assistant
Docker revolutionized...

At this stage, the model finally has data in a format it understands.

Fine-Tuning with Unsloth

Once the dataset was completely ready, the training process became surprisingly easy.

Using Unsloth’s SFTrainer, I configured LoRA training and launched the run.

trainer.train()

During training, the model repeatedly sees examples from the dataset and adjusts the LoRA adapter weights to better reproduce the expected responses. The original Llama weights remain frozen throughout the process.

Did the Model Actually Learn My Writing Style?

Now, after training, the main question was whether the model learned anything useful. To test this, I intentionally used prompts that were not included in the dataset.

Examples included:

Prompt:
Explain LLMs in short

Generated Output:
Most AI today runs on a small change. Instead of building LLM itself, you give an AI model large amounts of text data and tell it to only remember things it decides on its own. This creates a system where the AI learns by responding to other people’s queries, and where it has to think for itself in order to provide an accurate answer. That’s a major pattern behind the AI today. Instead of fine-tuning a model that thinks like a dictionary and a research paper in one, you fine-tune a model that thinks like a person. And instead of giving it years of training data, you give it hundreds of thousands of your own pages that it will decide what to keep in memory on its own. That is the system most AI use today, and it has worked well for a lot of things.

The output were not identical to my writing. That would actually be a bad sign because it would suggest memorization. The generated posts were more conversational, and more focused on helping readers understand concepts rather than simply describing them.

That was the behavior I expected to achieve.

Avoiding Overfitting

One challenge with small datasets is overfitting.

With only twenty examples, there is always a risk that the model memorizes responses instead of learning general writing patterns.

To reduce that risk, I intentionally included multiple content formats and multiple technical topics. Using diverse topics helped push the model toward learning style patterns instead of topic-specific answers.

Exporting the Model to GGUF

After training completed, I expected to see a complete model. But LoRA only trains adapters. Those adapters must be merged with the original model before deployment.

The workflow looks like this:

Using Unsloth’s export utilities, I generated a GGUF version of the model using Q4_K_M quantization. This format is widely used by tools such as Ollama and llama.cpp.

Running the Model Locally with Ollama

One of the coolest part was realizing that the model no longer needed Colab. Once exported as GGUF, it could run locally.

Using Ollama, the fine-tuned model can be loaded directly from a laptop without relying on external APIs.

This means the writing-style model becomes self-hosted. The same personalized behavior learned during fine-tuning can now be accessed locally whenever needed.

Ethical Considerations

Since this project involved training on personal content, it’s worth discussing about the privacy. I only used the content that I personally wrote and was comfortable using for experimentation.

I avoided using private messages, confidential conversations, or any data that could create privacy concerns.

The ability to train models on personal data is powerful, but consent and security should always be considered before building such AI systems.

Lessons Learned

While working on this project, I learnt a few things.

The most challenging parts were:

Building a useful dataset
Correctly formatting JSONL files
Understanding chat templates
Understanding LoRA adapters
Evaluating whether the model genuinely learned anything

The actual training process ended up being the easiest step. Modern tools such as Unsloth have simplified fine-tuning workflows. So the actual bottleneck is data quality, not just the hardware.

Final Thoughts

Before starting this project, fine-tuning felt like something made only for machine learning engineers.

After completing it, I think every developer should try it once.

Something that surprised me was how strongly the dataset influenced the final results. Even with a small collection of examples, the model began reproducing many of the patterns that consistently appear in my writing.

And after spending several days building, debugging, training, evaluating, and exporting the model, I’m convinced that the quality of the examples matters a lot.

Would I use this model for real content creation? Probably not yet. The dataset is still too small, and the outputs occasionally differ from my writing style.

However, the experiment successfully demonstrated that modern fine-tuning tools have lowered the barrier significantly. Something that once felt like a machine learning research project can now be completed on a free Colab GPU over a week.

Resources

Fine-Tuning Framework: Unsloth
Model Notebooks: Unsloth Notebook
Running Models Locally: Ollama
GitHub Repository: Fine-tuning-LLM-to-write-in-your-own-style
Lab Inspiration: VickyBytes Creator Labs

I Built a GitHub Bot That Reviews My Pull Requests Using a Local LLM

Shrestha Pandey — Wed, 20 May 2026 09:46:21 +0000

Last month I pushed a bug to production which I would have caught if I had somebody look at my code. Well, it's not like I was the only one working on the project, so I had to read it myself and then forget about it, only to spend an annoying Tuesday trying to figure out why it didn't ship.

I was thinking about Ollama. I had heard about it a couple of times but never really needed to use it for anything. So this weekend I built a GitHub bot that reviews pull requests with a local LLM, without sending any code out of the box. It makes a comment on the PR with what it finds.

Here's how I built it.

What the bot actually does

When you open a pull request, GitHub fires a webhook. The bot receives it, pulls the diff for each changed file using GitHub's REST API, sends that diff to a locally running LLM via Ollama, and posts the review back as a comment on the PR.

The output looks like this on your PR:

Project Structure:

self-hosted-ai-code-review-bot-for-github-prs/
├── src/
│   ├── server.js
│   ├── github.js
│   ├── ollama.js
│   ├── diffParser.js
│   └── chunker.js
├── private-key.pem
├── .env
├── package-lock.json
└── package.json

Setting up the GitHub App first

Before any code, you need a GitHub App. A GitHub App has its own identity, which gets installed on specific repos, and uses short-lived installation tokens instead of long-lived credentials.

Go to GitHub → Settings → Developer Settings → GitHub Apps → New GitHub App.

Permissions you need:

Pull requests: Read & write
Issues: Read & write
Metadata: Read-only

Under webhook events, subscribe to Pull request. Set the webhook URL to your ngrok address + /webhook (we'll come back to that).
Generate and download the private key (.pem file). GitHub uses this to sign the installation tokens.

Clone the repo and install dependencies:

git clone <your-repo-url>
cd <repo-name>
npm install

Your .env:

PORT=3000
GITHUB_APP_ID=your_app_id
GITHUB_PRIVATE_KEY_PATH=./privatekey.pem
WEBHOOK_SECRET=your_webhook_secret
OLLAMA_MODEL=deepseek-coder # or llama3, whichever you're running

Then install the app on your repository via the Install App option.

github.js — auth and API calls

GitHub Apps don't use static tokens. You sign a JWT with your private key, exchange it for a short-lived installation token, and use that to make API calls. @octokit/auth-app handles all of this:

The auth layer

GitHub Apps use installation tokens, not static keys. You sign a JWT with your private key, exchange it for a short-lived token, use that token for API calls. @octokit/auth-app handles the whole flow:

export async function getOctokit(installationId) {
  const privateKey = fs.readFileSync(
    process.env.GITHUB_PRIVATE_KEY_PATH,
    "utf8"
  );

  const auth = createAppAuth({
    appId: process.env.GITHUB_APP_ID,
    privateKey,
    installationId
  });

  const installationAuth = await auth({ type: "installation" });

  return new Octokit({ auth: installationAuth.token });
}

You pass the installationId from the webhook payload. GitHub includes it in every event so you always know which installation triggered it.
The other two functions in github.js are straightforward: one calls octokit.pulls.listFiles to get the changed files with their diffs, the other calls octokit.issues.createComment to post the final review.

Filtering and chunking the diff

Not every file in a PR deserves review time. package-lock.json, .vscode/ settings, minified files, we skip them. GitHub also sometimes returns files without a patch field (binary files, files too large to diff), those get skipped too.

For files that do make it through, large diffs need to be split before sending to the LLM. Context windows are limited and model quality drops toward the end of long inputs:

export function chunkDiff(diff, maxSize = 1500) {
  const chunks = [];
  if (!diff) return chunks;

  let current = "";
  const lines = diff.split("\n");

  for (const line of lines) {
    if ((current + line).length > maxSize) {
      chunks.push(current);
      current = "";
    }
    current += line + "\n";
  }

  if (current) chunks.push(current);
  return chunks;
}

1500 characters per chunk works reliably with both Llama 3 and DeepSeek Coder. Each chunk gets its own Ollama call, results get concatenated per file.

The prompt is important

This is where I spent the most time, and where the whole thing either works or doesn't.

My first version was something unclear like "review this diff and point out issues." The output was not good. The model would write paragraphs explaining what the file does, invent security vulnerabilities that weren't in the code, reference issue numbers that don't exist, and end with generic advice like "make sure to add proper error handling throughout your application."

The problem is that without constraints, the model tries to be helpful in every direction at once. It doesn't know you only care about what changed. So you have to be as explicit about what you don't want as what you do:

export async function reviewWithOllama(diffChunk) {
  const prompt = `
You are an AI GitHub pull request reviewer.
You are reviewing ONLY the provided git diff.

STRICT RULES:
- Review ONLY changed lines from the diff.
- Ignore unchanged code completely.
- Do NOT explain the whole application.
- Do NOT give generic software engineering advice.
- Do NOT hallucinate missing features.
- Do NOT invent security issues.
- Do NOT invent issue IDs, ticket numbers, PR references, or metadata.
- Do NOT mention issues outside the diff.
- Every issue MUST directly relate to a changed line.
- Keep response concise.
- If no real issue exists, reply exactly: "No significant issues found."

Focus ONLY on:
1. Potential bugs introduced
2. Style or readability issues introduced
3. Performance issues introduced
4. Security risks directly introduced
5. Small improvement suggestions directly related to the diff

Diff:
${diffChunk}
`;

  const response = await fetch("http://localhost:11434/api/generate", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: process.env.OLLAMA_MODEL,
      prompt,
      stream: false,
      options: { temperature: 0 }
    })
  });

  const data = await response.json();
  return data.response;
}

Once I added the negative rules, the output became useful. The model stopped inventing context and started pointing at specific changed lines.

temperature: 0 — code review isn't creative work. You want the model to be deterministic, same diff producing consistent output. It also reduces hallucinations noticeably.

The webhook handler

Express receives the event, checks if it's worth handling, and orchestrates everything:

app.post("/webhook", async (req, res) => {
  try {
    const event = req.headers["x-github-event"];
    const action = req.body.action;

    if (event !== "pull_request") return res.status(200).send("Ignored");

    if (!["opened", "synchronize", "reopened"].includes(action)) {
      return res.status(200).send("Ignored action");
    }

    const installationId = req.body.installation.id;
    const owner = req.body.repository.owner.login;
    const repo = req.body.repository.name;
    const pullNumber = req.body.pull_request.number;

    const octokit = await getOctokit(installationId);
    const files = await getPullRequestFiles(octokit, owner, repo, pullNumber);

    let finalReview = `# 🤖 AI Code Review\n\n`;

    for (const file of files) {
      if (!shouldReviewFile(file)) continue;

      const chunks = chunkDiff(file.patch);
      let fileReview = "";

      for (const chunk of chunks) {
        fileReview += await reviewWithOllama(chunk) + "\n\n";
      }

      finalReview += `## File: ${file.filename}\n\n${fileReview}---\n\n`;
    }

    await postReviewComment(octokit, owner, repo, pullNumber, finalReview);
    res.status(200).send("OK");
  } catch (error) {
    console.error("Webhook error:", error);
    res.status(500).send("Error processing webhook");
  }
});

Running it

Install Ollama from ollama.com and pull a model:

ollama run llama3
# or, better for code
ollama run deepseek-coder

Start the server and expose it with ngrok:

npm run dev
ngrok http 3000

Copy the ngrok HTTPS URL, update your GitHub App's webhook URL to https://your-ngrok-url/webhook, push a branch, open a PR.

CPU vs GPU: Depending on hardware, local inference on CPU may take several seconds per diff chunk. Llama 3 8B on a laptop CPU takes 10–20 seconds per diff chunk. For a background bot that's fine, you open the PR, switch tabs, comment shows up a minute later. With an NVIDIA GPU, Ollama picks it up automatically and it's under 2 seconds.

Deploying it properly

ngrok dies when you close your terminal. For anything persistent, a cheap VPS works. DigitalOcean's $6/month droplet, small EC2 instance, whatever. Install Node.js and Ollama on it, put Nginx in front with SSL, point your webhook URL at the domain.

Tools like GitHub Copilot Code Review send your diff to external servers. With this setup the diff goes from GitHub's API to your own machine into your local model which means nothing leaves your infrastructure. For proprietary codebases that's not a minor thing.

Where the idea came from

I came across VickyBytes a few weeks back. It's a platform that posts structured project ideas in Creator Labs, where you build something, write about it, and earn from it if the content is good enough. This was one of those.
Having a concrete spec helped. "Play with Ollama" had been on my list for months and never happened. A specific project with a clear outcome actually got it done. The bot runs on my own repos now, so that worked out.
If you write technical content and want structured project ideas to build around, visit vickybytes.com.

Full source is on GitHub. The prompt is where most of the interesting experimentation happens. That's where you'll actually learn how local LLMs behave in practice.

Why Reasoning Models Changed Everything

Shrestha Pandey — Fri, 10 Apr 2026 14:38:33 +0000

For years, making language models smarter meant making them bigger. Then someone asked a different question: what if, instead of training more, you let the model think longer?

In September 2024, OpenAI released o1. In January 2025, DeepSeek released R1. These two models together, invalidated the assumption that was governing the entire field since 2020, that the path to better AI runs through bigger training runs.

This assumption was backed by the Kaplan et al. scaling laws paper, which showed the language model performance follows a reliable power law with training compute. If you provide more parameters, more data, more GPU-hours, you get a more capable model. The field organized itself around this insight. Every major lab poured billions into pre-training.

o1 and R1 showed that there’s a second dimension to scale that the field had largely ignored: compute at inference time. And it turns out that for tasks requiring multi-step reasoning, this second dimension can be just as powerful as the first, and far cheaper to exploit.

What chain-of-thought actually is

Chain-of-thought prompting has been around since 2022, when Wei et al. at Google Brain showed that simply asking a language model to “think step by step” before answering dramatically improved its performance on math and logic tasks. This was a really surprising result. The model wasn’t being retrained, it was just prompted differently. The extra tokens the model generated while reasoning served as scratchpad that improved its final answer.

Transformers generate one token at a time, and each token is conditioned on all previous tokens. When a model solves a math problem by writing out steps, those steps become part of the context that informs the final answer. The model is using its own output as working memory. Without chain-of-thought, it has to compress all that computation into a single forward pass.

Nobody had figured out how to train a model to do this reliably, until o1. Prompting a model to “think step by step” helps, but the quality of reasoning is inconsistent. You might get careful and structured reasoning sometimes, but sometimes you get verbose filler that doesn’t really help. The model doesn’t know when to think hard and when not to.

The reinforcement learning connection

OpenAI’s o1 system card describes training the model with reinforcement learning to produce a chain of thought before answering. The core thing is that RL can teach the model how to think specifically, to develop reasoning strategies that lead to correct answers on verifiable tasks like mathematics and code.

According to OpenAI’s published description, o1 learns through RL to recognize and correct its mistakes in the middle of reasoning, break hard problems into simpler subproblems, and abandon approaches that aren’t working. These behaviors like self-correction, decomposition, backtracking, emerge from the training signal.

The result is a model where more thinking time produces better answers. On the AIME 2024 benchmark (American Invitational Mathematics Examination), o1 scored in the 74th percentile of human test-takers. GPT-4o, using the same benchmark, scored around 9%. That gap is from the model being trained to use its inference compute more productively.

DeepSeek R1: showing the mechanism

DeepSeek R1 published in January 2025 was an extremely important advancement because it achieved on benchmark scores the same level of performance as o1; however, this is the easy part. The significance of DeepSeek was that it provided an in-depth description of the training recipe allowing other people in the field the ability to study and replicate it.

They started with DeepSeek-V3-Base, a 671B parameter pretrained model built on a Mixture-of-Experts architecture. Their first experiment called DeepSeek-R1-Zero, applied reinforcement learning directly to this base model without supervised fine-tuning beforehand. The reward signal was such that, the model gets rewarded for producing a correct final answer, and for formatting its output with explicit reasoning inside <think> tags.

No direct human-labeled examples were used to provide the basis for generating a reward signal for successful attempts. Similarly, no preference-based human reward model was utilized for training and reward purposes for deep learning via reinforcement learning. Simply put, their reinforcement learning experiment used the following evaluation criteria: (1) Did you provide the correct final answer?

The results of R1-Zero were remarkable and, to be honest, a little unsettling. The model's average pass@1 score on AIME 2024 increased from 15.6% at the start of training to 71.0% by the end. More striking was what happened to the model's behavior during this process. The reasoning traces grew substantially longer as training progressed.

The model spontaneously developed strategies like re-reading the problem from the beginning partway through a solution, checking its own work, and explicitly noting when it suspected an error. None of this was designed in. It emerged from optimizing for correct answers.

The paper describes a notable emergent behavior where the model, while solving a math problem, pauses mid-reasoning, re-evaluates its approach, and switches to a different strategy to reach the correct answer. This emerges from reward training, where the model learns that revisiting its reasoning can sometimes lead to better outcomes.

The training algorithm: GRPO

The RL algorithm that DeepSeek used (Group Relative Policy Optimization (GRPO)), is worth understanding, because it's part of why this approach is tractable at scale.

Standard reinforcement learning for language models (the approach used in earlier RLHF pipelines) depends on Proximal Policy Optimization (PPO). This requires an additional network called a critic, which is basically another large neural network that can estimate the value of partially completed responses. The size of the critic is typically the same as the size of the model that is being trained, resulting in twice the memory/compute requirements of the PPO-based reinforcement learning algorithm, because you must perform two full forward passes for every training step.

GRPO, first introduced in the DeepSeekMath paper (2024), eliminates the critic. With GRPO, instead of estimating value through a learned value estimation model, GRPO now generates multiple responses for each prompt and receives a reward for each response. This group of multiple responses will provide an average reward, or baseline, for comparison when determining the advantage of any response (i.e., the signal that tells the model whether that particular response was better or worse than what was expected) through the formula:

A_i = (r_i - mean(r_1...r_G)) / std(r_1...r_G)

This is similar to the REINFORCE algorithm from the 90’s, however it is being created on larger scales with modern hardware and still holds the same clipping mechanism found within PPO for training stability. Because of this, you would see how using 50% fewer resources is huge, especially given that your policy model contains 671 billion parameters.

There were only two main types of rewards that were given in the reward function: correctness rewards (whether or not the final answer matched the ground truth for math problems and/or coding problems) and format rewards (whether or not the model used the structure expected for output).

There were no process rewards, nor step-by-step supervision. The implicit assumption of this approach was that if you give the model a correct-answer signal, along with a good deal of freedom to explore, the model will be able to find its own way to good reasoning.

Why R1-Zero wasn't the final model

R1-Zero had a problem because DeepSeek-V3-Base was pretrained on multilingual data, the model sometimes switched languages mid-reasoning. It would start a problem in English, shift into Chinese for a few sentences then continue in English. The reasoning was often correct but the output was unreadable. It also had readability issues. The thinking traces were sometimes clear internal monologues, but at other times, they were nearly incoherent.

DeepSeek R1 addressed this through a more involved training pipeline. They first collected a small number of high-quality chain-of-thought examples that demonstrated the kind of structured and readable reasoning they wanted. This was used for supervised fine-tuning before RL began, giving the RL process a better starting point. They also added a language consistency reward to penalize mid-reasoning language switching. A subsequent round of rejection sampling and SFT on the RL model's own outputs added coverage for non-reasoning tasks like writing and general question-answering. The result DeepSeek-R1, performs comparably to OpenAI o1-1217 on reasoning benchmarks.

DeepSeek released smaller models like 7B, 14B, 32B parameters, trained by fine-tuning on reasoning traces generated by the full R1 model. The 32B distilled version outperforms o1-mini on several benchmarks. This is knowledge distillation applied to reasoning i.e, smaller models learn reasoning patterns from larger models, without running the expensive RL training themselves.

Scaling just got a second dimension

The pre-training scaling laws described by Kaplan et al. in 2020 showed a clean relationship, i.e, training compute in, model capability out. The Chinchilla paper (Hoffmann et al., 2022) refined this further, showing that for a fixed compute budget, the optimal strategy is to train a smaller model on more data rather than a larger model on less data. These results organized the field for years.

Reasoning models introduce a second scaling curve. OpenAI’s o1 blog says performance improves when the model gets more time to think.

In “The Bitter Lesson” (2019), Richard Sutton makes the case that methods of making use of computing power in a general sense usually outperform in the long term methods that encode human knowledge explicitly. The first instance of this was the scaling of the amount of computational resources utilized for pre-training; the second instance is the scaling of the number of computational resources utilized for inference through the use of learned reasoning. This indicates that we are not done with increasing the scale of AI; in actuality, we have only just entered into a new chapter in that story.

That being said, scaling with regards to inference does work best for tasks that can have verified answers (i.e., math, formal proofs, code that can be tested). This makes it rather easy to determine whether the answer produced by a model is correct or not (there is either an answer or there is not).

Extending this to tasks that are open ended (e.g., writing) where there is no clear ground truth is a very active area of research and far more difficult. The current generation of reasoning models is likely very powerful in the STEM and coding fields, although exactly how broadly they will be able to extend beyond these fields remains an active area of research as well.

This one is different

There is often a claim every few years that AI has reached a major breakthrough, but it usually turns out to be a small improvement. But this time is different. Now models can use more compute during inference to produce better reasoning, which creates new possibilities that were not available before.

Tasks that previously needed human intervention because models can’t be trusted to reason carefully, now work differently. You can spend more compute at inference time to get more reliable results. For simple tasks the model can run fast, while harder or more important problems can be given more time to think, making the tool much more flexible than models from a few years ago.

DeepSeek R1 being released with open weights under an MIT license also changes the economics. The distilled R1-32B model, when used with quantization on a single high-end GPU, performs better than o1-mini on numerical tests. Now that there are multiple APIs, labs, researchers, and smaller organizations with limited resources can take advantage of them. This is a major change in the number of people who would have had access to this type of model before the DeepSeek R1 was made freely available.

Making models bigger and training them on more data helps them learn more, like reading lots of books makes someone more knowledgeable. But letting models “think longer” when answering questions helps them reason better and make fewer mistakes, like a person taking more time to work through a problem.

Both of these improvements are important and now we have both. What we do with that combination is still an open question.

References

DeepSeek-R1 Paper — DeepSeek-AI, 2025
Learning to Reason with LLMs — OpenAI, 2024
DeepSeekMath / GRPO Paper — Shao et al., 2024

For more such developer content, visit:
https://vickybytes.com

Why Terraform Breaks After Day-1 And How Terraform Actions Fix It

Shrestha Pandey — Wed, 01 Apr 2026 08:00:42 +0000

Let me start with something most infrastructure engineers might not say out loud — Terraform solves Day-1 beautifully and then kinda leaves you hanging.

You write your HCL, run terraform apply, and everything is provisioned perfectly. The state file appears impeccable. But six months later that same infrastructure has been poked, patched, manually changed and silently drifted away from what terraform thinks exists. No one realizes this until something breaks in production.

This article is about that “gap” between provisioning and actually managing infrastructure across its entire lifetime.

Day-2 Is Where Infrastructure Goes to Die (Slowly)

When a full stack is provisioned onto AWS using Terraform it has a good state and everything is the same, and then after some time passes and a deployment fails, someone logs into the console and changes a security group rule; now the deployment has been successful… but this change has not been documented and no tickets have been raised regarding this change.

When they run the scheduled terraform apply, Terraform sees the difference and resets the security group to the original state, resulting in production breaking. Everyone is confused because there were no code changes made.

The root cause of this issue is that the tools have not been designed for such usage; Terraform's core capability was to provide an infrastructure provisioning capability.

Therefore, what are teams doing for their Day-2 operations? Most have a combination of:

Bash scripts that contain parts nobody understands
AWS Console changes that are made manually and never documented
Ad-hoc Ansible runs that don't tie back to Terraform state in any way
Lambda functions that are each triggering another Lambda function creating a non-traceable chain

In total, over 30 different tools are managing a single hybrid infrastructure estate, which is being actively managed by organizations in the field.

The Lifecycle Nobody Talks About Enough

Infrastructure has four phases and most of the industry focuses heavily over two of them.

The first phase, or "Day-0", is the "Build Phase." In this phase an organisation will form their infrastructure and define policies. There has not been any provisioning yet and is done in partnership with the platform and security teams.

The second phase, or "Day-1", is "Deploy Phase." In this phase terraform apply will run, infrastructure will be built, and the application teams will deploy their workloads. This is where terraform really starts to show its capabilities.

Day-2 or "Manage Phase." This phase is where management happens, patches are installed, configurations are changed, certificates are renewed and scaled as needed and where compliance is checked for validity and accuracy. Day 2 can take years to complete and it is also were all of the operational pain will occur. Terraform traditionally has no place in this phase.

Day-N "Decommission Phase." This phase is where everything is removed and cleaned up.

Over the last ten years the DevOps industry has been focused on perfecting Day-1 tooling; however, there are very few tools available for Day-2.

Terraform Actions — What Changed in v1.14

Terraform Actions were added as stable functionality in Terraform v1.14 and were unveiled at HashiConf 2025. Now, providers can execute an action that does more than just CRUD - calling a lambda function, stopping an EC2, invalidating a CloudFront cache, or triggering an Ansible playbook.

These new actions are located in their own top-level action block in your HCL. Terraform can automatically execute them based on event triggers during a resource's lifecycle, or they can be invoked manually via the CLI without the need to do a complete terraform apply.

You can invoke an operational action (such as calling a lambda to warm up a cache) without having Terraform re-evaluate the entire state of your infrastructure. This is a significant change in how you will use your infrastructure from now on.

The AWS provider currently has:

aws_lambda_invoke
aws_ec2_stop_instance
aws_cloudfront_create_invalidation

How Actions Actually Work — The Syntax

There are two pieces. The action block itself, and the trigger that fires it.

Defining an Action

action "aws_lambda_invoke" "warm_cache" {
  config {
    function_name = aws_lambda_function.cache_warmer.function_name
    payload = jsonencode({
      source = "terraform_action"
    })
  }
}

Note the config {} wrapper. Provider-specific arguments go inside config, not directly in the action block.

Meta-arguments like count and provider exist outside config:

action "aws_lambda_invoke" "warm_cache" {
  count    = var.invoke_on_deploy ? 1 : 0
  provider = aws.us_east_1
  config {
    function_name = aws_lambda_function.cache_warmer.function_name
    payload       = jsonencode({ source = "terraform_action" })
  }
}

Triggering an Action on Resource Lifecycle Events

This goes inside the resource's lifecycle block:

resource "aws_lambda_function" "api" {
  function_name = "my-api-handler"
  # ... rest of config

  lifecycle {
    action_trigger {
      events  = [after_create, after_update]
      actions = [action.aws_lambda_invoke.warm_cache]
    }
  }
}

Two main things to understand:

events uses unquoted keywords — after_create and after_update
actions is plural and takes a list, not a single reference

You can also add a condition to guard the action:

lifecycle {
  action_trigger {
    events    = [after_create]
    actions   = [action.ansible_playbook.patch_instance]
    condition = var.enable_auto_patching
  }
}

When condition is false, the action is skipped completely. This is useful when the configuration should exist but only run in certain environments, like production.

Running Actions from the CLI

This is where it gets useful for Day-2 workflows:

# Just plan the action, don't run it
terraform plan -invoke=action.aws_lambda_invoke.warm_cache

# Actually run the action
terraform apply -invoke=action.aws_lambda_invoke.warm_cache

Terraform only executes that one action. No evaluation or change of any other part of your configuration occurs. Each action can only be executed once at a time; therefore, multiple -invoke cannot be run in a single command.

Provisioning EC2 + Immediate Patching via Ansible Automation Platform

One of the most important and widely used use cases is linking EC2 provisioning and automated patching through Ansible Automation Platform (AAP).

The challenge it solves is simple; there are usually many security patches pending for an Ubuntu AMI that has been provisioned several months prior. If EC2 instances are provisioned, and then you manually take the time to patch each one independently, then at some point (most likely within 30 days) you will not patch an instance you provisioned. Thus, the solution is to link the patching process to the lifecycle of the Terraform instance provisioning so that patching cannot be missed.

The Terraform Side

variable "instance_count" {
  type    = number
  default = 2
}

variable "ubuntu_ami" {
  type        = string
  description = "AMI ID — use a recent Ubuntu LTS, patching will handle the rest"
}

variable "aap_controller_url" {
  type      = string
  sensitive = true
}

variable "aap_oauth_token" {
  type      = string
  sensitive = true
}

variable "allow_instance_reboot" {
  type    = bool
  default = false
}

resource "aws_instance" "app_servers" {
  count         = var.instance_count
  ami           = var.ubuntu_ami
  instance_type = "t3.medium"
  subnet_id     = aws_subnet.public.id
  key_name      = aws_key_pair.deployer.key_name

  vpc_security_group_ids = [aws_security_group.allow_ssh.id]

  tags = {
    Name      = "app-server-${count.index}"
    ManagedBy = "terraform"
  }

  lifecycle {
    action_trigger {
      events  = [after_create, after_update]
      actions = [action.ansible_aap_job.patch_servers]
    }
  }
}

The after_update event is critical; should an instance be replaced (due to AMI update, instance type modification, or any reason that would force a new instance to be created), the patching will occur on the newly-created instance automatically without any manual intervention required.

The Action Block

action "ansible_aap_job" "patch_servers" {
  config {
    controller_url    = var.aap_controller_url
    oauth_token       = var.aap_oauth_token
    job_template_name = "EC2 Linux Patching"
    extra_vars = jsonencode({
      vm_hosts = [
        for instance in aws_instance.app_servers : {
          instance_id = instance.id
          public_ip   = instance.public_ip
        }
      ]
      allow_reboot = var.allow_instance_reboot
    })
  }
}

Credentials are stored in HCP Terraform's sensitive variable store. Instance IDs and IPs come straight from resource state at runtime, so AAP always gets current values.

Note: You must refer to your provider documentation to verify the argument names of your AAP action based upon the version you are using, while the structure remains valid.

The Ansible Playbook

AAP receives vm_hosts as an extra variable, builds inventory dynamically, and patches:

---
- name: Patch EC2 Instances
  hosts: all
  gather_facts: yes
  become: yes

  pre_tasks:
    - name: Wait for SSH connectivity
      ansible.builtin.wait_for_connection:
        timeout: 120
        delay: 10

    - name: Gather package facts
      ansible.builtin.package_facts:
        manager: apt

  tasks:
    - name: Update apt package index
      ansible.builtin.apt:
        update_cache: yes
        cache_valid_time: 3600

    - name: Apply security patches
      ansible.builtin.apt:
        upgrade: dist
        only_upgrade: yes
      register: patch_result

    - name: Check if reboot is required
      ansible.builtin.stat:
        path: /var/run/reboot-required
      register: reboot_required_file

    - name: Reboot if needed and allowed
      ansible.builtin.reboot:
        reboot_timeout: 300
        post_reboot_delay: 30
      when:
        - reboot_required_file.stat.exists
        - allow_reboot | default(false) | bool

  post_tasks:
    - name: Verify instance is up after patching
      ansible.builtin.ping:

/var/run/reboot-required is a file Ubuntu creates automatically when a package update (typically a kernel patch) requires a restart to take effect. The playbook checks for this file rather than blindly rebooting. And even then, it only reboots if allow_reboot is true, which is controlled from your Terraform variables.

AAP Job Template Configuration

With respect to Ansible Automation Platform:

Project is a reference to the Git repo containing your playbook.
Inventory is created dynamically from the vm_hosts value which is assigned to you by Terraform when it runs.
Credentials - Your SSH Private RSA key will be stored in AAP's credential vault and will be used to connect to the VM via SSH. This is a very secure separation of the enterprise applications and their configurations (Terraform) and how to connect to them (Ansible).

What the Full Workflow Looks Like

An engineer modifies the instance_count from 2 to 5 and then sends a Git commit with the modification.

The engineer pushes it to HCP Terraform. HCP Terraform recognizes the alteration made and initiates a plan, which indicates that Terraform will create 3 new AWS EC2 instances; and that Terraform will submit an action request once the provisioned instances are created.

After an engineer reviews and approves the proposed plan, Terraform executes the apply phase, creating three EC2 instances within AWS. At this time, the action_trigger will be invoked. Terraform calls AAP's API, sending the newly created instance IDs and public IP addresses, to activate the patching job.

The AAP will make a dynamic inventory through Terraform and subsequently will wait until all three instances can be accessed through SSH. Once that occurs, AAP will execute an apt dist-upgrade command to identify whether a reboot is required, and then reboot the instance, if allowed. Finally, AAP will, upon each instance coming back online & responding normally, send a report back to Terraform.

Upon the completion of the reports from AAP, Terraform will acknowledge completion of the run.

Other Places Actions Are Immediately Useful

CloudFront invalidation after S3 deployments

action "aws_cloudfront_create_invalidation" "bust_cache" {
  config {
    distribution_id = aws_cloudfront_distribution.website.id
    paths           = ["/*"]
  }
}

resource "aws_s3_object" "site_bundle" {
  lifecycle {
    action_trigger {
      events  = [after_update]
      actions = [action.aws_cloudfront_create_invalidation.bust_cache]
    }
  }
}

Lambda warm-up after deployments: After deployments, cold starts on the first production request are common sources of failures in Lambda functions. First, there may be cold starts on the first production request after deployments, which is typical of failures; hence, the function is invoked immediately after deployment to ensure users do not encounter any faults.

action "aws_lambda_invoke" "warm_up" {
  config {
    function_name = aws_lambda_function.api_handler.function_name
    payload       = jsonencode({ source = "warmup" })
  }
}

resource "aws_lambda_function" "api_handler" {
  lifecycle {
    action_trigger {
      events  = [after_create, after_update]
      actions = [action.aws_lambda_invoke.warm_up]
    }
  }
}

Stopping dev instances on demand

action "aws_ec2_stop_instance" "stop_dev" {
  config {
    instance_id = aws_instance.dev_server.id
  }
}

terraform apply -invoke=action.aws_ec2_stop_instance.stop_dev

Chaining multiple actions — actions is a list, order is respected, each one completes before the next starts:

lifecycle {
  action_trigger {
    events = [after_create]
    actions = [
      action.ansible_aap_job.patch_servers,
      action.aws_lambda_invoke.register_in_cmdb,
      action.aws_lambda_invoke.notify_slack
    ]
  }
}

Things That Will Catch You Out

An action that fails prevents a run's end:

By default, how long Terraform waits for an action to finish is determined by the status of the action being waited on. This allows visibility into the status of actions, but introduces potential for issues if AAP has gone down just prior to a critical deployment, preventing Terraform from being able to wait for AAP to complete. Use condition guards for actions that have minimal impact on the overall deployment if they're interrupted.

Idempotency is not a luxury:

Every time there is a resource change, the after_update event fires. That means that your playbooks and lambda handlers will be invoked multiple times over the lifecycle of your infrastructure. It is acceptable to run apt dist-upgrade multiple times but not acceptable to perform a database migration multiple times. You must design your programming for re-execution from the very beginning of the process.

Actions do not write to state:

When an action is executed there is no record of its execution being written to a statefile in terraform. The only way to tell that the action was executed is from the run history in HCP Terraform and in the logs for any other systems that were involved i.e., AAP job history, cloud watch, etc... You have to plan how you will be able to see/understand when your Terraform works based on its observability functionality.

The provider support continues to grow:

v1.14 of the AWS Provider supports a narrow set of operations (action types). Always refer to the Terraform Registry and Provider changelogs prior to assuming any operation is an action.

CLI invocation requires existing resources:

If your action references instance.id, but the instance doesn't exist in state, the -invoke option will fail during the plan phase. Actions that use CLI should reference existing resources that have already been provisioned.

The Actual Shift

Almost all infrastructure management involves Day-2 operations.

In the past, Day-2 operations were recorded in runbooks, as part of a Jenkins job that only a few people understood, or as a bash script that was modified years ago. Day-2 operations were delivered in a reactive manner, which means if something breaks, somebody performs an action.

With Terraform Actions, Day-2 operations can now be housed with the infrastructure they manage, from the same repository and pull request workflow and using the same audit trails as the infrastructure they are provisioned on. Patch management will be defined in terms of the infrastructure your organization will provision.

This kind of change reduces the number of incidents occurring at 2:00 AM.

Terraform Actions is stable from Terraform CLI v1.14.0. Check developer.hashicorp.com/terraform/language/invoke-actions for official documentation and your provider's registry page for supported action types.

Technical insights sourced from a community session on Terraform Day-2 operations.

For more developer content, visit vickybytes.com