DEV Community: Kopalachandran Abinash

How AI Applications Answer From Your Data, Not Their Training

Kopalachandran Abinash — Sat, 06 Jun 2026 06:26:01 +0000

Why retrieval-augmented generation has become the foundational pattern for building useful AI — and how it actually works.

The Problem With Relying on LLMs Alone

Large language models are impressive. They can write, reason, summarize, and explain across an enormous range of topics. But they have a hard boundary: their knowledge stops at their training cutoff. Anything that happened after that date, anything specific to your company, your codebase, or your documents — the model simply doesn't know it.

The naive solution is to paste your data directly into the prompt. For short content, this works. But prompts have limits. A model can only process so much text at once, and even within that limit, quality degrades when you stuff too much context in. The model loses track of things buried in the middle, confuses similar passages, and starts guessing when it should be reading.

RAG — Retrieval-Augmented Generation — solves this properly. Instead of sending everything to the model and hoping for the best, you send only what's actually relevant to the question being asked.

The Core Idea

The analogy that makes RAG click immediately: imagine a student sitting an open-book exam. They don't memorize the entire textbook. When they see a question, they flip to the right chapter, read the relevant section, and write their answer from what they just read. They're not guessing. They're grounding their answer in the source material.

RAG does exactly this. When a user asks a question, the system finds the most relevant pieces of information from your data, hands those pieces to the LLM as context, and the model answers from that context alone. The result is accurate, grounded, and verifiable — you can point to exactly which source the answer came from.

The process runs in two phases: ingestion, which prepares your data in advance, and retrieval, which happens at query time.

Phase One: Ingestion

Ingestion is the preparation step. Before any user asks anything, you process your data and store it in a way that makes future retrieval fast and accurate.

The first step is chunking — splitting your documents into smaller pieces. This seems simple but it's actually the most important decision in the entire pipeline. The quality of your chunking determines the quality of everything that follows.

The naive approach is splitting by a fixed character count — every 1,000 characters becomes a chunk. The problem is that ideas don't stop at character boundaries. A sentence that starts in one chunk and finishes in the next loses meaning in both halves. A definition that spans two chunks can't be found by searching for either half alone.

A better approach is to split on natural boundaries — paragraph breaks, section headers, or topic shifts — and add a small overlap between consecutive chunks. The overlap, typically 150–200 characters, means a sentence sitting at a boundary appears in both adjacent chunks. Nothing falls through the cracks.

Once you have your chunks, each one gets converted into a vector embedding. This is where the real power of RAG lives.

Understanding Embeddings

An embedding is a way of representing text as a list of numbers — typically hundreds or thousands of them. These numbers aren't arbitrary. They're produced by a model specifically trained to place semantically similar text close together in this numerical space.

The practical result is striking. "How do I cancel my subscription?" and "What is the process for terminating my account?" will produce embeddings that are numerically close to each other, despite sharing almost no words. Meanwhile, a chunk about shipping logistics will sit far away from both.

This property enables semantic search — searching by meaning, not by keyword matching. Traditional search fails when the user's words don't match the document's words. Semantic search doesn't have that problem. It finds what the user means, not just what they typed.

The Vector Database

Your embeddings need somewhere to live. A vector database stores them and, crucially, supports fast similarity search at scale. Given a query vector, it finds the most similar vectors in the collection — often across millions of entries — in milliseconds.

Popular options include Pinecone, Weaviate, Qdrant, and pgvector for teams already using PostgreSQL. The right choice depends on scale and infrastructure preferences, but they all support the same core operation: nearest-neighbor search in high-dimensional vector space.

One detail that matters in real applications: always filter your search by the relevant data source. In a multi-tenant application, you want to search within a specific user's documents, not across every document in the system. Vector databases support metadata filtering alongside similarity search, so you can combine both constraints in a single query.

Phase Two: Retrieval and Generation

When a user submits a question, the retrieval phase runs in three steps.

First, the question gets embedded using the same model used during ingestion. This produces a vector that represents the meaning of the question.

Second, the vector database finds the top matching chunks — typically the five most similar — using cosine similarity between the question vector and the stored chunk vectors.

Third, those chunks get assembled into a context block and passed to the LLM alongside the original question. The prompt explicitly instructs the model to answer only from the provided context. If the answer isn't there, the model should say so rather than guess.

That last instruction matters more than it might seem. LLMs, when they don't know something, have a tendency to fill the gap with a confident-sounding fabrication. Grounding them in retrieved context dramatically reduces this. The model has specific, relevant material to work from, and you've told it to stick to it.

The Similarity Threshold

Not every question will have a good answer in your data, and your system needs to handle that gracefully.

Vector similarity is scored between 0 and 1, where 1 means identical. Even when there's no relevant content at all, the search still returns results — they just have low scores. If you send those low-quality chunks to the LLM anyway, it will do its best with them and may produce something that sounds plausible but has no real grounding.

The fix is a minimum similarity threshold. If the best match falls below a certain score — 0.7 is a common starting point — skip the LLM entirely and return an honest "this information isn't in the available data." Users find this far more trustworthy than a confident wrong answer. The system knowing what it doesn't know is a feature, not a limitation.

Bridging the Vocabulary Gap

One persistent challenge in RAG is the mismatch between how users phrase questions and how the underlying documents are written. A user asking about "getting out of a contract" might be looking for a section titled "Termination of Agreement." The embeddings are related but not identical, and retrieval can miss the right chunk.

The most effective technique for this is called HyDE — Hypothetical Document Embeddings. Instead of embedding the raw user question, you first ask the LLM to generate a brief, formal answer to the question as it might appear in a document. Then you embed that hypothetical answer.

Because the hypothetical answer uses vocabulary closer to the source documents, the vector search tends to find the right chunks much more reliably. It's one extra LLM call — fast and cheap — that consistently improves retrieval quality on formal, technical, or legal content.

Where RAG Fits in the Broader AI Landscape

RAG has become the default pattern for any AI application that needs to work with private, specialized, or frequently updated information. The reasons are practical.

Fine-tuning a model on your data sounds appealing, but it's expensive, slow, and the model can still hallucinate. It also doesn't help when your data changes — you'd need to fine-tune again. RAG sidesteps all of this. You update your vector store when data changes, and the model immediately has access to the latest version without retraining.

The pattern powers a wide range of real applications: customer support bots that answer from internal documentation, legal tools that search across thousands of contracts, research assistants that work across academic papers, and enterprise tools that let employees query company knowledge in plain language.

The implementation varies — different chunking strategies for different content types, different embedding models for different languages and domains, different vector databases for different scales — but the fundamental loop never changes: embed, store, retrieve, generate.

The Thing That Actually Determines Quality

Developers new to RAG tend to focus on which LLM to use — GPT-4 versus Claude versus Llama. In practice, the model choice matters far less than the chunking strategy.

A powerful model fed irrelevant or broken chunks gives bad answers. A careful chunking strategy with a capable-but-not-state-of-the-art model gives good answers almost every time. The retrieval quality is the ceiling on the generation quality. If the right information doesn't make it into the context window, no amount of model intelligence compensates.

This is a useful mental shift for anyone building RAG systems: invest time in getting the chunking right, set a sensible similarity threshold, add overlap to handle boundaries, and test retrieval quality directly before worrying about which LLM is on the other end.

RAG is one of those patterns that looks simple on the surface and reveals depth the more seriously you build with it. If you're working on an AI application that needs to reason over your own data, this is the right foundation to start from.

#ai #llm #rag #machinelearning #webdev #buildinpublic

How I Deployed a MERN Stack App on AWS from Scratch — Step by Step published

Kopalachandran Abinash — Thu, 04 Jun 2026 09:01:19 +0000

I recently built a full-stack Hospital Management System and deployed it on AWS with a complete CI/CD pipeline. In this article I'll walk you through exactly how I did it — every command, every step, every mistake I made along the way.

By the end of this article you'll know how to:

Containerize a MERN app with Docker
Push images to AWS ECR
Deploy backend on EC2
Host frontend on S3
Set up an Application Load Balancer
Configure Auto Scaling
Automate everything with Jenkins CI/CD

🏗️ Architecture Overview

GitHub Push → Jenkins CI/CD
                ├── Build Docker Images
                ├── Push to AWS ECR
                ├── Deploy Backend → EC2 (via SSM)
                └── Deploy Frontend → S3

Users → S3 (Frontend) → ALB → EC2 (Backend) → MongoDB Atlas

Tech Stack:

Frontend: React.js + Vite + Tailwind CSS
Backend: Node.js + Express.js + Socket.IO
Database: MongoDB Atlas
Storage: Cloudinary
AI: Groq API

📋 Prerequisites

Before starting, make sure you have:

An AWS account (free tier works fine)
Docker Desktop installed
Node.js installed
Jenkins installed locally
Your MERN app code on GitHub

Phase 1 — AWS Account Setup

Create IAM Role for EC2

Your EC2 server needs permissions to talk to other AWS services. Instead of using secret keys on the server, we attach a Role.

Go to IAM → Roles → Create role
Select AWS service → EC2
Attach these 4 policies:
- AmazonS3FullAccess
- AmazonEC2ContainerRegistryFullAccess
- AmazonSSMManagedInstanceCore
- CloudWatchAgentServerPolicy
Name it: EC2-Hospital-Role

Create IAM User for CLI

IAM → Users → Create user → name: hospital-cli-user
Attach policies:
- AmazonEC2ContainerRegistryFullAccess
- AmazonS3FullAccess
- AmazonSSMFullAccess
- AmazonEC2FullAccess
- AWSCloudFormationFullAccess
Security credentials → Create access key → CLI
Download the CSV — you can't see the secret key again!

Phase 2 — AWS CLI Setup

Install AWS CLI from: https://awscli.amazonaws.com/AWSCLIV2.msi

Then configure it:

aws configure
# AWS Access Key ID: your-access-key
# AWS Secret Access Key: your-secret-key
# Default region name: eu-north-1
# Default output format: json

Verify it works:

aws sts get-caller-identity

You should see your Account ID. ✅

Phase 3 — ECR (Elastic Container Registry)

ECR is AWS's private Docker Hub. We create 2 repositories — one for backend, one for frontend.

aws ecr create-repository --repository-name hospital-backend --region eu-north-1
aws ecr create-repository --repository-name hospital-frontend --region eu-north-1

aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com

Build and push your Docker images:

# Build
docker-compose build

# Tag
docker tag hospital-app-backend:latest YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-backend:latest
docker tag hospital-app-frontend:latest YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-frontend:latest

# Push
docker push YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-backend:latest
docker push YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-frontend:latest

Phase 4 — EC2 Setup

Launch EC2 Instance

EC2 → Launch instance
Name: hospital-backend-server
AMI: Amazon Linux 2023
Instance type: t3.micro (free tier)
Create key pair: hospital-key (.pem format) — save this file!
Security group — open ports: 22, 80, 5000
IAM instance profile: EC2-Hospital-Role

Install Docker on EC2

Connect via EC2 Instance Connect, then run:

sudo yum update -y
sudo yum install -y docker
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker ec2-user
newgrp docker

Run Backend Container

# Login to ECR from EC2
aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com

# Create .env file
cat > /home/ec2-user/.env << 'EOF'
PORT=5000
MONGO_URI=your_mongodb_uri
JWT_SECRET=your_jwt_secret
# ... add all your env vars
EOF

# Run container
docker run -d \
  --name hospital-backend \
  --restart always \
  -p 5000:5000 \
  --env-file /home/ec2-user/.env \
  YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-backend:latest

Verify it's running:

docker logs hospital-backend
# Should see: Server running on port 5000 ✅
# Should see: MongoDB Connected! ✅

Phase 5 — S3 Frontend Hosting

# Create bucket
aws s3 mb s3://your-app-frontend --region eu-north-1

In AWS Console:

S3 → your bucket → Properties → Static website hosting → Enable
Index document: index.html, Error document: index.html
Permissions → Block public access → untick all
Add bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::your-app-frontend/*"
  }]
}

Build and upload frontend:

# Set your backend URL in frontend/.env
echo "VITE_API_URL=http://YOUR_ALB_DNS/api" > frontend/.env

# Build
cd frontend && npm run build

# Upload
aws s3 sync dist/ s3://your-app-frontend --region eu-north-1 --delete

Phase 6 — Application Load Balancer

WHY: The ALB gives you a stable DNS name that never changes even if EC2 restarts. It also enables Auto Scaling.

EC2 → Target Groups → Create
- Name: hospital-tg
- Protocol: HTTP, Port: 5000
- Health check path: /
- Register your EC2 instance
EC2 → Load Balancers → Create → Application Load Balancer
- Name: hospital-alb
- Internet-facing, all availability zones
- Listener: HTTP:80 → forward to hospital-tg

Wait for Active status, then copy the DNS name.

Update your frontend .env with the ALB DNS and redeploy to S3.

Phase 7 — Auto Scaling

Auto Scaling automatically adds more EC2 instances when CPU is high and removes them when traffic drops.

Create AMI from your running EC2 instance
Create Launch Template using that AMI
Create Auto Scaling Group:
- Min: 1, Desired: 1, Max: 3
- CPU tracking policy: 50%
- Attach to your ALB target group

Phase 8 — Jenkins CI/CD Pipeline

WHY: Every time you push code, Jenkins automatically builds, pushes, and deploys everything. No manual work needed.

Add credentials in Jenkins:

aws-access-key (Secret text)
aws-secret-key (Secret text)

Jenkinsfile:

pipeline {
    agent any
    environment {
        AWS_ACCESS_KEY_ID     = credentials('aws-access-key')
        AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key')
        AWS_REGION            = 'eu-north-1'
        AWS_ACCOUNT_ID        = 'YOUR_ACCOUNT_ID'
        ECR_BACKEND           = 'YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-backend'
        ECR_FRONTEND          = 'YOUR_ACCOUNT_ID.dkr.ecr.eu-north-1.amazonaws.com/hospital-frontend'
        S3_BUCKET             = 'your-app-frontend'
    }
    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/YOUR_USERNAME/YOUR_REPO.git'
            }
        }
        stage('Build Images') {
            steps {
                bat 'docker build -t hospital-backend ./backend'
                bat 'docker build -t hospital-frontend ./frontend'
            }
        }
        stage('Push to ECR') {
            steps {
                bat '"C:\\Program Files\\Amazon\\AWSCLIV2\\aws.exe" ecr get-login-password --region %AWS_REGION% | docker login --username AWS --password-stdin %AWS_ACCOUNT_ID%.dkr.ecr.%AWS_REGION%.amazonaws.com'
                bat 'docker tag hospital-backend:latest %ECR_BACKEND%:latest'
                bat 'docker tag hospital-frontend:latest %ECR_FRONTEND%:latest'
                bat 'docker push %ECR_BACKEND%:latest'
                bat 'docker push %ECR_FRONTEND%:latest'
            }
        }
        stage('Deploy Backend via SSM') {
            steps {
                script {
                    def instanceId = bat(
                        returnStdout: true,
                        script: '"C:\\Program Files\\Amazon\\AWSCLIV2\\aws.exe" ec2 describe-instances --filters "Name=tag:Name,Values=hospital-backend-server" "Name=instance-state-name,Values=running" --query "Reservations[0].Instances[0].InstanceId" --output text --region %AWS_REGION%'
                    ).trim().readLines().last()
                    bat """
                        "C:\\Program Files\\Amazon\\AWSCLIV2\\aws.exe" ssm send-command ^
                        --instance-ids ${instanceId} ^
                        --document-name "AWS-RunShellScript" ^
                        --parameters "commands=['aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin %AWS_ACCOUNT_ID%.dkr.ecr.eu-north-1.amazonaws.com && docker pull %ECR_BACKEND%:latest && docker stop hospital-backend || true && docker rm hospital-backend || true && docker run -d --name hospital-backend --restart always -p 5000:5000 --env-file /home/ec2-user/.env %ECR_BACKEND%:latest']" ^
                        --region %AWS_REGION%
                    """
                }
            }
        }
        stage('Deploy Frontend to S3') {
            steps {
                bat 'cd frontend && npm install && npm run build'
                bat '"C:\\Program Files\\Amazon\\AWSCLIV2\\aws.exe" s3 sync frontend\\dist\\ s3://%S3_BUCKET% --region %AWS_REGION% --delete'
            }
        }
    }
    post {
        success { echo 'Deployment successful!' }
        failure { echo 'Deployment failed!' }
    }
}

Key lesson: Keep secrets out of code!

Use Jenkins credentials for all secrets. GitHub will block your push if it detects API keys in your code (learned this the hard way! 😅).

Phase 9 — CloudFormation (Infrastructure as Code)

Define your infrastructure in a YAML file so you can recreate everything with one command:

AWSTemplateFormatVersion: '2010-09-09'
Description: Hospital App Infrastructure

Resources:
  BackendRepository:
    Type: AWS::ECR::Repository
    Properties:
      RepositoryName: hospital-backend

  FrontendBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: your-app-frontend
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: index.html

  HospitalSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: hospital-sg
      GroupDescription: Security group for Hospital App
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 5000
          ToPort: 5000
          CidrIp: 0.0.0.0/0

Deploy with:

aws cloudformation deploy \
  --template-file cloudformation.yml \
  --stack-name hospital-stack \
  --region eu-north-1

🐛 Mistakes I Made (So You Don't Have To)

1. Secrets in Jenkinsfile
I accidentally put my Groq API key directly in the Jenkinsfile. GitHub blocked the push immediately. Always use Jenkins credentials instead of hardcoding secrets.

2. Wrong VITE_API_URL
After deployment the frontend was hitting localhost:5000 instead of the ALB URL. Always double check your .env before building.

3. Missing IAM permissions
I kept getting UnauthorizedOperation errors because my IAM user was missing EC2 and CloudFormation permissions. Add all needed permissions upfront.

4. SSM parameter parsing
Passing environment variables through SSM commands with special characters caused parsing errors. The fix was using --env-file on EC2 instead of passing each variable individually.

🎯 Results

After all this setup, my CI/CD pipeline works like this:

I push code to GitHub
Jenkins automatically triggers
Docker images built and pushed to ECR
Backend deployed to EC2 via SSM (no SSH needed!)
Frontend built and uploaded to S3
Everything live in ~10 minutes

No manual deployment steps. Ever.

📊 Cost

Running this on AWS free tier costs approximately $0/month if you:

Use t3.micro EC2 (750 hours/month free)
Stop EC2 when not using
S3 storage is practically free for a small app

🔗 Resources

GitHub Repo: https://github.com/abinash1417/Hospital-app
Live Demo: http://hospital-app-frontend-159372.s3-website.eu-north-1.amazonaws.com
AWS Free Tier: https://aws.amazon.com/free

Conclusion

Deploying a full-stack app on AWS from scratch taught me more about DevOps than any tutorial. The key things I learned:

Always use IAM roles instead of access keys on servers
Keep secrets in credential managers, never in code
Load Balancers are essential for stable URLs
CI/CD pipelines save enormous amounts of time
Infrastructure as Code means you never lose your setup again

If you have any questions, drop them in the comments! Happy to help. 🚀

If this helped you, please leave a ❤️ and follow for more DevOps content!