DEV Community: piyushsachdeva

Build an AI Observability Pipeline on EKS Using Elasticsearch

piyushsachdeva — Sun, 14 Jun 2026 14:30:05 +0000

It's 2am and your pager is going off. You're on call, you were asleep, and now there's a pod stuck in CrashLoopBackOff. You know the drill: tail the logs, kubectl describe, scroll Git history, ask in Slack whether anyone shipped anything in the last hour.

Forty minutes later you've found it, and you've woken up your tech lead to confirm.

This walkthrough is about not doing that anymore.

We'll build an AI observability pipeline on Amazon EKS with Elastic Cloud and OpenTelemetry, end to end. The whole thing rests on one move: join two streams of data, your crash logs and your deployment history, and let a language model reason across the join.

Then you ask a plain question, "Why is paymentservice crashing?", and instead of another dashboard to squint at, you get the root cause:

paymentservice went OOMKilled three minutes after commit a1b2c3 by xyz, which dropped the memory limit from 128Mi to 24Mi.

The commit SHA, the author, what changed, what to revert. Two indices and one agent is all it takes. Over the next sections you'll wire OpenTelemetry, Elasticsearch, and ArgoCD into your EKS cluster until that 2am question answers itself.

Architecture
Prerequisites
Step 1: Provision Elasticsearch from AWS Marketplace
Step 2: Configure OpenTelemetry to ship Kubernetes logs
Step 3: Index Git commit metadata via GitHub Actions
Step 4: Set up Argo CD for GitOps deployment
Step 5: Build the AI diagnostic agent in Elastic
Step 6: Simulate and diagnose a crash
Why this works
FAQs

Architecture

Two data flows meet inside Elasticsearch.

On the infrastructure side, you push code, Argo CD syncs it to EKS, a pod crashes, and OpenTelemetry ships the crash event to the logs-* index.

On the metadata side, the same push fires a GitHub Action that POSTs the commit details (SHA, author, diff URL, and message) to the github-deploymentsindex.

The Elastic AI agent owns both queries. Ask it a question, and it runs one ES|QL query against the logs to find the failure and a second against the deployments to find the most recent push and stitches the two into a plain-English answer.

A few things to keep in mind before you start.

Push your Kubernetes manifests to GitHub before you create the Argo CD app. Argo CD syncs whatever is in the repo on first sync, so an empty repo sends you down a long debugging detour.
Setup Elastic Cloud before you install OpenTelemetry. The OTel chart wants the endpoint and API key at install time.
Create the github-deploymentsindex before you push to a repo that already has the workflow file, because the first push tries to write straight away.
Both indices need real data before you switch on the agent. It cannot reason over nothing.

Prerequisites

You'll need these tools installed locally:

AWS CLI (aws)
Kubernetes CLI (kubectl)
eksctl (for EKS cluster management)
Helm
Argo CD CLI
GitHub CLI (gh) and Git

Connect kubectl to your EKS cluster:

aws eks update-kubeconfig \
  --region us-east-1 \
  --name <CLUSTER_NAME>

Verify nodes are healthy:

kubectl get nodes
# Expect: 3 nodes, Status: Ready

One sizing note that will save you a Slack message later: use m5.xlarge instances, not t3.medium. You're running microservices, OTel DaemonSets, and Argo CD on the same cluster. Two vCPUs per node won't cut it. Three m5.xlargenodes is the sweet spot.

If you don't have an EKS cluster yet, spin one up quickly:

eksctl create cluster \
  --name demo-cluster \
  --region us-east-1 \
  --nodegroup-name standard-workers \
  --node-type m5.xlarge \
  --nodes 3 \
  --nodes-min 3 \
  --nodes-max 5 \
  --managed

Step 1: Provision Elasticsearch from AWS Marketplace

You could self-host Elasticsearch; however, the Elastic Cloud takes care of managing the service for you, which is another way to get peace of mind. The Marketplace deployment lands on the same AWS invoice and handles upgrades for you. (A free trial is enough for the demo.)

Open the AWS Console and navigate to AWS Marketplace.
Search for Elastic Cloud (Elasticsearch Service).
Subscribe, then click Set up your account. That kicks you over to the Elastic Cloud console.
The setup wizard will ask a few basic questions, along with "Which use case are you looking to try first?". Choose Elastic for Observability. This template pre-enables the bits you need: log aggregation, APM, and AIOps. The plain Elasticsearch template means turning those on later by hand.
The next question it will ask is "Which Deployment option would you like to try?" Click Create Cloud Serverless.
Then it will ask for the configuration options; select "Observability complete."
Lastly, it will ask the preferred region for cloud data storage, you can keep the default as cloud provider: Amazon Web Services and region to us-east-1.

Provisioning takes about five minutes. When the credentials screen appears, copy the elastic user password.

Get your endpoints

From the Elastic Cloud console, open your deployment and grab these two:

Elasticsearch → Copy endpoint — used with GitHub Actions secret ES_ENDPOINT and curl commands.
Integrations → Manage → APM → copy OTLP endpoint — used as the OTel kube-stack secret elastic_otlp_endpoint.

Step 1b: Generate a Kibana API key

You'll need one API key in two places (the OTel install and your GitHub repo secrets), so generate it now.

Navigate to:

https://<YOUR-KIBANA-URL>/app/management/security/api_keys

Or via the menu: dashboard -> settings -> Access -> Api Keys

Click Create API key, name it something obvious like github-deploy-key, and click Create.

Copy the key. Same rule as the password: shown once. You'll reuse it as:

elastic_api_key in the OTel secret (Step 2)
ES_API_KEY in your GitHub repo secrets (Step 3) Verify the key works:

curl -s -w "\nHTTP:%{http_code}" \
  -X GET "<YOUR-ES-ENDPOINT>" \
  -H "Authorization: ApiKey <YOUR-API-KEY>"
# Expect: HTTP:200

If you don't get a 200, fix that now. A bad key here causes OTel to silently drop logs later, and you'll spend an hour wondering why Discover is empty.

Step 2: Configure OpenTelemetry to ship Kubernetes logs

OpenTelemetry is the bridge between your cluster and Elasticsearch. Once it's installed as a DaemonSet, every pod log in the cluster ships automatically. You don't have to instrument individual services for this tutorial.

In Elastic, open your deployment and click Add Observability Data.
Choose Kubernetes.
Under Monitor your Kubernetes cluster using, pick OpenTelemetry: Full Observability. Elastic pre-fills the Helm commands with your endpoint and credentials.

Add the Helm repo:

helm repo add open-telemetry \
  https://open-telemetry.github.io/opentelemetry-helm-charts --force-update

Create the namespace, secret, and chart:

kubectl create namespace opentelemetry-operator-system

kubectl create secret generic elastic-secret-otel \
  --namespace opentelemetry-operator-system \
  --from-literal=elastic_otlp_endpoint='<YOUR-OTLP-INGEST-ENDPOINT>' \
  --from-literal=elastic_api_key='<YOUR-API-KEY>'

helm upgrade --install opentelemetry-kube-stack \
  open-telemetry/opentelemetry-kube-stack \
  --namespace opentelemetry-operator-system \
  --values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.3.3/deploy/helm/edot-collector/kube-stack/managed_otlp/values.yaml' \
  --version '0.12.4'

If Helm complains about a conflict, run helm uninstall opentelemetry-kube-stack -n opentelemetry-operator-system and reinstall. This happens to everyone the first time.

Verify all pods came up:

kubectl get all -n opentelemetry-operator-system

Step 2b: Verify logs are flowing in Kibana

Before you wire up anything else, confirm OTel is actually shipping logs. Open Kibana and go to Discover. The Kibana URL is on the Elastic Cloud deployment overview page, listed next to the Elasticsearch endpoint.

In Discover, pick the All logs data view, set the time range to Last 15 minutes, and look for documents arriving from your EKS pods. You should see system components, OTel collectors, and any applications running on the cluster.

If you see documents, you're good. If not, double-check the API key and the OTLP endpoint in the secret.

EKS-specific note: If you're using a private EKS cluster (no public endpoint), make sure the OTel collector pods can reach the Elastic endpoint. You may need to add a NAT Gateway or VPC endpoint for outbound HTTPS traffic on port 443.

Step 3: Index Git commit metadata via GitHub Actions

The agent answers "Who deployed this?" by querying a second index, and that index needs data in it. A GitHub Action handles that automatically on every push. Nothing runs inside your cluster for this part; Actions runs in GitHub's cloud and POSTs straight to Elasticsearch.

3a. Create the index

curl -X PUT "<YOUR-ES-ENDPOINT>/github-deployments" \
  -H "Authorization: ApiKey <YOUR-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "mappings": {
      "properties": {
        "timestamp":   { "type": "date" },
        "commit_sha":  { "type": "keyword" },
        "author":      { "type": "keyword" },
        "service":     { "type": "keyword" },
        "image_tag":   { "type": "keyword" },
        "change":      { "type": "text" },
        "diff_url":    { "type": "keyword" }
      }
    }
  }'
# Expect: {"acknowledged":true}

3b. Add GitHub repository secrets

gh secret set ES_ENDPOINT \
  --repo <YOUR-GITHUB-USERNAME>/<YOUR-REPO> \
  --body "<YOUR-ES-ENDPOINT>"

gh secret set ES_API_KEY \
  --repo <YOUR-GITHUB-USERNAME>/<YOUR-REPO> \
  --body "<YOUR-API-KEY>"

Step 3c: The workflow file

Create .github/workflows/index-deploy.yml. On every push to main, it figures out which service changed (by diffing release/kubernetes-manifests.yaml) and POSTs one document to the github-deployments index.

name: Index deployment to Elasticsearch
on:
  push:
    branches: [main]

jobs:
  index:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 2

      - name: Detect changed service
        id: changed
        run: |
          CHANGED=$(git diff HEAD~1 HEAD -- release/kubernetes-manifests.yaml \
            | grep '^+' \
            | grep -oP 'name:\s+\K[a-z][a-z0-9-]+service' \
            | head -1)
          if [ -z "$CHANGED" ]; then
            CHANGED=$(git diff HEAD~1 HEAD -- release/kubernetes-manifests.yaml \
              | grep -oE 'paymentservice|cartservice|frontend|recommendationservice|adservice|checkoutservice|productcatalogservice|currencyservice|shippingservice|emailservice|redis-cart|loadgenerator' \
              | head -1)
          fi
          echo "service=${CHANGED:-unknown}" >> $GITHUB_OUTPUT

      - name: Push deploy event to Elasticsearch
        run: |
          curl -X POST "${{ secrets.ES_ENDPOINT }}/github-deployments/_doc" \
            -H "Authorization: ApiKey ${{ secrets.ES_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d "{
              \"timestamp\":  \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",
              \"commit_sha\": \"${{ github.sha }}\",
              \"author\":     \"${{ github.actor }}\",
              \"service\":    \"${{ steps.changed.outputs.service }}\",
              \"image_tag\":  \"${{ github.sha }}\",
              \"change\":     \"${{ github.event.head_commit.message }}\",
              \"diff_url\":   \"${{ github.event.compare }}\"
            }"

Step 3d: Verify it works

git commit --allow-empty -m "test: verify ES indexing"
git push origin main

gh run list --limit 3
# Expect: "Index deployment to Elasticsearch" → completed success

curl -s "<YOUR-ES-ENDPOINT>/github-deployments/_count" \
  -H "Authorization: ApiKey <YOUR-API-KEY>"
# Expect: {"count":1,...}

Step 4: Set up Argo CD for GitOps deployment

Argo CD watches your repo and syncs changes to EKS on a schedule. The whole point of the demo is to push code and have it deploy on its own, and Argo CD is what makes that real.

# Install Argo CD
kubectl create namespace argocd
kubectl apply -n argocd \
  -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl rollout status deployment/argocd-server -n argocd --timeout=180s

# Expose publicly via LoadBalancer (creates an AWS ELB)
kubectl patch svc argocd-server -n argocd \
  -p '{"spec":{"type":"LoadBalancer"}}'
kubectl get svc argocd-server -n argocd
# Note the EXTERNAL-IP (AWS ELB hostname)

# Get admin password
kubectl get secret argocd-initial-admin-secret -n argocd \
  -o jsonpath='{.data.password}' | base64 -d && echo ""

# Log in
argocd login <EXTERNAL-IP> --username admin --password <PASSWORD> --insecure

Note: On EKS, the EXTERNAL-IP for the LoadBalancer will be an AWS ELB DNS hostname (e.g., xxxxxx.us-east-1.elb.amazonaws.com), not a plain IP. Use that hostname wherever <EXTERNAL-IP> appears.

Feel free to use this sample repo as your application.

Create the Argo CD application pointing at your repo:

kubectl apply -f - <<EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: online-boutique
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/<YOUR-USERNAME>/microservices-demo
    targetRevision: main
    path: release
  destination:
    server: https://kubernetes.default.svc
    namespace: online-boutique
  syncPolicy:
    automated:
      prune: true
    syncOptions:
      - CreateNamespace=true
EOF

Trigger the first sync. The default Argo CD poll interval is three minutes, which is fine in production and brutal in a demo. Drop it to thirty seconds so you can actually watch things happen.

argocd app sync online-boutique --force --prune

kubectl patch configmap argocd-cm -n argocd --type merge \
  -p '{"data": {"timeout.reconciliation": "30s"}}'
kubectl rollout restart deployment/argocd-repo-server -n argocd

# Verify all 12 pods are running
kubectl get pods -n online-boutique

Step 5: Build the AI diagnostic agent in Kibana

Both indices are live. Now build the agent that queries them.

Step 5a: LLM setup

The agent needs a language model behind it. Use Elastic Inference Service (EIS).

EIS ships with Elastic for Observability deployments and Marketplace trials, so the model dropdown is already populated when you create an agent. You'll see Claude for reasoning, JINA embeddings for vectorization, and the JINA reranker for retrieval ordering, all billed through the same Elastic invoice. Pick Claude (or whichever reasoning model you prefer) for the agent itself.

You'll want the JINA models later, if you extend this pattern to semantic search over logs or RAG over runbooks. For the diagnostic agent here, the reasoning model is all you need.

If EIS isn't available on your deployment, fall back to an external connector:

Kibana → ☰ → Stack Management → Connectors → Create connector → OpenAI

Paste an API key from platform.openai.com, set the model to gpt-4o, click Save & test, then pick the connector in Agent Builder.

Anthropic, Google Vertex, and AWS Bedrock connectors follow the same shape. Pick whichever you already have billing set up for.

Step 5b: Create the ES|QL tools

Kibana → ☰ → Search → Tools → Create tool.

Tool 1: get_crash_logs

Type: Elasticsearch query
Index: logs-*

FROM logs-*
| WHERE resource.attributes.k8s.namespace.name == "online-boutique"
| WHERE body.text LIKE "*OOMKill*" OR body.text LIKE "*memory*"
| SORT @timestamp DESC
| LIMIT 20
| KEEP @timestamp, resource.attributes.k8s.deployment.name, resource.attributes.k8s.pod.name, body.text

Tool 2: get_deploy_history

Type: Elasticsearch query
Index: github-deployments

FROM github-deployments
| SORT timestamp DESC
| LIMIT 5
| KEEP timestamp, author, commit_sha, service, change, diff_url

Step 5c: Create the agent

Kibana → ☰ → Search → Agent Builder → Create agent.

Name: blame-the-deploy
Model: Claude, or your configured connector

Paste these instructions:

You are an SRE assistant. When asked why a service is crashing:
1. Use get_crash_logs to find OOMKill or memory events in logs-*
2. Use get_deploy_history to find the most recent GitHub deployment to that service
3. If the deploy happened shortly before the crash, that is the likely cause
4. Report: crash time, commit SHA, author, what changed, and how to fix it
Always cite the commit SHA and author name in your answer.

Under Tools, add get_crash_logs and get_deploy_history and click save.

Step 6: Simulate and diagnose a crash

The whole reason you built this is to ask the agent a hard question. So break something on purpose.

Introduce the bad commit

Open release/kubernetes-manifests.yaml. Find the paymentservice deployment (around line 631) and squeeze the memory limits below what the service actually needs at startup:

resources:
  requests:
    memory: "24Mi"   # was 64Mi
  limits:
    memory: "24Mi"   # was 128Mi

git add release/kubernetes-manifests.yaml
git commit -m "perf: tune paymentservice memory limits for cost optimisation"
git push origin main

That commit message is the kind of thing a tired engineer actually writes at 4pm on a Friday. Argo CD picks it up within thirty seconds and applies the manifest. Because 24Mi isn't enough, the pod dies on startup.

kubectl get pods -n online-boutique -w
# paymentservice: Running → OOMKilled → CrashLoopBackOff

Confirm which pod is unhappy:

kubectl describe pod -n online-boutique <pod-name> | grep -A5 'Last State\|OOM'

Ask the agent

Open the Elastic Agent chat:

Why is paymentservice crashing? Check the logs and recent deployments.

The agent runs get_crash_logs, sees the OOMKill, runs get_deploy_history, finds the commit that lowered the memory limit, and reports back with the SHA, the author, the commit message, the diff URL, and a fix suggestion.

Fix it

git revert HEAD --no-edit
git push origin main
# Pod recovers in about 30 seconds

Ask the agent again:

Is paymentservice healthy now?

It checks the logs, finds no recent OOMKills, and confirms the service is stable.

Why this works

It comes down to the join. Kubernetes logs tell you what crashed and when. The github-deployments index tells you what was deployed and by whom. Either index alone leaves you guessing. Stitch them together behind an agent and you skip the part of the incident where you're bouncing between CloudWatch, GitHub, Slack, and kubectl.

The pattern extends. Want to diagnose CPU throttling? Add a tool that searches logs-* for CPUThrottlingHigh events. Want to catch failed health checks? Same idea with a different ES|QL clause. The tools are just saved queries, and the agent picks up a new diagnostic capability the next time you chat with it.

One caveat before you put this in front of the rest of your team. The agent is good at correlation, not causation. If two services deploy in the same five-minute window and one crashes, the agent will cheerfully blame the wrong commit. Treat the output as a strong lead, not a verdict.

Quick reference

Commands you will reuse

kubectl get pods -n online-boutique        # check all pods
kubectl get pods -n online-boutique -w     # watch live
kubectl get pods -n opentelemetry-operator-system  # check OTel
argocd app get online-boutique             # Argo CD sync status
argocd app sync online-boutique --force    # force sync

ES|QL: find OOMKill events

FROM logs-*
| WHERE resource.attributes.k8s.namespace.name == "online-boutique"
| WHERE body.text LIKE "*OOMKill*"
| SORT @timestamp DESC
| LIMIT 20
| KEEP @timestamp, resource.attributes.k8s.deployment.name, body.text

ES|QL: find recent deploys

FROM github-deployments
| SORT timestamp DESC
| LIMIT 10
| KEEP timestamp, author, commit_sha, service, change

FAQs

What is an AI observability pipeline?

An AI observability pipeline is a system that ships infrastructure telemetry and deployment metadata into the same data store, then puts an LLM in front of it so you can ask plain-English questions about failures. The model runs structured queries against your indices instead of asking you to remember which dashboard goes with which symptom.

Can I run this without AWS?

Yes. The architecture is portable. Swap EKS for GKE, AKS, or any conformant Kubernetes cluster. Elasticsearch has hosted offerings on GCP and Azure Marketplace, and OpenTelemetry doesn't care which cloud it runs on. The only AWS-specific steps are the Marketplace billing flow and the aws eks update-kubeconfig command, and both have direct equivalents on the other clouds.

Why use Elasticsearch for this use case?

Elasticsearch lets you keep your logs, metrics, deployment events, and vector embeddings in one place. The Kibana AI agent is built in. You define a tool, point it at an index, and chat with it. No LangChain wrapper, no FastAPI orchestrator, no model gateway to keep alive. And ES|QL is one query language across logs, metrics, and structured JSON metadata, which is the only reason the join in this tutorial (logs-* against github-deployments) is five lines instead of a Python service.

How much does this cost to run?

A small Elastic Cloud deployment on us-east-1 runs about $95 per month at the time of writing. EKS costs depend on node count; three m5.xlarge nodes are roughly $210 per month before Savings Plans or Reserved Instance discounts. The EKS control plane itself is $0.10/hour (~$73/month). If you're running this as a learning exercise, tear the cluster down between sessions with eksctl delete cluster --name demo-cluster --region us-east-1.

My GitHub action fires, but no document lands in Elasticsearch. What now?

Check three things in this order. First, do the repo secrets resolve? Add echo "endpoint length: ${#ES_ENDPOINT}" to the workflow to verify. Second, does the API key have write permission on github-deployments? Test with curl -X POST from your laptop using the same key. Third, is your ES_ENDPOINT value missing the https:// prefix? That one bites everyone.

Can the Kibana AI agent fix the bug or just identify it?

For this tutorial, it identifies. It doesn't open a pull request or call kubectl rollout undo on your behalf. You can extend it to do that with a custom tool that wraps the GitHub API or kubectl, but I'd rather you read the diagnosis first and decide. Autonomous remediation in production is a separate conversation.

Does this replace my on-call runbooks?

No. It collapses the first ten minutes of an incident, the part where you're hunting for the right tab. The runbook still tells you what to do once you know what broke. Think of the agent as the on-call's first responder, not the surgeon.

What IAM permissions does my EKS cluster need?

The OTel DaemonSet only needs outbound HTTPS to the Elastic endpoint; no AWS-specific IAM permissions are required. If your EKS nodes use an instance profile, make sure the security group allows outbound 443. Argo CD similarly needs no special AWS permissions, since it operates against the Kubernetes API only.

References

Deploy Your Own 24/7 AI Agent on AWS EC2 with Docker & Tailscale (The Secure Way)

piyushsachdeva — Wed, 11 Feb 2026 04:39:29 +0000

No, you don't need a Mac Mini or fancy hardware setup to run your own AI agent 24/7. All you need is a cloud virtual machine or VPS (Virtual Private Server).

I know what you're thinking: "What about security?" And you should be thinking about it. That's exactly why I'm going to walk you through the step-by-step process to securely set up an AI agent running as a Docker container.

Imagine having your own private AI assistant running 24/7 on a cloud server you control-accessible from anywhere, yet completely shielded from the public internet. That's exactly what we're building today.

In this guide, we'll deploy OpenClaw (also known as Moltbot/Clawdbot) on an AWS EC2 instance using Docker containers. But we're not just going to spin up a container and call it a day. We'll harden our server, lock down SSH access, and use Tailscale to create a secure private network that only you can access.

What You'll Have by the End

A hardened Ubuntu server with non-standard SSH configuration
Docker running OpenClaw in an isolated container
Secure private access via Tailscale (no public ports exposed)
A fully functional AI assistant accessible from your browser Let's get started.

If you prefer video tutorial, you can watch the below end-to-end video (else, keep reading the blog)

Prerequisites

Before we dive in, make sure you have the following ready:

Cloud Infrastructure

AWS Account with EC2 access
A fresh Ubuntu 24.04 LTS instance (Recommended specs: 2 vCPU, 4GB RAM, 15GB storage)
Key Pair: Your .pem file from AWS (e.g., moltbot.pem) saved on your local computer
Current Access: Ability to SSH into the instance as the default ubuntu user

API Keys

Anthropic API Key (for Claude Sonnet/Opus - recommended for best performance)
OpenAI API Key (Optional, as a backup)

Networking

Tailscale Account: Free tier is sufficient (this is how we'll securely access our bot)

Pro Tip: If you're new to AWS, providers like Hetzner, DigitalOcean, or Vultr also work perfectly with this guide.

Phase 1: VM Hardening (Do This FIRST)

Before we install anything fun, we need to secure our server. Think of this as locking the doors and windows before you move your valuables inside.

Step 1: Create a New User

First, SSH into your server as the default ubuntu user:

ssh -i "moltbot.pem" ubuntu@<YOUR-EC2-PUBLIC-IP>

Once connected, let's update the system and create a dedicated user for OpenClaw:

# Update system
sudo apt update && sudo apt upgrade -y

# Create a new user (replace 'openclaw' if you want a different name)
sudo adduser openclaw
# (Enter a strong password when prompted)

# Grant sudo rights
sudo usermod -aG sudo openclaw

Why a new user? Running applications as a dedicated user follows the principle of least privilege. If something goes wrong, the blast radius is contained.

Step 2: Set Up SSH Keys for the New User

Here's where many tutorials fail you. The AWS key (.pem file) only works for the ubuntu user by default. We need to copy the authorized keys to our new user, or we'll be locked out.

Run these commands while still logged in as ubuntu:

# 1. Create the SSH directory for the new user
sudo mkdir -p /home/openclaw/.ssh

# 2. Copy the authorized keys from 'ubuntu' to 'openclaw'
sudo cp /home/ubuntu/.ssh/authorized_keys /home/openclaw/.ssh/

# 3. Fix permissions (CRITICAL: If this is wrong, login will fail)
sudo chown -R openclaw:openclaw /home/openclaw/.ssh
sudo chmod 700 /home/openclaw/.ssh
sudo chmod 600 /home/openclaw/.ssh/authorized_keys

The Permission Trinity: Directory at 700, keys file at 600, owned by the user. Get any of these wrong, and SSH will silently reject your login.

Step 3: Configure AWS Security Group (External Firewall)

Time to switch to your web browser. We need to tell AWS to allow traffic on our new SSH port.

Go to EC2 Dashboard → Security Groups
Select your instance's security group → Edit inbound rules
Add Rule:
- Type: Custom TCP
- Port: 2222
- Source: 0.0.0.0/0 (or "My IP" for extra security)
DO NOT delete the rule for Port 22 yet - this is your safety net
Save rules

Step 4: Configure UFW (Internal Firewall)

Back in your terminal, let's set up the host-level firewall:

# Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow the CURRENT port (Safety Net)
sudo ufw allow 22/tcp

# Allow the FUTURE port
sudo ufw allow 2222/tcp

# Enable the firewall
sudo ufw enable

When prompted, type y to confirm. You now have two layers of firewall protection.

Step 5: The Golden Test

This is the moment of truth. Before we disable password authentication or change ports, we need to prove that our new user can log in with the SSH key.

Keep your current terminal window open (this is your lifeline).

Open a NEW terminal window on your local computer and run:

ssh -i "moltbot.pem" openclaw@<YOUR-IP-ADDRESS>

Result	Action
Success: You log in without being prompted for a password	Proceed to Step 6
Failure: "Permission denied"	Go back and re-check Step 2. Do not proceed.

Step 6: Lock Down SSH Configuration

Now that we've verified key-based login works, it's time to harden SSH. This is where Ubuntu 24.04 throws a curveball that catches many admins off guard.

Edit the SSH config file:

sudo vi /etc/ssh/sshd_config

Make the following changes:

Port 2222
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

Save and close the file (:wq in vi).

Disable Systemd Socket Activation (CRITICAL)

Here's the gotcha: Ubuntu 24.04 uses "socket activation" which holds Port 22 open regardless of what you put in sshd_config. You must disable this:

# Stop the socket listener
sudo systemctl stop ssh.socket
sudo systemctl disable ssh.socket

# Restart the SSH service to apply your new config
sudo systemctl restart ssh

Verify the Port Change:

sudo ss -tulpn | grep ssh

Output	Status
`0.0.0.0:2222`	Success!
`0.0.0.0:22`	Something went wrong. Do NOT disconnect.

Step 7: Final Verification & Cleanup

Open a NEW terminal window and test the new port:

ssh -p 2222 -i "moltbot.pem" openclaw@<YOUR-IP-ADDRESS>

Only if that works, go back to your server terminal and lock down Port 22:

sudo ufw delete allow 22/tcp

Then go to AWS Security Groups and delete the inbound rule for Port 22.

Important: From this point forward, you can no longer use "EC2 Instance Connect" (the browser console). You must always use the SSH command above.

Phase 2: Environment Setup

With our server hardened, it's time to install the software stack. We'll set up Docker to run OpenClaw in isolation and Tailscale to create our private access tunnel.

1. Install Docker & Docker Compose

Docker ensures OpenClaw's dependencies don't interfere with your host OS. Here's the official installation method:

# Install Docker dependencies
sudo apt install apt-transport-https ca-certificates curl software-properties-common -y

# Add Docker GPG key & Repository
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y

# Allow 'openclaw' user to run Docker without sudo
sudo usermod -aG docker ${USER}

Now log out and log back in for the group change to take effect:

exit

Then reconnect via SSH:

ssh -p 2222 -i "moltbot.pem" openclaw@<YOUR-IP-ADDRESS>

Verify Docker works without sudo:

docker run hello-world

2. Install Tailscale (The Secure Access Layer)

Here's where the magic happens. Instead of exposing OpenClaw's web port (18789) to the public internet, we'll use Tailscale to create a private mesh network. Only devices on your Tailnet can access the bot.

On your EC2 server:

curl -fsSL https://tailscale.com/install.sh | sh

After installation, authenticate with your Tailscale account:

sudo tailscale up

Follow the link provided to authorize your server.

On your local machine:

Install Tailscale from tailscale.com/download and sign in with the same account.

Once both devices are on your Tailnet, they can communicate securely without any public ports.

Phase 3: OpenClaw Installation

The infrastructure is ready. Let's deploy the bot.

1. Clone the Repository

git clone https://github.com/openclaw/openclaw.git
cd openclaw

2. Clean Up Old Configurations

If you've experimented with OpenClaw before, nuke any old config folders to prevent conflicts:

sudo rm -rf ~/.openclaw

3. Pre-Create the Configuration Folder

This step prevents the dreaded "Permission Denied" errors. We create the folder manually and set permissions so both you AND Docker can write to it:

# Create the folder structure
mkdir -p /home/openclaw/.openclaw/workspace

# Give full read/write access (fixes the "Permission Denied" error)
sudo chmod -R 777 /home/openclaw/.openclaw
sudo chown -R openclaw:openclaw ~/openclaw
sudo chmod -R 775 ~/openclaw

4. Run the Setup Script

Now we're ready for the main event:

cd ~/openclaw
./docker-setup.sh

A series of prompts will appear. Select EXACTLY these options to avoid crashes:

Prompt	Selection
Onboarding mode	Manual
Setup	Local gateway (this machine)
Workspace directory	(Press Enter to accept default)
Model/auth Provider	Anthropic
Anthropic API Key	(Enter your key)
Gateway port	18789
Gateway bind	Tailnet
Gateway auth	Token
Tailscale exposure	Off
Gateway Token	(Create a secure token)
Configure chat channel	(Whatsapp/Telegram - your choice)
Configure Skills	No
Hooks	Skip for Now

Critical Warning: Do NOT select "Serve" for Tailscale exposure. This bypasses complex proxy logic that causes crashes. Select "Off" - Tailscale already handles secure access.

Wait for completion. The container will start, but DO NOT LOGIN YET.

5. The "Insecure Auth" Injection

Here's a quirk with OpenClaw: At this stage, the bot is running but will block your browser because you're connecting over HTTP (not HTTPS). Since Tailscale already encrypts our traffic end-to-end, we can safely enable HTTP authentication.

First, install jq inside the container:

docker compose exec -u root openclaw-gateway bash -c "apt update && apt install -y jq"

Then inject the setting:

docker compose exec -T openclaw-gateway bash -c '
jq ".gateway.controlUi.allowInsecureAuth = true" \
/home/node/.openclaw/openclaw.json > /home/node/.openclaw/tmp.json && \
mv /home/node/.openclaw/tmp.json /home/node/.openclaw/openclaw.json'

Restart the bot to apply changes:

docker compose restart

Why is this safe? We're not exposing any ports to the public internet. Traffic flows exclusively through your encrypted Tailscale tunnel. The "insecure" part only refers to HTTP vs HTTPS - and Tailscale already provides encryption.

Phase 4: Login & Verify

The moment of truth. Your bot is running, security is configured, and Tailscale is connecting your devices. Let's access the interface.

1. Check Container Status

Make sure the containers are healthy and staying up:

docker compose ps

Watch it for a few seconds. If the status stays "Up" for more than 10 seconds, you're good.

2. Get Your Tailscale IP

tailscale ip -4

This returns something like 100.x.x.x - your private Tailscale IP address.

3. Access the Web Interface

On your local machine (which must also be connected to Tailscale), open your browser and navigate to:

http://<YOUR_TAILSCALE_IP>:18789

For example: http://100.64.0.1:18789

Enter the Gateway Token you created during setup and click Connect.

Congratulations! You now have a private AI assistant running on your own infrastructure, accessible only through your secure Tailscale network.

Phase 5: Best Practices & Maintenance

Your bot is running, but a few housekeeping practices will keep it healthy and your wallet safe.

Set API Spend Limits

Agentic AI can sometimes get stuck in loops and burn through API credits rapidly. Protect yourself:

Go to your Anthropic Console
Navigate to Settings → Limits
Set a hard monthly budget (e.g., $20)

Do the same for OpenAI if you configured it as a backup.

Enable Sandbox Mode

In the OpenClaw configuration, ensure Sandbox Mode is enabled if available. This restricts the bot's ability to execute unrestricted shell commands on the host system.

Back Up Your Bot's Memory

OpenClaw stores its "long-term memory" in Markdown files. If you lose this directory, the bot forgets everything it knows about you.

Create regular backups:

# Simple backup command
tar -czvf memory_backup_$(date +%F).tar.gz ./openclaw/memory

Consider automating this with a cron job for daily backups.

Monitor Logs

When things go sideways (and they sometimes do), logs are your best friend:

docker compose logs -f

The -f flag follows the log output in real-time. Press Ctrl+C to exit.

Conclusion

You've successfully deployed OpenClaw on a hardened EC2 instance with:

Non-standard SSH port (2222) with key-only authentication
Dual-layer firewall protection (AWS Security Groups + UFW)
Docker containerization for clean isolation
Tailscale private networking for secure, zero-trust access
No public ports exposed for the application itself

Your AI assistant is now running on infrastructure you control, accessible only to devices on your private Tailnet. No VPN configurations, no exposed ports, no attack surface for the public internet to probe.

Happy building!

DevOps and Cloud Engineering Roadmap 2025 🚀

piyushsachdeva — Fri, 20 Dec 2024 15:46:03 +0000

🚀 DevOps and Cloud Engineering Roadmap 2025

📋 Table of Contents

About
Learning Path
Projects
Resources

🎯 About

This repository contains a structured learning path for beginners entering the DevOps and Cloud Engineering field. The curriculum is designed for students, recent graduates, and career switchers, with an estimated completion time of 6-12 months of dedicated learning.

📚 Learning Path

Phase 1: Foundation (2-3 months)

Linux and Shell Scripting

# Topics:
├── Linux Administration
|   ├── Linux Installation
│   ├── Command Line Basics
│   ├── File System Management
│   ├── Process Management
│   ├── User Administration
│   └── Package Management
└── Bash Scripting
    ├── Variables and Data Types
    ├── Control Structures
    ├── Functions
    └── Automation Scripts

Programming Foundation

Choose one:

Python (Recommended for beginners)
- Basic syntax and data structures
- Object Oriented Programming
- File handling and automation
- API integration
- Libraries: requests, pandas, numpy
Golang
- Systems programming
- Concurrent programming
- Cloud-native development

Version Control

# Git & GitHub
├── Git Setup
├── Basic Commands
├── Branching Strategies
├── Git Flow
├── Pull Requests
├── Code Reviews
└── Collaborative Development

Phase 2: Core IT Skills (1 month)

Network Fundamentals

OSI Model - In-depth
TCP/IP Protocol Suite
Internet Protocols ( HTTP, HTTPS, SSH, SMTP, ICMP, etc)
How to analyze network packets using tools such as Wireshark
DDOS attack and how to mitigate that
CIDR and Subnetting Video Link
DNS and DHCP Video Link
Vertical V/s Horizontal Scaling
SSL/TLS, symmetric vs asymmetric encryption and how it works : Video Link
Reverse Proxy V/s Forward Proxy V/s load balancer
Troubleshooting application slowness, latency, unavailability etc
RAID
NAS vs SAN storage
SQL vs NoSQL
DB Sharding, Caching etc

Phase 3: Cloud Computing (2-3 months)

Choose your cloud provider:

Which Cloud Provider to choose? Confused? Watch the video below and you should be good Video link

Cloud provider	Certification type	Certification	Tutorials
AWS	Foundation Associate	• AWS Certified Cloud Practitioner • AWS Solutions Architect Associate
Azure	Foundation Associate	• AZ-900 Azure Fundamentals • AZ-104 Azure Administrator	Playlist
GCP	Foundation Associate	• Google Cloud Digital Leader • Associate Cloud Engineer	Playlist

Note: Got more questions about the Cloud/DevOps Certifications? Watch the video below:

Video Link

Phase 4: DevOps Tools (3-4 months)

1. Containerization (Docker)

# Docker Fundamentals
├── Container Basics
├── Dockerfile Creation, Multi-Stage Builds
├── Image Management
├── Docker Compose
├── Container Networking
├── Docker Best Practices
└── Volume Management

Checkout the free YouTube playlist 👉 here

2. Container Orchestration (Kubernetes)

Checkout the free YouTube playlist 👉 here

3. CI/CD Pipeline

Jenkins ( Checkout the free Youtube video here)
Modern CI/CD Tools(at least one from below)
- GitHub Actions
- Azure DevOps (Checkout the free Youtube Playlist here)
- GitLab CI/CD (Checkout the free Youtube video here)

4. Infrastructure as Code

# Terraform
├── HCL Syntax
├── Resource Management
├── State Management
├── Modules
└── Cloud Provider Integration

Checkout the free Terraform playlist here

5. Monitoring & Logging

Prometheus & Grafana
ELK Stack
Fluentd

Phase 5: 🛠️ Projects

Check out the Repo here for #10weeksofcloudops projects

📚 Learning Resources

Official Documentation

YouTube Channels

Tech Tutorials With Piyush

Practice Platforms

Made with ❤️ by [Piyush Sachdeva]
Last updated: December 2024

Top 3 Cloud and DevOps Projects To Supercharge Your Resume

piyushsachdeva — Wed, 08 May 2024 05:24:54 +0000

Introduction

In this blog, we'll be discussing three amazing DevOps and Cloud projects that you should have in your resume, and when I say have in your resume, it doesn't mean you add in your resume. It means that you should Implement those projects by yourself, get the learning out of it, create some useful artifacts like blogs or GitHub repositories, and then you should add them to your resume so that the recruiters/hiring managers will know that these projects you have implemented by yourself.

If you prefer watching a 10-minute video with all the details, you can check the below link; else, you can continue with the blog:

What can you expect from this blog?

In addition to sharing project ideas and steps, I will provide a complete end-to-end solution through blog posts, GitHub repositories, and YouTube videos. Anyone interested in implementing the solution can learn how to do so using these resources as guidance.

Project 1: Static website hosting and CICD

Difficulty Level: Beginner
Focus Area: Cloud Storage, CDN and CICD

Host a static website on the cloud of your choice, either AWS, Azure, or GCP, and Implement cicd on that.

Technologies Covered 📚 ( Use either of the below options)

AWS: Amazon S3, CloudFront, and Route 53
Azure: Azure Storage, Azure CDN, and DNS Management
Google Cloud: Cloud Storage, Load Balancing, and Content Delivery Networks (CDN)

Get Started 🚀

Review the project requirements. ✔️
Dive into AWS, Azure, or GCP documentation to familiarize yourself with the services mentioned. 📖
Start building your architecture diagram. 🏗️
Document your progress and implementation steps in a blog or GitHub Readme. 📝
Challenges Faced: Discuss any challenges you encountered and how you overcame them. 🤔
Key Takeaways: Share what you learned from this project. 🧐
Resources: List any helpful resources or references you used. 📚

Reference resources: ✅

💡 If you are an absolute beginner to the cloud and CICD, get yourself. Familiarize with the concepts, you can refer to the below documentation and study material:

For AWS
https://docs.aws.amazon.com/AmazonS3/latest/userguide/HostingWebsiteOnS3Setup.html
https://docs.aws.amazon.com/AmazonS3/latest/userguide/website-hosting-cloudfront-walkthrough.html

For Azure

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-static-website-how-to?tabs=azure-portal
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-custom-domain-name?tabs=azure-portal

For GCP
https://cloud.google.com/storage/docs/hosting-static-website
https://www.cloudskillsboost.google/focuses/1251?parent=catalog

If you are a visual learner, feel free to check out the video solution

Architecture Diagram for AWS by Ankit

Blog containing step-by-step instructions for AWS

Blog by Ankit

Architecture Diagram for Azure by Nishant

Blog By Nishant containing the step-by-step instructions for Azure

Blog by Nishant

Project #2: 3-tier architecture

Difficulty Level: Intermediate
Focus Area: Cloud Infrastructure, Networking, 3-tier application

For this project, your objective is to design a 3-tier architecture in a cloud platform, such as AWS, Azure, or GCP, with a focus on the following key considerations:

High Availability: Ensure your architecture is highly available, capable of withstanding failures, and can provide uninterrupted service.
Scalability: Design your architecture to be highly scalable, allowing for easy and efficient resource scaling as your application's demands increase.
Fault Tolerance: Implement fault-tolerant mechanisms to minimize downtime and service interruptions during failures.
Custom VPC/VNET: Consider using a custom Virtual Private Cloud (VPC) for AWS or GCP, a Virtual Network (VNET) for Azure, or a similar network customization instead of relying on the default configurations.
Security: Prioritize security by adhering to best practices for Identity and Access Management (IAM) and implementing robust security measures.

Your design should consider these considerations, resulting in a well-structured, efficient, and secure 3-tier architecture.
You can use your chosen cloud platform to implement this architecture effectively.

Reference resources: ✅

💡 To ensure you're ready to take on this challenge, it's essential to have a solid understanding of networking concepts. Check out the following resources for guidance:

👉 AWS: Click Here

👉 Azure: Click Here

👉 GCP: Click Here

🚀IP address calculation/CIDR and Subnet Masks: Click here

If you are a video person, feel free to check out the below video for end to end solution:👇

Architecture Diagram for AWS

Image Source

Architecture Diagram for Azure

Image Source

Detailed workshop for AWS with step-by-step instructions:

Workshop link

Detailed blog for Azure for step-by-step instructions

Blog by Nishant Singh

Project 3: Implement a 2-tier architecture in AWS, Azure, or GCP using Terraform 🚀

Difficulty Level: Intermediate
Focus Area: Iac using Terraform, Custom modules, Infrastructure.

This project aims to leverage the best practices of infrastructure as code (IaC) to create a reusable and shareable infrastructure setup. Our focus is on promoting modularity, flexibility, and maintainability.

Key Guidelines

1. Leverage Custom Modules

Build custom modules to break your infrastructure code into reusable and shareable components. This approach organizes your code and allows other team members to incorporate and adapt the components easily for their specific needs.

2. Use Variables and Data Sources

Please implement variables and data sources in your IaC code to enhance flexibility and maintainability. Variables make it easier to adapt and modify configuration settings, while data sources allow you to retrieve information from external sources to inform your infrastructure.

3. Remote State File

You can store your state file remotely. This practice enhances collaboration, security, and version control of your IaC code. Consider using your infrastructure's remote state storage service, such as Terraform Cloud or AWS S3.

4. Security First

Keep security in mind throughout your IaC development. Ensure your infrastructure is configured with appropriate security measures and adhere to best practices for secure and compliant deployments.

If you are a visual learner, feel free to check out the video solution

Repository containing the code for AWS Terraform 👉Click here👈

Architecture Diagram for AWS by Mahesh Upreti

Blog containing step-by-step instructions for AWS

Blog by Mahesh Upreti

Architecture Diagram for Azure by Joel

Blog containing the step-by-step instructions for Azure

Blog by Joel

AWS Networking - AWS VPC, Subnets, Security Groups, NAT Gateway & IP Addresses

piyushsachdeva — Wed, 22 Feb 2023 21:21:59 +0000

AWS Networking can be a complicated topic, but it's an essential part of building and managing resources on the AWS Cloud. In this comprehensive guide, we'll take a deep dive into the key components of AWS Networking and explore everything you need to know to get started.

let the game begins...

Virtual Private Clouds (VPCs)
At the heart of AWS Networking is the Virtual Private Cloud (VPC). A VPC is a virtual network that enables you to launch AWS resources into a virtual network that you define.It provides you with complete control over your virtual networking environment, including the selection of IP address ranges, subnets, and configuration of route tables and network gateways.

Let's look at each of these component in detail.

Resources that you create in AWS resides pysically in one or more Data Centres usually 100 miles apart from each other.
Collection of multiple data centres are referred to as an Availability Zone such as ca-central-1a , ca-central1b.
Collection of multiple Availability zones in a geographical location is referred to as an *AWS Region such as ca-central-1.

Below diagram shows an AWS Region(ca-central-1) that consists of two Availability Zones (ca-central-1a and ca-central-1b) that are part of VPC A with the CIDR range(10.0.0.0/16)

Subnets

A subnetwork or subnet is a logical subdivision of an IP network.
It further divides a VPC into multiple small networks so that they can be managed seperately.
The practice of dividing a network into two or more networks is called subnetting.

For example, a VPC having 10.0.0.0/16 = 65,536 IPs can be broken down into 4 subnets:

10.0.1.0/24 = 256 IPs
10.0.2.0/24 = 256 IPs
10.0.3.0/24 = 256 IPs
10.0.4.0/24 = 256 IPs

5 IPs per CIDR are reserved by AWS and rest of them will be available for further use.

Types of Subnets:
There are two types of subnets: Private and Public.
Public Subnets: If you want your instance in a public subnet to communicate with the internet then you use public subnet. Generally, web facing instances are placed in Public subnets.

Private Subnet: If a subnet doesn't have a route to the internet gateway, the subnet is known as a private subnet.
Generally, your DB servers are places in private subnets.

In the below diagram, we have added 1 Public and 1 Private subnet in each of the Availability zones.

Internet Gateway

Internet Gateway allows communication between your VPC and the internet. Only one IGW can be attached to one VPC and vice-versa.

In the below diagram, we have attached Internet Gateway to the VPC.

Internet gateway itself doesn’t provide access to the internet
Route table must be associated with the subnets and routes should be defined.

A route table contains a set of rules, called routes, that are used to determine where network traffic from your subnet or gateway is directed.

Public Subnet should have a route to the internet gateway while , Private subnet should have a route to the local network. As shown in the below diagram:

After attaching the route tables with the subnets, our diagram will look something like below

Now, lets talk about another important feature of VPC:
Security Groups

A security group acts as a virtual firewall for your EC2 instances to control incoming and outgoing traffic.
Inbound rules control the incoming traffic to your instance, and outbound rules control the outgoing traffic from your instance.
In the rules, you define, what type of traffic is allowed/denied from which source. e.g allow http traffic from 0.0.0.0 to the EC2.

Once you attach these security groups with your EC2 instances, the diagram will look something like below:

NACL(Network Access Control List) :
Like Security groups acts as a firewall on EC2/host level, NACL or Network Access control list acts as an additional layer of firewall on subnet level.

Default NACL allows all inbound and outbound traffic to your subnets.You can create a custom network ACL and associate it with a subnet.
A network ACL contains rules and a priority assigned to each rule, rules are evaluated based
on their priority, lower the number, higher the priority.

Below is a sample NACL Rule:

After adding the NACL rules to your subnets, our diagram will look something like below:

Now, there are a lot of usecases where your instances in private subnet needs access to the internet. For instance, Database instance needs regular updates/patching to be done by downloading updates from the internet. This can be done securely using NAT Gateway which allows instances in the private subnet to connect to the internet via a secure route.

Nat Gateways should be launched in Public Subnets (One per AZ). Something like below:

Conclusion
AWS Networking can be a complicated topic, but by following best practices and using the tools provided by AWS, you can build a secure and efficient network for your resources.

Want to see all the networking components with detailed explanation? Check out the below tutorial for the same:

References 📚:
https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html
https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html
https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html
https://aws.amazon.com/about-aws/global-infrastructure/
https://docs.aws.amazon.com/vpc/latest/userguide/default-vpc.html
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html

Boost Your Productivity with ChatGPT - A Tutorial for DevOps/Cloud

piyushsachdeva — Fri, 17 Feb 2023 03:03:09 +0000

If you’re an IT professional or a student, you must have come across the term *ChatGPT * at least once in the past few months. ChatGPT is an AI-based tool that allows users to have conversations and answer queries similar to that of a human being. The goal is to make it easier for people to communicate with computers and access information effortlessly. With its rising popularity, ChatGPT has become a vital tool in the field of DevOps and Cloud computing. In this tutorial, we will discuss how ChatGPT can help boost your productivity exponentially.

To get started, sign up for the free version of ChatGPT. The user-friendly interface is similar to that of a phone’s messaging application.

Simply visit the URL : https://chat.openai.com/ aand signup using your email account. Once logged in, it will look something like below:

you can ask ChatGPT to generate prompts, and it will provide you with a list of results. For example, if you’re a beginner in DevOps, you can ask ChatGPT to provide you with sample web applications that you could use in your CI/CD project.

As soon as you hit enter, it will generate the results similar to below:

If you’re a beginner and looking for more ideas, you can ask ChatGPT to generate more ideas by entering the prompt as below:

If you think that this is not ideal for a beginner in programming, you can enter another prompt to generate ideas for a beginner in coding as below:

Now that we have a lot of beginner friendly project ideas, let's pick one from the list and start working on that.For Instance, I have chosen : A simple calculator app.

You can prompt chatGPT to generate the code in any specific programming language or let it choose the language for you. Code generate by ChatGPT is not always 100% accurate but you can quickly fix that if you know code debugging a little bit. Below are the results:

Once you have the code, you can prompt ChatGPT to dockerize the web application.

It would have generated the dockerfile along with the steps to build the image and deploy the container.

Now you can take the usecase one step further and prompt ChatGPT to create a Jenkins CI CD pipeline using the above details.

The same way, it has generate the pipeline code along with all the prerequisites steps and details of each line of the code. Pretty Neat Eh!

You can perform a lot more tasks such as:

Generating documentation
Adding comments in your code using prompt
convert code from one language to another
generate readme file for your application code
troubleshooting the issue by pasting errors as a prompt and the usecases are numerous.

If you're interested in improving your productivity using ChatGPT, be sure to check out the full tutorial for DevOps/Cloud on YouTube! 🎥👨‍💻

Introducing the AWS Transfer Family Service

piyushsachdeva — Sun, 08 Jan 2023 04:26:12 +0000

Are you tired of slow, insecure data transfer processes? 💤 The AWS Transfer Family is here to help! 🚀

What is the AWS Transfer Family, you ask? It's a group of services that enables you to quickly and securely move data to and from the cloud. 🌩️ The family includes AWS Transfer for SFTP, AWS Transfer for FTP, and AWS Transfer for FTPS.

But what makes the AWS Transfer Family so great? 🤔 For starters, it's super easy to use. Setting up and managing your data transfer process is a breeze, thanks to the user-friendly interface and integration with other AWS services such as Amazon S3 and Amazon EC2. 💻 This saves you time and resources, so you can focus on running your business.

But that's not all! The AWS Transfer Family is also extremely secure. 🔒 All data transfer activities are encrypted, ensuring that your sensitive information is always protected. Plus, you have the option to enable multifactor authentication for an extra layer of security. 🛡️

But how does it all work? 🤔 It's actually quite simple. First, you choose the service that best fits your needs - SFTP, FTP, or FTPS. Next, you set up your server and configure your data transfer process, including the location of your Amazon S3 bucket. Finally, you can test your data transfer by uploading and downloading files to and from your S3 bucket. 📤📥

Checkout this cool animated video to see how it works!

Want to see the AWS Transfer Family in action? 💪 Check out our demo of how to set up an SFTP server and test the data transfer process:

And that's it! You've successfully set up an SFTP server using AWS Transfer Family and tested the data transfer process. 🎉 As you can see, it's a quick and easy way to move data to and from the cloud. 🌩️ So why not give the AWS Transfer Family a try today? 🚀

🙏Thank you for following along with the tutorial so far. If you found this blog to be helpful, please be sure to follow me and consider subscribing to my YouTube channel. Good luck 👍 on your cloud journey!

How To Install Multiple Gitlab Runners On a Single AWS EC2 machine

piyushsachdeva — Mon, 05 Dec 2022 17:19:52 +0000

In this quick tutorial, I am going to show you how you can install multiple Gitlab Runners on a single AWS EC2 machine.

Step 1

Provision a EC2 server

Give it a name such as Gitlab-Runner and select a suitable Linux Image. In this demo, I have selected an Ubuntu image

Then scroll down and select the key-pair that you would want to use for this instance or you can create a new key-pair. Once the key-pair is selected, you can create a new security group with the wizard and open access to port 22(SSH) from your local machine or a CIDR range and for your application server so that gitlab runner can ssh into your application server for code deployment. You can optionally use an existing security group as well.

Scroll down to the advance details section and keep everything as default except the user data section. Add the below user data script and hit Launch instance.



#!/bin/bash
sudo apt-get update -y
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

sudo apt-get install gitlab-runner -y

#install docker 
curl -fsSL https://get.docker.com -o get-docker.sh 
sudo bash get-docker.sh

cd /var/run/ 
sudo chmod 777 docker.pid docker.sock

Step 2

Register the Runner

Once server is running, login to the server using EC2 instance connect or ssh client such as Putty and enter the command

gitlab-runner register

then enter your gitlab running when prompted

Go to your Gitlab UI --> Settings --> CICD and Expand the section in front of Runners

Copy the registration token from here and paste the same in the terminal prompt

Enter the details as below to register the runner

Step 3

Start the Runner

Start the runner using the below command: sudo gitlab-runner start

once it is started, hit refresh on the Gitlab page and you would see the runner as active

Give yourself a pat on the back! you were able to perform the task successfully!

Step4

Register second runner(Optional)
Now , if you would like to register multiple runners on the same ec2 server, using can do so by following the same method. Give a different name and tag to the runner as follows

Now, refresh the page to see the new runner as well

You can use the runners based on the tags or the names.

That's all folks. If you have enjoyed the blog and learned something new today. Feel free to hit the like button and follow me for more such content.

I have also published a full end to end tutorial for Gitlab CI CD Pipeline, feel free to check that out using the below link

How To Mount AWS S3 Bucket On Amazon EC2 Linux Using S3FS

piyushsachdeva — Wed, 31 Aug 2022 17:16:21 +0000

In this blog, we will learn how to use S3 as a filesystem on EC2 Linux machine.

Let's start!

1) Create an EC2 Linux (I have used Ubuntu in this demo) instance

Keep everything as default and add the below user data script to install awscli and s3fs utlity from advance section of wizard



sudo apt-get update -y
sudo apt-get install awscli -y
sudo apt-get install s3fs -y

2) Create an IAM user for s3fs

3) Give the user a unique name and enable programmatic access

Set permission --> create a new policy

Select the service as S3 and include below access levels

Give the policy a unique name and click Create policy

Once the policy is created, go back to the IAM tab and hit refresh so that newly created policy is included in the list
, filter by policy name and hit the enable checkbox to add the policy to our IAM user.

Hit create user

Once the user is created, download the credentials. We are going to use it later.

4) Login to your Ec2 Instance

Go to your home directory and run below commands to create a new directory and to generate some sample files



mkdir /home/ubuntu/bucket; cd $HOME/bucket ;touch test1.txt test2.txt test3.txt

Next step is to create an S3 bucket.
5) Go to S3 service and create a new bucket

give it a unique name and leave reast of the settings as default.
Block public access to this bucket should be enabled by default

Hit create bucket.

6) Once the bucket is created, go to the ssh session and configure our AWS credentails for authentication using the IAM account that we have created.

Use the command



aws configure

and provide the credential details that we have downloaded before

7) Now run the below command to sync local directory with the S3 bucket



aws s3 sync path_on_filesystem s3://bucketname

For example,



aws s3 sync /home/ubuntu/bucket s3://test-s3fs-101

8) create the credential file for s3fs

s3fs supports the standard AWS credentials file stored in ${HOME}/.aws/credentials. Alternatively, s3fs supports a custom passwd file.
The default location for the s3fs password file can be created:
using a .passwd-s3fs file in the users home directory (i.e. ${HOME}/.passwd-s3fs)

file should have the below content:



$AWS_ACCESS_KEY_ID:$AWS_SECRET_KEY_ID

You can run the below command as well:



echo "AKIAQSCIQUH6XXYQMGDA:T5qM7rZmSaU3p/Y0xmuZyWv1/KUnT0Oc58sdCJ3t" > ${HOME}/.passwd-s3fs;
chmod 600 ${HOME}/.passwd-s3fs

9) Now you can run the command to mount S3 bucket as a filesystem



sudo s3fs bucketname path  -o passwd_file=$HOME/.passwd-s3fs,nonempty,rw,allow_other,mp_umask=002,uid=$UID,gid=$UID -o url=http://s3.aws-region.amazonaws.com
,endpoint=aws-region1,use_path_request_style

for example:



sudo s3fs s3fs-test-101 /home/ubuntu/bucket  -o passwd_file=$HOME/.passwd-s3fs,nonempty,rw,allow_other,mp_umask=002,uid=1000,gid=1000 -o url=http://s3.ca-central-1.amazonaws.com
,endpoint=ca-central-1,use_path_request_style

10) Once it is mounted successfully, you can verify by running the command



mount|grep s3fs

11) Add the entry in fstab using the below command so that the changes become persistent after the server reboot as well:



bucketname directoryonfs fuse.s3fs _netdev,allow_other 0 0

For example:



s3fs-test-101 /home/ubuntu/bucket fuse.s3fs _netdev,allow_other 0 0

12) Now the moment of truth, go to your S3 bucket and hit refresh, you should see the files that were present in your file system

13) Let's now verify whether it's getting synced properly after a object delete/addition

Go to your S3 bucket, and upload a new file

Go to your ssh session and do ls in the same directory

Eureka! The file that you just uploaded in your S3 bucket appears in your FileSystem.

Same way you can test the delete file operation. And it works both ways i.e if you perform any file operation on your filesystem, it will sync to your S3 bucket as well.

Feel free to checkout the below hands-on demo of what we have learned so far:

References:
https://github.com/s3fs-fuse/s3fs-fuse
https://aws.amazon.com/
https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

Limitations

Generally S3 cannot offer the same performance or semantics as a local file system. More specifically:

random writes or appends to files require rewriting the entire object, optimized with multi-part upload copy
metadata operations such as listing directories have poor performance due to network latency
non-AWS providers may have eventual consistency so reads can temporarily yield stale data (AWS offers read-after-write consistency since Dec 2020)
no atomic renames of files or directories
no coordination between multiple clients mounting the same bucket
no hard links inotify detects only local modifications, not external ones by other clients or tools

DEV Community: piyushsachdeva

Build an AI Observability Pipeline on EKS Using Elasticsearch

Table of contents

Architecture

Prerequisites

Step 1: Provision Elasticsearch from AWS Marketplace

Get your endpoints

Step 1b: Generate a Kibana API key

Step 2: Configure OpenTelemetry to ship Kubernetes logs

Step 2b: Verify logs are flowing in Kibana

Step 3: Index Git commit metadata via GitHub Actions

3a. Create the index

3b. Add GitHub repository secrets

Step 3c: The workflow file

Step 3d: Verify it works

Step 4: Set up Argo CD for GitOps deployment

Step 5: Build the AI diagnostic agent in Kibana

Step 5a: LLM setup

Step 5b: Create the ES|QL tools

Step 5c: Create the agent

Step 6: Simulate and diagnose a crash

Introduce the bad commit

Ask the agent

Fix it

Why this works

Quick reference

Commands you will reuse

ES|QL: find OOMKill events

ES|QL: find recent deploys

FAQs

References

Deploy Your Own 24/7 AI Agent on AWS EC2 with Docker & Tailscale (The Secure Way)

What You'll Have by the End

Prerequisites

Cloud Infrastructure

API Keys

Networking

Phase 1: VM Hardening (Do This FIRST)

Step 1: Create a New User

Step 2: Set Up SSH Keys for the New User

Step 3: Configure AWS Security Group (External Firewall)

Step 4: Configure UFW (Internal Firewall)

Step 5: The Golden Test

Step 6: Lock Down SSH Configuration

Disable Systemd Socket Activation (CRITICAL)

Step 7: Final Verification & Cleanup

Phase 2: Environment Setup

1. Install Docker & Docker Compose

2. Install Tailscale (The Secure Access Layer)

Phase 3: OpenClaw Installation

1. Clone the Repository

2. Clean Up Old Configurations

3. Pre-Create the Configuration Folder

4. Run the Setup Script

5. The "Insecure Auth" Injection

Phase 4: Login & Verify

1. Check Container Status

2. Get Your Tailscale IP

3. Access the Web Interface

Phase 5: Best Practices & Maintenance

Set API Spend Limits

Enable Sandbox Mode

Back Up Your Bot's Memory

Monitor Logs

Conclusion

DevOps and Cloud Engineering Roadmap 2025 🚀

🚀 DevOps and Cloud Engineering Roadmap 2025

📋 Table of Contents

🎯 About

📚 Learning Path

Phase 1: Foundation (2-3 months)

Linux and Shell Scripting

Programming Foundation

Version Control

Phase 2: Core IT Skills (1 month)

Network Fundamentals

Phase 3: Cloud Computing (2-3 months)

Phase 4: DevOps Tools (3-4 months)

1. Containerization (Docker)

2. Container Orchestration (Kubernetes)