DEV Community: SHIVAM UPADHYAY

Testing CAST AI on GKE: A Hands-On Kubernetes Workload Optimization Lab

SHIVAM UPADHYAY — Fri, 31 Jul 2026 12:38:02 +0000

I intentionally over-provisioned a Kubernetes application just to see whether CAST AI could detect it, recommend better resource requests, and actually optimize it.

Spoiler: it did—but the biggest lesson wasn't about cost savings. It was about understanding the difference between resource requests, actual usage, and infrastructure cost.

🤔 Why I Built This Lab

When we deploy applications to Kubernetes, we usually write something like this:

resources:
requests:
cpu: "1000m"
memory: "1Gi"

limits:
cpu: "1500m"
memory: "1500Mi"

But here's the question:

How do you know if those numbers are correct?

Most of us guess.

Sometimes we copy them from another project.

Sometimes we simply over-provision because we don't want the application to crash.

That's exactly what I did for this experiment.

🏗️ The Lab Setup

I built a very small application instead of using a complex microservice.

Tech Stack

FastAPI
Docker
Google Kubernetes Engine (GKE)
Google Artifact Registry
CAST AI

The deployment flow looked like this:

FastAPI
│
Docker Image
│
Artifact Registry
│
GKE Cluster
│
Deployment (2 Pods)
│
ClusterIP Service
│
CAST AI

☕ The Coffee API

The application only exposes two endpoints.

@app.get("/")
def home():
return {
"message": "Coffee Shop API",
"hostname": socket.gethostname()
}

@app.get("/coffee")
def coffee():
return {
"coffee": "Cappuccino",
"price": 120
}

Nothing fancy.

The goal wasn't the application.

The goal was the infrastructure.

🎯 Intentionally Over-Provisioning the Pods

Each Pod requested:

resources:
requests:
cpu: "1000m"
memory: "1Gi"

and I deployed 2 replicas.

So Kubernetes reserved:

2 CPU
2 GiB Memory

for an application that barely did anything.

🚀 Deploying to GKE

After building the Docker image, I pushed it to Artifact Registry and deployed it to GKE.

Everything looked healthy.

2 Pods
Running
Ready

At this point, Kubernetes had no idea the application was oversized.

It simply respected the resource requests I had defined.

🔍 Connecting CAST AI

I connected the cluster to CAST AI.

Interestingly, onboarding wasn't completely smooth.

I ran into several issues:

missing gke-gcloud-auth-plugin
expired CAST AI authentication
missing IAM permissions
Kubernetes RBAC

After fixing those, CAST AI started collecting workload metrics.

📊 The Interesting Part

After some time, CAST AI analyzed my workload.

My deployment requested:

CPU
1 CPU per Pod

Memory
1 GiB per Pod

But actual usage looked more like:

CPU
≈ 3-4m

Memory
≈ 30 MiB

That was a huge difference.

💡 CAST AI Recommendation

Instead of recommending something extreme like 4m CPU, CAST AI suggested:

Resource Before Recommendation
CPU 1 CPU 750m
Memory 1 GiB 768 MiB

It also estimated around 25% workload cost reduction.

Screenshot

(Insert your CAST AI optimization screenshot here)

🤔 Why Not Just Recommend 4m CPU?

This was actually my favorite learning.

If the application is only using 4m CPU...

...why recommend 750m?

Because optimization isn't simply:

request = current usage

Production workloads experience:

startup spikes
traffic bursts
background jobs
unpredictable load

A good optimizer leaves enough headroom for those situations.

✅ Verifying It From Kubernetes

I didn't want to trust only the dashboard.

So I verified everything directly from Kubernetes.

kubectl get pod \
-o jsonpath='{.status.containerStatuses[0].allocatedResources}'

Result:

{
"cpu":"750m",
"memory":"768Mi"
}

That was pretty satisfying.

🧠 Biggest Takeaways

This experiment completely changed how I think about Kubernetes resources.

I learned that these are not the same thing:

Requested Resources

↓

Actual Usage

↓

Recommended Resources

↓

Allocated Resources

↓

Infrastructure Cost

Those five numbers often get mixed together.

Understanding the difference is the first step toward Kubernetes cost optimization.

Final Thoughts

Before this project I thought Kubernetes cost optimization mostly meant:

"Find cheaper virtual machines."

Now I'd describe it differently:

Start by making sure your workloads request only the resources they actually need.

Node optimization comes later.

Workload optimization comes first.

📚 Project Repository

I documented the complete lab including:

GKE setup
CAST AI onboarding
Docker deployment
Kubernetes networking
Cost monitoring
Workload optimization
Troubleshooting

👉 GitHub:
https://github.com/Upshivam786/castai-gke-workload-optimization-lab

🙌 Thanks for Reading

If you've experimented with Kubernetes cost optimization tools like CAST AI, Karpenter, Goldilocks, or Vertical Pod Autoscaler, I'd love to hear about your experience.

Happy learning! 🚀

Tags:

kubernetes #gke #devops #docker #googlecloud #finops #mlops #castai

I Built an AI That Designs MLOps Infrastructure (Without Letting AI Generate YAML)

SHIVAM UPADHYAY — Thu, 09 Jul 2026 04:46:18 +0000

If you've ever started an AI project, you've probably asked yourself questions like:

Should I deploy on Kubernetes or Cloud Run?
Do I need Redis?
Which database fits this workload?
How should I structure CI/CD?
What monitoring stack should I use?

Choosing an architecture is often harder than writing the application itself.

I wanted to see if AI could help with those decisions while keeping the generated infrastructure deterministic and reliable.

That idea became DeployCraft AI.

The Goal

The application asks for a few high-level inputs:

Application Framework
Cloud Provider
Deployment Target
Database
Vector Database
Cache
Monitoring
Authentication
Traffic
Budget
High Availability

Instead of generating code from a prompt, it builds a structured deployment blueprint.

The output includes:

✅ Architecture reasoning
✅ Architecture score
✅ Mermaid diagram
✅ Docker Compose / Kubernetes / Cloud Run manifests
✅ GitHub Actions workflow
✅ Security report
✅ Monitoring recommendations
✅ Cost estimation
✅ Downloadable ZIP package
The Most Important Design Decision

One thing I intentionally didn't do:

I never let the LLM generate deployment manifests.

Instead, I separated the project into three layers.

                       Configuration
                           │
                           ▼
                      Rules Engine
                           │
                           ▼
                         Blueprint
                     ┌───────────────┐
                     │               │
                     ▼               ▼
                  Generators     AI Advisor
                     │               │
                     ▼               ▼
                    YAML          Explanations

Everything infrastructure-related is deterministic.

The rules layer decides:

which services exist
networking
environment variables
autoscaling
storage
deployment topology

No AI involved.

Template-Based Infrastructure

Once the blueprint is built, Jinja2 templates generate:

Docker Compose
Kubernetes manifests
Cloud Run YAML
GitHub Actions workflow

This means every deployment file is reproducible.

AI Only Explains

The LLM receives the architecture and answers questions like:

Why Cloud Run?
Why PostgreSQL?
Why Redis?
What are the risks?
How could this scale?

The AI becomes an architecture reviewer rather than an infrastructure generator.

I found this separation makes the outputs much more trustworthy.

Some Interesting Challenges

A few things that took more work than expected:

Supporting multiple deployment targets

Each target needs different templates while sharing the same architecture model.

Making AI output reliable

LLMs occasionally fail or return unexpected responses.

Instead of crashing the application, I implemented graceful fallbacks so the UI always remains usable.

Architecture scoring

Rather than asking AI for a score, the application evaluates:

Security
Scalability
Reliability
Cost Efficiency
Observability

using deterministic rules.

Packaging everything

Instead of displaying lots of text, the application bundles every generated artifact into a downloadable ZIP.

Tech Stack
Python
Gradio
Hugging Face Inference API
Jinja2
Docker
Kubernetes
Google Cloud Run
GitHub Actions
What I Learned

This project reinforced something I've been thinking about for a while:

AI doesn't need to replace software engineering.

It becomes much more useful when it's responsible for reasoning, while deterministic code remains responsible for correctness.

That balance made the application significantly more reliable.

Try It

🌐 Live Demo: https://huggingface.co/spaces/Upshivam786/deploycraft-ai

💻 GitHub: https://github.com/Upshivam786/DeployCraft-AI/tree/main

I'd appreciate any feedback or suggestions from people working in DevOps, Cloud Engineering, or MLOps.

Building an AI Cloud Cost Intelligence Platform That Doesn't Let AI Make Infrastructure Decisions

SHIVAM UPADHYAY — Mon, 29 Jun 2026 04:11:00 +0000

Most AI-powered cloud optimization demos follow a simple approach:

Azure Resources
      ↓
Large Language Model
      ↓
Recommendations

At first glance, this seems impressive.

But while building my own Azure Cost Intelligence Platform, I ran into a problem that completely changed my architecture.

The AI was generating infrastructure recommendations that looked correct—but some of them simply didn't exist.

For example, it suggested VM SKUs that Azure doesn't support and even produced Azure CLI commands with incorrect parameters. That was a wake-up call.

Instead of trying to "prompt engineer" my way out of the problem, I redesigned the application.

The New Architecture

Rather than letting AI make decisions, I split the system into independent components:

Azure Scanner → Discovers Azure resources.
Metrics Engine → Collects real CPU utilization from Azure Monitor.
Pricing Engine → Retrieves actual VM pricing.
Azure Advisor Integration → Fetches Microsoft's own optimization recommendations.
Recommendation Engine → Applies deterministic Python rules to decide what should happen.
Command Builder → Generates verified Azure CLI commands.
LLM → Explains why a recommendation matters in simple language.

The AI never decides what to do—it only explains decisions that have already been made.

Example

Instead of asking the LLM:

"What should I do with this VM?"

the backend first gathers facts:

Current VM SKU: Standard_B2ts_v2
Average CPU: 0.3%
Peak CPU: 1.9%
Current Cost: $1.64/month
Recommended Cost: $0.90/month

The recommendation engine then determines:

Resize VM
↓
Target SKU: Standard_B2ats_v2
↓
Estimated Savings: $0.74/month

Finally, the LLM produces a human-friendly explanation like:

"This virtual machine is significantly underutilized. Resizing it can reduce monthly costs without affecting current workloads."

Notice that the AI never invents the recommendation or the command—it only explains verified data.

Tech Stack

FastAPI
React + TypeScript
Docker
Azure Container Apps
Azure Monitor
Azure Advisor
Azure Pricing APIs
Azure Managed Identity
PostgreSQL
Groq API (LLM)

What I Learned

This project taught me an important lesson:

AI should not replace engineering logic.

Cloud infrastructure decisions should come from verified APIs, business rules, and deterministic code. AI adds the most value when it helps engineers understand those decisions, not when it generates them.

That's the architecture I'll continue using as I expand this platform with historical cost trends, anomaly detection, and multi-cloud (AWS & GCP) support.

If you're building AI-assisted DevOps or FinOps tools, I'd love to hear how you're balancing automation with reliability.

Building a Self-Hosted MLOps Platform from Scratch with FastAPI, PostgreSQL, GCS, and Docker

SHIVAM UPADHYAY — Tue, 23 Jun 2026 09:25:20 +0000

Introduction

Over the past few months, I set out to answer a simple question:

What does it take to build a production-style MLOps platform from scratch?

While tools like MLflow, SageMaker Model Registry, and Kubeflow provide powerful capabilities, I wanted to understand the underlying architecture and engineering decisions behind them. Instead of only using existing platforms, I decided to build my own.

The result is Kimchi, a self-hosted MLOps platform that supports:

Model Registration
Model Versioning
Artifact Management
Experiment Tracking
Audit Logging
Role-Based Access Control (RBAC)
Python SDK
CI/CD Automation
Google Cloud Storage Integration

This project was built using:

FastAPI
PostgreSQL
SQLAlchemy
Alembic
Docker
Google Cloud Storage (GCS)
GitHub Actions
JWT Authentication

In this article, I'll share how the project evolved from a simple model registry into a multi-phase MLOps platform.

The Problem

In many machine learning projects, models are often stored in random locations:

Local machines
Shared drives
Cloud storage buckets
Team chat channels

Over time, teams start asking difficult questions:

Which model version is currently in production?
Who promoted this version?
Which dataset was used to train it?
What accuracy did it achieve?
Where is the model artifact stored?

Without a centralized platform, answering these questions becomes difficult.

I wanted a system that could act as a single source of truth for the ML lifecycle.

Phase 1 — Building the Foundation

The first phase focused on creating a Model Registry.

Key capabilities:

Model registration
Model versioning
Artifact upload and download
JWT Authentication
Role-Based Access Control
Google Cloud Storage integration

Architecture

The platform separates metadata from artifacts:

PostgreSQL stores model metadata
GCS stores model artifacts
FastAPI exposes REST APIs

This architecture mirrors how many production MLOps systems are designed.

Phase 2 — Governance Through Audit Logging

Once the registry was functional, I realized something important:

The platform had no memory.

If someone promoted a model to production or deleted a version, there was no way to know:

Who made the change
What changed
When it happened

To solve this, I implemented an Audit Logging system.

Every write operation now creates an immutable audit record:

CREATE
UPDATE
DELETE
PROMOTE

This allows teams to trace every important lifecycle event.

Example:

Shivam
PROMOTE
Model Version 5
staging → production
2026-06-16 05:01

This feature brought governance and accountability to the platform.

Phase 3 — Experiment Tracking

At this stage, models could be registered, but training information was still buried inside artifact files.

For example:

metrics.json
params.json

could be uploaded, but the platform couldn't answer:

Which model has the highest accuracy?
Which version achieved the best F1 score?
What hyperparameters were used?

To solve this, I introduced Experiment Tracking.

A new Training Run layer stores:

Hyperparameters
Metrics
Dataset hashes
Framework information
Training duration

This enables powerful queries such as:

GET /experiments?min_accuracy=0.90

and

GET /models/{id}/versions/compare

The platform now supports experiment search, filtering, and version comparison.

Phase 4 — Improving Developer Experience

After building the core platform, I shifted focus to usability.

Python SDK

Instead of interacting with the API using raw curl commands, users can now work with a Python SDK.

Example:

from kimchi_sdk import ModelRegistry

registry = ModelRegistry(
    url="http://localhost:8000",
    username="user",
    password="password"
)

registry.create_model(
    name="fraud-detector"
)

GitHub Actions CI/CD

Every push automatically triggers:

Dependency installation
Automated test execution
Validation checks

This ensures platform stability and catches regressions early.

Admin APIs

Role management is now handled through APIs instead of direct database modifications.

Refresh Tokens

Refresh tokens provide a smoother authentication experience and prepare the platform for future UI integrations.

Lessons Learned

Building this platform taught me much more than simply writing APIs.

Some of the key lessons included:

Designing for Evolution

Each phase built on top of the previous one.

Instead of trying to build everything at once, I focused on creating a strong foundation and then extending it incrementally.

Governance Matters

Tracking who changed what becomes increasingly important as teams grow.

Audit logging turned out to be one of the most valuable additions.

Experiment Metadata is First-Class Data

Metrics and hyperparameters should not live only inside artifact files.

Making them queryable dramatically improves usability.

Developer Experience Matters

Features like SDKs and CI/CD pipelines are often overlooked but are critical for adoption.

What's Next?

The next phase focuses on Governance and Enterprise Readiness:

Signed GCS URLs
Model Cards
Dataset Lineage
Drift Monitoring
Kubernetes Deployments
Terraform Infrastructure

The goal is to continue evolving Kimchi into a production-grade MLOps platform.

Final Thoughts

This project started as an exercise to understand how model registries work internally.

Over time, it evolved into a hands-on exploration of:

MLOps
Backend Engineering
Cloud Infrastructure
Governance
Experiment Tracking
Developer Experience

Building systems from scratch is one of the best ways to understand the trade-offs behind production software.

If you're interested in MLOps, backend engineering, or cloud-native systems, I highly recommend building something similar yourself.

Project Repository

GitHub Repository:

[Add your GitHub repository URL here]

If you have suggestions, feedback, or ideas for future phases, feel free to connect with me.

🎉 Happy New Year, Devs! 🎉

SHIVAM UPADHYAY — Fri, 02 Jan 2026 05:15:22 +0000

Grateful for another year of learning, building, breaking, and fixing things.
Looking forward to writing cleaner code, learning new tools, and growing together as a community in 2026.

Wishing you all successful builds, fewer bugs, and lots of innovation ahead 🚀
Happy New Year!

Completed Linux for DevOps Beginners 🚀 #devops #linux #learninginpublic #resources

SHIVAM UPADHYAY — Thu, 24 Jul 2025 16:13:49 +0000

Hey everyone! 👋

Just wanted to share a small win — I’ve completed my first Linux course as part of my DevOps journey. 🐧
I started with the basics: terminal commands, file management, permissions, and a bit of scripting.

Why Linux?
Because it’s everywhere in DevOps — servers, containers, cloud. It’s like the home base, so I knew I had to start here.

If you’re also starting out, here are a few free resources I found super helpful:
🔹 The Linux Command Line book — linuxcommand.org/tlcl.php
🔹 Linux Journey (fun and simple) — linuxjourney.com
🔹 FreeCodeCamp YouTube video — search "Linux crash course freecodecamp"
🔹 Codecademy CLI basics — codecademy.com
🔹 Ubuntu command line guide — ubuntu.com/tutorials/command-line-for-beginners

Next, I’m going to explore Git and shell scripting. Excited to keep learning and building step by step.

If you're also learning DevOps, let’s connect. We can share tips and support each other. This is all part of learning in public. :)

DevOpsJourney #LinuxBasics #KeepLearning #100DaysOfDevOps

My First Step into DevOps: Linux Basics

SHIVAM UPADHYAY — Thu, 17 Jul 2025 12:34:37 +0000

Hey Dev Community! 👋

I'm excited to kick off my journey toward becoming a DevOps Engineer.
Step 1: I'm diving into Linux for Beginners — building the foundation brick by brick. 🧱

I'll be sharing my progress, learnings, and challenges here.
If you're on a similar path or have tips to share, let's connect and grow together!

DevOps #Linux #BeginnerJourney #LearningInPublic