DEV Community: Shuvojit Kar

Applicant and Candidate Privacy Policy

Shuvojit Kar — Tue, 05 Aug 2025 04:02:13 +0000

This policy explains:

What information we collect during our application and recruitment process and why we collect it;
How we use that information; and
How to access and update that information.
Your use of Google services is governed by any applicable terms and our general Privacy Policy.

Types of information we collect
This policy covers the information you share with us and/or which may be acquired or produced by Google LLC, its subsidiaries and its affiliates during the application or recruitment process including:

Your name, address, email address, telephone number and other contact information;
Your resume or CV, cover letter, previous and/or relevant work experience or other experience, education, transcripts, or other information you provide to us in support of an application and/or the application and recruitment process;
Information from interviews and phone-screenings you may have, if any;
Details of the type of employment you are or may be looking for, current and/or desired salary and other terms relating to compensation and benefits packages, willingness to relocate, or other job preferences;
Details of how you heard about the position you are applying for;
Any sensitive and/or demographic information processed during the application or recruitment process such as gender, information about your citizenship and/or nationality, medical or health information and/or your racial or ethnic origin;
Reference information and/or information received from background checks (where applicable), including information provided by third parties;
Information relating to any previous applications you may have made to Google and/or any previous employment history with Google;
Your information from publicly available sources, including online, that we believe is relevant to your application or a potential future application (e.g. your LinkedIn profile); and/or
Information related to any assessment you may take as part of the interview screening process.
How we use information we collect
Your information will be used by Google for the purposes of carrying out its application and recruitment process which includes:

Assessing your skills, qualifications and interests against our career opportunities;
Verifying your information and carrying out reference checks and/or conducting background checks (where applicable) if you are offered a job;
Communications with you about the recruitment process and/or your application(s), including, in appropriate cases, informing you of other potential career opportunities at Google;
Creating and/or submitting reports as required under any local laws and/or regulations, where applicable;
Where requested by you, assisting you with obtaining an immigration visa or work permit where required;
Making improvements to Google’s application and/or recruitment process;
Complying with applicable laws, regulations, legal processes or enforceable governmental requests; and/or
Proactively conducting research about your educational and professional background and skills and contacting you if we think you would be suitable for a role with us.
We may process information regarding your demographics as described above where we are required to do so to meet our legal and contractual obligations, such as reporting obligations to authorities and third parties and other similar requirements.
Google may also use your information to help develop, train, fine-tune, and improve machine learning models and foundational technologies that power various Google products and tools, including internal tools and where it is in our legitimate interests to do so.

We will also use your information to protect the rights and property of Google, our users, applicants, candidates, employees or the public as required or permitted by law.

If you are offered and accept employment with Google, the information collected during the application and recruitment process will become part of your employment record.

Internal Developer Platform Automation

Shuvojit Kar — Tue, 05 Aug 2025 02:42:11 +0000

Tech Stack: Go | Kubernetes | Ansible | Jenkins

As part of our DevOps transformation journey, I led the design and development of an internal developer platform (IDP) to streamline application onboarding, deployment, and environment provisioning. The goal: empower developers to self-serve infrastructure and deploy faster without waiting on ops teams.

Key
✅ Automated Application Deployment
Developed internal tooling in Go to automate application deployment workflows, reducing manual intervention and eliminating configuration drift. Integrated tightly with Jenkins pipelines for CI/CD automation.

✅ Standardized Onboarding Pipelines
Created reusable templates and a standardized onboarding framework, enabling teams to deploy microservices in a consistent and repeatable manner across staging and production environments.

✅ Optimized Provisioning with IaC
Leveraged Ansible and Kubernetes manifests to automate infrastructure provisioning. Enabled rapid spin-up of isolated dev/test environments using predefined IaC templates, cutting environment setup time by 60%.

📈 Impact
Reduced deployment lead time from hours to under 15 minutes
Unified tooling improved developer satisfaction and decreased handoffs
Enabled faster onboarding for new services—down from days to a few hours

IBM RAG and Agentic AI make a book

Shuvojit Kar — Fri, 01 Aug 2025 03:20:30 +0000

IBM’s Guidance and Official Materials
IBM’s RAG Cookbook: IBM offers a comprehensive "RAG Cookbook," designed as a practical guide for technical professionals seeking to build or optimize RAG-based AI pipelines. This resource covers architectures, best practices, and integration tips for deploying RAG solutions in enterprise environments. While RAG and Agentic AI are closely linked in advanced AI system designs, IBM typically addresses them as separate topics: practical RAG solutions and the architectural/topical aspects of Agentic AI. The Cookbook steers users to IBM's Agent Development Portal for more in-depth work on agentic systems.

Agentic AI with IBM: IBM defines Agentic AI as the construction of autonomous AI agents capable of carrying out specific goals with minimal supervision, often by combining retrieval-augmented techniques with agent workflows. IBM’s online resources provide deep dives into both topics, making them accessible to AI practitioners and business leaders alike.

Professional Certification: IBM offers a "RAG and Agentic AI Professional Certificate" through Coursera, guiding learners through RAG pipeline building, multimodal integration, and agent orchestration—ideal for those looking to develop production-grade, agentic AI solutions.

Books Covering RAG and Agentic AI (2024–2025)
Agentic AI with RAG in Action by Ronald Taylor (2025)

This book delivers a hands-on guide to building, scaling, and deploying autonomous AI agents using RAG. Readers learn practical prompt engineering, workflow design, case studies, and ethics for deploying agentic systems with retrieval-augmented backends. It blends technical details (code samples, architecture diagrams) with real-world application strategies, making it popular among AI developers and digital innovators.

Agentic AI System Leveraging RAG by Jerry Canter (2025)

Focused on practical frameworks, this guide integrates agentic system design with RAG 2.0, including cognitive architectures, advanced retrieval strategies, and real-time data handling. It addresses AI ethics, scalability, and implementation challenges, all through a lens of industrial use cases.

Agentic Artificial Intelligence: Harnessing AI Agents to Reinvent Business, Work, and Life by Pascal Bornet, Jochen Wirtz, and Thomas H Davenport (2025)

While not exclusively IBM-centric, this influential business guide demystifies agentic AI. It surveys hands-on deployments across major enterprises (including those using IBM systems), explains how agentic AI transforms industries, and offers strategic roadmaps for leaders. The book mixes case studies, technical explanations, and future trends for non-technical audiences as well.

Mastering Agentic AI in Cloud and Edge Computing by Brian Pitman (2025)

This book targets engineers and business leaders seeking a blend of RAG, agent architecture, and cloud/edge applications, supplying detailed coding guides, business case studies, and production design insights.

RAG-Driven Generative AI by Denis Rothman (2024)

This technical reference explains how to implement custom RAG pipelines with various open-source and IBM-relevant technologies, including best practices for minimizing hallucinations and supporting multimodal agents. It connects RAG’s impact to broader generative/agentic strategies in real-world applications.

Rolling Out Anthos Service Mesh for Secure and Observable Microservices

Shuvojit Kar — Sat, 19 Jul 2025 13:24:56 +0000

Goal
The objective was to deploy and operationalize Anthos Service Mesh (ASM) on our Google Kubernetes Engine (GKE) clusters to achieve:

Zero-trust security between microservices

Full observability into service-to-service communication

A foundation for scalable, policy-driven traffic management and control

With the increasing complexity of microservice interactions, we needed better visibility and tighter security without impacting developer velocity or uptime.

🔧 Actions Taken
The deployment involved several critical phases:

Architecture Planning & ASM Setup Defined the high-level architecture integrating ASM with GKE workloads.

Chose managed ASM control planes to reduce operational overhead.

mTLS and Zero Trust Security Enabled mutual TLS (mTLS) across all services by default.

Applied PeerAuthentication and AuthorizationPolicy to enforce strict identity-based access controls.

Observability Enhancements Integrated Cloud Trace, Cloud Monitoring, and Cloud Logging with Istio's telemetry features.

Configured dashboards to visualize request flows, latency, error rates, and SLOs/SLIs.

Traffic Control & Policy Management Implemented traffic shifting, canary deployments, and circuit breakers using Istio virtual services and destination rules.

Applied fine-grained traffic policies to separate dev, staging, and prod environments cleanly.

Training and Documentation Delivered internal training for developers and SREs on using the mesh and troubleshooting via telemetry tools.

Authored detailed runbooks for incident response, upgrades, and mesh monitoring.

📈 Results
✅ Achieved full compliance with our zero-trust architecture goals.

⏱️ Reduced troubleshooting time by 50% by leveraging built-in telemetry and service insights.

📉 Improved visibility into inter-service traffic, latency, and error distribution, enabling faster root cause analysis.

💡 Empowered developers to safely experiment with progressive delivery strategies (blue/green, canary) without downtime.

🧠 Lessons Learned
Planning for sidecar injection strategies early helped avoid rollout friction.

Observability becomes truly powerful when correlated with service metadata and mesh policies.

Consistent naming conventions and tagging in policies made troubleshooting dramatically easier.

🔍 What’s Next?
Expanding the mesh to hybrid clusters and non-Kubernetes workloads via Anthos Connect.

Automating policy rollouts with GitOps tooling (ArgoCD + Kustomize).

Exploring ambient mesh for lower overhead service mesh deployments.

Modernizing a Core Banking Platform with Google Cloud: A Cloud-Native Journey

Shuvojit Kar — Sat, 19 Jul 2025 13:16:24 +0000

Financial institutions are under increasing pressure to modernize legacy systems, improve agility, and meet the demands of digital banking customers. This blog post explores how we helped a Tier-1 financial client modernize their core banking system by leveraging Google Cloud Platform (GCP) and a cloud-native architecture built on microservices and event-driven design.

🚩 The Challenge
The client operated a monolithic core banking platform built over a decade ago. While robust, the system had several limitations:

Slow feature deployment cycles (monthly or quarterly).

Scalability issues during peak traffic (e.g., end-of-month processing).

Tight coupling between services and data layers, making maintenance and integration difficult.

High operational overhead due to reliance on manual processes and legacy middleware.

The goal was to re-architect the platform for resilience, scalability, and agility—without disrupting mission-critical services.

☁️ Why Google Cloud?
GCP was chosen for its strengths in container orchestration (GKE), event-driven processing (Pub/Sub), scalable serverless components, and strong security posture. Key advantages included:

Managed Kubernetes (GKE) for orchestrating microservices with auto-scaling and zero-downtime deployments.

Cloud Pub/Sub and Dataflow for handling high-volume, real-time event streams.

Cloud Spanner for global-scale, strongly consistent relational data needs.

Integrated DevOps tooling with Cloud Build, Artifact Registry, and Deployment Manager.

🛠️ Architecture Overview
We followed Domain-Driven Design (DDD) to isolate business capabilities and model them as independent bounded contexts. Here's a high-level breakdown of the architecture:

🔹 Microservices:
Decomposed core banking functions (e.g., Accounts, Transactions, Payments) into stateless microservices.

Each service communicates via gRPC and REST APIs, depending on latency and interoperability needs.

Resilience patterns (circuit breakers, retries, fallbacks) implemented using Istio service mesh.

🔹 Event-Driven Backbone:
Introduced Cloud Pub/Sub as the event backbone for decoupling services.

Payment workflows and account updates are now processed in real-time using Cloud Functions and Cloud Run.

Event replay and auditability supported using Dataflow and BigQuery.

🔹 Persistent Layer:
Cloud Spanner serves as the distributed transactional database for account and ledger data.

Immutable audit logs stored in Cloud Storage and BigQuery for compliance and reporting.

🔄 Migration Strategy
Modernizing a live banking system requires surgical precision. We followed a Strangler Fig pattern:

Baseline Assessment: Analyzed legacy system workflows and dependencies.

Service Extraction: Incrementally carved out services starting with low-risk domains (e.g., Notifications).

API Gateway Transition: Shifted API traffic through Apigee to orchestrate legacy and new services.

Shadow Testing & Canary Deployments: Used GKE + Istio to test microservices in parallel before full cutover.

Production Cutover: Transitioned critical traffic progressively, with full rollback plans in place.

✅ Business Outcomes
65% faster time-to-market for new banking features.

99.99% service availability, even during high-load periods like payroll processing.

35% reduction in infrastructure costs through autoscaling and optimized container resources.

Improved compliance via real-time audit trails and hardened security configurations.

📚 Lessons Learned
Design for failure: Distributed systems require thoughtful fallback and retry mechanisms.

Start with observability: Centralized logging, tracing (Cloud Trace), and monitoring were key to early debugging.

DDD pays off: Proper domain modeling made scaling teams and services more manageable.

Executive buy-in is critical: Clear communication with leadership helped mitigate resistance to change.

FREE Google AI Pro subscription

Shuvojit Kar — Fri, 18 Jul 2025 03:44:11 +0000

🚀 Big news for India’s next generation of innovators! Google just unlocked a one-year FREE Google AI Pro subscription for every eligible college student in India — a plan normally priced at ₹19,500 per year.

What the pass unlocks:
• Gemini 2.5 Pro for deep reasoning, coding & creativity
• Veo 3 for text-to-video storytelling
• Deep Research plus NotebookLM with 5× larger note limits
• Gemini Live inside Gmail, Docs, Sheets & more
• 2 TB of Google Cloud storage to keep every project, paper and prototype safe

Associate Software Engineer

Shuvojit Kar — Fri, 18 Jul 2025 03:40:49 +0000

Managing Kubernetes Clusters both Control and Data plane for On premises Infrastructures.
Developed the One-Day Setup DevOps Tool: Built a Django-based API integrating Ansible playbooks to automate the installation of DevOps infrastructure, including Kubernetes (multi-master and worker nodes), HAProxy, Istio, Helm, Jenkins, ArgoCD, Kafka, Redis, Vault, External Secrets, PostgreSQL, OpenSearch, Prometheus, Grafana, Kiali, OpenTelemetry, and other components.
Automated infrastructure provisioning with Terraform, Ansible reducing environment setup time from days to hours
Implemented GitOps practices using ArgoCD, reducing deployment errors by 40% and improving system reliability
Implemented chaos engineering practices to improve system resilience, reducing unplanned downtime by 25%
Developed custom Prometheus exporters for application-specific metrics, enhancing monitoring capabilities
Implemented robust CI/CD pipelines, reducing deployment time by 30% and enhancing overall development efficiency.

🎉 Thrilled to share that I’ve been renewed as an AWS Community Builder for the 3rd year in a row!

Shuvojit Kar — Fri, 18 Jul 2025 03:39:05 +0000

This program has been an incredible platform for growth, collaboration, and giving back to the Amazon Web Services (AWS) cloud community. Grateful for the opportunity to:
🔹 Learn from the brightest minds in AWS
🔹 Share knowledge through blogs, talks, and contributions
🔹 Connect with an inspiring global network of builders

AWS Community Builders team for their support—and for the awesome swag (loving the Glorious Gadget Pouch of Building!).

Code to Cloud: Deploying a Flask App with Docker, GCP, and Kubernetes

Shuvojit Kar — Thu, 17 Jul 2025 13:42:24 +0000

Modern application development demands not just building robust code but also ensuring it is portable, scalable, and easily maintainable across environments. In this blog, I’ll walk through how I developed, containerized, and deployed a Flask application using Docker and Google Cloud technologies like GKE, Cloud Build, and Container Registry.

Step 1: Building the Flask Application
The journey began with a lightweight Flask app designed for rapid development and REST API support. Flask’s modular nature made it perfect for this microservice-style project.

This setup ensured a safe, consistent, and efficient deployment process, with zero manual intervention.

Outcomes & Benefits
Environment Parity: By using Docker and Kubernetes, I maintained consistent environments across development, staging, and production.

Reduced Runtime Errors: Environment uniformity drastically reduced bugs that usually surface during deployment.

Operational Efficiency: The CI/CD pipeline with Cloud Build and GitHub streamlined the release cycle, ensuring fast and reliable updates.

Scalability & Load Balancing: GKE’s native support for scaling and load balancing ensured the app could handle increased traffic smoothly.

Conclusion
This project was a full-stack DevOps exercise in modern cloud-native application deployment. From building the app to deploying it on a scalable Kubernetes cluster, every step was optimized for automation, reliability, and performance.

If you're looking to deploy your applications on GCP with confidence, combining Docker, GKE, and Cloud Build is a powerful, production-ready solution.

SQL Performance Tuning Guide

Shuvojit Kar — Thu, 17 Jul 2025 13:39:36 +0000

Overview:
Designed a comprehensive, developer-facing guide focused on SQL performance optimization for PostgreSQL and BigQuery environments. The guide served as a reference for improving query speed, reducing resource consumption, and applying database best practices across analytics and transactional workloads.

Key Contributions:

Structured Documentation: Developed a modular guide with clear sections on indexing strategies, query planning, join optimization, partitioning, and caching.

Knowledge Extraction: Collaborated with senior database engineers to translate undocumented tribal knowledge and performance hacks into structured, reusable documentation.

Explain Plan Analysis: Included annotated EXPLAIN and EXPLAIN ANALYZE outputs with detailed breakdowns of cost, rows, and runtime behavior.

Hands-On Examples: Provided copy-paste-ready SQL examples with sample datasets, expected outputs, and anti-patterns.

Impact: Enabled teams to reduce query execution time by 30–60% on high-volume reporting workloads; improved BigQuery cost management by identifying and fixing inefficient scans.

Tech & Tools:
PostgreSQL, BigQuery, SQL, Google Cloud Console, Markdown, Data Studio, GitHub, Jupyter Notebooks.

MLOps Engineer – 2030

Shuvojit Kar — Thu, 17 Jul 2025 13:36:47 +0000

MLOps Engineer / AI Infrastructure Engineer / AI DevOps Architect
Domain: Machine Learning + DevOps + Security + Cloud + Responsible AI

Key Responsibilities
End-to-End ML Lifecycle Automation

Automate model development, training, validation, deployment, and monitoring using AI-native platforms.

Use no-code/low-code AI pipelines for rapid experimentation.

AI Infrastructure & Orchestration

Design and manage distributed training clusters (on Cloud, Edge, Quantum).

Leverage AI-optimized compute (TPUs, neuromorphic chips, quantum co-processors).

AI Observability & Explainability

Monitor real-time model performance and drift using self-healing systems.

Implement XAI (Explainable AI) tools to ensure transparency and compliance.

Responsible AI & Compliance

Enforce AI ethics: bias detection, privacy, and regulatory alignment (e.g., AI Act, GDPR v2.0).

Manage model cards and data sheets as compliance artifacts.

CI/CD/CT (Continuous Training)

Implement intelligent CI/CD/CT pipelines with adaptive retraining triggers.

Use synthetic data and simulation environments for safe model updates.

Collaboration Across Disciplines

Work with Data Scientists, Software Engineers, Model Risk Managers, and AI Policy Experts.

Operate in a multi-modal ecosystem (vision, speech, NLP, IoT).

🧠 Skills & Tools (Expected in 2030)
Languages: Python++, Julia AI, FlowLang (AI-native scripting), Rust

Platforms: Vertex AI 5.0, SageMaker++, Databricks Unity, HuggingFace Infra

Pipelines: Kubeflow++, Flyte, Airflow AI, ZenML

Infra: Multi-cloud (GCP/AWS/Azure/IBM Quantum), EdgeOps, Federated Learning

Monitoring: WhyLabs, Arize, TruEra, OpenTelemetry AI

Security & Governance: Confidential AI, Homomorphic Encryption, AI Chain of Custody

🧩 Future-Proof Mindset
Agile AI Ops: Continuously evolve workflows to adapt to model behavior and external factors.

Ethics by Design: Integrate ethical frameworks into deployment pipelines.

Cross-Skill Fluency: Understand ML models deeply and systems engineering thoroughly.