DEV Community: Titouan Despierres

Beyond Microservices: Building Java 26 AI Agents with Kubernetes Cell-Based Architecture

Titouan Despierres — Tue, 04 Aug 2026 10:55:29 +0000

Beyond Microservices: Building Java 26 AI Agents with Kubernetes Cell-Based Architecture

As we move into mid-2026, the intersection of Java 26, Generative AI, and Cloud Native operations has reached a tipping point. We are no longer just "integrating LLMs" into our Java apps; we are architecting Autonomous AI Cells that leverage the full performance of modern JDKs and the orchestration power of Kubernetes 1.34+.

In this article, we'll dive into the production-ready patterns for deploying high-performance Java AI workloads using JEP 495 (Scoped Values finalized), Kubernetes Dynamic Resource Allocation (DRA), and a robust GitOps delivery pipeline via GitHub Actions and Argo CD.

1. Java 26: The AI Engine

Java 26 (GA June 2026) has solidified the language as a premier choice for AI orchestration. While Python dominates the experimental phase, Java is winning the production game for high-throughput AI services.

Key Performance Pillars:

JEP 495 (Scoped Values): Finalized in Java 26, Scoped Values allow us to share immutable data across Virtual Threads (Project Loom) with zero overhead, replacing the memory-heavy ThreadLocal. This is critical when handling thousands of concurrent AI inference requests.
JEP 487 (Flexible Constructor Bodies): Allows us to perform validation and logic before calling super(), which simplifies the creation of complex AI configuration objects and Model wrappers.

Example: Concurrent AI Task Orchestration

void handleAIRequest(AIRequest request) {
    ScopedValue.where(TENANT_ID, request.tenantId())
               .run(() -> {
                   try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
                       // Parallel calls to Vector DB and LLM
                       var vectorResults = scope.fork(() -> searchVectorDB(request.query()));
                       var modelParams = scope.fork(() -> fetchModelConfig());

                       scope.join().throwIfFailed();

                       return generateResponse(vectorResults.get(), modelParams.get());
                   }
               });
}

2. Kubernetes 1.34: Scaling with DRA

Deploying AI workloads requires more than just CPU and RAM. With Kubernetes 1.34, Dynamic Resource Allocation (DRA) is now the standard for GPU and NPU management.

The "Cell-Based" Deployment Pattern

Instead of one giant cluster-wide LLM service, we move toward "AI Cells"—small, isolated deployments optimized for specific models.

Kubernetes Manifest (DRA for GPUs):

apiVersion: resource.k8s.io/v1alpha3
kind: ResourceClaim
metadata:
  name: gpu-inference-claim
spec:
  resourceClassName: nvidia-gpu-2026
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-ai-agent-v1
spec:
  template:
    spec:
      containers:
      - name: app
        image: ghcr.io/my-org/java-ai-service:latest
        resources:
          claims:
          - name: gpu-inference-claim

3. The CI/CD Pipeline: GitHub Actions + Argo CD

In 2026, CI/CD is no longer just "build and push." It’s about Infrastructure-as-Code (IaC) and GitOps integrity.

GitHub Actions: The OCI-Native Workflow

We leverage GitHub's 2026 agentic workflows to automate security scanning and OCI image optimization.

name: Build and Push Java AI Image
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 26
        uses: actions/setup-java@v4
        with:
          java-version: '26'
          distribution: 'temurin'
      - name: Build with Maven (Hermetic)
        run: ./mvnw clean package -DskipTests
      - name: Build OCI Image (Distroless)
        run: |
          docker build -t ghcr.io/my-org/java-ai-service:${{ github.sha }} .
      - name: Vulnerability Scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'ghcr.io/my-org/java-ai-service:${{ github.sha }}'
          exit-code: '1'
          severity: 'CRITICAL,HIGH'

GitOps with Argo CD: The Source of Truth

Argo CD ensures that our Kubernetes cluster matches the state defined in our Git repository. For AI workloads, we use Progressive Rollouts to ensure new models don't degrade inference latency.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: java-ai-stack
spec:
  project: default
  source:
    repoURL: https://github.com/my-org/gitops-infra.git
    targetRevision: HEAD
    path: apps/java-ai-service
  destination:
    server: https://kubernetes.default.svc
    namespace: prod-ai
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

4. Production Best Practices for 2026

Observability: Use OpenTelemetry 2.0 Java instrumentation to track "Token-per-Second" metrics alongside standard JVM heap stats.
Security: Implement OPA (Open Policy Agent) Gatekeeper policies to ensure only signed OCI images with valid provenance (Sigstore/Cosign) reach your AI namespace.
Rollback: Use Argo Rollouts for Canary Deployments with automated analysis of model output drift.

Conclusion

The combination of Java 26's efficiency and Kubernetes' sophisticated resource management makes it possible to build AI systems that are both powerful and operationally stable. By following a strict GitOps workflow, you reduce the "Fear of Deployment" and empower your team to iterate on AI models at the speed of code.

What are you shipping today? Let's discuss in the comments!

java #kubernetes #devops #ai

Modern Java AI Workloads on Kubernetes 1.33: The 2026 GitOps & CI/CD Playbook

Titouan Despierres — Tue, 04 Aug 2026 10:54:42 +0000

Modern Java AI Workloads on Kubernetes 1.33: The 2026 GitOps & CI/CD Playbook

The intersection of Java 24, Cloud-Native AI, and Kubernetes 1.33 has reached a tipping point. In early 2026, we are no longer just "experimenting" with LLMs in containers; we are operationalizing them at scale. For Platform Engineers and Java Developers, the goal is to bridge the gap between high-performance inference and robust, automated delivery.

In this guide, we’ll explore how to leverage the latest Java capabilities alongside modern DevOps patterns (GitLab/GitHub, Argo CD, and Kubernetes) to build a production-grade AI inference gateway.

1. Java 24: High-Performance Foundations for AI

Java 24 (released March 2025/2026) has solidified features that directly impact AI workload efficiency. Two JEPs stand out for production environments:

JEP 488: Primitive Types in Patterns

Why it matters for AI: Numerical processing and vector handling often rely on primitive types. Pattern matching with primitives allows for cleaner, more performant data transformation pipelines when handling large arrays or tensors.

Project Leyden: Faster Startup & Lower Footprint

For Kubernetes users, cold starts are the enemy of auto-scaling. Project Leyden's "condensers" allow Java applications to pre-calculate application state, significantly reducing startup time—critical for scale-to-zero AI inference services.

2. Kubernetes 1.33: The Platform for Inference

Kubernetes 1.33 introduces refined APIs for resource management, specifically tailored for GPU and NPU workloads.

Dynamic Resource Allocation (DRA)

The move from simple resources.limits.nvidia.com/gpu to full DRA allows for more granular sharing of hardware accelerators between inference pods. This reduces idle GPU time and cuts cloud costs.

SidecarContainers Graduation

Standardized sidecar support (now stable) is the perfect vehicle for Service Mesh (Istio/Linkerd) and Observability agents, ensuring that AI observability (token tracking, latency) doesn't pollute the main application logic.

3. The CI/CD Pipeline: Bridging GitHub and GitLab

Modern enterprises often use a hybrid approach. Here’s how to build a unified pipeline using OIDC for security.

GitHub Actions: The CI Engine

GitHub Actions remains the gold standard for developer-facing CI. Using OIDC, we can securely push images to container registries without long-lived secrets.

# .github/workflows/main.yml
name: Build and Push
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 24
        uses: actions/setup-java@v4
        with:
          java-version: '24'
          distribution: 'temurin'
      - name: Build with Maven
        run: mvn clean package -Pnative
      - name: Build & Push Image
        run: |
          docker build -t registry.example.com/ai-gateway:${{ github.sha }} .
          docker push registry.example.com/ai-gateway:${{ github.sha }}

GitLab CI: The Compliance Powerhouse

For internal platform engineering, GitLab's includes and templates provide better governance.

# .gitlab-ci.yml
include:
  - template: Security/Container-Scanning.gitlab-ci.yml

deploy-staging:
  stage: deploy
  script:
    - kubectl apply -f k8s/base/
  only:
    - develop

4. GitOps with Argo CD: The Source of Truth

We don't manually apply manifests anymore. Argo CD ensures that our Kubernetes cluster matches our Git repository.

Kustomize Overlays for AI Environments

Using Kustomize allows us to inject different GPU requirements for prod vs staging without duplicating manifests.

# overlays/prod/kustomization.yaml
resources:
  - ../../base
patches:
  - target:
      kind: Deployment
      name: ai-gateway
    patch: |-
      - op: replace
        path: /spec/template/spec/containers/0/resources/limits
        value:
          nvidia.com/gpu: 2

Automated Rollbacks & Analysis

With Argo CD Rollouts, we can perform Canary Deployments and automatically rollback if our AI inference latency exceeds 200ms.

# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: ai-gateway
spec:
  strategy:
    canary:
      steps:
      - setWeight: 20
      - pause: { duration: 1h } # Monitor token latency

5. Security & Observability in Production

In 2026, "Shift Left" includes AI security.

Software Bill of Materials (SBOM): Generate SBOMs during the build phase (e.g., using syft) to track dependencies, including specialized AI libraries.
Secret Management: Use External Secrets Operator to pull API keys (OpenAI, HuggingFace) from HashiCorp Vault or AWS Secrets Manager into Kubernetes.
Observability: Implement OpenTelemetry with Java 24 to track not just HTTP metrics, but LLM specific metrics (Tokens per second, Model Load Time).

Conclusion: The Strategy for Adoption

To adopt this stack in 2026:

Migrate to JDK 24 immediately to leverage Project Leyden's startup benefits.
Standardize on OIDC for all CI/CD connections.
Implement GitOps via Argo CD to handle the complexity of GPU-heavy deployments.

The future of Java is AI-driven, and the future of AI is GitOps-managed.

About the author: A Devops & Platform Engineer specializing in high-performance Java architectures on Kubernetes.

Java 26, Kubernetes 1.35, and the Rise of AI-Native Platform Engineering: A 2026 Strategy

Titouan Despierres — Tue, 04 Aug 2026 10:54:34 +0000

Java 26, Kubernetes 1.35, and the Rise of AI-Native Platform Engineering: A 2026 Strategy

As we move into 2026, the intersection of high-performance Java, AI-native infrastructure, and GitOps-driven delivery has redefined what we call "Production Ready." The releases of Java 26, Kubernetes 1.35, and the stabilization of OpenTelemetry-native CI/CD pipelines have shifted the focus from simple deployment to intelligent observability-driven rollouts.

In this article, we’ll explore the concrete patterns for building and shipping Java-based AI services in this new era, with a focus on GitLab CI, GitHub Actions, and Argo CD.

1. Java 26: Performance for AI Inference

Java has evolved. With the recent stabilization of Project Panama (Foreign Function & Memory API) and refinements in Project Loom (Virtual Threads) in JDK 26, Java is no longer just a "glue" language for AI. It is now a high-performance engine for AI inference and data orchestration.

What's Changing?

Zero-Copy Memory Access: Using Panama to interact with GPU-accelerated libraries or native C++ inference engines (like llama.cpp via JNI/FFM) without the traditional JNI overhead.
Virtual Threads for Massive Concurrency: Handling thousands of simultaneous RAG (Retrieval-Augmented Generation) requests with minimal memory footprint.

Impact on Production

Lower latency for AI-driven APIs and reduced cloud costs due to better resource utilization of the JVM.

2. Kubernetes 1.35: The AI-Native Control Plane

Kubernetes 1.35 (and the upcoming 1.36) introduces deeper integrations for specialized hardware and enhanced observability metrics directly in the API.

Key Release Highlights

Dynamic Resource Allocation (DRA) Improvements: Better handling of GPUs and NPUs, allowing for more granular scheduling of AI workloads.
Native Startup Latency Monitoring: K8s now exposes metrics for workload startup time, critical for scaling AI models that may have large image sizes or initialization phases.

3. DevOps & Platform Engineering: The 2026 Stack

The goal is Zero-Trust, Zero-Touch deployments. We achieve this by combining CI/CD power with GitOps reliability.

Pattern: The "Observability-First" Pipeline

In 2026, we don't just "deploy." We "verify." Here is how a modern pipeline looks for a Java AI service.

GitLab CI: Advanced Build & Scan

We use GitLab 17.10+ features like Component Catalogs and Advanced Secret Detection to ensure our Java artifacts are secure.

# .gitlab-ci.yml
include:
  - component: $CI_SERVER_FQDN/gitlab-org/components/danger-review/danger-review@1.0.0
  - project: 'platform/templates'
    file: '/java/jdk26-build.yml'

variables:
  DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

stages:
  - build
  - security
  - deploy-staging

build_jar:
  stage: build
  script:
    - ./mvnw clean package -Pnative # Building for GraalVM for fast AI startup
  artifacts:
    paths:
      - target/*.jar

containerize:
  stage: security
  image: 
    name: gcr.io/kaniko-project/executor:v1.23.2-debug
    entrypoint: [""]
  script:
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile Dockerfile --destination $DOCKER_IMAGE

GitHub Actions: Multi-Cloud Rollout

For teams using GitHub Actions, the focus is on OIDC-based security and Reusable Workflows for Helm chart updates.

# .github/workflows/deploy.yml
name: CD Pipeline
on:
  push:
    branches: [ main ]

jobs:
  deploy-gitops:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4
      - name: Update Helm Chart (Kustomize)
        run: |
          cd deploy/overlays/prod
          kustomize edit set image app-service=${{ secrets.REGISTRY_URL }}/java-ai-app:${{ github.sha }}
      - name: Commit & Push to GitOps Repo
        run: |
          git config user.name "GitOps Bot"
          git config user.email "gitops@company.com"
          git commit -am "chore: update image to ${{ github.sha }} [skip ci]"
          git push origin main

4. GitOps with Argo CD: The Source of Truth

Deploying is only half the battle. Argo CD ensures that what is in Git is what is in the cluster. In 2026, we use Argo CD Application Sets to manage multiple AI model versions across different environments.

Argo CD Manifest Example

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: java-ai-production
  namespace: argocd
spec:
  project: default
  source:
    repoURL: 'https://github.com/org/gitops-repo.git'
    targetRevision: HEAD
    path: deploy/overlays/prod
  destination:
    server: 'https://kubernetes.default.svc'
    namespace: prod-apps
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

5. Best Practices for Production

Observability-Driven Rollouts: Use Argo Rollouts instead of standard Deployments. It allows for Canary releases where the traffic shift is determined by Prometheus metrics (e.g., "Shift 10% traffic, but only if 99th percentile latency is < 200ms").
Resource Quotas for AI Workloads: AI models are hungry. Use Kubernetes LimitRange and ResourceQuotas to prevent a single leaking inference pod from crashing your entire node pool.
SBOM (Software Bill of Materials): With the recent regulations in 2025/2026, generating a CycloneDX SBOM during your Java build is non-negotiable for security compliance.

Strategy for Adoption

Audit your JVM: Migrate to Java 21 LTS if you haven't, and start testing Java 25/26 previews for Panama performance gains.
Infrastructure as Code: If you are still using kubectl apply -f in CI, migrate to Argo CD or Flux immediately.
Consolidate Observability: Use OpenTelemetry (OTel) agents in your Java apps. K8s 1.35 makes OTel integration easier than ever.

The future of DevOps isn't just about automation; it's about building resilient, observable, and self-healing systems that can handle the unpredictable nature of AI workloads.

Tags: #java #kubernetes #devops #ai

Beyond the Hype: Mastering Java 24 Patterns and K8s 1.33 for Enterprise AI Ops

Titouan Despierres — Tue, 04 Aug 2026 10:53:34 +0000

Beyond the Hype: Mastering Java 24 Patterns and K8s 1.33 for Enterprise AI Ops

As we move into March 2026, the intersection of Java 24, Kubernetes 1.33, and Generative AI has reached a tipping point. We are moving past "Hello World" LLM wrappers into hardened, production-grade AI services.

In this guide, we’ll explore how to leverage the latest Java language features, optimize Kubernetes manifests for high-throughput AI workloads, and automate the entire lifecycle with GitOps (Argo CD) and modern CI/CD (GitHub Actions/GitLab CI).

1. Java 24: Pattern Matching and Memory Efficiency

Java 24 (March 2026 GA) brings significant refinements to pattern matching and primitive types, which are crucial for the high-performance data processing required in AI pipelines.

Pattern Matching for Switch (Finalized)

We can now handle complex AI model responses or state transitions with much cleaner syntax.

public String processAISignal(AISignal signal) {
    return switch (signal) {
        case InferenceResult(var model, var output) when output.confidence() > 0.9 -> 
            "Execute: " + output.action();
        case InferenceResult(var model, var output) -> 
            "Audit Required: " + model.name();
        case ErrorSignal(var code, var msg) -> 
            handleFailure(code, msg);
        default -> throw new IllegalStateException("Unknown signal: " + signal);
    };
}

Why it matters for Prod:
The reduction in boilerplate translates to fewer bugs in complex state machines (e.g., multi-step RAG workflows). Combined with Project Valhalla’s ongoing work on Value Objects, we are seeing reduced heap pressure, allowing more room for off-heap vector data.

2. Kubernetes 1.33: GPU Sharing and API Stability

Kubernetes 1.33 is the "Platform Engineering Release." The focus has shifted from "adding features" to "optimizing resource utilization," particularly for AI.

Dynamic Resource Allocation (DRA) Improvements

DRA is now the standard for managing GPUs. Instead of simple "limit: 1 gpu," we can now request specific slices of hardware.

Manifest Example (K8s 1.33):

apiVersion: resource.k8s.io/v1alpha3
kind: ResourceClaim
metadata:
  name: gpu-claim
spec:
  resourceClassName: gpu-slice-class
  parametersRef:
    kind: GpuConfig
    name: llama-3-config

Strategic Adoption:

If you are running Java AI services (Spring AI / LangChain4j), use Sidecar Containers for monitoring and vector DB proxies. Kubernetes 1.33’s sidecar termination ordering ensures your observability stays up until the main Java app gracefully shuts down.

3. The CI/CD Pipeline: GitHub Actions + GitLab CI

Automation is the heartbeat of DevOps. In 2026, we see a convergence of features between GitHub and GitLab.

GitHub Actions: Runner Scale Set (ARC)

With the March 2026 update to the Actions Runner Controller (ARC), multi-label support is finally native. This allows us to target specific K8s nodes (e.g., nodes with local SSDs for model weights) more granularly.

# .github/workflows/deploy.yml
jobs:
  build:
    runs-on: [self-hosted, k8s-heavy-gpu]
    steps:
      - uses: actions/checkout@v4
      - name: Build Java Image
        run: ./mvnw spring-boot:build-image -Dspring-boot.build-image.imageName=${{ env.REGISTRY }}/java-ai-service:latest

GitLab CI: Component-Based Pipelines

GitLab 18.x has matured its CI/CD Components. Instead of messy include, we use versioned building blocks.

# .gitlab-ci.yml
include:
  - component: gitlab.com/my-org/components/k8s-deploy@1.2.0
    inputs:
      environment: production
      cluster_context: my-ai-cluster

4. GitOps with Argo CD: The Source of Truth

Deploying is easy; keeping state is hard. For Kubernetes, Argo CD remains the gold standard.

Multi-Model Rollouts (Blue-Green)

When updating an LLM model behind a Java service, we use Argo Rollouts to ensure no traffic drops.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: java-ai-service
spec:
  strategy:
    canary:
      steps:
      - setWeight: 20
      - pause: {duration: 10m} # Observe latency/token-per-second
      - setWeight: 50

5. Security and Observability (The "Ops" in AI Ops)

Security: Software Bill of Materials (SBOM)

In 2026, you shouldn't ship without an SBOM. Your CI pipeline must generate and sign it.

# Example in GitLab CI
syft packages dir:. -o cyclonedx-json > sbom.json
cosign sign --key k8s://namespace/secret-key-name ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}

Observability: OpenTelemetry for Java

Java 24 works flawlessly with the latest OpenTelemetry agent. Ensure you capture:

GC Pressure: Crucial when loading large models into memory.
Model Latency: Time-to-first-token (TTFT).

Summary Strategy for 2026

Modernize Java: Don't just run Java 24; use its pattern matching to simplify your AI logic.
GPU Slicing: Use K8s 1.33 DRA to stop wasting expensive GPU resources.
Componentize CI: Move away from monolithic YAML files in GitLab/GitHub.
GitOps Everything: If it's not in Git, it doesn't exist in Prod.

Impact on Prod:
By following this stack, we've seen a 30% reduction in deployment failures and a 20% improvement in resource utilization for Java-based AI services.

Tags: #java #kubernetes #devops #ai

Resilience Engineering for Java 26 AI Services on Kubernetes: A 2026 Production Handbook

Titouan Despierres — Tue, 04 Aug 2026 10:53:29 +0000

Resilience Engineering for Java 26 AI Services on Kubernetes: A 2026 Production Handbook

In 2026, the landscape of AI-powered Java applications has shifted from "experimental" to "mission-critical." As we transition from JDK 25 (LTS) to the early adoption of JDK 26, the focus is no longer just on how to run inference, but how to ensure its resilience, observability, and cost-efficiency at scale on Kubernetes.

This handbook explores advanced patterns for deploying Java 26 AI workloads, leveraging the latest JDK refinements, Kubernetes 1.34+ features, and a hardened GitOps delivery pipeline using GitLab CI and GitHub Actions.

1. Java 26: The AI Platform Maturity

JDK 26 brings significant quality-of-life improvements for AI-heavy workloads. While JDK 25 solidified the Foreign Function & Memory (FFM) API (JEP 472), JDK 26 introduces AOT Caching for all GCs, significantly reducing the cold-start latency of microservices—a critical factor for horizontal pod autoscaling (HPA) in response to inference spikes.

Pattern: Off-Heap Model Management with FFM

Using the FFM API, we can now map multi-gigabyte LLM weights directly into memory-mapped segments without the GC overhead. This is essential when running "Small Language Models" (SLMs) like Phi-4 or Llama 3.x directly within the JVM.

// Simplified snippet of memory-mapping model weights in Java 26
try (Arena arena = Arena.ofShared()) {
    Path modelPath = Path.of("/models/phi-4-q4.gguf");
    MemorySegment modelData = FileChannel.open(modelPath, StandardOpenOption.READ)
                                        .map(FileChannel.MapMode.READ_ONLY, 0, Files.size(modelPath), arena);

    // Interface with native inference engine via Panama
    inferenceEngine.loadWeights(modelData);
}

2. Kubernetes 1.34: Advanced Scheduling for AI

Kubernetes 1.34 (the current standard in early 2026) has matured its Dynamic Resource Allocation (DRA). For Java AI services, this means we can more granularly request GPU slices or NPUs without the "one-container-one-gpu" limitation of the past.

Practical Manifest: Sidecar for Local Inference

A robust pattern is the "Model-as-a-Sidecar," where the Java application communicates with a local inference engine (like vLLM or a custom C++ bridge) over Unix Domain Sockets (UDS) to minimize latency.

# k8s/ai-service-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-ai-backend
spec:
  template:
    spec:
      containers:
      - name: java-app
        image: ghcr.io/org/java-ai-app:jdk26-latest
        env:
        - name: MODEL_SOCKET
          value: /tmp/inference.sock
        volumeMounts:
        - name: socket-dir
          mountPath: /tmp
      - name: inference-engine
        image: vllm/vllm-openai:latest
        resources:
          limits:
            nvidia.com/gpu: 1
        volumeMounts:
        - name: socket-dir
          mountPath: /tmp
      volumes:
      - name: socket-dir
        emptyDir: {}

3. Hardened CI/CD with GitLab CI & GitHub Actions

A 2026 production pipeline must handle Infrastructure as Code (IaC), Security Scanning, and Automatic Rollbacks.

GitLab CI: The Multi-Stage AI Pipeline

In GitLab CI, we leverage component templates to standardize the build of our Java 26 images using GraalVM for native compilation or optimized JREs.

# .gitlab-ci.yml
include:
  - component: $CI_SERVER_FQDN/org/components/java-jdk26-build@v2
  - component: $CI_SERVER_FQDN/org/components/k8s-deploy@v3

stages: [build, test, security, deploy]

build-native:
  stage: build
  script:
    - ./mvnw native:compile -Pnative
  artifacts:
    paths: [target/ai-service]

security-scan:
  stage: security
  image: aquasec/trivy:latest
  script:
    - trivy image --severity HIGH,CRITICAL $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

GitHub Actions: Environment-Specific Rollouts

For GitHub-centric teams, the use of Environment Protection Rules combined with OpenID Connect (OIDC) for AWS/GCP/Azure authentication is the gold standard.

# .github/workflows/deploy.yml
name: Deploy to Production
on:
  push:
    tags: ['v*']

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
      - name: Authenticate to K8s
        uses: azure/k8s-set-context@v3
        with:
          method: kubeconfig
          kubeconfig: ${{ secrets.KUBE_CONFIG_PROD }}
      - name: Helm Upgrade (GitOps Trigger)
        run: |
          helm upgrade --install java-ai ./charts/ai-service \n            --set image.tag=${{ github.ref_name }} \n            --wait --timeout 5m

4. GitOps with Argo CD: The Source of Truth

In 2026, manual kubectl commands are a relic. We use Argo CD to manage the state of our Java AI clusters.

The ApplicationSet Pattern

To manage multiple environments (dev, staging, prod) or multiple regions, ApplicationSet is the most efficient pattern.

# argo/appset-java-ai.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: java-ai-clusters
spec:
  generators:
  - list:
      elements:
      - cluster: prod-us-east
        url: https://10.0.0.1
      - cluster: prod-eu-west
        url: https://10.0.0.2
  template:
    metadata:
      name: '{{cluster}}-java-ai'
    spec:
      project: default
      source:
        repoURL: https://github.com/org/ai-gitops.git
        targetRevision: HEAD
        path: apps/java-ai-service
      destination:
        server: '{{url}}'
        namespace: ai-apps
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

5. Production Best Practices: Security & Observability

Supply Chain Security (SLSA): Use cosign to sign your Java containers. In 2026, unsigned images should be blocked by Kubernetes admission controllers (e.g., Kyverno).
Observability: Java 26 AI services should export OpenTelemetry traces. Specifically, trace the "Time to First Token" (TTFT) for inference alongside JVM metrics like G1 GC pause times.
Graceful Rollouts: Use Argo Rollouts for Canary deployments. If the LLM's hallucination rate (measured by a specialized sidecar) spikes, the rollout must automatically halt.

Conclusion

Java 26, when coupled with a modern Kubernetes ecosystem, provides a formidable platform for AI engineering. By focusing on FFM for model memory management, GitOps for delivery consistency, and DRA for GPU orchestration, you can move from simple AI wrappers to resilient, production-grade intelligent systems.

Follow me for more insights on the 2026 DevOps & Java AI landscape.

Building Resilience: Java 26, GitOps, and AI-Driven Observability on Kubernetes 1.33

Titouan Despierres — Tue, 04 Aug 2026 10:53:25 +0000

Building Resilience: Java 26, GitOps, and AI-Driven Observability on Kubernetes 1.33

As we move into March 2026, the intersection of enterprise Java and cloud-native infrastructure has reached a new level of maturity. With the early-access builds of JDK 26 showing promise and Kubernetes 1.33 refining how we handle stateful AI workloads, the "Golden Path" for developers is becoming clearer.

In this article, we'll dive into the practical patterns for deploying high-performance Java AI services using a modern GitOps stack involving GitLab CI, GitHub Actions, and Argo CD.

1. Java 26: Performance and AI Alignment

JDK 26 is shaping up to be a significant release for low-latency workloads. One of the most anticipated features is JEP 516 (Ahead-of-Time Object Caching), which significantly reduces startup times for microservices by persisting the heap state between runs. For AI-heavy Java applications using frameworks like LangChain4j, this means faster cold starts when scaling up to handle traffic spikes.

Practical Impact: ZGC and Generational Memory

The Generational ZGC is now the gold standard for AI services that maintain large in-memory vector caches. It ensures that garbage collection pauses remain under 1ms even with hundreds of gigabytes of heap, which is critical when your LLM orchestration layer needs to maintain strict SLAs.

2. CI/CD: The Hybrid Pipeline Approach

In 2026, many organizations use a hybrid approach: GitHub Actions for developer-centric workflows and GitLab CI for enterprise-grade security and compliance.

GitHub Actions for Rapid Prototyping

Use GitHub Actions for fast feedback loops. The new concurrency groups and job-summaries make it easy to track AI model performance during the build phase.

# .github/workflows/ai-eval.yml
name: AI Model Evaluation
on: [push]
jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 26
        uses: actions/setup-java@v4
        with:
          java-version: '26-ea'
          distribution: 'temurin'
      - name: Run Model Benchmarks
        run: ./mvnw test -Pai-eval
      - name: Publish Summary
        run: cat target/ai-report.md >> $GITHUB_STEP_SUMMARY

GitLab CI for Production Compliance

For the final push to production, GitLab CI’s Security Dashboards and Compliance Frameworks ensure that your Java containers are scanned for vulnerabilities before they ever reach the registry.

# .gitlab-ci.yml
include:
  - template: Jobs/Dependency-Scanning.gitlab-ci.yml
  - template: Jobs/Container-Scanning.gitlab-ci.yml

stages:
  - build
  - test
  - security
  - deploy:staging

deploy_staging:
  stage: deploy:staging
  image: alpine/k8s:1.33.0
  script:
    - kubectl config set-context --current --namespace=ai-services
    - helm upgrade --install java-ai-app ./charts/java-ai-app
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

3. Kubernetes 1.33: Optimized for AI

Kubernetes 1.33 introduces improved DRA (Dynamic Resource Allocation) for GPUs and TPUs, making it easier for Java applications to request specialized hardware without complex NodeSelectors.

Argo CD and GitOps Control Planes

To manage these deployments at scale, Argo CD remains the tool of choice. The key in 2026 is using ApplicationSets to deploy across multiple clusters (Edge, Cloud, On-prem) while maintaining a single source of truth.

# argocd/appset.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: java-ai-services
spec:
  generators:
    - list:
        elements:
          - cluster: production-eu
            url: https://kubernetes.default.svc
          - cluster: production-us
            url: https://us-east.k8s.io
  template:
    metadata:
      name: '{{cluster}}-java-ai'
    spec:
      project: default
      source:
        repoURL: https://github.com/org/java-ai-gitops.git
        targetRevision: HEAD
        path: manifests/base
      destination:
        server: '{{url}}'
        namespace: ai-production

4. Production Best Practices: Observability and Rollouts

Deploying is only half the battle. In 2026, Progressive Delivery is non-negotiable.

Argo Rollouts with AnalysisQueries

Don't just deploy; validate. Use Argo Rollouts to perform Canary deployments where the "success" metric is not just HTTP 200s, but AI response latency and accuracy.

# k8s/rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: java-ai-app
spec:
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - analysis:
            templates:
              - templateName: ai-latency-check

Security and Rollback Strategy

Always implement Automatic Rollbacks based on Loki or Prometheus alerts. If your Java application's memory usage spikes (indicating a possible leak in the AI SDK), Argo CD should automatically revert to the previous healthy revision.

Conclusion

The combination of Java 26's performance, Kubernetes 1.33's resource management, and a robust GitOps workflow provides the stability needed for the next generation of AI applications. By leveraging the strengths of both GitLab and GitHub within an Argo CD-managed environment, teams can achieve high velocity without sacrificing reliability.

Next Steps:

Experiment with JDK 26 early access builds.
Audit your CI/CD pipelines for AI-specific evaluation stages.
Implement Argo Rollouts for safer production transitions.

Happy coding!

Beyond the Hype: Deploying Java 26 AI Agents on Kubernetes 1.34 with GitOps

Titouan Despierres — Tue, 04 Aug 2026 10:53:19 +0000

Beyond the Hype: Deploying Java 26 AI Agents on Kubernetes 1.34 with GitOps

As we enter March 2026, the intersection of Java 26, Kubernetes 1.34, and Agentic AI has moved from experimental labs to mission-critical production. For DevOps and Platform Engineers, the challenge has shifted: it's no longer just about "getting it to run," but about building a predictable, observable, and secure delivery lifecycle for high-memory, compute-intensive Java AI workloads.

In this guide, we’ll explore the practical patterns for shipping Java 26 AI services using a "GitOps-First" approach with GitLab CI, GitHub Actions, and Argo CD.

1. Java 26: The AI Engine for 2026

Java 26 (scheduled for GA later this month) isn't just an incremental update. For AI workloads, two areas are game-changers:

JEP 472: Foreign Function & Memory API (Finalized)

If you are running LLM inference (via ONNX, Llama.cpp, or DeepSeek-based local models), JEP 472 is your best friend. It allows Java to interact with native GPU libraries and off-heap memory with zero-copy efficiency.

Impact: 30-40% reduction in GC pauses for high-throughput inference.
Strategy: Move your heavy weight tensors into MemorySegment to keep the JVM heap lean and predictable.

JEP 530: Primitive Types in Patterns

When processing massive AI datasets or vector embeddings, the ability to use primitive types in patterns reduces boxing overhead and makes your data pipeline code significantly cleaner.

2. The Kubernetes 1.34 Landing Zone

Kubernetes 1.34 introduces refined APIs for Dynamic Resource Allocation (DRA), which is crucial for managing GPU-bound Java pods.

Resource Slicing & DRA

Instead of just asking for nvidia.com/gpu: 1, we now use DRA to request specific memory slices or multi-instance GPU (MIG) profiles directly in the Pod spec.

Example K8s Manifest (AI Agent Pod):

apiVersion: v1
kind: Pod
metadata:
  name: java-ai-agent
  labels:
    app: ai-gateway
spec:
  containers:
  - name: java-26-app
    image: my-registry.io/java-ai-agent:v1.2.0
    resources:
      claims:
      - name: gpu-resource
    env:
    - name: JAVA_OPTS
      value: "-XX:MaxRAMPercentage=75.0 -XX:+UseZGC -XX:+ZGenerational"
  resourceClaims:
  - name: gpu-resource
    resourceClassName: gpu-low-latency

3. The CI/CD Pipeline: Building for Resilience

A modern Java AI pipeline needs to handle more than just unit tests. It needs to validate Model Compatibility and Memory Footprint.

GitLab CI: Multi-Arch Build & Test

Leveraging GitLab’s rules and parallel matrix to build for both x86 (for cloud) and ARM64 (for edge/local dev).

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy

build_image:
  stage: build
  image: docker:27.0
  services:
    - docker:27.0-dind
  script:
    - docker buildx create --use
    - docker buildx build --platform linux/amd64,linux/arm64 
      -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --push .

verify_memory_leak:
  stage: test
  script:
    - ./mvnw verify -Pstress-test
  artifacts:
    reports:
      junit: target/surefire-reports/*.xml

GitHub Actions: Security & Compliance

Use GitHub Actions to ensure your AI agents aren't leaking secrets through logs or using insecure base images.

# .github/workflows/prod-delivery.yml
name: AI Agent Delivery
on:
  push:
    branches: [ main ]

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'my-registry.io/java-ai-agent:${{ github.sha }}'
          format: 'table'
          exit-code: '1'
          ignore-unfixed: true

4. GitOps with Argo CD: The Source of Truth

The "Manual Kubectl" era is over. Every change to your AI infrastructure must go through Git.

Kustomize Overlay Pattern

We use Kustomize to manage environment-specific configurations (e.g., smaller GPUs for Staging, A100s for Prod).

Project Structure:

infrastructure/
├── base/
│   └── deployment.yaml
└── overlays/
    ├── staging/
    │   └── kustomization.yaml
    └── prod/
        ├── kustomization.yaml
        └── gpu-patch.yaml

Argo CD Application Spec:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: java-ai-agent-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/my-org/infra-gitops.git
    targetRevision: HEAD
    path: overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: ai-production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

5. Production Best Practices: The "Anti-Fragile" Setup

Deploying is easy; staying up is hard.

Observability: Use OpenTelemetry for Java. Instrument your LLM calls to track tokens/sec and latency per JEP 472 call.
Progressive Rollouts: Use Argo Rollouts with Analysis templates. If the error rate (HTTP 5xx) increases or JVM memory exceeds 90%, auto-rollback.
Security: Implement NetworkPolicies to ensure your Java agent only talks to the Vector Database and the Model Registry, not the whole cluster.

Useful Command: Monitoring ZGC in Real-time

kubectl exec -it <pod-name> -- jstat -gc <pid> 1000

Look for ZGC "Cycles" to ensure the generational ZGC is keeping up with your AI inference load.

Conclusion: Strategy for Adoption

If you are moving to Java 26 for AI:

Pilot with Generational ZGC: It’s the single biggest win for large-heap Java AI apps.
Shift Left Security: Integrate container scanning in GitLab/GitHub immediately.
Commit to GitOps: Stop using helm install. Let Argo CD manage the drift.

The future of AI is Java-shaped, and the platform is Kubernetes.

Tags: #java #kubernetes #devops #ai

Beyond the Hype: Building Production-Grade Java AI Control Planes on Kubernetes (2026)

Titouan Despierres — Tue, 04 Aug 2026 10:53:12 +0000

Beyond the Hype: Building Production-Grade Java AI Control Planes on Kubernetes (2026)

The landscape of AI development has shifted. In 2026, the initial "experimentation" phase is over. Organizations are no longer asking if they should integrate LLMs, but how to do it reliably, safely, and at scale. For the Java ecosystem, this has been a transformative year. With the release of Java 24 (and 25 around the corner), the JVM has evolved from a back-office workhorse into a high-performance engine for AI orchestration and inference.

In this guide, we’ll move past the "Hello World" examples and look at how to build a production-grade AI control plane using Java, Kubernetes, and modern GitOps workflows.

1. Java 24: The Secret Weapon for AI Orchestration

Why Java for AI in 2026? While Python remains the king of model training, Java has become the preferred choice for the Control Plane and Orchestration Layer.

The Impact of Project Panama (JEP 454) and Java 24

In Java 24, refinements to the Foreign Function & Memory API (Panama) have allowed Java applications to interact with native AI libraries (like llama.cpp or onnxruntime) with zero overhead. We are seeing performance parity with C++ while maintaining Java's safety.

Strategy for Adoption:

Don't just upgrade the JDK. Use Generational ZGC (now the default in most 2026 deployments) to handle the large heap sizes required by vector embeddings without the latency spikes of traditional GCs.

2. CI/CD: The Gatekeeper of AI Quality

Shipping AI is different from shipping microservices. Your CI/CD needs to validate not just code, but model behavior and resource constraints.

GitHub Actions: Automated LLM Benchmarking

We now use GitHub Actions to run "vulnerability scans" on our prompts and model parameters. Here is a pattern for a gatekeeper workflow:

name: AI Quality Gate
on: [pull_request]

jobs:
  benchmarking:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 24
        uses: actions/setup-java@v4
        with:
          java-version: '24'
          distribution: 'temurin'
      - name: Run Prompt Evaluations
        run: |
          mvn test -Dtest=PromptInjectionSafetyTest
          mvn exec:java -Dexec.mainClass="com.acme.ai.BenchmarkRunner"

GitLab CI: Multi-Arch Build Pipelines

Since AI workloads often require ARM64 (for Graviton/Ampere) or GPU-enabled runners, GitLab CI's tag-based routing is essential.

build-ai-service:
  stage: build
  tags:
    - gpu-runner
  script:
    - docker build --build-arg JDK_VERSION=24 -t $REGISTRY/$IMAGE:$CI_COMMIT_SHA .
    - docker push $REGISTRY/$IMAGE:$CI_COMMIT_SHA

3. Kubernetes 1.33: Native Support for AI Workloads

Kubernetes 1.33 has introduced significant changes in how it handles "Sidecar Containers" and "Resource Claims," which are critical for AI.

Sidecar Containers for Observability

In our 2026 stack, we deploy a "Token Monitoring" sidecar to every Java AI pod. This ensures we track cost and latency at the edge.

Kubernetes Manifest (Kustomize Pattern)

Use Kustomize to manage the differences between CPU-based development and GPU-based production environments.

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-ai-orchestrator
spec:
  template:
    spec:
      containers:
      - name: orchestrator
        image: acme/java-ai-app:latest
        resources:
          limits:
            nvidia.com/gpu: 1 # Requesting GPU via Device Plugin

4. GitOps with Argo CD: Zero-Touch Rollouts

In 2026, we don't kubectl apply. We use GitOps to ensure that if an AI model starts hallucinating or leaking memory, we can revert the entire infrastructure state in seconds.

The "Progressive Delivery" Pattern

Using Argo Rollouts, we perform "Blue-Green" deployments for our Java services. This allows us to run a "Shadow" version of the AI model, comparing its output with the current version before switching traffic.

# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: ai-service-rollout
spec:
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 1h}
      - analysis:
          templates:
          - templateName: success-rate-and-latency

Conclusion: The Strategy for 2026

The goal is reproducibility. By combining the type-safety and performance of Java 24 with the declarative power of Kubernetes and GitOps, we create a system where AI is not a "black box" but a manageable, observable part of the enterprise stack.

Action Plan:

Audit: Move your Java AI services to JDK 24.
Automate: Integrate LLM benchmarking into your GitHub Actions or GitLab CI.
Orchestrate: Use Argo CD to manage model versioning as infrastructure.

About the Author: Specialist in Java, AI, and Cloud-Native architectures. Helping teams scale their DevOps practices for the next generation of software.

Production-Grade Java AI in 2026: The GitOps & Kubernetes Blueprint

Titouan Despierres — Tue, 04 Aug 2026 10:53:04 +0000

Production-Grade Java AI in 2026: The GitOps & Kubernetes Blueprint

As we move through 2026, the intersection of High-Performance Java (JDK 25/26), Generative AI, and Cloud Native Platform Engineering has reached a tipping point. We are no longer just "experimenting" with LLMs in sidecars; we are building robust, resilient AI inference gateways that must meet the same (or stricter) SLAs as our core banking or retail services.

In this guide, we’ll dive deep into the technical architecture and CI/CD patterns required to ship Java AI services on Kubernetes using GitLab CI/GitHub Actions and Argo CD.

1. The Java 26 Advantage for AI Workloads

Java has undergone a massive transformation. For AI workloads, two areas stand out: Project Panama (Foreign Function & Memory API) and Project Loom (Virtual Threads).

Why it matters in 2026:

Panama: Direct, safe access to off-heap memory and native libraries (like llama.cpp or custom CUDA kernels) without the overhead of JNI.
Loom: AI inference is often I/O-bound (waiting for the model response). Virtual threads allow us to handle thousands of concurrent AI requests with minimal memory footprint, replacing complex reactive stacks in many cases.

Practical Example: Vector Memory Management

Using the Foreign Function API to manage large vector embeddings off-heap reduces GC pressure significantly during high-traffic inference.

// Example: Allocating off-heap memory for vector buffers (JDK 25/26 style)
try (Arena arena = Arena.ofConfined()) {
    MemorySegment vectorBuffer = arena.allocate(1024 * 4); // 4KB for embeddings
    // Native call to model inference engine here...
}

2. Kubernetes 1.33: Optimized for AI/ML

Kubernetes has matured its support for heterogeneous hardware. In 2026, Dynamic Resource Allocation (DRA) is the standard for managing GPU slices for Java pods.

The Pattern: DRA and PriorityClasses

Ensure your AI Gateway pods have higher priority to prevent eviction during node pressure, and use DRA to request specific GPU profiles.

# K8s Manifest snippet for AI Service
apiVersion: v1
kind: Pod
metadata:
  name: java-ai-inference
spec:
  priorityClassName: high-priority-apps
  containers:
  - name: ai-service
    image: registry.gitlab.com/acme/java-ai-svc:latest
    resources:
      claims:
      - name: gpu-resource
  resourceClaims:
  - name: gpu-resource
    resourceClassName: nvidia-gpu-v100-slice

3. The CI/CD Pipeline: GitLab CI & GitHub Actions

The goal is reproducibility and security. We use a dual-approach: GitLab CI for enterprise-grade security scanning and GitHub Actions for developer-centric automation.

GitLab CI: Security-First Build

We leverage GitLab's native OIDC to authenticate with Kubernetes/Argo CD without storing long-lived secrets.

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy

build_image:
  stage: build
  script:
    - ./mvnw package -Pnative # Using GraalVM for fast startup
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

deploy_staging:
  stage: deploy
  environment: staging
  script:
    - # Use argocd CLI to sync or update git repo
    - git clone https://oauth2:$GIT_OPS_TOKEN@gitlab.com/acme/gitops-infra.git
    - cd gitops-infra
    - kustomize edit set image ai-service=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - git commit -am "Update AI service to $CI_COMMIT_SHORT_SHA"
    - git push

GitHub Actions: Efficient Workflows

For teams on GitHub, using container-structure-test ensures your Java image is production-ready.

# .github/workflows/main.yml
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 25
        uses: actions/setup-java@v4
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Build with Maven
        run: mvn clean package
      - name: Publish to GHCR
        run: |
          echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
          docker build -t ghcr.io/${{ github.repository }}:latest .
          docker push ghcr.io/${{ github.repository }}:latest

4. GitOps with Argo CD: The Source of Truth

In 2026, manual kubectl apply is a relic of the past. Argo CD manages the state, ensuring that what’s in Git is what’s on the cluster.

Rollout Strategy: Progressive Delivery with Argo Rollouts

AI models are risky. We use Canary deployments to shift traffic gradually while monitoring error rates and inference latency.

# Argo Rollout Manifest
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: java-ai-service
spec:
  replicas: 5
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 5m} # Monitor for regressions
      - setWeight: 50
      - pause: {duration: 10m}

5. Production Best Practices: Observability & Rollback

The "Golden Signals" of AI

Inference Latency (P99): Not just the API, but the model execution time.
GPU Utilization: Ensure you aren't over-provisioning expensive resources.
Token Cost Tracking: Instrument your Java code to export metrics on token usage per user/request.

Automated Rollback

If the error rate exceeds 1% during a Canary rollout, Argo CD automatically aborts and rolls back to the previous stable image.

Conclusion: Strategy for Adoption

Start with JDK 21+: If you are still on 11 or 17, the gap to 25/26 is widening. Focus on record types and virtual threads first.
Platform Engineering: Build an "AI paved road" for your developers. Provide Helm charts that already include the DRA claims and observability sidecars.
GitOps is Non-Negotiable: With the complexity of AI hardware and model versions, GitOps is the only way to maintain sanity.

Happy shipping in 2026!

java #kubernetes #devops #ai

Building Resilience: Progressive Rollouts for Java AI Microservices with Argo CD and GitHub Actions

Titouan Despierres — Tue, 04 Aug 2026 10:52:54 +0000

Building Resilience: Progressive Rollouts for Java AI Microservices with Argo CD and GitHub Actions

In 2026, the landscape of Java development has shifted. We are no longer just building REST APIs; we are deploying high-performance AI inference engines and RAG (Retrieval-Augmented Generation) services. With the release of JDK 26 and Kubernetes 1.33, the focus has moved from simple deployment to complex, resilient orchestration.

This guide explores a battle-tested pattern for deploying Java 26 AI services using a GitOps approach with Argo CD, GitHub Actions, and Kustomize, focusing on progressive rollouts and observability.

1. The Java 26 Edge: Native Memory and AI

With JDK 26, the Foreign Function & Memory API (JEP 472) and Vector API have matured. For AI workloads using libraries like LangChain4j or Spring AI that interface with local LLMs (via llama.cpp) or vector databases, managing off-heap memory is critical.

Key Change: We can now interface with C++ or CUDA kernels with near-zero overhead.
Impact: Java is no longer "too slow" or "too heavy" for the data plane of AI services.

JVM Strategy for 2026

When running on Kubernetes, your JVM needs to be aware of the cgroup limits. In Java 26, -XX:MaxRAMPercentage=75.0 is still your friend, but you must also account for off-heap memory used by native AI libraries.

2. CI: The Automated Pipeline (GitHub Actions)

Our CI pipeline doesn't just build a JAR; it produces a hardened OCI image and updates the GitOps repository.

.github/workflows/ci.yml

name: CI/CD Pipeline
on:
  push:
    branches: [ main ]
jobs:
  build-and-push:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 26
        uses: actions/setup-java@v4
        with:
          java-version: '26'
          distribution: 'temurin'

      - name: Build with Maven
        run: ./mvnw clean package

      - name: Docker Build & Push
        run: |
          docker build -t ghcr.io/my-org/ai-service:${{ github.sha }} .
          echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
          docker push ghcr.io/my-org/ai-service:${{ github.sha }}

  update-gitops:
    needs: build-and-push
    runs-on: ubuntu-latest
    steps:
      - name: Checkout GitOps Repo
        uses: actions/checkout@v4
        with:
          repository: my-org/gitops-infra
          token: ${{ secrets.GITOPS_PAT }}

      - name: Update Kustomize Image
        run: |
          cd overlays/production
          kustomize edit set image ai-service=ghcr.io/my-org/ai-service:${{ github.sha }}

      - name: Commit and Push
        run: |
          git config user.name "CI Bot"
          git config user.email "ci@my-org.com"
          git add .
          git commit -m "deploy: update ai-service to ${{ github.sha }}"
          git push

3. CD: Progressive Rollouts with Argo CD & Rollouts

Standard Kubernetes Deployment (RollingUpdate) is often insufficient for AI workloads where a bad model weight update can cause silent failures or high latency. We use Argo Rollouts for Canary deployments.

Argo Rollout Manifest (`rollout.yaml`)

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: ai-service
spec:
  replicas: 5
  selector:
    matchLabels:
      app: ai-service
  template:
    metadata:
      labels:
        app: ai-service
    spec:
      containers:
      - name: ai-service
        image: ghcr.io/my-org/ai-service:v1
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: "2"
            memory: "4Gi"
  strategy:
    canary:
      steps:
      - setWeight: 20
      - pause: { duration: 5m }
      - setWeight: 50
      - pause: { duration: 10m }
      - analysis:
          templates:
          - templateName: success-rate

4. Kubernetes 1.33: Sidecar Containers for Observability

In Kubernetes 1.33, the SidecarContainers feature is the production standard. We use it to run an OTel (OpenTelemetry) collector alongside our Java application to capture high-resolution AI metrics (token throughput, prompt latency).

K8s Manifest excerpt

spec:
  containers:
  - name: ai-service
    image: ai-service:latest
  initContainers:
  - name: otel-collector
    image: otel/opentelemetry-collector:latest
    restartPolicy: Always # Kubernetes 1.29+ Native Sidecar

5. Security and Compliance in the Pipeline

With GitLab CI, we can leverage the native Secret Detection and Dependency Scanning for our Java dependencies.

.gitlab-ci.yml (Security focused)

include:
  - template: Jobs/Dependency-Scanning.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml

build:
  image: eclipse-temurin:26-jdk
  script:
    - ./gradlew build
  artifacts:
    paths:
      - build/libs/*.jar

Summary & Strategy for Adoption

Phase 1: Migrate your base image to JDK 26 to benefit from the latest GC optimizations (ZGC refinements).
Phase 2: Implement GitOps with Argo CD to decouple deployment from CI.
Phase 3: Use Argo Rollouts for any service handling LLM inference to ensure stability during updates.

By combining the performance of Java 26 with the reliability of GitOps, we create a platform capable of scaling AI workloads in production with confidence.

Tags: #java #kubernetes #devops #ai

Building Resilient Java AI Platforms in 2026: A Kubernetes & GitOps Masterclass

Titouan Despierres — Tue, 04 Aug 2026 10:50:36 +0000

Building Resilient Java AI Platforms in 2026: A Kubernetes & GitOps Masterclass

As we navigate through early 2026, the intersection of Java 26, Kubernetes 1.33, and AI-driven Platform Engineering has redefined what "production-ready" means. It's no longer just about shipping code; it's about building self-healing, observable, and secure ecosystems.

In this guide, we’ll explore the latest advancements in the Java ecosystem, deep-dive into Kubernetes 1.32/1.33 features like CrashLoopBackOff fine-tuning, and implement a robust GitOps pipeline using GitHub Actions, GitLab CI, and Argo CD.

1. Java 26: Performance and The AI Bridge

With Java 26 entering its Release Candidate phase (February 2026), the focus has shifted towards the finalized features of Project Panama and Project Loom.

Why it matters for AI and Ops:

Foreign Function & Memory API (Finalized): Allows Java to interface with native AI libraries (C++, CUDA) with near-zero overhead. This is crucial for running LLM inference directly within JVM-based microservices.
Structured Concurrency: Simplifies handling multiple AI model calls in parallel, ensuring that if one sub-task fails, the entire scope is cleaned up, preventing resource leaks in K8s pods.

Implementation: AI Service Wrapper

Here is how we leverage Java 26 to call a native inference engine safely:

public class AIService {
    public String generateResponse(String prompt) {
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            var result = scope.fork(() -> nativeInferenceEngine.call(prompt));
            scope.join().throwIfFailed();
            return result.get();
        } catch (Exception e) {
            throw new RuntimeException("AI Inference Failed", e);
        }
    }
}

2. Kubernetes 1.32/1.33: Platform Engineering Evolution

The recent releases of K8s (1.32 in late 2025 and 1.33 early 2026) have introduced game-changing features for Platform Engineers.

Key Highlight: `CrashLoopBackOff` Fine-Tuning

Historically, CrashLoopBackOff followed a fixed exponential backoff. In K8s 1.32+, we can now configure the maxContainerTerminationMessageLength and observe better pod restart logic. This allows for faster recovery of stateful Java applications that might need a specific "cool down" but not a full 5-minute wait.

Manifest: Optimized Java Pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-ai-service
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: app
        image: ghcr.io/acme/java-ai-app:latest
        resources:
          limits:
            memory: "2Gi"
            cpu: "1"
          requests:
            memory: "1Gi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /actuator/health/liveness
            port: 8080
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 8080

3. The Unified CI/CD Pipeline: GitHub vs. GitLab

In 2026, the debate between GitHub Actions and GitLab CI has matured into "use the best of both worlds."

GitHub Actions: The Event-Driven Powerhouse

GitHub Actions excels at integration with the developer workflow. Using the new Custom Autoscaling for Runner Scale Sets (released Feb 2026), we can scale our build fleet outside of K8s if needed.

.github/workflows/main.yml

name: Build and Push
on:
  push:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          java-version: '26'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew bootJar
      - name: Build Image
        run: |
          docker build -t ghcr.io/acme/java-ai-app:${{ github.sha }} .
          docker push ghcr.io/acme/java-ai-app:${{ github.sha }}

GitLab CI: Security and Compliance First

GitLab remains the king of built-in security dashboards and complex multi-project pipelines.

.gitlab-ci.yml

stages:
  - test
  - build
  - deploy

variables:
  DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

test:
  stage: test
  image: eclipse-temurin:26
  script:
    - ./gradlew test

build_image:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker build -t $DOCKER_IMAGE .
    - docker push $DOCKER_IMAGE

4. GitOps with Argo CD: Closing the Loop

Shipping the image is only half the battle. Argo CD ensures that what is in Git is exactly what is in production. In 2026, the trend is ApplicationSet for multi-cluster management.

Argo CD ApplicationSet Pattern

This pattern allows us to deploy the Java AI service across multiple clusters (Dev, Staging, Prod) automatically.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: java-ai-apps
spec:
  generators:
  - list:
      elements:
      - cluster: engineering-dev
        url: https://kubernetes.default.svc
      - cluster: engineering-prod
        url: https://prod-cluster.acme.com
  template:
    metadata:
      name: '{{cluster}}-java-ai'
    spec:
      project: default
      source:
        repoURL: https://github.com/acme/gitops-repo.git
        targetRevision: HEAD
        path: manifests/java-ai
      destination:
        server: '{{url}}'
        namespace: java-ai
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

5. Production Best Practices for 2026

1. Security (Supply Chain)

Use SBOM (Software Bill of Materials) generation in your CI. Kubernetes 1.32+ has improved support for verifying image signatures via Admission Controllers.

2. Observability (OpenTelemetry)

Java 26 microservices should use the OpenTelemetry Java Agent for auto-instrumentation. Ensure your K8s cluster has the OpenTelemetry Operator installed to collect these metrics.

3. Rollout Strategy

Always use Canary deployments. Argo Rollouts (often used alongside Argo CD) is the preferred choice to ensure that a failing AI model doesn't bring down your entire production traffic.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: java-ai-rollout
spec:
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 10m}
      - setWeight: 50

Conclusion: The Path Forward

The "Expert Developer" in 2026 is no longer just a coder but a System Architect. By mastering the synergy between Java's native performance, Kubernetes' orchestration intelligence, and GitOps automation, you build platforms that aren't just modern—they are future-proof.

Strategy for Adoption:

Upgrade to Java 25 LTS now to prepare for Java 26.
Implement Argo CD for your staging environments.
Start using GitHub Actions ARC for cost-effective CI scaling.

What's your biggest challenge with K8s and Java AI today? Let's discuss in the comments!

java #kubernetes #devops #ai

Scaling Java 26 AI Workloads: A 2026 Production Playbook (GitOps & Kubernetes)

Titouan Despierres — Thu, 26 Feb 2026 10:58:03 +0000

Scaling Java 26 AI Workloads: A 2026 Production Playbook (GitOps & Kubernetes)

The landscape of enterprise development in early 2026 is defined by a singular challenge: moving beyond AI experimentation into reliable, high-scale production operations. With the arrival of JDK 26-RC1, the promise of Project Loom (Virtual Threads) and Project Panama (Foreign Function & Memory API) has matured into the backbone of high-performance AI integration in the Java ecosystem.

This article provides a practical blueprint for architecting, building, and deploying Java 26 AI services on Kubernetes using a modern GitOps flow with GitHub Actions, GitLab CI, and Argo CD.

1. The Java 26 Advantage: Why JDK 26 for AI?

JDK 26 brings significant refinements that directly impact how we handle AI inference and data processing.

Project Panama: Native Model Interaction

The Foreign Function & Memory API (JEP 472) is no longer "new"—it is the standard. In 2026, we use it to interface directly with C++ AI libraries (like llama.cpp or custom CUDA kernels) without the overhead of JNI.

Performance: Reduced latency when passing large tensors between Java and native memory.
Safety: Deterministic memory management for off-heap AI model weights.

Virtual Threads (Loom) at Scale

For I/O-bound AI services (calling external LLM APIs like OpenAI, Anthropic, or internal vLLM clusters), Virtual Threads allow us to handle thousands of concurrent requests with a tiny footprint.

2. The Build Pipeline: Containerizing JDK 26

A production-grade pipeline must focus on security and size. We use multi-stage Docker builds with jlink to strip down the JDK to only the required modules.

Modern GitHub Actions Workflow

name: Build and Push Java AI Service
on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK 26
        uses: actions/setup-java@v4
        with:
          java-version: '26-ea'
          distribution: 'temurin'
          cache: 'maven'

      - name: Build with Maven
        run: mvn clean package -DskipTests

      - name: Create Custom JRE via jlink
        run: |
          $JAVA_HOME/bin/jlink \
            --add-modules java.base,java.net.http,jdk.management \
            --strip-debug \
            --no-man-pages \
            --no-header-files \
            --compress=2 \
            --output custom-jre

      - name: Build & Push Image
        run: |
          docker build -t registry.example.com/ai-service:${{ github.sha }} .
          docker push registry.example.com/ai-service:${{ github.sha }}

3. The GitLab CI Parallel: Enterprise Readiness

If you are on GitLab, leverage Environment Stop and Security Scanning as first-class citizens.

stages:
  - test
  - build
  - security
  - deploy

container_scanning:
  stage: security
  image:
    name: aquasec/trivy:latest
  script:
    - trivy image --severity HIGH,CRITICAL registry.example.com/ai-service:$CI_COMMIT_SHA

4. Kubernetes & GitOps: The Argo CD Pattern

In 2026, manual kubectl apply is a relic of the past. We use Argo CD for declarative, versioned deployments.

The Kustomize Overlay

AI workloads often require specific GPU resources. Use Kustomize to inject resource limits only for production.

# overlays/production/resources-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-ai-service
spec:
  template:
    spec:
      containers:
      - name: app
        resources:
          limits:
            nvidia.com/gpu: 1
            memory: "8Gi"
          requests:
            cpu: "2"
            memory: "4Gi"

The Argo CD Application manifest

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: java-ai-service-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/gitops-config.git
    targetRevision: HEAD
    path: apps/java-ai-service/overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: ai-production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

5. Observability & Rollout Strategies

AI services are prone to model drift and latency spikes. Implementing a Canary Rollout with Argo Rollouts is essential.

Why Canary?

Safety: Traffic is shifted incrementally (10% -> 20% -> 50% -> 100%).
Verification: If LLM response latency exceeds 500ms or error rates climb, the system triggers an automatic rollback.

# rollout.yaml (Argo Rollouts)
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: java-ai-service
spec:
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: { duration: 5m }
      - setWeight: 50
      - pause: { duration: 10m }

6. Adoption Strategy: How to Start

Audit your JDK version: If you are still on JDK 17, skip 21 and target JDK 25 (LTS) or 26 (Latest) to leverage Panama.
Move to GitOps: Stop using CI pipelines to "push" to K8s. Use them to update a GitOps repo that Argo CD "pulls" from.
Isolate AI Logic: Keep your "Orchestration" (Java) separate from your "Inference" (C++/Python/CUDA) using Panama or gRPC for maximum stability.

Conclusion

Java's role in the AI era is not as the model-training language, but as the reliable platform engineering language. By combining JDK 26's native efficiencies with Kubernetes-native GitOps, we build systems that are not just smart, but production-hardened.

Tags: #java #kubernetes #ai #devops