DEV Community: Sushil Baligar

🚀 Major CI/CD Pipeline Enhancements Completed! 🐳

Sushil Baligar — Sat, 03 Jan 2026 17:05:05 +0000

Following up on my previous post about building a Jenkins CI/CD pipeline, I'm excited to share the advanced features I've implemented:

🔄 Production-Ready Deployments
✅ Docker containerization with multi-stage builds
✅ Automated deployment to AWS EC2
✅ Zero-downtime deployments
✅ Live at: https://sushilkumarsb.xyz/app/

🛡️ Intelligent Rollback Mechanism
✅ Automated health check validation (10 retries)
✅ Auto-rollback on deployment failure
✅ Manual rollback via Jenkins parameters
✅ Deployment state tracking for recovery

📊 Performance Monitoring
✅ Stage-level timing metrics
✅ Total pipeline duration tracking
✅ Build performance optimization (~35s end-to-end)
✅ Docker image cleanup automation

🌿 Flexible Deployment
✅ Build any branch via Jenkins parameter
✅ Feature branch testing before merge
✅ GitHub webhook integration for auto-triggers

🔐 Security & Best Practices
✅ Multi-stage Docker builds (reduced image size)
✅ Non-root container user
✅ Minimal attack surface
✅ Secure credential handling

📈 Key Metrics:
• 50% deployment time reduction
• 92% test coverage (30 comprehensive tests)
• 100% rollback success rate
• 40+ releases/year capability

What I Learned:

Automated rollback is critical for production reliability
Health checks prevent bad deployments from staying live
Performance metrics help identify bottlenecks
Docker multi-stage builds significantly reduce image size
Infrastructure automation saves hours of manual work

Next on my roadmap:
🔜 Kubernetes orchestration
🔜 Prometheus & Grafana monitoring
🔜 SonarQube security scanning
🔜 Blue-Green deployments

Building DevOps pipelines that handle failures gracefully is as important as handling success! Every production deployment should have a tested rollback strategy.

💡 What's your approach to deployment safety? Do you have automated rollbacks in your pipelines?

GitHub: https://github.com/sushilkumarsb/jenkins-cicd-automation
Linkedin: linkedin.com/in/sushilkumarsb

DevOps #Jenkins #CICD #Docker #AWS #Python #Automation #SoftwareEngineering #CloudComputing #DevSecOps #ContinuousDeployment #SiteReliabilityEngineering

Jenkins CI/CD Project

Sushil Baligar — Sat, 03 Jan 2026 07:11:07 +0000

🚀 From Local Development to Production: My Journey Building a Complete CI/CD Pipeline

I recently deployed a fully automated Jenkins CI/CD pipeline on AWS, and I want to share what I learned to help others navigate this journey.

🎯 What I Built

A production-ready CI/CD pipeline that reduced deployment time by 50% (10 minutes → 5 minutes) with:

✅ AWS EC2 deployment with custom domain and SSL
✅ GitHub webhook integration for instant build triggers (<5 seconds)
✅ 92% test coverage with 30 comprehensive unit tests
✅ Python virtual environments for isolated dependencies
✅ Nginx reverse proxy with Let’s Encrypt SSL
✅ Full automation — 40+ releases/year capability

Live Demo: https://sushilkumarsb.xyz/job/jenkins-cicd-automation/
GitHub: https://github.com/sushilkumarsb/jenkins-cicd-automation

— -

💡 Key Challenges & Solutions

Challenge 1: Ubuntu 24.04 PEP 668 Restrictions
Python’s new security model blocked system-wide pip installs. Solution? Implemented virtual environments in the Jenkins pipeline — better practice anyway!

Challenge 2: Windows → Linux Pipeline Conversion
My Jenkinsfile used Windows bat commands. Converted to sh with python3 for the Ubuntu EC2 instance. Cross-platform awareness is crucial.

Challenge 3: Professional Setup vs. Cost
Achieved enterprise-grade deployment (SSL, custom domain, reverse proxy) for just $12/month using:

AWS EC2 free tier eligible
Let’s Encrypt (FREE SSL)
Nginx (open source)
GoDaddy domain ($12/year)

Challenge 4: GitHub Webhook Integration
Configured real-time triggers requiring HTTPS, proper DNS, and security group configurations. The result? Instant builds on every push.

Press enter or click to view image in full size

— -

🏗️ Architecture Overview

Developer (Git Push)
↓
GitHub Webhook (HTTPS)
↓
AWS EC2 Ubuntu 24.04
↓
Nginx :443 (SSL Termination)
↓
Jenkins :8080 (CI/CD Pipeline)
↓
Build → Test (pytest + coverage) → Code Quality
↓
Automated Results & Reports

— -

Become a member

📚 What I Learned (For Others Starting Out)

Press enter or click to view image in full size

1. Start Small, Scale Up
I began with local Jenkins, then moved to AWS. Don’t try to build everything at once.

2. Infrastructure as Code
Everything in my Jenkinsfile is version-controlled. No manual configurations that can be lost.

3. Security First

SSL encryption via Let’s Encrypt
Jenkins accessible only through Nginx reverse proxy
Port 8080 blocked from internet (localhost only)
Matrix-based authorization with read-only public access

4. Test Thoroughly
30 unit tests covering edge cases, error handling, and concurrent requests. Testing isn’t optional — it’s your safety net.

5. Document Everything
Created comprehensive guides for setup, troubleshooting, and interview prep. Future you will thank present you.

— -

🎓 Technologies Used

Backend: Python 3.12, Flask 3.0
Testing: pytest, pytest-cov (92% coverage)
CI/CD: Jenkins 2.528, Groovy
Cloud: AWS EC2 (Ubuntu 24.04)
Web Server: Nginx (reverse proxy + SSL)
Security: Let’s Encrypt SSL, AWS Security Groups
Version Control: Git, GitHub webhooks
Automation: Bash scripting, Python venv

— -

💼 Impact Metrics

⚡ 50% deployment time reduction (10 min → 5 min)
🧪 92% code coverage with 30 comprehensive tests
🚀 40+ releases/year capability through automation
☁️ Production-ready AWS deployment with SSL
💰 Cost-effective (~$12/month for full stack)
🔒 Zero security incidents — proper auth & encryption

— -
Let’s connect! I’m happy to share knowledge, discuss opportunities, or help others on their DevOps journey.
🔗 Portfolio: https://github.com/sushilkumarsb, https://www.linkedin.com/in/sushilkumarsb/

— -

P.S. The entire project is open source! Check out my GitHub for the code, documentation, and detailed setup guides. Feel free to fork, learn, and ask questions. We all grow together. 🌱

If this helped you or inspired your own CI/CD journey, I’d love to hear about it in the comments!

#DevOps #Jenkins #CICD #AWS #CloudComputing #Automation #Python #SRE #CloudEngineer #InfrastructureAsCode

What Exactly Is the “Bazel Ecosystem”?

Sushil Baligar — Sat, 20 Dec 2025 19:42:11 +0000

When people say “Bazel ecosystem,” they’re not just talking about the bazel binary. They’re talking about the entire world of rules, tools, and community projects that make Bazel useful in real engineering teams.

Let’s unpack that in plain language.

Bazel: The build engine at the core.
At the center sits Bazel itself: an open‑source build and test tool originally open‑sourced by Google’s Blaze team.

It reads your BUILD, WORKSPACE or MODULE.bazel files.
It analyzes dependencies and creates an action graph.
It then executes only what’s needed, with heavy use of parallelism and caching to keep builds fast and reproducible. Think of this as the “compiler” for your entire monorepo build graph.
Rules and Starlark: How Bazel understands your world?
On its own, Bazel knows how to schedule work, but it doesn’t know what “build a Java library” or “build a Docker image” means. That knowledge lives in rules, written in the Starlark language.

You’ll see two broad categories:
a. Built‑in rules: For Java, C++, Go, Android, generic genrule, etc.
b. External rule sets: Community and official repositories like rules_python, rules_go, rules_docker, rules_k8s, and many more curated in “Awesome Bazel”.

If your team says “we use Bazel,” what they really mean is “we use Bazel plus a set of rules that teach it to our tech stack.”
Toolchains, platforms, and remote builds
Modern systems need to build for multiple platforms and sometimes offload heavy work to remote clusters. Bazel ecosystem covers that too.

Key pieces:
Platforms & toolchains: Describe OS/CPU plus the compilers and tools to use. This enables cross‑compilation and multi‑platform builds from a single configuration.
Remote cache & remote execution:
Optional services where build actions and outputs can be executed and cached remotely, so large teams don’t rebuild the same thing over and over. This is where Bazel moves from “just another build tool” to “infrastructure for build and test at scale”.
Developer tooling: Making Bazel pleasant to use.
Raw Bazel can feel low‑level. The ecosystem smooths that out with supporting tools. Typical examples:
a. Bazelisk and wrapper CLIs to manage Bazel versions and bootstrap workspaces.
b. IDE integrations for IntelliJ, VS Code, Android Studio and Xcode generators, so developers get code navigation and refactoring on top of Bazel builds.
c. Helper tools like bazel-diff, project generators, CI integrations, and repo‑specific utilities used at companies like Spotify, Atlassian, and others.

These tools turn Bazel from “powerful but painful” into something teams can adopt across an entire org.
Community, examples, and real‑world usage
Finally, an ecosystem is only as strong as its community.
The Bazel open‑source project is backed by a growing set of companies and OSS projects that use it for massive, multi‑language monorepos.

Curated lists like “Awesome Bazel” collect rules, tools, and example projects (from TensorFlow to Kubernetes and Envoy) so you can see how others structure real Bazel workspaces.
---------------------------------------------------------------------
If you’re evaluating Bazel, don’t just look at the core binary. Look at the rules you’ll rely on, the tooling your developers will touch daily, and the community resources you can learn from. That whole package is what people really mean when they talk about the Bazel ecosystem.

Modern Bazel with Python-Module 6: Production Deployment and CI/CD Integration

Sushil Baligar — Sat, 26 Jul 2025 07:35:25 +0000

Learning Objectives

By the end of this module, you will:

Master production-ready Bazel deployments
Implement comprehensive CI/CD pipelines with Bazel
Set up remote execution for scalable builds
Deploy containerized Python applications using Bazel
Monitor and maintain production Bazel systems
Implement security best practices for production builds

6.1 Production Build Configuration

Environment-Specific Configurations

Create robust configuration management for different environments:

# .bazelrc.production
# Production build configuration
build:prod --compilation_mode=opt
build:prod --strip=always
build:prod --define=environment=production
build:prod --stamp=true

# Security hardening
build:prod --sandbox_default_allow_network=false
build:prod --experimental_strict_action_env=true

# Performance optimization
build:prod --jobs=auto
build:prod --experimental_worker_memory_limit_mb=4096
build:prod --experimental_worker_multiplex

# Remote caching for production
build:prod --remote_cache=https://prod-cache.company.com
build:prod --remote_upload_local_results=true
build:prod --remote_timeout=300

# Production-specific flags
build:prod --define=log_level=ERROR
build:prod --define=enable_debug=false
build:prod --define=enable_metrics=true

# .bazelrc.staging  
build:staging --compilation_mode=opt
build:staging --define=environment=staging
build:staging --stamp=true
build:staging --remote_cache=https://staging-cache.company.com
build:staging --define=log_level=INFO
build:staging --define=enable_debug=true
build:staging --define=enable_metrics=true

# .bazelrc.development
build:dev --compilation_mode=fastbuild
build:dev --define=environment=development
build:dev --stamp=false
build:dev --spawn_strategy=local
build:dev --define=log_level=DEBUG
build:dev --define=enable_debug=true
build:dev --define=enable_metrics=false

Build Stamping for Production

Implement proper build stamping for version tracking:

# //tools/version.bzl
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

def _version_info_impl(ctx):
    """Generate version information from build stamp."""
    version_file = ctx.outputs.version_file

    # Read stamp information
    if ctx.attr.stamp:
        stamp_file = ctx.info_file
        stable_status = ctx.version_file
    else:
        stamp_file = None
        stable_status = None

    # Generate version.py
    ctx.actions.expand_template(
        template = ctx.file.template,
        output = version_file,
        substitutions = {
            "{BUILD_TIMESTAMP}": "$(BUILD_TIMESTAMP)",
            "{BUILD_USER}": "$(BUILD_USER)", 
            "{BUILD_HOST}": "$(BUILD_HOST)",
            "{GIT_COMMIT}": "$(BUILD_SCM_REVISION)",
            "{GIT_STATUS}": "$(BUILD_SCM_STATUS)",
        },
    )

    return [DefaultInfo(files = depset([version_file]))]

version_info = rule(
    implementation = _version_info_impl,
    attrs = {
        "template": attr.label(
            allow_single_file = True,
            default = "//tools:version_template.py",
        ),
        "stamp": attr.bool(default = False),
    },
    outputs = {"version_file": "%{name}.py"},
)

# //tools/version_template.py
"""Auto-generated version information."""

BUILD_TIMESTAMP = "{BUILD_TIMESTAMP}"
BUILD_USER = "{BUILD_USER}"
BUILD_HOST = "{BUILD_HOST}"
GIT_COMMIT = "{GIT_COMMIT}"
GIT_STATUS = "{GIT_STATUS}"

VERSION_INFO = {
    "build_timestamp": BUILD_TIMESTAMP,
    "build_user": BUILD_USER,
    "build_host": BUILD_HOST, 
    "git_commit": GIT_COMMIT,
    "git_status": GIT_STATUS,
}

def get_version():
    """Get formatted version string."""
    return f"{GIT_COMMIT[:8]}-{BUILD_TIMESTAMP}"

def get_build_info():
    """Get complete build information."""
    return VERSION_INFO

# //src/BUILD
load("//tools:version.bzl", "version_info")

version_info(
    name = "version",
    stamp = select({
        "//conditions:prod": True,
        "//conditions:staging": True,
        "//conditions:default": False,
    }),
)

py_library(
    name = "app_version",
    srcs = [":version"],
    visibility = ["//visibility:public"],
)

6.2 Container-Based Deployments

Docker Integration with Bazel

Create production-ready Docker images using Bazel:

# //deploy/BUILD
load("@rules_docker//python3:image.bzl", "py3_image")
load("@rules_docker//container:container.bzl", "container_push", "container_image")

# Base Python application image
py3_image(
    name = "app_base_image",
    srcs = ["//src:app_lib"],
    main = "//src:main.py",
    deps = [
        "//src:app_lib",
        "@pypi//fastapi",
        "@pypi//uvicorn",
        "@pypi//pydantic",
    ],
    # Use slim base image for production
    base = "@python3_base//image",
)

# Production image with optimizations
container_image(
    name = "app_prod_image", 
    base = ":app_base_image",
    env = {
        "ENVIRONMENT": "production",
        "LOG_LEVEL": "INFO",
        "PYTHONUNBUFFERED": "1",
        "PYTHONDONTWRITEBYTECODE": "1",
    },
    ports = ["8000"],
    workdir = "/app",
    # Add health check
    cmd = [
        "python", 
        "-m", "uvicorn",
        "src.main:app",
        "--host", "0.0.0.0", 
        "--port", "8000",
        "--workers", "4",
    ],
    # Add labels for production
    labels = {
        "maintainer": "devops@company.com",
        "version": "{BUILD_SCM_REVISION}",
        "environment": "production",
    },
)

# Multi-stage production image
container_image(
    name = "app_optimized_image",
    base = "@distroless_python3//image", 
    files = [
        "//src:app_binary_files",
    ],
    entrypoint = ["/app/main"],
    env = {
        "PYTHONPATH": "/app",
        "ENVIRONMENT": "production",
    },
    # Security: run as non-root
    user = "1000",
)

# Push to registry
container_push(
    name = "push_prod",
    image = ":app_prod_image",
    registry = "gcr.io",
    repository = "company-project/api-server",
    tag = "{BUILD_SCM_REVISION}",
)

container_push(
    name = "push_latest",
    image = ":app_prod_image", 
    registry = "gcr.io",
    repository = "company-project/api-server",
    tag = "latest",
)

Kubernetes Deployment

Generate Kubernetes manifests with Bazel:

# //k8s/BUILD
load("@io_bazel_rules_k8s//k8s:objects.bzl", "k8s_objects")
load("@io_bazel_rules_k8s//k8s:object.bzl", "k8s_object")

# Production deployment
k8s_object(
    name = "api_deployment_prod",
    kind = "deployment",
    template = ":deployment.yaml",
    substitutions = {
        "{IMAGE_TAG}": "{BUILD_SCM_REVISION}",
        "{ENVIRONMENT}": "production", 
        "{REPLICAS}": "3",
        "{CPU_REQUEST}": "100m",
        "{CPU_LIMIT}": "500m",
        "{MEMORY_REQUEST}": "128Mi",
        "{MEMORY_LIMIT}": "512Mi",
    },
)

k8s_object(
    name = "api_service_prod",
    kind = "service",
    template = ":service.yaml",
    substitutions = {
        "{ENVIRONMENT}": "production",
    },
)

k8s_object(
    name = "api_configmap_prod",
    kind = "configmap",
    template = ":configmap.yaml",
    substitutions = {
        "{ENVIRONMENT}": "production",
        "{LOG_LEVEL}": "INFO",
        "{DATABASE_URL}": "$(DATABASE_URL_PROD)",
        "{REDIS_URL}": "$(REDIS_URL_PROD)",
    },
)

# Complete application stack
k8s_objects(
    name = "api_stack_prod",
    objects = [
        ":api_deployment_prod",
        ":api_service_prod", 
        ":api_configmap_prod",
    ],
)

# Staging environment
k8s_objects(
    name = "api_stack_staging",
    objects = [
        ":api_deployment_staging",
        ":api_service_staging",
        ":api_configmap_staging", 
    ],
)

# //k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server-{ENVIRONMENT}
  namespace: {ENVIRONMENT}
  labels:
    app: api-server
    environment: {ENVIRONMENT}
    version: {IMAGE_TAG}
spec:
  replicas: {REPLICAS}
  selector:
    matchLabels:
      app: api-server
      environment: {ENVIRONMENT}
  template:
    metadata:
      labels:
        app: api-server
        environment: {ENVIRONMENT}
        version: {IMAGE_TAG}
    spec:
      containers:
      - name: api-server
        image: gcr.io/company-project/api-server:{IMAGE_TAG}
        ports:
        - containerPort: 8000
          name: http
        env:
        - name: ENVIRONMENT
          value: {ENVIRONMENT}
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: api-secrets
              key: database-url
        resources:
          requests:
            cpu: {CPU_REQUEST}
            memory: {MEMORY_REQUEST}
          limits:
            cpu: {CPU_LIMIT}
            memory: {MEMORY_LIMIT}
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 1000

6.3 CI/CD Pipeline Implementation

GitHub Actions Integration

Complete CI/CD pipeline with Bazel:

# .github/workflows/ci.yml
name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  BAZEL_VERSION: 7.0.0
  PYTHON_VERSION: 3.11

jobs:
  build-and-test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Full history for build stamping

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: ${{ env.PYTHON_VERSION }}

    - name: Install Bazel
      uses: bazel-contrib/setup-bazel@0.8.1
      with:
        bazelisk-version: 1.x

    - name: Configure Bazel caching
      run: |
        echo "build --remote_cache=https://storage.googleapis.com/${{ secrets.GCS_CACHE_BUCKET }}" >> .bazelrc.ci
        echo "build --google_default_credentials=true" >> .bazelrc.ci
        echo "build --remote_upload_local_results=true" >> .bazelrc.ci
        echo "build --disk_cache=/tmp/bazel-disk-cache" >> .bazelrc.ci
        echo "build --repository_cache=/tmp/bazel-repo-cache" >> .bazelrc.ci

    - name: Cache Bazel artifacts
      uses: actions/cache@v3
      with:
        path: |
          /tmp/bazel-disk-cache
          /tmp/bazel-repo-cache
        key: bazel-${{ runner.os }}-${{ hashFiles('WORKSPACE', 'MODULE.bazel', '**/*.bzl', 'requirements*.txt') }}
        restore-keys: |
          bazel-${{ runner.os }}-

    - name: Authenticate to Google Cloud
      uses: google-github-actions/auth@v1
      with:
        credentials_json: ${{ secrets.GCP_SA_KEY }}

    - name: Build all targets
      run: bazel build //... --config=ci

    - name: Run unit tests
      run: bazel test //... --test_tag_filters=-integration --config=ci --test_output=errors

    - name: Run integration tests  
      run: bazel test //... --test_tag_filters=integration --config=ci --test_output=errors

    - name: Run security scans
      run: |
        bazel run //tools/security:vulnerability_scan
        bazel run //tools/security:dependency_check

    - name: Generate coverage report
      run: |
        bazel coverage //... --combined_report=lcov --config=ci
        bash <(curl -s https://codecov.io/bash) -f bazel-out/_coverage/_coverage_report.dat

    - name: Build Docker images
      if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop'
      run: |
        bazel build //deploy:app_prod_image --config=ci --stamp

    - name: Upload build artifacts
      uses: actions/upload-artifact@v3
      with:
        name: bazel-build-artifacts
        path: |
          bazel-bin/
          bazel-testlogs/
        retention-days: 7

  security-scan:
    runs-on: ubuntu-latest
    needs: build-and-test

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Run Trivy vulnerability scanner
      uses: aquasecurity/trivy-action@master
      with:
        scan-type: 'fs'
        scan-ref: '.'
        format: 'sarif'
        output: 'trivy-results.sarif'

    - name: Upload Trivy scan results
      uses: github/codeql-action/upload-sarif@v2
      with:
        sarif_file: 'trivy-results.sarif'

  deploy-staging:
    runs-on: ubuntu-latest
    needs: [build-and-test, security-scan]
    if: github.ref == 'refs/heads/develop'
    environment: staging

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Bazel
      uses: bazel-contrib/setup-bazel@0.8.1
      with:
        bazelisk-version: 1.x

    - name: Authenticate to GCP
      uses: google-github-actions/auth@v1
      with:
        credentials_json: ${{ secrets.GCP_SA_KEY }}

    - name: Build and push Docker image
      run: |
        bazel run //deploy:push_staging --config=ci --stamp

    - name: Deploy to staging
      run: |
        bazel run //k8s:api_stack_staging.apply --config=ci

    - name: Run smoke tests
      run: |
        bazel test //tests/smoke:staging_smoke_tests --config=ci

    - name: Notify deployment
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: "Staging deployment completed: ${{ github.sha }}"
      env:
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

  deploy-production:
    runs-on: ubuntu-latest
    needs: [build-and-test, security-scan]
    if: github.ref == 'refs/heads/main'
    environment: production

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Bazel
      uses: bazel-contrib/setup-bazel@0.8.1
      with:
        bazelisk-version: 1.x

    - name: Authenticate to GCP
      uses: google-github-actions/auth@v1
      with:
        credentials_json: ${{ secrets.GCP_SA_KEY }}

    - name: Build and push Docker image
      run: |
        bazel run //deploy:push_prod --config=ci --stamp
        bazel run //deploy:push_latest --config=ci --stamp

    - name: Deploy to production (blue-green)
      run: |
        # Deploy to green environment first
        bazel run //k8s:api_stack_prod_green.apply --config=ci

        # Run production smoke tests
        bazel test //tests/smoke:production_smoke_tests --config=ci

        # Switch traffic to green
        bazel run //k8s:switch_to_green --config=ci

        # Clean up blue environment
        bazel run //k8s:cleanup_blue --config=ci

    - name: Create GitHub release
      uses: softprops/action-gh-release@v1
      with:
        tag_name: v${{ github.run_number }}
        name: Release v${{ github.run_number }}
        body: |
          ## Changes
          ${{ github.event.head_commit.message }}

          ## Build Info
          - Commit: ${{ github.sha }}
          - Build: ${{ github.run_number }}
          - Environment: Production
        draft: false
        prerelease: false
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

    - name: Notify production deployment
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: "🚀 Production deployment completed: v${{ github.run_number }}"
      env:
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

GitLab CI Integration

Alternative CI/CD setup for GitLab:

# .gitlab-ci.yml
stages:
  - build
  - test
  - security
  - deploy-staging
  - deploy-production

variables:
  BAZEL_VERSION: "7.0.0"
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"

.bazel_cache: &bazel_cache
  cache:
    key: bazel-$CI_COMMIT_REF_SLUG
    paths:
      - /tmp/bazel-cache/
      - /tmp/bazel-repo-cache/
    policy: pull-push

.bazel_setup: &bazel_setup
  before_script:
    - curl -LO "https://github.com/bazelbuild/bazelisk/releases/download/v1.19.0/bazelisk-linux-amd64"
    - chmod +x bazelisk-linux-amd64
    - mv bazelisk-linux-amd64 /usr/local/bin/bazel
    - echo "build --disk_cache=/tmp/bazel-cache" >> .bazelrc.ci
    - echo "build --repository_cache=/tmp/bazel-repo-cache" >> .bazelrc.ci
    - echo "build --remote_cache=$REMOTE_CACHE_URL" >> .bazelrc.ci
    - echo "build --remote_upload_local_results=true" >> .bazelrc.ci

build:
  stage: build
  image: ubuntu:22.04
  <<: *bazel_cache
  <<: *bazel_setup
  script:
    - apt-get update && apt-get install -y python3 python3-pip curl
    - bazel build //... --config=ci
  artifacts:
    paths:
      - bazel-bin/
    expire_in: 1 hour

unit-tests:
  stage: test
  image: ubuntu:22.04
  <<: *bazel_cache
  <<: *bazel_setup
  script:
    - apt-get update && apt-get install -y python3 python3-pip curl
    - bazel test //... --test_tag_filters=-integration --config=ci --test_output=errors
  artifacts:
    reports:
      junit: bazel-testlogs/**/test.xml
    paths:
      - bazel-testlogs/
    expire_in: 1 week

integration-tests:
  stage: test
  image: ubuntu:22.04
  <<: *bazel_cache
  <<: *bazel_setup
  services:
    - postgres:13
    - redis:6
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
  script:
    - apt-get update && apt-get install -y python3 python3-pip curl postgresql-client
    - bazel test //... --test_tag_filters=integration --config=ci --test_output=errors
  artifacts:
    reports:
      junit: bazel-testlogs/**/test.xml

security-scan:
  stage: security
  image: aquasec/trivy:latest
  script:
    - trivy fs --format json --output security-report.json .
  artifacts:
    reports:
      security: security-report.json
  allow_failure: true

deploy-staging:
  stage: deploy-staging
  image: google/cloud-sdk:alpine
  <<: *bazel_setup
  environment:
    name: staging
    url: https://api-staging.company.com
  only:
    - develop
  script:
    - gcloud auth activate-service-account --key-file $GCP_SERVICE_ACCOUNT_KEY
    - bazel run //deploy:push_staging --config=ci --stamp
    - bazel run //k8s:api_stack_staging.apply --config=ci
    - bazel test //tests/smoke:staging_smoke_tests --config=ci

deploy-production:
  stage: deploy-production
  image: google/cloud-sdk:alpine
  <<: *bazel_setup
  environment:
    name: production
    url: https://api.company.com
  only:
    - main
  when: manual
  script:
    - gcloud auth activate-service-account --key-file $GCP_SERVICE_ACCOUNT_KEY
    - bazel run //deploy:push_prod --config=ci --stamp
    - bazel run //k8s:api_stack_prod.apply --config=ci
    - bazel test //tests/smoke:production_smoke_tests --config=ci

6.4 Remote Execution Setup

Google Cloud Build Integration

Set up remote execution using Google Cloud Build:

# cloudbuild.yaml
steps:
  # Build with remote execution
  - name: 'l.gcr.io/google/bazel:latest'
    args:
      - 'build'
      - '//...'
      - '--remote_executor=grpc://remotebuildexecution.googleapis.com:443'
      - '--remote_cache=grpc://remotebuildexecution.googleapis.com:443'
      - '--bes_backend=grpc://buildeventservice.googleapis.com:443'
      - '--bes_results_url=https://source.cloud.google.com/results/invocations/'
      - '--project_id=$PROJECT_ID'
      - '--jobs=50'
    env:
      - 'BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1'

  # Run tests with remote execution
  - name: 'l.gcr.io/google/bazel:latest'
    args:
      - 'test'
      - '//...'
      - '--remote_executor=grpc://remotebuildexecution.googleapis.com:443'
      - '--remote_cache=grpc://remotebuildexecution.googleapis.com:443'
      - '--test_output=errors'
      - '--project_id=$PROJECT_ID'
      - '--jobs=50'
    env:
      - 'BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1'

# Build configuration
options:
  machineType: 'E2_HIGHCPU_32'
  diskSizeGb: 100

Self-Hosted Remote Execution

Set up your own remote execution cluster:

# remote-execution/docker-compose.yml
version: '3.8'

services:
  buildbarn-frontend:
    image: buildbarn/bb-remote-execution:latest
    command:
      - /app/bb_frontend
      - /config/frontend.jsonnet
    ports:
      - "8980:8980"
      - "8981:8981"
    volumes:
      - ./config:/config:ro
    depends_on:
      - buildbarn-storage

  buildbarn-scheduler:
    image: buildbarn/bb-remote-execution:latest
    command:
      - /app/bb_scheduler
      - /config/scheduler.jsonnet
    ports:
      - "8982:8982"
      - "8983:8983"
    volumes:
      - ./config:/config:ro
    depends_on:
      - buildbarn-storage

  buildbarn-worker:
    image: buildbarn/bb-remote-execution:latest
    command:
      - /app/bb_worker
      - /config/worker.jsonnet
    volumes:
      - ./config:/config:ro
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - buildbarn-scheduler
    deploy:
      replicas: 4

  buildbarn-storage:
    image: buildbarn/bb-storage:latest
    command:
      - /app/bb_storage
      - /config/storage.jsonnet
    ports:
      - "8984:8984"
    volumes:
      - ./config:/config:ro
      - bb-storage-data:/data

volumes:
  bb-storage-data:

6.5 Monitoring and Observability

Build Analytics and Monitoring

Implement comprehensive build monitoring:

# //tools/monitoring/BUILD
py_binary(
    name = "build_analytics",
    srcs = ["build_analytics.py"],
    deps = [
        "@pypi//requests",
        "@pypi//prometheus_client", 
        "@pypi//influxdb_client",
    ],
)

py_library(
    name = "metrics_collector",
    srcs = ["metrics_collector.py"],
    deps = [
        "@pypi//psutil",
        "@pypi//prometheus_client",
    ],
)

# //tools/monitoring/build_analytics.py
"""Build analytics and monitoring."""

import json
import time
import requests
from prometheus_client import CollectorRegistry, Gauge, Counter, push_to_gateway

class BuildMetricsCollector:
    def __init__(self, registry=None):
        self.registry = registry or CollectorRegistry()

        # Build metrics
        self.build_duration = Gauge(
            'bazel_build_duration_seconds',
            'Duration of Bazel builds',
            ['target', 'config', 'result'],
            registry=self.registry
        )

        self.cache_hit_rate = Gauge(
            'bazel_cache_hit_rate',
            'Cache hit rate percentage',
            ['cache_type'],
            registry=self.registry
        )

        self.test_results = Counter(
            'bazel_test_results_total',
            'Test results by status',
            ['status', 'target'],
            registry=self.registry
        )

        self.build_size = Gauge(
            'bazel_build_size_bytes',
            'Size of build outputs',
            ['target'],
            registry=self.registry
        )

    def collect_build_metrics(self, build_event_file):
        """Collect metrics from Bazel build event stream."""
        with open(build_event_file, 'r') as f:
            for line in f:
                event = json.loads(line)
                self._process_build_event(event)

    def _process_build_event(self, event):
        """Process individual build event."""
        if 'buildFinished' in event:
            result = event['buildFinished']
            self.build_duration.labels(
                target='//...',
                config='prod',
                result=result.get('exitCode', 'unknown')
            ).set(result.get('finishTimeMillis', 0) / 1000)

        elif 'testResult' in event:
            test = event['testResult']
            self.test_results.labels(
                status=test.get('status', 'unknown'),
                target=test.get('label', 'unknown')
            ).inc()

    def push_metrics(self, gateway_url, job_name):
        """Push metrics to Prometheus gateway."""
        push_to_gateway(gateway_url, job=job_name, registry=self.registry)

# Usage example
if __name__ == "__main__":
    collector = BuildMetricsCollector()
    collector.collect_build_metrics('/tmp/build_events.json')
    collector.push_metrics('http://prometheus-gateway:9091', 'bazel_builds')

Health Monitoring for Deployed Applications

# //src/monitoring/health.py
"""Application health monitoring."""

from fastapi import FastAPI, Response, status
from prometheus_client import Counter, Histogram, generate_latest
import time
import psutil
import logging

app = FastAPI()

# Metrics
request_count = Counter('http_requests_total', 'Total HTTP requests', ['method', 'endpoint', 'status'])
request_duration = Histogram('http_request_duration_seconds', 'HTTP request duration')

@app.middleware("http")
async def metrics_middleware(request, call_next):
    start_time = time.time()
    response = await call_next(request)
    duration = time.time() - start_time

    request_count.labels(
        method=request.method,
        endpoint=request.url.path,
        status=response.status_code
    ).inc()

    request_duration.observe(duration)
    return response

@app.get("/health")
async def health_check():
    """Basic health check endpoint."""
    return {
        "status": "healthy",
        "timestamp": time.time(),
        "version": "1.0.0"
    }

@app.get("/ready")
async def readiness_check():
    """Readiness check for Kubernetes."""
    # Check database connectivity
    try:
        # Add your database health check here
        # db.execute("SELECT 1")
        pass
    except Exception as e:
        logging.error(f"Database health check failed: {e}")
        return Response(status_code=status.HTTP_503_SERVICE_UNAVAILABLE)

    # Check system resources
    memory_usage = psutil.virtual_memory().percent
    if memory_usage > 90:
        return Response(status_code=status.HTTP_503_SERVICE_UNAVAILABLE)

    return {"status": "ready", "memory_usage": memory_usage}

@app.get("/metrics")
async def metrics():
    """Prometheus metrics endpoint."""
    return Response(generate_latest(), media_type="text/plain")

@app.get("/info")
async def build_info():
    """Build and deployment information."""
    from src.app_version import get_build_info
    return get_build_info()

Alerting Configuration

# //monitoring/alerts/bazel-alerts.yml
groups:
- name: bazel_build_alerts
  rules:
  - alert: BuildFailureRate
    expr: rate(bazel_build_failures_total[5m]) > 0.1
    for: 2m
    labels:
      severity: warning
      team: devops
    annotations:
      summary: "High build failure rate detected"
      description: "Build failure rate is {{ $value }} failures per second"

  - alert: CacheHitRateLow
    expr: bazel_cache_hit_rate < 0.5
    for: 5m
    labels:
      severity: warning
      team: devops
    annotations:
      summary: "Bazel cache hit rate is low"
      description: "Cache hit rate is {{ $value }}%, below 50% threshold"

  - alert: BuildDurationHigh
    expr: bazel_build_duration_seconds > 1800
    for: 1m
    labels:
      severity: critical
      team: devops
    annotations:
      summary: "Build duration exceeded 30 minutes"
      description: "Build took {{ $value }} seconds to complete"

- name: application_alerts
  rules:
  - alert: ApplicationDown
    expr: up{job="api-server"} == 0
    for: 1m
    labels:
      severity: critical
      team: sre
    annotations:
      summary: "Application is down"
      description: "API server has been down for more than 1 minute"

  - alert: HighErrorRate
    expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.05
    for: 2m
    labels:
      severity: warning
      team: sre
    annotations:
      summary: "High error rate detected"
      description: "Error rate is {{ $value }} errors per second"

6.6 Security Best Practices

Secure Build Configuration

# .bazelrc.security
# Security-focused build configuration
build:secure --sandbox_default_allow_network=false
build:secure --experimental_strict_action_env=true
build:secure --experimental_guard_against_concurrent_changes=true

# Disable potentially unsafe features
build:secure --noincompatible_strict_action_env
build:secure --experimental_sandbox_default_allow_network=false

# Use hermetic toolchains
build:secure --incompatible_enable_cc_toolchain_resolution
build:secure --action_env=PATH=/usr/bin:/bin

# Enable security features
build:secure --experimental_enable_runfiles_manifest
build:secure --experimental_inprocess_symlink_creation=false

Dependency Security Scanning

# //tools/security/BUILD
py_binary(
    name = "vulnerability_scan",
    srcs = ["vulnerability_scan.py"],
    deps = [
        "@pypi//safety",
        "@pypi//requests",
        "@pypi//packaging",
    ],
)

py_binary(
    name = "dependency_check",
    srcs = ["dependency_check.py"], 
    deps = [
        "@pypi//cyclonedx_bom",
        "@pypi//packageurl_python",
    ],
)

# //tools/security/vulnerability_scan.py
"""Security vulnerability scanning for Python dependencies."""

import json
import subprocess
import sys
from packaging.version import parse
import requests

class VulnerabilityScanner:
    def __init__(self):
        self.safety_db_url = "https://raw.githubusercontent.com/pyupio/safety-db/master/data"

    def scan_requirements(self, requirements_file):
        """Scan requirements file for vulnerabilities."""
        try:
            result = subprocess.run([
                "safety", "check", 
                "--requirements", requirements_file,
                "--output", "json"
            ], capture_output=True, text=True)

            if result.returncode != 0:
                vulnerabilities = json.loads(result.stdout)
                self._report_vulnerabilities(vulnerabilities)
                return False
            return True

        except Exception as e:
            print(f"Error scanning requirements: {e}")
            return False

    def _report_vulnerabilities(self, vulnerabilities):
        """Report found vulnerabilities."""
        print("🚨 SECURITY VULNERABILITIES FOUND:")
        print("=" * 50)

        for vuln in vulnerabilities:
            print(f"Package: {vuln['package_name']}")
            print(f"Installed: {vuln['installed_version']}")
            print(f"Vulnerability: {vuln['vulnerability_id']}")
            print(f"Description: {vuln['advisory']}")
            print(f"More info: {vuln['more_info_url']}")
            print("-" * 30)

    def generate_sbom(self, output_file):
        """Generate Software Bill of Materials."""
        try:
            subprocess.run([
                "cyclonedx-bom", 
                "--requirements", "requirements.txt",
                "--output", output_file
            ], check=True)
            print(f"SBOM generated: {output_file}")
            return True
        except subprocess.CalledProcessError:
            print("Failed to generate SBOM")
            return False

if __name__ == "__main__":
    scanner = VulnerabilityScanner()

    # Scan all requirements files
    requirements_files = [
        "//third_party:requirements.txt",
        "//third_party:requirements-dev.txt", 
        "//third_party:requirements-test.txt"
    ]

    all_secure = True
    for req_file in requirements_files:
        if not scanner.scan_requirements(req_file):
            all_secure = False

    # Generate SBOM
    scanner.generate_sbom("sbom.json")

    if not all_secure:
        sys.exit(1)

Secret Management

# //tools/secrets/BUILD
py_library(
    name = "secret_manager",
    srcs = ["secret_manager.py"],
    deps = [
        "@pypi//google_cloud_secret_manager",
        "@pypi//azure_keyvault_secrets",
        "@pypi//boto3",
    ],
)

# //tools/secrets/secret_manager.py
"""Centralized secret management."""

import os
from abc import ABC, abstractmethod
from google.cloud import secretmanager
import boto3
from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential

class SecretProvider(ABC):
    @abstractmethod
    def get_secret(self, secret_name: str) -> str:
        pass

class GCPSecretProvider(SecretProvider):
    def __init__(self, project_id: str):
        self.client = secretmanager.SecretManagerServiceClient()
        self.project_id = project_id

    def get_secret(self, secret_name: str) -> str:
        name = f"projects/{self.project_id}/secrets/{secret_name}/versions/latest"
        response = self.client.access_secret_version(request={"name": name})
        return response.payload.data.decode("UTF-8")

class AWSSecretProvider(SecretProvider):
    def __init__(self, region: str):
        self.client = boto3.client('secretsmanager', region_name=region)

    def get_secret(self, secret_name: str) -> str:
        response = self.client.get_secret_value(SecretId=secret_name)
        return response['SecretString']

class AzureSecretProvider(SecretProvider):
    def __init__(self, vault_url: str):
        credential = DefaultAzureCredential()
        self.client = SecretClient(vault_url=vault_url, credential=credential)

    def get_secret(self, secret_name: str) -> str:
        secret = self.client.get_secret(secret_name)
        return secret.value

class SecretManager:
    def __init__(self):
        self.provider = self._get_provider()

    def _get_provider(self) -> SecretProvider:
        """Select secret provider based on environment."""
        if os.getenv('GCP_PROJECT'):
            return GCPSecretProvider(os.getenv('GCP_PROJECT'))
        elif os.getenv('AWS_REGION'):
            return AWSSecretProvider(os.getenv('AWS_REGION'))
        elif os.getenv('AZURE_VAULT_URL'):
            return AzureSecretProvider(os.getenv('AZURE_VAULT_URL'))
        else:
            raise ValueError("No secret provider configured")

    def get_secret(self, secret_name: str) -> str:
        return self.provider.get_secret(secret_name)

6.7 Production Troubleshooting

Debugging Production Issues

# //tools/debug/BUILD
py_binary(
    name = "production_debugger",
    srcs = ["production_debugger.py"],
    deps = [
        "@pypi//kubernetes",
        "@pypi//requests",
        "@pypi//click",
    ],
)

# //tools/debug/production_debugger.py
"""Production debugging utilities."""

import click
import subprocess
import json
from kubernetes import client, config

class ProductionDebugger:
    def __init__(self):
        try:
            config.load_incluster_config()
        except:
            config.load_kube_config()
        self.k8s = client.CoreV1Api()

    def check_build_artifacts(self, commit_sha: str):
        """Check if build artifacts exist for commit."""
        print(f"Checking build artifacts for {commit_sha}...")

        # Check if Docker image exists
        image_tag = f"gcr.io/company-project/api-server:{commit_sha}"
        try:
            result = subprocess.run([
                "docker", "manifest", "inspect", image_tag
            ], capture_output=True, text=True)

            if result.returncode == 0:
                print(f"✅ Docker image exists: {image_tag}")
                manifest = json.loads(result.stdout)
                print(f"   Size: {manifest.get('schemaVersion', 'unknown')}")
            else:
                print(f"❌ Docker image not found: {image_tag}")

        except Exception as e:
            print(f"Error checking Docker image: {e}")

    def check_deployment_status(self, namespace: str, deployment: str):
        """Check Kubernetes deployment status."""
        try:
            deployment_obj = client.AppsV1Api().read_namespaced_deployment(
                name=deployment, namespace=namespace
            )

            print(f"Deployment: {deployment}")
            print(f"Replicas: {deployment_obj.status.replicas}")
            print(f"Ready: {deployment_obj.status.ready_replicas}")
            print(f"Available: {deployment_obj.status.available_replicas}")

            # Check pod status
            pods = self.k8s.list_namespaced_pod(
                namespace=namespace,
                label_selector=f"app={deployment}"
            )

            for pod in pods.items:
                print(f"Pod: {pod.metadata.name}")
                print(f"  Status: {pod.status.phase}")
                print(f"  Ready: {sum(1 for c in pod.status.conditions if c.type == 'Ready' and c.status == 'True')}")

        except Exception as e:
            print(f"Error checking deployment: {e}")

    def get_recent_logs(self, namespace: str, deployment: str, lines: int = 100):
        """Get recent application logs."""
        try:
            pods = self.k8s.list_namespaced_pod(
                namespace=namespace,
                label_selector=f"app={deployment}"
            )

            for pod in pods.items:
                print(f"\n=== Logs for {pod.metadata.name} ===")
                logs = self.k8s.read_namespaced_pod_log(
                    name=pod.metadata.name,
                    namespace=namespace,
                    tail_lines=lines
                )
                print(logs)

        except Exception as e:
            print(f"Error getting logs: {e}")

@click.group()
def cli():
    """Production debugging CLI."""
    pass

@cli.command()
@click.argument('commit_sha')
def check_build(commit_sha):
    """Check build artifacts for commit."""
    debugger = ProductionDebugger()
    debugger.check_build_artifacts(commit_sha)

@cli.command()
@click.argument('namespace')
@click.argument('deployment')
def check_deployment(namespace, deployment):
    """Check deployment status."""
    debugger = ProductionDebugger()
    debugger.check_deployment_status(namespace, deployment)

@cli.command()
@click.argument('namespace')
@click.argument('deployment')
@click.option('--lines', default=100, help='Number of log lines to retrieve')
def logs(namespace, deployment, lines):
    """Get application logs."""
    debugger = ProductionDebugger()
    debugger.get_recent_logs(namespace, deployment, lines)

if __name__ == "__main__":
    cli()

Performance Debugging

#!/bin/bash
# //scripts/performance_debug.sh

echo "🔍 Production Performance Debug Report"
echo "======================================"

# Build performance analysis
echo "📊 Recent Build Performance:"
bazel query "//..." --output=build | wc -l
echo "Total targets: $(bazel query '//...' | wc -l)"

# Cache analysis
echo "💾 Cache Performance:"
echo "Local cache size: $(du -sh ~/.cache/bazel 2>/dev/null || echo 'Unknown')"

# Resource usage
echo "🖥️  System Resources:"
echo "CPU usage: $(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1)%"
echo "Memory usage: $(free | grep Mem | awk '{printf "%.1f%%", $3/$2 * 100.0}')"
echo "Disk usage: $(df -h | grep '/dev/' | head -1 | awk '{print $5}')"

# Application metrics
if command -v kubectl &> /dev/null; then
    echo "🚀 Kubernetes Resources:"
    kubectl top nodes 2>/dev/null || echo "kubectl not configured"
    kubectl get pods -A --field-selector=status.phase=Running | wc -l | xargs echo "Running pods:"
fi

echo "✅ Debug report complete"

6.8 Rollback and Recovery Procedures

Automated Rollback System

# //tools/rollback/BUILD
py_binary(
    name = "rollback_manager",
    srcs = ["rollback_manager.py"],
    deps = [
        "@pypi//kubernetes", 
        "@pypi//google_cloud_container",
        "@pypi//click",
    ],
)

# //tools/rollback/rollback_manager.py
"""Automated rollback and recovery system."""

import click
import time
import logging
from kubernetes import client, config
from google.cloud import container_v1

class RollbackManager:
    def __init__(self):
        config.load_kube_config()
        self.k8s_apps = client.AppsV1Api()
        self.k8s_core = client.CoreV1Api()

    def rollback_deployment(self, namespace: str, deployment: str, revision: int = None):
        """Rollback Kubernetes deployment."""
        try:
            if revision:
                # Rollback to specific revision
                body = {'spec': {'rollbackTo': {'revision': revision}}}
            else:
                # Rollback to previous version
                body = {'spec': {'rollbackTo': {}}}

            self.k8s_apps.patch_namespaced_deployment_rollback(
                name=deployment,
                namespace=namespace,
                body=body
            )

            print(f"Initiated rollback for {deployment} in {namespace}")

            # Wait for rollback to complete
            self._wait_for_rollback_completion(namespace, deployment)

        except Exception as e:
            print(f"Rollback failed: {e}")
            raise

    def _wait_for_rollback_completion(self, namespace: str, deployment: str, timeout: int = 300):
        """Wait for rollback to complete."""
        start_time = time.time()

        while time.time() - start_time < timeout:
            deployment_obj = self.k8s_apps.read_namespaced_deployment(
                name=deployment, namespace=namespace
            )

            if (deployment_obj.status.ready_replicas == deployment_obj.spec.replicas and 
                deployment_obj.status.updated_replicas == deployment_obj.spec.replicas):
                print(f"✅ Rollback completed for {deployment}")
                return True

            print(f"⏳ Waiting for rollback... ({int(time.time() - start_time)}s)")
            time.sleep(10)

        raise TimeoutError(f"Rollback timeout after {timeout}s")

    def health_check_after_rollback(self, namespace: str, deployment: str):
        """Perform health checks after rollback."""
        # Get service endpoint
        try:
            service = self.k8s_core.read_namespaced_service(
                name=deployment, namespace=namespace
            )

            # Perform basic health check
            import requests
            health_url = f"http://{service.spec.cluster_ip}/health"

            for attempt in range(5):
                try:
                    response = requests.get(health_url, timeout=10)
                    if response.status_code == 200:
                        print("✅ Health check passed")
                        return True
                except Exception as e:
                    print(f"Health check attempt {attempt + 1} failed: {e}")
                    time.sleep(5)

            print("❌ Health check failed")
            return False

        except Exception as e:
            print(f"Error during health check: {e}")
            return False

@click.group()
def cli():
    """Rollback management CLI."""
    pass

@cli.command()
@click.argument('namespace')
@click.argument('deployment')
@click.option('--revision', type=int, help='Specific revision to rollback to')
@click.option('--confirm', is_flag=True, help='Skip confirmation prompt')
def rollback(namespace, deployment, revision, confirm):
    """Rollback deployment to previous version."""
    if not confirm:
        click.confirm(f'Are you sure you want to rollback {deployment} in {namespace}?', abort=True)

    manager = RollbackManager()
    manager.rollback_deployment(namespace, deployment, revision)
    manager.health_check_after_rollback(namespace, deployment)

@cli.command()
@click.argument('namespace')
@click.argument('deployment')
def history(namespace, deployment):
    """Show deployment rollout history."""
    try:
        config.load_kube_config()
        k8s = client.AppsV1Api()

        # Get replica sets for deployment
        replica_sets = k8s.list_namespaced_replica_set(
            namespace=namespace,
            label_selector=f"app={deployment}"
        )

        print(f"Rollout history for {deployment}:")
        for rs in replica_sets.items:
            revision = rs.metadata.annotations.get('deployment.kubernetes.io/revision', 'unknown')
            created = rs.metadata.creation_timestamp
            print(f"  Revision {revision}: {created}")

    except Exception as e:
        print(f"Error getting history: {e}")

if __name__ == "__main__":
    cli()

6.9 Best Practices Summary

Production Deployment Checklist

# Production Deployment Checklist

## Pre-Deployment
- [ ] All tests passing (unit, integration, e2e)
- [ ] Security scans completed with no critical issues
- [ ] Performance benchmarks within acceptable range
- [ ] Database migrations tested and ready
- [ ] Rollback plan documented and tested
- [ ] Monitoring and alerting configured
- [ ] Load testing completed

## During Deployment  
- [ ] Blue-green deployment strategy used
- [ ] Health checks passing before traffic switch
- [ ] Monitoring dashboards active
- [ ] Team notified of deployment start
- [ ] Gradual traffic ramp-up if applicable

## Post-Deployment
- [ ] All health checks passing
- [ ] Error rates within normal bounds
- [ ] Performance metrics stable
- [ ] User acceptance testing completed
- [ ] Rollback plan ready if needed
- [ ] Documentation updated
- [ ] Team notified of successful deployment

Performance Optimization Guidelines

Build Performance
- Use remote caching and execution
- Optimize target granularity
- Monitor cache hit rates
- Use incremental builds effectively
Application Performance
- Implement proper monitoring
- Use performance profiling
- Optimize Docker images
- Configure resource limits appropriately
Infrastructure Performance
- Use appropriate machine types
- Configure autoscaling
- Monitor resource utilization
- Implement proper load balancing

Module 6 Exercises

Exercise 1: Complete CI/CD Pipeline

Set up a complete CI/CD pipeline for a Python web application including automated testing, security scanning, and deployment to staging and production environments.

Exercise 2: Monitoring Implementation

Implement comprehensive monitoring for your Bazel builds and deployed applications, including custom metrics, alerting rules, and dashboards.

Exercise 3: Disaster Recovery Testing

Create and test a complete disaster recovery procedure including automated rollbacks, data backup verification, and service restoration.

Exercise 4: Security Hardening

Implement security best practices including vulnerability scanning, secret management, and secure build configurations.

Course Completion Project

Final Project: Production-Ready Microservices Platform

Build a complete microservices platform demonstrating all concepts learned:

Architecture: Multiple Python microservices with shared libraries
Build System: Advanced Bazel configuration with custom rules
Dependencies: Complex dependency management with version pinning
Caching: Multi-tier caching strategy with remote execution
CI/CD: Complete pipeline with automated testing and deployment
Monitoring: Comprehensive observability and alerting
Security: Vulnerability scanning and secret management
Production: Blue-green deployments with automated rollback

This project should demonstrate expert-level Bazel usage that would impress any technical reviewer and showcase production-ready skills.

Key Takeaways

Production deployments require comprehensive automation and monitoring
Security must be built into every stage of the pipeline
Observability is crucial for maintaining production systems
Rollback capabilities are as important as deployment capabilities
Performance optimization is an ongoing process requiring measurement
Documentation and runbooks are essential for team success
Testing in production-like environments prevents deployment issues

Congratulations! You've completed the Modern Bazel with Python tutorial.

Modern Bazel with Python-Module 5: Advanced Python Rules and Toolchains

Sushil Baligar — Sat, 26 Jul 2025 06:44:37 +0000

Learning Objectives

By the end of this module, you will:

Master custom Python rules and macros
Configure Python toolchains for different environments
Implement hermetic Python builds
Handle complex Python packaging scenarios
Use aspects for code analysis and transformation

5.1 Custom Python Rules and Macros

Creating Custom Rules

Bazel's extensibility shines when you need custom build logic. Let's create a custom rule for Python code generation:

# //tools/python_rules.bzl
def _python_proto_library_impl(ctx):
    """Implementation of python_proto_library rule."""
    proto_files = ctx.files.srcs
    output_dir = ctx.actions.declare_directory("proto_gen")

    # Generate Python files from proto
    ctx.actions.run(
        inputs = proto_files,
        outputs = [output_dir],
        executable = ctx.executable._protoc,
        arguments = [
            "--python_out=" + output_dir.path,
            "--proto_path=" + ctx.files.srcs[0].dirname,
        ] + [f.path for f in proto_files],
        mnemonic = "ProtocPython",
    )

    return [
        DefaultInfo(files = depset([output_dir])),
        PyInfo(
            transitive_sources = depset([output_dir]),
            imports = depset([output_dir.path]),
        ),
    ]

python_proto_library = rule(
    implementation = _python_proto_library_impl,
    attrs = {
        "srcs": attr.label_list(
            allow_files = [".proto"],
            mandatory = True,
        ),
        "_protoc": attr.label(
            default = "@com_google_protobuf//:protoc",
            executable = True,
            cfg = "exec",
        ),
    },
)

Advanced Macros

Create sophisticated macros that generate multiple targets:

# //tools/python_macros.bzl
def python_microservice(name, srcs, deps = [], **kwargs):
    """Macro for creating a complete Python microservice."""

    # Main library
    native.py_library(
        name = name + "_lib",
        srcs = srcs,
        deps = deps,
        **kwargs
    )

    # Binary
    native.py_binary(
        name = name,
        srcs = [name + "_main.py"],
        deps = [":" + name + "_lib"],
        main = name + "_main.py",
    )

    # Tests
    test_srcs = native.glob([name + "_*test.py"])
    if test_srcs:
        native.py_test(
            name = name + "_test",
            srcs = test_srcs,
            deps = [
                ":" + name + "_lib",
                "@pytest",
            ],
        )

    # Docker image
    native.genrule(
        name = name + "_docker",
        srcs = [":" + name],
        outs = [name + ".tar"],
        cmd = """
        docker build -t {name}:latest - <<EOF
FROM python:3.11-slim
COPY $(location :{name}) /app/
WORKDIR /app
ENTRYPOINT ["python", "{name}"]
EOF
        docker save {name}:latest > $@
        """.format(name = name),
    )

5.2 Python Toolchains

Configuring Multiple Python Versions

Set up toolchains for different Python versions:

# //tools/python_toolchain/BUILD
load("@rules_python//python:defs.bzl", "py_runtime", "py_runtime_pair")

py_runtime(
    name = "python38_runtime",
    interpreter_path = "/usr/bin/python3.8",
    python_version = "PY3",
)

py_runtime(
    name = "python39_runtime", 
    interpreter_path = "/usr/bin/python3.9",
    python_version = "PY3",
)

py_runtime(
    name = "python311_runtime",
    interpreter_path = "/usr/bin/python3.11", 
    python_version = "PY3",
)

py_runtime_pair(
    name = "python38_runtime_pair",
    py3_runtime = ":python38_runtime",
)

py_runtime_pair(
    name = "python39_runtime_pair",
    py3_runtime = ":python39_runtime", 
)

py_runtime_pair(
    name = "python311_runtime_pair",
    py3_runtime = ":python311_runtime",
)

toolchain(
    name = "python38_toolchain",
    toolchain = ":python38_runtime_pair",
    toolchain_type = "@rules_python//python:toolchain_type",
)

toolchain(
    name = "python39_toolchain", 
    toolchain = ":python39_runtime_pair",
    toolchain_type = "@rules_python//python:toolchain_type",
)

toolchain(
    name = "python311_toolchain",
    toolchain = ":python311_runtime_pair", 
    toolchain_type = "@rules_python//python:toolchain_type",
)

Hermetic Python Builds

Create completely hermetic builds with embedded interpreters:

# WORKSPACE
load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python_3_11",
    python_version = "3.11.4",
    # This makes the Python interpreter hermetic
    ignore_root_user_error = True,
)

load("@python_3_11//:defs.bzl", "interpreter")

5.3 Advanced Dependency Management

Custom Repository Rules

Create custom repository rules for complex dependency scenarios:

# //tools/custom_repos.bzl
def _python_wheel_impl(repository_ctx):
    """Download and extract a Python wheel."""
    url = repository_ctx.attr.url
    sha256 = repository_ctx.attr.sha256

    repository_ctx.download_and_extract(
        url = url,
        sha256 = sha256,
        stripPrefix = repository_ctx.attr.strip_prefix,
    )

    # Generate BUILD file
    repository_ctx.file("BUILD", """
load("@rules_python//python:defs.bzl", "py_library")

py_library(
    name = "pkg",
    srcs = glob(["**/*.py"]),
    data = glob(["**/*"], exclude=["**/*.py", "BUILD"]),
    visibility = ["//visibility:public"],
)
""")

python_wheel = repository_rule(
    implementation = _python_wheel_impl,
    attrs = {
        "url": attr.string(mandatory = True),
        "sha256": attr.string(mandatory = True), 
        "strip_prefix": attr.string(default = ""),
    },
)

Version Pinning and Lock Files

Implement sophisticated version management:

# //third_party/python/requirements.bzl
def install_python_deps():
    """Install all Python dependencies with exact versions."""

    # Auto-generated from requirements.lock
    pip_install(
        name = "pypi_deps",
        requirements_lock = "//third_party/python:requirements.lock",
        python_interpreter_target = "@python_3_11_host//:python",
    )

    # Custom wheels
    python_wheel(
        name = "custom_ml_lib",
        url = "https://files.pythonhosted.org/packages/.../custom_ml_lib-1.0.0-py3-none-any.whl",
        sha256 = "abc123...",
    )

5.4 Code Generation and Aspects

Protocol Buffer Integration

Advanced protobuf handling with custom rules:

# //proto/BUILD
load("//tools:python_rules.bzl", "python_proto_library")

proto_library(
    name = "api_proto",
    srcs = ["api.proto"],
)

python_proto_library(
    name = "api_py_proto",
    srcs = [":api_proto"],
)

py_library(
    name = "api_client",
    srcs = ["api_client.py"],
    deps = [":api_py_proto"],
)

Using Aspects for Analysis

Create aspects for code analysis and transformation:

# //tools/analysis.bzl
def _python_coverage_aspect_impl(target, ctx):
    """Collect Python source files for coverage analysis."""
    if PyInfo not in target:
        return []

    py_info = target[PyInfo]
    source_files = py_info.transitive_sources.to_list()

    coverage_file = ctx.actions.declare_file(target.label.name + ".coverage")

    ctx.actions.write(
        output = coverage_file,
        content = "\n".join([f.path for f in source_files if f.path.endswith(".py")]),
    )

    return [OutputGroupInfo(coverage_files = depset([coverage_file]))]

python_coverage_aspect = aspect(
    implementation = _python_coverage_aspect_impl,
    attr_aspects = ["deps"],
)

5.5 Performance Optimization

Build Performance Tuning

Optimize build performance with advanced configurations:

# .bazelrc.performance
# Enable persistent workers for Python tools
build --worker_sandboxing=false
build --experimental_worker_multiplex

# Optimize Python rule execution
build --experimental_python_import_all_repositories

# Use faster Python stub generation
build --experimental_python_stub_imports

# Memory optimization
build --experimental_worker_memory_limit_mb=2048

# Remote execution optimization
build:remote --experimental_remote_merkle_tree_cache
build:remote --experimental_remote_cache_compression

Incremental Build Optimization

Configure rules for optimal incremental builds:

# //tools/optimized_rules.bzl
def optimized_py_library(name, srcs, deps = [], **kwargs):
    """Optimized Python library with better incremental builds."""

    # Separate interface and implementation
    native.py_library(
        name = name + "_interface",
        srcs = [s for s in srcs if s.endswith("_interface.py")],
        **kwargs
    )

    native.py_library(
        name = name,
        srcs = [s for s in srcs if not s.endswith("_interface.py")],
        deps = deps + [":" + name + "_interface"],
        **kwargs
    )

5.6 Practical Exercise: ML Pipeline

Let's build a complete machine learning pipeline demonstrating advanced concepts:

# //ml_pipeline/BUILD
load("//tools:python_macros.bzl", "python_microservice")
load("//tools:python_rules.bzl", "python_proto_library")

# Data schema
proto_library(
    name = "schema_proto",
    srcs = ["schema.proto"],
)

python_proto_library(
    name = "schema_py_proto", 
    srcs = [":schema_proto"],
)

# Data processing service
python_microservice(
    name = "data_processor",
    srcs = [
        "data_processor.py",
        "data_utils.py",
    ],
    deps = [
        ":schema_py_proto",
        "@pypi_deps//pandas",
        "@pypi_deps//numpy",
    ],
)

# Model training service  
python_microservice(
    name = "model_trainer",
    srcs = [
        "model_trainer.py",
        "model_utils.py",
    ],
    deps = [
        ":schema_py_proto",
        "@pypi_deps//scikit_learn", 
        "@pypi_deps//joblib",
    ],
)

# Model serving service
python_microservice(
    name = "model_server",
    srcs = [
        "model_server.py",
        "serving_utils.py", 
    ],
    deps = [
        ":schema_py_proto",
        "@pypi_deps//flask",
        "@pypi_deps//joblib",
    ],
)

# Integration test
py_test(
    name = "pipeline_integration_test",
    srcs = ["pipeline_integration_test.py"],
    deps = [
        ":data_processor_lib",
        ":model_trainer_lib", 
        ":model_server_lib",
        "@pypi_deps//pytest",
        "@pypi_deps//requests",
    ],
    data = [
        "test_data.csv",
        "expected_model.pkl",
    ],
)

5.7 Best Practices Summary

Rule Design Principles

Keep rules focused and composable
Use proper input/output declarations
Implement hermetic execution
Provide clear error messages
Document rule attributes and behavior

Toolchain Management

Use hermetic toolchains when possible
Version pin all dependencies
Test with multiple Python versions
Implement proper toolchain selection

Performance Considerations

Minimize rule overhead
Use aspects judiciously
Optimize for incremental builds
Profile build performance regularly

Module 5 Exercises

Exercise 1: Custom Rule

Create a custom rule that generates Python dataclasses from JSON schema files.

Exercise 2: Multi-Version Testing

Set up a test matrix that runs your tests against Python 3.8, 3.9, and 3.11.

Exercise 3: Aspect Implementation

Write an aspect that collects all Python import statements across your build graph.

Exercise 4: Performance Analysis

Profile your build and identify the top 3 bottlenecks, then implement optimizations.

Next Steps

In Module 6, we'll cover "Production Deployment and CI/CD Integration" where you'll learn to:

Set up remote caching and execution
Integrate with CI/CD systems
Implement automated testing pipelines
Deploy applications using Bazel

Key Takeaways

Custom rules and macros provide powerful extensibility
Toolchains enable hermetic, reproducible builds
Aspects offer cross-cutting analysis capabilities
Performance optimization requires systematic profiling
Advanced dependency management prevents version conflicts

https://www.linkedin.com/in/sushilbaligar/
https://github.com/sushilbaligar
https://dev.to/sushilbaligar
https://medium.com/@sushilbaligar

Modern Bazel with Python-Module 4: Caching and Dependencies

Sushil Baligar — Sat, 26 Jul 2025 06:40:53 +0000

Learning Objectives

By the end of this module, you will:

Master Bazel's caching mechanisms (local and remote)
Implement efficient dependency management for Python projects
Set up remote caching for team collaboration
Optimize build performance through smart caching strategies
Handle complex dependency scenarios and version conflicts

4.1 Understanding Bazel Caching

Local Caching Fundamentals

Bazel automatically caches build outputs locally. Understanding how this works is crucial for optimization:

# Check cache status
bazel info output_base
bazel info repository_cache

# Clean specific caches
bazel clean --expunge  # Nuclear option - removes everything
bazel clean            # Removes output files but keeps cache

Cache Key Components

Bazel creates cache keys based on:

Input file contents (not timestamps)
Build command and flags
Toolchain configuration
Environment variables that affect the build

# //BUILD - This will be cached efficiently
py_library(
    name = "utils",
    srcs = ["utils.py"],
    deps = ["@pypi//requests"],
)

# Changes to utils.py content will invalidate cache
# Changes to modification time won't affect cache

4.2 Remote Caching Setup

Basic Remote Cache Configuration

Set up a simple HTTP remote cache:

# .bazelrc
build --remote_cache=https://storage.googleapis.com/my-bazel-cache
build --remote_upload_local_results=true
build --remote_timeout=60
build --remote_retries=3

# For authentication
build --google_default_credentials=true
# OR
build --remote_header=Authorization=Bearer=your-token

Advanced Remote Cache Setup

Configure a more sophisticated remote cache with build event publishing:

# .bazelrc.remote
# Remote cache configuration
build:remote --remote_cache=grpc://cache.example.com:443
build:remote --remote_timeout=600
build:remote --remote_retries=5
build:remote --remote_upload_local_results=true

# Build event publishing
build:remote --bes_backend=grpc://analytics.example.com:443
build:remote --bes_results_url=https://analytics.example.com/build/

# Remote execution (if available)
build:remote --remote_executor=grpc://executor.example.com:443
build:remote --jobs=50

# Platform configuration
build:remote --host_platform=@bazel_tools//platforms:linux_x86_64
build:remote --platforms=@bazel_tools//platforms:linux_x86_64

# Use remote config
build --config=remote

Docker-based Remote Cache

Set up a local remote cache using Docker for team development:

# cache-server.dockerfile
FROM nginx:alpine

COPY nginx.conf /etc/nginx/nginx.conf
COPY cache.conf /etc/nginx/conf.d/default.conf

EXPOSE 8080

# cache.conf
server {
    listen 8080;
    client_max_body_size 2G;

    location / {
        root /var/cache/bazel;
        dav_methods PUT DELETE;
        create_full_put_path on;
        dav_access user:rw group:rw all:r;
    }
}

# docker-compose.yml for team cache
version: '3.8'
services:
  bazel-cache:
    build:
      context: .
      dockerfile: cache-server.dockerfile
    ports:
      - "8080:8080"
    volumes:
      - bazel-cache-data:/var/cache/bazel
    restart: unless-stopped

volumes:
  bazel-cache-data:

4.3 Python Dependency Management

Advanced pip_parse Configuration

Set up sophisticated Python dependency management:

# WORKSPACE
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
load("@rules_python//python:pip.bzl", "pip_parse")

python_register_toolchains(
    name = "python3_11",
    python_version = "3.11.4",
)

# Main dependencies
pip_parse(
    name = "pypi",
    requirements_lock = "//third_party:requirements.lock",
    python_interpreter_target = "@python3_11_host//:python",
)

load("@pypi//:requirements.bzl", "install_deps")
install_deps()

# Development dependencies (separate namespace)
pip_parse(
    name = "pypi_dev", 
    requirements_lock = "//third_party:requirements-dev.lock",
    python_interpreter_target = "@python3_11_host//:python",
)

load("@pypi_dev//:requirements.bzl", dev_install_deps = "install_deps")
dev_install_deps()

# Testing dependencies
pip_parse(
    name = "pypi_test",
    requirements_lock = "//third_party:requirements-test.lock", 
    python_interpreter_target = "@python3_11_host//:python",
)

load("@pypi_test//:requirements.bzl", test_install_deps = "install_deps")
test_install_deps()

Dependency Lock Files

Create comprehensive lock files for reproducible builds:

# //third_party/requirements.txt
# Production dependencies
fastapi>=0.104.0,<0.105.0
uvicorn[standard]>=0.24.0,<0.25.0
pydantic>=2.5.0,<3.0.0
sqlalchemy>=2.0.0,<2.1.0
alembic>=1.13.0,<1.14.0
redis>=5.0.0,<6.0.0
celery>=5.3.0,<5.4.0

# //third_party/requirements-dev.lock
# Auto-generated - DO NOT EDIT
# This file was generated by pip-compile with python 3.11
# To update, run: pip-compile requirements-dev.txt

fastapi==0.104.1
    # via -r requirements-dev.txt
starlette==0.27.0
    # via fastapi
pydantic==2.5.2
    # via fastapi
pydantic-core==2.14.5
    # via pydantic
typing-extensions==4.8.0
    # via pydantic
uvicorn==0.24.0
    # via -r requirements-dev.txt
# ... complete locked versions

Multi-Platform Dependencies

Handle platform-specific dependencies:

# //third_party/BUILD
load("@rules_python//python:defs.bzl", "py_library")

# Platform-specific dependencies
py_library(
    name = "platform_deps",
    deps = select({
        "@platforms//os:linux": [
            "@pypi//psutil",
            "@pypi//linux_specific_lib",
        ],
        "@platforms//os:macos": [
            "@pypi//psutil", 
            "@pypi//pyobjc_framework_cocoa",
        ],
        "@platforms//os:windows": [
            "@pypi//psutil",
            "@pypi//pywin32",
        ],
        "//conditions:default": ["@pypi//psutil"],
    }),
)

4.4 Advanced Caching Strategies

Repository Caching

Configure repository-level caching for external dependencies:

# .bazelrc
build --repository_cache=/home/user/.cache/bazel/repos
build --experimental_repository_cache_hardlinks=true

# Force repository re-fetch when needed
build --repository_cache_hits_threshold=10

Action Caching vs Output Caching

Understand the difference and optimize accordingly:

# //tools/cache_optimization.bzl
def cache_friendly_genrule(name, srcs, cmd, **kwargs):
    """Genrule optimized for caching."""
    native.genrule(
        name = name,
        srcs = srcs,
        cmd = cmd,
        # Ensure deterministic output
        stamp = 0,
        # Add cache-friendly attributes
        **kwargs
    )

Cache Warming Strategies

Implement cache warming for CI/CD:

#!/bin/bash
# scripts/warm_cache.sh

# Warm cache with common targets
bazel build //... --keep_going
bazel test //... --test_tag_filters=-slow --keep_going

# Pre-build common development targets
bazel build //src/main:app //src/tests:unit_tests

# Cache commonly used external dependencies
bazel build @pypi//requests @pypi//fastapi @pypi//pytest

echo "Cache warming complete"

4.5 Dependency Resolution and Conflicts

Version Conflict Resolution

Handle complex version conflicts systematically:

# //third_party/overrides.bzl
def apply_dependency_overrides():
    """Apply necessary dependency version overrides."""

    # Override conflicting versions
    override_targets = {
        # Force specific numpy version across all deps
        "@pypi//numpy": "@pypi_pinned//numpy",
        # Use our patched version of requests
        "@pypi//requests": "//third_party/patched:requests",
    }

    return override_targets

# //third_party/patched/BUILD
py_library(
    name = "requests",
    srcs = ["requests_patched.py"],
    deps = [
        "@pypi//urllib3", 
        "@pypi//certifi",
        "@pypi//charset_normalizer",
    ],
    visibility = ["//visibility:public"],
)

Custom Dependency Resolution

Implement custom resolution for complex scenarios:

# //tools/custom_deps.bzl
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

def custom_python_deps():
    """Install Python dependencies with custom resolution."""

    # Custom ML library not available on PyPI
    http_archive(
        name = "custom_ml_lib",
        urls = ["https://github.com/company/ml-lib/archive/v2.1.0.tar.gz"],
        sha256 = "abcd1234...",
        strip_prefix = "ml-lib-2.1.0",
        build_file = "//third_party:custom_ml_lib.BUILD",
    )

    # Forked dependency with patches
    http_archive(
        name = "patched_fastapi",
        urls = ["https://github.com/our-org/fastapi/archive/patched-v0.104.1.tar.gz"],
        sha256 = "efgh5678...",
        strip_prefix = "fastapi-patched-v0.104.1", 
        build_file = "//third_party:patched_fastapi.BUILD",
    )

4.6 Performance Monitoring and Optimization

Build Performance Analysis

Monitor and analyze build performance:

# Generate build profile
bazel build //... --profile=build_profile.json
bazel analyze-profile build_profile.json

# Memory usage analysis  
bazel build //... --memory_profile=memory_profile.json

# Detailed timing
bazel build //... --experimental_profile_additional_tasks

Cache Hit Rate Monitoring

Track cache effectiveness:

#!/bin/bash
# scripts/monitor_cache.sh

echo "Cache Statistics:"
echo "=================="

# Local cache info
echo "Local cache location: $(bazel info output_base)"
echo "Repository cache: $(bazel info repository_cache)"

# Build with cache stats
bazel build //... --profile=cache_profile.json 2>&1 | tee build.log

# Extract cache hit statistics
grep -E "(cache hit|remote cache)" build.log | wc -l
echo "Remote cache hits: $(grep -c 'remote cache hit' build.log)"
echo "Local cache hits: $(grep -c 'local cache hit' build.log)"

Optimizing for Cache Efficiency

Best practices for cache-friendly builds:

# //BUILD - Cache-optimized targets
py_library(
    name = "stable_utils",
    srcs = ["utils.py"],
    # Stable dependencies cache better
    deps = [
        "@pypi//requests",  # Pinned version
        "//common:constants",  # Rarely changing
    ],
)

py_library(
    name = "feature_code", 
    srcs = ["feature.py"],
    # Separate frequently changing code
    deps = [
        ":stable_utils",  # Reuse cached stable parts
        "//config:dynamic_config",  # Accept this changes often
    ],
)

4.7 Practical Examples

Complete Web Application Setup

Real-world example with optimized caching and dependencies:

# //web_app/BUILD
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")

# Core application library (stable, caches well)
py_library(
    name = "app_core",
    srcs = [
        "core/__init__.py",
        "core/models.py", 
        "core/database.py",
        "core/auth.py",
    ],
    deps = [
        "@pypi//fastapi",
        "@pypi//sqlalchemy", 
        "@pypi//pydantic",
        "@pypi//passlib",
        "//common:config",
    ],
)

# API routes (changes more frequently)
py_library(
    name = "api_routes",
    srcs = [
        "api/__init__.py",
        "api/users.py",
        "api/posts.py", 
        "api/auth.py",
    ],
    deps = [
        ":app_core",
        "@pypi//fastapi",
        "//common:validators",
    ],
)

# Main application
py_binary(
    name = "web_app",
    srcs = ["main.py"],
    deps = [
        ":app_core",
        ":api_routes", 
        "@pypi//uvicorn",
    ],
    main = "main.py",
)

# Comprehensive test suite
py_test(
    name = "integration_test",
    srcs = ["test_integration.py"],
    deps = [
        ":web_app",
        "@pypi_test//pytest",
        "@pypi_test//httpx",
        "@pypi_test//pytest_asyncio",
    ],
    data = ["test_data.json"],
)

CI/CD Cache Configuration

Optimize CI/CD with proper cache configuration:

# .github/workflows/build.yml
name: Build and Test

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4

    - name: Mount bazel cache
      uses: actions/cache@v3
      with:
        path: |
          ~/.cache/bazel
          ~/.cache/bazel-repo
        key: bazel-${{ runner.os }}-${{ hashFiles('WORKSPACE', '**/*.bzl', 'requirements*.txt') }}
        restore-keys: |
          bazel-${{ runner.os }}-

    - name: Configure Bazel
      run: |
        echo "build --repository_cache=/home/runner/.cache/bazel-repo" >> .bazelrc.ci
        echo "build --disk_cache=/home/runner/.cache/bazel" >> .bazelrc.ci
        echo "build --remote_cache=${{ secrets.REMOTE_CACHE_URL }}" >> .bazelrc.ci
        echo "build --remote_upload_local_results=true" >> .bazelrc.ci

    - name: Build
      run: bazel build //... --config=ci

    - name: Test  
      run: bazel test //... --config=ci --test_output=errors

4.8 Troubleshooting Common Issues

Cache Invalidation Problems

Debug cache invalidation issues:

# Check why target was rebuilt
bazel build //target:name --explain=explain.log
bazel build //target:name --verbose_explanations

# Compare action keys
bazel aquery //target:name --output=textproto > action1.txt
# Make change and run again
bazel aquery //target:name --output=textproto > action2.txt
diff action1.txt action2.txt

Dependency Resolution Issues

Debug complex dependency problems:

# Analyze dependency graph
bazel query "deps(//your:target)" --output=graph | dot -Tpng > deps.png

# Find conflicting versions
bazel query "//... intersect deps(@pypi//problematic_package//:*)"

# Check why a dependency was selected
bazel query --output=build //external:pypi_problematic_package

Remote Cache Issues

Troubleshoot remote cache problems:

# Test remote cache connectivity
bazel build //simple:target --remote_cache=your-cache-url --execution_log_json_file=exec.json

# Check cache upload/download
bazel build //target --remote_cache=your-cache-url --experimental_remote_cache_eviction_retries=3 -v

# Verify authentication
curl -H "Authorization: Bearer $TOKEN" https://your-cache-url/status

4.9 Best Practices Summary

Caching Best Practices

Use remote caching for team collaboration
Separate stable and volatile dependencies
Monitor cache hit rates regularly
Implement cache warming in CI/CD
Use repository caching for external dependencies

Dependency Management Best Practices

Pin all dependency versions in lock files
Separate production, development, and test dependencies
Handle platform-specific dependencies explicitly
Use custom resolution for complex scenarios
Monitor for security vulnerabilities in dependencies

Performance Optimization

Profile builds regularly to identify bottlenecks
Structure targets to maximize cache reuse
Use hermetic builds for reproducibility
Implement incremental build strategies
Monitor and optimize resource usage

Module 4 Exercises

Exercise 1: Remote Cache Setup

Set up a remote cache using Google Cloud Storage or AWS S3 and measure the cache hit rate improvement.

Exercise 2: Dependency Conflict Resolution

Create a scenario with conflicting dependency versions and resolve it using custom overrides.

Exercise 3: Cache Performance Analysis

Profile a complex build, identify cache inefficiencies, and implement optimizations.

Exercise 4: CI/CD Cache Integration

Set up cache optimization in a CI/CD pipeline and measure build time improvements.

Next Steps

In Module 5, we'll cover "Advanced Python Rules and Toolchains" where you'll learn to:

Create custom Python rules and macros
Configure multiple Python toolchains
Implement hermetic Python builds
Use aspects for code analysis

Key Takeaways

Bazel's caching is content-based, not timestamp-based
Remote caching enables massive build speedups for teams
Proper dependency management prevents version conflicts
Cache performance should be monitored and optimized
Structured targets maximize cache reuse efficiency
Lock files ensure reproducible builds across environments

https://www.linkedin.com/in/sushilbaligar/
https://github.com/sushilbaligar
https://dev.to/sushilbaligar
https://medium.com/@sushilbaligar

Modern Bazel with Python-Module 3: Testing Framework - Quality Assurance

Sushil Baligar — Sat, 26 Jul 2025 05:39:41 +0000

Learning Objectives

By the end of this module, you will:

Master Bazel's testing framework and test execution
Implement comprehensive unit, integration, and end-to-end testing
Configure test sharding, parallelization, and coverage reporting
Set up continuous integration with quality gates
Build a robust testing pipeline for production-ready applications

3.1 Bazel Testing Fundamentals

Understanding Bazel Test Rules

Bazel provides several test rules for Python:

py_test: Basic Python test execution
py_pytest: Advanced testing with pytest integration
py_test_suite: Grouping multiple tests

Basic Test Structure

# BUILD file
load("@rules_python//python:defs.bzl", "py_test", "py_library")

py_library(
    name = "calculator",
    srcs = ["calculator.py"],
    visibility = ["//visibility:public"],
)

py_test(
    name = "calculator_test",
    srcs = ["calculator_test.py"],
    deps = [":calculator"],
    python_version = "PY3",
)

# calculator.py
class Calculator:
    def add(self, a: int, b: int) -> int:
        return a + b

    def subtract(self, a: int, b: int) -> int:
        return a - b

    def multiply(self, a: int, b: int) -> int:
        return a * b

    def divide(self, a: int, b: int) -> float:
        if b == 0:
            raise ValueError("Cannot divide by zero")
        return a / b

# calculator_test.py
import unittest
from calculator import Calculator

class TestCalculator(unittest.TestCase):
    def setUp(self):
        self.calc = Calculator()

    def test_add(self):
        self.assertEqual(self.calc.add(2, 3), 5)
        self.assertEqual(self.calc.add(-1, 1), 0)

    def test_subtract(self):
        self.assertEqual(self.calc.subtract(5, 3), 2)
        self.assertEqual(self.calc.subtract(1, 1), 0)

    def test_multiply(self):
        self.assertEqual(self.calc.multiply(3, 4), 12)
        self.assertEqual(self.calc.multiply(0, 5), 0)

    def test_divide(self):
        self.assertEqual(self.calc.divide(10, 2), 5.0)
        with self.assertRaises(ValueError):
            self.calc.divide(5, 0)

if __name__ == '__main__':
    unittest.main()

3.2 Advanced Testing with pytest

Setting up pytest with Bazel

# WORKSPACE file additions
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# pytest integration
http_archive(
    name = "rules_python_pytest",
    sha256 = "...",
    urls = ["..."],
)

load("@rules_python_pytest//:repositories.bzl", "rules_python_pytest_dependencies")
rules_python_pytest_dependencies()

pytest Configuration

# BUILD file
load("@rules_python//python:defs.bzl", "py_test")
load("@rules_python_pytest//python:pytest.bzl", "py_pytest_test")

py_pytest_test(
    name = "advanced_calculator_test",
    srcs = ["advanced_calculator_test.py"],
    deps = [
        ":calculator",
        "@pip//pytest",
        "@pip//pytest-cov",
        "@pip//pytest-mock",
    ],
    args = [
        "--verbose",
        "--tb=short",
        "--cov=calculator",
        "--cov-report=term-missing",
    ],
)

# advanced_calculator_test.py
import pytest
from unittest.mock import Mock, patch
from calculator import Calculator

class TestAdvancedCalculator:
    @pytest.fixture
    def calculator(self):
        return Calculator()

    @pytest.mark.parametrize("a,b,expected", [
        (2, 3, 5),
        (-1, 1, 0),
        (0, 0, 0),
        (100, -50, 50),
    ])
    def test_add_parametrized(self, calculator, a, b, expected):
        assert calculator.add(a, b) == expected

    @pytest.mark.parametrize("a,b,expected", [
        (10, 2, 5.0),
        (9, 3, 3.0),
        (1, 2, 0.5),
    ])
    def test_divide_parametrized(self, calculator, a, b, expected):
        assert calculator.divide(a, b) == expected

    def test_divide_by_zero_exception(self, calculator):
        with pytest.raises(ValueError, match="Cannot divide by zero"):
            calculator.divide(5, 0)

    @patch('calculator.Calculator.add')
    def test_add_mocked(self, mock_add, calculator):
        mock_add.return_value = 42
        result = calculator.add(1, 2)
        mock_add.assert_called_once_with(1, 2)
        assert result == 42

3.3 Test Organization and Structure

Test Suites and Categories

# BUILD file
load("@rules_python//python:defs.bzl", "py_test")

# Unit tests
py_test(
    name = "unit_tests",
    srcs = glob(["*_unit_test.py"]),
    deps = [
        "//src:core_library",
        "@pip//pytest",
    ],
    size = "small",
    tags = ["unit"],
)

# Integration tests
py_test(
    name = "integration_tests",
    srcs = glob(["*_integration_test.py"]),
    deps = [
        "//src:core_library",
        "//src:database",
        "@pip//pytest",
        "@pip//testcontainers",
    ],
    size = "medium",
    tags = ["integration"],
    data = ["//testdata:sample_data"],
)

# End-to-end tests
py_test(
    name = "e2e_tests",
    srcs = glob(["*_e2e_test.py"]),
    deps = [
        "//src:application",
        "@pip//selenium",
        "@pip//pytest",
    ],
    size = "large",
    tags = ["e2e"],
    timeout = "long",
)

# Test suite combining all tests
test_suite(
    name = "all_tests",
    tests = [
        ":unit_tests",
        ":integration_tests",
        ":e2e_tests",
    ],
)

Test Data Management

# BUILD file for test data
filegroup(
    name = "test_fixtures",
    srcs = glob(["fixtures/**/*.json"]),
    visibility = ["//tests:__subpackages__"],
)

filegroup(
    name = "sample_database",
    srcs = ["sample.db"],
    visibility = ["//tests:__subpackages__"],
)

3.4 Test Coverage and Quality Metrics

Coverage Configuration

# .coveragerc
[run]
source = src/
omit = 
    */tests/*
    */test_*
    setup.py
    */__pycache__/*

[report]
exclude_lines =
    pragma: no cover
    def __repr__
    raise AssertionError
    raise NotImplementedError

[html]
directory = coverage_html_report

Bazel Coverage Integration

# BUILD file
py_test(
    name = "coverage_test",
    srcs = ["test_coverage.py"],
    deps = [
        "//src:main_library",
        "@pip//coverage",
        "@pip//pytest",
        "@pip//pytest-cov",
    ],
    args = [
        "--cov=src",
        "--cov-report=xml:coverage.xml",
        "--cov-report=html:htmlcov",
        "--cov-fail-under=80",
    ],
)

3.5 Test Sharding and Parallelization

Sharding Configuration

# BUILD file
py_test(
    name = "large_test_suite",
    srcs = ["large_test_suite.py"],
    deps = [":test_dependencies"],
    shard_count = 4,  # Split test into 4 shards
    size = "large",
)

# Custom sharding strategy
py_test(
    name = "custom_sharded_test",
    srcs = ["custom_test.py"],
    deps = [":dependencies"],
    env = {
        "TEST_TOTAL_SHARDS": "$(TEST_TOTAL_SHARDS)",
        "TEST_SHARD_INDEX": "$(TEST_SHARD_INDEX)",
    },
)

Parallel Test Execution

# Run tests in parallel
bazel test //tests/... --jobs=8 --test_output=errors

# Run with specific test strategy
bazel test //tests/... --test_strategy=standalone --jobs=auto

3.6 Integration Testing Patterns

Database Integration Tests

# database_integration_test.py
import pytest
import testcontainers.postgres
from sqlalchemy import create_engine
from src.database import DatabaseManager

class TestDatabaseIntegration:
    @pytest.fixture(scope="class")
    def postgres_container(self):
        with testcontainers.postgres.PostgresContainer() as postgres:
            yield postgres

    @pytest.fixture
    def db_manager(self, postgres_container):
        connection_url = postgres_container.get_connection_url()
        engine = create_engine(connection_url)
        return DatabaseManager(engine)

    def test_user_creation(self, db_manager):
        user_id = db_manager.create_user("test@example.com", "Test User")
        assert user_id is not None

        user = db_manager.get_user(user_id)
        assert user.email == "test@example.com"
        assert user.name == "Test User"

    def test_user_deletion(self, db_manager):
        user_id = db_manager.create_user("delete@example.com", "Delete User")
        assert db_manager.delete_user(user_id) is True
        assert db_manager.get_user(user_id) is None

API Integration Tests

# api_integration_test.py
import pytest
import requests
from src.api import create_app

class TestAPIIntegration:
    @pytest.fixture(scope="class")
    def app(self):
        app = create_app(testing=True)
        with app.test_client() as client:
            yield client

    def test_health_endpoint(self, app):
        response = app.get('/health')
        assert response.status_code == 200
        assert response.json['status'] == 'healthy'

    def test_user_crud_operations(self, app):
        # Create user
        user_data = {'name': 'Test User', 'email': 'test@example.com'}
        response = app.post('/api/users', json=user_data)
        assert response.status_code == 201
        user_id = response.json['id']

        # Get user
        response = app.get(f'/api/users/{user_id}')
        assert response.status_code == 200
        assert response.json['name'] == 'Test User'

        # Update user
        updated_data = {'name': 'Updated User'}
        response = app.put(f'/api/users/{user_id}', json=updated_data)
        assert response.status_code == 200

        # Delete user
        response = app.delete(f'/api/users/{user_id}')
        assert response.status_code == 204

3.7 End-to-End Testing

Selenium E2E Tests

# e2e_web_test.py
import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class TestWebApplicationE2E:
    @pytest.fixture(scope="class")
    def driver(self):
        options = webdriver.ChromeOptions()
        options.add_argument("--headless")
        options.add_argument("--no-sandbox")
        options.add_argument("--disable-dev-shm-usage")

        driver = webdriver.Chrome(options=options)
        driver.implicitly_wait(10)
        yield driver
        driver.quit()

    def test_user_registration_flow(self, driver):
        driver.get("http://localhost:8080/register")

        # Fill registration form
        driver.find_element(By.ID, "name").send_keys("Test User")
        driver.find_element(By.ID, "email").send_keys("test@example.com")
        driver.find_element(By.ID, "password").send_keys("password123")

        # Submit form
        driver.find_element(By.ID, "submit").click()

        # Wait for success message
        wait = WebDriverWait(driver, 10)
        success_message = wait.until(
            EC.presence_of_element_located((By.CLASS_NAME, "success"))
        )
        assert "Registration successful" in success_message.text

    def test_login_flow(self, driver):
        driver.get("http://localhost:8080/login")

        driver.find_element(By.ID, "email").send_keys("test@example.com")
        driver.find_element(By.ID, "password").send_keys("password123")
        driver.find_element(By.ID, "login").click()

        # Verify redirect to dashboard
        wait = WebDriverWait(driver, 10)
        wait.until(EC.url_contains("/dashboard"))
        assert "/dashboard" in driver.current_url

3.8 Test Configuration and Environment Management

Test Environment Configuration

# test_config.py
import os
from dataclasses import dataclass

@dataclass
class TestConfig:
    database_url: str = "sqlite:///:memory:"
    redis_url: str = "redis://localhost:6380"
    api_base_url: str = "http://localhost:8080"
    test_data_dir: str = "testdata"

    @classmethod
    def from_env(cls):
        return cls(
            database_url=os.getenv("TEST_DATABASE_URL", cls.database_url),
            redis_url=os.getenv("TEST_REDIS_URL", cls.redis_url),
            api_base_url=os.getenv("TEST_API_BASE_URL", cls.api_base_url),
            test_data_dir=os.getenv("TEST_DATA_DIR", cls.test_data_dir),
        )

pytest Configuration

# conftest.py
import pytest
import tempfile
import shutil
from test_config import TestConfig

@pytest.fixture(scope="session")
def test_config():
    return TestConfig.from_env()

@pytest.fixture
def temp_dir():
    temp_dir = tempfile.mkdtemp()
    yield temp_dir
    shutil.rmtree(temp_dir)

@pytest.fixture(autouse=True)
def setup_test_environment(test_config):
    # Setup code before each test
    yield
    # Teardown code after each test

3.9 Continuous Integration Integration

GitHub Actions with Bazel

# .github/workflows/test.yml
name: Test Suite

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.8, 3.9, 3.10, 3.11]

    steps:
    - uses: actions/checkout@v3

    - name: Setup Bazel
      uses: bazelbuild/setup-bazelisk@v2

    - name: Cache Bazel
      uses: actions/cache@v3
      with:
        path: ~/.cache/bazel
        key: ${{ runner.os }}-bazel-${{ hashFiles('**/BUILD', '**/WORKSPACE') }}
        restore-keys: |
          ${{ runner.os }}-bazel-

    - name: Run unit tests
      run: |
        bazel test //tests:unit_tests --test_output=errors --test_summary=detailed

    - name: Run integration tests
      run: |
        bazel test //tests:integration_tests --test_output=errors

    - name: Generate coverage report
      run: |
        bazel coverage //tests:all_tests --combined_report=lcov

    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v3
      with:
        file: ./bazel-out/_coverage/_coverage_report.dat

    - name: Run E2E tests
      run: |
        bazel test //tests:e2e_tests --test_output=streamed
      if: github.event_name == 'push' && github.ref == 'refs/heads/main'

3.10 Quality Gates and Metrics

Quality Gate Configuration

# BUILD file
load("@rules_python//python:defs.bzl", "py_test")

py_test(
    name = "quality_gates",
    srcs = ["quality_gates_test.py"],
    deps = [
        "//src:all_modules",
        "@pip//pytest",
        "@pip//pytest-cov",
        "@pip//flake8",
        "@pip//mypy",
        "@pip//black",
    ],
    args = [
        "--cov=src",
        "--cov-fail-under=80",
        "--flake8",
        "--mypy",
        "--black-check",
    ],
    tags = ["quality"],
)

Performance Testing

# performance_test.py
import pytest
import time
from src.calculator import Calculator

class TestPerformance:
    def test_calculation_performance(self):
        calc = Calculator()

        start_time = time.time()
        for i in range(10000):
            calc.add(i, i + 1)
        end_time = time.time()

        execution_time = end_time - start_time
        assert execution_time < 1.0, f"Performance test failed: {execution_time}s"

    @pytest.mark.benchmark
    def test_benchmark_operations(self, benchmark):
        calc = Calculator()
        result = benchmark(calc.multiply, 123, 456)
        assert result == 56088

3.11 Test Reporting and Analytics

Custom Test Reporter

# test_reporter.py
import json
import sys
from datetime import datetime

class TestReporter:
    def __init__(self):
        self.results = {
            'timestamp': datetime.utcnow().isoformat(),
            'tests': [],
            'summary': {
                'total': 0,
                'passed': 0,
                'failed': 0,
                'skipped': 0
            }
        }

    def add_test_result(self, test_name, status, duration, error=None):
        self.results['tests'].append({
            'name': test_name,
            'status': status,
            'duration': duration,
            'error': error
        })

        self.results['summary']['total'] += 1
        self.results['summary'][status] += 1

    def generate_report(self, output_file='test_report.json'):
        with open(output_file, 'w') as f:
            json.dump(self.results, f, indent=2)

        print(f"Test Report Generated: {output_file}")
        print(f"Total: {self.results['summary']['total']}")
        print(f"Passed: {self.results['summary']['passed']}")
        print(f"Failed: {self.results['summary']['failed']}")
        print(f"Skipped: {self.results['summary']['skipped']}")

3.12 Best Practices and Advanced Patterns

Test Utilities and Helpers

# test_utils.py
import functools
import time
from typing import Callable, Any

def retry(max_attempts: int = 3, delay: float = 1.0):
    """Decorator to retry flaky tests"""
    def decorator(func: Callable) -> Callable:
        @functools.wraps(func)
        def wrapper(*args, **kwargs) -> Any:
            last_exception = None
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    last_exception = e
                    if attempt < max_attempts - 1:
                        time.sleep(delay)
                    continue
            raise last_exception
        return wrapper
    return decorator

def timeout(seconds: int):
    """Decorator to add timeout to tests"""
    def decorator(func: Callable) -> Callable:
        @functools.wraps(func)
        def wrapper(*args, **kwargs) -> Any:
            import signal

            def timeout_handler(signum, frame):
                raise TimeoutError(f"Test timed out after {seconds} seconds")

            signal.signal(signal.SIGALRM, timeout_handler)
            signal.alarm(seconds)

            try:
                return func(*args, **kwargs)
            finally:
                signal.alarm(0)
        return wrapper
    return decorator

Test Data Factories

# test_factories.py
import factory
from datetime import datetime, timedelta
from src.models import User, Product, Order

class UserFactory(factory.Factory):
    class Meta:
        model = User

    id = factory.Sequence(lambda n: n)
    name = factory.Faker('name')
    email = factory.Faker('email')
    created_at = factory.LazyFunction(datetime.utcnow)
    is_active = True

class ProductFactory(factory.Factory):
    class Meta:
        model = Product

    id = factory.Sequence(lambda n: n)
    name = factory.Faker('word')
    price = factory.Faker('pydecimal', left_digits=3, right_digits=2, positive=True)
    description = factory.Faker('text', max_nb_chars=200)

class OrderFactory(factory.Factory):
    class Meta:
        model = Order

    id = factory.Sequence(lambda n: n)
    user = factory.SubFactory(UserFactory)
    products = factory.List([factory.SubFactory(ProductFactory) for _ in range(3)])
    total_amount = factory.LazyAttribute(lambda obj: sum(p.price for p in obj.products))
    created_at = factory.LazyFunction(datetime.utcnow)

Practical Exercise: Complete Testing Pipeline

Create a comprehensive testing pipeline for a simple e-commerce application:

Setup Project Structure:

   ecommerce/
   ├── WORKSPACE
   ├── BUILD
   ├── src/
   │   ├── BUILD
   │   ├── models/
   │   ├── services/
   │   └── api/
   ├── tests/
   │   ├── BUILD
   │   ├── unit/
   │   ├── integration/
   │   └── e2e/
   └── testdata/

Implement Test Categories:
- Unit tests for business logic
- Integration tests for database operations
- API integration tests
- End-to-end user journey tests
Configure Quality Gates:
- 80% code coverage requirement
- Performance benchmarks
- Code quality checks (linting, type checking)
Setup CI/CD Pipeline:
- Automated test execution
- Coverage reporting
- Quality gate enforcement

Module Summary

In this module, you've learned:

Comprehensive testing strategies with Bazel
Advanced testing patterns and best practices
Test organization, sharding, and parallelization
Integration and end-to-end testing approaches
Quality gates and continuous integration
Performance testing and monitoring

https://www.linkedin.com/in/sushilbaligar/
https://github.com/sushilbaligar
https://dev.to/sushilbaligar
https://medium.com/@sushilbaligar

Modern Bazel with Python- Module 2 - Libraries and Dependencies

Sushil Baligar — Wed, 23 Jul 2025 06:25:28 +0000

Why Python Libraries Matter in Bazel

In Module 1, we created a single executable file. But real-world Python projects need:

Code reuse across multiple programs
Modular architecture for maintainability
Dependency management between components
Visibility controls for code organization

Traditional Python handles this with packages and imports. Bazel takes it further with explicit dependency graphs and incremental builds.

The Problem with Traditional Python Organization

Consider this typical Python project structure:

my_project/
├── utils/
│   ├── __init__.py
│   ├── math_helpers.py
│   └── string_helpers.py
├── main.py
└── requirements.txt

Issues at scale:

Implicit dependencies: Hard to know what depends on what
No incremental builds: Change one util, rebuild everything
Import path confusion: Relative vs absolute imports
No visibility control: Everything can import everything

How Bazel Solves Library Management

Bazel introduces:

Explicit dependencies: Every target declares what it needs
Incremental builds: Only rebuild affected targets
Clear visibility: Control who can use your code
Dependency graphs: Visualize and analyze relationships

Let's build this step by step!

Project Structure for Module 2

We'll expand our project to demonstrate library concepts:

bazel-python-tutorial/
├── MODULE.bazel                    # Same as Module 1
├── .bazelrc                       # Same as Module 1  
├── .bazelversion                  # Same as Module 1
├── BUILD.bazel                    # Root build file
├── lib/                           # Our library package
│   ├── BUILD.bazel               # Library build instructions
│   ├── math_utils.py             # Mathematical utilities
│   └── string_utils.py           # String processing utilities
└── apps/                          # Applications using our libraries
    ├── BUILD.bazel               # Application build instructions
    ├── calculator.py             # Uses math_utils
    └── text_processor.py         # Uses string_utils

Step 1: Create Library Code

Create the lib/ directory and files

File: lib/math_utils.py

#!/usr/bin/env python3
"""
Mathematical utility functions for Bazel tutorial.
This demonstrates how to create reusable Python libraries in Bazel.
"""

import math
from typing import List, Union

Number = Union[int, float]

def fibonacci(n: int) -> int:
    """
    Calculate the nth Fibonacci number using dynamic programming.

    Args:
        n: Position in Fibonacci sequence (0-indexed)

    Returns:
        The nth Fibonacci number

    Example:
        >>> fibonacci(5)
        5
        >>> fibonacci(10)
        55
    """
    if n <= 1:
        return n

    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b

def prime_factors(n: int) -> List[int]:
    """
    Find all prime factors of a number.

    Args:
        n: Number to factorize

    Returns:
        List of prime factors

    Example:
        >>> prime_factors(12)
        [2, 2, 3]
    """
    if n <= 1:
        return []

    factors = []
    d = 2
    while d * d <= n:
        while n % d == 0:
            factors.append(d)
            n //= d
        d += 1
    if n > 1:
        factors.append(n)
    return factors

def is_perfect_square(n: int) -> bool:
    """
    Check if a number is a perfect square.

    Args:
        n: Number to check

    Returns:
        True if n is a perfect square, False otherwise
    """
    if n < 0:
        return False
    root = int(math.sqrt(n))
    return root * root == n

class Calculator:
    """
    Advanced calculator with memory functionality.
    Demonstrates class-based library design in Bazel.
    """

    def __init__(self):
        """Initialize calculator with zero memory."""
        self.memory: Number = 0
        self.history: List[str] = []

    def add(self, a: Number, b: Number) -> Number:
        """Add two numbers and store result in memory."""
        result = a + b
        self._update_memory(result, f"{a} + {b} = {result}")
        return result

    def multiply(self, a: Number, b: Number) -> Number:
        """Multiply two numbers and store result in memory."""
        result = a * b
        self._update_memory(result, f"{a} × {b} = {result}")
        return result

    def power(self, base: Number, exponent: Number) -> Number:
        """Calculate base^exponent and store result in memory."""
        result = base ** exponent
        self._update_memory(result, f"{base}^{exponent} = {result}")
        return result

    def recall(self) -> Number:
        """Recall the last calculated value."""
        return self.memory

    def get_history(self) -> List[str]:
        """Get calculation history."""
        return self.history.copy()

    def clear(self) -> None:
        """Clear memory and history."""
        self.memory = 0
        self.history.clear()

    def _update_memory(self, value: Number, operation: str) -> None:
        """Update memory and add to history."""
        self.memory = value
        self.history.append(operation)

File: lib/string_utils.py

#!/usr/bin/env python3
"""
String processing utilities for Bazel tutorial.
This demonstrates another library component with different functionality.
"""

import re
from typing import List, Dict, Optional
from collections import Counter

def reverse_words(text: str) -> str:
    """
    Reverse the order of words in a string.

    Args:
        text: Input string

    Returns:
        String with words in reverse order

    Example:
        >>> reverse_words("Hello Bazel World")
        "World Bazel Hello"
    """
    return ' '.join(text.split()[::-1])

def count_vowels(text: str) -> int:
    """
    Count vowels in a string (case insensitive).

    Args:
        text: Input string

    Returns:
        Number of vowels found
    """
    return len(re.findall(r'[aeiouAEIOU]', text))

def camel_to_snake(text: str) -> str:
    """
    Convert CamelCase to snake_case.

    Args:
        text: CamelCase string

    Returns:
        snake_case string

    Example:
        >>> camel_to_snake("HelloBazelWorld")
        "hello_bazel_world"
    """
    # Insert underscore before uppercase letters (except first character)
    result = re.sub(r'(?<!^)(?=[A-Z])', '_', text)
    return result.lower()

def extract_emails(text: str) -> List[str]:
    """
    Extract email addresses from text using regex.

    Args:
        text: Text containing potential email addresses

    Returns:
        List of found email addresses
    """
    pattern = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'
    return re.findall(pattern, text)

def find_longest_word(text: str) -> Optional[str]:
    """
    Find the longest word in a string.

    Args:
        text: Input string

    Returns:
        Longest word, or None if no words found
    """
    words = text.split()
    return max(words, key=len) if words else None

class TextAnalyzer:
    """
    Advanced text analysis functionality.
    Demonstrates stateful library classes.
    """

    def __init__(self, text: str):
        """
        Initialize analyzer with text.

        Args:
            text: Text to analyze
        """
        self.original_text = text
        self.words = text.split()
        self.clean_words = [
            word.lower().strip('.,!?;:"()[]{}') 
            for word in self.words
        ]

    def word_frequency(self) -> Dict[str, int]:
        """
        Calculate word frequency distribution.

        Returns:
            Dictionary mapping words to their frequencies
        """
        return dict(Counter(self.clean_words))

    def most_common_words(self, n: int = 5) -> List[tuple]:
        """
        Get the most common words.

        Args:
            n: Number of top words to return

        Returns:
            List of (word, frequency) tuples
        """
        counter = Counter(self.clean_words)
        return counter.most_common(n)

    def average_word_length(self) -> float:
        """
        Calculate average word length.

        Returns:
            Average length of words in the text
        """
        if not self.words:
            return 0.0
        return sum(len(word) for word in self.words) / len(self.words)

    def sentence_count(self) -> int:
        """
        Count sentences in the text.

        Returns:
            Number of sentences (approximate)
        """
        sentence_endings = re.findall(r'[.!?]+', self.original_text)
        return len(sentence_endings)

    def readability_score(self) -> Dict[str, float]:
        """
        Calculate basic readability metrics.

        Returns:
            Dictionary with readability statistics
        """
        word_count = len(self.words)
        sentence_count = max(self.sentence_count(), 1)  # Avoid division by zero

        return {
            'words_per_sentence': word_count / sentence_count,
            'average_word_length': self.average_word_length(),
            'unique_word_ratio': len(set(self.clean_words)) / max(word_count, 1)
        }

Step 2: Create Library BUILD.bazel

File: lib/BUILD.bazel

"""
Build instructions for utility libraries.
This demonstrates how to create py_library targets in Bazel.
"""

load("@rules_python//python:defs.bzl", "py_library")

# Mathematical utilities library
py_library(
    name = "math_utils",
    srcs = ["math_utils.py"],
    visibility = ["//visibility:public"],  # Allow other packages to use this
)

# String processing utilities library  
py_library(
    name = "string_utils",
    srcs = ["string_utils.py"],
    visibility = ["//visibility:public"],  # Allow other packages to use this
)

# Combined utilities library (depends on both above libraries)
py_library(
    name = "all_utils",
    deps = [
        ":math_utils",    # Depend on math_utils target in same package
        ":string_utils",  # Depend on string_utils target in same package
    ],
    visibility = ["//visibility:public"],
)

Key Concepts Explained:

py_library: Creates a reusable Python library (not executable)
visibility: Controls which other packages can use this library
- ["//visibility:public"] = Anyone can use it
- ["//visibility:private"] = Only this package can use it
- ["//apps:__pkg__"] = Only the //apps package can use it
deps: Lists other libraries this target depends on
:target_name: References a target in the same package

Step 3: Create Applications Using Libraries

Create apps/ directory

File: apps/calculator.py

#!/usr/bin/env python3
"""
Calculator application demonstrating library usage in Bazel.
This shows how to import and use py_library targets.
"""

# Import from our library - Bazel handles the import path
from lib.math_utils import Calculator, fibonacci, prime_factors, is_perfect_square

def demonstrate_calculator():
    """Demonstrate the Calculator class functionality."""
    print("🧮 Calculator Demo")
    print("=" * 50)

    calc = Calculator()

    # Basic operations
    result1 = calc.add(15, 25)
    print(f"15 + 25 = {result1}")

    result2 = calc.multiply(6, 7)
    print(f"6 × 7 = {result2}")

    result3 = calc.power(2, 10)
    print(f"2^10 = {result3}")

    print(f"\nLast result in memory: {calc.recall()}")

    print("\nCalculation History:")
    for operation in calc.get_history():
        print(f"  {operation}")

def demonstrate_math_functions():
    """Demonstrate standalone math functions."""
    print("\n🔢 Math Functions Demo")
    print("=" * 50)

    # Fibonacci sequence
    print("Fibonacci numbers:")
    for i in range(8):
        fib = fibonacci(i)
        print(f"  F({i}) = {fib}")

    # Prime factorization
    numbers_to_factor = [12, 17, 100, 97]
    print(f"\nPrime factorization:")
    for num in numbers_to_factor:
        factors = prime_factors(num)
        print(f"  {num} = {' × '.join(map(str, factors))}")

    # Perfect squares
    test_numbers = [16, 17, 25, 30, 36]
    print(f"\nPerfect square check:")
    for num in test_numbers:
        is_perfect = is_perfect_square(num)
        status = "✅" if is_perfect else "❌"
        print(f"  {num}: {status}")

def main():
    """Main application entry point."""
    print("🚀 Bazel Python Library Demo - Calculator App")
    print("This app uses //lib:math_utils library")
    print()

    demonstrate_calculator()
    demonstrate_math_functions()

    print(f"\n✅ Success! This demonstrates:")
    print("  • Using py_library targets from other packages")
    print("  • Importing library code with clean imports")
    print("  • Bazel's dependency management")

if __name__ == "__main__":
    main()

File: apps/text_processor.py

#!/usr/bin/env python3
"""
Text processing application demonstrating string utilities.
Shows how to use multiple library functions together.
"""

# Import from our string utilities library
from lib.string_utils import (
    TextAnalyzer, reverse_words, count_vowels, 
    camel_to_snake, extract_emails, find_longest_word
)

def demonstrate_text_functions():
    """Demonstrate standalone text processing functions."""
    print("📝 Text Processing Functions Demo")
    print("=" * 50)

    sample_text = "Hello Bazel World! This is Amazing."

    print(f"Original: '{sample_text}'")
    print(f"Reversed words: '{reverse_words(sample_text)}'")
    print(f"Vowel count: {count_vowels(sample_text)}")
    print(f"Longest word: '{find_longest_word(sample_text)}'")

    # CamelCase conversion
    camel_examples = ["HelloBazelWorld", "PythonIsAwesome", "BuildSystemsRock"]
    print(f"\nCamelCase to snake_case:")
    for camel in camel_examples:
        snake = camel_to_snake(camel)
        print(f"  {camel} → {snake}")

    # Email extraction
    email_text = "Contact us at hello@bazel.build or support@example.com for help!"
    emails = extract_emails(email_text)
    print(f"\nEmail extraction from: '{email_text}'")
    print(f"Found emails: {emails}")

def demonstrate_text_analyzer():
    """Demonstrate the TextAnalyzer class."""
    print("\n🔍 Text Analyzer Demo")
    print("=" * 50)

    sample_document = """
    Bazel is a fast, scalable, multi-language build system. 
    Bazel helps developers build software efficiently. 
    The build system is designed for large codebases and teams.
    Bazel supports many programming languages including Python.
    """

    analyzer = TextAnalyzer(sample_document.strip())

    print("Analyzing document:")
    print(f"'{sample_document.strip()[:60]}...'")
    print()

    # Word frequency analysis
    frequency = analyzer.word_frequency()
    print("Word frequencies:")
    sorted_words = sorted(frequency.items(), key=lambda x: x[1], reverse=True)
    for word, count in sorted_words[:8]:  # Top 8 words
        print(f"  '{word}': {count}")

    # Most common words
    print(f"\nTop 5 most common words:")
    for word, count in analyzer.most_common_words(5):
        print(f"  {word} ({count} times)")

    # Readability metrics
    metrics = analyzer.readability_score()
    print(f"\nReadability metrics:")
    print(f"  Average words per sentence: {metrics['words_per_sentence']:.1f}")
    print(f"  Average word length: {metrics['average_word_length']:.1f}")
    print(f"  Unique word ratio: {metrics['unique_word_ratio']:.1%}")

def main():
    """Main application entry point."""
    print("🚀 Bazel Python Library Demo - Text Processor App")
    print("This app uses //lib:string_utils library")
    print()

    demonstrate_text_functions()
    demonstrate_text_analyzer()

    print(f"\n✅ Success! This demonstrates:")
    print("  • Using multiple functions from a py_library")  
    print("  • Importing specific functions with clean syntax")
    print("  • Complex library classes (TextAnalyzer)")
    print("  • Bazel's module import system")

if __name__ == "__main__":
    main()

Step 4: Create Applications BUILD.bazel

File: apps/BUILD.bazel

"""
Build instructions for applications that use our libraries.
This demonstrates py_binary targets with library dependencies.
"""

load("@rules_python//python:defs.bzl", "py_binary")

# Calculator application - depends on math utilities
py_binary(
    name = "calculator",
    srcs = ["calculator.py"],
    main = "calculator.py",
    deps = [
        "//lib:math_utils",  # Depend on math_utils library from lib package
    ],
)

# Text processor application - depends on string utilities  
py_binary(
    name = "text_processor",
    srcs = ["text_processor.py"],
    main = "text_processor.py",
    deps = [
        "//lib:string_utils",  # Depend on string_utils library from lib package
    ],
)

# Combined application - depends on all utilities
py_binary(
    name = "combined_demo",
    srcs = ["combined_demo.py"],
    main = "combined_demo.py", 
    deps = [
        "//lib:all_utils",  # Depend on combined utilities library
    ],
)

Understanding Cross-Package Dependencies:

//lib:math_utils: Reference to math_utils target in lib package
//package:target: Standard Bazel label format
Dependencies are explicit: You must declare what you use
Build optimization: Bazel only rebuilds changed dependencies

Step 5: Create Combined Demo

File: apps/combined_demo.py

#!/usr/bin/env python3
"""
Combined demo showing both math and string utilities working together.
Demonstrates how libraries can be composed in applications.
"""

from lib.math_utils import Calculator, fibonacci, prime_factors
from lib.string_utils import TextAnalyzer, reverse_words, camel_to_snake

def math_and_text_demo():
    """Demonstrate using both libraries together."""
    print("🔀 Combined Math + Text Demo")
    print("=" * 50)

    # Generate some math data
    calc = Calculator()
    fib_numbers = [fibonacci(i) for i in range(8)]

    # Convert to text and analyze
    fib_text = " ".join(map(str, fib_numbers))
    print(f"Fibonacci sequence: {fib_text}")

    # Analyze the number sequence as text
    analyzer = TextAnalyzer(fib_text)
    print(f"Average 'word' length: {analyzer.average_word_length():.1f}")
    print(f"Unique numbers: {len(analyzer.word_frequency())}")

    # Do some calculations
    total = calc.add(sum(fib_numbers), 0)
    print(f"Sum of Fibonacci numbers: {total}")

    # Convert calculation description to different formats
    description = "FibonacciSequenceSum"
    snake_case = camel_to_snake(description)
    reversed_desc = reverse_words("Fibonacci Sequence Sum")

    print(f"\nText transformations:")
    print(f"  Original: {description}")
    print(f"  Snake case: {snake_case}")
    print(f"  Reversed: {reversed_desc}")

def main():
    """Main entry point."""
    print("🚀 Combined Library Demo")
    print("Using both //lib:math_utils and //lib:string_utils")
    print()

    math_and_text_demo()

    print(f"\n✅ This demonstrates:")
    print("  • Using multiple libraries in one application")
    print("  • Library composition and integration")
    print("  • Clean import statements across packages")

if __name__ == "__main__":
    main()

Step 6: Build and Test Everything

Build Individual Libraries

# Build just the math utilities library
bazel build //lib:math_utils

# Build just the string utilities library  
bazel build //lib:string_utils

# Build the combined utilities library
bazel build //lib:all_utils

Build and Run Applications

# Build and run calculator app
bazel run //apps:calculator

# Build and run text processor app
bazel run //apps:text_processor

# Build and run combined demo
bazel run //apps:combined_demo

Build Everything at Once

# Build all targets in the project
bazel build //...

# Query to see all available targets
bazel query //...

Understanding Bazel's Dependency Graph

Visualize Dependencies

# See dependency graph for calculator app
bazel query --output=graph //apps:calculator

# See all dependencies for all targets
bazel query 'deps(//...)' --output=graph

Test Incremental Builds

# Build everything once
bazel build //...

# Change one line in lib/math_utils.py, then rebuild
bazel build //...
# Notice: Only affected targets rebuild!

Key Concepts Mastered

1. py_library vs py_binary

py_library: Reusable code, can be imported by other targets
py_binary: Executable programs with main() functions

2. Dependency Declaration

Must explicitly declare all dependencies in deps = []
Bazel enforces this - missing deps cause build failures
Enables precise incremental builds

3. Visibility Control

//visibility:public: Anyone can use this library
//visibility:private: Only current package can use it
Custom visibility: ["//apps:__pkg__"] - only apps package

4. Cross-Package References

//package:target format for referencing other packages
Clear separation between packages
Explicit dependency management

5. Import Path Handling

Bazel automatically handles Python import paths
Import using package structure: from lib.math_utils import ...
No need for complex PYTHONPATH manipulation

What's Next?

In Module 3, we'll add testing to our libraries:

py_test targets for unit testing
Test data management
Test discovery and execution
Coverage reporting

Ready to continue with Module 3: Testing Framework?

Summary

✅ Created reusable py_library targets

✅ Demonstrated cross-package dependencies

✅ Learned visibility controls

✅ Built multiple applications using shared libraries

✅ Understood Bazel's incremental build system

Your project now has a solid library architecture that scales!

Follow me for more such updates!
https://www.linkedin.com/in/sushilbaligar/
https://github.com/sushilbaligar
https://dev.to/sushilbaligar
https://medium.com/@sushilbaligar

Modern Bazel with Python- Module 1 - Basics

Sushil Baligar — Tue, 22 Jul 2025 18:13:07 +0000

Why Bazel is Revolutionary for Python Development

Before diving into code, let's understand why Google open-sourced their internal build system and why it's becoming essential for serious Python development.

The Problem with Traditional Python Build Systems

Most Python developers are familiar with this workflow:

pip install -r requirements.txt
python setup.py build
python -m pytest

This approach breaks down at scale:

Inconsistent builds: "Works on my machine" syndrome
Slow rebuilds: Everything rebuilds even when only one file changes
Dependency hell: Version conflicts across projects
No parallelization: Tests and builds run sequentially
Language barriers: Hard to integrate C++, Java, or other languages

How Bazel Solves These Problems

Bazel introduces several revolutionary concepts:

Hermetic Builds: Same inputs always produce identical outputs, regardless of the machine or environment.

Incremental Builds: Only rebuilds what actually changed, using cryptographic hashing to detect changes.

Massive Parallelization: Builds independent targets simultaneously across multiple cores.

Remote Caching: Share build artifacts across your entire team or CI/CD pipeline.

Multi-language Support: Python, Java, C++, Go, and more in a single build system.

Why MODULE.bazel Over WORKSPACE

Bazel is transitioning from WORKSPACE files to MODULE.bazel (called "Bzlmod") for several reasons:

Better dependency resolution: Handles version conflicts automatically
Simplified syntax: Less boilerplate, more intuitive
Improved performance: Faster loading and resolution
Future-proof: This is where Bazel is heading

Now let's build our first modern Bazel Python project!

Setting Up Your First Modern Bazel Python Project

Project Structure Overview

We'll create a minimal but complete Bazel project:

bazel-python-tutorial/
├── MODULE.bazel          # Modern dependency management (replaces WORKSPACE)
├── .bazelrc             # Build configuration
├── .bazelversion        # Lock Bazel version for team consistency
├── BUILD.bazel          # Build instructions for this directory
└── hello.py             # Our Python source code

Step 1: Initialize Your Project

Create your project directory:

mkdir bazel-python-tutorial
cd bazel-python-tutorial

Step 2: Create MODULE.bazel - The Modern Way

The MODULE.bazel file is your project's dependency manifest. It's cleaner and more powerful than the old WORKSPACE approach.

# MODULE.bazel
module(
    name = "bazel_python_tutorial",
    version = "1.0.0",
)

bazel_dep(name = "rules_python", version = "0.29.0")

# Simplified Python setup - let Bazel find system Python
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    python_version = "3.11",
)

Key Differences from WORKSPACE:

No need for http_archive or SHA hashes
Automatic version resolution
Cleaner, more declarative syntax
Built-in dependency management

Step 3: Create .bazelrc - Build Configuration

# .bazelrc
# Configuration file that sets default behavior for Bazel commands
# Think of this as your "build preferences"

# === Build Configuration ===
build --verbose_failures
build --show_progress_rate_limit=5

test --test_output=errors
test --test_summary=detailed

common --enable_bzlmod

Step 4: Lock Bazel Version (.bazelversion)

7.1.1

Step 5: Create BUILD.bazel - Build Instructions

# BUILD.bazel
"""
Build instructions for the root directory.
Every directory containing source code needs a BUILD.bazel file.
"""

# Import the py_binary rule from Python rules
load("@rules_python//python:defs.bzl", "py_binary")

# Define a Python executable target
py_binary(
    name = "hello",
    srcs = ["hello.py"],           # Source files to include
    main = "hello.py",             # Entry point file
)

Understanding py_binary:

name: What you'll type in bazel run //:{name}
srcs: List of Python files this target includes
main: Which file contains if __name__ == "__main__":
python_version: Ensures Python 3 compatibility

Step 6: Write Your Python Code

#!/usr/bin/env python3
# hello.py
"""
Your first Bazel Python program!
This demonstrates modern Bazel with Python.
"""

import sys
from typing import List

def create_greeting(name: str, enthusiasm_level: int = 1) -> str:
    """
    Create a personalized greeting with variable enthusiasm.

    Args:
        name: The name to greet
        enthusiasm_level: Number of exclamation marks (1-3)

    Returns:
        A formatted greeting string
    """
    exclamation = "!" * min(max(enthusiasm_level, 1), 3)
    return f"Hello, {name}{exclamation}"

def display_bazel_info() -> None:
    """Display information about this Bazel build."""
    print("🚀 Modern Bazel + Python Demo")
    print(f"Python version: {sys.version}")
    print(f"Running from: {__file__}")
    print("Built with: Bazel + MODULE.bazel (Bzlmod)")
    print("-" * 50)

def main(args: List[str] = None) -> None:
    """
    Main entry point of our application.

    Args:
        args: Command line arguments (optional)
    """
    display_bazel_info()

    # Basic greeting
    print(create_greeting("Bazel World"))

    # Enthusiastic greeting
    print(create_greeting("Modern Python Developer", 3))

    # Success message
    print("\n✅ Congratulations! You've successfully:")
    print("   • Set up modern Bazel with MODULE.bazel")
    print("   • Built your first py_binary target")
    print("   • Used Bzlmod for dependency management")
    print("   • Created a reproducible, scalable build")

if __name__ == "__main__":
    main(sys.argv[1:])

Building and Running Your First Modern Bazel Python Program

Build the Project

# Build your hello target
bazel build //:hello

What happens behind the scenes:

Bazel reads MODULE.bazel and downloads Python rules
Sets up Python 3.11 toolchain
Analyzes BUILD.bazel to understand dependencies
Compiles and packages your Python code
Creates executable in bazel-bin/

Run the Program

# Run your hello target
bazel run //:hello

Expected Output:

🚀 Modern Bazel + Python Demo
Python version: 3.11.x (main, ...)
Running from: /path/to/your/project/hello.py
Built with: Bazel + MODULE.bazel (Bzlmod)
--------------------------------------------------
Hello, Bazel World!
Hello, Modern Python Developer!!!

✅ Congratulations! You've successfully:
   • Set up modern Bazel with MODULE.bazel
   • Built your first py_binary target
   • Used Bzlmod for dependency management
   • Created a reproducible, scalable build

Understanding Bazel Labels

The //:hello syntax is called a "Bazel label":

// = Root of the workspace (where MODULE.bazel lives)
: = Separator between package and target
hello = Target name (from name = "hello" in BUILD.bazel)

Useful Commands to Try

# List all available targets in your project
bazel query //...

# Get detailed information about the hello target
bazel query //:hello --output=build

# See what files Bazel generated
ls -la bazel-bin/

# Clean all build outputs
bazel clean

# Build with verbose output (great for learning)
bazel build //:hello --verbose_failures --announce_rc

What Makes This Better Than Traditional Python?

Reproducible Builds

Every developer on your team will get identical builds because:

Python version is locked via MODULE.bazel
Bazel version is locked via .bazelversion
All dependencies are precisely specified

Incremental Builds

Change one line in hello.py and rebuild:

# Edit hello.py, then rebuild
bazel build //:hello

Bazel only rebuilds what changed - incredibly fast!

Scalability Foundation

This simple setup scales to:

Hundreds of Python modules
Mixed-language projects (Python + C++ + Java)
Microservices architectures
Thousands of developers

Hope this helps! Follow me for more such updates!
https://www.linkedin.com/in/sushilbaligar/
https://github.com/sushilbaligar
https://dev.to/sushilbaligar
https://medium.com/@sushilbaligar