DEV Community

Allen Elzayn
Allen Elzayn

Posted on

Forge: Lightweight, Fast, and Reliable Local CI/CD

#rust #programming #cicd #devops

Originally published at allenarch.dev

here's a frustrating moment every developer has experienced: push code to GitHub, wait for GitHub Actions to run, realize there's a typo in the config file. Push again, wait again. Repeat until the CI/CD pipeline finally passes.

The problem? We can't easily test pipelines locally. GitHub Actions only runs in the cloud. GitLab CI is the same. CircleCI? You need to set up the project first. Everything requires pushing to remote, which means unnecessary delays.

That's why I built Forge - a lightweight, fast, and reliable local CI/CD tool. Built with Rust for performance, runs using Docker, and has syntax similar to GitHub Actions so you don't need to learn a new format.

Why Do We Need Local CI/CD?

Imagine this scenario: you have a monorepo with 5 different services. Each service has its own test suite. Before pushing, you need to:

  1. Run tests for all services manually
  2. Build each service
  3. Lint check
  4. Format check
  5. And many more

Or... you could push first, wait for cloud CI/CD, then find out something failed. Waste of time.

Forge solves this simply: run the same pipeline as GitHub Actions, but locally. Test before pushing. Fix before committing. Save time, save frustration.

Architecture: Rust + Docker = Speed + Isolation

Forge is built with Rust for performance and reliability. It ships as a single binary that can run without additional dependencies (except Docker).

Docker is used for isolation. Each step in the pipeline runs in a separate container, so:

  • Dependencies don't conflict between steps
  • Environment can be customized per step
  • Execution results are consistent and reproducible

Three-Tier Architecture

Forge uses a three-layer architecture:

1. Main Orchestrator (src/main.rs)

  • Parse CLI commands
  • Load and validate YAML config
  • Resolve dependencies between stages
  • Schedule execution (sequential or parallel)

2. Container Management (src/container.rs)

  • Docker abstraction layer
  • Image pulling and caching
  • Container lifecycle management
  • Volume mounting for workspace
  • Real-time log streaming

3. Execution Engine

  • Stage dependency resolution (topological sort using Kahn's algorithm)
  • Parallel execution with proper log aggregation
  • Cache key derivation from lockfiles
  • Secrets management and environment variable injection

Familiar Syntax: YAML Like GitHub Actions

One important design decision: Forge uses syntax similar to GitHub Actions and GitLab CI. Why? Because developers are already familiar with this format. No need to learn new syntax.

# forge.yaml
cache:
  enabled: true
  directories:
    - /workspace/node_modules

stages:
  - name: install
    steps:
      - name: Install dependencies
        image: node:20
        command: npm ci
        working_dir: /workspace

  - name: test
    depends_on:
      - install
    steps:
      - name: Run tests
        image: node:20
        command: npm test
        working_dir: /workspace
        env:
          NODE_ENV: test

  - name: build
    depends_on:
      - test
    steps:
      - name: Build application
        image: node:20
        command: npm run build
        working_dir: /workspace
Enter fullscreen mode Exit fullscreen mode

Simple, clean, and immediately understandable by anyone who has used GitHub Actions.

Key Features

1. Container Isolation

Each step runs in its own Docker container. This means:

  • Dependencies don't interfere with each other
  • Can use different images per step (Node.js for frontend, Python for ML, Rust for backend)
  • Clean environment every time you run

2. Parallel Execution

Stages that don't depend on each other can run in parallel. Forge automatically detects dependencies and schedules execution optimally.

stages:
  - name: lint-frontend
    steps: [...]

  - name: lint-backend
    steps: [...]

  - name: build
    depends_on:
      - lint-frontend
      - lint-backend
    steps: [...]
Enter fullscreen mode Exit fullscreen mode

lint-frontend and lint-backend will run simultaneously, then build runs after both complete.

3. Intelligent Caching

Forge has a smart caching system. Cache is configured at the top level of forge.yaml and applies to all stages. Cache keys are automatically derived from lockfiles (package-lock.json, Cargo.lock, requirements.txt, etc.), so cache automatically invalidates when dependencies change.

cache:
  enabled: true
  directories:
    - /workspace/node_modules
    - /workspace/.next/cache
Enter fullscreen mode Exit fullscreen mode

Cache is stored locally at ./.forge/cache/ and automatically managed by Forge. Cache keys are derived from lockfiles (package-lock.json, Cargo.lock, etc.), so cache automatically invalidates when dependencies change.

4. Secrets Management

Forge supports secure secrets management. Secrets are defined in forge.yaml and loaded from host environment variables (or .env files). They are automatically injected into all containers as environment variables.

secrets:
  - name: API_KEY
    env_var: FORGE_API_KEY

stages:
  - name: deploy
    steps:
      - name: Deploy
        image: alpine:latest
        command: deploy.sh
        env:
          API_KEY: ${{ secrets.API_KEY }}
Enter fullscreen mode Exit fullscreen mode

5. Real-Time Logging

Logs from each step are streamed in real-time with proper formatting. If a step fails, Forge immediately stops execution (fail-fast) and displays a clear error.

6. Multi-Language Support

Forge isn't tied to one programming language. Since it uses Docker, you can use any image:

  • node:20 for Node.js
  • rust:1.75 for Rust
  • python:3.11 for Python
  • golang:1.21 for Go
  • Or your own custom image

Dependency Resolution: Topological Sort

One feature I'm most proud of is dependency resolution. Forge uses Kahn's algorithm for topological sorting, so:

  • Circular dependencies are detected before execution
  • Self-dependencies are detected and rejected
  • Missing dependencies are detected with clear error messages
  • Optimal execution order 
stages:
  - name: A
    depends_on: []

  - name: B
    depends_on: [A]

  - name: C
    depends_on: [A]

  - name: D
    depends_on: [B, C]
Enter fullscreen mode Exit fullscreen mode

Forge will execute: A → (B, C parallel) → D.

Use Cases: From Simple Scripts to Monorepos

Simple Node.js Project 

cache:
  enabled: true
  directories:
    - /workspace/node_modules

stages:
  - name: install
    steps:
      - name: Install
        image: node:20
        command: npm ci
        working_dir: /workspace

  - name: test
    depends_on: [install]
    steps:
      - name: Test
        image: node:20
        command: npm test
        working_dir: /workspace

  - name: build
    depends_on: [test]
    steps:
      - name: Build
        image: node:20
        command: npm run build
        working_dir: /workspace
Enter fullscreen mode Exit fullscreen mode

Rust Project with Cargo 

cache:
  enabled: true
  directories:
    - /workspace/target

stages:
  - name: test
    steps:
      - name: Run tests
        image: rust:1.75
        command: cargo test --all-features
        working_dir: /workspace

  - name: build
    depends_on: [test]
    steps:
      - name: Build release
        image: rust:1.75
        command: cargo build --release
        working_dir: /workspace
Enter fullscreen mode Exit fullscreen mode

Multi-Language Monorepo 

stages:
  - name: frontend-test
    steps:
      - name: Test frontend
        image: node:20
        working_dir: /workspace/frontend
        command: npm test

  - name: backend-test
    steps:
      - name: Test backend
        image: python:3.11
        working_dir: /workspace/backend
        command: pytest

  - name: build-all
    depends_on: [frontend-test, backend-test]
    steps:
      - name: Build frontend
        image: node:20
        working_dir: /workspace/frontend
        command: npm run build

      - name: Build backend
        image: python:3.11
        working_dir: /workspace/backend
        command: python setup.py build
Enter fullscreen mode Exit fullscreen mode

Performance: Fast Thanks to Caching and Parallel Execution

Forge is designed to be fast. The combination of smart caching and parallel execution means pipelines can complete in seconds for small projects, or minutes for large monorepos.

Benchmark from my own project:

  • Without cache: ~2 minutes (download dependencies, compile, test)
  • With cache: ~30 seconds (only compile and test)
  • Parallel stages: ~20 seconds (multiple stages running simultaneously)

For monorepos with 10+ services, parallel execution can cut execution time from 15 minutes to 5 minutes.

Installation: Simple and Cross-Platform

Forge can be installed in several ways:

Pre-compiled binaries (recommended):

# Linux
curl -L https://github.com/0xReLogic/Forge/releases/latest/download/forge-linux-amd64 -o forge
chmod +x forge
sudo mv forge /usr/local/bin/

# macOS
curl -L https://github.com/0xReLogic/Forge/releases/latest/download/forge-macos-amd64 -o forge
chmod +x forge
sudo mv forge /usr/local/bin/

# Windows (PowerShell)
Invoke-WebRequest -Uri "https://github.com/0xReLogic/Forge/releases/latest/download/forge.exe" -OutFile "forge.exe"
Enter fullscreen mode Exit fullscreen mode

Via Cargo:

cargo install forge
Enter fullscreen mode Exit fullscreen mode

From source:

git clone https://github.com/0xReLogic/Forge
cd Forge
cargo build --release
Enter fullscreen mode Exit fullscreen mode

The only dependency is Docker. If Docker is already installed, Forge is ready to use.

Workflow: Init, Validate, Run

Forge has three main commands:

1. forge init
Creates a new forge.yaml with a template that can be used immediately.

2. forge validate
Validates config without executing the pipeline. Useful for checking syntax before committing.

3. forge run
Executes the pipeline. Supports several flags:

  • --file <FILE>: Use a custom configuration file (default: forge.yaml)
  • --stage <STAGE>: Run a specific stage and its dependencies
  • --cache: Force enable caching
  • --no-cache: Force disable caching
  • --verbose: Enable verbose output with performance metrics
  • --dry-run: Validate and preview execution without running containers 
# Initialize new project
forge init

# Validate config
forge validate

# Run full pipeline
forge run

# Run specific stage
forge run --stage test

# Preview execution without running
forge run --dry-run

# Run without cache
forge run --no-cache

# Run with verbose output
forge run --verbose
Enter fullscreen mode Exit fullscreen mode

Error Handling: Fail-Fast with Clear Messages

Forge has comprehensive error handling:

  • Validation errors: Detected before execution, with clear messages about what's wrong
  • Dependency errors: Circular dependencies, self-dependencies, missing dependencies are all detected
  • Execution errors: If a step fails, Forge immediately stops and displays an error with clear context

Every error message is designed to help developers fix issues quickly, not frustrate them with cryptic error messages.

Security: Secrets and Container Isolation

Security is a priority. Forge uses several strategies:

  • Container isolation: Each step runs in a separate container, so there's no cross-contamination
  • Secrets management: Secrets are not logged, only injected into containers that need them
  • No network access: Containers don't have network access unless explicitly needed (can be configured)

Roadmap: The Future of Forge

Forge is still in early development, but stable enough for production use. Some features being considered:

  • Remote execution: Execute pipelines on remote servers or cloud
  • Web UI: Dashboard to monitor pipeline execution
  • Plugin system: Support for custom plugins
  • Integration: Integration with GitHub Actions, GitLab CI, etc.

But for now, Forge is already powerful enough to handle most local CI/CD use cases.

Conclusion: Local CI/CD That Should Have Existed Long Ago

Forge is a tool that should have existed long ago. Local CI/CD should be standard practice, not an exception. With Forge, developers can:

  • Test pipelines before pushing
  • Debug issues locally without waiting for cloud CI/CD
  • Save time and reduce frustration
  • Maintain consistency between local and remote environments

Built with Rust for performance, using Docker for isolation, and familiar syntax for good developer experience. Lightweight, fast, and reliable.

If you've ever been frustrated by having to push-push-push just to test CI/CD config, or if you have a monorepo that needs a complex test suite, Forge might be worth a try.

Repository: github.com/0xReLogic/Forge

Forge is an open-source project still in active development. Contributions, feedback, and bug reports are very welcome!

Resources:

Further Reading:

Connect

Top comments (0)