DEV Community: Lucien Chemaly

HTML to PDF API: Convert Web Content to PDF Programmatically with Foxit

Lucien Chemaly — Thu, 16 Apr 2026 17:04:49 +0000

Your Puppeteer setup works fine at low volume. You launch a Chrome process, load the page, call page.pdf(), and write the bytes to disk. Then your invoice generation hits 500 documents per night, your report export feature goes live across three time zones simultaneously, and the wheels come off. Chrome processes time out waiting for JavaScript hydration. Memory climbs until your container OOMs. The font that renders correctly on your MacBook looks wrong on the Linux build server. You spend a Friday afternoon tuning networkidle2 timeouts per template instead of shipping features.

That failure mode comes from treating a rendering engine as a conversion service. Headless Chrome is a browser, and running it at production document volume means operating a browser fleet: process pooling, memory isolation, crash recovery, rendering consistency across OS environments. All of that infrastructure overhead comes directly out of engineering time.

A managed REST API sidesteps that entirely. You POST your HTML (or a URL), the service renders the PDF, and you download the result. The rendering infrastructure becomes the API provider's problem. This guide covers how to build that conversion pipeline end-to-end using Foxit PDF Services API, from authentication through batch processing and production error handling.

The Production Problem with Headless Browser PDF Conversion

A standard Puppeteer setup looks like this:

const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(url, { waitUntil: "networkidle2" });
const pdf = await page.pdf({ format: "A4", printBackground: true });
await browser.close();

At five documents a day, this is fine. At five hundred concurrent, each puppeteer.launch() spins up a full Chromium process, roughly 100-200MB RSS on Linux. In a container with 2GB of memory and 20 concurrent requests, you're at the memory ceiling before accounting for the Node.js process or any other application memory.

The standard fix is a Chrome process pool (libraries like puppeteer-cluster or generic-pool). Now you're managing pool size tuning, handling pool exhaustion under burst traffic, and writing cleanup logic for crashed Chrome instances. You've added significant operational complexity to what started as a one-liner.

Font rendering is its own category of pain. Chrome on macOS uses CoreText. Chrome on Linux uses FreeType with fontconfig. The same CSS font-family: 'Inter' declaration produces visibly different output depending on whether Inter is installed as a system font or loaded via a @font-face declaration, and whether the fallback stack resolves differently across environments. Teams that ship invoice PDFs to customers discover this in production.

JavaScript execution adds another dimension. If your page renders a data table via a React component that fetches data on mount, networkidle2 is an unreliable wait condition. Network activity can go idle before the DOM finishes updating. You end up tuning waitForSelector or adding arbitrary timeouts per template, and those timeouts become technical debt that breaks when the page changes.

A managed REST API with consistent rendering environments and no infrastructure to maintain solves these problems at the architectural level.

How Cloud HTML-to-PDF APIs Handle Rendering

Cloud conversion APIs typically accept input in two modes: URL mode and file upload mode.

In URL mode, you pass a public URL. The API fetches the page, renders it, and returns a PDF. This works when your page is publicly accessible and all assets (fonts, images, stylesheets) load from the same domain or CDN. The tradeoff is that the API's rendering environment must reach your server, which creates a dependency on network reachability and your server's response time. If you're generating PDFs from an internal dashboard behind a VPN, URL mode doesn't work without additional networking.

In file upload mode, you construct the complete HTML file (with inlined CSS and assets where needed) and upload it to the API. The service processes the file and returns a PDF. This eliminates the external asset dependency and makes your conversion more deterministic. The same HTML file always produces the same PDF, regardless of what's deployed on your web server at the time.

Beyond input mode, rendering fidelity depends on several factors:

CSS @media print rules control what renders into the PDF. Navigation bars, sidebars, and hover states should be hidden via print stylesheets so they don't appear in the output.
Font loading strategy determines rendering consistency. Relying on system fonts produces different output across environments. Embedding fonts via @font-face with a CDN URL or base64-inlined data guarantees consistent rendering.
Page layout properties (paper size, margins, orientation) can be controlled through CSS @page rules embedded in the HTML itself. This keeps layout configuration in the document rather than in API parameters.
JavaScript execution matters for pages that render content dynamically. Some APIs wait for the page to stabilize before capturing; others capture immediately.

These factors are the same ones you'd manage with Puppeteer's page.pdf() options, but with a cloud API you handle them through your HTML and CSS rather than through in-process code.

Setting Up Foxit PDF Services API: Authentication and First Conversion

Foxit PDF Services API is a cloud-hosted REST API built on Foxit's proprietary PDF engine, backed by over 20 years of PDF technology development. Create an account at the Foxit Developer Portal (the Developer plan is free, includes 500 credits/year, and requires no credit card). Generate your API credentials (client_id and client_secret) from the Developer Dashboard.

Understanding the Async Workflow

Foxit PDF Services uses an asynchronous task-based workflow. Every operation follows the same pattern:

Submit the job (upload a file, or POST a URL)
Receive a taskId in the response
Poll the task status until it completes or fails
Download the result using the resultDocumentId from the completed task

This design handles long-running operations cleanly. A complex HTML page might take several seconds to render, and the async pattern means your client never blocks on a single HTTP request waiting for rendering to finish.

URL-to-PDF Conversion

For pages that are publicly accessible, URL-to-PDF is the simplest path. You POST the URL directly and the API fetches, renders, and converts it. The complete workflow in Python uses the requests library:

import os
import requests
from time import sleep

HOST = os.environ["FOXIT_API_HOST"]  # e.g., https://na1.fusion.foxit.com
CLIENT_ID = os.environ["FOXIT_CLIENT_ID"]
CLIENT_SECRET = os.environ["FOXIT_CLIENT_SECRET"]

AUTH_HEADERS = {
    "client_id": CLIENT_ID,
    "client_secret": CLIENT_SECRET,
}


def create_url_to_pdf_task(url: str) -> str:
    """Submit a URL for PDF conversion. Returns a taskId."""
    headers = {**AUTH_HEADERS, "Content-Type": "application/json"}
    response = requests.post(
        f"{HOST}/pdf-services/api/documents/create/pdf-from-url",
        json={"url": url},
        headers=headers,
    )
    response.raise_for_status()
    return response.json()["taskId"]


def poll_task(task_id: str, interval: int = 5) -> dict:
    """Poll until the task completes or fails. Returns the task status object."""
    headers = {**AUTH_HEADERS, "Content-Type": "application/json"}
    while True:
        response = requests.get(
            f"{HOST}/pdf-services/api/tasks/{task_id}",
            headers=headers,
        )
        response.raise_for_status()
        status = response.json()
        if status["status"] == "COMPLETED":
            return status
        elif status["status"] == "FAILED":
            raise RuntimeError(f"Task {task_id} failed: {status}")
        sleep(interval)


def download_document(document_id: str, output_path: str) -> None:
    """Download the resulting PDF by its document ID."""
    response = requests.get(
        f"{HOST}/pdf-services/api/documents/{document_id}/download",
        headers=AUTH_HEADERS,
        stream=True,
    )
    response.raise_for_status()
    with open(output_path, "wb") as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)

# Full workflow: URL to PDF
task_id = create_url_to_pdf_task("https://example.com/invoice/1042")
result = poll_task(task_id)
download_document(result["resultDocumentId"], "invoice_1042.pdf")
print("PDF generated successfully.")

Three reusable functions map to the async workflow: create_url_to_pdf_task() submits a public URL and returns a taskId, poll_task() checks task status in a loop until it reaches COMPLETED or FAILED, and download_document() streams the resulting PDF to disk. The final three lines wire them together into the complete conversion pipeline.

Before running: Set your FOXIT_API_HOST, FOXIT_CLIENT_ID, and FOXIT_CLIENT_SECRET environment variables with the values from your Foxit Developer Dashboard. Never commit credentials to source control; use environment variables or a secrets manager.

HTML File-to-PDF Conversion

When your content isn't publicly accessible (internal dashboards, dynamically generated reports), you can upload an HTML file directly. This follows the same 4-step async pattern:

def upload_document(file_path: str) -> str:
    """Upload a file to Foxit. Returns a documentId."""
    with open(file_path, "rb") as f:
        response = requests.post(
            f"{HOST}/pdf-services/api/documents/upload",
            files={"file": f},
            headers=AUTH_HEADERS,
        )
    response.raise_for_status()
    return response.json()["documentId"]


def create_html_to_pdf_task(document_id: str) -> str:
    """Create an HTML-to-PDF conversion task. Returns a taskId."""
    headers = {**AUTH_HEADERS, "Content-Type": "application/json"}
    response = requests.post(
        f"{HOST}/pdf-services/api/documents/create/pdf-from-html",
        json={"documentId": document_id},
        headers=headers,
    )
    response.raise_for_status()
    return response.json()["taskId"]

# Full workflow: HTML file to PDF
doc_id = upload_document("report.html")
task_id = create_html_to_pdf_task(doc_id)
result = poll_task(task_id)
download_document(result["resultDocumentId"], "report.pdf")
print("HTML converted to PDF successfully.")

You first upload a local .html file via upload_document(), which returns a documentId referencing the uploaded file on Foxit's servers. Then create_html_to_pdf_task() submits that documentId for conversion. The rest of the workflow is identical: poll for completion, then download the result.

Note: Replace "report.html" with the path to your own HTML file. This code reuses the poll_task() and download_document() functions from the URL-to-PDF example above, so make sure both are defined in the same script.

URL-to-PDF skips the upload step because you POST the URL directly. HTML file conversion requires uploading the .html file first via the /documents/upload endpoint. Both use the same poll-and-download pattern after task creation.

Refer to the Foxit API documentation and the Postman workspace for the complete parameter reference, including any additional rendering options supported by these endpoints. The GitHub demo repository contains working examples in Python, Node.js, and PHP.

Controlling CSS and JavaScript Rendering in HTML-to-PDF Conversion

Regardless of which API you use for HTML-to-PDF conversion, output quality depends on how well you prepare the HTML. The rendering parameters live in your document, not in API request fields.

The most common rendering problem between "looks right in a browser" and "looks wrong in a PDF" is the CSS media type. By default, browsers render with screen styles, which means your navigation bar, sidebar, and hover states all appear in the output. For PDF output, you want your @media print rules to take over.

@media print {
  nav,
  .sidebar,
  .no-print {
    display: none;
  }

  body {
    font-size: 11pt;
    font-family: "Inter", Arial, sans-serif;
    color: #000;
  }

  .invoice-table {
    page-break-inside: avoid;
  }

  .page-header {
    page-break-before: always;
  }

  @page {
    size: A4;
    margin: 20mm 15mm;
  }
}

This stylesheet hides non-essential UI elements (navigation, sidebars) when printing, sets a clean body font, and uses page-break-inside: avoid to prevent the renderer from splitting a table row across pages. The nested @page rule sets the paper size and margins at the CSS level, so layout configuration stays in the document rather than in API parameters.

For font rendering consistency, don't rely on system fonts. Include a @font-face declaration in your HTML that loads from a CDN, or inline the font as base64:

<style>
  @font-face {
    font-family: "Inter";
    src: url("https://fonts.gstatic.com/s/inter/v13/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfAZ9hiJ.woff2")
      format("woff2");
    font-weight: 400;
    font-style: normal;
  }
</style>

This embeds the Inter font directly in the HTML using a @font-face declaration pointing to Google Fonts. Inter renders in the PDF regardless of what fonts are installed in the API's container environment. The tradeoff is latency: the rendering engine fetches the font file during conversion. If you're running high-volume batch jobs, consider inlining the font as a base64 data URI to eliminate that network round trip.

For JavaScript-heavy pages, make sure the content has fully rendered before the API captures it. If you're using the URL-to-PDF endpoint, the API fetches and renders the live page, so your page's JavaScript will execute. For the HTML file upload path, keep your HTML self-contained with all data already rendered in the markup, rather than relying on client-side JavaScript to populate it after load.

Batch HTML-to-PDF Conversion at Scale

Sequential conversion is the naive starting point:

for invoice in invoices:
    doc_id = upload_document(invoice.html_path)
    task_id = create_html_to_pdf_task(doc_id)
    result = poll_task(task_id)
    download_document(result["resultDocumentId"], f"output/{invoice.id}.pdf")

Each invoice is processed one at a time, uploading, converting, polling, and downloading before moving to the next. Each iteration blocks on the poll loop before starting the next conversion. At a few seconds per document (upload, render, poll, download), 500 invoices could take over 30 minutes.

Concurrent dispatch with a semaphore to cap parallelism fixes that. Check your plan's rate limits before setting the semaphore ceiling in production.

import asyncio
import aiohttp
import os
from pathlib import Path

HOST = os.environ["FOXIT_API_HOST"]
CLIENT_ID = os.environ["FOXIT_CLIENT_ID"]
CLIENT_SECRET = os.environ["FOXIT_CLIENT_SECRET"]
MAX_CONCURRENT = 10  # Adjust based on your plan's rate limits


async def convert_one(
    session: aiohttp.ClientSession,
    sem: asyncio.Semaphore,
    invoice_id: str,
    html_path: str,
    output_dir: Path,
) -> tuple[str, bool]:
    async with sem:
        try:
            auth = {"client_id": CLIENT_ID, "client_secret": CLIENT_SECRET}

            # Step 1: Upload the HTML file
            with open(html_path, "rb") as f:
                form = aiohttp.FormData()
                form.add_field("file", f, filename="document.html")
                async with session.post(
                    f"{HOST}/pdf-services/api/documents/upload",
                    data=form,
                    headers=auth,
                ) as resp:
                    if resp.status != 200:
                        return invoice_id, False
                    upload_result = await resp.json()
                    doc_id = upload_result["documentId"]

            # Step 2: Create the conversion task
            async with session.post(
                f"{HOST}/pdf-services/api/documents/create/pdf-from-html",
                json={"documentId": doc_id},
                headers={**auth, "Content-Type": "application/json"},
            ) as resp:
                if resp.status != 200:
                    return invoice_id, False
                task_result = await resp.json()
                task_id = task_result["taskId"]

            # Step 3: Poll for completion
            while True:
                async with session.get(
                    f"{HOST}/pdf-services/api/tasks/{task_id}",
                    headers={**auth, "Content-Type": "application/json"},
                ) as resp:
                    status = await resp.json()
                    if status["status"] == "COMPLETED":
                        result_doc_id = status["resultDocumentId"]
                        break
                    elif status["status"] == "FAILED":
                        print(f"Task failed for {invoice_id}")
                        return invoice_id, False
                await asyncio.sleep(5)

            # Step 4: Download the result
            async with session.get(
                f"{HOST}/pdf-services/api/documents/{result_doc_id}/download",
                headers=auth,
            ) as resp:
                if resp.status == 200:
                    pdf_bytes = await resp.read()
                    (output_dir / f"{invoice_id}.pdf").write_bytes(pdf_bytes)
                    return invoice_id, True
                return invoice_id, False

        except Exception as e:
            print(f"Error converting {invoice_id}: {e}")
            return invoice_id, False


async def batch_convert(invoices: list[dict], output_dir: str = "output") -> dict:
    output_path = Path(output_dir)
    output_path.mkdir(exist_ok=True)

    sem = asyncio.Semaphore(MAX_CONCURRENT)
    connector = aiohttp.TCPConnector(limit=MAX_CONCURRENT)

    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = [
            convert_one(session, sem, inv["id"], inv["html_path"], output_path)
            for inv in invoices
        ]
        results = await asyncio.gather(*tasks)

    succeeded = [r[0] for r in results if r[1]]
    failed = [r[0] for r in results if not r[1]]
    return {"succeeded": len(succeeded), "failed": failed}


# Usage
invoices = [
    {"id": "inv_1042", "html_path": "templates/invoice_1042.html"},
    {"id": "inv_1043", "html_path": "templates/invoice_1043.html"},
    # ... up to thousands of entries
]

result = asyncio.run(batch_convert(invoices))
print(f"Converted {result['succeeded']} PDFs. Failed: {result['failed']}")

asyncio and aiohttp let you process multiple conversions concurrently. The convert_one() function runs the full 4-step workflow (upload, create task, poll, download) for a single invoice, while batch_convert() dispatches all invoices in parallel, capped by a semaphore. Results are collected via asyncio.gather() and split into succeeded and failed lists.

Before running: Set FOXIT_API_HOST, FOXIT_CLIENT_ID, and FOXIT_CLIENT_SECRET as environment variables with your credentials from the Developer Dashboard. Adjust MAX_CONCURRENT based on your plan's rate limits, and update the invoices list with your actual file paths.

With MAX_CONCURRENT = 10 and several seconds per conversion (including polling), the batch processes 10 documents at a time. The semaphore prevents you from flooding the API with simultaneous requests and hitting the rate limit ceiling. asyncio is part of Python's standard library, so no additional dependencies beyond aiohttp are needed.

Credit consumption matters at scale. The Developer plan includes 500 credits/year. The Startup plan ($1,750/year) provides 3,500 credits. Each conversion typically costs 1 credit. For higher volumes, the Business plan ($4,500/year) includes 150,000 credits. Check your remaining credit balance via the Developer Dashboard before launching a large batch job.

For volumes beyond what a single process can handle efficiently, a queue-based architecture decouples submission from processing. Services like Amazon SQS or Redis Streams handle the message brokering:

App Server -> Message Queue (SQS / Redis Streams) -> Worker Pool (N workers)
  Worker: upload HTML -> create task -> poll -> download PDF -> store in S3/GCS
  Worker: update job status in Postgres / Redis

Each worker picks a job from the queue, runs the 4-step conversion workflow, writes the resulting PDF to S3 or GCS, and updates the job status in a database. This pattern handles burst volume naturally: jobs queue up during spikes, workers drain at the rate the API allows, and your app server is never blocked waiting for conversions to complete.

Production Deployment Patterns for HTML-to-PDF Pipelines

Error Handling and Retry Logic

Map HTTP status codes to decisions before writing any retry logic, because not all errors warrant a retry.

A 400 Bad Request means your request body is malformed. Retrying the same payload returns another 400, so fix the payload. A 429 Too Many Requests and a 503 Service Unavailable are transient: back off and retry. A FAILED task status means the conversion itself failed (possibly due to invalid HTML or unreachable URLs). Check the task response for diagnostic details.

import time
import random
import requests
from requests.exceptions import RequestException

PERMANENT_ERRORS = {400, 401, 403, 422}
TRANSIENT_ERRORS = {429, 500, 502, 503, 504}


def post_with_retry(
    url: str,
    max_retries: int = 4,
    base_delay: float = 1.0,
    **kwargs,
) -> requests.Response:
    """POST with exponential backoff and jitter for transient errors."""
    for attempt in range(max_retries + 1):
        try:
            response = requests.post(url, timeout=60, **kwargs)

            if response.status_code in range(200, 300):
                return response

            if response.status_code in PERMANENT_ERRORS:
                raise ValueError(
                    f"Permanent error {response.status_code}: {response.text}"
                )

            if response.status_code in TRANSIENT_ERRORS:
                if attempt == max_retries:
                    raise RuntimeError(
                        f"Max retries exceeded. Last status: {response.status_code}"
                    )
                delay = base_delay * (2 ** attempt) + random.uniform(0, 0.5)
                print(f"Transient error {response.status_code}. Retrying in {delay:.1f}s...")
                time.sleep(delay)

        except RequestException as e:
            if attempt == max_retries:
                raise
            delay = base_delay * (2 ** attempt) + random.uniform(0, 0.5)
            time.sleep(delay)

    raise RuntimeError("Unexpected: exhausted retries without returning or raising")

# Usage with the URL-to-PDF endpoint
auth_headers = {
    "client_id": CLIENT_ID,
    "client_secret": CLIENT_SECRET,
    "Content-Type": "application/json",
}

response = post_with_retry(
    f"{HOST}/pdf-services/api/documents/create/pdf-from-url",
    json={"url": "https://example.com/invoice/1042"},
    headers=auth_headers,
)
task_id = response.json()["taskId"]

Every POST request goes through a retry loop with exponential backoff. The function distinguishes between permanent errors (like 400 or 401, which you shouldn't retry) and transient errors (like 429 or 503, which resolve on their own). Each retry doubles the wait time and adds random jitter to avoid synchronized retry waves.

Before running: Replace CLIENT_ID, CLIENT_SECRET, and HOST with your Foxit credentials and API host, or load them from environment variables as shown in the earlier examples.

The jitter (random.uniform(0, 0.5)) prevents a thundering herd where every worker wakes up and retries simultaneously after a 429 burst. Plain exponential backoff produces synchronized retry waves when all workers hit the rate limit at the same time.

Output Optimization: Compression and Linearization

After conversion, you can chain additional PDF operations using the same async pattern. Upload the resulting PDF, call the compression or linearization endpoint, poll, and download the optimized version.

For PDFs served directly in a browser, linearization enables Fast Web View, which lets the browser display page one while the rest of the file downloads:

def compress_and_linearize(input_pdf_path: str, output_path: str) -> None:
    """Compress a PDF, then linearize it for fast web viewing."""
    auth = {"client_id": CLIENT_ID, "client_secret": CLIENT_SECRET}
    json_headers = {**auth, "Content-Type": "application/json"}

    # Upload the PDF
    doc_id = upload_document(input_pdf_path)

    # Compress
    resp = requests.post(
        f"{HOST}/pdf-services/api/documents/modify/pdf-compress",
        json={"documentId": doc_id, "compressionLevel": "MEDIUM"},
        headers=json_headers,
    )
    resp.raise_for_status()
    task = poll_task(resp.json()["taskId"])
    compressed_doc_id = task["resultDocumentId"]

    # Linearize the compressed result (no need to re-upload; use the resultDocumentId)
    resp = requests.post(
        f"{HOST}/pdf-services/api/documents/optimize/pdf-linearize",
        json={"documentId": compressed_doc_id},
        headers=json_headers,
    )
    resp.raise_for_status()
    task = poll_task(resp.json()["taskId"])

    # Download the final optimized PDF
    download_document(task["resultDocumentId"], output_path)

You chain two PDF operations back-to-back here. First, you upload the PDF and compress it at MEDIUM level (valid options are LOW, MEDIUM, and HIGH). Once compression completes, you pass the resultDocumentId directly into the linearization step, which avoids a second upload. The final download gives you a PDF that's both smaller and optimized for progressive loading in browsers.

Note: This function reuses upload_document(), poll_task(), and download_document() from the earlier examples. Make sure those functions are defined in the same script with your credentials configured. The Foxit developer blog post on chaining PDF actions covers this pattern in detail.

Monitoring and Secret Management

Monitor three things per conversion job: how long each call takes (to spot API degradation early), credits consumed per job type (to forecast when you'll hit your plan ceiling), and failure rate broken down by error code (to catch template regressions before customers do). Set an alert when your remaining credits fall below 20% of the plan allocation. The Foxit Developer Dashboard surfaces real-time usage data worth checking before kicking off large batch runs.

Store API credentials in environment variables or a secrets manager (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager). When team members leave or you suspect a credential leak, rotate them from the Developer Dashboard. New credentials can be generated and old ones revoked without downtime, as long as you update your environment before revoking.

Start Converting HTML to PDF

The Foxit Developer plan is free, requires no credit card, and gives you 500 credits to start with. Grab your client_id and client_secret from the Developer Dashboard, then either clone the demo repository for ready-made examples in Python, Node.js, and PHP, or drop the URL-to-PDF snippet from this guide into a script and run it against any public page.

Once your first conversion comes back, check credit usage in the Dashboard to project costs at your production volume. If you need more throughput, the Startup plan ($1,750/year for 3,500 credits) is self-serve with no sales call.

Get started for free on the Foxit Developer Portal

The 10 Best Next.js Starter Kits for SaaS in 2026

Lucien Chemaly — Wed, 15 Apr 2026 06:45:10 +0000

You've decided to build a SaaS product with Next.js. The smart move is starting from a production-ready starter kit instead of spending three months wiring up auth, payments, and a dashboard from scratch. The not-so-smart move is picking the wrong one and realizing it six weeks in.

There are dozens of Next.js boilerplates competing for your money right now. Some are genuinely excellent. Some are glorified create-next-app templates with a Stripe webhook bolted on. This comparison cuts through the noise with real technical details: actual tech stacks, actual pricing, actual feature coverage, and honest trade-offs.

I've built products on several of these and evaluated the rest against real-world SaaS requirements: authentication with RBAC, payment flexibility, database architecture, CMS capabilities, and how much code you'll actually need to write after checkout.

Here are the 10 best Next.js starter kits for SaaS in 2026, ranked.

1. RapidLaunch

Price: $99 (Starter) / $199 (Pro/Unlimited) - one-time, early bird pricing
URL: rapidlaun.ch

Tech Stack:

Next.js 16 with App Router
TypeScript (strict mode)
Supabase (PostgreSQL + Auth + Storage)
Stripe + LemonSqueezy (choose via env variable)
Tailwind CSS + DaisyUI
TanStack Query
TipTap rich text editor
Zod validation
Vitest + Playwright testing

RapidLaunch is the most feature-complete starter kit on this list. Where most boilerplates give you auth, payments, and a landing page, RapidLaunch ships a full drag-and-drop CMS with 20+ section templates, an AI-powered chat widget with lead capture, real-time analytics with a 3D interactive visitor globe, and an AI template generator that builds complete websites from a business description.

What sets it apart:

The CMS alone justifies the price. You get 25+ templates, 35+ customizable components, a visual navigation editor, footer editor, and a media library. Your non-technical co-founder can update the marketing site without touching code. That's a feature most teams don't get until they integrate a headless CMS as a separate service.

The AI chat widget runs on configurable OpenAI models (GPT-4, O3 Mini, and five others) with real-time streaming, conversation memory, and an intent detection system that automatically triggers a contact form when it detects purchase intent. That replaces tools like Intercom or Drift that run $50-$150/month.

The analytics dashboard tracks visitors in real-time with UTM attribution, browser fingerprinting, individual visitor session histories, and a WebGL globe showing geographic distribution. That's the kind of feature you'd pay $200+/month for from a standalone analytics tool.

The full feature set:

Authentication with RBAC (Owner + Admin roles)
Stripe and LemonSqueezy billing (subscriptions, one-time, guest checkout)
Automated webhook handling and two-way product sync
Customer portal with account, billing, and subscription management
Admin panel (CMS, users, billing, analytics, chat management)
Maintenance mode and Coming Soon mode (one-click toggle)
Cookie consent system
SEO optimization
15+ DaisyUI theme presets
52+ unit tests, 15 database migrations
CLAUDE.md file for AI-assisted development

Trade-offs: Supabase is the only database option. If you need raw PostgreSQL with a different hosting provider or want MongoDB, you'll need to adapt the data layer. Auth is email/password only out of the box (no social OAuth yet, though Supabase supports adding it). The project is newer than established competitors like ShipFast or Makerkit, so the community is still growing.

Verdict: If you want the most complete out-of-the-box experience (CMS + AI + analytics + auth + payments) at a price that undercuts most competitors, RapidLaunch is the strongest option available right now.

2. ShipFast

Price: $199 (Starter) / $249 (All-in) / $299 (Bundle with CodeFast) - one-time
URL: shipfa.st

Tech Stack:

Next.js 15
TypeScript or JavaScript
MongoDB or Supabase
Google OAuth + Magic Links
Stripe + LemonSqueezy
Tailwind CSS
Mailgun or Resend for email

ShipFast is the most popular Next.js boilerplate by sheer community size. Built by Marc Lou, it has a 5,000+ member Discord with revenue leaderboards and a culture of shipping fast. The average user launches in 7 days.

What you get: Auth, payments, email with DNS setup, SEO, a blog, pre-built UI components, and $1,210 in partner discounts. It's laser-focused on getting an MVP out the door with minimal friction.

Trade-offs: No CMS, no admin dashboard beyond basic user management, no analytics, no AI features. The database choice is MongoDB or Supabase only. If your SaaS grows beyond MVP stage, you'll be building significant infrastructure on top of what ShipFast provides. The community and shipping culture are genuinely valuable, but the actual codebase is thinner than competitors at the same price point.

Verdict: Best for solo founders who prioritize speed over feature completeness and value the community aspect. If you want to ship an MVP in a week and figure out the rest later, ShipFast delivers on that promise.

3. Supastarter

Price: $349 (Solo) / $799 (Startup, 5 seats) / $1,499 (Agency, 10 seats, white-label) - one-time
URL: supastarter.dev

Tech Stack:

Next.js, Nuxt, or SvelteKit (same features across frameworks)
TypeScript
Prisma or Drizzle ORM
better-auth (2FA, passkeys, RBAC)
Stripe, LemonSqueezy, Polar, Creem, Dodo Payments (5 providers)
Hono.js with oRPC
Tailwind CSS + Radix UI
TanStack Query
trigger.dev / QStash for background jobs

Supastarter is the enterprise option. Five payment providers, framework-agnostic (the same feature set works on Next.js, Nuxt, or SvelteKit), i18n baked in, team/organization management, and a super admin dashboard.

What sets it apart: The payment flexibility is unmatched. Five providers means you can support customers in markets where Stripe isn't available without rebuilding your billing layer. The Agency tier at $1,499 includes white-label licensing, which is the right call if you're building SaaS products for clients. It's also optimized for AI coding agents (Cursor and Claude Code ready).

Trade-offs: No built-in CMS (blog and docs only), no AI features, no analytics dashboard. The starting price at $349 is higher than most alternatives, and the Agency tier at $1,499 is the most expensive kit on this list. The breadth of options (three frameworks, two ORMs, five payment providers) means more decisions upfront. If you're a solo developer building one product, Supastarter may be more kit than you need.

Verdict: Best for teams and agencies that need multi-framework support, maximum payment flexibility, and white-label capabilities. Overkill for a solo founder building one product.

4. Makerkit

Price: $299 (Pro, individual) / $599 (Teams, 5 collaborators) - one-time, lifetime access
URL: makerkit.dev

Tech Stack:

Next.js 16, React 19
TypeScript 5
Supabase (with RLS), Drizzle, or Prisma (choose your stack)
OAuth + MFA (Google, GitHub, Facebook, X, Discord, TOTP)
Stripe Checkout + Customer Portal
Tailwind CSS v4 + Shadcn UI
Playwright for E2E testing

Makerkit has been around since 2022, making it one of the longest-running Next.js SaaS kits. It's focused on multi-tenant B2B SaaS with row-level security, RBAC, and team management built in from the start.

What sets it apart: The database flexibility is strong. You choose between Supabase with row-level security, Drizzle ORM with Better Auth, or Prisma 7 with Better Auth. Each stack is fully implemented, not just a config toggle. The Figma UI kit and React.Email templates are nice touches that save design time. Auth is the most complete of any kit here: email/password, magic links, social OAuth (five providers), and TOTP MFA.

Trade-offs: No CMS, no AI features, no built-in analytics. The $299 starting price is on the higher side for an individual license. The multiple stack options, while flexible, mean the documentation and codebase are more complex to navigate than a more opinionated kit.

Verdict: Best for B2B SaaS products that need multi-tenancy, strong auth, and row-level security from day one. The longest track record of any kit on this list.

5. Nextbase

Price: $99 (Essential) / $299 (Pro) / $399 (Ultimate) - one-time, lifetime access
URL: usenextbase.com

Tech Stack:

Next.js 15+
TypeScript
Supabase
Stripe
Tailwind CSS + Shadcn
Jest + Playwright
Sentry error tracking
PostHog + Google Analytics
OpenAI GPT-4

Nextbase takes a tiered approach. The $99 Essential tier covers auth, payments, multi-tenancy, admin panel, and MDX docs. The $299 Pro tier adds a blog, user feedback collection, roadmap functionality, changelog, and user impersonation. The $399 Ultimate tier unlocks everything.

What sets it apart: AI integration with GPT-4 is built in, and the monitoring stack (Sentry + PostHog) means you start with observability from day one instead of bolting it on later. The $99 entry price for a solid set of core features is competitive.

Trade-offs: Supabase only. Features are locked by tier, so you need to check exactly what you're getting at each price point. No CMS beyond the MDX blog. The community is smaller than ShipFast or Makerkit.

Verdict: Strong mid-range option with a genuinely useful tiered pricing model. The $99 Essential tier is one of the best value propositions for a production-ready starter.

6. LaunchFast

Price: $99 (single framework) / $149 (Astro + Next.js + SvelteKit bundle) - one-time
URL: launchfa.st

Tech Stack:

Next.js (also Astro and SvelteKit versions)
TypeScript
5+ database options (MongoDB, Firebase, PostgreSQL via Neon/Supabase/Xata, Redis, SQLite)
Flexible auth (email, magic links, OAuth)
Stripe + LemonSqueezy
Tailwind CSS
Resend, Postmark, SendGrid, Mailgun, AutoSend
S3, Cloudflare R2, Firebase, Supabase Storage

LaunchFast is the "bring your own stack" option. It supports more database providers, email services, file storage backends, and deployment targets than any other kit on this list.

What sets it apart: At $99 you get maximum flexibility. Swap databases, auth providers, payment processors, and deployment targets without rewriting your app. The $149 bundle gives you the same feature set across three frameworks. Deployment options include Vercel, Cloudflare Workers, Render, Fly.io, Netlify, and AWS Amplify.

Trade-offs: Less opinionated means more decisions and more wiring. No CMS, no AI features, no analytics dashboard. The documentation covers breadth over depth for each provider combination. If you want something that works out of the box with minimal configuration, this isn't it.

Verdict: Best for developers who have strong preferences about their stack and want a starter kit that adapts to them, not the other way around. The $99 price point is excellent for the flexibility offered.

7. Shipped.club

Price: $149 - one-time, lifetime access
URL: shipped.club

Tech Stack:

Next.js 14
TypeScript
Supabase
NextAuth + Supabase Auth
LemonSqueezy
ChakraUI + Tailwind CSS
MailChimp + Loops
MDX blog

Shipped.club is the budget option that doesn't feel like one. At $149 you get a complete starter with auth, payments, a landing page, waitlist page, pre-order page, and an affiliate program built in.

What sets it apart: The affiliate program is a feature no other kit on this list includes out of the box. Dual UI framework support (Chakra UI and Tailwind) gives you styling flexibility. PPP (Purchasing Power Parity) pricing makes it accessible globally. There's also a Chrome Extension version if that's your product format.

Trade-offs: Stuck on Next.js 14 (not the latest). LemonSqueezy is the only payment option (no Stripe). Supabase only. Less documentation and a smaller community than the top-tier competitors.

Verdict: Best bang for your buck if you're on a tight budget and LemonSqueezy fits your payment needs. The affiliate program inclusion is a smart differentiator.

8. Indie Starter

Price: $69 (Tier 1) / $199 (Tier 2) / $299 (Tier 3) - one-time, unlimited projects
URL: indie-starter.dev

Tech Stack:

Next.js (latest)
TypeScript
Supabase (PostgreSQL)
Magic links + social OAuth (Google, GitHub)
Stripe with webhooks
Tailwind CSS + Shadcn/ui
Sanity CMS
Resend for email
Umami + Google Analytics
Zod validation

Indie Starter stands out for its tiered pricing model starting at just $69. The Sanity CMS integration is a differentiator; most kits ship a basic MDX blog, but Indie Starter gives you a proper headless CMS for content management.

What sets it apart: The $69 entry tier makes it the cheapest paid option on this list. SEO infrastructure (sitemaps, robots.txt, Schema.org markup) is built in, not an afterthought. Sanity CMS integration means you get a real content management workflow, not just markdown files.

Trade-offs: Feature availability varies by tier. Supabase is the only database option. The community is smaller. Limited auth options compared to Makerkit or Supastarter.

Verdict: Best for indie hackers who want a solid foundation at minimal cost. The $69 tier is genuinely usable, not a stripped-down demo.

9. Nextless.js

Price: $699 (Single, 1 project) / $2,099 (Unlimited) - one-time + optional $199-$599/year renewal
URL: nextlessjs.com

Tech Stack:

Next.js 14
TypeScript
AWS Lambda (serverless)
PostgreSQL, MySQL, MongoDB, or DynamoDB
AWS Cognito (email, social login, MFA)
Stripe
AWS SES, SendGrid, Postmark, Mailgun, Mandrill
AWS CDK (Infrastructure as Code)
Tailwind CSS
Jest (unit, integration, E2E)

Nextless.js is the AWS-native option. If your company runs on AWS and needs serverless architecture with Infrastructure as Code, this is the only kit that delivers that out of the box.

What sets it apart: Full AWS CDK templates for infrastructure provisioning. Four database options in the Unlimited tier. Silo multi-tenancy model for proper data isolation. React Native boilerplate included for mobile support. This is the only kit on this list with a mobile-ready path.

Trade-offs: The most expensive option by far ($699-$2,099). Annual renewal for updates adds ongoing cost. AWS vendor lock-in is real. The serverless architecture has a learning curve if you're not already AWS-comfortable. Stuck on Next.js 14. No CMS, no AI features, no analytics.

Verdict: The right choice only if you're building on AWS and need serverless architecture with IaC. For everyone else, the price and complexity aren't justified.

10. Next.js SaaS Starter (by Vercel)

Price: Free (open source)
URL: github.com/nextjs/saas-starter

Tech Stack:

Next.js (latest)
TypeScript (93% of codebase)
PostgreSQL + Drizzle ORM
Email/password auth with JWTs in cookies
Stripe
Shadcn/ui
Zod validation
pnpm

The official Next.js SaaS starter from Vercel. It's intentionally minimal: auth, Stripe checkout, subscription management via Customer Portal, Server Actions with Zod validation, an activity logging system, and protected routes with middleware.

What sets it apart: It's free, it's from Vercel, and it uses the latest Next.js patterns. The codebase is clean and readable, making it an excellent learning resource. No bloat, no unnecessary abstractions. If you understand what you're building and just need a solid starting point, this does the job.

Trade-offs: Intentionally minimal means you'll add everything yourself: email service, landing pages, admin dashboard, CMS, AI integration, analytics, advanced auth (social OAuth, MFA), file storage. This is a starting point, not a finished product. If you factor in the hours to build what paid kits include, the "free" label is misleading.

Verdict: Best for experienced developers who want maximum control and are comfortable building features from scratch. Also the best learning resource for understanding Next.js SaaS architecture patterns.

Feature	RapidLaunch	ShipFast	Supastarter	Makerkit	Nextbase	LaunchFast	Shipped.club	Indie Starter	Nextless.js	Vercel Starter
Starting Price	$99	$199	$349	$299	$99	$99	$149	$69	$699	Free
Pricing Model	One-time	One-time	One-time	One-time	One-time	One-time	One-time	One-time	One-time + renewal	Free
Next.js Version	16	15	Latest	16	15+	Latest	14	Latest	14	Latest
TypeScript	Strict	Optional	Yes	Strict	Yes	Yes	Yes	Yes	Yes	Yes
Database	Supabase	MongoDB/Supabase	Prisma/Drizzle	Supabase/Drizzle/Prisma	Supabase	5+ options	Supabase	Supabase	4 options	PostgreSQL
Auth: Email/Password	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes (magic links)	Yes	Yes
Auth: Social OAuth	No (extensible)	Google	5+ providers	5 providers	Yes	20+ providers	Yes	Google, GitHub	Yes	No
Auth: MFA/2FA	No	No	Yes (passkeys)	TOTP MFA	No	No	No	No	Yes	No
Auth: RBAC	Yes	No	Yes	Yes (RLS)	Yes	No	No	No	Yes	No
Stripe	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
LemonSqueezy	Yes	Yes	Yes	No	No	Yes	Yes (only)	No	No	No
Other Payments	No	No	Polar, Creem, Dodo	No	No	No	No	No	No	No
CMS/Page Builder	Drag-and-drop (20+ sections)	No	No	No	No	No	No	Sanity CMS	No	No
Blog	Via CMS	Yes	Yes	Yes	MDX (Pro tier)	MDX	MDX	Sanity	No	No
Admin Dashboard	Yes (full)	Basic	Yes	Yes	Yes	No	No	No	Yes	No
AI Chat Widget	GPT-4 (7 models)	No	No	No	GPT-4	No	No	No	No	No
AI Site Generator	Yes	No	No	No	No	No	No	No	No	No
Built-in Analytics	Yes (3D globe, UTM, fingerprinting)	No	No	No	PostHog + GA	No	No	Umami + GA	No	No
Customer Portal	Yes	No	Yes	Yes	Yes	No	No	No	Yes	Yes (Stripe)
Email Integration	Yes	Mailgun/Resend	Multiple	React.Email	Resend	5 providers	MailChimp/Loops	Resend	5 providers	No
i18n	No	No	Yes	No	No	No	No	No	No	No
Multi-tenancy	No	No	Yes	Yes (RLS)	Yes	No	No	No	Yes (silo)	No
Testing Suite	Vitest + Playwright	No	Yes	Playwright	Jest + Playwright	No	No	No	Jest	No
Maintenance Mode	Yes	No	No	No	No	No	No	No	No	No
Affiliate Program	No	No	No	No	No	No	Yes	No	No	No
Mobile Support	No	No	No	No	No	No	No	No	React Native	No
Figma UI Kit	No	No	No	Yes	No	No	No	No	No	No
Unlimited Projects	Pro tier	Yes	Solo tier+	Pro tier	Yes	Yes	Yes	Yes	Unlimited tier	Yes
Community Size	Growing	8,000+	1,300+	Established	Growing	110+	330+	Small	Small	15.7K stars

Final Thoughts

The Next.js starter kit market has matured significantly. Every option on this list will save you time compared to starting from scratch. The difference comes down to how much time.

Kits like the Vercel SaaS Starter and LaunchFast give you a foundation and expect you to build. Kits like ShipFast and Shipped.club get you to MVP fast with the essentials. And kits like RapidLaunch, Supastarter, and Makerkit aim to give you a near-complete product from day one.

If I'm starting a new SaaS project today, my pick is RapidLaunch. The combination of a full CMS with drag-and-drop builder, AI-powered chat with lead capture, enterprise-grade analytics, and dual payment provider support (Stripe + LemonSqueezy) at $99-$199 one-time is the best value proposition in this space. You'd spend more than that in a single month on the standalone tools it replaces.

Get started with RapidLaunch and launch your SaaS in days, not months.

Backstage SaaS & Open Source Alternatives

Lucien Chemaly — Thu, 29 Jan 2026 07:26:39 +0000

Spotify's Backstage created the Internal Developer Portal category. It showed that a central hub for services, documentation, and tooling could actually improve how developers work. But here's what a lot of engineering leaders figure out the hard way: Backstage isn't right for every team.

If you're reading this, you've probably realized that adopting Backstage means committing serious engineering resources. You need a dedicated team, TypeScript expertise, and months of setup time. For a lot of teams, the operational overhead just isn't worth it.

But the problem Backstage solves doesn't go away. As your organization grows past about 150 people, you hit what's called the Dunbar Number Effect. Anthropologist Robin Dunbar found that humans can only maintain stable social networks of around 150 people. Go beyond that, and tribal knowledge disappears. Nobody knows who owns what. The informal Slack channels that worked when you were 30 people turn into complete chaos.

You need a system to fix this chaos, a "system of record" for your engineering organization. But you don't need the complexity of self-hosted Backstage to get there.

This guide covers the best Backstage alternatives in 2026. I'll break down three paths you can take, Build, Buy, or Hybrid, and help you figure out which one makes sense without drowning your platform team in maintenance work.

The Three Paths to an IDP: Build, Buy, or Hybrid

Before we look at specific tools, you need to understand the three approaches you can take:

Build (Self-Hosted Backstage): You take the open-source Backstage project and dedicate engineers to build, customize, and maintain your own portal. Ultimate flexibility, but significant headcount and operational costs.

Buy (Proprietary IDPs): You purchase a SaaS solution from a vendor like Cortex or Port. Quick to set up and feature-rich, but you're locked into their proprietary data model.

Hybrid (Managed Backstage): You use a service like Roadie that handles the hosting and maintenance of Backstage for you. You get the open-source ecosystem without the operational burden.

Backstage Alternatives: At a Glance

Here's a quick comparison of your options:

Tool	Core Technology	Hosting Model	Key Strength	Ideal For
Roadie	Backstage	SaaS (Managed)	Backstage ecosystem without the overhead	Teams who want Backstage's power but need a managed solution
Cortex	Proprietary	SaaS	Engineering metrics and scorecards	Organizations focused on measuring service quality
Port	Proprietary	SaaS	Developer-friendly API and flexibility	Teams building custom workflows
Atlassian Compass	Proprietary	SaaS	Deep Atlassian integration	Companies invested in the Atlassian stack
OpsLevel	Proprietary	SaaS	Service maturity and reliability checks	SRE teams enforcing production standards
Self-Hosted Backstage	Backstage (OSS)	Self-Hosted	Ultimate customization	Large orgs with dedicated platform teams (5+ engineers)

The Hybrid Approach: Managed Backstage

Roadie

Roadie isn't an alternative to Backstage, it's a different way to adopt it. The core idea is that you shouldn't have to choose between the power of an open-source community and the convenience of a SaaS product.

I've talked to a lot of teams who committed to self-hosting Backstage, only to realize it requires 3-12 engineers and 6-12 months to get production-ready. That's a significant investment. Roadie solves this by providing a secure, scalable, and fully managed Backstage experience out of the box.

Best for: Teams who've decided on Backstage but want to accelerate their timeline and reduce operational burden.

Key Features:

Get a production-ready Backstage instance running in minutes. Roadie handles upgrades, security, and maintenance.
Enterprise features like Role-Based Access Control, enterprise-grade search, and scorecards come built-in.
Install any open-source Backstage plugin without rebuilding your instance.

Considerations: Roadie uses Backstage as its foundation, so you get the same data model and core experience. If you want a completely different, highly opinionated UI, a proprietary vendor might fit better.

The "Buy" Approach: Proprietary IDPs

Cortex

Cortex has established itself as a leader in the IDP space. They focus heavily on service quality, reliability, and engineering metrics. Their Scorecards feature is particularly strong for defining standards and tracking adoption.

Best for: Organizations focused on establishing and measuring engineering standards.

Key Features: A central inventory for microservices and APIs, scorecards to track service health, and a scaffolder for creating new services from templates.

Considerations: Cortex is proprietary. You're locked into their data model, and migrating away later could be painful.

Port

Port is built around flexibility. Their developer-friendly API lets you ingest any data and build custom workflows. They position themselves as a platform for building a developer portal, not a rigid out-of-the-box solution.

Best for: Platform teams with strong dev skills who want to build highly custom experiences.

Key Features: A flexible "blueprint" model to define any asset, a self-service hub for custom actions, and scorecards for tracking quality.

Considerations: Port's flexibility is powerful but comes with a steeper learning curve. Expect more initial setup compared to more opinionated platforms.

Atlassian Compass

Atlassian Compass is Atlassian's entry into developer experience. Its main advantage is seamless integration with Jira, Confluence, and Bitbucket.

Best for: Companies already standardized on Atlassian tools.

Key Features: A component catalog for tracking ownership, health scorecards, and deep native integration with other Atlassian products.

Considerations: If you're not an Atlassian-centric organization, Compass may feel less compelling compared to other options.

OpsLevel

OpsLevel is a mature player focused on service ownership and reliability. SRE and platform teams like them because they help answer, "Is our software ready for production?"

Best for: SRE-driven organizations enforcing service maturity standards.

Key Features: A complete service catalog, an extensive library of automated maturity checks, and integrations with on-call tools like PagerDuty.

Considerations: OpsLevel's focus is more on reliability and standards than on developer self-service, which is stronger in other platforms.

The "Build" Approach: Self-Hosted Backstage

Choosing to self-host Backstage is a significant commitment. You should treat it like building an internal product, not just deploying a tool.

Best for: Large organizations with a well-funded platform team (5+ engineers) that has a clear mandate to build and maintain a customized developer portal.

Key Features: Complete control over the code and data model. You can customize it to your exact specifications.

Considerations: This path has a high cost of ownership. You need to account for the fully-loaded salaries of a dedicated engineering team, 6-12 months of initial build time, and ongoing operational burden for maintenance and upgrades.

How to Choose the Right Path

Your choice depends on your organization's priorities, resources, and philosophy. Here are the questions you should ask:

How important is the open-source ecosystem to us?

If you want to avoid vendor lock-in and tap into community innovation, choose between self-hosting Backstage or using a managed service like Roadie. If you prefer an all-in-one vendor experience, a proprietary option like Cortex or Port makes more sense.

What's the size and skill set of our platform team?

If you have 5-10 engineers with TypeScript experience and a mandate to build a custom portal, self-hosted Backstage is viable. If your platform team is smaller or focused on other priorities, Roadie or a proprietary vendor is more efficient.

What's our most critical problem right now?

If you need scorecards and don't mind vendor lock-in, tools like Cortex or OpsLevel offer polished solutions.

If you want to build custom workflows from scratch and are comfortable in a closed-source ecosystem, Port gives you a flexible API.

If your organization lives entirely in the Atlassian suite, Compass is a natural extension.

If you want enterprise features combined with the freedom of the open-source ecosystem, Roadie gives you both.

Final Thoughts

An Internal Developer Portal is a long-term investment in your developer experience. The choice between Build, Buy, and Hybrid depends entirely on your team's size, skills, and priorities.

I've seen teams succeed with all three approaches. The key is being honest about what you can realistically maintain and what problems you're actually trying to solve.

I'm curious what path you're evaluating. Are you leaning toward self-hosting Backstage? Considering a proprietary vendor? Looking at the hybrid approach with something like Roadie? What's the biggest factor driving your decision, team size, budget, or something else? Drop a comment and share what's working (or not working) in your evaluation process.

The "Glue Work" Trap: Why Your Best Engineer Looks Like Your Worst Performer

Lucien Chemaly — Wed, 28 Jan 2026 07:46:49 +0000

Your top engineer closed half the tickets the new grad did last quarter. The performance dashboard says they're underperforming. Your gut says the opposite.

I ran into this problem after moving from data engineering into management. One of my engineers spent three hours debugging why the nightly ETL job was timing out, which was blocking the entire team from running integration tests. They pair-programmed with our new hire to help them understand the data architecture. They wrote detailed documentation for the ETL migration because no one else would. They jumped into a Microsoft Teams thread at 9 PM to unblock a production deployment that couldn't wait until morning.

Then I pulled up the performance dashboard. They had closed half the tickets that our new grad closed that quarter.

The metrics said they were underperforming. My experience working with them said the opposite. They were the glue holding our team together.

This isn't a data engineering problem. It's an engineering management problem. Your best engineers spend their time on essential work that keeps the team functional, but none of it shows up in the systems that measure performance.

Why Your Metrics Are Lying to You

Jira and similar project management tools measure intent, not reality. They track what you planned to do, not what actually happened.

The problem compounds as engineers get more senior. Junior engineers can focus on tickets. Senior engineers spend most of their time on work that never gets a ticket: reviewing 40% of the team's PRs, helping others debug gnarly issues, fixing the CI pipeline that's been broken for weeks, sitting in architecture reviews to share context.

Creating a Jira ticket for "helped someone debug their environment setup" feels absurd. It took 20 minutes. But do that five times a week, and you've spent nearly two hours on invisible work. Multiply that across code reviews, production firefighting, mentoring conversations, and architectural guidance, and engineers can easily spend 50-70% of their time on work that has no corresponding ticket.

The result: visible output shrinks as actual value increases. The ticket dashboard shows your best engineer as underperforming while they're actually the team's MVP.

Tanya Reilly calls this "being glue", the technical leadership work that holds teams together but doesn't map cleanly to traditional promotion criteria.

Why Brag Documents Don't Work

The standard advice is to have engineers maintain brag documents. Julia Evans popularized this approach: engineers keep a running log of everything they do, then compile it for performance reviews.

This rarely works in practice.

It requires constant discipline. Engineers need to remember to update it after every contribution. Most forget until the week before their review, then try to reconstruct six months of work from memory.

It also feels self-aggrandizing. Engineers are trained to let their work speak for itself. Writing "I am great because I did X" feels uncomfortable, even when X is genuinely valuable.

And here's the real problem: without data backing it up, a brag document is just the engineer's word against the metrics. When you're under pressure to justify ratings, you'll default to the numbers.

Brag documents work for some people. But they put the burden on the wrong person. As a manager, you should have systems that capture your team's actual work, not force them to manually log it.

What the Data Actually Shows

Your engineers' work leaves digital traces everywhere. Every PR they review generates data in GitHub. Every commit they make has metadata. Every Teams conversation is timestamped. Every calendar event is logged.

This digital exhaust contains proof of the glue work. The challenge is assembling it into a coherent picture.

When an engineer reviews 127 PRs while their peers review 30, that's measurable. It explains why their ticket count is lower. They're multiplying everyone else's productivity.

Those 50-line PRs fixing type definitions, updating the Dockerfile, or adding error handling don't map to epics. But they're real work that keeps the system running.

When an engineer spends a week unblocking another team's API integration, that work might not even show up in your team's metrics. But it has real business impact.

The data exists. You just need to surface it.

How Tools Like Span Solve This Problem

I came across Span and I wish I'd known about them when I was dealing with the invisible work issue on my team.

The approach is different from traditional productivity tools. Instead of trying to make engineers track their work better, Span analyzes the actual work in your codebase and other systems. It looks at your code, PRs, Jira tickets, incident management tools, and calendars to understand what's really happening.

The platform uses AI to automatically categorize work into different types. That random Tuesday afternoon an engineer spends fixing the ETL timeout? Span classifies it as infrastructure work without them needing to create a ticket. The three hours spent reviewing PRs? That shows up in the data.

One feature that particularly addresses the glue work problem is called Investment Mix. According to one of their customers, Chad Bayer (VP of Technology at The Helper Bees), "Span's Investment Mix is the best on the market. It now powers our executive updates and board discussions, replacing what used to be a time-consuming manual process."

Investment Mix shows how engineering time is distributed across different categories of work. Instead of just counting tickets, you can see the actual breakdown between feature development, code reviews, infrastructure improvements, and maintenance work. That paints a completely different picture than "closed 8 tickets this quarter."

The platform also tracks metrics like PR cycle time and team velocity, helping you spot patterns in how work flows through your team. You can see if code reviews are a bottleneck or if infrastructure work is eating into feature development time.

The key thing is that it uses the actual work as the source of truth. It doesn't matter if an engineer created a ticket. If they made substantial changes to the infrastructure, that work gets classified and measured automatically.

How to Identify Glue Work in Your Team

You need to actively look for this. Your best engineers won't complain that their glue work is invisible. They're too busy doing the work.

Here are the patterns to watch for:

The Review Burden: Pull up your team's PR activity. If one engineer is reviewing 2-3x more PRs than everyone else, that's glue work. They're acting as a quality gate for the entire team.

The Unblocking Pattern: Watch for engineers who are constantly mentioned in other people's PRs or Teams/Slack threads. If someone's name keeps coming up when people need help, they're doing glue work.

The Infrastructure Tax: Some engineers volunteer to fix the annoying things that everyone complains about but no one wants to own. The flaky tests. The slow CI pipeline. The missing documentation. This work is nearly invisible but high-impact.

The Onboarding Load: If you have an engineer who every new hire gets paired with for their first few weeks, that's glue work. They're transferring institutional knowledge that doesn't exist anywhere else.

The Cross-Team Coordination: Watch for engineers who spend time in meetings with other teams or helping unblock dependencies. This work often doesn't even show up in your team's metrics, but it has real business impact.

How to Credit Glue Work in Performance Reviews

Once you identify the glue work, you need to explicitly credit it in performance reviews and compensation decisions.

Instead of apologizing for an engineer's ticket count, reframe the conversation around leverage and impact. Present the review with data, not gut feelings.

Show that they spent 35% of their time on developer enablement. Then show how that translated into team velocity improvements. If the team shipped 23% more features in the second half of the quarter compared to the first half, and the engineer's increased review load preceded that improvement, connect those dots.

Quantify specific contributions:

They reviewed 130 PRs that quarter, representing a significant portion of the team's total output. Their review feedback reduced the revision cycle time by an average of 8 hours per PR.
They spent 15% of their time on infrastructure improvements that cut the CI pipeline time from 45 minutes to 12 minutes, saving the team approximately 180 hours in aggregate wait time.
They pair-programmed with three junior engineers on complex features. All three shipped their work on schedule with minimal revision cycles.

The data doesn't just prove they were busy. It proves they were strategic about where they invested time.

When you present this to your director, the response will be immediate. The engineer gets promoted instead of getting a "meets expectations" rating.

How to Change Your Team's Incentives

Measuring glue work isn't enough. You need to actively reward it, or engineers will optimize for what gets measured.

Change how you run performance reviews. Explicitly evaluate engineers on three dimensions:

Individual Contribution: Traditional feature development and bug fixes. This is what Jira measures.

Team Multiplication: Code reviews, mentoring, infrastructure work, and knowledge sharing. This is what tools like Span measure.

Cross-Team Impact: Work that helps other teams or the company, even if it doesn't show up in your team's metrics.

Weight all three dimensions equally. An engineer who excels at team multiplication but has lower individual contribution numbers can still get an "exceeds expectations" rating.

Also change how you assign work. When planning sprints, explicitly allocate time for glue work. If an engineer is going to spend 30% of their time on code reviews, plan for them to close 30% fewer tickets. This prevents the trap where engineers get overloaded with both feature work and glue work, then get dinged for not completing enough tickets.

The Long-Term Impact

Six months after implementing these changes, the results are clear.

Team velocity increases by 30%+. Not because people work more hours, but because the glue work gets distributed more evenly. Junior engineers close more tickets because they get unblocked faster. Mid-level engineers contribute to code reviews and documentation. Senior engineers still do the most glue work, but they're not penalized for it.

Retention improves. Engineers who were considering leaving because they felt like their contributions weren't valued become your strongest advocates and refer excellent engineers to your team.

Hiring gets easier. When candidates ask how you evaluate performance, you can show them the actual data on how you measure and reward different types of contributions. Engineers who care about doing high-quality work are attracted to teams that recognize all forms of contribution, not just ticket velocity.

What You Should Do Tomorrow

Start by identifying who on your team is doing the most glue work. Pull up your GitHub data and look at PR review patterns. Talk to your junior engineers about who they go to when they're stuck.

Then look at how you're measuring performance. If you're only looking at ticket velocity, you're missing half the picture. Find ways to quantify the other half, whether that's through manual data gathering or tools like Span.

Finally, have explicit conversations with your team about glue work. Tell them you value it. Show them you're measuring it. Make it clear that doing glue work won't hurt their career trajectory.

Glue work is career-making if it's visible. It's career-ending if it stays invisible. As a manager, making it visible is your job, not your engineers'.

The engineers who succeed at senior levels aren't the ones who close the most tickets. They're the ones who multiply their team's effectiveness. Code review, mentoring, infrastructure work, and technical guidance are how they create that leverage.

But leverage without measurement looks like low productivity. You need systems that capture the full picture of contributions, not just the subset that maps cleanly to ticket boards.

If you have engineers spending time on glue work, make sure it's being captured and credited. They shouldn't have to choose between doing high-impact work and having a successful performance review.

How do you measure glue work on your team? I'd love to hear what's worked (or hasn't worked) for other engineering managers. Drop a comment below.

10 Proven Ways to Improve Developer Productivity (Without Burning Out Your Team)

Lucien Chemaly — Tue, 13 Jan 2026 14:49:12 +0000

Engineering leaders face relentless pressure to ship faster. The tempting response is to push harder, add hours, and demand more. But this approach fails. The 2025 DORA State of AI-Assisted Software Development report, surveying nearly 5,000 technology professionals, reveals that 90% now use AI tools at work, spending a median of two hours daily with them. Yet only 24% trust the output. More telling: extended crunch periods create an almost perfect correlation between overtime and defect rates.

The path to sustainable velocity isn't raw effort. It's removing friction, protecting focus, and building systems that let your engineers do their best work. Here are ten strategies backed by recent research that actually move the needle.

1. Protect focus time by clustering interruptions

The cost of context switching is staggering. Dr. Gloria Mark's research at UC Irvine established that workers need an average of 23 minutes and 15 seconds to fully refocus after an interruption. Her 2023 book Attention Span updated this to roughly 25 minutes and revealed something worse: the average time spent on any screen before switching has dropped to just 47 seconds.

For developers, the damage compounds. GitHub's Good Day Project found that developers have an 82% chance of having a good day with minimal interruptions, but only a 7% chance when interrupted frequently. Paul Graham captured this perfectly in his 2009 essay on the maker's schedule: "A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in."

Action: Implement no-meeting days or cluster all meetings into specific blocks. A 2022 MIT Sloan study of 76 companies found that one meeting-free day per week increased productivity by 35%. Two meeting-free days pushed that to 71%. The optimal balance was three no-meeting days per week, protecting 60% of the work week for focused work.

2. Cut CI/CD wait times to eliminate forced context switches

When a build takes 20 minutes, developers switch to something else. Then they need another 25 minutes to regain context when the build completes. That single build just cost 45 minutes of cognitive overhead.

The 2025 DORA report shows that AI adoption increased throughput but also instability, with teams shipping faster but experiencing higher change failure rates when they lack robust control systems. The difference isn't just speed, it's that fast feedback loops let developers stay in flow.

Action: Invest in incremental builds, parallelized test suites, and intelligent test impact analysis. AI-driven test selection can reduce CI run times by 40-80% by running only tests affected by recent changes. Kent Beck's original 10-minute rule from Extreme Programming Explained: "Automatically build the whole system and run all of the tests in ten minutes. A build that takes longer than ten minutes will be used much less often, missing the opportunity for feedback." Every minute beyond that threshold increases the likelihood of costly context switches.

3. Treat internal documentation as a product

The 2024 Stack Overflow Developer Survey found that 61% of developers spend more than 30 minutes daily searching for answers, with 26% spending over an hour. That's one entire workday per week burned on hunting for answers that should be documented.

The impact on teams is severe. New hires take two to three months longer to become productive when documentation is poor. The 2024 Stack Overflow survey also found that while 70% of developers know where to find answers, only 56% can find them quickly, and 53% report that waiting on answers disrupts their workflow.

Action: Allocate dedicated time for documentation maintenance. Treat your internal wikis like a product with owners, roadmaps, and quality standards. Organizations with poor documentation waste hundreds of hours weekly for mid-sized engineering teams, equivalent to losing multiple full-time engineers to information hunting.

4. Deploy AI coding tools, but measure what matters

GitHub Copilot has crossed 20 million users, with 90% of Fortune 100 companies now using it. GitHub's controlled study showed developers completed tasks 55% faster with Copilot, and Duolingo's engineering team estimated 25% speed increases for engineers new to a codebase.

But the data isn't universally positive. A December 2025 CodeRabbit analysis of 470 open-source PRs found AI-generated code produces 1.7x more issues than human-written code, including 1.4x more critical bugs. The 2024 DORA report noted that while AI adoption increased documentation quality by 7.5% and code review speed by 3.1%, it was also associated with higher instability. Meanwhile, only about 33% of developers trust AI code accuracy according to the 2025 Stack Overflow survey, with 46% actively distrusting AI-generated code.

The real risk is "AI sprawl," where teams adopt multiple tools without understanding impact.

Action: Roll out AI assistants to reduce boilerplate and repetitive tasks, where they excel. But measure outcomes, not just adoption. Platforms like Span help leaders see if AI is reducing coding time or just increasing review time, allowing you to optimize the toolchain without guessing. Span's AI code detector can identify AI-generated code with 95% accuracy, giving you ground truth on adoption and impact. The goal is reduced toil, not just more code.

5. Standardize the inner development loop

"It works on my machine" remains one of the most expensive phrases in engineering. Each developer maintaining their own unique environment creates configuration drift, onboarding delays, and debugging nightmares.

Cloud development environments (CDEs) and dev containers eliminate this friction. Teams using mature internal developer platforms report significant reductions in cognitive load and faster onboarding. Spotify's platform engineering team found a 55% improvement in time-to-tenth-pull-request for new developers using standardized environments.

Action: Standardize development environments using containers, devcontainers, or cloud-based development tools. Define the inner loop (edit, build, test, debug) as a first-class product. When any developer can clone a repo and start contributing within hours instead of days, you've removed one of the most persistent friction points in engineering.

6. Set explicit code review SLAs

Code sitting in review is inventory on a shelf. It's not delivering value, it's accumulating merge conflicts, and the author has already context-switched away. Meta's engineering research found a correlation between time-in-review and engineer dissatisfaction.

High-performing teams treat review latency as a key metric. Google's engineering practices mandate responding to code reviews within one business day maximum.

Action: Establish team SLAs for review turnaround. A reasonable target: first review within four hours for internal team PRs, full cycle time under 24 hours. Keep PRs small (under 400 lines) for higher defect discovery rates. Research shows that reducing the time between acceptance and merge can improve code velocity by up to 63%.

7. Create on-call rotations to protect maker time

The "tap on the shoulder" through Slack is insidious. It feels harmless, but it fragments focus and distributes interruptions unpredictably across the team. Shadow work, the unplanned requests and ad-hoc support that never appears on a sprint board, silently drains capacity. When everyone is "available," no one is protected.

Action: Implement explicit on-call rotations where designated engineers handle interruptions while others focus. During on-call workdays, engineers do only on-call work with no feature development expected. If your on-call engineers regularly handle more than 2 incidents per 12-hour shift, Google SRE data suggests your system has reliability problems that need addressing - either through better automation, improved monitoring, or expanding the rotation. This protects the rest of the team's time while ensuring responsive support.

8. Build self-service infrastructure through platform engineering

Filing tickets for AWS access, waiting for DevOps to configure a database, or requesting permission to deploy are all symptoms of underdeveloped platform capabilities. Each handoff adds days of latency and forces developers into waiting mode.

Gartner predicts that by 2026, 80% of large software engineering organizations will have platform engineering teams, up from 45% in 2022. A 2024 survey of Kubernetes experts found 96% of organizations already have a platform engineering function, and the 2024 DORA report showed teams using internal developer platforms saw 10% increases in team performance.

Action: Build "golden paths" for common workflows: deploying a new service, provisioning a database, setting up monitoring. Spotify pioneered this approach, creating opinionated, well-documented pathways that reduce cognitive load while still allowing teams to deviate when needed.

9. Measure systems and processes, not individual output

Stack-ranking developers by commits or lines of code pushes them to game the numbers instead of doing good work, and it destroys psychological safety. The 2024 Stack Overflow survey found that only 20% of developers report being happy at work, with 62% citing technical debt as their top frustration. The DORA research team explicitly warns against using their metrics for individual evaluation—these metrics exist to identify systemic bottlenecks, not to rank people.

Action: Use metrics to debug your engineering system, not your engineers. Look for process flaws: calendar fragmentation, review bottlenecks, deployment friction, documentation gaps. Span's platform is valuable here because it highlights systemic issues, like calendars full of fragmented time, rather than simply counting lines of code. This preserves psychological safety while surfacing real obstacles.

10. Regularly survey developers and act on feedback

Happy developers write better code. The DORA Accelerate research found that high-performing teams are 2x as likely to exceed organizational performance goals. Google's 2022 research established that perceived code quality causally increases productivity. Yet most organizations never systematically ask their engineers what's getting in the way.

Productivity cannot be reduced to a single metric. You need satisfaction data alongside performance data. And you need to actually fix what developers say is broken.

Action: Run quarterly developer experience surveys. Ask what sucks. Then visibly address the top pain points. The investment pays compound returns.

Start with two changes this quarter

You don't need to implement all ten strategies at once. Pick two that address your team's biggest friction points. If your calendar is a disaster, start with meeting-free days. If code rots in review, set explicit SLAs. If developers waste hours searching for answers, invest in documentation.

The goal isn't maximum velocity for one sprint. It's sustainable pace that your team can maintain indefinitely. The 2025 DORA data is clear: AI functions as an amplifier, magnifying the strengths of high-performing teams and the dysfunctions of struggling ones. Teams that prioritize stability and developer well-being outperform those that sacrifice them for short-term speed. Build the system that lets your engineers do their best work, and the velocity will follow.

The 7 Best Developer Portals for Enterprise Teams in 2025

Lucien Chemaly — Thu, 08 Jan 2026 09:07:29 +0000

Your platform team spent a year and $2 million building an internal developer portal. Or maybe you bought one off the shelf. Either way, you're probably wondering if you made the right choice.

Here's what I've learned after working with dozens of companies implementing internal developer portals: the "build versus buy" decision isn't really the question anymore. The market has changed completely since 2020. Companies that went with self-hosted Backstage discovered that "free and open-source" actually means paying for 3-12 full-time engineers. Organizations that bought proprietary platforms found themselves locked into data models they can't escape. And the ones who picked wrong are now facing painful migrations.

I've spent the last few years talking to engineering leaders about their IDP implementations, and I've noticed the landscape has evolved into three distinct approaches:

Build: Self-hosted Backstage installations that give you maximum flexibility but cost over $1M per year to operate.

Buy: Proprietary SaaS platforms like Cortex and Port with polished interfaces but permanent vendor lock-in.

Hybrid: Managed Backstage solutions that give you the open-source ecosystem without the engineering overhead.

In this guide, I'll walk through the seven platforms that enterprise teams are actually using at scale. I'm focusing on the strategic trade-offs that matter three years after your initial decision, when you'll discover whether you made the right choice or not.

What Actually Matters in an Enterprise Developer Portal

Before I dive into specific platforms, let me share what I've learned matters most when evaluating IDPs. A lot of these platforms look great in demos but fall apart in production.

Ecosystem and Extensibility

Can you integrate with your entire toolchain, or are you stuck with whatever the vendor decides to support? At enterprise scale, you're probably using 50+ different tools across CI/CD, monitoring, security, and cloud infrastructure. The difference between supporting 20 integrations versus 250+ becomes critical when half your value comes from centralizing everything in one place.

Vendor Lock-In Risk

What happens to your data if you need to switch platforms in two years? Open-source-based solutions like Backstage use standardized YAML entity definitions you can export and migrate anywhere. Proprietary platforms often use custom data models that trap all your organizational knowledge inside their systems forever.

Maintenance Overhead

Does running the platform require a dedicated team, or can your existing platform engineers manage it alongside their other work? I've seen self-hosted solutions consume 3-5 full-time engineers just for maintenance, upgrades, and troubleshooting. This is what I call the "TypeScript tax," the hidden cost of maintaining frontend infrastructure that most DevOps teams aren't equipped to handle.

Enterprise Readiness

Does the platform provide role-based access control (RBAC), single sign-on (SSO), and SOC2 compliance out of the box? For regulated industries or companies with strict security requirements, these aren't nice-to-haves. They're table stakes. Building them yourself in a self-hosted environment can take months.

Day 2 Operations

Look beyond the initial setup. How difficult is it to upgrade when breaking changes occur? How do you handle search infrastructure at scale? What happens when you need to migrate to a new backend system? The platforms that look easiest on day one often become maintenance nightmares by day 700.

1. Roadie: Managed Backstage for Teams Who Value Their Time

Category: Hybrid (Managed Backstage)

Best For: Teams that want Backstage's ecosystem without the operational burden

Roadie delivers the full power of Spotify's open-source Backstage platform as a managed SaaS service. This is what I call the hybrid approach. You get access to the entire Backstage ecosystem with 211 open-source plugins, standardized data models, and active community development. But you don't need to maintain any of the infrastructure.

The platform handles everything you'd normally spend engineering time on: hosting, security patches, database management, enterprise-grade search, and those painful upgrades like the New Backend System migration that challenged self-hosted teams throughout 2024. Your team focuses on configuring integrations and building workflows, not debugging TypeScript or updating React components.

Key Features:

Minimal-maintenance Backstage platform with automated upgrades
Access to entire open-source plugin ecosystem (211 plugins, 82 supported out-of-the-box)
Built-in Tech Insights for scorecards and engineering standards (paid add-on)
Enterprise RBAC (basic RBAC in Teams plan, custom RBAC in Growth plan)
No vendor lock-in since the data model is standard Backstage YAML

What I Like:

The biggest advantage is what I call avoiding the TypeScript tax. Platform engineering teams can focus on building platform capabilities instead of maintaining TypeScript and React frontends. Any Backstage plugin works, including community-developed integrations. Complex upgrades like the New Backend System transition happen automatically. Most customers see value within weeks, not months. Your catalog definitions are portable YAML files, not proprietary formats.

The Trade-offs:

You're working within Backstage's UI constraints, which means less drag-and-drop flexibility compared to tools like Port. It's more structured, which some teams see as a feature and others see as a limitation. While Roadie offers secure connectivity options like the Roadie Broker for on-premises resources, it's primarily a hosted service. If your security team absolutely requires on-premises deployment, self-hosted Backstage might be your only option.

Pricing: Teams plan starts at $24 per developer per month with a 50-seat minimum (50-150 developers). Growth plan pricing is custom with a 100-seat minimum (100+ developers). Only active contributors to your source control management incur costs. Non-coding team members like product managers and leadership can access for free. Tech Insights is an optional paid add-on. View pricing details.

2. Cortex: When You're Obsessed with Scorecards

Category: Proprietary SaaS

Best For: Organizations focused on service maturity scorecards and reliability metrics

Cortex built a polished, opinionated developer portal that's heavily focused on measuring and improving service quality through scorecards. The platform excels at gamifying service ownership with detailed maturity models, SLO tracking, and automated scoring based on engineering best practices using Bronze/Silver/Gold levels.

The UI feels modern and intuitive, especially if your team is already familiar with SaaS tools like Datadog or PagerDuty. Cortex's scorecard system is more sophisticated than most alternatives, offering fine-grained control over scoring criteria with flexible rule definitions and excellent visualization of how your services stack up against engineering standards.

Key Features:

Advanced scorecard system with customizable rubrics using Bronze/Silver/Gold levels
Strong reliability engineering focus with SLOs, incidents, and on-call tracking
Polished, modern UI optimized for service discovery
AI-powered features including Ownership Prediction and Velocity Dashboard for DORA metrics
60+ out-of-the-box integrations with major monitoring and development tools

What I Like:

The scorecard sophistication is genuinely best-in-class. If you want to track service maturity with Bronze/Silver/Gold level visualization and detailed point-based scoring, nothing else comes close. The UI is beautiful and impresses stakeholders. The reliability focus is excellent for teams prioritizing SRE practices. You can get a basic catalog running quickly. New AI features for ownership prediction and metrics analysis are interesting additions.

The Trade-offs:

Cortex is known for being expensive at enterprise scale, especially compared to Backstage-based alternatives. The data model is Cortex-specific, making migration difficult if you ever want to leave. You're limited to whatever integrations Cortex builds. You can't leverage community plugins like you can with Backstage. The template and workflow capabilities lag behind Backstage's Software Templates.

Pricing: Not publicly disclosed. You need to sign up for a demo to get pricing information. However, the Forrester Total Economic Impact study from July 2024 lists pricing at approximately $65 per user per month at scale. Multiple tiers available (Engineering Intelligence, Accelerate, Full IDP, Site License) with features scaling from basic catalog and scorecards to full platform capabilities. Request pricing.

3. Port: The Builder's Platform

Category: Proprietary SaaS

Best For: Teams that need to model non-standard assets or want maximum UI customization

Port takes a completely different approach. Instead of giving you an opinionated developer portal, it gives you building blocks to create your own. The platform's no-code interface lets you define custom data models (called Blueprints) for any asset type, not just services and APIs, but environments, IoT devices, or cloud resources.

This flexibility makes Port uniquely suited for organizations with complex, non-standard infrastructure that doesn't fit typical service catalog patterns. You can build custom views, define relationships between any entity types, and create workflows that match your exact processes. Port recently rebranded as an "Agentic Internal Developer Portal" with enhanced AI capabilities.

Key Features:

Fully customizable data models and UI views via Blueprints
No-code interface for defining entities and relationships
Strong visualization capabilities for complex systems
Self-service actions using Cookiecutter templates
50+ integrations including DORA metrics tracking
AI agent capabilities and Engineering360 dashboard

What I Like:

Port gives you ultimate flexibility to model anything your organization needs, not just traditional services. You can build exactly the interface your teams need. It's excellent for multi-cloud, hybrid environments with diverse asset types. The visual workflow builder lets you create automation without writing code.

The Trade-offs:

Maximum flexibility means you're building everything from scratch. There's less out-of-the-box value compared to opinionated platforms. You can't leverage the Backstage open-source plugin ecosystem, though the Ocean Framework provides extensibility through data integrations and workflow automation. While Port supports Markdown, it lacks the full TechDocs build pipeline and search capabilities you get with Backstage. Teams need time to master the data modeling concepts, which means a steeper learning curve.

Pricing: Free tier available (up to 15 seats, 10,000 entities). Startup tier at $30 per developer per month. Enterprise tier available with premium features including SSO, advanced RBAC, ISO 27001 and SOC2 Type 2 certifications, and dedicated support. View pricing details.

4. OpsLevel: Fast Setup, Limited Growth

Category: Proprietary SaaS

Best For: Teams focused primarily on service ownership and maturity tracking

OpsLevel started as a service maturity and ownership tool before evolving into a broader developer portal. This shows in its excellent service ownership features and straightforward approach to tracking engineering standards through its Rubric system with Bronze/Silver/Gold levels, plus separate Scorecards for team-specific standards.

The platform offers the fastest time-to-initial-value for basic service cataloging. Typical deployments complete in 30-45 days. You can have a working catalog with ownership information and basic checks running within hours. But this simplicity comes at a cost. OpsLevel's feature set is more constrained than platforms built on extensible architectures. Recent additions include AI-powered features for check generation and catalog enrichment.

Key Features:

Fast setup for basic service catalog (typical 30-45 day deployment)
Strong focus on service ownership and maturity rubrics
Good integration with CI/CD systems for automated checks (60+ integrations)
AI-generated checks and AI-enriched catalog
Package version inventories for SBOM visibility
Clean, straightforward UI

What I Like:

You can get a basic catalog running faster than with any alternative. The service ownership focus is excellent for clarifying who owns what. Good built-in templates for measuring service quality. The interface is less complex than more feature-rich platforms, which some teams prefer. New AI features streamline catalog management.

The Trade-offs:

You can't easily add community plugins or build custom integrations. The UI is less flexible than Port or Backstage for customization. The feature set is primarily focused on cataloging and checks, with less emphasis on documentation or scaffolding. The migration path is unclear if you outgrow the platform since it uses a proprietary data model.

Pricing: Not publicly disclosed. Pricing is based on team size with custom quotes. Per-developer pricing model with volume discounts available. Includes SOC2 Type 2 compliance and SAML-based SSO. Pricing customizable based on needs including self-hosted options and support levels. Request pricing.

5. Atlassian Compass: The Jira Extension

Category: Proprietary SaaS

Best For: Organizations deeply invested in the Atlassian ecosystem

If your company runs on Jira, Bitbucket, and Confluence, Compass offers the most seamless native integration you'll find. The platform leverages Atlassian's identity system, pulls in data from other Atlassian products automatically, and feels like a natural extension of your existing toolchain.

Compass provides automated service health monitoring, tracking metrics from integrated tools and surfacing problems before they escalate. For teams already paying for Atlassian products, Compass represents an incremental cost with minimal integration effort. The platform has scorecards with a new Maturity Levels feature added in 2025.

Important Note: Atlassian deprecated the Templates and scaffolding feature on December 1, 2025. This is a significant capability reduction for teams requiring self-service service creation.

Key Features:

Native integration with Jira, Bitbucket, Confluence, and Opsgenie
Automated service health monitoring
Component tracking with Atlassian-native data models
Integrated incident management through Opsgenie
Scorecards with Maturity Levels feature
Built on Atlassian Forge with GraphQL APIs for extensibility

What I Like:

The Atlassian integration is unbeatable if you use Jira for everything. The UX feels consistent with other Atlassian products, so there's minimal learning curve. Automated health monitoring works well. You can get it as a standalone product or bundled with some enterprise packages.

The Trade-offs:

This is what I call the Atlassian trap. The platform struggles with non-Atlassian tools like GitHub, GitLab, or CircleCI. The scaffolding feature was removed on December 1, 2025, so it's no longer available for service creation. You can't extend it with community plugins since it uses a proprietary ecosystem. It's more of a service catalog with add-ons than a complete platform interface.

Pricing: Free tier available (3 full users, unlimited basic users). Standard tier at $8 per user per month includes basic features. Premium tier at $25 per user per month includes IP Allowlisting, advanced integrations, 99.9% uptime SLA, and premium support. Discounted rates available for teams above 101 users. Compass is a standalone product with separate billing from other Atlassian tools. View pricing details.

6. Backstage: The Industry Standard (Self-Hosted)

Category: Open Source (Self-Hosted)

Best For: Large enterprises with dedicated platform engineering teams and specific compliance requirements

Spotify's Backstage is the industry-standard open-source developer portal framework. It powers IDPs at Spotify, American Airlines, Pinterest, and thousands of other organizations. You own the code, control the infrastructure, and can customize anything.

But this flexibility comes with significant operational costs. Self-hosting Backstage requires 3-5 dedicated engineers to manage infrastructure, handle upgrades, maintain search systems, and keep up with the rapidly evolving codebase. Roadie's survey of the Backstage community found that successful self-hosted deployments had at least three engineers dedicated full-time. Some companies have teams of 12 people just for Backstage.

Breaking changes occur regularly with monthly releases. Major migrations like the New Backend System transition that completed in 2024 consumed months of engineering time for self-hosted teams. For perspective on the true cost, Zalando invested over $4 million across four years developing their internal platform before open-sourcing their work as part of Backstage.

Key Features:

Fully open-source with Apache 2.0 license
250+ community plugins covering every major tool
Extensible architecture for custom plugins and integrations
Active community and regular monthly releases
CNCF Incubating project with strong enterprise adoption

What I Like:

You have ultimate control and can customize without limits. There are no license costs since it's free to download and use. The ecosystem is massive with the largest community and plugin library (250+ plugins). You have no vendor dependency and can run it anywhere, modify anything. It's the industry standard backed by CNCF and major enterprises.

The Trade-offs:

The operational overhead is substantial. Successful deployments require 3-5 full-time engineers minimum. Some teams reach 12 full-time employees just maintaining Backstage. Most DevOps teams lack frontend skills for React and TypeScript customization, which is what I call the TypeScript tax. Monthly breaking changes and major migrations like the New Backend System require significant engineering investment.

You manage databases, search infrastructure with Elasticsearch, monitoring, and security patches. The hidden costs are enormous. At typical senior platform engineer compensation of $250K per year fully loaded, 3-5 engineers cost $750K to $1.25M annually, plus infrastructure costs of $12K to $24K per year. Total cost of ownership typically exceeds $2M+ over three years when you factor in engineering time, opportunity cost, and infrastructure.

Pricing: Free (open source under Apache 2.0 license). However, total cost of ownership includes 3-5 engineer salaries ($750K to $1.25M+ annually) plus infrastructure costs ($12K to $24K+ annually). TCO typically exceeds $2M+ over three years.

7. Configure8: The Discovery Platform

Category: Proprietary SaaS

Best For: Organizations prioritizing discovery and cost analytics

Configure8 positions itself as a universal catalog that can ingest and relate data from virtually any source. The platform emphasizes discovery features, helping teams understand what they have and how it's interconnected. It also offers strong cloud cost integration, surfacing spending data alongside technical resources.

While Configure8 has solid core features including 30+ integrations and workflow-based Self-Serve Actions, its smaller market presence and proprietary nature make it a riskier choice than platforms with larger ecosystems or open-source foundations.

Key Features:

Universal catalog supporting diverse asset types
Strong discovery and search capabilities
Cloud cost analytics integration
Relationship mapping across resources
Workflow-based Self-Serve Actions
Available as SaaS or on-premises deployment

What I Like:

It's good at helping teams understand what exists in their infrastructure. The cost integration is unique, combining cloud spending data with technical resources. It can pull data from many systems. You get deployment flexibility with both SaaS and on-premises options.

The Trade-offs:

The ecosystem is smaller with less community support and fewer integrations (30+) than larger platforms. The data model is Configure8-specific, which means proprietary lock-in. There are fewer public case studies and enterprise deployments than alternatives. As a smaller player in a competitive market, there's some uncertainty about long-term viability.

Pricing: Free tier available (up to 10 users for scorecards). Paid tiers available with SOC2 certification and RBAC features. Enterprise pricing available with additional features and volume discounts. Available as both SaaS and on-premises deployment. Contact Configure8 for detailed pricing. View pricing page.

Quick Comparison

Platform	Foundation	Maintenance	Ecosystem Size	Lock-In Risk	Enterprise Features
Roadie	Open Source (Backstage)	Minimal (Managed)	211 plugins	Low (standard YAML)	RBAC, SSO, SOC2 Day 1
Cortex	Proprietary	Minimal (SaaS)	60+ integrations	High (proprietary)	Strong
Port	Proprietary	Minimal (SaaS)	50+ integrations	High (proprietary)	Good
OpsLevel	Proprietary	Minimal (SaaS)	60+ integrations	High (proprietary)	Basic
Compass	Proprietary	Minimal (SaaS)	Atlassian-centric	High (proprietary)	Good (if Atlassian)
Backstage	Open Source	High (3-12 engineers)	250+ plugins	None	DIY
Configure8	Proprietary	Minimal (SaaS)	30+ integrations	High (proprietary)	Moderate

Making Your Choice

The right IDP depends on your organization's constraints, technical culture, and platform engineering maturity. Here's how I'd think about it:

Choose Self-Hosted Backstage if you have a team of 10+ platform engineers, unlimited budget, and specific requirements that absolutely can't be met by managed solutions. You're willing to invest significant engineering time in maintenance and customization for maximum control. Be prepared for 3-12 dedicated engineers and $2M+ total cost of ownership over three years.

Choose Cortex or Port if you value polished UI above everything else, don't mind proprietary lock-in, and want specific workflow capabilities their platforms emphasize. Budget for potentially higher costs at scale (around $65 per user per month for Cortex, around $30+ per user per month for Port enterprise tiers).

Choose OpsLevel if you need a basic service catalog immediately and your primary use case is tracking ownership and maturity, not building complex workflows or maintaining extensive documentation.

Choose Compass if you live entirely in the Atlassian ecosystem and can accept its limitations with non-Atlassian tools. Note that scaffolding and templates were deprecated on December 1, 2025. The integration efficiency may outweigh the platform's constraints if you're already deep in the Atlassian world.

Choose Roadie if you want the industry standard (Backstage) with its entire ecosystem and community support, but you value your engineers' time too much to spend it on infrastructure maintenance. It's what I consider the golden path for enterprises that want modern platform capabilities without the platform tax or the $2M+ cost of building from scratch.

The key question isn't which platform has the most features. It's which platform lets your engineers focus on building platform capabilities instead of maintaining platform infrastructure. At the 150+ engineer scale where IDPs become critical, that distinction determines whether your portal becomes a force multiplier or just another thing to maintain, potentially at a cost exceeding $2M over three years if you go the self-hosted route.

Final Thoughts

After evaluating these platforms for the past few years, I keep coming back to a simple principle: your platform team should build platform capabilities, not maintain platform infrastructure. The $2M+ you'd spend on self-hosting Backstage could fund a lot of actual platform work.

If you're serious about Backstage but want to skip the TypeScript tax, request a personalized demo of Roadie that'll show you what managed Backstage looks like in practice. Worth checking out before you commit to building everything yourself.

Developer Productivity vs Developer Experience: Why You Can't Fix One Without the Other

Lucien Chemaly — Mon, 05 Jan 2026 07:32:43 +0000

Your engineering team lost 8+ hours per developer this week to broken CI pipelines, unclear requirements, and context switching between tools. That's 20% of your engineering capacity. Your manager brings sprint velocity reports. You bring developer complaints about the flaky test suite. Nobody wins because you're treating these as separate problems.

They're not. Developer Experience is the leading indicator of Productivity. Fix one without the other and you're building either a burnout factory or an expensive hobby club.

Here's what these terms mean:

Productivity is the rate of value delivery to users. Features shipped, bugs fixed, deployments completed. The measurable output.

Developer Experience (DevEx) is the friction encountered while delivering that value. Build failures, context switches, review delays, broken tooling, ambiguous requirements. The daily obstacles that slow everything down.

The relationship is causal. Bad DevEx degrades Productivity over time. High Productivity with poor DevEx is temporary. You can't optimize one metric while ignoring the human cost.

The Burnout Factory: High Productivity, Low DevEx

You can force short-term productivity gains. Extend sprints, skip code reviews, ignore tooling problems, accumulate technical debt. The velocity metrics look great for six months.

Then the system breaks.

Your best engineers start quiet quitting. Code quality drops. Hero culture kicks in where 20% of the team carries 80% of the work. On call becomes unbearable because nobody understands the hastily shipped features. Six months later, you lose three senior engineers in a single month.

The 2024 DORA State of DevOps Report found that teams with unstable organizational priorities experience 40% higher risk of burnout. Even strong leadership and good documentation can't compensate for constantly shifting goals. The human cost isn't visible in your sprint reports until it's too late.

The AI Amplification Effect

The 2025 DORA report introduces a critical insight: AI acts as an amplifier. It magnifies whatever foundation you give it. High-performing teams with solid practices see acceleration. Teams with dysfunction see amplified problems.

The data shows AI adoption now positively correlates with throughput. Teams ship code faster and recover from failures more quickly. But here's the catch: AI adoption still correlates with higher instability. More change failures, increased rework, longer cycle times to resolve issues.

90% of technology professionals use AI at work, up 14% from 2024. Developers spend a median of two hours per day with AI tools. Yet only 24% trust AI outputs strongly. The trust paradox: nearly everyone uses it, but few fully trust it.

Your senior engineers now spend 60% of their time reviewing AI-generated code. The throughput metric goes up. The experience reality is code review fatigue, debugging syntactically correct but architecturally wrong solutions, and fixing subtle bugs that pass basic checks. Six months later, your tech leads quit.

The Country Club: Low Productivity, High DevEx

The opposite failure is equally dangerous. Some organizations optimize purely for developer happiness. Perfect tooling, unlimited refactoring time, zero pressure to ship. Engineers love it. The business can't sustain it.

Tech companies have been cutting significant portions of their workforce. Crunchbase tracked over 95,000 tech layoffs in 2024, down from 200,000 in 2023 but still substantial. Companies cutting 20-30% of headcount are common: Stack Overflow cut 28%, Bumble 30%, Mozilla 30%. When organizations can't demonstrate value from engineering investments, the cuts are severe.

The trap is vibes-based management. Running surveys, collecting feedback, investing in tooling but never connecting improvements to business outcomes. When the CFO asks "what did we get for the \$2M tooling budget?" you have no answer.

The SPACE Framework: Measuring Both Dimensions

The SPACE framework from Microsoft Research, GitHub, and the University of Victoria recognizes that developer productivity is multi-dimensional:

Satisfaction and Well-being: How developers feel about their work
Performance: The outcome of developer work
Activity: Developer actions and outputs
Communication and Collaboration: How teams interact
Efficiency and Flow: Ability to complete work with minimal interruptions

You need both objective telemetry (GitHub activity, deployment frequency, cycle time) and subjective data (developer surveys, sentiment analysis). One without the other gives you an incomplete picture.

Example: Your deployment telemetry says builds complete in 5 minutes (fast). Your survey data says builds fail randomly 50% of the time, forcing developers to restart and wait another 5 minutes (frustrating). You need both signals to understand the real problem.

Most tools measure half the picture. Traditional metrics platforms like LinearB focus on quantitative signals (DORA metrics, cycle time). Survey platforms like Culture Amp capture sentiment across organizations but aren't developer-specific. DX (founded by DORA/SPACE research creators) combines developer surveys with SDLC analytics. These approaches require deliberate implementation and buy-in.

The AI Variable: Speed Gains with Quality Costs

AI coding tools have shifted the productivity-experience equation. Teams are getting faster but the stability costs are real.

The Throughput Story

The 2025 DORA report confirms what earlier studies suggested: AI improves throughput. Teams using AI move work through the system faster than those who don't. The speed gains are measurable and consistent across organizations.

But the report asks the critical question: "Faster, but are we any better?"

Large-scale field studies show:

Microsoft: 12.92% to 21.83% more pull requests per week in production environments
Accenture: 7.51% to 8.69% more pull requests per week
Real-world tasks: Studies using proprietary codebases report 30-40% time savings on repetitive tasks

These are meaningful gains. Not the 55% from controlled lab studies, but significant improvements in real work.

The Quality Tradeoff

Veracode's 2025 GenAI Code Security Report tested over 100 large language models across 80 coding tasks:

45% of AI-generated code introduced OWASP Top 10 vulnerabilities
Java had a 72% failure rate (highest risk)
Cross-site scripting tasks failed 86% of the time
Newer, larger models performed no better than smaller ones on security

Security performance hasn't improved despite massive gains in code generation capabilities.

GitClear's analysis found code churn (lines reverted or updated within two weeks) projected to double in 2024 compared to pre-AI baseline. AI-generated code shows 41% higher churn rate.

The 2025 DORA research confirms this at scale: AI adoption correlates with higher instability. More change failures. Increased rework. Longer recovery times. Teams are shipping faster but breaking more.

The Amplifier Effect

DORA's central thesis is that AI doesn't create elite organizations, it anoints them. Organizations with solid foundations (robust platforms, clear processes, strong culture) see AI accelerate everything. Organizations with dysfunction see AI magnify the problems.

High-quality platforms, data ecosystems, and governance are the difference. Without them, AI just helps you hit the wall faster.

The Measurement Problem: Time Loss at Scale

The 2024 State of Developer Experience report from Atlassian and DX surveyed 2,100+ developers and leaders:

69% of developers lose 8+ hours per week to inefficiencies. That's 20% of their time. The top causes:

Technical debt
Insufficient documentation
Flawed build processes
Context switching between tools
Unclear requirements

The disconnect: Only 44% of developers believe their leaders are aware of these issues. Meanwhile, 86% of leaders recognize they can't attract and retain talent without improving developer experience.

This gap is expensive. A Bay Area developer making \$180,000 annually who loses 8 hours per week to inefficiencies represents \$36,000 in wasted productivity per year. For a team of 50 developers, that's \$1.8M annually lost to friction that leadership doesn't know exists.

The 2025 DORA report found no correlation between AI adoption and increased burnout. Developers are adapting to AI-enhanced workflows despite handling more concurrent workstreams. This is surprising given the increased context switching (9% more task contexts, 47% more pull requests daily). Teams are absorbing the complexity without burning out, at least not yet.

How to Measure Both (Without Creating Overhead)

The failure mode is measurement theater. Weekly surveys, 50 different metrics, reports nobody reads, bureaucracy that creates more friction than insight.

Do this instead:

Don't spam surveys. Run comprehensive DevEx surveys quarterly using research-backed frameworks like SPACE metrics. Use targeted pulse checks triggered by specific events: after major incidents, when cycle time spikes, during oncall rotations.

Don't weaponize metrics. Metrics should debug the system, not judge individual developers. If your team thinks productivity tracking is surveillance, you've lost. Make the data visible to everyone. Explain what you're measuring and why. Focus on team-level trends, not individual performance.

Correlate, don't report in silos. The real insight comes from connecting the dots. When rework rate spikes, does developer sentiment drop? When you improve build times, does deployment frequency increase? When AI adoption hits 80%, what happens to code review cycle time and change failure rate?

Microsoft Research studied this directly. Their 2024 "Time Warp" study found that developers who felt "very productive" had the highest correlation (0.52) between their actual and ideal workweeks. For unproductive developers, that correlation dropped to 0.18. The gap between how developers want to spend their time and how they actually spend it predicts productivity and satisfaction.

What I'm Using to Solve This

I've been evaluating tools that connect productivity metrics with developer experience data. The problem is that most platforms do one or the other, not both.

After testing several options, I'm using Span. What sold me: it combines metrics, team surveys, and behavioral context to show the complete picture of productivity and team health. Not just isolated numbers.

You can correlate rework rate (productivity) with developer sentiment scores (experience) to understand if technical debt is creating team burnout. You can compare deployment frequency (productivity) with build satisfaction (experience) to measure if your CI/CD improvements actually helped. You can track AI code adoption (activity) alongside code review time and change failure rate (efficiency and stability) to see if your Copilot investment is shifting bottlenecks instead of eliminating them.

The span-detect-1 model provides 95% accuracy in detecting AI-assisted code across any tool: Copilot, Cursor, Claude, or custom solutions. This matters because you need to measure actual impact, not just adoption rates. Is AI code getting merged faster? Does it require more review cycles? How does the change failure rate compare between AI-assisted and human-written code?

The platform doesn't require perfect Jira hygiene or process changes. AI-native classification automatically categorizes work, identifies patterns, and surfaces insights without adding overhead to your team. That was the dealbreaker for me. I'm not asking my team to change their workflow to feed a metrics tool.

What Actually Works

The organizations winning in 2025 stopped treating productivity and experience as competing priorities. Here's what they do differently:

They measure time loss, not just time spent. Understanding where 8 hours per week disappears matters more than tracking story points completed.

They connect DevEx investments to business outcomes. When you improve build reliability, track deployment frequency and change failure rate before and after. When you fix documentation gaps, measure onboarding time and time-to-first-commit for new engineers.

They treat AI tools as transformations, not features. The 2025 DORA AI Capabilities Model identifies seven organizational practices that determine AI success: clear governance, high-quality data ecosystems, robust version control, small-batch delivery, user-centric feedback, and strong platforms. Organizations that invest in these capabilities see AI accelerate their work. Organizations that skip them see AI magnify dysfunction.

They ask developers what's broken. Less than half of developers think their leaders understand the obstacles they face. The simplest fix is also the most effective: ask.

They prioritize stable goals over constant pivots. The "move fast and constantly pivot" mentality increases burnout by 40% even with strong leadership and good documentation. Stability matters.

They understand AI is an amplifier. If your foundation is shaky (technical debt, unclear requirements, flaky builds, poor documentation), AI will make it worse. Get your platform house in order first. Then AI becomes a force multiplier instead of a chaos accelerator.

The Bottom Line

You can't fix Developer Productivity without improving Developer Experience. You can't improve Developer Experience without measuring Productivity outcomes. The organizations winning in 2025 stopped treating these as competing priorities.

AI tools are forcing this reckoning. The 2025 DORA report confirms teams are getting faster with AI. But the instability costs are real and persistent. You need visibility into both sides to make informed decisions about where to invest.

The teams that figure this out first will have a sustainable competitive advantage. The teams that don't will keep losing their best engineers while wondering why the velocity charts look so good.

If you're ready to stop choosing between speed and happiness, check out Span. Their AI Code Detector gives you ground truth on how much AI code is being written and merged. You can request a demo to see how it correlates metrics with sentiment and surfaces where your investments are actually paying off.

Stop Measuring Noise: The Productivity Metrics That Really Matter in Software Engineering

Lucien Chemaly — Tue, 23 Dec 2025 11:55:48 +0000

"Productivity" has become a dirty word in engineering.

Mention it in a Slack channel, and the immediate assumption is that management is looking for a reason to fire the bottom 10%, or that McKinsey is back with another controversial report. (For what it's worth, their 2023 report actually warns against using "overly simple measurements, such as lines of code produced, or number of code commits" rather than recommending them. The backlash came from other aspects of their approach.)

The skepticism is earned. For decades, productivity metrics in software engineering have been weaponized to micromanage individual contributors rather than optimize systems.

But ignoring metrics entirely is just as dangerous. When you run an engineering organization on vibes and anecdotal evidence, you end up playing a game of Telephone. The reality of what's happening on the ground gets distorted as it passes through layers of management. You lose the ground truth.

The question isn't if we should measure productivity. The question is what we should measure.

Most standard dashboards are filled with noise. In the era of AI coding tools, DORA metrics are no longer enough. Here's what actually matters in 2025.

The Problem with Velocity at All Costs

For the last few years, the industry converged on DORA metrics (Deployment Frequency, Change Lead Time, Change Fail Percentage, Failed Deployment Recovery Time) as the gold standard.

DORA is excellent, but it's a smoke alarm. It tells you when the house is burning down (Change Fail Percentage spikes) or when you're moving painfully slow (Deployment Frequency drops). But it doesn't tell you why.

The rise of AI tools like GitHub Copilot and Cursor has broken velocity as a standalone metric. A developer can now generate a 500-line Pull Request in 30 seconds using an LLM. The cycle time for the coding phase looks incredible. But if that code is a hallucinated mess that clogs up the review process for three days, that personal velocity came at the expense of the team's throughput.

If you only measure speed, you'll incentivize behaviors that destroy quality. Research from Tilburg University analyzing GitHub activity found that while less-experienced developers gain productivity from AI tools, core developers now review 6.5% more code and show a 19% drop in their own original code productivity. The time shifted to reviewing AI-generated submissions.

The New Framework: Inputs, Outputs, and Internals

To get a real signal, stop treating engineering like a black box. Start treating it like a system. You need to measure three distinct areas:

Inputs: What are we investing? (Headcount, Tooling Costs, Cloud Spend)
Internals: How is the work actually happening? (PR workflow, Rework, Focus Time, Context Switching)
Outputs: What's the result? (Reliability, Feature Adoption, Customer Value)

Here are the four specific metrics that are proving most valuable for modern engineering leaders.

1. Rework Rate (The AI Counter-Balance)

This is the most underrated metric in software development right now. Rework Rate measures the percentage of code that is rewritten or reverted shortly after being merged. The 2024 DORA Report now includes rework rate as part of their evolved framework, categorizing it alongside Change Fail Percentage as an "instability metric."

In an AI-augmented world, it's very easy to ship bad code fast. Data from platforms analyzing hundreds of engineering teams reveals a fascinating U-shaped curve regarding AI adoption:

Low AI usage: Standard rework rates
High AI usage (Boilerplate): Low rework rates (AI excels at unit tests and scaffolding)
Hybrid usage (25-50% AI): Highest rework rates

When developers use AI in a "mixed context" (half human logic, half AI autocomplete), it creates code that looks correct at a glance but fails in edge cases. This hybrid approach causes cognitive whiplash for reviewers who must switch between evaluating human and AI logic. PRs that are clearly all-human OR all-AI are easier to review than mixed-ratio PRs.

The red flag: If you see your cycle time improving but your rework rate creeping up, you're not moving faster. You're just building technical debt faster.

GitClear's analysis of 211 million lines of code found that code churn is projected to double in 2024 versus their 2021 baseline, with 7.9% of newly added code being revised within two weeks (compared to 5.5% in 2020). Copy-pasted code rose from 8.3% to 12.3%.

The challenge: you can't fix what you can't see. Most teams have no idea what percentage of their code is AI-generated, let alone how that AI code correlates with rework. Tools like Span's AI Code Detector now measure AI-authored code with 95% accuracy (for Python, TypeScript, and JavaScript), giving you ground truth on adoption patterns and quality impact.

2. Investment Distribution (The Truth vs. Your Task Tracker)

Ask a VP of Engineering what their team is working on, and they'll show you a roadmap: "40% New Features, 20% Tech Debt, 40% KTLO."

Ask the engineers what they're working on, and you'll hear about the "shadow work": "I'm supposed to be on the Platform team, but I'm spending 20 hours a week helping the Checkout team fix bugs because I'm the only one who knows the legacy codebase."

Project management tools are often a lagging indicator of intent, not reality. Tickets get rolled over, scope creeps, and quick fixes go untracked. An IDC report analyzing developer time allocation found that application development accounts for just 16% of developers' time, with the rest consumed by meetings, context switching, and what engineering manager Anton Zaides calls "shadow work".

Zaides identifies three main types of invisible work stealing team capacity:

Invisible production support: Investigating alerts, answering questions, ad-hoc requests
Technical glue work: Code reviews, planning, mentoring, documenting
Shadow backlog: Off-the-record PM requests, engineers doing things "right" without approval

In one case study, a senior engineer had more than 40% of their time as invisible work. An internal team performed roughly 65% shadow work, all without cost codes or billing.

The red flag: You need a metric that looks at actual code activity (commits, PRs, and reviews) to classify where effort is going. This automated analysis often reveals that the "Innovation" team is actually spending 70% of their time on maintenance. You can't fix that allocation if you rely solely on project management tools.

Platforms like Span automatically categorize engineering work by analyzing git activity, creating what they call an "automated P&L of engineering time." You can finally answer questions like "How much time did we actually spend on that platform migration?" with data instead of guesswork.

3. Review Burden Ratio

This metric tracks the relationship between the time spent writing code and the time spent reviewing it.

As AI drives the marginal cost of writing code to zero, the bottleneck shifts to the reviewer. AI generates code instantly, humans review code linearly. If your review burden is skyrocketing, your senior engineers are becoming human spell-checkers for LLMs.

The Tilburg University research quantified this shift: each core contributor now reviews approximately 10 additional PRs annually. Meanwhile, Faros AI's analysis found that code review time increases 91% as PR volume outpaces reviewer capacity, with PR size growing 154% and bug rates climbing 9%.

The red flag: Watch for "LGTM" culture versus "Nitpick" culture.

Too fast: If massive AI-generated PRs are getting approved in minutes, your Review Burden is suspiciously low, and you likely have a quality issue looming
Too slow: If review burden is high, your senior engineers are trapped in Review Hell, leading to burnout and a halt in architectural innovation

4. Fragmented Time (The Anti-Flow Metric)

Engineering requires flow. Yet, most organizations schedule their way out of productivity. Fragmented Time measures the blocks of available deep work time (2+ hours) versus time fractured by meetings and interruptions.

You can have the best AI tools in the world, but if an engineer has 30 minutes between standup and a planning meeting, they're not shipping complex features. They're answering emails.

Research from UC Irvine professor Gloria Mark found that it takes an average of 23 minutes and 15 seconds to fully return to a task after an interruption. The nuance: this measures returning to your original "working sphere" (project-level work), not recovering from every small distraction. Her updated 2023 research in "Attention Span" found this has increased to approximately 25 minutes, while average attention on screen dropped from 2.5 minutes (2004) to just 47 seconds (2021).

The red flag: If you look at calendar data and see that 40% of your engineering capacity is lost to context switching costs (the 30-minute dead zones between meetings), you've found the cheapest way to improve productivity: cancel meetings. No tool can fix a broken calendar.

How to Measure Without Being Big Brother

The biggest risk with productivity metrics is cultural, not technical. If you roll these out as a way to rank engineers, you've already lost. You'll incentivize gaming the system (splitting one PR into ten tiny ones to boost "throughput").

Here's the golden rule: Metrics are for debugging systems, not people.

Don't use data to ask: "Why is Alice slower than Bob?"
Do use data to ask: "Why is the Checkout Team stuck in code review twice as long as the Platform Team? Do they need better tooling? Is their tech debt unmanageable?"

Leaders shouldn't want a dashboard that tells them who to fire. They should want a dashboard that acts as a neutral third party, objective data to validate what engineers are already saying in 1:1s. When an engineer says, "I'm swamped with maintenance work," you want the data to prove it so you can secure the budget and resources the team actually needs.

Getting Started

So how do you actually implement these metrics?

For Rework Rate:
Use your version control system to track code that gets reverted or significantly modified within 14-21 days of merging. Most engineering intelligence platforms can calculate this automatically. If you're rolling your own, start with a git analysis script that flags commits that touch the same files within your chosen time window.

For Investment Distribution:
Engineering intelligence platforms can automatically categorize work based on code patterns, repo structure, and commit messages. The key is classification: New Features vs. Bug Fixes vs. Tech Debt vs. Operations. If you don't have budget for tools, start manually by sampling one week per month and having engineers log their actual time allocation, then compare it to what your task tracker says.

For Review Burden:
Calculate the ratio of time PRs spend in review versus time spent in active development. GitHub, GitLab, and Bitbucket all expose this data through their APIs. A healthy ratio varies by team, but if you see review time consistently exceeding development time, or if senior engineers are spending more than 50% of their time reviewing, you've got a problem.

For Fragmented Time:
Use calendar analysis tools or simple calendar exports. Count the number of 2+ hour blocks available per engineer per week. Anything less than 10 hours of uninterrupted time per week per engineer is a red flag.

The AI Quality Question

The elephant in the room: how do you actually measure AI code quality?

You can't manage what you can't measure. The first step is knowing how much AI-generated code you're actually shipping. Most teams are flying blind here, relying on surveys that are notoriously unreliable (self-reported data has serious limitations).

Platforms like Span use code-level detection to identify AI-authored code with high accuracy, then correlate that with downstream metrics like rework rate, review cycles, and bug density. This gives you a complete picture: where AI is helping, where it's hurting, and how to coach your team to use it more effectively.

The counterintuitive finding: it's not about reducing AI usage. It's about using AI more deliberately. Teams that coach engineers to use AI for complete, well-scoped tasks (not mixed human-AI work) see better outcomes.

The Bottom Line

We're entering a new era of engineering intelligence. The old proxies (lines of code and commit counts) are dead. Even the modern standards like cycle time are insufficient on their own.

To navigate the next few years, you need to understand the interplay between human creativity and AI leverage. You need to measure the quality of AI code, not just the volume.

If you want to build a high-performing team in 2025, stop measuring how busy everyone looks. Start measuring the friction that slows them down.

The metrics discussed here won't give you a single number to optimize. Instead, they'll give you a system view: where time actually goes, where quality breaks down, where your senior engineers are drowning in review work, and where your calendar is stealing focus time.

Use them to debug your system, not to rank your people.

If you're ready to move beyond vanity metrics and get real insights into your engineering organization, check out Span to see how engineering intelligence platforms are helping teams measure what actually matters.

AI Tools for Developer Productivity: Hype vs. Reality in 2025

Lucien Chemaly — Wed, 17 Dec 2025 10:18:48 +0000

If you believe the marketing, AI "Agents" are about to replace us all, and "Autonomous Software Engineering" has arrived. If you ask actual developers, we're mostly just fixing hallucinations in boilerplate code.

Every week brings another breathless announcement. Another demo video showing an AI building a complete app in 47 seconds. Another VC-funded startup claiming to have cracked AGI for code. Meanwhile, you're sitting in a PR review, staring at AI-generated code that confidently imports a library that doesn't exist.

The noise is exhausting. But here's the thing: AI coding tools are useful. Just not in the way the hype machine suggests.

The real value isn't "autonomy," where AI does the job for you. It's "flow," where AI keeps you in the zone. That distinction matters more than any feature comparison, and understanding it will save you from chasing the wrong tools.

Category 1: The Sidekicks (IDE Extensions)

The Hype: "It reads your mind and writes the function."

The Reality: It's a super-powered StackOverflow copy-paste.

GitHub Copilot, Cody, and Supermaven fall into this category. They live inside your existing IDE as extensions, watching what you type and suggesting completions. The marketing implies they understand your codebase, your intentions, and your architectural decisions.

They don't. What they actually do is pattern-match against billions of lines of training data and make educated guesses about what comes next.

That sounds dismissive, but it shouldn't be. For certain tasks, this is genuinely transformative. Writing regex patterns that would otherwise require fifteen minutes of trial and error becomes a two-second autocomplete. Generating boilerplate for unit tests goes from tedious to trivial. Converting between data formats, writing SQL queries, scaffolding API endpoints, all the repetitive work that burns hours across a sprint becomes nearly instant.

The problem emerges when developers start trusting these suggestions for anything beyond boilerplate. Ask Copilot to implement business logic and you'll get something that looks correct, compiles successfully, and silently breaks edge cases. Ask it for security-sensitive code and you might ship a vulnerability that passed all your reviews because the code looked reasonable.

The Verdict: Essential for reducing toil, but requires heavy verification. Use them for the boring stuff. Don't use them for the important stuff. And definitely don't measure productivity by how many suggestions you accept.

Category 2: The Native Editors (AI-First IDEs)

The Hype: "The IDE that codes for you."

The Reality: This is the biggest actual leap in 2025.

Cursor and Windsurf represent a fundamentally different approach. Instead of bolting AI onto an existing editor, they build the entire IDE around AI-assisted workflows. This architectural decision has practical consequences that matter.

Because these tools own the runtime, the terminal, and the diff view, they reduce context switching in ways plugins can't. When Cursor's "Composer" mode suggests a multi-file refactor, you see the changes across your entire project in a unified interface. You can accept, modify, or reject at the file level without jumping between tabs.

This might sound like a minor UX improvement. It's not. The friction cost of context switching compounds throughout a coding session. Every time you break focus to check another file, run a test, or verify a change, you're burning cognitive resources. Tools that reduce those interruptions keep you in flow state longer.

The multi-file editing capabilities are particularly strong for refactoring work. Renaming a function, updating its call sites, adjusting tests, and modifying documentation can happen in a single coordinated operation. For migration tasks, dependency updates, or architectural changes that touch dozens of files, this is genuinely faster than doing it manually.

The Verdict: Superior to plugins for anyone doing regular refactoring or multi-file changes. The investment in learning a new editor pays off for teams with complex, interconnected codebases.

Category 3: The Autonomous Agents (AI Software Engineers)

The Hype: "The First AI Software Engineer."

The Reality: Still experimental. High potential, but high babysitting cost.

Devin and SWE-agent represent the bleeding edge: AI systems that theoretically work independently, browsing documentation, writing code, running tests, and iterating until the task is complete. The demo videos are impressive. An AI figures out an unfamiliar codebase, implements a feature, and opens a PR, all without human intervention.

In practice, the experience differs from the demos.

These agents work best for isolated "janitorial" tasks. Version bumps across multiple packages. Dependency migrations where the changes are mechanical. Adding logging to existing functions. Tasks with clear success criteria and minimal interaction with core business logic.

Where they struggle is anywhere that requires understanding context beyond the immediate code. Implementing a feature that needs to match existing patterns in your codebase. Making architectural decisions that align with your team's conventions. Anything touching authentication, authorization, or data integrity.

The oversight cost is the killer. You can't just fire off a task and forget about it. You need to review the agent's approach, check its intermediate steps, and validate the final output. For complex tasks, the time spent supervising often exceeds the time you would have spent just doing the work yourself.

The Verdict: Useful for sandbox tasks and mechanical migrations. Not ready to touch core production logic without significant human oversight. Worth experimenting with, but don't bet your roadmap on them.

The Hidden Cost of AI: Rework and Review

Here's what nobody talks about in the productivity demos: the Jevons Paradox applied to code.

When writing code becomes cheap and fast, we write more of it. But reading code is still expensive. Someone has to review every AI-generated PR. Someone has to debug the subtle issues that passed initial review. Someone has to maintain the extra abstraction layers that seemed like a good idea when they took three seconds to generate.

The nastiest problem is the "Hybrid PR." Code that's 50% AI-generated and 50% human-written often takes the longest to review. The context switches between AI patterns and human patterns. The reviewer can't develop a mental model of the author's intent because there are effectively two authors with different styles.

This isn't theoretical. A rigorous study from METR in July 2025 found that experienced developers took 19% longer to complete tasks when using AI tools, despite believing they were faster. Teams report that review times have increased even as coding times have decreased. The net effect on cycle time is smaller than the productivity demos suggest, sometimes negligible.

This doesn't mean AI tools aren't worth using. It means you need to measure the entire pipeline, not just the "lines written per hour" number that looks good in executive reports.

How to Separate Hype from Reality in Your Own Team

Don't trust demo videos. Don't trust vendor benchmarks. Trust your own repository data.

The questions that actually matter are: Are developers accepting AI suggestions, or ignoring them? Is AI-generated code causing more rework in PRs? Which tools are your power users actually using daily versus which ones got installed and forgotten?

This is where measurement becomes critical. Self-reported usage surveys are unreliable because developers often overestimate their AI adoption. Vendor telemetry only shows you data for their specific tool, not how it fits into your broader workflow.

What you need is code-level visibility: the ability to see which PRs contain AI-assisted code, how that code performs in review, and whether it requires more or less revision than human-written code.

Span provides this visibility. The platform can detect AI-generated code with over 95% accuracy across all major AI coding tools, giving you ground truth on adoption rather than self-reported estimates. More importantly, it correlates AI usage with downstream metrics like review cycles and rework rates, so you can see whether your AI investments are actually paying off.

The goal isn't to discourage AI tool usage. It's to ensure you're investing in tools that genuinely improve your team's output rather than just creating the appearance of productivity.

Conclusion

The best AI coding tool is the one that fits your workflow, not the one with the coolest Twitter demo.

Sidekicks like Copilot reduce toil on repetitive tasks but require constant verification. Native editors like Cursor and Windsurf genuinely improve multi-file workflows and refactoring. Autonomous agents like Devin show promise for isolated tasks but aren't ready for unsupervised production work.

None of them are magic. All of them require thoughtful integration into your development process. And measuring their actual impact requires looking beyond the metrics that vendors want you to track.

Experiment aggressively. Try the new tools, push their limits, and see what works for your specific codebase and team structure. But measure the results with something more rigorous than vibes.

The AI revolution in software development is real. It's just more mundane than the hype suggests, and that's okay. Mundane improvements that compound across thousands of developer-hours are worth more than revolutionary demos that don't survive contact with production code.

Unsure if your team's AI tools are hype or reality? Connect Span to your repo to see exactly how much AI code is actually sticking in production.

Cheat Sheet for Basic Python for Postgres

Lucien Chemaly — Mon, 22 Sep 2025 06:34:17 +0000

Combining Postgres with Python is an effective solution for complex data-driven tasks thanks to Python's straightforward programming experience and extensive libraries.

In real-world applications, effective data management is essential. Whether you're working with user profiles, financial transactions, product inventories, or any other structured data, you'll often need to create, read, update, or delete records. These four operations—collectively known as CRUD—are the foundation of all database interactions. Mastering CRUD with Python and Postgres lays the foundation for building web applications, automating data workflows, and developing robust backend services for use cases like e-commerce platforms, automation scripts, and mobile apps.

This tutorial shows how I set up a Postgres database using Supabase and connected a Python application to it. I show how to perform create, update, and delete operations, and how to make sure your database operations are safe, efficient, and maintainable.

Setting Up the Environment and Project

If you don't have Python installed on your machine, go to the official Python website and download Python 3.10 or later.

Create a new directory for your project by executing the following commands from your terminal or shell:

mkdir python-postgres-crud
cd python-postgres-crud

To keep your project isolated, create a virtual environment by running the following command:

python3 -m venv venv

Now activate the virtual environment. If you're on Windows, run the command below:

.\venv\Scripts\activate

On macOS/Linux, run the following command:

source venv/bin/activate

You also need a dedicated adapter to interact with Postgres using Python. Run the following command to install psycopg2, the most commonly used library for this purpose:

pip install psycopg2-binary

With your project created, you're ready to start setting up the database.

Setting Up a Postgres Database

Instead of setting up a local Postgres instance, I decided to use Supabase because it provides a fully managed, hosted Postgres database out of the box, simplifying the setup time and database management. This allows us to focus entirely on writing and deploying the application code without worrying about infrastructure details. First, visit Supabase and create a free account if you don't have one yet.

Note: If you prefer to work with a local database, you can install Postgres locally instead. However, this tutorial assumes you're using Supabase.

Once you're logged in, go to your dashboard and click New Project. Fill out the form by providing a project name (eg python-tutorial), setting a strong database password (make sure to save it), and selecting a region close to your location. After you submit the form, Supabase will take a minute or two to provision your project.

After your project is ready, click Connect on the top bar. Then scroll down to Session pooler and click View parameters. Here you'll find your connection details.

Copy your connection details (host, database, and user) and keep them somewhere safe because you'll need them later to connect your Python application to your database.

Connecting Python to Your Supabase Database

Inside your project directory, create a file named db_connect.py and add the following code to establish a connection to the database (make sure to replace YOUR_HOST, YOUR_USER, and YOUR_PASSWORD with your credentials):

import psycopg2

def create_connection():
    try:
        conn = psycopg2.connect(
            host="YOUR_HOST",
            database="postgres",
            user="YOUR_USER",
            password="YOUR_PASSWORD",
            port=5432
        )
        print("Connection to the database successful!")
        return conn
    except Exception as e:
        print(f"Failed to connect to database: {e}")
        return None

If the connection is successful, the code will generate a success message. If the connection fails, an error message will display.

To test the connection, create a new file named test_connection.py and add the following code:

from db_connect import create_connection

conn = create_connection()

if conn:
    conn.close()

Run this code from your terminal or shell:

python test_connection.py

This test verifies your database credentials, connection parameters, and network accessibility, confirming that your application can successfully communicate with the Supabase database. If everything is set up correctly, you'll see Connection to the database successful! printed in the console.

Preparing Your Database

Before your Python code can interact with any data, you'll need to set up a database table. In this example, you'll create a users table to store user information, including names and email addresses.

From the left navigation in the Supabase dashboard, navigate to SQL Editor and paste the following SQL command, then click Run to create a users table:

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

This command defines a table where each user will have a unique ID, a name, a unique email address, and an automatically set creation timestamp. After you run the query, open Table Editor from the left navigation to ensure your table has been created successfully.

Inserting Records

To insert records, create a new file named insert_user.py and add the following code:

from db_connect import create_connection

conn = create_connection()
cur = conn.cursor()

try:
    cur.execute(
        """
        INSERT INTO users (name, email)
        VALUES (%s, %s)
        """,
        ("Alice", "alice@example.com")
    )
    conn.commit()
    print("User inserted successfully!")
except Exception as e:
    print(f"Failed to insert user: {e}")

cur.close()
conn.close()

This code uses the create_connection() function to create the connection, which is the communication channel between your Python code and the database. The conn.cursor() function creates a cursor that is used to execute SQL commands. The cur.execute() method executes the SQL insert, and the conn.commit() function saves the change to the database. If the operation is successful, you will get a success message, and a new user with the name Alice and email alice@example.com will be created.

Run the code by executing the following:

python insert_user.py

You should see an output similar to the following:

Connection to the database successful!
User inserted successfully!

Now, if you check your database from Table Editor, you should see that the user named Alice has been added to your table.

Adding Multiple Records

When you need to insert many records at once—such as during initial setup or data imports—you can do a bulk insert. To do so, create a new file named insert_many_users.py and add the following code:

from db_connect import create_connection

# Number of users to insert
x = 20

# Generate dummy users
users = [
    (f"User{i}", f"user{i}@example.com")
    for i in range(1, x + 1)
]

# Connect to the database
conn = create_connection()
cur = conn.cursor()

try:
    # Bulk insert using executemany
    cur.executemany(
        "INSERT INTO users (name, email) VALUES (%s, %s)",
        users
    )
    conn.commit()
    print(f"✅ Inserted {cur.rowcount} users successfully!")
except Exception as e:
    conn.rollback()
    print(f"❌ Failed to insert users: {e}")
finally:
    cur.close()
    conn.close()

This code specifies the number of users (twenty) through the variable x before it uses a for loop to fill the dummy users array. The code establishes a database connection before running an SQL query to insert all users simultaneously through the executemany method. If the operation succeeds, it prints a success message after committing changes. If something goes wrong, it rolls back the transaction and prints an error message.

Run the code by executing the following:

python insert_many_users.py

You should see an output similar to the following:

Connection to the database successful!
✅ Inserted 20 users successfully!

Now, if you check your database from Table Editor, you should see that twenty new user records have been added to the table.

Updating Records

Most applications experience data changes through time because users perform updates to their email addresses, name changes, and preference modifications. Updating operations enable you to modify existing database records without needing to delete or recreate them. To update the records in the example, create a file named update_user.py and add the following code:

from db_connect import create_connection

conn = create_connection()
cur = conn.cursor()

try:
    cur.execute(
        """
        UPDATE users
        SET email = %s
        WHERE id = %s
        """,
        ("newalice@example.com", 1)
    )
    conn.commit()
    print("User updated successfully!")
except Exception as e:
    print(f"Failed to update user: {e}")

cur.close()
conn.close()

It's important that you use a WHERE clause when performing updates so you don't accidentally modify all your records. You can use the primary key (id) or a unique field (eg email) to precisely identify the record you want to update. Avoid using other fields that are not unique (eg name) as they can contain duplicate values and lead to unintended changes.

The code is similar to the previous one for creating a connection and handling errors, but this time you're running an update SQL query to update the email of the record that has the id 1.

Run the code by executing the following:

python update_user.py

You should see an output similar to the following:

Connection to the database successful!
User updated successfully!

Now, if you check your database from Table Editor, you should see that the record has been updated with a new email: newalice@example.com.

Deleting Records

In real-world applications, you may need to remove records, for example, when you delete a user account or clean up test data. While deleting data permanently is sometimes necessary (eg complying with privacy regulations or removing irrelevant test entries), many applications use "soft deletes" instead. Soft deletes mark records as deleted (eg with an is_deleted flag) rather than actually removing them, allowing for later recovery and preserving historical data. That said, to perform a permanent delete using SQL, create a new file named delete_user.py and add the following code:

from db_connect import create_connection

conn = create_connection()
cur = conn.cursor()

try:
    cur.execute(
        """
        DELETE FROM users
        WHERE id = %s
        """,
        (1,)
    )
    conn.commit()
    print("User deleted successfully!")
except Exception as e:
    print(f"Failed to delete user: {e}")

cur.close()
conn.close()

Again, this is similar to the previous code for creating a connection and handling errors, but you're running an update SQL query to delete a record that has the id 1, the email newalice@example.com, and the name Alice.

Run the code by executing the following:

python delete_user.py

You should see an output similar to the following:

Connection to the database successful!
User deleted successfully!

Now, if you check your database from Table Editor, you should see that the record with the email newalice@example.com has been deleted.

Best Practices and Error Handling

When you're building a production-grade application, it's not just about making your code work—it's about making sure it works reliably, securely, and efficiently.

Parameterized Queries

To begin with, always use parameterized queries. These help defend your database from SQL injection, which can happen if user inputs are directly interpolated into SQL commands. Consider this vulnerable code:

cur.execute(f"DELETE FROM users WHERE email = '{email}'")

If the variable email contains malicious SQL code, your entire database could be compromised. For example, an attacker can put something like '; DROP TABLE users; –, and this will turn the SQL command into DELETE FROM users WHERE email = ''; DROP TABLE users; --'. This query will first delete users with empty email addresses, then completely remove your users table, which will delete all your user data.

Instead, use this:

cur.execute("DELETE FROM users WHERE email = %s", (email,))

This way, the input is safely handled by the database driver.

Using Context Managers

You should also use context managers with the with statement when dealing with connections and cursors. This approach ensures proper resource cleanup, even if your code crashes midway. This prevents potential resource leaks, such as leaving database connections open unintentionally. Here's how it works:

from contextlib import closing
import psycopg2

with closing(psycopg2.connect(...)) as conn:
    with conn.cursor() as cur:
        cur.execute("SELECT * FROM users")
        result = cur.fetchall()
        print(result)

Transactions

If you perform multiple operations that should succeed or fail as a group, wrap them in a transaction. Transactions help preserve data integrity. An example is a user registration where you want to create a new user and update their preferences at the same time. Here's how it works:

try:
    cur.execute("INSERT INTO users (username, email) VALUES (%s, %s)", (username, email))
    user_id = cur.fetchone()[0]
    cur.execute("INSERT INTO user_preferences (user_id, preference) VALUES (%s, %s)", (user_id, preference))
    conn.commit()
except Exception as e:
    conn.rollback()
    print("User registration failed:", e)

Exception Handling

You also need to consider how you'll handle exceptions specifically. Use targeted exception blocks like psycopg2.IntegrityError to give more accurate feedback to users. If you try to insert a duplicate email, catching this specific error allows you to tell the user exactly what went wrong rather than presenting a vague or generic message. Here's how it works:

try:
    cur.execute("INSERT INTO users(email) VALUES(%s)", (email,))
    conn.commit()
except psycopg2.IntegrityError:
    conn.rollback()
    print(f"The email '{email}' already exists.")

Fetching Data Efficiently

There are also a few ways you can improve performance. Avoid fetching more data than you need. If you're only displaying a user's name and email in your app, don't query every column from the database.

Instead of writing a query like this:

SELECT * FROM users;

You should write a more specific query like this:

SELECT name, email FROM users;

Pagination

When your data set grows large, pagination becomes essential. Loading thousands of records at once can crash your app or slow it down. Fetch a limited number of rows at a time:

SELECT * FROM users ORDER BY id LIMIT 50 OFFSET 100;

This retrieves fifty records starting from the 101st entry. It's great for building paginated UIs.

Indexing

Indexing also helps speed up searches and filtering. For example, if you frequently look up users by email, create an index:

CREATE INDEX idx_users_email ON users(email);

The use of indexes speeds up read operations but results in slower insert and update operations. The selection of your indexed columns depends on the actual behavior of your application. You should never run UPDATE or DELETE without a WHERE clause because this can unintentionally affect all rows in your table. You can even enforce safety nets in development environments by limiting update/delete operations with application-level checks.

Foreign Key Constraints and Cascading Deletes

A foreign key establishes a link between tables by ensuring that each value in one table points to an existing record in another table, such as linking each post to its corresponding user. The ON DELETE CASCADE feature provides a safe deletion method by automatically deleting related records in child tables whenever a parent record gets deleted.

For example, you can create a posts table that references the user's id as a foreign key, and on delete, you do cascade deletes. Here's what it will look like:


CREATE TABLE posts (

id SERIAL PRIMARY KEY,
user_id uuid REFERENCES users(id) ON DELETE CASCADE,
content TEXT
);

Use cascading deletes with caution, as they can unintentionally remove large amounts of related data. Always design your deletion logic with a clear understanding of the relationships in your schema.

Conclusion

The foundational skills explained in this tutorial are your "cheat sheet" that will allow you to interact with a Postgres database directly from Python. I explained how to do basic CRUD operations using Python and Postgres. I also covered how to implement transactions, handle exceptions properly, avoid SQL injection, and write optimized queries to ensure that your application not only works but remains fast and secure as it scales.

You can find the code used in this tutorial in the following GitHub repository.