DEV Community

Cover image for Your CI/CD Pipeline Has a Blind Spot (and It's Not What You Think)
Liora
Liora

Posted on

Your CI/CD Pipeline Has a Blind Spot (and It's Not What You Think)

#webdev #devops #documentation #tutorial

Your pipeline catches a missing semicolon in thirty seconds.

It runs four thousand unit tests, flags security vulnerabilities, checks code style, enforces branch naming conventions, and sends a slightly passive-aggressive Slack notification if someone pushes directly to main.

It does not check whether your API documentation still describes your API.

Think about this for a second. Your docs are the first thing a developer reads before integrating with your product. If your quickstart references a token format you stopped using in January, you'll find out from a support ticket three weeks later. Not from your pipeline. Your pipeline doesn't know the docs exist. The docs don't know the pipeline exists. They're roommates who've never met, living in the same repository, communicating through the medium of customer frustration.

Here's how to introduce them. One afternoon. Zero budget.

Step 1: Lint your prose like you lint your code

Vale is an open-source linter for prose. You give it rules. It yells at your documentation. Same principle as ESLint, but instead of catching unused variables, it catches things like "your docs say 'workspace' on page 1 and 'project' on page 7 for the same concept."

Install it:

snap install vale    # Linux
pip install vale     # Any OS with Python
# or download a binary:
# https://github.com/errata-ai/vale/releases
Enter fullscreen mode Exit fullscreen mode

Create .vale.ini in your docs root:

StylesPath = .vale/styles
MinAlertLevel = warning

Packages = MyCompany

[*.md]
BasedOnStyles = Vale, MyCompany
Enter fullscreen mode Exit fullscreen mode

Now the useful part - custom rules. Say your product renamed "workspace" to "project" three months ago, but half the docs missed the memo:

# .vale/styles/MyCompany/Terminology.yml
extends: substitution
message: "Use '%s' instead of '%s'."
level: error
swap:
  workspace: project
  log in: sign in
  click on: select
  repo: repository
Enter fullscreen mode Exit fullscreen mode

Run it:

vale docs/

The first time I ran this on a real documentation site - 340 pages - it flagged 918 inconsistencies. The docs had been "reviewed and approved" two weeks earlier. By humans. Who presumably read them. The machine wasn't smarter. It was just more literal.

Step 2: Validate that your code examples actually work

Your docs contain cURL examples. Lovely. Do they return what the docs claim they return?

Not "probably." Do they. Right now.

The embarrassingly simple approach: a script that pulls code blocks from your markdown and runs them against staging.

#!/bin/bash
set -uo pipefail

DOCS_DIR="${1:-docs}"
FAILED=0
TOTAL=0
TMPFILE=$(mktemp)

grep -rh "curl " "$DOCS_DIR" --include="*.md" \
  | sed 's/^[[:space:]]*//' \
  | grep "^curl " > "$TMPFILE"

while IFS= read -r cmd; do
  TOTAL=$((TOTAL + 1))

  if ! echo "$cmd" | grep -q -- '--max-time\|--connect-timeout'; then
    cmd="$cmd --max-time 5 --connect-timeout 3"
  fi

  echo "Testing: ${cmd:0:80}..."
  if eval "$cmd" > /dev/null 2>&1; then
    echo "  OK"
  else
    echo "  FAILED (exit $?)"
    FAILED=$((FAILED + 1))
  fi
done < "$TMPFILE"

rm -f "$TMPFILE"

echo "Results: $TOTAL tested, $FAILED failed."
exit $FAILED
Enter fullscreen mode Exit fullscreen mode

Crude? Very. Better than discovering broken examples from a customer tweet? Enormously.

Step 3: Add a freshness gate to CI

Every documentation page should know how old it is relative to the code it describes. Add last_verified to your frontmatter:

title: "Authentication Guide"
last_verified: "2025-11-15"

api_version: "4.7"

Then a GitHub Action that complains when pages go stale:

name: Docs Freshness Check
on: [pull_request]
jobs:
  freshness:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Flag stale docs
        run: |
          set -uo pipefail

           #!/bin/bash
  set -uo pipefail

  DOCS_DIR="${1:-docs}"
  MAX_AGE_DAYS="${2:-90}"
  STALE=0
  MISSING=0

  THRESHOLD=$(date -d "$MAX_AGE_DAYS days ago" +%s)

  echo "Checking docs freshness (max age: $MAX_AGE_DAYS days)..."
  echo ""

  while IFS= read -r file; do
    reviewed=$(grep -oP 'last_reviewed:\s*"\K[^"]+' "$file" 2>/dev/null || true)

    if [ -z "$reviewed" ]; then
      echo "WARNING: $file - no last_reviewed date"
      MISSING=$((MISSING + 1))
      continue
    fi

    reviewed_ts=$(date -d "$reviewed" +%s 2>/dev/null || echo 0)

    if [ "$reviewed_ts" -lt "$THRESHOLD" ]; then
      echo "STALE:   $file (last reviewed: $reviewed)"
      STALE=$((STALE + 1))
    else
      echo "OK:      $file (last reviewed: $reviewed)"
    fi
  done < <(find "$DOCS_DIR" -name "*.md" -type f | sort)

  echo ""
  echo "Results: $STALE stale, $MISSING without date."

  if [ "$STALE" -gt 0 ]; then
    exit 1
  fi
Enter fullscreen mode Exit fullscreen mode

90 days is generous. Adjust to taste. The point isn't the number - the point is that a number exists where previously there was a vague feeling that "someone should probably look at that."

What this doesn't cover

This is three checks. A real documentation quality pipeline would also validate OpenAPI spec alignment, cross-reference integrity between pages, heading hierarchy for RAG-readiness, link rot detection, and about fourteen other things I could list but won't, because the goal here is to start, not to achieve perfection by Thursday.

Start with these three. Your docs won't be perfect. But they'll stop lying quite so confidently.

Top comments (0)