DEV Community: Ayi NEDJIMI

LLM Context Window Management: Strategies and Patterns

Ayi NEDJIMI — Sat, 20 Jun 2026 10:04:29 +0000

Managing context windows in production LLM applications is one of those problems that everyone underestimates until their app crashes or costs spiral out of control. Token limits are hard walls, not soft guidelines, and the strategies you choose upfront determine whether your system stays reliable at scale.

Why Context Windows Break Production Apps

Most developers hit context limits the same way: they chain a few prompts together, test with small inputs, ship, then watch the system fail when a user uploads a 50-page PDF. Context windows are finite buffers — once you exceed them, the API returns an error, not a graceful degradation.

The practical pain points:

Input truncation: silently cutting content means the model reasons on incomplete data
Cost explosion: naively stuffing the window on every call burns tokens on static content (system prompts, tool schemas)
Latency degradation: larger contexts mean slower TTFT (time to first token)

Understanding the trade-offs between different management strategies is the first step to avoiding these failure modes.

Strategy 1: Sliding Window with Hard Token Limits

The simplest approach is a sliding window that keeps only the most recent N tokens of conversation history. It is not smart — it discards early context — but it is predictable and cheap to implement.

import tiktoken

def truncate_messages(messages: list[dict], max_tokens: int, model: str = "gpt-4o") -> list[dict]:
    enc = tiktoken.encoding_for_model(model)

    total = 0
    result = []

    # Always keep the system message
    if messages and messages[0]["role"] == "system":
        system_msg = messages[0]
        total += len(enc.encode(system_msg["content"])) + 4  # +4 for message overhead
        result = [system_msg]
        messages = messages[1:]

    # Walk backwards through history, keep what fits
    for msg in reversed(messages):
        tokens = len(enc.encode(msg["content"])) + 4
        if total + tokens > max_tokens:
            break
        total += tokens
        result.insert(1 if result else 0, msg)

    return result

This works for conversational bots where recency matters more than total history. The critical constraint: count tokens the same way the API does. tiktoken handles this for OpenAI-compatible models; for other providers, check their tokenizer documentation.

Strategy 2: Semantic Compression with Summarization

When history matters — support tickets, code sessions, research threads — truncation loses information. A better pattern: periodically summarize older conversation segments and replace them with a compressed version.

import tiktoken
from typing import Callable

def compress_history(
    messages: list[dict],
    summarize_fn: Callable[[list[dict]], str],
    threshold_tokens: int = 3000,
    model: str = "gpt-4o"
) -> list[dict]:
    enc = tiktoken.encoding_for_model(model)

    def count_tokens(msgs: list[dict]) -> int:
        return sum(len(enc.encode(m["content"])) + 4 for m in msgs)

    if count_tokens(messages) <= threshold_tokens:
        return messages

    system = [m for m in messages if m["role"] == "system"]
    tail   = [m for m in messages if m["role"] != "system"]

    if len(tail) <= 4:
        return messages

    to_compress = tail[:-4]
    recent      = tail[-4:]

    summary     = summarize_fn(to_compress)
    summary_msg = {
        "role": "system",
        "content": f"[Earlier conversation summary]
{summary}"
    }

    return system + [summary_msg] + recent


def call_llm_for_summary(msgs: list[dict]) -> str:
    # Call your language model here with a summarization prompt.
    # Return the summary as a plain string.
    ...

# compressed = compress_history(conversation_history, call_llm_for_summary)

The cost: one extra LLM call per compression event. The benefit: the model retains the gist of the full conversation without unbounded context growth. For applications handling sensitive data, review your security hardening checklist before routing conversation history to external API endpoints — what goes into the prompt is often more sensitive than what comes out.

Strategy 3: Chunked Retrieval for Document-Heavy Workloads

For document-heavy workloads — legal review, codebase Q&A, long reports — you do not want to send the entire document on every call. You want only the chunks that are relevant to the current query.

The pattern: embed your documents at ingestion time, then at query time retrieve only the top-K chunks by semantic similarity.

import numpy as np
from dataclasses import dataclass

@dataclass
class Chunk:
    text: str
    embedding: list[float]
    metadata: dict

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a_arr = np.array(a)
    b_arr = np.array(b)
    return float(np.dot(a_arr, b_arr) / (np.linalg.norm(a_arr) * np.linalg.norm(b_arr)))

def retrieve_relevant_chunks(
    query_embedding: list[float],
    chunks: list[Chunk],
    top_k: int = 5,
    max_tokens: int = 2000,
    model: str = "gpt-4o"
) -> list[Chunk]:
    import tiktoken
    enc = tiktoken.encoding_for_model(model)

    scored = sorted(
        chunks,
        key=lambda c: cosine_similarity(query_embedding, c.embedding),
        reverse=True
    )

    selected     = []
    total_tokens = 0

    # Fetch 2x top_k candidates, trim to token budget
    for chunk in scored[:top_k * 2]:
        chunk_tokens = len(enc.encode(chunk.text))
        if total_tokens + chunk_tokens > max_tokens:
            break
        selected.append(chunk)
        total_tokens += chunk_tokens

    return selected

Key observation: top_k is a starting point, not a ceiling. Always apply a secondary token budget — individual chunk sizes vary, and a fixed top_k can blow your context limit without warning.

Context Packing: Order and Structure Matter

Beyond which content to include, the order within the context window affects both cost and quality. A reliable structure:

[System prompt — static, always first]
[Retrieved context — RAG chunks, relevant docs]
[Compressed history summary — if applicable]
[Recent conversation turns — last 4-8 messages]
[Current user message]

A few non-obvious rules that trip people up:

Keep system prompts static — any variation between calls disables prompt caching, which can meaningfully cut costs on high-volume deployments
Do not inject user data into the system prompt — it defeats caching and introduces a prompt-injection attack surface
Separate tool schemas from instructions — some APIs cache tool definitions independently; mixing them into the main prompt body resets the cache

Measuring What You Actually Send

You cannot optimize what you do not measure. A simple decorator logs token usage per call:

import logging
from functools import wraps

logger = logging.getLogger(__name__)

def log_token_usage(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        response = func(*args, **kwargs)
        if hasattr(response, "usage"):
            logger.info(
                "llm_call_tokens",
                extra={
                    "prompt_tokens":     response.usage.prompt_tokens,
                    "completion_tokens": response.usage.completion_tokens,
                    "total_tokens":      response.usage.total_tokens,
                }
            )
        return response
    return wrapper

Feed this into whatever observability stack you use. Over time, the distribution of prompt token counts reveals whether your management strategies are working — or whether certain query patterns are silently bypassing them and sending far more than intended.

The Takeaway

Context window management is not a set-it-and-forget-it problem. The right strategy depends on your workload:

Short chat sessions: a sliding window with a hard token cap is sufficient
Long-running sessions with continuity requirements: periodic summarization keeps coherence without unbounded growth
Document or codebase Q&A: chunked retrieval with a secondary token budget cap
All production deployments: instrument token usage from day one

All three patterns compose naturally — sliding window for recent turns, summarization for older history, RAG for external documents. The added complexity pays off once you have been paged at 2 am because an LLM call returned a context overflow error on a payload that always worked fine in testing.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Building a Multi-Step AI Pipeline with Automatic Retry Logic

Ayi NEDJIMI — Thu, 18 Jun 2026 10:03:56 +0000

When you chain language model calls into a pipeline, things break in unexpected ways. A step times out, the API returns a 429, or the model outputs malformed JSON that crashes your parser two steps later. Without deliberate retry logic, one bad response kills the whole run — and re-running from scratch wastes time and money. This article walks through building a multi-step LLM pipeline in Python with per-step retry, exponential backoff, and clean error propagation.

What a Multi-Step AI Pipeline Looks Like

A pipeline here means: take raw input → process through N language model calls, each depending on the previous → produce a final structured result. A realistic example:

Extract entities from a user document
Classify each entity by type and risk level
Summarize the findings into a final report

Each step has its own prompt, its own expected output format, and its own failure modes. Step 2 failing should not force you to re-run step 1. You need per-step isolation, not a single try/except wrapped around the whole chain.

The Naive Approach and Why It Fails

Most tutorials wrap every call in a try/except and call it done:

def call_llm(prompt: str) -> str:
    try:
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}]
        )
        return response.choices[0].message.content
    except Exception as e:
        raise RuntimeError(f"LLM call failed: {e}")

This silently swallows all failures in a single bucket. A transient 503 and a hard authentication error look identical. You cannot retry one without retrying the other, and you cannot tell which step caused the failure downstream.

The real failure mode: a 429 at step 3 after 45 seconds of processing means you either crash the whole run or retry from step 1. Neither is acceptable in production.

Per-Step Retry with Exponential Backoff

The right pattern gives each step its own retry budget. Here is a decorator that does it:

import time
import random
from functools import wraps
from typing import Callable, TypeVar

T = TypeVar("T")

def with_retry(
    max_attempts: int = 3,
    base_delay: float = 1.0,
    max_delay: float = 30.0,
    retryable_exceptions: tuple = (TimeoutError, ConnectionError),
):
    def decorator(fn: Callable[..., T]) -> Callable[..., T]:
        @wraps(fn)
        def wrapper(*args, **kwargs) -> T:
            for attempt in range(max_attempts):
                try:
                    return fn(*args, **kwargs)
                except retryable_exceptions as e:
                    if attempt + 1 >= max_attempts:
                        raise
                    delay = min(
                        base_delay * (2 ** (attempt + 1)) + random.uniform(0, 1),
                        max_delay,
                    )
                    print(f"[retry] {fn.__name__} attempt {attempt+1}/{max_attempts} in {delay:.1f}s: {e}")
                    time.sleep(delay)
        return wrapper
    return decorator

The jitter (random.uniform(0, 1)) prevents thundering herd when multiple pipeline instances retry simultaneously after an outage. Without it, all instances wake up at the same moment and hit the API together.

Usage:

@with_retry(max_attempts=3, retryable_exceptions=(TimeoutError, ConnectionError))
def extract_entities(text: str) -> list[str]:
    # your LLM call here, returns a parsed list
    ...

Tracking Pipeline State

Beyond per-step retry, you need to know where a pipeline failed and what each step produced. A simple state object makes this explicit:

from dataclasses import dataclass, field
from typing import Optional, Any

@dataclass
class StepResult:
    name: str
    success: bool
    output: Optional[Any] = None
    error: Optional[str] = None
    attempts: int = 0

@dataclass
class PipelineState:
    input: str
    steps: list[StepResult] = field(default_factory=list)

    def last_output(self) -> Any:
        for step in reversed(self.steps):
            if step.success:
                return step.output
        return self.input

    def first_failure(self) -> Optional[StepResult]:
        return next((s for s in self.steps if not s.success), None)

Then a run_step helper that wraps retry logic and records the outcome:

import httpx

RETRYABLE = (httpx.TimeoutException, httpx.ConnectError, httpx.RemoteProtocolError)

def run_step(state: PipelineState, name: str, fn, *args) -> PipelineState:
    for attempt in range(3):
        try:
            result = fn(*args)
            state.steps.append(StepResult(name, success=True, output=result, attempts=attempt + 1))
            return state
        except RETRYABLE as e:
            if attempt == 2:
                state.steps.append(StepResult(name, success=False, error=str(e), attempts=3))
                return state
            time.sleep(min(1.0 * (2 ** (attempt + 1)), 30))
        except Exception as e:
            # Non-retryable: parse failures, auth errors, validation issues
            state.steps.append(StepResult(name, success=False, error=str(e), attempts=attempt + 1))
            return state
    return state

Validation errors like json.JSONDecodeError fall through to the non-retryable branch. Retrying a step that produced malformed output with the same prompt rarely fixes anything — you need a different prompt or a fallback parser.

Wiring the Full Pipeline

def run_pipeline(raw_input: str) -> str:
    state = PipelineState(input=raw_input)

    state = run_step(state, "extract", extract_entities, raw_input)
    if state.first_failure():
        raise RuntimeError(f"Pipeline aborted at 'extract': {state.first_failure().error}")

    state = run_step(state, "classify", classify_entities, state.last_output())
    if state.first_failure():
        raise RuntimeError(f"Pipeline aborted at 'classify': {state.first_failure().error}")

    state = run_step(state, "summarize", summarize, state.last_output())
    if state.first_failure():
        raise RuntimeError(f"Pipeline aborted at 'summarize': {state.first_failure().error}")

    return state.last_output()

Each step failure short-circuits with a precise message: which step failed, what error was raised, and how many attempts were made before giving up.

Handling Rate Limits Correctly

LLM APIs return HTTP 429 with a Retry-After header when you hit quota. Ignoring this header and using fixed backoff can get your account throttled further. Read it:

import httpx

def call_with_rate_limit_awareness(prompt: str, client) -> str:
    for attempt in range(4):
        try:
            return client.complete(prompt)
        except httpx.HTTPStatusError as e:
            status = e.response.status_code
            if status == 429:
                wait = int(e.response.headers.get("retry-after", 5))
                print(f"Rate limited. Respecting retry-after: {wait}s")
                time.sleep(wait)
            elif status >= 500:
                time.sleep(2 ** attempt)
            else:
                raise  # 4xx that is not 429: request is broken, do not retry
    raise RuntimeError("Exceeded retry budget")

The key distinction: 4xx errors (except 429) mean your request is malformed or unauthorized. Retrying will not fix them. 5xx and 429 are transient — retry with respect for server-side signals.

The Takeaway

Multi-step LLM pipelines fail in predictable, manageable ways. The pattern here — per-step retry with jittered backoff, explicit state tracking, and differentiated error handling — covers the common failure modes without overengineering.

What matters most:

Retry transient errors only — do not retry parse failures or auth errors
Track state per step — know exactly where and why a pipeline failed
Honor Retry-After headers — ignoring them makes throttling worse

In production, add structured logging per step and consider persisting PipelineState to a database for runs longer than a few seconds. This lets you resume mid-run after a crash instead of starting over from step 1.

If you are hardening the security layer around your LLM stack, our free security hardening checklists cover API key rotation, rate limit handling, and output validation patterns for production deployments.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

How to Evaluate LLM Output Quality Programmatically

Ayi NEDJIMI — Tue, 16 Jun 2026 10:05:57 +0000

When you ship an LLM-powered feature, "does it work?" is not a binary question. An answer can be grammatically correct, topically on-point, factually wrong, and subtly biased — all at the same time. Without a systematic way to measure output quality, regressions silently creep in with every model update, prompt tweak, or temperature change. By the time users complain, you've already lost trust.

This article walks through a practical evaluation framework: what to measure, how to automate it, and how to integrate quality gates into your CI/CD pipeline.

Why Manual Testing Doesn't Scale

The typical early-stage approach is to write 10–20 example prompts, run them by hand, and eyeball the outputs. This works until you change the prompt — then you need to re-run everything. It breaks entirely when you swap models or add a retrieval layer.

The problems compound quickly:

Inconsistency: Different reviewers apply different quality bars. Even the same person reviews differently on Monday morning versus Friday afternoon.
Coverage: You test the happy path. Edge cases only surface in production, where they cause visible failures.
Speed: Manual review doesn't fit in a CI pipeline. You can't block a merge because someone needs to manually evaluate 40 outputs.

The solution is a repeatable test suite that runs automatically and produces a numeric score you can trend over time.

What to Actually Measure

Not every quality dimension matters equally. Start with the metrics that map directly to user pain:

Factual correctness — did the model get the facts right? This requires ground-truth answers and is the hardest to automate, but it's the most important for high-stakes applications.

Relevance — is the output actually responsive to the input? A perfectly written paragraph that ignores the actual question is useless, regardless of how fluent it sounds.

Format compliance — if you asked for JSON, did you get valid JSON? If you asked for a numbered list, did you get prose instead? Format failures break downstream parsing and are trivially easy to detect.

Verbosity — outputs that are 10x longer than needed cost tokens, increase latency, and frustrate users. Outputs that are too short are often missing substance.

Groundedness (for RAG pipelines) — does the answer reference only facts present in the retrieved context? Answers that go beyond the source documents are fabrications, even when they sound plausible.

For a first pass, format compliance and verbosity are the easiest to automate with zero ground-truth data. Start there before tackling the harder problems.

Building an Evaluation Harness in Python

Here's a minimal but useful evaluation harness. It defines test cases with expected format and length constraints, runs them against your model, and computes a per-case score:

import json
import re
from dataclasses import dataclass, field
from typing import Callable

@dataclass
class TestCase:
    name: str
    prompt: str
    expected_format: str  # "json", "list", "prose"
    max_words: int = 300
    min_words: int = 10
    validator: Callable[[str], bool] | None = None

@dataclass
class EvalResult:
    case_name: str
    output: str
    format_ok: bool
    length_ok: bool
    custom_ok: bool | None
    score: float = field(init=False)

    def __post_init__(self):
        checks = [self.format_ok, self.length_ok]
        if self.custom_ok is not None:
            checks.append(self.custom_ok)
        self.score = sum(checks) / len(checks)

def check_format(output: str, expected: str) -> bool:
    if expected == "json":
        try:
            json.loads(output)
            return True
        except json.JSONDecodeError:
            return False
    elif expected == "list":
        return bool(re.search(r"^(\s*[-*]\s+|\d+\.\s+)", output, re.MULTILINE))
    return True  # prose format has no structural constraint

def evaluate(case: TestCase, output: str) -> EvalResult:
    word_count = len(output.split())
    return EvalResult(
        case_name=case.name,
        output=output,
        format_ok=check_format(output, case.expected_format),
        length_ok=case.min_words <= word_count <= case.max_words,
        custom_ok=case.validator(output) if case.validator else None,
    )

def run_suite(cases: list[TestCase], model_fn: Callable[[str], str]) -> list[EvalResult]:
    results = []
    for case in cases:
        output = model_fn(case.prompt)
        results.append(evaluate(case, output))
    avg = sum(r.score for r in results) / len(results)
    print(f"Suite score: {avg:.2%} ({len(results)} cases)")
    return results

Wire model_fn to your actual model API call. Define test cases that cover your application's key behaviors, and you have a repeatable baseline. Run it in CI: if the average score drops below a threshold, fail the build.

Using a Language Model as a Judge

Format checks are cheap and useful, but they don't catch subtler quality problems: hallucinated facts, unhelpful vagueness, or off-tone responses.

One effective pattern is LLM-as-judge: you pass the original prompt and the model's output to a second language model and ask it to score specific dimensions. This sounds circular, but it works well in practice when:

The judge model is different from (or larger than) the model under test.
The scoring rubric is explicit and narrow.
You're measuring relative quality (A/B comparisons) rather than absolute truth.

import json
import urllib.request

JUDGE_PROMPT = (
    "You are an evaluator. Score the following LLM response on two dimensions.\n\n"
    "Original prompt: {prompt}\n"
    "Response to evaluate: {response}\n\n"
    "Rate each dimension from 1 to 5:\n"
    "- relevance: Does the response actually answer the original prompt?\n"
    "- clarity: Is the response well-structured and easy to understand?\n\n"
    'Return valid JSON only, no explanation:\n'
    '{{"relevance": <int>, "clarity": <int>}}'
)

def judge_output(prompt: str, response: str, api_key: str, model: str) -> dict:
    payload = json.dumps({
        "model": model,
        "messages": [
            {
                "role": "user",
                "content": JUDGE_PROMPT.format(prompt=prompt, response=response),
            }
        ],
        "max_tokens": 100,
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        "https://api.openai.com/v1/chat/completions",
        data=payload,
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        data = json.loads(resp.read())
    return json.loads(data["choices"][0]["message"]["content"])

Keep the judge prompt narrow. Asking a model to simultaneously rate relevance, tone, factual accuracy, and helpfulness in one call produces noisy, inconsistent scores. One or two dimensions per call, with temperature: 0 to make scores deterministic across runs.

Integrating Quality Gates into CI

Once you have scores, you need to act on them. The goal is not to block every deployment when a single test case regresses — that's too brittle. The goal is to catch systematic degradation before it reaches users.

A practical threshold setup:

Suite average score below 0.90 → fail the build
Any individual test case score below 0.60 → add a warning comment to the PR
LLM-judge average relevance below 3.5 → flag for human review before merging

Store scores per git commit to a lightweight JSON file or a time-series database. A score that drifts from 0.97 to 0.89 over 10 commits is actionable even if no individual commit crossed the hard threshold. Trend visibility is what separates monitoring from a one-off check.

One often-overlooked concern: the test inputs and outputs flowing through your evaluation pipeline may contain sensitive data — user queries, PII, internal business logic. The evaluation infrastructure itself becomes an attack surface. Teams formalizing their AI deployment process will find our free security hardening checklists useful — they cover LLM pipeline security alongside traditional application hardening.

The Takeaway

LLM evaluation doesn't need to be complicated to be useful. Start with the cheapest checks: format compliance and length bounds. Add an LLM-as-judge layer for subjective quality dimensions. Integrate both into your CI pipeline with explicit pass/fail thresholds.

The metric you don't measure is the one that bites you in production. A failing format check caught in a 10-second CI run is infinitely less painful than a broken JSON response cascading through a production system at 2am.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Implementing AI-Powered Log Anomaly Detection

Ayi NEDJIMI — Mon, 15 Jun 2026 10:04:52 +0000

Production systems generate thousands of log lines per minute. Most are noise. The signals that matter — a failed authentication spike, an unexpected process crash, a database query that suddenly takes 10x longer — are buried in that noise. Rule-based alerting catches what you anticipated; anomaly detection catches what you didn't.

Why Rule-Based Alerting Falls Short

Writing alert rules requires knowing what failure looks like before it happens. That works for well-understood failure modes but leaves you blind to novel attacks, silent degradation, or emergent issues across services. A statistical model can learn what "normal" looks like from historical data and flag deviations automatically — without you enumerating every failure scenario upfront.

This article shows how to build a working log anomaly detector in Python that:

Parses raw log lines into numeric features
Trains an Isolation Forest on a baseline window
Scores new lines in near real-time and surfaces the most anomalous ones
Optionally routes flagged lines through a language model for triage

Parsing Logs Into Features

Raw log text is the worst possible input for a statistical model. We need numeric features. The simplest useful set includes: log level, HTTP status code, response latency, message length, and a coarse hash of the message template.

import re
import hashlib
from dataclasses import dataclass

LOG_PATTERN = re.compile(
    r'\S+\s+(?P<level>DEBUG|INFO|WARN|ERROR|FATAL)\s+'
    r'(?P<status>\d{3})?\s*(?P<latency_ms>\d+)?ms?\s*(?P<message>.+)'
)

LEVEL_MAP = {"DEBUG": 0, "INFO": 1, "WARN": 2, "ERROR": 3, "FATAL": 4}

@dataclass
class LogFeatures:
    level: int
    status: int
    latency_ms: int
    message_len: int
    message_hash: int

def parse_line(raw: str) -> LogFeatures | None:
    m = LOG_PATTERN.match(raw.strip())
    if not m:
        return None
    return LogFeatures(
        level=LEVEL_MAP.get(m.group("level"), 1),
        status=int(m.group("status") or 200),
        latency_ms=int(m.group("latency_ms") or 0),
        message_len=len(m.group("message")),
        message_hash=int(hashlib.md5(m.group("message").encode()).hexdigest()[:8], 16) % 10_000,
    )

message_hash is a cheap way to bucket log message templates without full NLP. It handles the common case where the same event type repeats with different variable values — the hash groups them while still separating genuinely novel messages from the baseline.

Training an Isolation Forest

Isolation Forest is a strong fit for log anomaly detection: it requires no labeled examples, handles high-dimensional sparse data well, and its anomaly score is easy to interpret. The algorithm isolates anomalies by randomly partitioning the feature space — outliers require fewer partitions because they live in sparse regions.

import numpy as np
from sklearn.ensemble import IsolationForest

def build_matrix(lines: list[str]) -> np.ndarray:
    features = [parse_line(l) for l in lines]
    features = [f for f in features if f is not None]
    return np.array([
        [f.level, f.status, f.latency_ms, f.message_len, f.message_hash]
        for f in features
    ], dtype=float)

def train_detector(baseline_lines: list[str], contamination: float = 0.01):
    X = build_matrix(baseline_lines)
    model = IsolationForest(
        n_estimators=200,
        contamination=contamination,
        random_state=42,
        n_jobs=-1,
    )
    model.fit(X)
    return model

contamination=0.01 tells the model to expect roughly 1% of the baseline to already be anomalous — a defensible prior for most production environments. If your baseline window included an incident, push this to 0.05. If it was an unusually clean period, drop it to 0.005. The parameter is worth measuring rather than guessing: sample 500 lines, review them manually, and set contamination from the actual observed fraction of weird lines.

Scoring Lines and Surfacing Anomalies

Once trained, scoring is fast. predict returns -1 for anomalies and 1 for normal samples. decision_function gives a continuous score — lower values are more anomalous — which lets you prioritize when dozens of anomalies appear at once.

def score_lines(model, lines: list[str]) -> list[dict]:
    parsed = [(l, parse_line(l)) for l in lines]
    valid = [(raw, f) for raw, f in parsed if f is not None]
    if not valid:
        return []

    X = np.array([
        [f.level, f.status, f.latency_ms, f.message_len, f.message_hash]
        for _, f in valid
    ], dtype=float)

    predictions = model.predict(X)
    scores = model.decision_function(X)

    results = []
    for (raw, _), pred, score in zip(valid, predictions, scores):
        if pred == -1:
            results.append({"line": raw.strip(), "score": round(float(score), 4)})

    return sorted(results, key=lambda r: r["score"])

You can pipe results directly into any alerting channel — Slack webhook, PagerDuty, Telegram bot. Sorting by score ascending puts the most extreme outliers first.

Tailing a Live Log File

For production use, wire the detector into a file tail loop with a rolling baseline. Retrain periodically so the model adapts to deployment changes and organic traffic growth.

import time
from collections import deque

BASELINE_MAX = 50_000
RETRAIN_EVERY = 500  # batches

def tail_and_detect(log_path: str, contamination: float = 0.01):
    baseline: deque[str] = deque(maxlen=BASELINE_MAX)
    model = None
    batch_n = 0

    with open(log_path) as fh:
        fh.seek(0, 2)  # start at EOF — only tail new lines
        while True:
            lines = fh.readlines()
            if not lines:
                time.sleep(0.5)
                continue

            for line in lines:
                baseline.append(line)

            if len(baseline) >= 1_000 and model is None:
                model = train_detector(list(baseline), contamination)
                print(f"Model trained on {len(baseline)} lines")

            if model is not None:
                for a in score_lines(model, lines):
                    print(f"[ANOMALY {a['score']:.4f}] {a['line']}")

            batch_n += 1
            if model is not None and batch_n % RETRAIN_EVERY == 0:
                model = train_detector(list(baseline), contamination)
                print("Model retrained on rolling baseline")

In a real deployment, replace print with structured JSON output, add a watchdog that alerts if the detector stops processing (a silent failure is worse than no detector), and persist the trained model to disk so restarts don't lose the accumulated baseline.

Optional: LLM-Based Triage

The Isolation Forest flags outliers but cannot explain them in plain language. For the most severe anomalies, route the flagged line and its surrounding context to a language model for a human-readable summary:

def triage_anomaly(line: str, context: list[str], llm_client) -> str:
    prompt = (
        "You are an SRE reviewing a log anomaly.\n\n"
        f"Context (10 lines before):\n{''.join(context[-10:])}\n\n"
        f"Anomalous line:\n{line}\n\n"
        "In 2-3 sentences: what is likely happening and how urgent is it?"
    )
    # Works with any chat-completions-compatible API (OpenAI, Ollama, Mistral, etc.)
    resp = llm_client.chat.completions.create(
        model="your-model-id",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=200,
    )
    return resp.choices[0].message.content

This two-stage pattern — statistical detection first, LLM triage second — keeps costs under control. You only spend LLM tokens on the small fraction of lines flagged as anomalous, not the full stream. On a service generating 10k lines/minute with a 0.5% anomaly rate, that's at most 50 LLM calls per minute.

For teams building security-aware systems, it is worth auditing your logging and monitoring posture before shipping. We publish free security hardening checklists covering log retention policies, alerting gaps, and incident response readiness.

The Takeaway

This architecture is production-ready today with no exotic dependencies — just scikit-learn and Python's standard library for the core detector:

Parse into features first. Do not pass raw log text to statistical models; numeric features are orders of magnitude more effective.
Use a rolling baseline. Log patterns shift with every deployment. A static baseline from three months ago generates too many false positives.
Tune contamination from measurement. Sample your actual logs, review them, then set the parameter from real data rather than intuition.
Reserve LLM calls for triage. Detection must be fast and cheap; explanation can afford to be slower.
Add a watchdog. A detector that silently stops processing is worse than having no detector at all.

The full pipeline runs under 200 lines of Python, has no database dependency, and handles tens of thousands of log lines per second on a single core.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Pytest vs unittest vs hypothesis: Python testing frameworks in 2026

Ayi NEDJIMI — Sun, 14 Jun 2026 10:05:26 +0000

You pick a testing framework when you start a project and rarely revisit the decision. Pick the wrong one and you end up refactoring your entire test suite six months later — or shipping bugs that your tests structurally cannot catch. In 2026, three frameworks dominate the Python testing landscape: unittest (standard library), pytest (ecosystem default), and hypothesis (property-based testing). Here is what each one actually does, where it falls short, and how to combine them effectively.

unittest: the built-in baseline

unittest ships with Python. No installation required. It follows the xUnit pattern: subclass TestCase, prefix methods with test_, and use self.assert* helpers.

import unittest
from myapp.auth import TokenValidator, TokenExpiredError

class TokenValidatorTest(unittest.TestCase):

    def setUp(self):
        self.validator = TokenValidator(secret="s3cr3t", max_age=3600)

    def test_valid_token_passes(self):
        token = self.validator.sign({"user_id": 42})
        result = self.validator.verify(token)
        self.assertEqual(result["user_id"], 42)

    def test_expired_token_raises(self):
        token = self.validator.sign({"user_id": 42}, issued_at=-7200)
        with self.assertRaises(TokenExpiredError):
            self.validator.verify(token)

if __name__ == "__main__":
    unittest.main()

Where it shines: zero-dependency libraries, corporate environments where the approved package list is locked down, or codebases that already use it and don't justify migration cost.

Where it falls short: sharing fixtures across modules requires careful use of setUpModule or test inheritance hierarchies that grow awkward fast. Parametrization is verbose. Assertion failure output is minimal compared to what pytest produces.

The ergonomics gap is real. A failing self.assertEqual(expected, result) shows you a diff. A plain assert expected == result in raw Python gives you AssertionError with no context. Pytest solves this with rewritten assertion introspection.

Pytest: where most projects land

Pytest's key features are fixture injection with configurable scope, smart assertion introspection, and @pytest.mark.parametrize. No boilerplate class required:

import pytest
from myapp.auth import TokenValidator, TokenExpiredError

@pytest.fixture(scope="module")
def validator():
    return TokenValidator(secret="s3cr3t", max_age=3600)

def test_valid_token_passes(validator):
    token = validator.sign({"user_id": 42})
    assert validator.verify(token)["user_id"] == 42

def test_expired_token_raises(validator):
    token = validator.sign({"user_id": 42}, issued_at=-7200)
    with pytest.raises(TokenExpiredError):
        validator.verify(token)

@pytest.mark.parametrize("payload,expected_error", [
    ({"user_id": None}, ValueError),
    ({}, KeyError),
    ({"user_id": "not-an-int"}, TypeError),
])
def test_invalid_payloads(validator, payload, expected_error):
    with pytest.raises(expected_error):
        validator.sign(payload)

Fixtures are composable and scoped at function, class, module, or session level. A db_session fixture can depend on a db_engine fixture without any global state. This scales cleanly across hundreds of tests without duplication.

Pytest runs unittest-based tests transparently, so incremental migration is painless. The plugin ecosystem is extensive — pytest-asyncio for async code, pytest-django for Django, pytest-xdist for parallel execution:

# Run tests in parallel across all CPU cores
pytest -n auto tests/

# Run only last-failed tests, then the full suite
pytest --lf tests/

Performance note: pytest's collection overhead becomes noticeable past 10k tests. Use -x (exit on first failure) during development and --lf for quick iteration. Save full suite runs for CI.

Hypothesis: what example-based tests structurally miss

The fundamental problem with hand-written test cases is that you only test what you think of. Hypothesis inverts this: you describe the shape of valid inputs and the library generates thousands of them — including boundary conditions you would never write manually.

from hypothesis import given, settings
from hypothesis import strategies as st
from myapp.auth import TokenValidator

validator = TokenValidator(secret="s3cr3t", max_age=3600)

@given(
    user_id=st.integers(min_value=1, max_value=2**31 - 1),
    roles=st.lists(
        st.sampled_from(["admin", "user", "guest"]),
        min_size=1,
        max_size=10
    )
)
@settings(max_examples=500)
def test_round_trip_never_corrupts_payload(user_id, roles):
    payload = {"user_id": user_id, "roles": roles}
    token = validator.sign(payload)
    result = validator.verify(token)
    assert result["user_id"] == user_id
    assert result["roles"] == roles

When Hypothesis finds a failing case, it shrinks the input to the minimal reproduction. Instead of user_id=1_732_891_043, roles=["admin","user","guest","user","admin"], it reduces to user_id=1, roles=["admin"]. The shrunk example is persisted in a local database (.hypothesis/) and replays on every subsequent run.

This catches real classes of bugs: integer overflow when a user_id exceeds a 32-bit signed boundary, Unicode normalization issues when a role name contains combining characters, off-by-one in expiry logic when max_age exactly equals the time offset. These are precisely the kinds of inputs attackers probe for manually.

Hypothesis integrates directly with pytest — @given decorates a standard def test_ function and failures surface as normal pytest output.

When to use what

Keep unittest if you have an existing test suite where migration cost isn't justified, or if you're writing a library that must ship with zero additional dependencies.

Default to pytest for everything new. The ergonomics, fixture system, and plugin ecosystem are significantly better. It also runs existing unittest tests transparently.

Add hypothesis on top of pytest for any module that processes external input — parsers, serializers, authentication helpers, API input validators. It does not replace example-based tests; it finds the bugs your examples can't.

A practical configuration for production projects:

# pyproject.toml
[tool.pytest.ini_options]
addopts = "-ra -q --tb=short"
testpaths = ["tests"]

[tool.hypothesis]
max_examples = 100
database = ".hypothesis"

In CI, increase coverage: HYPOTHESIS_MAX_EXAMPLES=1000 pytest tests/.

Security-critical code deserves property-based tests

If your codebase includes token validation, permission checks, input sanitization, or any authentication logic, hypothesis belongs in your test suite. The security hardening checklists we publish include property-based test coverage as a requirement for auth and authorization modules — alongside SAST tooling and manual review. Example-based tests systematically miss integer boundary conditions and encoding edge cases. Those are precisely the inputs attackers try first.

The takeaway

Don't overthink the initial choice: start with pytest. Add hypothesis to any module that handles external input or implements security-sensitive logic. Keep unittest only where you already have it.

The combination of pytest + hypothesis + pytest-cov covers 95% of what production Python projects need. Install pytest-xdist once test runtime crosses two minutes. Everything else is noise.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Docker vs Podman vs nerdctl: Container Runtimes for Go Developers

Ayi NEDJIMI — Sat, 13 Jun 2026 10:02:51 +0000

When you containerize a Go service, the runtime choice is usually an afterthought — until something breaks in CI or a security audit flags your daemon running as root. Docker, Podman, and nerdctl each make different trade-offs around privilege, daemon architecture, and OCI compatibility. This article compares them from a Go developer's perspective.

What makes these runtimes different

All three can build and run OCI-compliant container images. The divergence is in how they do it.

Docker runs a daemon (dockerd) that listens on a Unix socket owned by root. The Docker CLI talks to this daemon. On Linux, any user in the docker group can send arbitrary commands to that socket — which is functionally equivalent to root access on the host. Docker Desktop on macOS/Windows wraps this in a VM, but on Linux servers and in CI, it is a real privilege escalation path.

Podman is daemonless. Each podman invocation forks the container process directly. It defaults to rootless — containers run as your user, mapped into a user namespace. There is no long-lived daemon to attack. The CLI is intentionally Docker-compatible, so aliasing docker=podman works for most workflows.

nerdctl is a thin CLI built by the containerd project. It wraps containerd directly instead of going through dockerd. Containerd is what Docker uses under the hood anyway, so nerdctl gives you direct access to that layer. It supports rootless via rootlesskit and accepts the same Dockerfiles and image formats.

Building a Go binary in a container

The Dockerfile is identical across all three runtimes:

FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /bin/server ./cmd/server

FROM scratch
COPY --from=builder /bin/server /server
EXPOSE 8080
ENTRYPOINT ["/server"]

The build commands are nearly identical:

# Docker
docker build -t myservice:latest .

# Podman — daemonless, rootless by default
podman build -t myservice:latest .

# nerdctl — wraps containerd directly
nerdctl build -t myservice:latest .

The scratch base image plus a statically compiled Go binary results in a final image that is just your binary: no shell, no package manager, minimal attack surface. All three runtimes handle multi-stage builds identically.

Rootless containers and what they actually change

Rootless is not a security checkbox — it changes what an attacker lands as if a container escape happens.

In rootless mode (Podman's default, opt-in for Docker and nerdctl), container processes run as the invoking user. A user namespace maps UID 0 inside the container to a non-privileged UID on the host. A container escape puts an attacker inside an unprivileged user namespace.

With Docker's default setup, the daemon socket is owned by root. Any member of the docker group can mount host paths, read host filesystems, and run privileged containers. The privilege escalation path is short.

Enabling rootless Docker requires an explicit setup step:

dockerd-rootless-setuptool.sh install
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock
docker run --rm golang:1.22-alpine go version

Podman runs rootless without any configuration:

# Runs as your user, no daemon, no root
podman run --rm golang:1.22-alpine go version

If you are reviewing your container security posture, the container hardening checklists at ayinedjimi-consultants.fr/checklists cover rootless configuration, seccomp profiles, and capability dropping in a format you can run against your own deployments.

CI/CD integration for Go projects

This is where the practical differences matter most.

Docker in GitHub Actions usually requires Docker-in-Docker (DinD) with --privileged, or socket mounting — both are risky in shared CI environments.

# GitHub Actions — Docker, daemon-based
- name: Build
  uses: docker/build-push-action@v5
  with:
    context: .
    push: false
    tags: myservice:latest

Podman in GitHub Actions does not need a daemon. Ubuntu runners have Podman pre-installed:

# GitHub Actions — Podman, daemonless
- name: Build with Podman
  run: |
    podman build -t myservice:latest .
    podman run --rm myservice:latest /server --version

No privileged flag, no socket mounts. Podman forks container processes directly under the runner user. For Go projects that compile to a static binary and run smoke tests in CI, this is the lowest-friction secure option.

nerdctl with containerd fits naturally in Kubernetes-adjacent CI like kind or k3s, where containerd is already the cluster runtime. If your CI nodes run containerd, nerdctl avoids adding a separate daemon:

# GitHub Actions — nerdctl on a containerd-backed runner
- name: Build with nerdctl
  run: |
    nerdctl build -t myservice:latest .
    nerdctl run --rm myservice:latest /server --version

One gotcha with nerdctl: it uses containerd namespaces (defaults to default). If the same node runs a Kubernetes cluster, images built by nerdctl are not visible to the kubelet unless you specify the k8s.io namespace explicitly: nerdctl --namespace k8s.io build ....

Compose workflows for local development

Go services almost always need dependencies — Postgres, Redis, a mock OAuth server. For local compose workflows:

Docker: docker compose (v2, built-in plugin) — widest compatibility
Podman: podman compose (wraps docker-compose binary) or podman-compose (Python package) — some gaps with complex volume and network configs
nerdctl: nerdctl compose (built-in, Docker Compose v2 syntax) — solid compatibility

In practice, nerdctl compose has the best parity with existing docker-compose.yml files after Docker itself. Podman-compose works for straightforward setups but can break on edge cases like network_mode: host and certain healthcheck configurations.

Which one to pick

	Docker	Podman	nerdctl
Rootless by default	No	Yes	No (opt-in)
Daemon required	Yes	No	Yes (containerd)
CI without privileges	Harder	Easy	Medium
Compose compatibility	Best	Partial	Good
Kubernetes integration	Via Docker Desktop	Via podman-remote	Native (containerd)

For Go developers building microservices who care about the security of their build pipeline: Podman's rootless-by-default model eliminates the daemon socket risk with minimal migration cost. The CLI compatibility means alias docker=podman in your shell profile covers most local workflows.

For teams embedded in the Kubernetes ecosystem where containerd is already the node runtime: nerdctl gives direct access to what is already running, with no added daemon layer.

Docker remains the default for good reasons — ecosystem depth, tooling, documentation. But if a security reviewer asks why your CI pipeline runs a root-owned socket, Podman is the answer that requires the least justification.

The takeaway

The container image format is the same across all three. The divergence is in privilege model, daemon architecture, and CI integration patterns. For Go services compiled to static binaries and deployed from scratch images, the runtime is almost invisible at build time — until it is not. If your current setup has the docker group granting implicit root on CI nodes, switching to Podman costs an afternoon and removes a real attack vector.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Cloudflare vs Nginx vs Caddy: Reverse Proxy for Developers in 2026

Ayi NEDJIMI — Fri, 12 Jun 2026 10:02:18 +0000

Your service is running. Now you need something in front of it: TLS termination, routing, rate limiting, maybe auth headers. Three tools come up in every conversation — Nginx, Caddy, and Cloudflare. They solve the same surface-level problem but with very different tradeoffs. Picking the wrong one costs you either operational complexity or flexibility you'll need in six months.

What we're actually comparing

These tools aren't equivalents. Nginx and Caddy are software reverse proxies you deploy and operate yourself. Cloudflare is a managed edge network — your traffic goes through their infrastructure before reaching your origin. That distinction matters for latency, security posture, and what you can control.

	Nginx	Caddy	Cloudflare
Hosting	Self-hosted	Self-hosted	Managed (SaaS)
TLS	Manual (certbot)	Automatic	Automatic
Config format	Custom DSL	Caddyfile or JSON	Dashboard / Terraform
Performance ceiling	Very high	High	Depends on plan
Pricing	Free	Free (BSL license)	Free to expensive

Let's go through each with real configuration examples.

Nginx: the workhorse that never surprises you

Nginx has been the default reverse proxy for over a decade. It's fast, well-documented, and every DevOps engineer knows how to debug it. The configuration syntax is verbose but explicit — which is either a feature or a flaw depending on your tolerance for ceremony.

A typical Nginx config for proxying a Go or Python backend with HTTPS:

server {
    listen 80;
    server_name api.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate     /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_ciphers         HIGH:!aNULL:!MD5;

    location / {
        proxy_pass         http://127.0.0.1:8080;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_read_timeout 30s;
        proxy_buffering    off;
    }
}

You'll still need certbot and a systemd timer for certificate renewal. Add nginx_status and scrape it with Prometheus for metrics. Add limit_req_zone for rate limiting. Everything is possible but nothing is automatic.

Where Nginx wins: high-traffic production systems where you need precise control, environments where your team already operates it, and anywhere institutional knowledge matters more than convenience.

Where it hurts: new projects where you don't want to think about certs, or any setup where ops overhead isn't justified by the traffic.

Caddy: automatic HTTPS and a sane config format

Caddy's main selling point is automatic TLS via ACME (Let's Encrypt by default). Zero-touch certificate issuance and renewal. That alone eliminates a common failure mode — the cert that expires because the renewal cron job was silently broken for two weeks.

The equivalent reverse proxy setup in a Caddyfile:

api.example.com {
    reverse_proxy localhost:8080 {
        header_up Host {host}
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-Proto {scheme}
    }

    log {
        output file /var/log/caddy/access.log
        format json
    }

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options nosniff
        X-Frame-Options DENY
        -Server
    }
}

No certbot. No cron. No ssl_certificate path to manage. Caddy handles OCSP stapling, automatic HTTP→HTTPS redirects, and certificate renewal out of the box. The security headers block is also zero-cost to add — no separate module needed.

Caddy also exposes a JSON API for dynamic config updates, which is useful if you're building a control plane on top of it. The Caddyfile compiles to JSON internally, so both interfaces are first-class.

One thing to know: Caddy changed its license from Apache 2.0 to the BSL (Business Source License) in 2024. Free for most use cases, but read the terms if you're distributing it as part of a commercial product.

Where Caddy wins: developer-operated production systems, small teams, projects where TLS lifecycle management shouldn't be anyone's job, and situations where you want security headers with zero boilerplate.

Where it hurts: very high throughput (Nginx's event loop still has an edge at extreme load), or organizations with compliance requirements around third-party license review.

Cloudflare: when the edge is the product

Cloudflare isn't a reverse proxy in the traditional sense — it's a CDN/WAF/edge network that can act like one. Your DNS points to Cloudflare, and they proxy requests to your origin. The security features are substantial: DDoS protection, bot management, WAF rules, and certificate management happen at their edge, not on your server.

For many teams, this changes the security conversation. Instead of hardening your origin's surface area yourself, you're managing Cloudflare rules. The tradeoff is dependency on their availability and pricing, and your traffic passes through their infrastructure — which is fine until it isn't, or until your compliance team asks where TLS terminates.

The free tier covers most small production workloads. But meaningful features — Advanced Bot Protection, custom WAF rules at scale, Argo routing — require paid plans that add up quickly.

If you're using Cloudflare, lock your origin to accept connections only from Cloudflare's published IP ranges, and enable authenticated origin pull certificates. Cloudflare being in the path doesn't replace origin hardening — it just moves the perimeter. For a practical checklist of what to configure on both layers, see our security hardening checklists.

Where Cloudflare wins: global audiences needing CDN, teams without dedicated ops, or anyone who needs DDoS mitigation without running their own infrastructure.

Where it hurts: latency-sensitive applications where an extra network hop matters, regulated environments where data sovereignty means traffic can't traverse a third-party network, or teams that need full observability into raw request data at the edge.

The decision matrix

Don't choose based on what's familiar. Choose based on what breaks in your specific failure mode:

Scenario	Pick
Small to medium production, team of 1–5	Caddy
High-traffic, ops team present	Nginx
Global audience, no ops overhead	Cloudflare
mTLS between internal services	Nginx or Caddy
WAF + DDoS without dedicated infra	Cloudflare
Compliance requires self-hosted TLS termination	Nginx or Caddy
Side project or internal tool	Caddy
Already running Nginx in prod	Keep Nginx

There's rarely a wrong answer, only a wrong context. The mistake teams make is choosing based on habit — using Nginx because "everyone uses Nginx" or defaulting to Cloudflare because the free tier is convenient.

The takeaway

In 2026, Caddy has earned its place as the default for most developer-operated production systems. Automatic TLS, a clean config format, sensible defaults for security headers, and enough performance for the vast majority of workloads. Nginx is still the right call when you need maximum control, extreme throughput, or have a team already operating it. Cloudflare belongs in your stack when you need global edge features and are comfortable with the external dependency and pricing model.

The right question isn't "which is best" — it's "which failure mode am I most willing to own."

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Vault vs AWS Secrets Manager vs Doppler: Secrets Management Compared

Ayi NEDJIMI — Thu, 11 Jun 2026 10:02:47 +0000

Hardcoded credentials in source code remain one of the leading causes of cloud breaches. The fix sounds simple — use a secrets manager — but picking between Vault, AWS Secrets Manager, and Doppler involves real trade-offs in operational complexity, cost, and developer workflow friction.

What You're Actually Comparing

These three tools represent different philosophies, not just different implementations:

HashiCorp Vault: open-source, self-hosted (or managed via HCP Vault), feature-rich, with a steep operational learning curve.
AWS Secrets Manager: fully managed, deeply integrated with IAM, priced per secret and per API call.
Doppler: developer-focused SaaS that treats secrets like environment variables with proper access control.

Comparing them on price alone misses the point. The real questions are: who manages the infrastructure, how granular is your access control, and how painful is onboarding a new developer?

Developer Experience and Setup

Vault requires the most upfront work. You spin up a server (or use vault server -dev locally), choose an auth method (AppRole, Kubernetes, AWS IAM, LDAP...), write policies, and define secret mounts. For teams with dedicated infrastructure engineers, this is manageable. For a three-person startup, it's often overkill.

AWS Secrets Manager has near-zero setup if you're already on AWS. You create a secret via the console or CLI, attach an IAM policy to your service role, and start reading. The friction is the IAM complexity you already have — not something Secrets Manager adds.

Doppler's pitch is "you shouldn't need to read documentation." You sign up, create a project, add secrets in a UI that looks like a spreadsheet, and inject them into your process with doppler run -- python app.py. Secrets are scoped by environment (dev/staging/prod) by default.

Here's how fetching a secret looks in Python for each:

# Vault — using hvac
import hvac, os

client = hvac.Client(url=os.environ["VAULT_ADDR"], token=os.environ["VAULT_TOKEN"])
secret = client.secrets.kv.v2.read_secret_version(path="myapp/db", mount_point="secret")
password = secret["data"]["data"]["password"]

# AWS Secrets Manager — using boto3
import boto3, json

def get_secret(name: str, region: str = "eu-west-1") -> dict:
    client = boto3.client("secretsmanager", region_name=region)
    return json.loads(client.get_secret_value(SecretId=name)["SecretString"])

password = get_secret("prod/myapp/db")["password"]

# Doppler — environment variables injected by the CLI
import os
password = os.environ["DB_PASSWORD"]

# Or programmatically via the REST API
import httpx

def get_doppler_secret(name: str, token: str) -> str:
    resp = httpx.get(
        "https://api.doppler.com/v3/configs/config/secret",
        params={"name": name},
        headers={"authorization": f"Bearer {token}"},
    )
    resp.raise_for_status()
    return resp.json()["secret"]["value"]["raw"]

Secret Rotation and Dynamic Credentials

This is where Vault separates itself from the pack. Vault supports dynamic secrets — instead of storing a fixed password for your database, Vault generates fresh credentials on demand with a configurable TTL. When the lease expires, Vault revokes them automatically. There's no rotation job to write or schedule.

Here's how you'd request dynamic database credentials in Go:

package main

import (
    "context"
    "os"

    vault "github.com/hashicorp/vault-client-go"
)

func getDBCredentials(ctx context.Context) (string, string, error) {
    client, err := vault.New(
        vault.WithAddress(os.Getenv("VAULT_ADDR")),
    )
    if err != nil {
        return "", "", err
    }
    client.SetToken(os.Getenv("VAULT_TOKEN"))

    resp, err := client.Secrets.DatabaseGenerateCredentials(
        ctx,
        "myapp-role",
        vault.WithMountPath("database"),
    )
    if err != nil {
        return "", "", err
    }

    return resp.Data["username"].(string), resp.Data["password"].(string), nil
}

AWS Secrets Manager supports automatic rotation via Lambda. AWS provides ready-made templates for RDS, Redshift, and DocumentDB. It works well within the AWS ecosystem, but rotation is schedule-based rather than on-demand — your app still caches a long-lived secret between rotation events and must handle the brief window when old credentials expire.

Doppler has no built-in rotation mechanism. You update secrets manually or via API, and Doppler syncs the new value to connected integrations (GitHub Actions secrets, Kubernetes Secrets, Vercel env vars, etc.). For static API keys that change infrequently, this is fine. For database passwords or short-lived tokens, you need a rotation layer upstream.

Cost and Operational Overhead

	Vault (self-hosted)	AWS Secrets Manager	Doppler
Base infrastructure	Your servers	None	SaaS (~$6/user/month)
Per-secret cost	None	$0.40/secret/month	None
Per-call cost	None	$0.05/10k API calls	None
Operational burden	High	Low	None
Cross-cloud	Yes	AWS only	Yes
Dynamic secrets	Yes	No	No
Audit logs	Yes (audit device)	CloudTrail	Yes (paid tier)

At 100 secrets and 2M API calls per month on AWS Secrets Manager, you're looking at roughly $50/month before Lambda rotation costs. Self-hosted Vault with HA adds on-call burden for upgrades, unsealing, and disaster recovery on top of your infra bill. Doppler is predictable flat-rate SaaS.

One often-overlooked cost with Vault is the unsealing ceremony after a restart. Without auto-unseal (via AWS KMS, GCP Cloud KMS, or Azure Key Vault), every Vault restart requires quorum from key holders. At 2 AM during an incident, this is painful.

When to Use Each

Choose Vault when:

You need dynamic secrets for databases, PKI, or cloud IAM roles
You operate multi-cloud or on-premises with strict data residency requirements
You have an infrastructure team that can own upgrades, HA, and disaster recovery

Choose AWS Secrets Manager when:

Your stack is 100% AWS
You want automatic RDS password rotation with no code to maintain
Operational simplicity matters more than cost at your secret count

Choose Doppler when:

Developer experience is the bottleneck (secrets in .env files, Slack DMs, or worse)
You want to sync secrets across environments, CI/CD, and Kubernetes without writing glue code
You're a small team and the SaaS operational model is worth the monthly cost

In practice, these tools compose well. Many teams run Vault or Secrets Manager at the infrastructure layer and use Doppler for developer and CI/CD workflows. The environments stay consistent without anyone pasting secrets into config files.

Whatever tool you pick, the fundamentals don't change: least privilege access, audit logging enabled, and a documented rotation schedule. We maintain free security hardening checklists covering secrets management and related controls — useful for validating your setup against concrete criteria.

The Takeaway

Vault is the most powerful option, but power comes with a real operational tax. AWS Secrets Manager is the pragmatic default for AWS-native teams. Doppler is the fastest path to eliminating secrets from .env files without investing in infrastructure.

If you're starting from scratch: Doppler for developer workflows today, with a plan to add Vault or Secrets Manager when dynamic credentials or compliance requirements demand it. Getting secrets out of your codebase matters more than picking the optimal tool on day one.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Trivy vs Grype vs Snyk: Container Vulnerability Scanners Shootout

Ayi NEDJIMI — Wed, 10 Jun 2026 10:05:22 +0000

Every container image you ship is a potential attack surface. Running nginx with an unpatched OpenSSL? A base image with a known privilege-escalation CVE? Your scanner should catch it before your attacker does. The real question is which tool is worth integrating into your pipeline without drowning your team in noise.

I tested Trivy, Grype, and Snyk across real-world images — from a lean Alpine-based Go service to a bloated Python data science container. Here's what I found.

The Contenders at a Glance

Trivy (Aqua Security) is a general-purpose vulnerability scanner covering container images, filesystems, Git repositories, Kubernetes configs, and IaC. It's fast, fully open-source, and ships as a single binary with no daemon required.

Grype (Anchore) is narrower in focus — containers and filesystems. It's also open-source, uses the Syft SBOM engine under the hood for package enumeration, and integrates cleanly with the broader Anchore ecosystem.

Snyk takes a commercial-platform approach (with a free tier). It wraps vulnerability scanning in a developer-experience layer: fix suggestions, pull request checks, IDE plugins, and a web dashboard. Useful, but with trade-offs discussed below.

Installation and First Scan

All three are easy to run locally:

# Trivy — single binary, no auth needed
brew install aquasecurity/trivy/trivy
trivy image python:3.11-slim

# Grype — also single binary, no auth needed
brew install anchore/grype/grype
grype python:3.11-slim

# Snyk — requires an account and token
npm install -g snyk
snyk auth
snyk container test python:3.11-slim

For CI/CD, Trivy and Grype both work without authentication. Snyk requires a token, adding friction in ephemeral build environments unless you already use Snyk's platform. Here's a minimal GitHub Actions step for Trivy that blocks the build on fixable HIGH or CRITICAL CVEs:

- name: Scan container image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ env.IMAGE_TAG }}
    format: 'table'
    exit-code: '1'
    ignore-unfixed: true
    severity: 'CRITICAL,HIGH'

Grype has an equivalent action (anchore/scan-action). Snyk offers a GitHub App that handles scanning at the repository level via a dashboard, but the CI step still requires an API token injected as a secret.

What Each Scanner Actually Finds

I scanned three images and compared raw finding counts:

python:3.11-slim — a common base image
node:18-alpine — leaner, Alpine-based
A custom Go API image built on golang:1.21-alpine

Trivy reported the most CVEs overall. On python:3.11-slim, it returned 78 vulnerabilities (14 HIGH, 2 CRITICAL on the test date) across OS packages. With --scanners vuln,secret, it also flags hardcoded credentials in image layers.

Grype found 71 on the same image. The delta stems from differences in how each tool maps package versions to CVE databases. Both pull from NVD, GitHub Advisory, and OS-specific sources (Debian Security Tracker, Alpine SecDB, Red Hat OVAL), but their normalisation logic diverges on edge cases.

Snyk found 65, but surfaced the most actionable output: a recommended base image (python:3.11.9-slim-bookworm) that would eliminate 42 findings in a single change. That kind of signal cuts real triage time.

For custom triage, parsing JSON output programmatically beats reading tables. Here's a Python snippet that reads Trivy's JSON output and prints only fixable HIGH/CRITICAL findings:

import json
import sys

def parse_trivy_report(path: str) -> None:
    with open(path) as f:
        report = json.load(f)

    for result in report.get("Results", []):
        target = result.get("Target", "")
        for vuln in result.get("Vulnerabilities", []):
            severity = vuln.get("Severity", "")
            fixed = vuln.get("FixedVersion", "")
            if severity in ("HIGH", "CRITICAL") and fixed:
                print(
                    f"[{severity}] {vuln['VulnerabilityID']} "
                    f"{target}: {vuln['PkgName']} {vuln['InstalledVersion']} "
                    f"-> fix available: {fixed}"
                )

if __name__ == "__main__":
    parse_trivy_report(sys.argv[1])

Run it with:

trivy image --format json --output report.json my-image:latest
python parse_trivy.py report.json

Grype and Snyk both produce structured JSON as well, though with different schemas. Trivy's is the most consistent for scripting.

Speed, DB Updates, and Offline Use

Trivy and Grype both download and cache their vulnerability databases locally. On first run, expect a 20-50 MB download. Subsequent runs use the cache and pull only diffs.

Approximate scan times on a 2024 M3 MacBook Pro:

Scanner	Cold (no cache)	Warm (cached)
Trivy	~35s	~8s
Grype	~40s	~10s
Snyk	~25s	~20s*

*Snyk sends image metadata to their servers for analysis on every scan — no true local cache. This has a privacy implication: proprietary package names, internal library versions, and image layer metadata all leave your environment. For regulated industries, that's a non-starter.

Trivy and Grype run entirely air-gapped after the DB download. This matters in finance, healthcare, and government contexts. If you're building a formal container security posture, tooling choice belongs in your threat model — our container security hardening checklists cover this decision point with specific control mappings.

Suppressing False Positives

Every scanner flags CVEs for package features you don't compile, vulnerabilities that require local shell access in an environment with no shell, or disputed findings. Managing that noise is where workflow matters.

Trivy uses a .trivyignore file at the repo root:

# HTTP/2 rapid reset — not exposed externally in this service
CVE-2023-44487
# Disputed, no practical impact in our configuration
CVE-2024-12345

Grype uses .grype.yaml:

ignore:
  - vulnerability: CVE-2023-44487
    reason: "HTTP/2 rapid reset — not exposed in this service"
  - fix-state: not-fixed

The fix-state: not-fixed rule is particularly useful: it silences all unfixed CVEs globally, letting you focus on what is actually actionable. Trivy's --ignore-unfixed flag does the same thing.

Snyk handles suppressions through its web UI, which is friendlier for non-security engineers but requires everyone to be in the Snyk platform to see the reasoning behind each ignored finding.

Which One Should You Use?

After running all three in CI for several months, here is my honest recommendation:

Use Trivy as your default. It covers containers, IaC, secrets, and Kubernetes manifests in one binary. The GitHub Actions integration is stable, the JSON output is scriptable, and there is no vendor dependency. This is the right choice for most teams.

Use Grype if SBOM generation is a compliance requirement. The Syft + Grype combination produces a reproducible, auditable artifact trail — a container's software bill of materials plus its known vulnerabilities, tied together. If your security team or customers are asking for SBOMs, this pipeline is the cleanest way to deliver them.

Use Snyk if your team has few security specialists and needs fix guidance embedded in developer tools. The "upgrade this base image to fix 40% of your findings" recommendation saves real time. The free tier works for open-source projects; serious production use pushes you toward a paid plan quickly.

In practice, many teams run Trivy in the pipeline (fast, free, CI-native) and offer Snyk IDE plugins to developers who want inline fix suggestions. That combination covers pipeline enforcement and developer experience without full vendor lock-in.

The Takeaway

Container scanning is table stakes now. The real problem is avoiding alert fatigue. Trivy is the best default for most teams: fast, open-source, no auth required, wide coverage. Grype wins on the SBOM compliance path. Snyk earns its seat when developer adoption and fix guidance matter more than control.

Whichever tool you integrate, block on CRITICAL findings with a fix available, suppress known-irrelevant CVEs explicitly, and review your ignore list quarterly. Noise without action is worse than no scanning at all.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Wazuh vs Elastic SIEM vs Splunk Free: Open-Source SIEM Compared (2026)

Ayi NEDJIMI — Tue, 09 Jun 2026 10:03:21 +0000

Your servers are generating thousands of log lines per minute, and you have no idea which ones matter. A SIEM (Security Information and Event Management) platform is supposed to solve that — correlate events, detect threats, and alert your team before damage is done. The problem: commercial SIEM licenses can cost $50k/year and up. The free alternatives are genuinely capable, but they make very different trade-offs.

This article compares three realistic options for teams without an enterprise security budget: Wazuh, the Elastic SIEM stack, and Splunk Free.

What We're Actually Comparing

"Free" means different things for each platform:

Wazuh: Fully open-source (GPLv2 for the manager, Apache 2.0 for the dashboard). No license caps, no feature gating. Community-supported with commercial support options.
Elastic SIEM: The security analytics layer on top of Elasticsearch + Kibana. The SIEM features themselves are free, but Elastic's licensing changed in 2021 — you're running SSPL unless you use the managed cloud offering. Self-hosted is fine, just know what you're agreeing to.
Splunk Free: Hard-capped at 500 MB/day ingestion. Above that, it stops indexing. Not viable for anything beyond a small homelab or a single application.

That 500 MB cap effectively eliminates Splunk Free from most real deployments, so this comparison becomes primarily Wazuh vs Elastic SIEM, with Splunk Free as a distant third for specific niche use cases.

Architecture Differences

Wazuh is agent-based. You install a Wazuh agent on each host you want to monitor. Agents ship logs, file integrity events, process activity, and vulnerability scan results back to the Wazuh manager. The manager runs correlation rules and sends alerts. The UI is a fork of Kibana, skinned for security workflows.

# Query Wazuh alerts via its REST API
import requests

WAZUH_API = "https://your-wazuh-manager:55000"

def authenticate(user: str, password: str) -> str:
    resp = requests.post(
        f"{WAZUH_API}/security/user/authenticate",
        auth=(user, password),
        verify=False,
    )
    resp.raise_for_status()
    return resp.json()["data"]["token"]

def get_recent_alerts(token: str, level_min: int = 10, limit: int = 50) -> list:
    headers = {"Authorization": f"Bearer {token}"}
    params = {
        "limit": limit,
        "sort": "-timestamp",
        "q": f"rule.level>={level_min}",
    }
    resp = requests.get(
        f"{WAZUH_API}/alerts",
        headers=headers,
        params=params,
        verify=False,
    )
    resp.raise_for_status()
    return resp.json().get("data", {}).get("affected_items", [])

if __name__ == "__main__":
    token = authenticate("wazuh-wui", "your-password")
    alerts = get_recent_alerts(token, level_min=12)
    for alert in alerts:
        agent_name = alert.get("agent", {}).get("name", "unknown")
        print(f"[{alert['rule']['level']}] {alert['rule']['description']} — {agent_name}")

Elastic SIEM works differently. You use Beats (Filebeat, Winlogbeat, Packetbeat) or Elastic Agent to ship logs into Elasticsearch. Kibana's Security app then provides detection rules, timelines, and case management on top of that raw data. Detection rules are written in EQL (Event Query Language) or KQL.

The architecture is more flexible but also more complex. You're assembling a stack (Elasticsearch + Kibana + ingestion layer), whereas Wazuh ships as a more cohesive bundle.

Detection Quality Out of the Box

This is where Wazuh has a genuine advantage for teams that want something working on day one. Wazuh ships with 3,000+ detection rules covering Linux syscalls, Windows event IDs, web application attacks, Docker/Kubernetes activity, and common CVEs. The rules are maintained by the community and mapped to MITRE ATT&CK.

Elastic's detection rules are also MITRE-mapped and actively maintained, but they live in a separate GitHub repository and need to be imported. Once imported, the quality is comparable — there's just more setup friction.

For custom rules, Elastic wins on expressiveness. EQL is designed for threat hunting and lets you correlate sequences of events across time:

# Run an EQL sequence query with the Elasticsearch Python client
from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://your-elastic:9200",
    api_key=("key-id", "api-key-value"),
)

# Detect process injection: cmd.exe from a non-shell parent,
# then outbound TLS connection within 30 seconds
eql_query = (
    "sequence by host.name with maxspan=30s\n"
    "  [process where process.name == 'cmd.exe'\n"
    "   and process.parent.name != 'explorer.exe'\n"
    "   and process.parent.name != 'powershell.exe']\n"
    "  [network where destination.port == 443\n"
    "   and process.name == 'cmd.exe']"
)

resp = es.eql.search(
    index="logs-*",
    body={"query": eql_query, "size": 20},
)

for hit in resp["hits"]["sequences"]:
    events = hit["events"]
    host = events[0]["_source"]["host"]["name"]
    print(f"Suspicious cmd.exe network activity on {host}")
    for e in events:
        dst = e["_source"].get("destination", {}).get("ip", "N/A")
        proc = e["_source"]["process"]["name"]
        print(f"  {proc} -> {dst}")

This kind of event-sequence correlation is harder to express in Wazuh's XML rule format, though Wazuh 4.x added composite rules that partially close the gap.

Scalability and Operational Cost

At small scale (under 500 agents, 10k events per second), both platforms work fine on modest hardware. Wazuh's recommended minimum for production is 4 CPU cores and 8 GB RAM for the manager.

Elastic's RAM appetite is well-known. Elasticsearch JVM heap should be 50% of available RAM, capped at 31 GB for GC performance reasons. A production Elastic stack that handles real log volumes comfortably needs 16-32 GB RAM and fast SSDs. Multi-node clusters multiply that.

Wazuh also uses Elasticsearch under the hood (the indexer component is a rebranded OpenSearch fork), but operational complexity is lower because Wazuh manages it for you. You don't tune JVM settings manually unless something breaks.

For compliance use cases (PCI-DSS, HIPAA, SOC 2), Wazuh includes built-in compliance dashboards mapped to specific control frameworks. Elastic requires you to build these views yourself or pay for the Elastic Security subscription. Our security hardening checklists include the specific Wazuh rules and agent policies needed to satisfy each control — saved a few teams days of mapping work.

The Honest Comparison

Criterion	Wazuh	Elastic SIEM	Splunk Free
License	Open-source	SSPL (self-hosted)	Free, 500 MB/day cap
Setup time	2-4 hours	4-8 hours	1-2 hours
Default rules	3,000+	~650 (must import)	Limited
Advanced queries	Limited	EQL - excellent	SPL - excellent
RAM requirements	Moderate	High	Moderate
Compliance dashboards	Built-in	Manual	Manual
Scalability ceiling	High	Very high	Hard-capped

Choose Wazuh if: you want a cohesive security platform with agent-based monitoring, built-in compliance reports, and lower operational overhead. It's the pragmatic choice for teams of 1-5 people responsible for security.

Choose Elastic SIEM if: you already run Elasticsearch for other workloads, have large log volumes that need full-text search flexibility, or your team has threat hunters who need EQL for complex correlation queries.

Avoid Splunk Free for anything beyond a demo environment. The 500 MB/day cap hits immediately in any real deployment.

The Takeaway

There's no universally better option — the right SIEM depends on your team size, existing infrastructure, and what "free" actually means operationally. Wazuh wins on out-of-box security coverage and setup simplicity. Elastic SIEM wins on query power and ecosystem depth. Splunk Free is a demo, not a production tool.

For most small-to-medium teams starting from scratch, Wazuh is where to begin. You can have detection rules running, agents deployed, and dashboards live in an afternoon. Once you've outgrown it or need more advanced correlation, the migration path to Elastic is well-documented.

Start with working detection, not the most powerful platform you can't configure.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Terraform vs Pulumi vs Ansible: IaC for small teams

Ayi NEDJIMI — Mon, 08 Jun 2026 10:05:14 +0000

You're a team of three engineers. One of you just broke staging by running a script manually. Nobody knows what the current state of infra actually is. IaC is the obvious fix — but Terraform, Pulumi, and Ansible all claim to solve this. Which one?

The short answer: they're not the same kind of tool, and the right choice depends on your team's profile more than the tools' feature lists.

What each tool actually does

It's worth being precise about categories because "IaC" gets used too loosely:

Terraform (HashiCorp, now BSL licensed): declarative, state-file-based cloud provisioning. You describe desired state; Terraform computes the diff and applies it.
Pulumi: same concept as Terraform, but you write real code (Python, Go, TypeScript) instead of HCL.
Ansible: imperative, agentless configuration management. Strong for day-2 ops (installing packages, configuring services), weaker for provisioning.

These aren't identical tools competing for the same niche.

When Ansible makes sense — and when it doesn't

Ansible is solid for: configuring existing servers, deploying applications, running ordered multi-step procedures across a fleet.

It's awkward for: managing cloud resources from scratch. The modules exist (for EC2, VPCs, RDS), but state management is painful. Ansible doesn't track what it created — each run answers "did this task succeed?", not "does this resource still exist?". You end up with orphaned resources and no clear inventory.

For a small team provisioning cloud infra, Ansible alone isn't sufficient. The pattern that actually works: provision with Terraform, configure with Ansible.

# playbook.yml — configure a Python app on a freshly provisioned EC2 instance
- name: Deploy Python app
  hosts: web
  become: yes
  tasks:
    - name: Install system dependencies
      apt:
        name: [python3, python3-pip, nginx]
        state: present
        update_cache: yes

    - name: Deploy application files
      copy:
        src: ./app/
        dest: /opt/myapp/
        owner: www-data

    - name: Install Python requirements
      pip:
        requirements: /opt/myapp/requirements.txt
        executable: pip3

    - name: Enable and start service
      systemd:
        name: myapp
        state: started
        enabled: yes

This works well. But you need Terraform or Pulumi to create that instance in the first place.

Terraform: the safe default

Terraform has been the standard for cloud provisioning for years. HCL is learnable in a day, the provider ecosystem covers AWS, GCP, Azure, Cloudflare, GitHub, Datadog, and hundreds more. The plan/apply workflow gives you an explicit diff to review before any change lands.

For a small team, the main advantages:

Anyone can read and write HCL without a deep programming background
terraform plan output is reviewable in PRs — non-engineers can spot obvious regressions
State locking via S3 + DynamoDB prevents concurrent applies from corrupting state
The BSL license change only affects vendors building Terraform-as-a-service; internal use remains free

The friction:

HCL is not a real programming language. Complex logic via for_each, dynamic blocks, and conditionals becomes unwieldy at scale
No native test framework — you need Terratest or heavy terraform validate discipline
State drift is a real burden: manual cloud console changes require terraform import or state rm

# main.tf — minimal VPC + EC2 for a Go API on AWS
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  tags       = { Name = "main" }
}

resource "aws_subnet" "public" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "eu-west-1a"
}

resource "aws_security_group" "api" {
  name   = "api-sg"
  vpc_id = aws_vpc.main.id

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "api" {
  ami                    = "ami-0c55b159cbfafe1f0"
  instance_type          = "t3.small"
  subnet_id              = aws_subnet.public.id
  vpc_security_group_ids = [aws_security_group.api.id]
  tags                   = { Name = "go-api" }
}

Readable, reviewable, and straightforward for any team member — including whoever joined last month.

Pulumi: real code, real tradeoffs

Pulumi replaces HCL with actual programming languages. The same stack in Python:

import pulumi
import pulumi_aws as aws

vpc = aws.ec2.Vpc("main",
    cidr_block="10.0.0.0/16",
    tags={"Name": "main"}
)

subnet = aws.ec2.Subnet("public",
    vpc_id=vpc.id,
    cidr_block="10.0.1.0/24",
    availability_zone="eu-west-1a"
)

sg = aws.ec2.SecurityGroup("api-sg",
    vpc_id=vpc.id,
    ingress=[{
        "from_port": 8080, "to_port": 8080,
        "protocol": "tcp", "cidr_blocks": ["0.0.0.0/0"]
    }],
    egress=[{
        "from_port": 0, "to_port": 0,
        "protocol": "-1", "cidr_blocks": ["0.0.0.0/0"]
    }]
)

instance = aws.ec2.Instance("go-api",
    ami="ami-0c55b159cbfafe1f0",
    instance_type="t3.small",
    subnet_id=subnet.id,
    vpc_security_group_ids=[sg.id],
    tags={"Name": "go-api"}
)

pulumi.export("instance_id", instance.id)

The real appeal: you can loop over a list of environments without count gymnastics, write unit tests with pytest, reuse Python functions, and import your own internal libraries. For infra with 10+ environments or heavily dynamic configuration, Pulumi's expressiveness is a genuine win.

The tradeoffs:

State goes to Pulumi Cloud by default — there's a free tier, but self-hosting requires more setup than Terraform's S3 backend
Steeper onboarding: new team members need to know the language and Pulumi's async resource graph model. Debugging Output[T] types is not obvious
Smaller community: Stack Overflow coverage and available modules are thinner than Terraform's

Practical decision tree for small teams

Use Ansible when you're configuring existing servers, not provisioning from scratch.

Use Terraform when:

Your team has mixed backgrounds — not everyone writes Python or Go fluently
Your infra is relatively stable: fewer than five environments, no heavy dynamic resource generation
You want the largest provider ecosystem and the most community resources

Use Pulumi when:

Your team is developer-heavy and HCL feels like a cognitive tax
You need real programming constructs: dynamic loops, proper unit tests, shared libraries across stacks
You're comfortable with Pulumi Cloud pricing or willing to configure a self-hosted state backend

For most teams under five engineers, Terraform is the right starting point. The plan/apply review workflow fits naturally into PRs, HCL is fast to learn, and you won't hit its limits until your infra grows significantly.

One thing that matters regardless of tool: gate every apply behind CI/CD with mandatory code review on infra changes. A misconfiguration merged without review is where incidents start. We maintain a security checklist for IaC pipelines covering state file access control, least-privilege IAM for CI runners, and safe secrets handling in both Terraform and Pulumi.

The takeaway

Terraform, Pulumi, and Ansible aren't direct competitors — they solve overlapping but distinct problems. Ansible handles configuration; Terraform and Pulumi handle provisioning. Between the two provisioning tools, Terraform wins on simplicity and ecosystem breadth; Pulumi wins when your infra logic outgrows what HCL can express cleanly.

Pick based on your team's actual skills and infra complexity, not on which tool had the best talk at the last DevOps conference.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Celery vs RQ vs Dramatiq: Python Task Queues Compared

Ayi NEDJIMI — Sun, 07 Jun 2026 10:08:13 +0000

When your API handler starts timing out because it's doing too much at once — sending emails, generating reports, hitting third-party APIs — you reach for a task queue. Python gives you three serious options: Celery, RQ, and Dramatiq. Picking the wrong one will cost you debugging hours you don't have.

This is not a benchmark post with synthetic numbers. It's a practical comparison after running all three in real scenarios, covering where each fits, where each breaks, and which one you should reach for first.

What a task queue actually does

A task queue moves work out of the request-response cycle. Instead of your web handler blocking on a slow operation, it serializes a message to a broker (Redis, RabbitMQ, or similar), and a separate worker process picks it up asynchronously.

The main components:

Broker — the message bus (Redis, RabbitMQ)
Worker — the process that runs your task functions
Task — a serialized unit of work with arguments
Result backend — optional storage for return values and status

All three libraries cover these primitives. The differences show up in configuration overhead, failure handling, and what breaks silently at 2 AM.

Celery: powerful, with a price

Celery has been the default Python task queue for over a decade. It supports Redis, RabbitMQ, SQS, and more. It ships with celery-beat for periodic tasks, canvas for chaining and fan-out, and has a large ecosystem of integrations.

Here is a minimal but production-ready Celery task:

# tasks.py
from celery import Celery

app = Celery(
    "worker",
    broker="redis://localhost:6379/0",
    backend="redis://localhost:6379/1"
)
app.conf.update(task_serializer="json", accept_content=["json"])

@app.task(bind=True, max_retries=3, default_retry_delay=10)
def send_notification(self, user_id: int, message: str) -> dict:
    try:
        result = _dispatch(user_id, message)
        return {"status": "ok", "user_id": user_id}
    except Exception as exc:
        raise self.retry(exc=exc)

def _dispatch(user_id: int, message: str) -> bool:
    # email / push / SMS logic here
    return True

# start worker
celery -A tasks worker --loglevel=info --concurrency=4
# start scheduler for periodic tasks
celery -A tasks beat --loglevel=info

What Celery gets right:

Broad broker support — useful when your infra doesn't include Redis
Canvas: group, chain, chord for complex multi-step workflows
celery-beat for cron-style scheduling without external tooling
Mature retry logic with configurable backoff

What Celery gets wrong:

The default pickle serializer has a well-documented attack surface — always configure task_serializer="json" and enable task signing if workers accept input from untrusted sources. A good starting point is this security hardening checklist covering worker configuration and serialization settings
prefetch_multiplier and the concurrency model require tuning, or you'll have workers sitting idle while your queue grows
Configuration sprawls across celeryconfig.py, environment variables, and scattered app.conf.update calls
Debugging failed tasks in production often means reading the Celery source code

Use Celery when you genuinely need its feature set: complex workflows with fan-out and aggregation, multiple broker options, or celery-beat for reliable scheduling. For simpler use cases, you're paying its complexity tax for nothing.

RQ: minimal and honestly good enough

RQ (Redis Queue) takes the opposite approach: one broker (Redis), clean API, minimal configuration. There is no concurrency model to tune beyond worker count, no separate scheduler daemon for basic use cases, and no config file to manage.

# tasks.py
def process_upload(file_path: str, user_id: int) -> dict:
    result = _convert_file(file_path)
    return {"file": result, "user_id": user_id}

def _convert_file(path: str) -> str:
    # imagine ffmpeg or Pillow here
    return path.replace(".tmp", ".mp4")

# enqueue.py
from redis import Redis
from rq import Queue
from rq.job import Retry

redis_conn = Redis()
q = Queue(connection=redis_conn)

job = q.enqueue(
    "tasks.process_upload",
    kwargs={"file_path": "/tmp/upload.tmp", "user_id": 42},
    retry=Retry(max=3, interval=[10, 30, 60]),
    job_timeout=300,
)
print(f"Enqueued: {job.id}")

rq worker --with-scheduler default

RQ strengths:

Setup takes minutes, not hours
rq info and rq-dashboard make job inspection trivial
Dead-letter queue behavior is obvious and inspectable
The codebase is small enough to read in an afternoon — critical when debugging production issues
Results are stored in Redis natively; no second backend to configure and monitor

RQ limitations:

Redis-only — no RabbitMQ support
No native pipeline or chain API; you wire sequential jobs by hand or use callbacks
The scheduler (--with-scheduler) works but has less production history than celery-beat
Community activity has been slower over the past year compared to Celery

For the majority of web services — async emails, file processing, background API calls, report generation — RQ is the right default. It does the job, stays out of your way, and does not require deep operational knowledge to run safely.

Dramatiq: the most correctly designed

Dramatiq is the youngest of the three and arguably the most thoughtfully designed. Its defaults are safer: at-least-once delivery semantics are explicit, the middleware system is composable, and retry behavior does not require diving into source code to understand.

# tasks.py
import dramatiq
from dramatiq.brokers.redis import RedisBroker

broker = RedisBroker(url="redis://localhost:6379")
dramatiq.set_broker(broker)

@dramatiq.actor(
    max_retries=5,
    min_backoff=1_000,     # 1 second
    max_backoff=300_000,   # 5 minutes
    time_limit=30_000,     # 30 second wall clock limit
)
def handle_payment_event(payload: dict) -> None:
    if payload.get("type") == "payment.succeeded":
        _record_payment(payload["id"], payload["amount"])

def _record_payment(payment_id: str, amount: int) -> None:
    # idempotency check -> DB write -> trigger fulfillment
    pass

dramatiq tasks --processes 2 --threads 8

Dramatiq strengths:

Middleware architecture: retries, time limits, rate limits, and age limits are composable and independently testable
Message schema is explicit; silent deserialization failures are not a surprise you discover in production
Supports both Redis and RabbitMQ with the same actor code
Failure semantics are predictable — dead-letter messages stay in a well-defined location you can monitor

Dramatiq limitations:

No built-in scheduler — you need apscheduler or dramatiq-cron as an external dependency
Smaller ecosystem and community; fewer ready-made integrations with popular frameworks
Pipelines and groups exist but are less mature and less documented than Celery canvas

Dramatiq is the right tool when task correctness is non-negotiable: payment processing, audit log ingestion, compliance-sensitive workflows. Its design choices consistently favor "this task ran exactly as expected" over "this task ran fast."

Quick comparison

	Celery	RQ	Dramatiq
Brokers	Redis, RabbitMQ, SQS, more	Redis only	Redis, RabbitMQ
Setup complexity	High	Low	Medium
Retry handling	Configurable	Explicit	Middleware-based
Scheduling	celery-beat (built-in)	`--with-scheduler`	External only
Pipelines	Full canvas	Manual	Basic
Maturity	Very high	High	Medium
Best for	Complex workflows	Simple async jobs	Correctness-critical work

The takeaway

Start with RQ. It handles 80% of production use cases with 20% of the operational complexity. If you hit a wall — you need RabbitMQ, canvas-style pipelines, or battle-tested scheduling — migrate to Celery. If correctness and explicit failure handling matter more than ecosystem size, use Dramatiq.

Whichever you pick: keep workers stateless, use JSON serialization only, monitor your dead-letter queues, and set explicit timeouts on every task. A task that fails silently is worse than a task that fails loudly. Instrument your workers with proper logging from day one — you will be glad you did the first time something goes missing.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.