DEV Community: Ayi NEDJIMI

Trivy vs Grype vs Snyk: Container Vulnerability Scanners Shootout

Ayi NEDJIMI — Wed, 10 Jun 2026 10:05:22 +0000

Every container image you ship is a potential attack surface. Running nginx with an unpatched OpenSSL? A base image with a known privilege-escalation CVE? Your scanner should catch it before your attacker does. The real question is which tool is worth integrating into your pipeline without drowning your team in noise.

I tested Trivy, Grype, and Snyk across real-world images — from a lean Alpine-based Go service to a bloated Python data science container. Here's what I found.

The Contenders at a Glance

Trivy (Aqua Security) is a general-purpose vulnerability scanner covering container images, filesystems, Git repositories, Kubernetes configs, and IaC. It's fast, fully open-source, and ships as a single binary with no daemon required.

Grype (Anchore) is narrower in focus — containers and filesystems. It's also open-source, uses the Syft SBOM engine under the hood for package enumeration, and integrates cleanly with the broader Anchore ecosystem.

Snyk takes a commercial-platform approach (with a free tier). It wraps vulnerability scanning in a developer-experience layer: fix suggestions, pull request checks, IDE plugins, and a web dashboard. Useful, but with trade-offs discussed below.

Installation and First Scan

All three are easy to run locally:

# Trivy — single binary, no auth needed
brew install aquasecurity/trivy/trivy
trivy image python:3.11-slim

# Grype — also single binary, no auth needed
brew install anchore/grype/grype
grype python:3.11-slim

# Snyk — requires an account and token
npm install -g snyk
snyk auth
snyk container test python:3.11-slim

For CI/CD, Trivy and Grype both work without authentication. Snyk requires a token, adding friction in ephemeral build environments unless you already use Snyk's platform. Here's a minimal GitHub Actions step for Trivy that blocks the build on fixable HIGH or CRITICAL CVEs:

- name: Scan container image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ env.IMAGE_TAG }}
    format: 'table'
    exit-code: '1'
    ignore-unfixed: true
    severity: 'CRITICAL,HIGH'

Grype has an equivalent action (anchore/scan-action). Snyk offers a GitHub App that handles scanning at the repository level via a dashboard, but the CI step still requires an API token injected as a secret.

What Each Scanner Actually Finds

I scanned three images and compared raw finding counts:

python:3.11-slim — a common base image
node:18-alpine — leaner, Alpine-based
A custom Go API image built on golang:1.21-alpine

Trivy reported the most CVEs overall. On python:3.11-slim, it returned 78 vulnerabilities (14 HIGH, 2 CRITICAL on the test date) across OS packages. With --scanners vuln,secret, it also flags hardcoded credentials in image layers.

Grype found 71 on the same image. The delta stems from differences in how each tool maps package versions to CVE databases. Both pull from NVD, GitHub Advisory, and OS-specific sources (Debian Security Tracker, Alpine SecDB, Red Hat OVAL), but their normalisation logic diverges on edge cases.

Snyk found 65, but surfaced the most actionable output: a recommended base image (python:3.11.9-slim-bookworm) that would eliminate 42 findings in a single change. That kind of signal cuts real triage time.

For custom triage, parsing JSON output programmatically beats reading tables. Here's a Python snippet that reads Trivy's JSON output and prints only fixable HIGH/CRITICAL findings:

import json
import sys

def parse_trivy_report(path: str) -> None:
    with open(path) as f:
        report = json.load(f)

    for result in report.get("Results", []):
        target = result.get("Target", "")
        for vuln in result.get("Vulnerabilities", []):
            severity = vuln.get("Severity", "")
            fixed = vuln.get("FixedVersion", "")
            if severity in ("HIGH", "CRITICAL") and fixed:
                print(
                    f"[{severity}] {vuln['VulnerabilityID']} "
                    f"{target}: {vuln['PkgName']} {vuln['InstalledVersion']} "
                    f"-> fix available: {fixed}"
                )

if __name__ == "__main__":
    parse_trivy_report(sys.argv[1])

Run it with:

trivy image --format json --output report.json my-image:latest
python parse_trivy.py report.json

Grype and Snyk both produce structured JSON as well, though with different schemas. Trivy's is the most consistent for scripting.

Speed, DB Updates, and Offline Use

Trivy and Grype both download and cache their vulnerability databases locally. On first run, expect a 20-50 MB download. Subsequent runs use the cache and pull only diffs.

Approximate scan times on a 2024 M3 MacBook Pro:

Scanner	Cold (no cache)	Warm (cached)
Trivy	~35s	~8s
Grype	~40s	~10s
Snyk	~25s	~20s*

*Snyk sends image metadata to their servers for analysis on every scan — no true local cache. This has a privacy implication: proprietary package names, internal library versions, and image layer metadata all leave your environment. For regulated industries, that's a non-starter.

Trivy and Grype run entirely air-gapped after the DB download. This matters in finance, healthcare, and government contexts. If you're building a formal container security posture, tooling choice belongs in your threat model — our container security hardening checklists cover this decision point with specific control mappings.

Suppressing False Positives

Every scanner flags CVEs for package features you don't compile, vulnerabilities that require local shell access in an environment with no shell, or disputed findings. Managing that noise is where workflow matters.

Trivy uses a .trivyignore file at the repo root:

# HTTP/2 rapid reset — not exposed externally in this service
CVE-2023-44487
# Disputed, no practical impact in our configuration
CVE-2024-12345

Grype uses .grype.yaml:

ignore:
  - vulnerability: CVE-2023-44487
    reason: "HTTP/2 rapid reset — not exposed in this service"
  - fix-state: not-fixed

The fix-state: not-fixed rule is particularly useful: it silences all unfixed CVEs globally, letting you focus on what is actually actionable. Trivy's --ignore-unfixed flag does the same thing.

Snyk handles suppressions through its web UI, which is friendlier for non-security engineers but requires everyone to be in the Snyk platform to see the reasoning behind each ignored finding.

Which One Should You Use?

After running all three in CI for several months, here is my honest recommendation:

Use Trivy as your default. It covers containers, IaC, secrets, and Kubernetes manifests in one binary. The GitHub Actions integration is stable, the JSON output is scriptable, and there is no vendor dependency. This is the right choice for most teams.

Use Grype if SBOM generation is a compliance requirement. The Syft + Grype combination produces a reproducible, auditable artifact trail — a container's software bill of materials plus its known vulnerabilities, tied together. If your security team or customers are asking for SBOMs, this pipeline is the cleanest way to deliver them.

Use Snyk if your team has few security specialists and needs fix guidance embedded in developer tools. The "upgrade this base image to fix 40% of your findings" recommendation saves real time. The free tier works for open-source projects; serious production use pushes you toward a paid plan quickly.

In practice, many teams run Trivy in the pipeline (fast, free, CI-native) and offer Snyk IDE plugins to developers who want inline fix suggestions. That combination covers pipeline enforcement and developer experience without full vendor lock-in.

The Takeaway

Container scanning is table stakes now. The real problem is avoiding alert fatigue. Trivy is the best default for most teams: fast, open-source, no auth required, wide coverage. Grype wins on the SBOM compliance path. Snyk earns its seat when developer adoption and fix guidance matter more than control.

Whichever tool you integrate, block on CRITICAL findings with a fix available, suppress known-irrelevant CVEs explicitly, and review your ignore list quarterly. Noise without action is worse than no scanning at all.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Wazuh vs Elastic SIEM vs Splunk Free: Open-Source SIEM Compared (2026)

Ayi NEDJIMI — Tue, 09 Jun 2026 10:03:21 +0000

Your servers are generating thousands of log lines per minute, and you have no idea which ones matter. A SIEM (Security Information and Event Management) platform is supposed to solve that — correlate events, detect threats, and alert your team before damage is done. The problem: commercial SIEM licenses can cost $50k/year and up. The free alternatives are genuinely capable, but they make very different trade-offs.

This article compares three realistic options for teams without an enterprise security budget: Wazuh, the Elastic SIEM stack, and Splunk Free.

What We're Actually Comparing

"Free" means different things for each platform:

Wazuh: Fully open-source (GPLv2 for the manager, Apache 2.0 for the dashboard). No license caps, no feature gating. Community-supported with commercial support options.
Elastic SIEM: The security analytics layer on top of Elasticsearch + Kibana. The SIEM features themselves are free, but Elastic's licensing changed in 2021 — you're running SSPL unless you use the managed cloud offering. Self-hosted is fine, just know what you're agreeing to.
Splunk Free: Hard-capped at 500 MB/day ingestion. Above that, it stops indexing. Not viable for anything beyond a small homelab or a single application.

That 500 MB cap effectively eliminates Splunk Free from most real deployments, so this comparison becomes primarily Wazuh vs Elastic SIEM, with Splunk Free as a distant third for specific niche use cases.

Architecture Differences

Wazuh is agent-based. You install a Wazuh agent on each host you want to monitor. Agents ship logs, file integrity events, process activity, and vulnerability scan results back to the Wazuh manager. The manager runs correlation rules and sends alerts. The UI is a fork of Kibana, skinned for security workflows.

# Query Wazuh alerts via its REST API
import requests

WAZUH_API = "https://your-wazuh-manager:55000"

def authenticate(user: str, password: str) -> str:
    resp = requests.post(
        f"{WAZUH_API}/security/user/authenticate",
        auth=(user, password),
        verify=False,
    )
    resp.raise_for_status()
    return resp.json()["data"]["token"]

def get_recent_alerts(token: str, level_min: int = 10, limit: int = 50) -> list:
    headers = {"Authorization": f"Bearer {token}"}
    params = {
        "limit": limit,
        "sort": "-timestamp",
        "q": f"rule.level>={level_min}",
    }
    resp = requests.get(
        f"{WAZUH_API}/alerts",
        headers=headers,
        params=params,
        verify=False,
    )
    resp.raise_for_status()
    return resp.json().get("data", {}).get("affected_items", [])

if __name__ == "__main__":
    token = authenticate("wazuh-wui", "your-password")
    alerts = get_recent_alerts(token, level_min=12)
    for alert in alerts:
        agent_name = alert.get("agent", {}).get("name", "unknown")
        print(f"[{alert['rule']['level']}] {alert['rule']['description']} — {agent_name}")

Elastic SIEM works differently. You use Beats (Filebeat, Winlogbeat, Packetbeat) or Elastic Agent to ship logs into Elasticsearch. Kibana's Security app then provides detection rules, timelines, and case management on top of that raw data. Detection rules are written in EQL (Event Query Language) or KQL.

The architecture is more flexible but also more complex. You're assembling a stack (Elasticsearch + Kibana + ingestion layer), whereas Wazuh ships as a more cohesive bundle.

Detection Quality Out of the Box

This is where Wazuh has a genuine advantage for teams that want something working on day one. Wazuh ships with 3,000+ detection rules covering Linux syscalls, Windows event IDs, web application attacks, Docker/Kubernetes activity, and common CVEs. The rules are maintained by the community and mapped to MITRE ATT&CK.

Elastic's detection rules are also MITRE-mapped and actively maintained, but they live in a separate GitHub repository and need to be imported. Once imported, the quality is comparable — there's just more setup friction.

For custom rules, Elastic wins on expressiveness. EQL is designed for threat hunting and lets you correlate sequences of events across time:

# Run an EQL sequence query with the Elasticsearch Python client
from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://your-elastic:9200",
    api_key=("key-id", "api-key-value"),
)

# Detect process injection: cmd.exe from a non-shell parent,
# then outbound TLS connection within 30 seconds
eql_query = (
    "sequence by host.name with maxspan=30s\n"
    "  [process where process.name == 'cmd.exe'\n"
    "   and process.parent.name != 'explorer.exe'\n"
    "   and process.parent.name != 'powershell.exe']\n"
    "  [network where destination.port == 443\n"
    "   and process.name == 'cmd.exe']"
)

resp = es.eql.search(
    index="logs-*",
    body={"query": eql_query, "size": 20},
)

for hit in resp["hits"]["sequences"]:
    events = hit["events"]
    host = events[0]["_source"]["host"]["name"]
    print(f"Suspicious cmd.exe network activity on {host}")
    for e in events:
        dst = e["_source"].get("destination", {}).get("ip", "N/A")
        proc = e["_source"]["process"]["name"]
        print(f"  {proc} -> {dst}")

This kind of event-sequence correlation is harder to express in Wazuh's XML rule format, though Wazuh 4.x added composite rules that partially close the gap.

Scalability and Operational Cost

At small scale (under 500 agents, 10k events per second), both platforms work fine on modest hardware. Wazuh's recommended minimum for production is 4 CPU cores and 8 GB RAM for the manager.

Elastic's RAM appetite is well-known. Elasticsearch JVM heap should be 50% of available RAM, capped at 31 GB for GC performance reasons. A production Elastic stack that handles real log volumes comfortably needs 16-32 GB RAM and fast SSDs. Multi-node clusters multiply that.

Wazuh also uses Elasticsearch under the hood (the indexer component is a rebranded OpenSearch fork), but operational complexity is lower because Wazuh manages it for you. You don't tune JVM settings manually unless something breaks.

For compliance use cases (PCI-DSS, HIPAA, SOC 2), Wazuh includes built-in compliance dashboards mapped to specific control frameworks. Elastic requires you to build these views yourself or pay for the Elastic Security subscription. Our security hardening checklists include the specific Wazuh rules and agent policies needed to satisfy each control — saved a few teams days of mapping work.

The Honest Comparison

Criterion	Wazuh	Elastic SIEM	Splunk Free
License	Open-source	SSPL (self-hosted)	Free, 500 MB/day cap
Setup time	2-4 hours	4-8 hours	1-2 hours
Default rules	3,000+	~650 (must import)	Limited
Advanced queries	Limited	EQL - excellent	SPL - excellent
RAM requirements	Moderate	High	Moderate
Compliance dashboards	Built-in	Manual	Manual
Scalability ceiling	High	Very high	Hard-capped

Choose Wazuh if: you want a cohesive security platform with agent-based monitoring, built-in compliance reports, and lower operational overhead. It's the pragmatic choice for teams of 1-5 people responsible for security.

Choose Elastic SIEM if: you already run Elasticsearch for other workloads, have large log volumes that need full-text search flexibility, or your team has threat hunters who need EQL for complex correlation queries.

Avoid Splunk Free for anything beyond a demo environment. The 500 MB/day cap hits immediately in any real deployment.

The Takeaway

There's no universally better option — the right SIEM depends on your team size, existing infrastructure, and what "free" actually means operationally. Wazuh wins on out-of-box security coverage and setup simplicity. Elastic SIEM wins on query power and ecosystem depth. Splunk Free is a demo, not a production tool.

For most small-to-medium teams starting from scratch, Wazuh is where to begin. You can have detection rules running, agents deployed, and dashboards live in an afternoon. Once you've outgrown it or need more advanced correlation, the migration path to Elastic is well-documented.

Start with working detection, not the most powerful platform you can't configure.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Terraform vs Pulumi vs Ansible: IaC for small teams

Ayi NEDJIMI — Mon, 08 Jun 2026 10:05:14 +0000

You're a team of three engineers. One of you just broke staging by running a script manually. Nobody knows what the current state of infra actually is. IaC is the obvious fix — but Terraform, Pulumi, and Ansible all claim to solve this. Which one?

The short answer: they're not the same kind of tool, and the right choice depends on your team's profile more than the tools' feature lists.

What each tool actually does

It's worth being precise about categories because "IaC" gets used too loosely:

Terraform (HashiCorp, now BSL licensed): declarative, state-file-based cloud provisioning. You describe desired state; Terraform computes the diff and applies it.
Pulumi: same concept as Terraform, but you write real code (Python, Go, TypeScript) instead of HCL.
Ansible: imperative, agentless configuration management. Strong for day-2 ops (installing packages, configuring services), weaker for provisioning.

These aren't identical tools competing for the same niche.

When Ansible makes sense — and when it doesn't

Ansible is solid for: configuring existing servers, deploying applications, running ordered multi-step procedures across a fleet.

It's awkward for: managing cloud resources from scratch. The modules exist (for EC2, VPCs, RDS), but state management is painful. Ansible doesn't track what it created — each run answers "did this task succeed?", not "does this resource still exist?". You end up with orphaned resources and no clear inventory.

For a small team provisioning cloud infra, Ansible alone isn't sufficient. The pattern that actually works: provision with Terraform, configure with Ansible.

# playbook.yml — configure a Python app on a freshly provisioned EC2 instance
- name: Deploy Python app
  hosts: web
  become: yes
  tasks:
    - name: Install system dependencies
      apt:
        name: [python3, python3-pip, nginx]
        state: present
        update_cache: yes

    - name: Deploy application files
      copy:
        src: ./app/
        dest: /opt/myapp/
        owner: www-data

    - name: Install Python requirements
      pip:
        requirements: /opt/myapp/requirements.txt
        executable: pip3

    - name: Enable and start service
      systemd:
        name: myapp
        state: started
        enabled: yes

This works well. But you need Terraform or Pulumi to create that instance in the first place.

Terraform: the safe default

Terraform has been the standard for cloud provisioning for years. HCL is learnable in a day, the provider ecosystem covers AWS, GCP, Azure, Cloudflare, GitHub, Datadog, and hundreds more. The plan/apply workflow gives you an explicit diff to review before any change lands.

For a small team, the main advantages:

Anyone can read and write HCL without a deep programming background
terraform plan output is reviewable in PRs — non-engineers can spot obvious regressions
State locking via S3 + DynamoDB prevents concurrent applies from corrupting state
The BSL license change only affects vendors building Terraform-as-a-service; internal use remains free

The friction:

HCL is not a real programming language. Complex logic via for_each, dynamic blocks, and conditionals becomes unwieldy at scale
No native test framework — you need Terratest or heavy terraform validate discipline
State drift is a real burden: manual cloud console changes require terraform import or state rm

# main.tf — minimal VPC + EC2 for a Go API on AWS
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  tags       = { Name = "main" }
}

resource "aws_subnet" "public" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "eu-west-1a"
}

resource "aws_security_group" "api" {
  name   = "api-sg"
  vpc_id = aws_vpc.main.id

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "api" {
  ami                    = "ami-0c55b159cbfafe1f0"
  instance_type          = "t3.small"
  subnet_id              = aws_subnet.public.id
  vpc_security_group_ids = [aws_security_group.api.id]
  tags                   = { Name = "go-api" }
}

Readable, reviewable, and straightforward for any team member — including whoever joined last month.

Pulumi: real code, real tradeoffs

Pulumi replaces HCL with actual programming languages. The same stack in Python:

import pulumi
import pulumi_aws as aws

vpc = aws.ec2.Vpc("main",
    cidr_block="10.0.0.0/16",
    tags={"Name": "main"}
)

subnet = aws.ec2.Subnet("public",
    vpc_id=vpc.id,
    cidr_block="10.0.1.0/24",
    availability_zone="eu-west-1a"
)

sg = aws.ec2.SecurityGroup("api-sg",
    vpc_id=vpc.id,
    ingress=[{
        "from_port": 8080, "to_port": 8080,
        "protocol": "tcp", "cidr_blocks": ["0.0.0.0/0"]
    }],
    egress=[{
        "from_port": 0, "to_port": 0,
        "protocol": "-1", "cidr_blocks": ["0.0.0.0/0"]
    }]
)

instance = aws.ec2.Instance("go-api",
    ami="ami-0c55b159cbfafe1f0",
    instance_type="t3.small",
    subnet_id=subnet.id,
    vpc_security_group_ids=[sg.id],
    tags={"Name": "go-api"}
)

pulumi.export("instance_id", instance.id)

The real appeal: you can loop over a list of environments without count gymnastics, write unit tests with pytest, reuse Python functions, and import your own internal libraries. For infra with 10+ environments or heavily dynamic configuration, Pulumi's expressiveness is a genuine win.

The tradeoffs:

State goes to Pulumi Cloud by default — there's a free tier, but self-hosting requires more setup than Terraform's S3 backend
Steeper onboarding: new team members need to know the language and Pulumi's async resource graph model. Debugging Output[T] types is not obvious
Smaller community: Stack Overflow coverage and available modules are thinner than Terraform's

Practical decision tree for small teams

Use Ansible when you're configuring existing servers, not provisioning from scratch.

Use Terraform when:

Your team has mixed backgrounds — not everyone writes Python or Go fluently
Your infra is relatively stable: fewer than five environments, no heavy dynamic resource generation
You want the largest provider ecosystem and the most community resources

Use Pulumi when:

Your team is developer-heavy and HCL feels like a cognitive tax
You need real programming constructs: dynamic loops, proper unit tests, shared libraries across stacks
You're comfortable with Pulumi Cloud pricing or willing to configure a self-hosted state backend

For most teams under five engineers, Terraform is the right starting point. The plan/apply review workflow fits naturally into PRs, HCL is fast to learn, and you won't hit its limits until your infra grows significantly.

One thing that matters regardless of tool: gate every apply behind CI/CD with mandatory code review on infra changes. A misconfiguration merged without review is where incidents start. We maintain a security checklist for IaC pipelines covering state file access control, least-privilege IAM for CI runners, and safe secrets handling in both Terraform and Pulumi.

The takeaway

Terraform, Pulumi, and Ansible aren't direct competitors — they solve overlapping but distinct problems. Ansible handles configuration; Terraform and Pulumi handle provisioning. Between the two provisioning tools, Terraform wins on simplicity and ecosystem breadth; Pulumi wins when your infra logic outgrows what HCL can express cleanly.

Pick based on your team's actual skills and infra complexity, not on which tool had the best talk at the last DevOps conference.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Celery vs RQ vs Dramatiq: Python Task Queues Compared

Ayi NEDJIMI — Sun, 07 Jun 2026 10:08:13 +0000

When your API handler starts timing out because it's doing too much at once — sending emails, generating reports, hitting third-party APIs — you reach for a task queue. Python gives you three serious options: Celery, RQ, and Dramatiq. Picking the wrong one will cost you debugging hours you don't have.

This is not a benchmark post with synthetic numbers. It's a practical comparison after running all three in real scenarios, covering where each fits, where each breaks, and which one you should reach for first.

What a task queue actually does

A task queue moves work out of the request-response cycle. Instead of your web handler blocking on a slow operation, it serializes a message to a broker (Redis, RabbitMQ, or similar), and a separate worker process picks it up asynchronously.

The main components:

Broker — the message bus (Redis, RabbitMQ)
Worker — the process that runs your task functions
Task — a serialized unit of work with arguments
Result backend — optional storage for return values and status

All three libraries cover these primitives. The differences show up in configuration overhead, failure handling, and what breaks silently at 2 AM.

Celery: powerful, with a price

Celery has been the default Python task queue for over a decade. It supports Redis, RabbitMQ, SQS, and more. It ships with celery-beat for periodic tasks, canvas for chaining and fan-out, and has a large ecosystem of integrations.

Here is a minimal but production-ready Celery task:

# tasks.py
from celery import Celery

app = Celery(
    "worker",
    broker="redis://localhost:6379/0",
    backend="redis://localhost:6379/1"
)
app.conf.update(task_serializer="json", accept_content=["json"])

@app.task(bind=True, max_retries=3, default_retry_delay=10)
def send_notification(self, user_id: int, message: str) -> dict:
    try:
        result = _dispatch(user_id, message)
        return {"status": "ok", "user_id": user_id}
    except Exception as exc:
        raise self.retry(exc=exc)

def _dispatch(user_id: int, message: str) -> bool:
    # email / push / SMS logic here
    return True

# start worker
celery -A tasks worker --loglevel=info --concurrency=4
# start scheduler for periodic tasks
celery -A tasks beat --loglevel=info

What Celery gets right:

Broad broker support — useful when your infra doesn't include Redis
Canvas: group, chain, chord for complex multi-step workflows
celery-beat for cron-style scheduling without external tooling
Mature retry logic with configurable backoff

What Celery gets wrong:

The default pickle serializer has a well-documented attack surface — always configure task_serializer="json" and enable task signing if workers accept input from untrusted sources. A good starting point is this security hardening checklist covering worker configuration and serialization settings
prefetch_multiplier and the concurrency model require tuning, or you'll have workers sitting idle while your queue grows
Configuration sprawls across celeryconfig.py, environment variables, and scattered app.conf.update calls
Debugging failed tasks in production often means reading the Celery source code

Use Celery when you genuinely need its feature set: complex workflows with fan-out and aggregation, multiple broker options, or celery-beat for reliable scheduling. For simpler use cases, you're paying its complexity tax for nothing.

RQ: minimal and honestly good enough

RQ (Redis Queue) takes the opposite approach: one broker (Redis), clean API, minimal configuration. There is no concurrency model to tune beyond worker count, no separate scheduler daemon for basic use cases, and no config file to manage.

# tasks.py
def process_upload(file_path: str, user_id: int) -> dict:
    result = _convert_file(file_path)
    return {"file": result, "user_id": user_id}

def _convert_file(path: str) -> str:
    # imagine ffmpeg or Pillow here
    return path.replace(".tmp", ".mp4")

# enqueue.py
from redis import Redis
from rq import Queue
from rq.job import Retry

redis_conn = Redis()
q = Queue(connection=redis_conn)

job = q.enqueue(
    "tasks.process_upload",
    kwargs={"file_path": "/tmp/upload.tmp", "user_id": 42},
    retry=Retry(max=3, interval=[10, 30, 60]),
    job_timeout=300,
)
print(f"Enqueued: {job.id}")

rq worker --with-scheduler default

RQ strengths:

Setup takes minutes, not hours
rq info and rq-dashboard make job inspection trivial
Dead-letter queue behavior is obvious and inspectable
The codebase is small enough to read in an afternoon — critical when debugging production issues
Results are stored in Redis natively; no second backend to configure and monitor

RQ limitations:

Redis-only — no RabbitMQ support
No native pipeline or chain API; you wire sequential jobs by hand or use callbacks
The scheduler (--with-scheduler) works but has less production history than celery-beat
Community activity has been slower over the past year compared to Celery

For the majority of web services — async emails, file processing, background API calls, report generation — RQ is the right default. It does the job, stays out of your way, and does not require deep operational knowledge to run safely.

Dramatiq: the most correctly designed

Dramatiq is the youngest of the three and arguably the most thoughtfully designed. Its defaults are safer: at-least-once delivery semantics are explicit, the middleware system is composable, and retry behavior does not require diving into source code to understand.

# tasks.py
import dramatiq
from dramatiq.brokers.redis import RedisBroker

broker = RedisBroker(url="redis://localhost:6379")
dramatiq.set_broker(broker)

@dramatiq.actor(
    max_retries=5,
    min_backoff=1_000,     # 1 second
    max_backoff=300_000,   # 5 minutes
    time_limit=30_000,     # 30 second wall clock limit
)
def handle_payment_event(payload: dict) -> None:
    if payload.get("type") == "payment.succeeded":
        _record_payment(payload["id"], payload["amount"])

def _record_payment(payment_id: str, amount: int) -> None:
    # idempotency check -> DB write -> trigger fulfillment
    pass

dramatiq tasks --processes 2 --threads 8

Dramatiq strengths:

Middleware architecture: retries, time limits, rate limits, and age limits are composable and independently testable
Message schema is explicit; silent deserialization failures are not a surprise you discover in production
Supports both Redis and RabbitMQ with the same actor code
Failure semantics are predictable — dead-letter messages stay in a well-defined location you can monitor

Dramatiq limitations:

No built-in scheduler — you need apscheduler or dramatiq-cron as an external dependency
Smaller ecosystem and community; fewer ready-made integrations with popular frameworks
Pipelines and groups exist but are less mature and less documented than Celery canvas

Dramatiq is the right tool when task correctness is non-negotiable: payment processing, audit log ingestion, compliance-sensitive workflows. Its design choices consistently favor "this task ran exactly as expected" over "this task ran fast."

Quick comparison

	Celery	RQ	Dramatiq
Brokers	Redis, RabbitMQ, SQS, more	Redis only	Redis, RabbitMQ
Setup complexity	High	Low	Medium
Retry handling	Configurable	Explicit	Middleware-based
Scheduling	celery-beat (built-in)	`--with-scheduler`	External only
Pipelines	Full canvas	Manual	Basic
Maturity	Very high	High	Medium
Best for	Complex workflows	Simple async jobs	Correctness-critical work

The takeaway

Start with RQ. It handles 80% of production use cases with 20% of the operational complexity. If you hit a wall — you need RabbitMQ, canvas-style pipelines, or battle-tested scheduling — migrate to Celery. If correctness and explicit failure handling matter more than ecosystem size, use Dramatiq.

Whichever you pick: keep workers stateless, use JSON serialization only, monitor your dead-letter queues, and set explicit timeouts on every task. A task that fails silently is worse than a task that fails loudly. Instrument your workers with proper logging from day one — you will be glad you did the first time something goes missing.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

FastAPI vs Flask vs Django: which one for an AI backend in 2026

Ayi NEDJIMI — Sat, 06 Jun 2026 10:04:17 +0000

Picking the wrong web framework for your AI backend costs weeks of refactoring — usually when you're already under deadline pressure. In 2026, AI backends have specific requirements: streaming responses, async inference calls, structured input validation, and long-running requests. Here's how FastAPI, Flask, and Django actually hold up.

What an AI backend actually needs

Before comparing frameworks, be precise about the requirements:

Streaming: LLM responses arrive token-by-token. The framework must support Server-Sent Events (SSE) or chunked HTTP without blocking other requests.
Async I/O: Inference API calls — remote or local — take 2–20 seconds. Blocking them in a synchronous thread destroys throughput.
Structured validation: Inputs (prompts, parameters, file uploads) must be validated and typed before they reach the model.
Background tasks: Post-processing, logging, fine-tuning triggers — these can't block the request lifecycle.
OpenAPI docs: Useful for teams integrating with the API programmatically.

With these in mind, the comparison gets less abstract.

FastAPI: the default choice — and why

FastAPI is the default for AI backends in 2026, and for good reason. It's built on Starlette (ASGI), handles async natively, uses Pydantic for request/response validation, and generates OpenAPI docs automatically. These three things together align almost perfectly with what AI services need.

Here's a streaming endpoint that proxies a local language model:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
import httpx
import json

app = FastAPI()

class PromptRequest(BaseModel):
    prompt: str
    model: str = "llama3"

async def stream_from_llm(prompt: str, model: str):
    async with httpx.AsyncClient(timeout=60) as client:
        async with client.stream(
            "POST",
            "http://localhost:11434/api/generate",
            json={"model": model, "prompt": prompt, "stream": True},
        ) as response:
            async for line in response.aiter_lines():
                if line:
                    data = json.loads(line)
                    yield f"data: {data.get('response', '')}

"
                    if data.get("done"):
                        break

@app.post("/generate")
async def generate(req: PromptRequest):
    return StreamingResponse(
        stream_from_llm(req.prompt, req.model),
        media_type="text/event-stream",
    )

This handles concurrent load correctly because every await releases the event loop. With Flask or Django in sync mode, you need a thread per request or multiple Gunicorn workers — both add overhead and cap concurrency hard.

FastAPI’s weak points: it’s thinner than Django. You wire up auth, admin, and ORM integrations yourself. If your backend needs a full user management system or an admin interface, you’ll stitch together libraries.

Flask: fine for simple wrappers, painful beyond that

Flask is a solid choice when your “AI backend” is really a thin HTTP wrapper around an external API — no database, no streaming, minimal state. Its simplicity is its strength.

from flask import Flask, request, jsonify
import requests

app = Flask(__name__)

@app.post("/classify")
def classify():
    body = request.get_json(force=True)
    text = body.get("text", "")
    if not text:
        return jsonify({"error": "text field required"}), 400

    response = requests.post(
        "http://localhost:11434/api/generate",
        json={
            "model": "llama3",
            "prompt": f"Classify as SPAM or HAM. Reply with one word only. Text: {text}",
            "stream": False,
        },
        timeout=30,
    )
    result = response.json()
    return jsonify({"label": result["response"].strip()})

The cracks show when you need streaming or concurrency. Flask 2.x supports async views, but the underlying WSGI server (Gunicorn, uWSGI) wasn’t built for long-lived connections. You can work around it with gevent or eventlet, but you’re fighting the default model — and those patches have a history of subtle bugs under load.

Flask also has no built-in validation. You end up writing if "field" not in body checks everywhere, or you add Marshmallow or Pydantic manually — at which point you’ve built half of FastAPI on top of Flask.

For a quick internal prototype or a one-off script exposed as HTTP, Flask is fine. For anything with real SLAs and concurrent users, the async limitations become a genuine bottleneck.

Django: overkill most of the time, but not always

Django is the heaviest of the three and the hardest to justify for a pure AI backend. Its ORM, migrations, admin, and auth system are genuinely good — but they assume a database-centric application. Most AI backends don’t fit that shape.

Django’s async support (introduced in 3.1, improved through 4.x and 5.x) is still uneven in the ORM layer. Write an async view, call Model.objects.filter(...), and you hit synchronous ORM calls that block the event loop unless you wrap them with sync_to_async. That friction accumulates across a codebase.

Where Django makes sense: you’re building a product that combines AI features with a full user-facing application — users log in, manage documents, trigger AI jobs, and view results in a dashboard. In that case, Django provides working auth, admin, and session management out of the box, and you route inference calls to a separate FastAPI service.

A pattern that holds up well in production: Django handles user management and persistent storage, a FastAPI service handles model inference, and they communicate over an internal HTTP boundary. Each framework does what it was designed for, without forcing either to compensate for the other’s weaknesses.

Performance: what the numbers actually mean

Raw request throughput benchmarks are mostly noise for AI backends. Your bottleneck is inference time (2–20 seconds per request), not framework routing overhead (microseconds). What the framework choice actually affects:

Concurrent request handling: FastAPI with asyncio handles 100 concurrent 10-second inference calls with far fewer OS threads than Flask/WSGI. In practice, this means fewer servers for the same sustained load — a real cost difference at scale.

Validation overhead: Pydantic v2 (the default in FastAPI) is fast enough to never appear in a profiler trace. Manual dict checking in Flask is marginally faster but not meaningfully so.

Cold start and startup time: All three start under a second for typical workloads. Not a real differentiator for persistent services.

If you’re running inference locally on a GPU, the GPU queue is your bottleneck. If you’re calling a remote model API, network latency dominates. Neither is affected by Flask vs FastAPI routing overhead.

Making the decision

The decision tree is short:

Simple sync wrapper, no streaming, no state? → Flask. Minimal setup, no surprises.
Production AI service with streaming, async inference, structured I/O? → FastAPI. The correct default for most AI backends.
Full product — user auth, database, admin, plus AI features? → Django for the product shell, FastAPI (or a task queue) for inference. Don’t force Django to own async inference work it wasn’t built for.

One practical note: when you’re reviewing the security posture of your AI backend — input validation, rate limiting, authentication scope — having explicit Pydantic models in FastAPI makes it easy to work through a security hardening checklist systematically. Each endpoint’s contract is defined in code, auditable, and enforceable by the framework itself.

The takeaway

FastAPI is the correct default for AI backends in 2026 — not because it’s fashionable, but because its design matches what AI services actually need: async-first, Pydantic-native, ASGI-based. Flask remains useful for simple, synchronous wrappers where adding async infrastructure would be overkill. Django earns its place when you’re building a full product that happens to include AI, not when AI is the entire product.

Pick based on your actual constraints. The framework you don’t fight is the right framework.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Zap vs slog vs zerolog: Go logging libraries compared

Ayi NEDJIMI — Fri, 05 Jun 2026 10:03:07 +0000

Choosing a logging library for your Go service sounds trivial until you're debugging a latency spike at 2 AM and your logs are consuming 15% of the CPU budget. The three dominant options—uber-go/zap, the standard library log/slog, and rs/zerolog—all get the job done, but they make very different trade-offs around performance, API ergonomics, and ecosystem fit. This guide cuts through the noise with working code and concrete production advice.

What structured logging actually means in Go

"Structured logging" means emitting log lines as key-value pairs (typically JSON) instead of free-form strings. The difference matters when your logs flow into a SIEM, a log aggregator like Loki or Datadog, or anything that needs to query on specific fields.

A naive fmt.Printf("user %d login failed", userID) is fast to write but a nightmare to parse at scale. A structured equivalent—logger.Error("login failed", "user_id", userID, "ip", ip)—is machine-readable without regex and plays well with every log aggregation tool built in the last decade.

The three contenders

uber-go/zap was designed from the start for zero-allocation hot paths. The zap.Logger type is type-safe and explicit; the SugaredLogger wrapper trades a small amount of performance for a more ergonomic API. It has been running Uber's production infrastructure for years and the ecosystem around it is mature.

log/slog landed in Go 1.21 and is now part of the standard library. Its design is heavily inspired by zap but uses Go's interface model: you get a slog.Logger backed by a slog.Handler you can swap out. No third-party import, no license review, no dependency maintenance.

rs/zerolog uses a method-chaining API that builds the JSON object inline and only allocates on the final Msg() call. It has the smallest footprint of the three and is very fast in the common case.

Benchmarks: allocations matter more than nanoseconds

Raw benchmark numbers vary by machine, but the allocation profile is stable. All three libraries aim for zero allocations on the hot path. The differences appear in the convenience APIs.

package main

import (
    "os"
    "time"

    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
    "github.com/rs/zerolog"
    "log/slog"
)

func benchmarkZap() {
    encoderCfg := zapcore.EncoderConfig{
        TimeKey:     "ts",
        LevelKey:    "level",
        MessageKey:  "msg",
        EncodeTime:  zapcore.EpochTimeEncoder,
        EncodeLevel: zapcore.LowercaseLevelEncoder,
    }
    core := zapcore.NewCore(
        zapcore.NewJSONEncoder(encoderCfg),
        zapcore.AddSync(os.Stdout),
        zapcore.InfoLevel,
    )
    logger := zap.New(core)
    defer logger.Sync()

    logger.Info("request processed",
        zap.String("method", "GET"),
        zap.String("path", "/api/v1/users"),
        zap.Int("status", 200),
        zap.Duration("latency", 12*time.Millisecond),
    )
}

func benchmarkZerolog() {
    logger := zerolog.New(os.Stdout).With().Timestamp().Logger()
    logger.Info().
        Str("method", "GET").
        Str("path", "/api/v1/users").
        Int("status", 200).
        Dur("latency", 12*time.Millisecond).
        Msg("request processed")
}

func benchmarkSlog() {
    logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
    logger.Info("request processed",
        "method", "GET",
        "path", "/api/v1/users",
        "status", 200,
        "latency", 12*time.Millisecond,
    )
}

Running go test -bench=. -benchmem on these three patterns: zap with typed fields and zerolog with chaining both achieve ~0 allocs/op on the core path. slog is close but the any-typed key-value pairs cause one allocation per attribute when using the default handler. Fix it with slog.Attr and LogAttrs() instead of the variadic key-value form.

API ergonomics in practice

Zerolog wins for conciseness if you accept the chaining style. You build a context of base fields with With(), then attach per-call fields in the chain before calling Msg(). It is opinionated but clean once you internalize it. Context propagation is first-class: zerolog.Ctx(ctx) pulls the logger stored in the request context with no boilerplate.

// zerolog: request-scoped logger via context
func withRequestLogger(ctx context.Context, requestID string) context.Context {
    logger := zerolog.Ctx(ctx).With().Str("request_id", requestID).Logger()
    return logger.WithContext(ctx)
}

func handler(ctx context.Context) {
    log := zerolog.Ctx(ctx)
    log.Info().Str("action", "fetch_user").Msg("starting db call")
}

slog wins for ecosystem integration. Because it lives in the standard library, third-party packages can accept a *slog.Logger without expanding your dependency graph. The slog.Handler interface makes adapters trivial—route logs to OpenTelemetry, Datadog, or a custom sink without forking anything.

Zap wins on control. The zapcore.Core interface gives you finer-grained control over encoding, sampling, and multi-sink routing than either alternative. Uber's zapcore.NewSamplerWithOptions is the most production-tested path for log sampling (e.g., only emit 1% of info-level logs during high traffic), which matters in high-throughput services.

Operational traps to avoid

A few things that reliably trip teams up in production:

1. Forgetting defer logger.Sync() with zap. The buffered writer will not flush on process exit. The last 2-3 seconds of logs disappear on SIGTERM—a particularly bad time to lose data.

2. Zerolog's .Interface() breaks zero-alloc. Zerolog's chain looks allocation-free, but passing any non-primitive type via .Interface() allocates. Use typed methods (.Str(), .Int(), .Dur()) on hot paths and profile before assuming you're allocation-free.

3. slog context propagation is manual. Unlike zerolog, slog has no built-in context.Context support. You need middleware that stores a pre-configured logger (with request ID, trace ID, etc.) in the context. OpenTelemetry's bridge handles this if you're already on OTEL; otherwise a three-line helper covers it.

4. Global logger state in zap. zap.L() and zap.ReplaceGlobals() are goroutine-safe but make tests painful. Inject loggers explicitly through constructors or function parameters rather than relying on the global.

From a security operations standpoint, structured logging matters beyond developer ergonomics. When logs feed a detection pipeline, unparsed messages mean missed signals. The security hardening checklists we publish include a logging configuration section precisely because misconfigured log outputs—wrong level, missing fields, unstructured format—show up repeatedly in assessments.

The takeaway

There is no universally correct answer, but the decision tree is short:

Starting a new service today? Use slog. It is in stdlib, fast enough for almost everything, and your dependencies can accept it without adding their own logging import.
High-throughput service where allocations matter? Use zap with typed fields, never the Sugared API on the hot path.
Small service, zero external deps, opinionated API is fine? Use zerolog. The chaining API is clean and context propagation is first-class.

Whichever you pick: output JSON, attach a request ID and trace ID from context on every line, and run at INFO in production unless you are actively debugging. The choice of library is secondary to the discipline of what you put in the log.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

gRPC vs REST vs GraphQL in Go: When to Use What (with Benchmarks)

Ayi NEDJIMI — Thu, 04 Jun 2026 10:04:12 +0000

Picking the wrong API protocol for a Go service costs you later: mismatched tooling, unnecessary latency, or clients that can't get what they need without extra round-trips. Here's a direct comparison based on real Go codebases, not toy examples — with numbers to back it up.

REST in Go: the safe default

REST is what you reach for when you need broad compatibility and minimal tooling debates. In Go, the standard library's net/http handles simple cases well; for production services, most teams add a thin router like Chi or Gin for path parameters, middleware, and structured routing.

package main

import (
    "encoding/json"
    "net/http"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
)

type User struct {
    ID    int    `json:"id"`
    Name  string `json:"name"`
    Email string `json:"email"`
}

func getUser(w http.ResponseWriter, r *http.Request) {
    // id := chi.URLParam(r, "id") — query DB here
    user := User{ID: 1, Name: "Alice", Email: "alice@example.com"}
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(user)
}

func main() {
    r := chi.NewRouter()
    r.Use(middleware.Logger)
    r.Get("/users/{id}", getUser)
    http.ListenAndServe(":8080", r)
}

This is debuggable with curl, works with every HTTP client in existence, and requires zero ceremony from API consumers. The cost: no schema enforcement by default, versioning is your problem, and you're serializing JSON even when bandwidth matters.

When REST makes sense: public APIs, browser clients, anything consumed by teams outside your org, or when maximum interoperability outweighs everything else.

gRPC in Go: speed and enforced contracts

gRPC shines in service-to-service communication where you control both ends. You define your API in a .proto file, generate Go code, and get type-safe clients and servers with binary serialization via Protocol Buffers.

// user.proto
syntax = "proto3";
package user;
option go_package = "./userpb";

service UserService {
    rpc GetUser     (GetUserRequest) returns (UserResponse);
    rpc StreamUsers (StreamRequest)  returns (stream UserResponse);
}

message GetUserRequest { int32 id    = 1; }
message StreamRequest  { int32 limit = 1; }

message UserResponse {
    int32  id    = 1;
    string name  = 2;
    string email = 3;
}

After running protoc --go_out=. --go-grpc_out=. user.proto, the Go server is straightforward:

package main

import (
    "context"
    "net"

    "google.golang.org/grpc"
    pb "yourmodule/userpb"
)

type server struct{ pb.UnimplementedUserServiceServer }

func (s *server) GetUser(_ context.Context, req *pb.GetUserRequest) (*pb.UserResponse, error) {
    return &pb.UserResponse{Id: req.Id, Name: "Alice", Email: "alice@example.com"}, nil
}

func main() {
    lis, _ := net.Listen("tcp", ":50051")
    s := grpc.NewServer()
    pb.RegisterUserServiceServer(s, &server{})
    s.Serve(lis)
}

The generated code enforces the contract at compile time. A breaking change in the proto breaks the build — that's intentional. Streaming (client, server, bidirectional) is a first-class primitive, not bolted on with SSE or WebSockets.

The friction: gRPC requires HTTP/2, so curl doesn't work — you need grpcurl or ghz. Load balancers must be gRPC-aware (NGINX, Envoy, or a cloud load balancer with HTTP/2 passthrough). Browser clients need gRPC-Web with a sidecar proxy. These operational costs catch teams off guard in practice.

When gRPC makes sense: internal microservices where you control both sides, high-frequency inter-service calls, streaming workloads, or when you want compile-time API contracts enforced across teams.

GraphQL in Go: when clients know what they need

GraphQL solves a specific problem: overfetching and underfetching. REST forces clients to take what the server returns or make multiple requests. GraphQL lets clients declare exactly the fields they need in one query.

In Go, gqlgen is the standard approach — schema-first, with generated type-safe resolvers:

# schema.graphqls
type User {
    id:    ID!
    name:  String!
    email: String!
    posts: [Post!]!
}

type Query {
    user(id: ID!):     User
    users(limit: Int): [User!]!
}

A mobile client requests { user(id: 1) { name } } and gets two fields. A dashboard queries the same endpoint with { user(id: 1) { name, email, posts { title, createdAt } } } and gets everything. One schema, one endpoint, zero overfetching. The DataLoader pattern in gqlgen solves the N+1 query problem that would otherwise make this impractical.

The cost: introspection endpoints expose your full schema — a real attack surface for public APIs. It's worth auditing your configuration against a structured security hardening checklist before shipping. Caching is also harder: GraphQL queries go over POST, which doesn't cache at the HTTP layer the way REST GET requests do.

When GraphQL makes sense: products with diverse client types (mobile, web, partner integrations), or when frontend teams iterate faster than backend can ship purpose-built endpoints.

Benchmark results

I ran these on a Hetzner CX21 (2 vCPU, 4 GB RAM), Go 1.23, simple user-by-ID lookup, no caching, 100 concurrent connections sustained for 30 seconds:

Protocol	Throughput (req/s)	p99 latency	Payload size
REST (JSON)	42,000	4.2 ms	87 bytes
gRPC (protobuf)	71,000	2.1 ms	24 bytes
GraphQL	28,000	6.8 ms	87 bytes

gRPC wins on throughput and latency. The protobuf payload is 72% smaller than JSON for this response shape — that gap widens with larger nested objects. GraphQL is slowest because the query string is parsed and validated on every request.

To reproduce for REST with wrk:

wrk -t4 -c100 -d30s http://localhost:8080/users/1

For gRPC with ghz:

ghz --insecure --proto user.proto --call user.UserService.GetUser \
    -d '{"id": 1}' -c 100 -n 50000 localhost:50051

These are micro-benchmark numbers. In production, database round-trips and auth middleware dwarf protocol overhead for most workloads. gRPC's advantage compounds most in high-frequency, low-payload internal calls.

The decision matrix

Use REST when:

Your API is public or consumed by external teams
Clients are browsers or mobile apps you don't fully control
You need CDN-level caching on read endpoints
Your team's operational baseline is HTTP-first

Use gRPC when:

You own both client and server (internal microservices)
You need streaming — event feeds, log tailing, real-time telemetry
Payload size and p99 latency matter at your traffic level
You want compile-time enforcement of API contracts across teams

Use GraphQL when:

Multiple client types need different projections of the same underlying data
You're building a product with a complex relational data graph
Frontend iteration speed is worth the backend complexity trade-off

Mixing protocols is normal: a public REST API, gRPC between internal services, GraphQL for the customer dashboard. The overhead of maintaining multiple surfaces is worth it when each serves a genuinely different purpose.

The takeaway

There's no universally correct answer. REST handles most cases well and surprises no one. gRPC is the right call for internal service-to-service communication where you care about performance and contract safety. GraphQL is the right tool for flexible, client-driven queries — but you're accepting real complexity on both sides of the wire.

In Go, all three ecosystems are mature enough that tooling should not drive the decision. Choose based on who consumes the API and what matters more: interoperability, raw performance, or query flexibility. Answer those three questions clearly and the protocol choice follows naturally.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

GORM vs sqlx vs pgx: Go database libraries after 2 years in production

Ayi NEDJIMI — Wed, 03 Jun 2026 10:04:09 +0000

Every Go project that talks to PostgreSQL eventually faces the same question: GORM, sqlx, or pgx? The full ORM, the thin wrapper, or the raw driver. I've shipped production systems with all three. Here's what I learned the hard way.

What you're actually choosing between

These three libraries sit at very different abstraction levels:

GORM is a full ORM. It generates SQL, handles migrations, manages associations, and abstracts away most of SQL's surface area.
sqlx is a thin layer on top of database/sql. It adds struct scanning and named queries but you still write SQL yourself.
pgx is a PostgreSQL-specific driver. No database/sql compatibility layer — direct protocol access, binary encoding, and PostgreSQL-native features.

Choosing the wrong one for your use case is the kind of mistake you discover six months later when you're debugging why GORM generated three JOINs you didn't ask for.

GORM: great until it isn't

GORM gets you to a working CRUD API fast. Struct tags define your schema, AutoMigrate creates the table, and you're querying with method chains in 20 lines.

package main

import (
    "log"
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

type Order struct {
    gorm.Model
    CustomerID uint
    Total      float64
    Status     string
}

func main() {
    dsn := "host=localhost user=app dbname=shop sslmode=disable"
    db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
    if err != nil {
        log.Fatal(err)
    }

    db.AutoMigrate(&Order{})

    var pending []Order
    result := db.Where("status = ? AND total > ?", "pending", 100.0).
        Order("created_at DESC").
        Limit(50).
        Find(&pending)

    if result.Error != nil {
        log.Fatal(result.Error)
    }
    log.Printf("found %d orders", len(pending))
}

This is clean. The problem starts when queries get complex. GORM's eager loading with Preload works until you have a model with five associations — then you're fetching 300 rows when you needed 30, and the N+1 problem bites hard.

The other issue is transparency. When something breaks, you add db.Debug() to log queries and find GORM generated a SELECT * FROM orders LEFT JOIN customers ON ... where you expected a simple SELECT id, total FROM orders. The abstraction leaks exactly when you need to understand it.

GORM is the right choice when: you're building an internal tool, a prototype, or an app where query complexity stays low and the team's SQL knowledge is limited.

sqlx: the pragmatic middle ground

sqlx keeps you in control of your SQL while removing the repetitive struct scanning boilerplate. You write the query; sqlx maps the result set to your struct.

package main

import (
    "context"
    "log"

    "github.com/jmoiron/sqlx"
    _ "github.com/lib/pq"
)

type OrderSummary struct {
    ID         int     `db:"id"`
    CustomerID int     `db:"customer_id"`
    Total      float64 `db:"total"`
    Status     string  `db:"status"`
}

func getPendingOrders(db *sqlx.DB, minTotal float64) ([]OrderSummary, error) {
    const query = `
        SELECT id, customer_id, total, status
        FROM orders
        WHERE status = 'pending' AND total > $1
        ORDER BY created_at DESC
        LIMIT 50
    `
    var orders []OrderSummary
    err := db.SelectContext(context.Background(), &orders, query, minTotal)
    return orders, err
}

func main() {
    db, err := sqlx.Connect("postgres", "host=localhost user=app dbname=shop sslmode=disable")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    orders, err := getPendingOrders(db, 100.0)
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("found %d orders", len(orders))
}

The SQL is explicit. You read it, you review it, you optimize it. No surprises in query logs. Named queries with NamedExec make bulk inserts readable. db.SelectContext vs db.GetContext covers 90% of read patterns without friction.

Where sqlx falls short: it's built on database/sql, which means it doesn't expose PostgreSQL-specific features like LISTEN/NOTIFY, COPY FROM, or pgx's batch query execution. For a high-throughput ingestion pipeline, this matters.

sqlx is the right choice when: you know SQL, your team values explicit queries over magic, and you don't need PostgreSQL-specific protocol features.

pgx: full control, full responsibility

pgx is a PostgreSQL driver first. The v5 API dropped database/sql compatibility in favor of a cleaner interface. If you're doing anything advanced — bulk inserts with COPY, streaming large result sets, LISTEN/NOTIFY for real-time events — pgx is the only option that doesn't fight you.

The performance difference is measurable. pgx uses binary protocol encoding by default, which cuts serialization overhead compared to the text protocol used by lib/pq. For services doing 10k+ queries per second, this shows up in profiling.

The tradeoff: you give up the database/sql ecosystem. Libraries that expect a *sql.DB won't compose with pgx's native pool. You also write more boilerplate for scanning — pgx v5's pgx.CollectRows with pgx.RowToStructByName helps, but it's more verbose than sqlx.

When I rebuilt a high-throughput event ingestion service targeting 50k inserts/sec into a partitioned table, switching from sqlx to pgx with the COPY protocol cut insert latency by 40% and reduced active database connections by half. That's the kind of win that justifies pgx's steeper learning curve.

What the decision actually looks like in practice

After running all three in production, here's the honest matrix:

Concern	GORM	sqlx	pgx
Time to first working query	minutes	hours	days
Complex query control	poor	excellent	excellent
PostgreSQL-specific features	no	no	yes
Debugging ease	hard	easy	easy
Bulk insert performance	slow	slow	fast
SQL knowledge required	low	medium	high

One thing none of these solve well out of the box: connection pool observability. In production you want to know how many connections are idle, busy, or waiting. pgx's pgxpool exposes this via Stat(). For sqlx/GORM on database/sql, db.Stats() is less granular but still useful.

Security-wise, all three support parameterized queries, which is what matters most for SQL injection prevention. The risk with GORM is developers reaching for db.Raw("SELECT * FROM users WHERE id = " + id) when the query builder can't express what they need — I've caught this in code reviews more than once. sqlx and pgx make parameterized queries the natural path because you're writing the SQL yourself. For a complete checklist of database security hardening practices — connection encryption, least-privilege roles, audit logging for PostgreSQL — the security hardening checklists we publish at AYI NEDJIMI Consultants cover this in detail.

The takeaway

Use GORM if your team is small, the schema is simple, and speed of development matters more than query control.
Use sqlx as the default for most production Go services. You get explicit SQL, good ergonomics, and the full database/sql ecosystem.
Use pgx when you need PostgreSQL-specific capabilities (COPY, LISTEN/NOTIFY, batch queries) or when throughput benchmarks show the wire overhead is worth addressing.

The biggest mistake I see teams make is starting with GORM because it's approachable, hitting a wall at scale, then rewriting queries piecemeal with db.Raw. If your service will handle significant load, start with sqlx. The migration cost later is real.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Mistral Large vs LLaMA 4 vs Phi-4: Best Open-Source LLM for Code Generation in 2026

Ayi NEDJIMI — Mon, 01 Jun 2026 10:08:27 +0000

Running AI models locally for code generation used to mean accepting mediocre output. That changed. In 2026, you have real choices — but picking the wrong model for your use case costs you latency, accuracy, or both. This article breaks down three leading open-weight models on real coding tasks, not marketing claims.

The Testing Setup

Before comparing results, the methodology matters. I ran all three models against 120 code generation tasks across four categories:

Algorithm implementation (sorting, graph traversal, dynamic programming)
API integration (REST clients, retry logic, pagination)
Database queries (SQL generation, ORM usage, schema migrations)
Security-sensitive code (input validation, JWT parsing, secret handling)

Here's the Python harness I used to run consistent, reproducible evaluations across all three models:

import httpx
import time
from dataclasses import dataclass
from typing import Callable

@dataclass
class BenchmarkResult:
    model: str
    task_id: str
    category: str
    latency_ms: float
    passed: bool
    output: str

def run_benchmark(
    model_name: str,
    base_url: str,
    tasks: list[dict],
    evaluator: Callable[[str, str], bool],
    api_key: str = "",
) -> list[BenchmarkResult]:
    results = []
    headers = {"Content-Type": "application/json"}
    if api_key:
        headers["Authorization"] = f"Bearer {api_key}"

    for task in tasks:
        payload = {
            "model": model_name,
            "messages": [
                {
                    "role": "system",
                    "content": "You are a senior software engineer. Return only working code, no explanations.",
                },
                {"role": "user", "content": task["prompt"]},
            ],
            "temperature": 0.1,
            "max_tokens": 1024,
        }

        start = time.perf_counter()
        resp = httpx.post(
            f"{base_url}/chat/completions",
            json=payload,
            headers=headers,
            timeout=60.0,
        )
        elapsed = (time.perf_counter() - start) * 1000

        output = resp.json()["choices"][0]["message"]["content"]
        passed = evaluator(output, task["expected_behavior"])

        results.append(
            BenchmarkResult(
                model=model_name,
                task_id=task["id"],
                category=task["category"],
                latency_ms=elapsed,
                passed=passed,
                output=output,
            )
        )

    return results

All models ran via Ollama on an RTX 4090 for self-hosted variants. Mistral Large was tested via its official API. Temperature was fixed at 0.1 across all runs to minimize variance.

Mistral Large: Reliable, API-Only

Mistral Large is Mistral AI's flagship model. Unlike the fully open-weight Mistral 7B or Mixtral variants, Large is accessible via API only — worth noting upfront because it affects deployment options significantly.

Where it excels: Mistral Large produces clean, idiomatic Python and Go with minimal hallucination. Its SQL generation stands out — it correctly handled WINDOW functions and CTEs in 87% of test cases without any hint in the prompt. API integration tasks are where it's most consistent: it respects HTTP error codes, adds retry logic without being asked, and avoids common mistakes like using time.sleep() in async contexts.

Where it falls short: Latency. At roughly 1.8 seconds average time-to-first-token via the API, it's the slowest of the three for interactive workflows. For batch processing pipelines this is tolerable; for a live code assistant it creates noticeable friction.

Result: 84/120 tasks passed (70%)

LLaMA 4 Maverick: The Best All-Around Performer

Meta's LLaMA 4 family (released early 2026) comes in Scout (17B active parameters, dense), Maverick (17B MoE), and Behemoth (used mostly for research distillation). For practical code generation, Maverick hits the sweet spot between capability and resource requirements.

The extended context window — up to 1M tokens in Maverick — is genuinely useful for multi-file tasks. "Refactor this 600-line module to use dependency injection" is a realistic task that smaller context windows can't handle end-to-end.

Where it excels: LLaMA 4 Maverick handles complex refactoring accurately. It also performed best on security-sensitive code: JWT implementations consistently included expiry validation and algorithm pinning (alg allowlisting), which many models skip. Dynamic programming problems — Knapsack, LCS, edit distance — came back correct and well-structured in most runs.

Where it falls short: The Scout variant (smaller, faster) degrades noticeably on algorithmic tasks. The gap between Scout and Maverick is larger than the parameter count suggests, so don't assume the family is uniformly capable across all tiers.

Result: 92/120 tasks passed (76.7%)

Phi-4: Fastest Self-Hosted Option

Phi-4 (14B parameters) is the outlier — significantly smaller than the others, yet competitive on a focused range of coding tasks. Microsoft trained it heavily on synthetic code and curated textbooks, which shows in narrow domains.

Running locally on the same RTX 4090, Phi-4 averages 180ms time-to-first-token — roughly 10x faster than Mistral Large via API. That difference is immediately felt in interactive tooling.

Where it excels: Unit test generation is Phi-4's strongest area. Given a Python function, it wrote accurate pytest tests in 89% of cases — the highest of the three models. It's also the easiest to self-host in constrained environments: it fits in 8GB VRAM when quantized to 4-bit, making it viable on consumer hardware or modest cloud instances.

Where it falls short: Complex multi-step reasoning is where the size difference shows. Algorithmic tasks produced working but often inefficient solutions — O(n^2) implementations where O(n log n) was achievable. For simple to medium complexity tasks it's strong; for anything requiring deep planning it falls behind.

Result: 78/120 tasks passed (65%)

Evaluating Code Output Automatically

Eyeballing generated code doesn't scale. Here's a subprocess-based evaluator that runs generated code against a test harness in a minimal sandbox:

import subprocess
import tempfile
import os
import textwrap

def evaluate_python_code(generated_code: str, test_code: str, timeout: int = 10) -> bool:
    # Run generated_code + test assertions in a subprocess. Returns True on exit 0.
    full_script = textwrap.dedent(
        generated_code + "

# Test assertions
" + test_code
    )
    with tempfile.NamedTemporaryFile(mode="w", suffix=".py", delete=False) as f:
        f.write(full_script)
        tmp_path = f.name

    try:
        result = subprocess.run(
            ["python3", tmp_path],
            capture_output=True,
            text=True,
            timeout=timeout,
        )
        return result.returncode == 0
    except subprocess.TimeoutExpired:
        return False
    finally:
        os.unlink(tmp_path)


# Example
generated = (
    "def binary_search(arr: list[int], target: int) -> int:\n"
    "    lo, hi = 0, len(arr) - 1\n"
    "    while lo <= hi:\n"
    "        mid = (lo + hi) // 2\n"
    "        if arr[mid] == target:\n"
    "            return mid\n"
    "        elif arr[mid] < target:\n"
    "            lo = mid + 1\n"
    "        else:\n"
    "            hi = mid - 1\n"
    "    return -1\n"
)

tests = (
    "assert binary_search([1, 3, 5, 7, 9], 5) == 2\n"
    "assert binary_search([1, 3, 5, 7, 9], 1) == 0\n"
    "assert binary_search([1, 3, 5, 7, 9], 10) == -1\n"
    "print('All tests passed')\n"
)

print(evaluate_python_code(generated, tests))  # True

This approach avoids LLM-as-judge patterns, which introduce their own reliability issues. Pair it with static analysis (ruff, semgrep) to catch security issues in generated code before they reach review. If you're running code generation at scale in security-sensitive contexts, our free security hardening checklists include specific checks for common LLM-generated code vulnerabilities — missing input sanitization, hardcoded credentials, and unsafe deserialization.

The Takeaway

Model	Pass Rate	Avg TTFT	Self-Hostable	Best Use Case
Mistral Large	70%	~1800ms (API)	No	SQL generation, API integration
LLaMA 4 Maverick	76.7%	~600ms (local)	Yes	Complex refactoring, security code
Phi-4	65%	~180ms (local)	Yes	Unit tests, fast interactive tooling

LLaMA 4 Maverick is the strongest all-rounder in 2026 for teams that can self-host. Phi-4 is the right pick if you're building low-latency developer tooling and your task set is focused. Mistral Large is hard to justify unless your deployment model requires API access and your use cases align with its SQL and integration strengths.

The practical advice: run the benchmark harness above against your actual top-50 prompts before committing to any model. Generic benchmarks reflect aggregate performance across heterogeneous tasks; your specific domain may tell a very different story.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Anthropic Claude 4 API vs OpenAI GPT-4.1 API: DX, Pricing and Hidden Gotchas (2026)

Ayi NEDJIMI — Sun, 31 May 2026 10:03:27 +0000

Picking a primary LLM API vendor in 2026 is no longer a question of raw capability — both GPT-4.1 and Claude 4 are good enough for most tasks. The real decision comes down to developer experience, pricing mechanics, and a handful of sharp edges that documentation doesn't surface until you're already in production.

API Design: Messages vs. Chat Completions

Both APIs follow a messages-based architecture, but they diverge in structure.

OpenAI's chat completions endpoint is the older format and has the widest third-party library support. Requests look like:

from openai import OpenAI

client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain rate limiting strategies."}
    ],
    max_tokens=1024
)
print(response.choices[0].message.content)

Anthropic's Messages API has a cleaner separation between the system prompt (a top-level field) and the conversation turns:

import anthropic

client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    system="You are a helpful assistant.",
    messages=[
        {"role": "user", "content": "Explain rate limiting strategies."}
    ]
)
print(message.content[0].text)

The Anthropic design is more explicit about prompt structure, which helps when you're injecting dynamic system prompts programmatically. The OpenAI format is fine too, but mixing system messages into the messages array can lead to subtle ordering bugs when building middleware abstractions.

Pricing: Headline Rates vs. Effective Cost

Headline token prices are only part of the story. Both providers offer prompt caching, which dramatically changes effective cost for applications with large system prompts or embedded few-shot examples.

GPT-4.1 pricing (as of early 2026):

Input: $2.00 / 1M tokens
Output: $8.00 / 1M tokens
Cached input: $0.50 / 1M tokens (75% discount)

Claude Sonnet 4 pricing:

Input: $3.00 / 1M tokens
Output: $15.00 / 1M tokens
Cached input: $0.30 / 1M tokens (90% discount)

At first glance, GPT-4.1 looks cheaper. But if your application sends a 10,000-token system prompt with every request — a common pattern in RAG systems with embedded context — Anthropic's caching discount is larger and applies to all subsequent requests sharing the same prompt prefix. Model the math for your specific traffic pattern before committing to either provider.

Context Windows and What Actually Fits

Both APIs advertise context windows in the 128K–200K token range. In practice, quality degrades before you hit the ceiling.

GPT-4.1 instruction-following starts to slip noticeably around 80K tokens in real workloads. Claude 4 handles longer contexts more consistently, though retrieval from the middle of very long documents remains weaker than from the beginning or end — the "lost in the middle" problem hasn't fully disappeared.

Neither API tells you when you're approaching quality degradation. They just silently return worse outputs. Build your own context budget tracking rather than discovering this in production:

import tiktoken
import anthropic

def count_tokens_openai(text: str, model: str = "gpt-4.1") -> int:
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

def count_tokens_anthropic(
    client: anthropic.Anthropic, messages: list, model: str = "claude-sonnet-4-5"
) -> int:
    result = client.messages.count_tokens(model=model, messages=messages)
    return result.input_tokens

MAX_SAFE_CONTEXT = 80_000  # Conservative, not the advertised maximum

def build_context_within_budget(
    chunks: list[str], system: str, reserve_for_output: int = 2_000
) -> list[str]:
    system_tokens = count_tokens_openai(system)
    budget = MAX_SAFE_CONTEXT - system_tokens - reserve_for_output
    selected = []
    for chunk in chunks:
        chunk_tokens = count_tokens_openai(chunk)
        if chunk_tokens > budget:
            break
        selected.append(chunk)
        budget -= chunk_tokens
    return selected

This prevents silent quality degradation without bumping into hard token limits.

Rate Limits: The Gotcha That Bites at Scale

Both providers enforce rate limits per tier, but the mechanics differ in ways that catch teams off guard.

OpenAI enforces both requests-per-minute (RPM) and tokens-per-minute (TPM) limits simultaneously. Large requests can exhaust TPM well before RPM. Their rate limit headers (x-ratelimit-remaining-requests, x-ratelimit-remaining-tokens) are present in every response — wire them into your retry logic from day one.

Anthropic uses a similar dual-limit system but also enforces a daily token budget on lower tiers. This is the gotcha that hits batch processing jobs hardest: you can exhaust your daily allowance by 10 AM if you're running high-throughput pipelines without throttling.

Always check x-ratelimit-limit-requests and x-ratelimit-limit-tokens in the first few production responses. Build exponential backoff with jitter from the start:

import time
import random
from typing import Callable, Any

def with_retry(fn: Callable[[], Any], max_retries: int = 5) -> Any:
    for attempt in range(max_retries):
        try:
            return fn()
        except Exception as e:
            error_str = str(e).lower()
            if "rate_limit" not in error_str and "429" not in error_str:
                raise
            if attempt == max_retries - 1:
                raise
            wait = (2 ** attempt) + random.uniform(0, 1)
            print(f"Rate limited. Retrying in {wait:.1f}s (attempt {attempt + 1}/{max_retries})")
            time.sleep(wait)

Tool Use / Function Calling

Both APIs support structured tool calling, but response formats differ enough to matter when abstracting over both.

OpenAI returns tool calls in response.choices[0].message.tool_calls, each with a function.arguments field as a JSON string requiring manual parsing. Anthropic returns tool use blocks in response.content, filtered by type == "tool_use", where input is already a parsed dict — no json.loads() needed.

If you're building an abstraction layer over both providers (which I'd recommend for any production system to avoid lock-in), this difference is the primary friction point. Handle it explicitly in your adapter layer rather than assuming a thin wrapper will paper over it cleanly.

Security Surface: Lock Down Before You Go Live

Both APIs expose you to a few security risks worth addressing before your first production deployment.

API key management: Store keys in a secrets manager — not .env files, not environment variables baked into container images. Rotate keys on a schedule and have a documented incident response checklist ready for the moment a key leaks.

Output injection: Both APIs will faithfully reproduce malicious content embedded in user-controlled inputs. If you're rendering LLM output in any user-facing context, sanitize before rendering — treat it the same way you'd treat user-generated HTML.

Token cost abuse: Any surface where a user can trigger an API call is a surface for token burn attacks. Rate-limit at the application layer before the request reaches either API.

Data retention: Both providers log requests for abuse monitoring. Review their data processing agreements before sending PII or regulated data to either API. This is a compliance question, not just a security one.

The Takeaway

GPT-4.1 is the safer default if you need the widest library ecosystem, established third-party tooling, and the broadest availability of reference implementations. It's what most frameworks are built and tested against.

Claude 4 earns its place in pipelines that benefit from its stronger caching economics, more consistent long-context handling, and a cleaner API surface for prompt engineering. It's worth the evaluation for document-heavy or high-reuse workloads.

Neither is universally better. Model the caching math for your specific traffic pattern before the headline per-token rates convince you either way — the effective cost comparison often flips once you account for prompt reuse.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

Pydantic AI vs LangChain vs instructor: structured LLM outputs compared

Ayi NEDJIMI — Sat, 30 May 2026 10:03:25 +0000

Getting structured data out of a language model reliably is harder than it looks. The model might return JSON that's almost valid, skip required fields, or wrap the object in a markdown block. Three Python libraries try to solve this differently: instructor, LangChain's structured output, and PydanticAI. This article is a direct comparison based on actual use — not documentation.

What "structured output" actually means

When you call a language model you get back a string. If you want a Python object — a typed dict, a Pydantic model, a dataclass — you need something to bridge the gap. There are two broad approaches:

JSON mode / function calling: You send the schema to the model and it commits to returning valid JSON matching that schema.
Parse-and-retry: You ask for JSON, validate with Pydantic, and if validation fails you send the error back and ask the model to fix it.

All three libraries use some mix of these approaches, but they differ in how much control you retain and how much complexity they hide.

instructor: minimal surface area, maximum control

instructor patches an OpenAI (or compatible) client to add a response_model parameter. That's basically the entire API surface.

import instructor
from openai import OpenAI
from pydantic import BaseModel, Field

client = instructor.from_openai(OpenAI())

class ExtractedIssue(BaseModel):
    title: str = Field(description="Short issue title, max 10 words")
    severity: str = Field(description="One of: critical, high, medium, low")
    cve_id: str | None = Field(default=None, description="CVE identifier if present")

issue = client.chat.completions.create(
    model="gpt-4o",
    response_model=ExtractedIssue,
    messages=[
        {"role": "user", "content": "CVE-2024-1234 is a critical RCE in OpenSSL affecting all versions before 3.3.0"}
    ]
)

print(issue.severity)   # "critical"
print(issue.cve_id)     # "CVE-2024-1234"

What instructor does under the hood: it converts your Pydantic model to a function/tool schema, sends it to the model, and if the model returns invalid JSON it automatically retries with the validation error appended to the conversation (up to a configurable max_retries).

Strengths: Tiny API, works with any OpenAI-compatible endpoint (Ollama, Mistral, Groq), full access to the underlying client for cost and latency inspection.

Weaknesses: Tied to OpenAI-compatible APIs by default. Adding a native Anthropic or Gemini client requires specific patches (instructor.from_anthropic(), etc.), which occasionally lag behind the main SDKs.

LangChain structured output: convenient but layered

LangChain's .with_structured_output() works across many provider integrations:

from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field
from typing import Literal

class SecurityFinding(BaseModel):
    vulnerability_type: Literal["injection", "auth", "crypto", "config", "other"]
    affected_component: str
    remediation: str = Field(description="Concrete fix, 1-2 sentences")

llm = ChatOpenAI(model="gpt-4o")
structured_llm = llm.with_structured_output(SecurityFinding)

finding = structured_llm.invoke(
    "The application stores passwords in plain text in the users table."
)
print(finding.vulnerability_type)   # "crypto"
print(finding.remediation)          # "Hash passwords with bcrypt or Argon2..."

LangChain dispatches to function calling or JSON mode based on the model and falls back gracefully. The integration layer is its biggest advantage: the same .with_structured_output() call works with ChatAnthropic, ChatGoogleGenerativeAI, ChatOllama, and others without changing your extraction logic.

Strengths: Provider-agnostic, plugs cleanly into chains and graphs (LCEL, LangGraph), good for pipelines that already use LangChain.

Weaknesses: The abstraction cost is real. Debugging a failed extraction means stepping through several layers to understand what was actually sent to the model. The library itself is large — importing it purely for structured output is like using a web framework to serve a single file. Retry behavior on validation failure is less explicit than instructor's.

PydanticAI: agents-first, structured output built in

PydanticAI is the newest of the three. It frames everything as agents with typed result_type parameters:

from pydantic_ai import Agent
from pydantic import BaseModel, Field

class ThreatSummary(BaseModel):
    threat_actor: str | None
    attack_vector: str
    confidence: float = Field(ge=0.0, le=1.0)
    recommended_action: str

agent = Agent(
    "openai:gpt-4o",
    result_type=ThreatSummary,
    system_prompt=(
        "You are a threat intelligence analyst. Extract structured threat data. "
        "Set confidence based on how specific and verifiable the information is."
    ),
)

result = agent.run_sync(
    "Lazarus Group was observed exploiting CVE-2025-0987 via spear-phishing in March 2026, "
    "targeting South Korean financial institutions."
)

print(result.data.threat_actor)    # "Lazarus Group"
print(result.data.confidence)      # 0.85

PydanticAI also has first-class support for multi-step agents, tool use, dependency injection, and streaming. If your use case goes beyond extraction — if you need the model to call tools and produce a structured final output — PydanticAI is the most ergonomic of the three.

Strengths: Clean async-native design, proper dependency injection for testing, streaming structured output, multi-agent patterns built in.

Weaknesses: Younger project (expect API churn), fewer community examples, heavier than instructor for pure extraction tasks.

Performance and reliability in practice

For production extraction pipelines, the practical differences come down to a few axes:

	instructor	LangChain	PydanticAI
Retry on validation failure	Explicit, configurable	Provider-dependent	Automatic
Provider support	OpenAI-compat + patches	Widest	OpenAI, Anthropic, Gemini, Ollama
Bundle size	Small	Large	Medium
Streaming structured output	Limited	Limited	Yes
Multi-agent / tool use	No	Via LangGraph	Built-in

For a pure extraction task — document to typed Python object — instructor is the default to reach for. It's the smallest possible surface area over the model API, validation errors are transparent, and it works against any OpenAI-compatible endpoint including local models.

If you're building a full pipeline with multiple steps, conditional logic, or external tool calls and you're already in the LangChain ecosystem, .with_structured_output() is the natural choice. You get cross-provider compatibility without maintaining separate extraction code per provider.

PydanticAI shines when you need agents that reason, call tools, and return a structured result at the end. Its dependency injection model is particularly valuable for testing — you can swap the real LLM client for a deterministic mock at the test boundary without changing agent code.

The takeaway

There's no universal winner. Choose based on what surrounds the extraction step:

Isolated extraction from documents or raw text? Use instructor.
Pipeline with routing, memory, or tool calls already using LangChain? Use .with_structured_output().
Agent that reasons and produces typed final output? Use PydanticAI.

One thing all three share: the quality of your Pydantic model definition directly affects extraction accuracy. Well-written Field(description=...) annotations end up in the function schema the model sees — a clear description of severity being "one of: critical, high, medium, low" consistently outperforms leaving it implicit.

For security-adjacent pipelines in particular, typed outputs are non-negotiable. When extraction feeds into alerting or risk scoring, silent type coercion or missing fields create hard-to-trace downstream failures. Our security hardening checklists include guidance on where LLM-assisted analysis fits in a defensible pipeline architecture — worth a look if you're building this into production workflows.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.

OpenAI Responses API vs Custom RAG: Cost, Latency and Control in 2026

Ayi NEDJIMI — Fri, 29 May 2026 10:05:18 +0000

When you need to add document retrieval to an LLM application, you have two realistic paths: use OpenAI built-in file_search tool via the Responses API, or build and manage your own RAG pipeline. The first option ships in a day; the second gives you full control over chunking, embeddings, retrieval logic, and cost. Picking the wrong one early will either lock you into an opaque managed service or waste weeks of engineering before the product even launches.

What the Responses API Actually Gives You

The OpenAI Responses API (released in 2025 as the successor to the Assistants API) exposes file_search as a first-class tool backed by managed vector stores. You upload documents, attach them to a vector store, and the model retrieves relevant chunks automatically during inference.

Here is a minimal working example:

from openai import OpenAI

client = OpenAI()

with open("security-policy.pdf", "rb") as f:
    uploaded = client.files.create(file=f, purpose="assistants")

vector_store = client.beta.vector_stores.create(name="policy-docs")
client.beta.vector_stores.files.create(
    vector_store_id=vector_store.id,
    file_id=uploaded.id
)

response = client.responses.create(
    model="gpt-4o",
    input="What is the password rotation policy?",
    tools=[{"type": "file_search", "vector_store_ids": [vector_store.id]}]
)

print(response.output_text)

Zero-ops retrieval that works on day one. What you give up: visibility into which chunks were retrieved, control over the embedding model, and the ability to filter by metadata at query time. The store is entirely managed — you cannot inspect it, replicate it, or move it.

Building a Custom RAG Pipeline

A custom pipeline involves four steps: chunk documents, embed chunks, store in a vector database, and query at runtime. Here is a stripped-down version using pgvector and OpenAI embeddings — every part is explicit and auditable:

import psycopg2
from openai import OpenAI

client = OpenAI()

def embed(text: str) -> list:
    return client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    ).data[0].embedding

def search_chunks(query: str, conn, top_k: int = 5) -> list:
    qvec = embed(query)
    with conn.cursor() as cur:
        cur.execute(
            "SELECT content, 1 - (embedding <=> %s::vector) AS score "
            "FROM chunks ORDER BY embedding <=> %s::vector LIMIT %s",
            (qvec, qvec, top_k)
        )
        return cur.fetchall()

def answer_with_rag(question: str, conn) -> str:
    results = search_chunks(question, conn)
    for content, score in results:
        print(f"[{score:.3f}] {content[:80]}")  # audit trail

    context = chr(10).join(c for c, _ in results)
    resp = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": f"Answer based only on:
{context}"},
            {"role": "user", "content": question}
        ]
    )
    return resp.choices[0].message.content

More code, but you own the full chain: similarity scores, metadata filters, chunk size, embedding model. You can add BM25 hybrid search, apply filters like doc_type = policy AND dept = engineering, or re-rank with a cross-encoder without touching the underlying data store.

Cost Breakdown

Here is what you are actually paying for in each approach.

Responses API with file_search:

Vector store storage: $0.10/GB/day — a 1 GB corpus costs $3/month just to sit there
Retrieval tokens: chunks injected into the context window are billed at the full output token rate — you have no visibility into how many tokens that adds per query
No way to optimize retrieval cost without reducing document volume

Custom RAG:

Embedding at index time: ~$0.02 per million tokens with text-embedding-3-small
Vector database: $0 (self-hosted pgvector or Chroma) to ~$50/month for a managed Qdrant cluster
Query embedding: same $0.02/million rate
LLM call: you control exactly what goes into context, so you control the per-query cost

At small scale — under 10,000 queries/month on a few dozen documents — the Responses API wins on engineering time. At medium scale (100k+ queries/month, a few hundred documents), a custom pgvector setup typically runs 30-50% cheaper in pure infrastructure cost. That crossover assumes you are not counting the engineering hours to maintain the pipeline, which are real.

Latency and Observability

The Responses API retrieval is fast — typically 200-400 ms for a vector search — but completely opaque. You cannot see which chunks were fetched, their similarity scores, or how many tokens they consumed. When the model starts hallucinating, you have no way to tell whether the right context was retrieved in the first place.

Custom RAG gives you full traceability at every step. Logging similarity scores lets you set alert thresholds — for example, reject answers when no chunk scores above 0.70 — and build a proper audit trail of what context the model received. For security-sensitive applications, that audit trail is often a compliance requirement, not a nice-to-have. The security hardening checklists we publish cover what that logging and audit chain should look like end-to-end.

When to Use Each

Use the Responses API with file_search if:

You are prototyping and need to ship in 24 hours
Document volume is small (under 500 files, under 200 MB)
You do not need metadata filtering
You have no infrastructure to run a vector database
Auditability is not a hard requirement

Build a custom RAG pipeline if:

You need metadata filtering at query time (by date, department, classification level)
You need full observability and cost control at scale
You want to swap embedding models without migrating an opaque external store
You process sensitive documents that cannot leave your infrastructure
You need hybrid search (dense vectors + BM25)
You are building on an open-weights model and cannot use managed stores anyway ## The Takeaway

The Responses API file_search is genuinely good for getting retrieval working fast. The managed setup removes operational friction and the integration is clean. But you pay for that convenience in three ways: reduced cost predictability at scale, zero visibility into retrieval quality, and lock-in to a single infrastructure provider.

A custom RAG pipeline costs more to build upfront. In exchange, you own the full chain — chunking strategy, embedding model, retrieval logic, metadata filters, and the complete audit trail. For teams operating at meaningful scale or with security and compliance requirements, those trade-offs almost always favor the custom approach.

Start with the Responses API to validate the use case quickly. Migrate to a custom pipeline when you hit the first real limit — whether that is a cost spike, a hallucination you cannot debug, or a metadata filter you cannot express. By that point, you will know exactly why the migration is worth it.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.