DEV Community: Veduis

Hermes Agent: A Beginner's Guide to Getting It Running and Configured

Veduis — Tue, 16 Jun 2026 00:00:00 +0000

Hermes Agent: A Beginner's Guide to Getting It Running and Configured

I have spent the last couple weeks running Hermes Agent on my home server, and the experience convinced me that most people who would benefit from this tool never get past the first install step. Not because it's hard. It's not. But the documentation assumes a comfort level with terminals and config files that not everyone has going in.

This guide walks through the whole thing from scratch. I have installed Hermes on three different machines over the past two weeks. By the end, you will have a running Hermes instance, a model configured, and the option to connect it to a messaging platform so you can talk to it from your phone.

What Hermes Agent Actually Is

Before touching any commands, it helps to understand what you are actually installing.

Hermes Agent is an open source autonomous AI agent built by Nous Research. The tagline is "the agent that grows with you," and that description is pretty accurate. Unlike a standard chatbot that forgets everything when the window closes, Hermes maintains persistent memory across sessions. It builds a library of "skills" over time based on tasks you give it, and it can reach you on messaging platforms like Telegram, Discord, or Slack.

The project is open source under the MIT license, meaning you can read the code, self-host everything, and modify it however you want. Your data stays on your own machine or server.

It is not a coding assistant tethered to an IDE. It is closer to a personal agent that you deploy and then interact with however you want, doing tasks in the background and reporting back to you.

What You Need Before Starting

Hermes runs on Linux, macOS, and WSL2 on Windows. If you are on a Windows machine without WSL2 set up, do that first. The agent does not run natively in PowerShell or CMD.

You will also need:

An API key from at least one AI provider (Anthropic, OpenAI, OpenRouter, or Ollama for local models)
A basic comfort level typing commands into a terminal
About 15 to 20 minutes

That's genuinely it. Hermes handles the rest of the installation automatically.

Step 1: Running the Installer

Open your terminal (or WSL2 if you're on Windows) and run the one-line installer:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

The script checks for dependencies, downloads what it needs, and sets everything up. When it finishes, reload your shell:

source ~/.bashrc

If you use zsh, swap that for source ~/.zshrc.

To confirm everything went through, run:

hermes --version

You should see a version number. If you see a "command not found" error, your shell did not pick up the new path. Log out and back in, then try again.

If you want to double-check that your environment is healthy, Hermes ships with a diagnostic command:

hermes doctor

This checks all the required pieces and tells you about anything that's missing or misconfigured.

Step 2: Running the Setup Wizard

Once installed, the fastest way to configure everything is through the interactive setup wizard:

hermes setup

The wizard walks you through:

Choosing your AI provider
Entering your API key
Selecting a model
Basic preferences

This is the recommended path for beginners. The wizard handles all the file writes for you, so you don't need to touch config files manually.

That said, understanding where those files live is useful for troubleshooting later.

Understanding the Config Files

Hermes stores all its settings in a hidden folder in your home directory: ~/.hermes/

Inside that folder, two files do most of the work:

~/.hermes/config.yaml holds non-sensitive settings: your model selection, which tools are enabled, memory settings, and similar preferences. You can open and edit this in any text editor.

~/.hermes/.env holds secrets: API keys, bot tokens, and anything else that should never appear in a log or a git repository.

The separation matters. If you share your config for troubleshooting, you share config.yaml. The .env file stays private.

You can update individual values without opening files directly by using the CLI:

hermes config set OPENROUTER_API_KEY sk-or-your-key-here

The agent knows whether a value is a secret and routes it to the right file automatically.

Step 3: Connecting Your API Key

If you went through hermes setup, your key is already in. If you skipped it or want to add a second provider, here is how to set keys manually.

OpenRouter (Recommended for Beginners)

OpenRouter is a good starting point because it gives you access to many different models under one API key. You pay per token used rather than committing to a subscription.

Go to openrouter.ai and create an account
Go to Keys in your dashboard and create a new key
Copy the key and run:

hermes config set OPENROUTER_API_KEY sk-or-your-key-here

Anthropic

If you have an Anthropic account and want to use Claude directly:

hermes config set ANTHROPIC_API_KEY sk-ant-your-key-here

OpenAI

hermes config set OPENAI_API_KEY sk-your-key-here

Local Models via Ollama

If you are running models locally with Ollama, Hermes supports that too. Set your endpoint:

hermes config set OLLAMA_BASE_URL http://localhost:11434

For local models, keep in mind that Hermes needs a model with at least 64,000 context tokens to perform well. Small models tend to struggle with complex agentic tasks and may produce unreliable tool calls.

Step 4: Picking a Model

With your API key in place, choose which model Hermes should use:

hermes model

This opens an interactive selector that lists the models available through your configured providers. Scroll through and select one.

For most people starting out, a solid choice is one of the Claude Sonnet models via Anthropic or OpenRouter. I tested Claude 3.5 Sonnet through OpenRouter and it handled complex instructions without the tool-call failures I saw with smaller models. The large context window is the reason Hermes needs it.

If cost is a concern, I compared pricing across models before settling on my setup. Claude Haiku costs roughly $0.25 per million input tokens versus $3.00 for Claude 3.5 Sonnet. For basic tasks, the savings add up quickly. Smaller GPT-4o variants fall somewhere in between.

You can change your model at any time by running hermes model again. The change takes effect immediately without restarting anything.

Step 5: Starting Hermes

With everything configured, you can start the agent in a couple ways depending on how you want to interact with it.

Classic CLI

The simplest option is to launch the standard command-line interface:

hermes

This drops you into a conversation with the agent. Type a task, hit enter, and watch it work.

Terminal UI (TUI) Mode

If you want a slightly richer terminal experience with mouse support, modal overlays, and non-blocking input:

hermes --tui

The TUI mode is useful when you want to run longer tasks and monitor progress without the output scrolling past you.

Step 6: Connecting to a Messaging Platform

This is where Hermes gets genuinely interesting. Rather than only working through the terminal, you can connect it to a messaging platform and talk to it like a contact on your phone.

Supported platforms include Telegram, Discord, Slack, WhatsApp, and Signal.

I'll walk through Telegram since it's the most straightforward.

Setting Up the Telegram Gateway

Part 1: Create a Telegram Bot

Open Telegram and search for @BotFather
Send /newbot to start the creation flow
Give your bot a display name (anything you like)
Give your bot a username ending in bot (example: myhermes_bot)
BotFather will reply with your Bot Token. Copy it and keep it somewhere safe.

Part 2: Get Your Telegram User ID

Hermes needs to know which Telegram account to trust, so it doesn't respond to arbitrary people who message your bot.

Search for @userinfobot on Telegram
Send it any message
It will reply with your numeric User ID. Copy that number.

Part 3: Configure the Gateway

Run the gateway setup:

hermes gateway setup

Select Telegram, then enter your Bot Token and User ID when prompted. Alternatively, you can add them manually to ~/.hermes/.env:

TELEGRAM_BOT_TOKEN=your_token_here
TELEGRAM_ALLOWED_USERS=your_numeric_id_here

Part 4: Start the Gateway

hermes gateway

The agent is now listening for messages from your Telegram account. Open Telegram, find your bot, and send it a task.

From here, you can run automations from your phone, ask it to look something up, or give it recurring tasks while you're away from your desk.

Keeping Hermes Running Persistently

If you want Hermes to stay active even when you close your terminal session, run it inside a screen or tmux session:

# Using screen
screen -S hermes
hermes gateway

# Detach with Ctrl+A then D

Or on a Linux server, you can set it up as a systemd service so it starts automatically on boot. The Hermes documentation covers this in more detail.

Understanding Skills and Memory

Two features separate Hermes from a standard chatbot: persistent memory and skills.

Persistent memory means Hermes remembers context between sessions. If you told it last week that your main project is in /home/you/projects/webapp, it will still know that today. It uses FTS5 full-text search to recall relevant past conversations.

Skills are something different. When Hermes completes a task successfully, it can extract the sequence of steps that worked and save it as a reusable skill document. Think of skills as a growing playbook: if you asked it to generate a weekly report from your data folder last Tuesday, it might save that workflow as a skill and use it automatically next time you ask for the same thing.

You don't have to actively manage skills. They accumulate in the background as you use the agent. The longer you run it, the better it gets at the specific things you ask of it.

Useful Commands to Know

Here is a quick reference for the commands you will use most often:

Command	What it does
`hermes setup`	Run the interactive configuration wizard
`hermes model`	Switch your AI model or provider
`hermes config set KEY VALUE`	Update a single config value
`hermes gateway`	Start the messaging gateway
`hermes gateway setup`	Configure a messaging platform
`hermes doctor`	Run environment diagnostics
`hermes --tui`	Launch in Terminal UI mode
`hermes --version`	Check your installed version

Common Beginner Mistakes

Using a model with too small a context window. Hermes needs at least 64K tokens of context to handle complex tasks reliably. If you pick a cheap small model and find the agent producing weird results or hallucinated tool calls, model size is usually the reason.

Leaving your API key in config.yaml instead of .env. The setup wizard handles this correctly, but if you're editing files manually, put API keys only in .env.

Not reloading the shell after install. If hermes gives a command not found error right after install, run source ~/.bashrc (or ~/.zshrc) before assuming something went wrong.

Trying to run Hermes in native Windows PowerShell. It needs Linux/macOS or WSL2. This is the most common point of confusion for Windows users.

A Good First Task to Try

Once everything is running, a simple first task to test the setup is a research query:

Find the latest release notes for Hermes Agent from the GitHub repository and summarize what changed

This tests that your model connection works, that Hermes can browse the web, and that it can return a coherent response. I ran this exact query after my first install and it returned a clean summary of the latest three commits. If that works for you, you are in good shape.

From there, give it something more personal: schedule a daily briefing, have it monitor a folder for new files, or ask it to keep notes on a project you're working on.

If you want to go further, try connecting Hermes to a self-hosted n8n workflow(https://veduis.com/blog/automating-business-workflows-n8n-make-zapier/) and have it trigger automations based on messages you send. Our API design patterns guide also covers how to build custom endpoints that Hermes can call through its tool system.

Where to Go from Here

The official Hermes Agent documentation covers advanced topics including Docker sandboxing, SSH backends, subagent delegation, and MCP tool integrations. The GitHub repository is the authoritative source for changelogs and the latest features.

The Nous Research Discord is also active if you run into something this guide doesn't cover.

The setup takes about 20 minutes. The value compounds the longer you run it. Start small, give it a few recurring tasks, and let the skill library build up before you throw anything complicated at it.

Conclusion

Hermes Agent is not a magic solution. It is a tool that rewards consistent use. The persistent memory and skills system mean it gets better the longer you run it, but that improvement requires giving it real tasks and letting it learn from the outcomes.

Start with the setup wizard, connect one messaging platform, and give it a simple recurring task. Within a week, you will have a useful assistant that knows your preferences. Within a month, the skill library will handle common workflows without you needing to explain them again.

The 20 minutes spent on initial setup pays for itself quickly. The key is starting small and building up rather than trying to automate everything on day one.

Multi-Region Architecture: Global Deployment, Data Residency, and Designing for Regional Failures

Veduis — Tue, 16 Jun 2026 00:00:00 +0000

Multi-region deployments are sold as simple solutions to two problems: serving users globally with low latency, and surviving regional cloud outages. The reality is more complicated. Multi-region architecture introduces consistency challenges, data synchronization complexity, and operational overhead that can produce more incidents than a well-designed single-region setup.

This guide covers when multi-region is the right choice, the architectural patterns for implementing it, the data consistency challenges you will face, and the compliance requirements that sometimes force the decision regardless of technical preference.

When Multi-Region Is Worth It

The business case for multi-region usually falls into one of three categories.

Latency for global users. A 200ms round trip from London to a US-east data center is acceptable for many applications but unacceptable for others. Real-time applications (trading, gaming, video calls), user-interactive applications where perceived responsiveness matters, and any application where 200ms round trips compound across many API calls all benefit from geographic proximity.

Regional failure resilience. AWS us-east-1 has had significant outages. GCP us-central1 has had significant outages. Every cloud region has had significant outages. If your business cannot tolerate downtime during a full regional failure, multi-region is a requirement rather than a nice-to-have.

Data residency compliance. GDPR requires that EU personal data not be transferred outside the EU without adequate protections. Some countries have data localization laws requiring that citizen data remain within national borders. Healthcare regulations in some jurisdictions require data to stay in specific regions. These compliance requirements may force multi-region regardless of the technical trade-offs.

If none of these apply, a highly available single-region deployment (multiple Availability Zones, read replicas, failover) is simpler, cheaper, and less risky.

Architectural Patterns

Active-Passive

One region handles all traffic (the active region). The other region maintains a synchronized replica and is ready to accept traffic if the active region fails (the passive region).

Active-Passive pattern: one region serves all traffic while a second region maintains a hot standby.

Traffic flows to Region A. Data is replicated to Region B continuously. If Region A fails, DNS updates to point at Region B. Region B becomes the new active.

Advantages: Simpler to operate. No write consistency challenges. Cheaper than active-active (passive region only needs to handle replica traffic).

Disadvantages: Users far from Region A still experience high latency. Failover requires DNS propagation, which takes time. The passive region's resources are partially wasted in normal operation.

Failover time: DNS TTL (controlled, can be 60-300 seconds) plus time for Region B to become fully operational. This can range from a few minutes to 30+ minutes depending on application warm-up requirements.

Active-Active

Both regions handle traffic simultaneously. Users are routed to the nearest region. Each region can handle the full request load in isolation.

Active-Active pattern with GeoDNS routing: users connect to their nearest region while data replicates bidirectionally.

Advantages: True geographic latency reduction. Survives regional failure without failover (traffic simply routes to the other region). Resources are utilized in both regions.

Disadvantages: Write consistency across regions is a hard problem. If a user in the EU and a user in the US both update the same record simultaneously, which write wins? How does each region know about the other's writes before serving reads?

Most applications avoid this by restricting writes to one region and handling reads from both. Or they carefully design which data is region-specific versus globally shared.

The Write Consistency Problem

This is where active-active architectures fail in practice. Cross-region database replication introduces latency. An EU database write may take 80-100ms to replicate to the US. During that window, a US read will see stale data.

The write consistency problem: a write in one region takes time to replicate, creating a window where other regions see stale data.

Acceptable in some cases: User preferences, profile information, content that changes rarely. Stale reads for a few hundred milliseconds are not harmful.

Not acceptable: Financial transactions, inventory levels, booking systems. Stale reads here cause double-spending, overselling, and double-booking.

For writes that require global consistency, there are a few approaches:

Route all writes to a single home region. Users are geographically routed for reads, but writes go to a designated write region. Adds round-trip latency for writes from distant regions.

Conflict-free Replicated Data Types (CRDTs). Data structures designed to merge concurrent writes without conflict. Counters that merge by addition, sets that merge by union. Applicable for specific use cases but not general-purpose databases.

Global databases. AWS Aurora Global Database provides sub-second replication across regions with a primary write region and read replicas globally. CockroachDB and Google Spanner are purpose-built globally distributed databases with strong consistency guarantees at significant cost.

Data Residency and Compliance

Data residency requirements add constraints on top of the architectural choices.

GDPR (EU)

The General Data Protection Regulation restricts transfers of EU personal data to countries without adequate data protection unless specific safeguards are in place. For most SaaS applications serving EU customers, this means EU personal data should be stored and processed in the EU or in countries with adequacy decisions.

Practical implementation:

Assign EU users to an EU region at signup
Store all data for EU users in EU infrastructure
Ensure analytics, logging, and third-party services do not transfer EU user data to non-EU regions
Document your data flows to demonstrate compliance

// Assign region based on user location at signup
function assignUserRegion(signupCountry) {
  const euCountries = new Set([
    'AT', 'BE', 'BG', 'HR', 'CY', 'CZ', 'DK', 'EE', 'FI', 'FR',
    'DE', 'GR', 'HU', 'IE', 'IT', 'LV', 'LT', 'LU', 'MT', 'NL',
    'PL', 'PT', 'RO', 'SK', 'SI', 'ES', 'SE'
  ]);

  if (euCountries.has(signupCountry)) {
    return 'eu-west-1';
  }
  return 'us-east-1';
}

Data Localization Laws

Some countries (Russia, China, India, Brazil) have data localization requirements mandating that certain categories of data (personal data, financial records, health records) be stored on servers physically located within the country. Serving these markets may require separate regional deployments.

Before building multi-region infrastructure for compliance, consult with legal counsel who is current on the specific requirements for your data categories and target markets. Regulations change and their interpretation varies.

Traffic Routing

Getting users to the right region requires DNS-level routing.

GeoDNS

DNS responds with different IP addresses based on the geographic location of the DNS resolver making the query. A resolver in Frankfurt gets the EU region IP. A resolver in Virginia gets the US region IP.

AWS Route 53 geolocation routing supports this natively. Cloudflare Load Balancing with proximity steering provides similar functionality with Cloudflare's network.

GeoDNS routing is approximate because DNS resolver location does not always match user location. A user in Germany might use Google's 8.8.8.8 resolver, which is located in the US. Route 53 geolocation routing uses resolver IP location, not the client IP. For more precise routing, use a CDN or anycast network that routes by client IP.

Latency-Based Routing

Route 53 latency-based routing measures actual network latency from AWS regions to DNS resolvers and routes to the region with the lowest measured latency, regardless of geographic proximity. This is often more accurate than pure geolocation.

Anycast

Large CDNs and traffic networks use anycast routing, where the same IP address is announced from multiple geographic locations. BGP routes each user's packets to the nearest network node that announces that IP. This provides genuinely client-IP-based routing without the approximations of DNS-based approaches.

Cloudflare, Fastly, and AWS CloudFront all use anycast for their networks. Placing your application behind one of these services provides geographic routing without managing your own multi-region infrastructure for the traffic layer.

Deploying to Multiple Regions

Infrastructure as code makes multi-region deployment manageable.

Terraform with Multiple Providers

# providers.tf
provider "aws" {
  alias = "us_east"
  region = "us-east-1"
}

provider "aws" {
  alias = "eu_west"
  region = "eu-west-1"
}

# modules/app_region/main.tf - reusable module for one region's stack
module "app_us_east" {
  source = "./modules/app_region"
  providers = {
    aws = aws.us_east
  }
  region_name = "us-east"
  db_snapshot_id = var.us_snapshot_id
}

module "app_eu_west" {
  source = "./modules/app_region"
  providers = {
    aws = aws.eu_west
  }
  region_name = "eu-west"
  db_snapshot_id = var.eu_snapshot_id
}

The reusable module creates identical infrastructure stacks in each region. Changes to the module deploy consistently to all regions.

Testing Regional Failover

Failover testing is as important as restore testing for backups. The steps that work in a planned drill work under pressure. The steps that have never been tested will fail at 3 AM.

Test steps for active-passive failover:

Verify the passive region is synchronized and healthy before the test.
Simulate region failure by blocking traffic to the active region at the load balancer level (do not actually take down the region).
Execute the documented failover procedure: update DNS records, confirm replication has caught up, verify the passive region can accept writes.
Measure the actual failover time.
Verify the application functions correctly in the now-active passive region.
Fail back to the original active region and measure that time too.
Document everything that did not go as expected.

Run this drill at least twice a year as part of your CI/CD pipeline testing strategy. The first time reveals problems in the runbook. Subsequent runs build team confidence and muscle memory.

Cost Considerations

Multi-region infrastructure roughly doubles your baseline infrastructure cost. Two database clusters. Two application server fleets. Cross-region data transfer charges (which are often underestimated).

AWS cross-region data transfer is currently priced at $0.02 per GB. An application transferring 10TB of data per month between regions pays $200 per month just for transfer. At 100TB, that is $2,000 per month. Understand your data transfer volume before committing to an architecture that depends on high-bandwidth cross-region replication.

For applications that do not yet need full multi-region infrastructure, caching strategies and API gateway patterns can provide significant latency improvements without the complexity of full multi-region deployment. Consider intermediate approaches: active-passive with a warm standby in a second region (lower cost than fully active), or relying on your cloud provider's managed services for geographic redundancy without managing a full second region yourself.

The architecture question is always: what level of availability and geographic performance does the business actually require, and what is the cost of achieving it? This ties back to broader infrastructure decisions like load balancing and disaster recovery planning.

The cost-complexity spectrum: start simple, add redundancy only when the data demands it.

Multi-region is often the right answer. It is rarely the starting point.

Conclusion

Multi-region architecture is a powerful tool when the business case is clear. It reduces latency for global users, provides resilience against regional failures, and enables compliance with data residency requirements. But it is not free. The complexity of cross-region consistency, the operational overhead of managing multiple deployments, and the cost of duplicated infrastructure all demand careful evaluation.

Start with a highly available single-region deployment. Add read replicas and CDN caching. Measure your actual latency and availability requirements. When the data shows that single-region is insufficient, then invest in multi-region with a clear understanding of the trade-offs. The best multi-region architecture is the one you understand well enough to operate at 3 AM when something breaks.

Data Consistency Models: From Strong Consistency to Eventual Consistency

Veduis — Sun, 14 Jun 2026 00:00:00 +0000

When you store data in a distributed system, you face a fundamental choice. Do you require that all copies of the data agree at all times? This ensures correctness but limits availability and performance. Or do you allow temporary divergence between copies, accepting that they will eventually agree? This improves availability and performance but complicates application logic.

This is the consistency trade-off. It is not a technical detail to be delegated to the database administrator. It is an architectural decision that affects your application's behavior, your user experience, and your business logic. A banking system that shows different balances depending on which ATM you use is unacceptable. A social media feed that shows slightly different posts to different users for a few seconds is tolerable.

I have built systems at both extremes and many points between. I have seen the confusion that eventual consistency causes in application code and the downtime that strict consistency requirements can create. This guide covers the patterns that work: understanding ACID and BASE, the CAP theorem's practical implications, read and write quorum patterns for tuning consistency, and matching consistency models to business requirements.

The Consistency Spectrum

Consistency is not binary. It exists on a spectrum from strong guarantees to weak guarantees, each with different trade-offs.

Strong Consistency : All reads see the most recent write. The system behaves as if there is only one copy of the data.

Eventual Consistency : Reads may see stale data. If no new writes occur, all copies eventually converge to the same value.

Between these extremes exist intermediate models: causal consistency, session consistency, monotonic reads, read-your-writes consistency.

ACID: Strong Consistency in Practice

ACID (Atomicity, Consistency, Isolation, Durability) describes the guarantees provided by traditional relational databases.

Atomicity

A transaction is all-or-nothing. Either all operations complete successfully, or none do. There is no partial completion.

Example : A funds transfer debits one account and credits another. With atomicity, both operations happen, or neither happens. You cannot have the debit without the credit or vice versa.

Consistency

Transactions bring the database from one valid state to another. All integrity constraints and business rules are maintained.

Note : This is different from the "consistency" in CAP theorem. ACID consistency refers to application-level invariants (foreign keys, check constraints). CAP consistency refers to all nodes agreeing on the same data.

Isolation

Concurrent transactions do not interfere with each other. Each transaction executes as if it were the only one running.

Isolation levels :

Read Uncommitted : Transactions see uncommitted changes from others. Fastest, least safe.
Read Committed : Transactions see only committed changes. Default in PostgreSQL.
Repeatable Read : A transaction sees the same data if it reads twice. Prevents non-repeatable reads.
Serializable : Transactions are completely isolated as if they ran sequentially. Highest safety, lowest concurrency.

Practical isolation :

-- Default (Read Committed)
BEGIN;
SELECT balance FROM accounts WHERE id = 1; -- Sees committed data
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
COMMIT;

-- Serializable (for critical operations)
BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT balance FROM accounts WHERE id = 1;
-- Other transactions cannot modify this row until we commit
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
COMMIT;

Durability

Once a transaction commits, it persists even if the system crashes immediately after.

Implementation : Write-ahead logging (WAL). Changes are written to a log before being applied to the database. After a crash, the log is replayed to recover committed transactions.

BASE: Accepting Weak Consistency

BASE (Basically Available, Soft state, Eventual consistency) describes the approach of distributed NoSQL systems.

Basically Available

The system remains operational for reads and writes, even during failures. This contrasts with ACID systems that may reject operations to maintain consistency.

Soft State

The state of the system can change without external input. This refers to background processes like replication and read repair that modify data without explicit user operations.

Eventual Consistency

If no new updates are made, eventually all replicas will converge to the same value. "Eventually" could be milliseconds or hours depending on the system.

Example : You update your profile picture. Your friend sees the old picture for a few seconds, then sees the new one. The system was inconsistent temporarily, then became consistent.

The CAP Theorem

CAP theorem states that in a distributed system with network partitions, you can have at most two of:

Consistency : All nodes see the same data at the same time
Availability : Every request receives a response (success or failure)
Partition Tolerance : The system continues to operate despite network partitions

The Practical Reality

Network partitions happen. They are a fact of distributed systems. Therefore, the real choice is between:

CP systems : Sacrifice availability for consistency during partitions. If nodes cannot communicate, some nodes refuse requests to prevent divergence.

AP systems : Sacrifice consistency for availability during partitions. All nodes continue accepting requests, accepting that they may diverge and need reconciliation later.

Examples

CP Systems :

Traditional relational databases with synchronous replication
etcd, ZooKeeper (consensus systems)
HBase with strong consistency settings

AP Systems :

Cassandra with eventual consistency
DynamoDB (default settings)
Riak
DNS (the canonical eventual consistency example)

PACELC: Extending CAP

PACELC adds nuance: if there is a Partition, choose between Availability and Consistency; Else, choose between Latency and Consistency.

Even without partitions, you face a latency-consistency trade-off. Synchronous replication to all nodes adds latency. Asynchronous replication is faster but provides weaker consistency.

Consistency Models in Detail

Linearizability (Strong Consistency)

Linearizability is the strongest consistency model. Operations appear to execute atomically at a single point in time between their invocation and response.

Guarantees :

Every read sees the most recent write
All operations appear to happen in a total order
If operation A completes before operation B begins, A appears before B

Cost : Requires coordination between nodes. Adds latency. Reduces availability during partitions.

Use cases : Financial ledgers, inventory management, anything where stale reads are unacceptable.

Sequential Consistency

Operations appear to execute in some sequential order that respects the order of operations on each individual node.

Weaker than linearizability but still strong. Allows reordering of operations from different nodes as long as each node's own operations stay in order.

Example : If Node A writes X then Y, and Node B reads Y then X, that is acceptable under sequential consistency (different order than real-time) but not under linearizability.

Causal Consistency

If operation A causally influences operation B (A happens before B, and B knows about A), then all nodes see A before B. Concurrent, unrelated operations may be seen in different orders.

Implementation : Vector clocks or version vectors track causality.

Use case : Collaborative editing (Google Docs). Your edits must appear in order to you, but concurrent edits by others may be ordered differently.

Session Consistency

Consistency guarantees apply within a session (a sequence of operations by one client). A client sees its own writes immediately, even if other clients see stale data.

Example : You post a comment. You see it immediately. Other users may see it after a delay.

Implementation : Route reads from the same replica that handled the write, or use sticky sessions.

Read-Your-Writes Consistency

A client that writes a value will always read that value or a later value, never an earlier one.

Subset of session consistency. Fundamental for user experience: users must see their own actions reflected.

Monotonic Reads

If a client reads a value, subsequent reads will not see earlier values. You cannot "go back in time."

Example : You see a post with 10 comments. You refresh and see 8 comments. This violates monotonic reads. You should see 10 or more, never fewer.

Eventual Consistency (Weakest)

With no new updates, all replicas eventually converge. No guarantees about what you see before convergence.

Acceptable when : Temporary inconsistency does not harm the business. Social media feeds, analytics dashboards, recommendation engines.

Quorum-Based Consistency

Distributed databases often use quorum mechanisms to balance consistency and availability.

Read and Write Quorums

In a system with N replicas:

Write quorum (W): Number of replicas that must acknowledge a write
Read quorum (R): Number of replicas that must respond to a read

Strong consistency : W + R > N

This ensures any read quorum and write quorum overlap. The read will include at least one replica that has the latest write.

Examples :

N=3, W=2, R=2: W+R=4 > 3. Strong consistency. Writes acknowledged by 2, reads from 2 (must overlap).
N=3, W=1, R=3: W+R=4 > 3. Strong consistency. Fast writes, slow reads.
N=3, W=1, R=1: W+R=2 < 3. Eventual consistency. Fastest, but may read stale data.

Dynamo-Style Quorum (Sloppy Quorum)

Cassandra and DynamoDB use a relaxed quorum:

Write sent to N nodes (where N is replication factor)
Wait for W acknowledgments
Read from R nodes, return most recent value

If nodes are unavailable, the system may write to alternative nodes and reconcile later (hinted handoff).

Tunable Consistency

Many systems allow per-operation consistency:

# Cassandra consistency levels
from cassandra import ConsistencyLevel

# Strong consistency for payment
session.execute(
    payment_query,
    consistency_level=ConsistencyLevel.QUORUM # W+R > N
)

# Eventual consistency for analytics
session.execute(
    analytics_query,
    consistency_level=ConsistencyLevel.ONE # Fast, may be stale
)

Consistency in Practice: Implementation Patterns

Optimistic Concurrency Control

Allow conflicts to occur, detect them, and resolve them.

Implementation : Version numbers or timestamps. Every update includes the version being modified. If the version has changed since read, the update fails.

-- Table with version column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT,
    version INT DEFAULT 1
);

-- Update with version check
UPDATE documents 
SET content = 'new content', version = version + 1
WHERE id = 1 AND version = 5; -- Expected version

-- Check if update succeeded
IF rows_affected = 0:
    raise ConflictError("Document modified by another process")

Use case : Collaborative editing, inventory management where conflicts are rare.

Pessimistic Concurrency Control

Prevent conflicts by locking data during access.

Implementation : Database locks, distributed locks (Redis, ZooKeeper).

# Distributed lock with Redis
import redis
import time

r = redis.Redis()
lock_key = "lock:document:123"
lock_timeout = 30 # seconds

# Acquire lock
if r.set(lock_key, "locked", nx=True, ex=lock_timeout):
    try:
        # Critical section: only one process can execute this
        document = read_document(123)
        document.content = update_content(document)
        write_document(document)
    finally:
        # Release lock
        r.delete(lock_key)
else:
    raise LockAcquisitionError("Document is being edited by another user")

Use case : Financial transactions, resource allocation where conflicts are common and expensive.

Conflict-Free Replicated Data Types (CRDTs)

Data structures designed such that concurrent modifications can be merged automatically without conflict resolution.

Types :

G-Counter : Increment-only counter. Merge takes maximum of replicas.
G-Set : Grow-only set. Merge is set union.
LWW-Element-Set : Last-write-wins element set. Timestamps determine winner.
OR-Set : Observed-remove set. Removes only affect elements the replica has seen.

Use case : Collaborative editing (where multiple users may edit simultaneously), distributed counters, shopping carts.

Vector Clocks

Track causality in distributed systems. Each node maintains a counter. When nodes communicate, they exchange and update their clocks.

Conflict detection : If vector clocks are incomparable (neither is greater than or equal to the other), operations are concurrent and conflict.

Use case : Version control (Git uses a form of this), document stores detecting concurrent modifications.

Read Repair and Anti-Entropy

Eventually consistent systems need mechanisms to detect and repair divergence.

Read Repair

When a read detects inconsistent replicas, repair the divergent replicas in the background.

Process :

Read from R replicas
Compare values
If values differ, identify the most recent
Write the correct value back to out-of-date replicas

Example : Cassandra reads from 3 replicas. Two return value A (timestamp t1), one returns value B (timestamp t0). The system returns A to the client and writes A to the replica with B.

Anti-Entropy Repair

Background process that compares and reconciles all data between replicas, not just data being read.

Implementation : Merkle trees (hash trees) allow efficient comparison of large datasets. If root hashes differ, compare children to find differing segments.

Example : Cassandra runs nodetool repair to synchronize replicas.

Choosing Consistency for Your Use Case

Match consistency requirements to business needs.

Financial Transactions

Requirement : Strong consistency. No stale reads. No lost updates.

Implementation : ACID transactions with serializable isolation. Synchronous replication. Consensus systems for distributed coordination.

Example : Bank transfer must debit one account and credit another atomically. All users must see the same balance.

E-Commerce Inventory

Requirement : Bounded staleness or strong consistency. Overselling is bad, but brief inconsistency may be acceptable.

Implementation : Optimistic locking with version numbers. Reserve inventory during checkout with short TTL.

Example : Two users attempt to buy the last item. First succeeds, second gets "out of stock" even if they saw "in stock" seconds ago.

User Profiles

Requirement : Session consistency minimum. User sees their own updates immediately.

Implementation : Sticky sessions or read-after-write consistency.

Example : User updates profile picture. They see the new picture immediately. Friends see the old picture for a few seconds.

Analytics Dashboard

Requirement : Eventual consistency acceptable. Staleness of minutes or hours is fine.

Implementation : Async aggregation pipelines, cached results, eventual consistency database.

Example : Dashboard shows sales data from 15 minutes ago. Real-time is not necessary.

Social Media Feed

Requirement : Eventual consistency. Different users seeing slightly different feeds is acceptable.

Implementation : Asynchronous fan-out. Cache heavily. Prioritize availability over consistency.

Example : Two users refresh at the same time. User A sees post X, User B does not yet. Both are acceptable experiences.

Implementation Examples

PostgreSQL Synchronous Replication

For strong consistency across PostgreSQL replicas:

-- Primary configuration
synchronous_commit = remote_apply # Wait for replica to apply
synchronous_standby_names = 'replica1, replica2'

With this configuration, commits wait until the specified replicas have applied the change. Higher latency, stronger consistency.

Cassandra Consistency Levels

from cassandra import ConsistencyLevel
from cassandra.cluster import Cluster

cluster = Cluster(['127.0.0.1'])
session = cluster.connect()

# Write with QUORUM consistency (strong)
session.execute(
    "INSERT INTO users (id, name) VALUES (?, ?)",
    (user_id, name),
    consistency_level=ConsistencyLevel.QUORUM
)

# Read with ONE consistency (fast, possibly stale)
rows = session.execute(
    "SELECT * FROM users WHERE id = ?",
    (user_id,),
    consistency_level=ConsistencyLevel.ONE
)

DynamoDB Consistency

import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('users')

# Strongly consistent read
table.get_item(
    Key={'user_id': '123'},
    ConsistentRead=True # Default is eventually consistent
)

# Eventually consistent read (faster, half the cost)
table.get_item(
    Key={'user_id': '123'}
    # ConsistentRead=False is default
)

Common Pitfalls

Pitfall 1: Assuming Strong Consistency

Using an eventually consistent database but writing application code that assumes immediate consistency. Leads to race conditions and stale data bugs.

Pitfall 2: Ignoring Conflict Resolution

In distributed systems, conflicts happen. Not planning how to resolve them leads to data loss or corruption.

Pitfall 3: One Consistency Level Everywhere

Using the same consistency for all operations. Analytics queries do not need strong consistency. Financial transactions do not tolerate eventual consistency.

Pitfall 4: Not Testing Failure Scenarios

Testing only the happy path. Network partitions and node failures expose consistency bugs that normal testing misses.

Pitfall 5: Over-Engineering for Strong Consistency

Building a complex consensus system when eventual consistency would suffice. Wastes development time and harms performance.

Pitfall 6: Ignoring Clock Synchronization

Using timestamps for conflict resolution without synchronized clocks. Clock skew causes incorrect conflict resolution.

Conclusion

Consistency is not a boolean choice. It is a spectrum of trade-offs between correctness, availability, and performance. Strong consistency (ACID) ensures correctness at the cost of availability during failures and higher latency. Eventual consistency (BASE) maximizes availability and performance at the cost of temporary divergence.

Match your consistency model to your business requirements. Financial systems need strong consistency. Social feeds tolerate eventual consistency. Most systems have components at different points on the spectrum.

Understand the CAP theorem's implications for your architecture. During partitions, choose whether to be consistent or available. Design for the failures you can tolerate.

Implement appropriate concurrency control: optimistic for low-contention scenarios, pessimistic for high-contention. Plan for conflict detection and resolution in distributed systems.

Test your consistency guarantees under failure conditions. Network partitions reveal whether your system actually provides the consistency you designed for.

Consistency choices are architectural decisions with long-term consequences. Make them deliberately, with clear understanding of trade-offs, not by default or accident.

Further Reading

"Designing Data-Intensive Applications" by Martin Kleppmann: Chapters on consistency and replication
"CAP Twelve Years Later" by Eric Brewer: Updated perspective on the CAP theorem
Dynamo paper (Amazon): Eventual consistency in practice
Spanner paper (Google): Strong consistency at global scale
CRDT research: Conflict-free replicated data types

Building a RAG System for Your Business: Custom AI Knowledge Bases Without Fine-Tuning

Veduis — Sat, 13 Jun 2026 00:00:00 +0000

Every business I've worked with hits the same wall. They get ChatGPT or Claude working on general tasks, then someone asks it about the company's refund policy, their specific product specs, or a contract clause from six months ago, and the model either makes something up or admits it doesn't know. The model has never seen your data. That's not a bug; that's how language models work.

Fine-tuning sounds like the answer. Train the model on your internal documents and it will know your stuff. In practice, fine-tuning is expensive, slow to update, and doesn't actually give the model reliable recall of specific facts. It shifts the model's general behavior. It doesn't give it a searchable memory.

If you're weighing fine-tuning against other approaches, our guide on fine-tuning open-source LLMs for business breaks down when training your own model makes sense and when it does not.

Retrieval-Augmented Generation (RAG) is the architecture that actually solves this. Instead of baking knowledge into model weights, you build a searchable index of your documents and retrieve the relevant ones at query time, then pass them to the model as context. The model reasons over real source material rather than guessing from memory. Updates to your knowledge base take effect immediately, no retraining required.

I've built RAG systems for internal documentation search, customer support knowledge bases, contract analysis tools, and compliance Q&A systems. The core architecture is the same every time. This guide covers it completely.

How RAG Works

The RAG pipeline has two phases: ingestion (indexing your documents) and retrieval (finding relevant content at query time).

Ingestion phase:

Load documents from your source (PDFs, databases, wikis, URLs)
Split documents into chunks of manageable size
Convert each chunk to a vector embedding (a numerical representation of meaning)
Store embeddings in a vector database alongside the original text

Retrieval phase:

User submits a question
Convert the question to an embedding using the same model
Search the vector database for chunks with the most similar embeddings
Pass the retrieved chunks as context to the LLM
LLM generates an answer grounded in the retrieved material

The magic is in step 3: semantic similarity search. A vector database doesn't match keywords; it finds chunks that are conceptually similar to the question, even when the wording differs completely. If you're new to vector search, our vector search fundamentals guide explains how embeddings and similarity scoring work under the hood.

Choosing a Vector Database

The vector database is the foundation of the system. Your choice affects query latency, operational complexity, and cost.

Pinecone

Pinecone is a fully managed vector database built for production scale. There's no infrastructure to manage; you create an index, insert vectors, and query it via API.

Where Pinecone excels:

Production deployments with variable traffic
Teams who want zero infrastructure management
Applications needing 99.99% availability SLAs

Where it falls short:

Paid service (free tier is limited to one index with 100K vectors)
Data leaves your infrastructure (relevant for sensitive industries)
Less control over underlying storage and indexing behavior

For most growing businesses, Pinecone's managed nature is worth the cost. The free tier is enough to prototype and demo, and the starter plan covers most small-to-medium knowledge bases. If you are already running PostgreSQL, pgvector (covered below) lets you add vectors without adding another vendor to your stack.

Chroma

Chroma is an open-source vector database that runs embedded in your application process or as a separate server. It requires zero external services for local development and small deployments.

Where Chroma excels:

Local development and prototyping
Deployments where data must stay on-premises
Smaller knowledge bases (under a few million vectors)
Python-native workflows

Where it falls short:

Horizontal scaling requires custom work
Production deployments over significant scale need attention to resource sizing

Chroma is my default for new projects during development. I switch to Pinecone or pgvector when I have clear production requirements.

pgvector

pgvector is a PostgreSQL extension that adds vector storage and similarity search to a database you likely already operate. If your application data lives in Postgres, storing embeddings there too eliminates a separate infrastructure component entirely.

Where pgvector excels:

Teams already running PostgreSQL in production
Applications needing transactional consistency between application data and embeddings
Deployments where adding another managed service isn't feasible
Cost-conscious setups where you're already paying for a Postgres instance

Where it falls short:

Approximate nearest-neighbor (ANN) search at very large scale is slower than purpose-built vector databases
Requires PostgreSQL operational knowledge

The performance limitation matters at scale. For knowledge bases under a few million vectors, which covers the vast majority of business applications, pgvector performs adequately. I run it in production for several clients who are already on managed PostgreSQL services. If your application is hitting Postgres limits, our guide to database sharding for SaaS applications covers horizontal scaling strategies that apply to pgvector workloads as well.

Quick decision framework

Prototype or small-scale production with Postgres already in use → pgvector
On-premises requirement or open-source preference → Chroma (self-hosted)
Production scale with managed infrastructure preference → Pinecone
Very large scale with complex filtering requirements → Weaviate or Qdrant

Embedding Models

Embeddings convert text to vectors. The model you choose determines how well semantic similarity search performs, and how much each query costs.

OpenAI text-embedding-3-small

OpenAI's embedding models are the most widely used in production RAG systems. text-embedding-3-small produces 1536-dimensional vectors and costs $0.02 per million tokens. For most knowledge bases, this is a rounding error.

text-embedding-3-large produces higher-quality embeddings for complex semantic matching but costs 13x more. I use text-embedding-3-small unless I'm working with dense technical or legal content where nuanced similarity matters.

Open-source alternatives

If data privacy requires keeping text on-premises, several open-source models perform comparably:

BAAI/bge-large-en-v1.5: Consistently top-ranked on the MTEB leaderboard, free to run locally
nomic-ai/nomic-embed-text-v1.5: Strong performance with support for variable context lengths
sentence-transformers/all-MiniLM-L6-v2: Small, fast, good for prototyping

For on-premises deployments, I run these via Ollama or sentence-transformers and see latencies under 50ms for single-document embedding on CPU. If your business is deciding between cloud and local AI infrastructure, our comparison of local LLMs versus cloud AI for small business walks through the cost, privacy, and performance trade-offs.

Critical rule : Use the same embedding model for ingestion and retrieval. Vectors from different models are not comparable; mixing them produces nonsense search results.

Chunking Strategies

How you split documents into chunks has a bigger impact on retrieval quality than almost any other architectural decision. Chunks that are too large overwhelm the LLM's context window and dilute relevance. Chunks that are too small lose the surrounding context the model needs to answer correctly.

Fixed-size chunking

Split documents every N characters or tokens, with overlap between adjacent chunks to preserve context across boundaries.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, # characters per chunk
    chunk_overlap=200, # overlap between consecutive chunks
    separators=["\n\n", "\n", ". ", " ", ""]
)

chunks = splitter.split_text(document_text)

RecursiveCharacterTextSplitter from LangChain tries to split on paragraph breaks first, then line breaks, then sentence boundaries, preserving natural language units wherever possible. I use 1000 characters with 200 overlap as my starting point and adjust based on the source material.

Semantic chunking

Semantic chunking groups sentences that discuss the same topic, regardless of where paragraph breaks fall. It's more expensive to compute but produces more coherent chunks for documents with irregular structure.

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

semantic_splitter = SemanticChunker(
    OpenAIEmbeddings(),
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=95
)

chunks = semantic_splitter.split_text(document_text)

I use semantic chunking for user manuals, legal documents, and support articles where section boundaries matter significantly for answer quality.

Document-specific strategies

Different document types benefit from different approaches:

Markdown and HTML: Split on heading levels (H1, H2, H3) first, then by size
PDFs with tables: Extract tables separately from prose; tables often answer precise factual queries
Code files: Split on function or class boundaries, never mid-function
Q&A content: Keep question-answer pairs together as single chunks

The metadata you store alongside each chunk matters as much as the text itself. Always include the source document name, page number if applicable, section heading, and creation date. This lets you filter results by recency or document type, and lets the model cite sources accurately.

End-to-End Implementation

Here's a complete RAG pipeline using Chroma for local development, with easy migration to Pinecone for production.

Setup and dependencies

pip install langchain langchain-openai langchain-chroma chromadb pypdf python-dotenv

Document ingestion pipeline

import os
from pathlib import Path
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader, TextLoader, DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

load_dotenv()

CHROMA_PATH = "./chroma_db"
DOCS_PATH = "./documents"

def load_documents(directory: str) -> list:
    """Load all PDFs and text files from a directory."""
    loaders = [
        DirectoryLoader(directory, glob="**/*.pdf", loader_cls=PyPDFLoader),
        DirectoryLoader(directory, glob="**/*.txt", loader_cls=TextLoader),
        DirectoryLoader(directory, glob="**/*.md", loader_cls=TextLoader),
    ]
    documents = []
    for loader in loaders:
        try:
            documents.extend(loader.load())
        except Exception as e:
            print(f"Warning loading documents: {e}")
    return documents

def chunk_documents(documents: list) -> list:
    """Split documents into chunks with overlap."""
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
        add_start_index=True, # records position in source doc
    )
    chunks = splitter.split_documents(documents)
    print(f"Split {len(documents)} documents into {len(chunks)} chunks")
    return chunks

def build_vector_store(chunks: list) -> Chroma:
    """Embed chunks and store in Chroma."""
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

    # Remove existing database if rebuilding
    if Path(CHROMA_PATH).exists():
        import shutil
        shutil.rmtree(CHROMA_PATH)

    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory=CHROMA_PATH
    )
    print(f"Stored {len(chunks)} chunks in vector database")
    return vector_store

if __name__ == " __main__":
    docs = load_documents(DOCS_PATH)
    chunks = chunk_documents(docs)
    build_vector_store(chunks)
    print("Ingestion complete.")

Query and answer pipeline

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain.prompts import ChatPromptTemplate
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.output_parser import StrOutputParser

CHROMA_PATH = "./chroma_db"

PROMPT_TEMPLATE = """
You are a helpful assistant. Answer the question based only on the following context.
If the context does not contain enough information to answer confidently, say so rather than guessing.

Context:
{context}

Question: {question}

Answer:
"""

def load_retriever(k: int = 5):
    """Load vector store and return retriever."""
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    vector_store = Chroma(
        persist_directory=CHROMA_PATH,
        embedding_function=embeddings
    )
    return vector_store.as_retriever(
        search_type="mmr", # Maximum Marginal Relevance for diversity
        search_kwargs={"k": k, "fetch_k": 20}
    )

def format_docs(docs) -> str:
    """Format retrieved documents into context string."""
    formatted = []
    for i, doc in enumerate(docs):
        source = doc.metadata.get("source", "Unknown")
        page = doc.metadata.get("page", "")
        citation = f"[Source: {Path(source).name}" + (f", page {page+1}" if page != "" else "") + "]"
        formatted.append(f"{citation}\n{doc.page_content}")
    return "\n\n---\n\n".join(formatted)

def build_rag_chain():
    """Build the complete RAG chain."""
    retriever = load_retriever(k=5)
    prompt = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    chain = (
        {"context": retriever | format_docs, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )
    return chain

def answer_question(question: str) -> str:
    chain = build_rag_chain()
    return chain.invoke(question)

if __name__ == " __main__":
    question = input("Ask a question: ")
    answer = answer_question(question)
    print(f"\nAnswer: {answer}")

Migrating to Pinecone for production

Switching from Chroma to Pinecone requires minimal code changes:

from pinecone import Pinecone
from langchain_pinecone import PineconeVectorStore

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

# Create index (run once)
pc.create_index(
    name="company-knowledge",
    dimension=1536, # matches text-embedding-3-small
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

# Replace Chroma with Pinecone
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = PineconeVectorStore.from_documents(
    documents=chunks,
    embedding=embeddings,
    index_name="company-knowledge"
)

The rest of the pipeline (retriever, prompt, LLM chain) stays identical. For teams running RAG in production, monitoring and evaluation are non-negotiable. Our guide on evaluating LLM outputs with automated quality checks covers the metrics and guardrails that keep generated answers accurate and on-brand.

Improving Retrieval Quality

A naive RAG system retrieves the top-k chunks by vector similarity and feeds them to the model. This works adequately for simple queries but breaks down when:

The question is ambiguous or uses different vocabulary than the source documents
The answer spans multiple sections that don't appear near each other in vector space
The relevant chunk is buried in a document that contains many tangentially related topics

These techniques significantly improve results in practice.

Hybrid search

Combine vector similarity with traditional keyword (BM25) search. Keyword search catches exact term matches that semantic search sometimes misses; semantic search catches meaning-based matches that keyword search misses. Merging both result sets with reciprocal rank fusion produces better top-k results than either alone.

Weaviate and Elasticsearch have built-in hybrid search. For pgvector, you can implement BM25 alongside vector queries using PostgreSQL full-text search and combine the ranked results in application code.

Query rewriting

User questions are often poorly worded for retrieval. A question like "what's the deal with our enterprise plan?" won't match chunks describing "Enterprise tier pricing and feature inclusions."

Before running the similarity search, use an LLM to rewrite the query into search-optimized phrasing:

from langchain_openai import ChatOpenAI

query_rewriter_prompt = """
Rewrite the following user question as an optimized search query for a document retrieval system.
Return only the rewritten query, nothing else.

User question: {question}
"""

def rewrite_query(question: str) -> str:
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    prompt = ChatPromptTemplate.from_template(query_rewriter_prompt)
    chain = prompt | llm | StrOutputParser()
    return chain.invoke({"question": question})

In my testing, query rewriting improves retrieval precision by 15-30% on real user questions, particularly for short or colloquial queries.

Re-ranking retrieved chunks

After initial retrieval, use a cross-encoder model to re-rank chunks by true relevance to the question. Cross-encoders are slower than embedding similarity (they process the query and each chunk together rather than independently) but much more accurate at distinguishing genuinely relevant chunks from topically-adjacent ones.

Cohere's Rerank API is the easiest managed option. For open-source, cross-encoder/ms-marco-MiniLM-L-6-v2 runs efficiently on CPU.

import cohere

co = cohere.Client(os.environ["COHERE_API_KEY"])

def rerank_chunks(query: str, chunks: list, top_n: int = 3) -> list:
    """Re-rank retrieved chunks using Cohere's cross-encoder."""
    results = co.rerank(
        model="rerank-english-v3.0",
        query=query,
        documents=[doc.page_content for doc in chunks],
        top_n=top_n
    )
    return [chunks[r.index] for r in results.results]

Retrieve 10-15 candidates from the vector database, re-rank them, and pass only the top 3-5 to the LLM. This keeps context window usage low while maximizing relevance.

Metadata filtering

When users ask questions scoped to specific document types, dates, or categories, filter the vector search before running similarity comparison:

# Search only documents from the last 90 days
from datetime import datetime, timedelta

cutoff = (datetime.now() - timedelta(days=90)).timestamp()

results = vector_store.similarity_search(
    query=question,
    k=10,
    filter={"created_at": {"$gte": cutoff}}
)

Pinecone, Chroma, and Weaviate all support metadata filters. The filter syntax varies by database, but the concept is identical.

Reducing Hallucinations

The system prompt is your most important hallucination control mechanism. I use this structure consistently:

You are a helpful assistant with access to [company name]'s documentation.

Rules you must follow:
1. Answer only from the provided context. Do not use knowledge from outside the context.
2. If the context doesn't contain enough information to answer fully, say so explicitly.
3. When answering, cite which document and section your answer comes from.
4. Never make up product names, prices, policies, or dates.
5. If you're uncertain, say you're uncertain rather than asserting confidently.

Three additional practices reduce hallucinations meaningfully:

Temperature of 0: Set temperature=0 on the LLM for factual Q&A. Higher temperatures increase creativity but also fabrication.

Self-consistency check: For high-stakes answers, generate 3 answers independently and compare them. Answers that consistently appear across all three runs are more likely grounded in context than answers that appear only once.

Confidence scoring: Ask the model to rate its confidence and explain its reasoning alongside the answer. When the model can't explain which context passage supports its answer, that's a signal the answer may be fabricated.

Production Considerations

Incremental re-indexing

Your knowledge base changes. New documents get added, old ones become outdated, corrections get made. A re-indexing strategy needs to handle this without full rebuilds.

I track document hashes to detect changes:

import hashlib

def get_document_hash(filepath: str) -> str:
    with open(filepath, "rb") as f:
        return hashlib.sha256(f.read()).hexdigest()

def needs_reindex(filepath: str, stored_hashes: dict) -> bool:
    current_hash = get_document_hash(filepath)
    return stored_hashes.get(filepath) != current_hash

On each scheduled re-index run (I typically schedule nightly):

Compute hash for each source document
Skip documents whose hash matches the stored value
Delete all chunks for changed documents from the vector database
Re-ingest and re-embed the changed documents
Update stored hashes

This keeps incremental runs fast even for large knowledge bases.

Cost management

Embedding costs are usually small. With text-embedding-3-small at $0.02/million tokens, a 10,000-document knowledge base averaging 500 tokens per document costs $0.10 to fully re-embed. The recurring cost comes from query-time embeddings: one embedding call per user question.

LLM costs are the main variable. Monitor token usage per query and set alerts if average context length grows unexpectedly (this usually indicates retrieval is pulling too many chunks). For gpt-4o-mini, keeping context under 4000 tokens per query typically results in costs under $0.002 per answer.

Set hard limits with LangSmith or your LLM provider's usage controls to prevent runaway costs from prompt injection attacks or misbehaving clients.

Evaluating system quality

Measure RAG quality with these metrics:

Context precision: What fraction of retrieved chunks are actually relevant to the question?
Context recall: Does the retrieved set contain all the information needed to answer correctly?
Answer faithfulness: Does the generated answer accurately reflect the retrieved context?
Answer relevance: Does the answer address what was actually asked?

RAGAS is the standard evaluation framework for these metrics. Build an evaluation dataset of 50-100 representative questions with known correct answers and run RAGAS against your system before every significant change to the pipeline.

Common Mistakes

Ignoring document quality: Garbage in, garbage out. PDF extraction often produces garbled text from scanned documents. Spend time on document preprocessing: clean OCR output, normalize whitespace, strip boilerplate headers and footers before chunking.

Chunk size too large: Chunks over 1500 characters often contain multiple distinct topics. The embedding vector represents a blended average of all those topics, diluting similarity scores for any one of them. When retrieval feels imprecise, smaller chunks usually help.

No overlap between chunks: Without overlap, a sentence straddling a chunk boundary might be split, making both resulting chunks harder to understand. Keep overlap at 15-20% of chunk size.

Same model for everything: A model well-suited for embeddings isn't necessarily the best for generation, and vice versa. Pick each component based on its role.

Not filtering by recency: For knowledge bases with time-sensitive content (policies, pricing, announcements), retrieved context from outdated documents produces outdated answers. Always store and filter by document date.

RAG is the practical architecture for businesses that need AI to reason over their own information. Fine-tuning teaches a model new behaviors; RAG gives it access to current, specific facts. For most business applications, RAG delivers better results faster, at a fraction of the cost, with knowledge that stays current as your documents change.

+++

LLM Token Cost Optimization: Cutting Your API Bills Without Cutting Quality

Veduis — Fri, 12 Jun 2026 20:51:04 +0000

Traditional search matches keywords. Users must know the exact words in the documents they seek. Vector search matches meaning. Users describe what they are looking for in natural language, and the system finds semantically similar content even when keywords differ. "Car trouble" finds documents about "automotive repair" and "engine problems."

Vector search powers modern semantic search, recommendation systems, and retrieval-augmented generation (RAG) for LLMs. It converts text, images, or other content into high-dimensional vectors (embeddings) that capture semantic meaning. It then searches for vectors most similar to a query vector using specialized algorithms that find approximate nearest neighbors efficiently at scale.

I have built vector search systems for knowledge bases, product catalogs, and content recommendation. I have learned that embedding model selection dramatically impacts quality, that similarity metric choice affects results, and that approximate nearest neighbor (ANN) algorithms enable search at million-document scale. This guide covers the patterns that work: understanding embeddings and their properties, similarity metrics and when to use each, ANN algorithms that make vector search practical, vector database selection, and building complete semantic search pipelines.

Understanding Embeddings

What Are Embeddings

Embeddings are dense numerical vectors that represent semantic meaning. Similar items have similar vectors.

Text: "The quick brown fox"
Embedding: [0.12, -0.45, 0.89, ..., 0.34]  # 384 to 1536 dimensions

Text: "A fast brown animal"
Embedding: [0.14, -0.42, 0.87, ..., 0.31]  # Similar vector

Text: "Quantum computing"
Embedding: [-0.78, 0.23, -0.12, ..., 0.91]  # Very different vector

Properties of Good Embeddings

Semantic similarity: Similar meaning = close in vector space

Linear relationships: Analogies work as vector arithmetic

king - man + woman ≈ queen
Paris - France + Italy ≈ Rome

Dense representation: All dimensions have values (vs sparse one-hot encoding)

Similar meanings produce vectors that cluster together in high-dimensional space.

Embedding Models

Model	Dimensions	Best For	Provider
text-embedding-3-small	1536	General purpose	OpenAI
text-embedding-3-large	3072	High accuracy	OpenAI
text-embedding-ada-002	1536	Legacy	OpenAI
sentence-transformers/all-MiniLM	384	Cost-effective	Open source
sentence-transformers/all-mpnet-base	768	Balanced	Open source
voyage-2	1024	High quality	Voyage AI
Cohere embed	1024	Multilingual	Cohere

Selection criteria:

Quality: Benchmark on your specific data
Cost: API costs or compute for self-hosted
Dimensions: Higher dimensions = more accurate but more storage
Latency: Model inference time

Generating Embeddings

# OpenAI
from openai import OpenAI

client = OpenAI()

def get_embedding(text: str, model: str = "text-embedding-3-small") -> list[float]:
    # Replace newlines, which can affect results
    text = text.replace("\n", " ")

    response = client.embeddings.create(
        input=text,
        model=model
    )

    return response.data[0].embedding

# Batch processing for efficiency
async def get_embeddings_batch(texts: list[str], batch_size: int = 100) -> list[list[float]]:
    embeddings = []

    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        response = await client.embeddings.create(input=batch, model="text-embedding-3-small")
        embeddings.extend([item.embedding for item in response.data])

    return embeddings

# Open source (sentence-transformers)
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')

def get_embedding(text: str) -> list[float]:
    embedding = model.encode(text)
    return embedding.tolist()

# Batch processing
embeddings = model.encode(texts, batch_size=32, show_progress_bar=True)

For teams evaluating whether to run embeddings locally or via API, cost and latency tradeoffs matter. Our guide to locally run AI breaks down when self-hosting saves money versus using managed embedding APIs.

Similarity Metrics

Cosine Similarity

Measures angle between vectors, ignoring magnitude.

import numpy as np

def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Ranges from -1 (opposite) to 1 (identical)
# Most common for text embeddings

When to use:

Text embeddings (most models normalized)
When direction matters more than magnitude
Most common default choice

Euclidean Distance

Straight-line distance between vectors.

def euclidean_distance(a: np.ndarray, b: np.ndarray) -> float:
    return np.linalg.norm(a - b)

# Lower = more similar

When to use:

When magnitude matters
Computer vision embeddings
When vectors are not normalized

Dot Product

Simple sum of element-wise products.

def dot_product(a: np.ndarray, b: np.ndarray) -> float:
    return np.dot(a, b)

When to use:

When vectors are normalized (equivalent to cosine)
Fast computation
Some vector databases optimize for this

Metric Selection Guide

Metric	Use When	Range
Cosine	Text search, normalized vectors	[-1, 1]
Euclidean	Magnitude matters, vision	[0, ∞)
Dot Product	Normalized vectors, speed	(-∞, ∞)

Approximate Nearest Neighbor (ANN) Algorithms

Why ANN

Exact nearest neighbor search is O(n) with high dimensional data. At million-document scale, it is too slow.

ANN algorithms trade small accuracy loss for massive speed gains (1000x+).

ANN algorithms build hierarchical graphs that navigate directly to the most similar vectors without scanning every entry.

HNSW (Hierarchical Navigable Small World)

Graph-based algorithm. Most popular for production.

Building:
1. Insert points into layered graph structure
2. Each layer is a proximity graph
3. Top layer is sparse, lower layers denser

Searching:
1. Start at random point in top layer
2. Greedy walk to closest point
3. Drop to lower layer when local minimum reached
4. Repeat until bottom layer

Characteristics:

High recall (typically >95%)
Fast queries (milliseconds)
Memory intensive (stores graph)
Good for million-scale datasets

IVF (Inverted File Index)

Clustering-based approach.

Building:
1. Cluster vectors into N groups (voronoi cells)
2. Store cluster centroids

Searching:
1. Find nearest cluster centroids to query
2. Search only vectors in those clusters
3. Refine with exact search on candidates

Characteristics:

Tunable speed/accuracy tradeoff
Memory efficient
Good for billion-scale datasets

LSH (Locality Sensitive Hashing)

Hash-based approach.

Building:
1. Create multiple hash functions
2. Similar vectors hash to same buckets

Searching:
1. Hash query vector
2. Check all vectors in matching buckets
3. Refine with exact similarity

Characteristics:

Very fast
Lower recall than HNSW
Good for very large datasets where recall can be lower

PQ (Product Quantization)

Compression technique often combined with other indexes.

Compress vectors by:
1. Split vector into sub-vectors
2. Quantize each sub-vector to a codebook
3. Store codes instead of full vectors

Enables:
- 10-20x memory reduction
- Faster distance computation
- Slight accuracy loss

Vector Databases

Pinecone

Pinecone is a managed vector database.

from pinecone import Pinecone, ServerlessSpec

pc = PineCone(api_key="your-api-key")

# Create index
pc.create_index(
    name="my-index",
    dimension=1536,  # Matches embedding model
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

index = pc.Index("my-index")

# Upsert vectors
index.upsert(
    vectors=[
        {
            "id": "doc1",
            "values": embedding,
            "metadata": {"source": "article1", "category": "tech"}
        }
    ]
)

# Query
results = index.query(
    vector=query_embedding,
    top_k=10,
    filter={"category": {"$eq": "tech"}},
    include_metadata=True
)

Weaviate

Weaviate is an open-source vector database with a managed option.

import weaviate

client = weaviate.Client("http://localhost:8080")

# Create schema
class_obj = {
    "class": "Article",
    "vectorizer": "text2vec-openai",
    "moduleConfig": {
        "text2vec-openai": {
            "vectorizeClassName": False
        }
    },
    "properties": [
        {"name": "title", "dataType": ["text"]},
        {"name": "content", "dataType": ["text"]},
        {"name": "category", "dataType": ["text"]}
    ]
}

client.schema.create_class(class_obj)

# Insert (automatically vectorized)
client.data_object.create(
    data_object={
        "title": "Vector Search Guide",
        "content": "Content here...",
        "category": "tech"
    },
    class_name="Article"
)

# Query
result = (
    client.query
    .get("Article", ["title", "content"])
    .with_near_text({"concepts": ["semantic search"]})  # Auto-vectorized
    .with_limit(10)
    .do()
)

pgvector

pgvector is a PostgreSQL extension that adds vector similarity search to existing Postgres databases.

-- Enable extension
CREATE EXTENSION vector;

-- Create table
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT,
    embedding VECTOR(1536)
);

-- Create index
CREATE INDEX ON documents 
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

-- Insert
INSERT INTO documents (content, embedding)
VALUES ('text here', '[0.1, 0.2, ...]');

-- Query
SELECT content, 1 - (embedding <=> query_embedding) AS cosine_similarity
FROM documents
ORDER BY embedding <=> query_embedding
LIMIT 10;

# Using with SQLAlchemy
from sqlalchemy import create_engine, Column, Integer, String
from pgvector.sqlalchemy import Vector

class Document(Base):
    __tablename__ = 'documents'

    id = Column(Integer, primary_key=True)
    content = Column(String)
    embedding = Column(Vector(1536))

# Query
docs = session.query(Document).order_by(
    Document.embedding.cosine_distance(query_embedding)
).limit(10).all()

Selection Guide

Database	Best For	Deployment
Pinecone	Managed, easy start	SaaS
Weaviate	Flexibility, features	Self-hosted or SaaS
pgvector	Existing Postgres	Self-hosted
Milvus	High scale, hybrid search	Self-hosted or SaaS
Chroma	Local development, simplicity	Embedded
Qdrant	Rust-based, fast	Self-hosted or SaaS

Each vector database optimizes for different deployment models and scale requirements.

Building a Semantic Search Pipeline

Architecture

Documents
    |
    v
Chunking → Embedding → Vector DB
                              |
Query → Embedding → Similarity Search → Reranking → Results

Chunking Strategy

def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    """Chunk text with overlap for context preservation"""
    chunks = []
    start = 0

    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]

        # Try to end at sentence boundary
        if end < len(text):
            last_period = chunk.rfind('.')
            if last_period > chunk_size * 0.7:  # If found in last 30%
                chunk = chunk[:last_period + 1]
                end = start + len(chunk)

        chunks.append(chunk.strip())
        start = end - overlap  # Overlap for context

    return chunks

# Semantic chunking with embeddings
def semantic_chunk(text: str, similarity_threshold: float = 0.8) -> list[str]:
    """Chunk based on semantic similarity"""
    sentences = text.split('. ')
    chunks = []
    current_chunk = [sentences[0]]

    for i in range(1, len(sentences)):
        prev_embedding = get_embedding(sentences[i-1])
        curr_embedding = get_embedding(sentences[i])

        similarity = cosine_similarity(prev_embedding, curr_embedding)

        if similarity > similarity_threshold:
            current_chunk.append(sentences[i])
        else:
            chunks.append('. '.join(current_chunk))
            current_chunk = [sentences[i]]

    if current_chunk:
        chunks.append('. '.join(current_chunk))

    return chunks

Hybrid Search

Combine vector similarity with keyword matching.

def hybrid_search(query: str, vector_weight: float = 0.7) -> list[dict]:
    """Combine BM25 and vector search"""

    # Vector search
    query_embedding = get_embedding(query)
    vector_results = vector_db.search(query_embedding, k=100)

    # Keyword search
    keyword_results = keyword_index.search(query, k=100)

    # Reciprocal Rank Fusion
    scores = {}

    for rank, result in enumerate(vector_results):
        doc_id = result['id']
        scores[doc_id] = scores.get(doc_id, 0) + vector_weight / (rank + 60)

    for rank, result in enumerate(keyword_results):
        doc_id = result['id']
        scores[doc_id] = scores.get(doc_id, 0) + (1 - vector_weight) / (rank + 60)

    # Sort by fused score
    ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)

    return [get_document(doc_id) for doc_id, _ in ranked[:10]]

Reranking

Initial retrieval (fast, approximate) → Reranking (slower, accurate).

def search_with_reranking(query: str) -> list[dict]:
    # Initial retrieval (ANN)
    query_embedding = get_embedding(query)
    candidates = vector_db.query(query_embedding, top_k=100)

    # Rerank with cross-encoder (more accurate)
    from sentence_transformers import CrossEncoder

    reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

    pairs = [[query, candidate['text']] for candidate in candidates]
    scores = reranker.predict(pairs)

    # Sort by reranker score
    for candidate, score in zip(candidates, scores):
        candidate['rerank_score'] = score

    reranked = sorted(candidates, key=lambda x: x['rerank_score'], reverse=True)

    return reranked[:10]

Filtering and Metadata

# Pinecone with metadata filter
results = index.query(
    vector=query_embedding,
    top_k=10,
    filter={
        "category": {"$eq": "documentation"},
        "created_at": {"$gte": "2024-01-01"},
        "$or": [
            {"author": {"$eq": "team-a"}},
            {"author": {"$eq": "team-b"}}
        ]
    }
)

Performance Optimization

Index Tuning

# HNSW parameters
index_params = {
    "M": 16,        # Connections per layer (higher = more accurate, more memory)
    "efConstruction": 200,  # Size of dynamic candidate list during construction
    "ef": 100       # Size of dynamic candidate list during search
}

# Tradeoffs:
# M: 8-64 (default 16). Higher = better recall, more memory
# efConstruction: 64-512. Higher = better index quality, slower build
# ef: 16-512. Higher = better recall, slower queries

Batch Operations

# Batch embedding (much faster)
texts = [doc['content'] for doc in documents]
embeddings = model.encode(texts, batch_size=64)

# Batch upsert
vectors_to_upsert = [
    {
        "id": doc['id'],
        "values": embedding.tolist(),
        "metadata": {"source": doc['source']}
    }
    for doc, embedding in zip(documents, embeddings)
]

# Upsert in batches
for i in range(0, len(vectors_to_upsert), 100):
    batch = vectors_to_upsert[i:i+100]
    index.upsert(vectors=batch)

Caching

from functools import lru_cache

@lru_cache(maxsize=10000)
def get_cached_embedding(text: str) -> tuple[list[float], str]:
    """Cache embeddings by text hash"""
    embedding = get_embedding(text)
    return tuple(embedding)  # Must be hashable for cache

Evaluation

Metrics

def evaluate_search(queries: list[dict]) -> dict:
    """
    queries: [{"query": str, "relevant_ids": [str]}]
    """
    results = {
        'recall@10': [],
        'precision@10': [],
        'mrr': [],
        'ndcg': []
    }

    for q in queries:
        search_results = search(q['query'], k=10)
        retrieved_ids = [r['id'] for r in search_results]
        relevant_ids = set(q['relevant_ids'])

        # Recall@10
        recall = len(relevant_ids & set(retrieved_ids)) / len(relevant_ids)
        results['recall@10'].append(recall)

        # Precision@10
        precision = len(relevant_ids & set(retrieved_ids)) / len(retrieved_ids)
        results['precision@10'].append(precision)

        # MRR
        for rank, doc_id in enumerate(retrieved_ids, 1):
            if doc_id in relevant_ids:
                results['mrr'].append(1 / rank)
                break
        else:
            results['mrr'].append(0)

    return {
        'recall@10': np.mean(results['recall@10']),
        'precision@10': np.mean(results['precision@10']),
        'mrr': np.mean(results['mrr'])
    }

Common Pitfalls

Pitfall 1: Wrong Embedding Model

Using general embeddings for domain-specific content. Use domain-tuned models.

Pitfall 2: Poor Chunking

Chunks that break semantic coherence. Use overlap and semantic boundaries.

Pitfall 3: No Metadata Filtering

Searching across all documents when users need filtered results. Index metadata.

Pitfall 4: Ignoring Exact Matches

Relying only on vectors when users search for specific IDs or names. Use hybrid search.

Pitfall 5: Wrong Similarity Metric

Using Euclidean distance on normalized embeddings. Use cosine for text.

Pitfall 6: Not Monitoring Quality

Search quality degrades over time without measurement. Evaluate regularly.

Conclusion

Vector search enables semantic understanding at scale. Choose embedding models that match your domain and quality requirements. Select similarity metrics appropriate for your embeddings. Use ANN algorithms for production-scale performance. Consider vector databases based on your operational requirements.

Build complete pipelines with proper chunking, hybrid search for best results, and reranking for precision. Monitor quality metrics and iterate.

Vector search is foundational technology for modern AI applications. Master it to build systems that understand user intent, not just match keywords. For teams building production AI systems, understanding how to cut LLM costs without sacrificing quality and protecting against prompt injection attacks are the next logical steps after getting search working.

Further Reading

Pinecone documentation: Vector search tutorials and API reference
Weaviate documentation: Vector database guides and schema design
pgvector GitHub repository: PostgreSQL vector extension source and docs
ANN Benchmarks: Comprehensive algorithm comparison across datasets
Sentence Transformers documentation: Embedding models and training guides
HNSW paper (Malkov & Yashunin, 2016): Algorithm fundamentals and performance analysis
OpenAI Embeddings Guide: Best practices for text embedding generation
MTEB Leaderboard: Massive Text Embedding Benchmark rankings

DNS Deep Dive for Developers: Records, TTLs, and Debugging When Resolution Breaks

Veduis — Fri, 12 Jun 2026 19:09:02 +0000

Every developer touches DNS constantly. You set up domains, configure mail records, point subdomains at servers, and wonder why your changes are not visible yet. Most of that work happens through a control panel that hides what is actually going on. That is fine until something breaks, and then the lack of understanding becomes a real problem.

DNS failures are among the hardest to debug quickly because the symptoms look like many different things. Your application appears down. Users in one region cannot reach your site but users elsewhere can. Email is bouncing. A deployment moved your app to a new IP address and some users still hit the old one. Understanding what DNS is doing is the difference between a 10-minute fix and a 3-hour incident.

If you are building on a static site architecture, DNS is often the only infrastructure layer you directly control. Getting it right matters.

How DNS Resolution Works

When a browser needs to connect to api.example.com, it needs an IP address. DNS resolution is the process of converting that hostname to an IP.

Recursive resolver: Your device sends a query to a recursive resolver, which is typically your ISP's DNS server, Google Public DNS (8.8.8.8), or Cloudflare DNS (1.1.1.1). This resolver does the work of finding the answer by querying other servers on your behalf. It also caches results to avoid repeating the same work.

Root nameservers: If the recursive resolver does not have the answer cached, it asks one of the 13 root nameserver clusters. Root servers do not know the IP of api.example.com. They know which nameserver is authoritative for the .com TLD and refer the resolver there.

TLD nameserver: The .com TLD nameserver knows which nameserver is authoritative for example.com specifically. It refers the resolver to the authoritative nameservers for example.com.

Authoritative nameserver: This server holds the actual DNS records for example.com. It returns the IP address (or other record) for api.example.com.

The recursive resolver caches this result according to the TTL (Time to Live) value in the record, then returns it to your device. Your device caches it too.

This entire chain typically completes in 20-100 milliseconds for uncached queries. For context on how this fits into overall page delivery, see our guide on how CDN caching works at the edge.

Record Types and When to Use Each

A Record

Maps a hostname to an IPv4 address. The most common record type.

api.example.com.  300  IN  A  203.0.113.42

Use A records to point subdomains or the apex domain at server IP addresses. When you have multiple A records for the same hostname, DNS returns all of them and clients can use any. This is the simplest form of load balancing, though it has no awareness of server health.

AAAA Record

Maps a hostname to an IPv6 address. The structure is identical to an A record.

api.example.com.  300  IN  AAAA  2001:db8::1

Most services should serve both A and AAAA records. Clients that support IPv6 will prefer AAAA. Clients that do not will fall back to A.

CNAME Record

An alias. Points a hostname to another hostname rather than an IP address.

www.example.com.  300  IN  CNAME  example.com.

When a client resolves www.example.com, it gets example.com as the result, then resolves example.com to an IP. CNAMEs add a resolution hop but let you point multiple subdomains at a canonical hostname. If the IP behind example.com changes, all CNAMEs pointing to it automatically follow.

Critical restriction: You cannot use a CNAME at the zone apex (the root of a domain like example.com itself). The zone apex must have an A record or use a DNS provider that supports ALIAS or ANAME records, which behave like CNAMEs but are implemented differently to avoid this restriction.

MX Record

Specifies mail server hostnames for a domain. Required for email delivery.

example.com.  300  IN  MX  10  mail1.example.com.
example.com.  300  IN  MX  20  mail2.example.com.

The priority number determines preference. Lower numbers have higher priority. A sender tries mail1 first. If unavailable, it tries mail2. The hostname in an MX record must be an A or AAAA record, not a CNAME.

TXT Record

Stores arbitrary text, used for domain verification and email authentication.

example.com.  300  IN  TXT  "v=spf1 include:_spf.google.com ~all"

TXT records serve several functions: SPF email authentication, DKIM public key publication, DMARC policy, domain ownership verification for SSL certificates and third-party services, and Google Search Console verification. Multiple TXT records on the same hostname are valid and common. If you are working through a full technical SEO audit checklist, verifying your TXT records for domain ownership and email authentication should be on it.

NS Record

Identifies the authoritative nameservers for a domain or subdomain.

example.com.  86400  IN  NS  ns1.registrar.com.
example.com.  86400  IN  NS  ns2.registrar.com.

NS records at the zone apex are set by your domain registrar. They tell the world which nameservers hold the authoritative records for your domain. Changing your DNS provider means updating NS records at the registrar level, which takes 24-48 hours to propagate globally.

NS records on a subdomain are used for delegation: handing control of sub.example.com to a different nameserver. This is common in large organizations where different teams manage different parts of the DNS namespace.

SRV Record

Specifies the hostname and port for a service.

_sip._tcp.example.com.  300  IN  SRV  10 20 5060 sipserver.example.com.

Format: _service._proto.domain TTL IN SRV priority weight port target. SRV records are used by protocols like SIP, XMPP, and Minecraft servers. Kubernetes uses SRV records for service discovery within clusters.

CAA Record

Certificate Authority Authorization. Specifies which certificate authorities are permitted to issue SSL certificates for your domain.

example.com.  300  IN  CAA  0 issue "letsencrypt.org"
example.com.  300  IN  CAA  0 issuewild "letsencrypt.org"

CAA records prevent unauthorized certificate issuance. If a CA receives a certificate request for a domain, it checks for CAA records. If the CA is not listed, it must refuse. This stops attackers from obtaining fraudulent certificates through less rigorous CAs even if they compromise the certificate request process.

RFC 8659 defines the CAA record format. The issue tag controls single-name certificates. The issuewild tag controls wildcard certificates.

TTL Strategy

TTL tells resolvers and caches how long to hold a record before re-querying. It is the primary lever you have over DNS propagation speed.

Low TTL (60-300 seconds): Changes propagate within minutes. Appropriate for records that change frequently: load balancer IPs, CDN origins during migrations, anything you might need to update quickly in an incident.

High TTL (3600-86400 seconds): Reduces DNS query load and improves resolution speed for end users. Appropriate for stable records that rarely change: mail server configuration, nameservers, long-established infrastructure.

Migration strategy: Before a planned change, drop the TTL of the record to 60-300 seconds and wait for the current TTL to expire. Then make the change. The new value propagates quickly. After the migration is confirmed stable, raise the TTL back to a higher value.

Failing to lower TTL before a migration is the most common DNS mistake. If you change a record with a 24-hour TTL, some users will continue hitting the old value for up to 24 hours. You have no control over this after the fact. This is especially painful during zero-downtime database migrations or infrastructure cutovers where every request needs to land on the right target.

Understanding Propagation

"DNS propagation" is a common source of confusion. DNS does not propagate outward from a central source in a broadcast fashion. Resolvers fetch records on demand and cache them based on TTL. When you change a record:

Your authoritative nameserver immediately serves the new value.
Resolvers that already have the old value cached continue serving the old value until their cached TTL expires.
New queries, or queries from resolvers whose cache has expired, fetch the new value.

This means propagation time equals the TTL of the old record. Not global propagation to all resolvers. Each resolver independently fetches the new value when its cache expires.

You can check what different resolvers see with:

# Check Google's resolver
dig @8.8.8.8 api.example.com

# Check Cloudflare's resolver
dig @1.1.1.1 api.example.com

# Check your local resolver
dig api.example.com

# Check the authoritative nameserver directly
dig @ns1.your-dns-provider.com api.example.com

If the authoritative nameserver returns the new value but 8.8.8.8 returns the old value, the old TTL has not expired in Google's cache yet. This is expected behavior, not an error.

Split-Horizon DNS

Split-horizon DNS returns different records for the same hostname depending on the source of the query. Internal queries get internal IP addresses. External queries get public IP addresses.

This is common in corporate networks where internal services live on RFC 1918 address space but must also be reachable externally under the same hostname. It is also used in development to return localhost for services when running locally.

On AWS, Route 53 supports split-horizon through separate public and private hosted zones with the same domain name. Internal resolvers hit the private zone. External resolvers hit the public zone.

For local development, /etc/hosts provides the simplest form of split-horizon:

# /etc/hosts
127.0.0.1 api.example.com

Tools like dnsmasq provide more sophisticated local DNS control, letting you return custom values for specific patterns while forwarding everything else to upstream resolvers.

Debugging DNS Problems

Systematic Approach

When DNS is suspected as the cause of a problem, work from the authoritative source outward.

Step 1: Verify the authoritative nameserver has the correct record.

# Find the authoritative nameservers for the domain
dig NS example.com

# Query the authoritative nameserver directly
dig @ns1.your-dns-provider.com api.example.com

If the authoritative nameserver returns the wrong answer, the problem is in your DNS provider settings. No amount of waiting will fix it.

Step 2: Check what major resolvers see.

dig @8.8.8.8 api.example.com
dig @1.1.1.1 api.example.com

If the authoritative server is correct but resolvers are returning old values, wait for TTL expiry. If TTL is very long, you cannot force external resolvers to flush their caches.

Step 3: Check what your application server sees.

# From the application server
nslookup api.example.com
host api.example.com

An application server might be using a different resolver than your workstation, or it might have its own cache.

Common DNS Failure Modes

NXDOMAIN when a hostname should exist: The authoritative nameserver has no record for that hostname. Check spelling, check that the record was created at the correct DNS provider (not a cached provider), and check that NS records at the registrar match the provider you are editing.

Resolution works intermittently: Multiple A records for the same hostname with one returning an unhealthy IP. Or a nameserver that is intermittently unavailable. Check all A records and verify each IP responds correctly.

Email bouncing: Usually an MX record problem or SPF/DKIM misconfiguration. Check that MX records exist, that they point to valid hostnames (not CNAMEs), and that the SPF TXT record includes all mail-sending services.

SSL certificate validation failure: Common during domain migrations. The CA validation requires the domain to resolve correctly. If the domain recently changed nameservers and the new records have not propagated, certificate issuance fails. For a broader look at keeping certificates current, see our guide on SSL/TLS certificate automation with Let's Encrypt.

The DNS Attack Surface

DNS is a target for attackers. Understanding the attack surface helps you configure defenses.

DNS cache poisoning: An attacker feeds false DNS records to a resolver, causing it to return attacker-controlled IPs. DNSSEC (DNS Security Extensions) mitigates this by digitally signing records so resolvers can verify authenticity. Cloudflare's DNS over HTTPS and DNS over TLS prevent eavesdropping on DNS queries.

DNS hijacking: An attacker gains access to your DNS provider account and changes records to redirect your domain. Protect against this with strong credentials, two-factor authentication on your DNS provider account, and registry locks at the registrar level for high-value domains. If you are responsible for application security, our web security fundamentals guide covers additional layers of defense worth implementing.

Dangling DNS records: A subdomain (like subdomain.example.com) points via CNAME to a cloud service you no longer use. An attacker can claim that cloud service and serve content under your subdomain. Audit CNAME records periodically and remove any pointing at decommissioned services.

Zone transfer exposure: By default, some older DNS servers allow zone transfers (AXFR queries) that enumerate every record in a zone. This reveals your complete infrastructure layout. Restrict zone transfers to authorized secondary nameservers only.

Tools Worth Having

dig is the primary DNS debugging tool on Linux and macOS. Covers every record type and resolver option.

nslookup is available on Windows and most Unix systems. More limited than dig but sufficient for basic lookups.

MXToolbox provides browser-based DNS lookup, MX record validation, and blacklist checking. Useful for verifying mail configuration from an external perspective.

DNSViz visualizes the full DNS resolution chain and highlights DNSSEC validation issues. Invaluable for diagnosing delegation and signing problems.

WhatsMyDNS shows what DNS resolvers in different geographic regions are returning for a hostname. Useful for confirming propagation across global resolvers.

Understanding DNS at this level changes how you approach infrastructure. Domain migrations, SSL certificate renewals, email deliverability problems, and load balancer changes all become more predictable when you understand what DNS is doing at each step. The 10-minute fixes come from knowing where to look first.

If you are managing infrastructure for a growing business, DNS is just one piece of the puzzle. Our infrastructure as code guide with Terraform shows how to manage DNS records alongside the rest of your infrastructure declaratively, and our Ubuntu server guide covers the foundational server setup that DNS ultimately points to.

We Built a VS Code Extension That Installs 90+ MCP Servers in One Click

Veduis — Thu, 11 Jun 2026 18:13:36 +0000

I have been using Claude Desktop, Cursor, and Cline for months. The Model Context Protocol (MCP) is genuinely useful — it lets your AI assistant query databases, browse the web, manage files, and interact with APIs. But the setup process is painful. You find a server on GitHub, read the README, figure out which config file to edit, add the JSON manually, restart your client, and hope you did not typo the command path.

We got tired of doing this by hand. So we built something to fix it.

1 Click MCP Installer By VePrompts is a VS Code extension that lets you browse, search, and install any of our 90+ curated MCP servers directly from your editor. No JSON editing. No restarts. One click and it is done.

It is free, open source, and available now on the VS Code Marketplace and Open VSX Registry.

What MCP Is and Why It Matters
MCP is an open protocol from Anthropic that standardizes how AI assistants connect to external tools. Instead of every AI client building its own plugin system, MCP gives you one interface that works everywhere.

A single MCP server can expose multiple tools. A database server might let your AI run SQL queries, list tables, and inspect schemas. A browser automation server might let it click elements, fill forms, and scrape pages. The AI decides which tool to use based on your request.

The protocol is gaining serious traction. Claude Desktop supports it natively. Cursor, Cline, and Devin (formerly Windsurf) all have MCP integrations. The problem is not the protocol — it is the friction of getting servers installed and configured.

What the Extension Actually Does
The extension adds a VePrompts MCP panel to your VS Code sidebar. Here is what happens when you use it:

Browse by category. Servers are organized into groups — databases, browser automation, developer tools, cloud platforms, AI/ML, communication, and more. You scroll, you click, you see what is available.

See everything before you install. Each server page shows install methods, required environment variables, which clients it works with, available tools, GitHub stars, and last commit date. You know exactly what you are getting.

One-click install. Click the install button. The extension detects which MCP clients you have installed, validates the config, writes it atomically, and tells you to restart. That is it. No hand-editing JSON.

Secure env var handling. If a server needs an API key, the extension prompts you securely. Nothing gets logged or transmitted. It stays local.

Config validation. Before writing anything, the extension validates your MCP client config. If something is wrong, it tells you before it breaks your setup.

Smart search. Type a keyword and filter across all 90+ servers by name, tag, or description.

Supported Clients
The extension works with every major MCP client:

Claude Desktop — macOS and Windows
Cursor — the AI-powered code editor
Cline — the VS Code extension for autonomous coding
Devin (formerly Windsurf) — the AI-native IDE
It detects which clients you have installed and configures each one correctly. Claude Desktop uses claude_desktop_config.json. Cursor uses .cursor/mcp.json. Cline uses its own settings panel. The extension handles the differences so you do not have to.

How to Use It
Install the extension from the VS Code Marketplace
Open the VePrompts MCP panel from the activity bar
Browse or search for a server
Click install
Restart your AI client
Start using the tools
That is the entire workflow. The extension is open source on GitHub if you want to see how it works or contribute.

The Server Directory Behind It
The extension is powered by VePrompts, our free MCP server directory. We curate and maintain metadata for every server — install commands, environment variables, compatibility matrices, tool listings, and ratings.

The directory is searchable at veprompts.com/mcp/servers/. You can filter by category, client, transport type, programming language, and whether the server is official or community-built. Each server has its own detail page with full documentation.

We also publish guides on the Veduis blog for specific setups. If you are new to MCP, start with our complete setup guide.

Why We Built This
We run Veduis, a digital agency, and VePrompts, a free AI tooling directory. We use MCP servers daily for client work — querying databases, automating browser tasks, managing cloud resources. The manual setup was costing us time every time we onboarded a new server or switched machines.

The extension solves a real problem we had. We use it ourselves. If you are working with MCP, it will probably save you time too.

Get It
VS Code Marketplace: 1 Click MCP Installer By VePrompts
Open VSX Registry: 1 Click MCP Installer By VePrompts
GitHub: https://Veduis/1clickmcp
Documentation: https://veprompts.com/mcp/vscode-extension/
Server Directory: https://veprompts.com/mcp/servers/
Setup Guide: https://veduis.com/blog/how-to-install-mcp-servers-vs-code/
Our Agency: https://veduis.com
Free. Open source. MIT license. No signup required.

HTTP/3 and QUIC for Web Developers: What Changes and What You Need to Configure

Veduis — Sun, 24 May 2026 20:42:42 +0000

This post was original published on Veduis

HTTP/3 has been shipping in production for several years now. Chrome uses HTTP/3 for roughly 20% of web traffic. Cloudflare, Fastly, and AWS CloudFront all support it. Google has used QUIC (the protocol underlying HTTP/3) in production since 2015. Understanding what changed and why matters for making informed decisions about your infrastructure configuration.

The motivation for HTTP/3 was one specific problem with HTTP/2: head-of-line blocking at the TCP layer. Fixing that problem required replacing TCP with a new transport protocol, which is why HTTP/3 is a more substantial change than HTTP/2 was.

The Problem HTTP/3 Solves

HTTP/2 solved head-of-line blocking at the HTTP layer by multiplexing multiple streams over a single TCP connection. In HTTP/1.1, you needed multiple connections to fetch multiple resources in parallel. HTTP/2 sends all resources over one connection with stream multiplexing.

But TCP has its own head-of-line blocking. If a TCP packet is lost, TCP halts all delivery on the connection until that packet is retransmitted and received. All HTTP/2 streams over that connection wait, even streams that did not depend on the lost packet. On reliable networks this is negligible. On mobile networks with 1-2% packet loss rates, it causes real performance degradation.

QUIC solves this by implementing streams at the transport layer over UDP. When a packet is lost, only the stream that depended on that packet stalls. Other streams continue delivering. On lossy networks (cellular, high-latency connections), this produces meaningful improvements. These gains compound with other optimizations covered in our CDN comparison guide, which benchmarks how major providers implement HTTP/3 at the edge.

What Changed

UDP Instead of TCP

QUIC runs over UDP rather than TCP. UDP has no built-in reliability or ordering — QUIC implements its own reliability, ordering, and congestion control on top. This is why QUIC can provide per-stream reliability without the connection-level head-of-line blocking that TCP imposes.

Some corporate firewalls and networks block or rate-limit UDP traffic, which means HTTP/3 may not work for a portion of your users. All HTTP/3 implementations fall back to HTTP/2 when HTTP/3 is unavailable, so this is not a hard dependency.

Integrated TLS 1.3

HTTP/3 requires TLS 1.3. TLS is integrated into QUIC at the transport layer rather than layered on top as it is with TLS over TCP. This enables 0-RTT connection establishment: a client that previously connected to a server can resume with zero round trips before sending application data.

Traditional TLS 1.2 required 2 round trips before any application data. TLS 1.3 over TCP reduced this to 1 round trip on initial connection and 0-RTT on resumption. QUIC gets the same 1-RTT initial / 0-RTT resumption performance, but with the transport-level QUIC benefits on top. If you are not yet managing your TLS certificates with automation, our guide on SSL/TLS certificate automation covers Let's Encrypt and ACME-based renewal — a prerequisite for any production HTTP/3 deployment.

Connection Migration

HTTP/3 connections are identified by a connection ID rather than by the 4-tuple (source IP, source port, destination IP, destination port) that identifies TCP connections. This means a connection can survive a change in the client's IP address or port — common when a mobile device switches from WiFi to cellular.

In practice, connection migration means HTTP/3 connections survive network transitions without re-establishing the connection and going through TLS negotiation again. For mobile users who frequently switch between WiFi and cellular, this eliminates the reconnection delay.

Multiplexed Streams Without Head-of-Line Blocking

QUIC streams are independent at the transport layer. A lost packet affecting stream 3 does not stall streams 1, 2, and 4. This is the core performance improvement over HTTP/2 for users on networks with any meaningful packet loss.

Browser Support and Adoption

HTTP/3 is supported by all major browsers: Chrome 87+, Firefox 88+, Safari 14+, Edge 87+. Clients always fall back gracefully to HTTP/2 or HTTP/1.1 if the server does not support HTTP/3 or if UDP is blocked at the network level.

Servers advertise HTTP/3 support via the Alt-Svc HTTP header:

Alt-Svc: h3=":443"; ma=86400

This tells the client that HTTP/3 is available on port 443, with the information valid for 86400 seconds (one day). The client uses HTTP/2 for the first connection, receives this header, and uses HTTP/3 for subsequent connections.

Enabling HTTP/3

Cloudflare

HTTP/3 is enabled automatically for all Cloudflare zones. You can verify it is enabled in the dashboard under Speed > Optimization > Protocol Optimization. No configuration is required.

To verify HTTP/3 is being used, check the cf-cache-status header or the network protocol in Chrome DevTools (right-click on columns in the Network tab to add the Protocol column). For a detailed look at how Cloudflare compares to other CDN providers on HTTP/3 and broader performance, see our CDN comparison: Cloudflare vs Fastly vs BunnyCDN.

Nginx (1.25+)

QUIC support was merged into Nginx mainline in version 1.25. Earlier versions require a patch or building from source with QUIC support.

# nginx.conf
http {
  server {
    listen 443 quic reuseport;  # QUIC/HTTP/3
    listen 443 ssl;              # TCP/HTTP/2

    ssl_certificate /etc/ssl/cert.pem;
    ssl_certificate_key /etc/ssl/key.pem;
    ssl_protocols TLSv1.3;       # HTTP/3 requires TLS 1.3

    # Advertise HTTP/3 support
    add_header Alt-Svc 'h3=":443"; ma=86400';

    # QUIC-specific settings
    ssl_early_data on;           # Enable 0-RTT
    quic_retry on;               # Mitigates reflection attacks
  }
}

The reuseport option is required for QUIC. Without it, multiple worker processes cannot share the UDP port.

Security consideration: ssl_early_data on enables 0-RTT resumption. Early data is vulnerable to replay attacks. Do not enable 0-RTT for non-idempotent requests (POST, PUT, DELETE) without replay protection. Static asset servers are generally safe to enable it on.

Caddy

Caddy 2 enables HTTP/3 by default when serving HTTPS. No additional configuration is required.

Verifying HTTP/3 Is Working

# Using curl (requires curl 7.66+ with QUIC support)
curl --http3 -I https://yourdomain.com

# Using the h2load tool from nghttp2
h3load https://yourdomain.com --npn-list=h3

Chrome DevTools shows the protocol used for each request in the Network tab. h3 indicates HTTP/3. h2 indicates HTTP/2. http/1.1 indicates HTTP/1.1. The http3check.net tool lets you verify HTTP/3 support from your browser without any local tooling.

Performance Impact

HTTP/3 performance improvements are most significant:

On mobile networks with packet loss. The elimination of TCP head-of-line blocking directly helps when network conditions are degraded. Studies from Google and Cloudflare show 10-20% improvements in Time to First Byte on mobile networks under lossy conditions. These gains feed into your Core Web Vitals scores, particularly LCP on mobile devices where network conditions are less predictable.

On high-latency connections. The 0-RTT resumption and 1-RTT initial connection reduce connection setup overhead meaningfully when latency is already high.

For connections that survive network transitions. Mobile users who switch between WiFi and cellular benefit from connection migration.

HTTP/3 performance improvements on wired low-latency connections are more modest. The TCP head-of-line blocking issue is largely irrelevant when packet loss is near zero.

What Developers Do Not Need to Change

HTTP/3 is transparent at the application layer. Your HTTP headers, request formats, response formats, caching behavior, and authentication patterns are all unchanged. The change is at the transport and session layer, below where application code operates.

You do not need to rewrite any application code to support HTTP/3. You enable it at the server or CDN level, and clients that support it use it automatically.

The Alt-Svc header and HTTP/3 negotiation happen automatically. Your application code sees the same request objects regardless of whether the underlying transport is HTTP/1.1, HTTP/2, or HTTP/3. Your existing caching strategies — browser cache headers, CDN TTLs, and ETags — continue to work without modification under HTTP/3.

Practical Recommendation

If you use a CDN (Cloudflare, Fastly, AWS CloudFront), HTTP/3 is either already enabled or available with a single toggle. Enable it. The fallback to HTTP/2 is automatic and robust. The downside risk is near zero.

If you serve directly from your own infrastructure with Nginx 1.25+ or Caddy, enabling HTTP/3 requires the configuration changes above and opening UDP port 443 in your firewall rules. Test in a staging environment first and verify that your network allows UDP traffic on port 443.

The overhead of enabling HTTP/3 on modern infrastructure is low. The performance benefits for your mobile users and users on less-than-ideal network connections are real and accumulate meaningfully at scale.

AI-Assisted Development Workflows: Code Review, Testing, and Documentation

Veduis — Tue, 14 Apr 2026 20:24:35 +0000

AI has transformed software development. Code completion predicts what you will type next. AI reviewers catch bugs before human review. Test generation covers edge cases you might miss. Documentation updates happen automatically as code changes. This is not science fiction; it is the current state of the art for teams using AI-assisted development workflows.

But integrating AI effectively requires more than installing tools. It requires understanding where AI helps and where it hinders. It requires maintaining code quality when AI generates code. It requires verifying AI suggestions rather than accepting them blindly. It requires updating workflows and team practices to incorporate AI assistance.

I have integrated AI tools into development workflows across multiple teams. I have learned that AI excels at pattern recognition and boilerplate but struggles with architectural decisions. I have seen teams double their velocity with AI assistance and others create technical debt by accepting every suggestion. This guide covers the patterns that work: AI-powered code review that catches issues early, automated test generation that increases coverage, documentation maintenance that stays synchronized, refactoring assistance that accelerates modernization, and integrating AI tools into existing workflows.

(Want to cut the API costs of these workflows? Read our complete guide to *reducing AI coding tool token usage*)

The AI-Assisted Development Stack

Current Capabilities

Code completion: Predict and generate code as you type
Code review: Automated review for common issues
Test generation: Create unit tests from code
Documentation: Generate and update docs from code
Refactoring: Suggest and apply transformations
Debugging: Explain errors and suggest fixes

Tool Landscape

Code completion: GitHub Copilot, Cursor, Amazon CodeWhisperer
Code review: GitHub Copilot Review, Amazon CodeGuru, SonarQube with AI
Test generation: CodiumAI, GitHub Copilot Chat
Documentation: Mintlify, ReadMe.com AI, custom LLM pipelines
Refactoring: Sourcegraph Cody, Continue.dev

AI Code Review

Automated Review Pipeline

# .github/workflows/ai-review.yml
name: AI Code Review
on: [pull_request]

jobs:
  ai-review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: AI Code Review
        uses: ai-reviewer-action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
          review-level: 'detailed'

What AI Reviewers Catch

Security issues:

SQL injection vulnerabilities
Hardcoded secrets
Unsafe deserialization
XSS vulnerabilities

Performance problems:

N+1 queries
Unnecessary computations
Memory leaks
Inefficient algorithms

Code quality:

Unused imports
Dead code
Complex functions
Missing error handling

Implementation Example

# AI review script
import openai
import difflib

def ai_review_diff(diff: str) -> list[dict]:
    prompt = f"""Review this code diff for issues:
{diff}

Check for:
1. Security vulnerabilities
2. Performance problems
3. Logic errors
4. Code smells

Return findings as JSON array:
[{{
    "line": number,
    "severity": "high|medium|low",
    "category": "security|performance|logic|quality",
    "message": "description",
    "suggestion": "recommended fix"
}}]"""

    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        response_format={"type": "json_object"}
    )

    return json.loads(response.choices[0].message.content)

# Post as PR comments
for finding in findings:
    post_pr_comment(
        path=file_path,
        line=finding['line'],
        body=f"**{finding['category'].upper()}: {finding['severity']}**\n\n"
              f"{finding['message']}\n\n"
              f"**Suggestion:** {finding['suggestion']}"
    )

Combining with Human Review

AI as first pass:

AI reviews immediately on PR open
Addresses trivial issues automatically
Flags issues for human attention

Human focuses on:

Architecture decisions
Business logic correctness
Maintainability concerns
Knowledge sharing

Test Generation

Automated Unit Test Creation

# CodiumAI example
def calculate_discount(price: float, customer_type: str) -> float:
    """Calculate discount based on customer type."""
    if customer_type == "premium":
        return price * 0.8
    elif customer_type == "regular":
        return price * 0.95
    else:
        raise ValueError(f"Unknown customer type: {customer_type}")

# AI generates:
import pytest

def test_calculate_discount_premium():
    assert calculate_discount(100.0, "premium") == 80.0

def test_calculate_discount_regular():
    assert calculate_discount(100.0, "regular") == 95.0

def test_calculate_discount_unknown_type():
    with pytest.raises(ValueError, match="Unknown customer type: invalid"):
        calculate_discount(100.0, "invalid")

def test_calculate_discount_zero_price():
    assert calculate_discount(0.0, "premium") == 0.0

def test_calculate_discount_negative_price():
    # Edge case: should handle or reject negative prices
    with pytest.raises(ValueError):
        calculate_discount(-10.0, "premium")

Property-Based Test Generation

# AI generates property-based tests
from hypothesis import given, strategies as st

@given(st.floats(min_value=0), st.sampled_from(["premium", "regular"]))
def test_discount_never_exceeds_original(price, customer_type):
    """Discount should never make price higher"""
    result = calculate_discount(price, customer_type)
    assert result <= price
    assert result >= 0

@given(st.floats(min_value=0))
def test_premium_always_cheaper_than_regular(price):
    """Premium discount should be better than regular"""
    premium = calculate_discount(price, "premium")
    regular = calculate_discount(price, "regular")
    assert premium <= regular

Integration Test Generation

# Generate integration tests from API specs
openapi_spec = load_openapi_spec()

for endpoint in openapi_spec['paths']:
    test_code = ai_generate_test(
        endpoint=endpoint,
        spec=openapi_spec['paths'][endpoint],
        framework="pytest",
        style="arrange-act-assert"
    )

    write_test_file(endpoint, test_code)

Documentation Maintenance

Auto-Generated API Docs

# Generate OpenAPI from code annotations
from flask import Flask
from flasgger import Swagger

app = Flask(__name__)
Swagger(app)

@app.route('/users', methods=['POST'])
def create_user():
    """
    Create a new user
    ---
    post:
      summary: Create user
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                email:
                  type: string
                name:
                  type: string
    responses:
      201:
        description: User created successfully
    """
    # Implementation

Code Comment Generation

# AI generates docstrings
def complex_algorithm(data: list[dict]) -> dict:
    # AI analyzes and generates:
    """
    Aggregate transaction data by category and calculate statistics.

    Args:
        data: List of transaction dictionaries with 'category' and 'amount' keys

    Returns:
        Dictionary mapping category to statistics dict with 'total', 
        'average', and 'count' keys

    Raises:
        ValueError: If data contains negative amounts

    Example:
        >>> data = [
        ...     {"category": "food", "amount": 10.50},
        ...     {"category": "food", "amount": 25.00}
        ... ]
        >>> complex_algorithm(data)
        {"food": {"total": 35.50, "average": 17.75, "count": 2}}
    """
    # Implementation

README Maintenance

# GitHub Action to update README
name: Update Documentation
on:
  push:
    paths:
      - 'src/**'
      - 'api/**'

jobs:
  update-docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Generate updated README
        run: |
          python scripts/generate_readme.py

      - name: Commit changes
        run: |
          git config --local user.email "action@github.com"
          git config --local user.name "GitHub Action"
          git diff --quiet && git diff --staged --quiet || 
            (git add README.md && git commit -m "docs: Auto-update README" && git push)

Refactoring Assistance

Pattern-Based Refactoring

# Before: AI identifies pattern
users = []
for user in database.query(User).all():
    if user.is_active:
        users.append(user)

# AI suggests:
users = [user for user in database.query(User).all() if user.is_active]

# Or better (database-level filtering):
users = database.query(User).filter(User.is_active == True).all()

Modernization

# Legacy callback pattern
process_data(data, callback=handle_result)

# AI suggests async/await
async def process():
    result = await process_data(data)
    await handle_result(result)

Type Annotation Addition

# AI adds type hints
def calculate(a, b, operation):
    if operation == "add":
        return a + b

# AI generates:
from typing import Literal

def calculate(
    a: float,
    b: float,
    operation: Literal["add", "subtract", "multiply", "divide"]
) -> float:
    if operation == "add":
        return a + b
    elif operation == "subtract":
        return a - b
    # ...

Workflow Integration

Pre-Commit Hooks

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: ai-lint
        name: AI Code Review
        entry: python scripts/ai_lint.py
        language: python
        types: [python]

      - id: ai-test-gen
        name: AI Test Generation
        entry: python scripts/ai_generate_tests.py --check
        language: python
        types: [python]

IDE Integration

Cursor IDE workflow:

Write high-level comment describing intent
AI generates implementation
Review and refine
AI generates tests
Run and iterate

GitHub Copilot Chat:

/explain - Explain selected code
/fix - Suggest fix for error
/tests - Generate tests
/docs - Generate documentation

CI/CD Integration

# .github/workflows/ai-enhanced.yml
name: AI-Enhanced CI
on: [push]

jobs:
  ai-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: AI Security Scan
        run: |
          python scripts/ai_security_scan.py --fail-on-high

      - name: AI Test Coverage Check
        run: |
          python scripts/ai_suggest_missing_tests.py

      - name: AI Documentation Check
        run: |
          python scripts/ai_check_docs.py

Best Practices

Verification Required

Always review AI output:

Security implications
Business logic correctness
Performance characteristics
Edge cases

Red flags:

Generated code that looks plausible but is wrong
Missing error handling
Hardcoded values that should be configurable
Over-engineered solutions

Code Quality Standards

AI-generated code must pass:

All existing linting rules
Type checking
Unit tests
Security scanning

Do not lower standards for AI code.

Knowledge Preservation

Document AI decisions:

# AI-assisted refactoring: Converted from callbacks to async/await
# Date: 2026-04-10
# Rationale: Improve readability and error handling
# Reviewed by: @username

Gradual Adoption

Start with:

Code completion for boilerplate
Documentation generation
Test generation for new code
Code review assistance

Expand to:

Refactoring assistance
Legacy code modernization
Architecture suggestions

Common Pitfalls

Pitfall 1: Blind Acceptance

Accepting all AI suggestions without review. Always verify.

Pitfall 2: Skill Atrophy

Developers forgetting how to code without AI. Maintain fundamental skills.

Pitfall 3: Over-Reliance on Boilerplate

AI generates repetitive code instead of abstracting. Review for refactoring opportunities.

Pitfall 4: Security Blind Spots

AI may generate vulnerable code. Security review remains essential.

Pitfall 5: Context Loss

AI lacking full project context produces suboptimal solutions. Provide context in prompts.

Pitfall 6: No Attribution

Not tracking what code is AI-generated. Document AI assistance for accountability.

Conclusion

AI-assisted development increases productivity and code quality when used thoughtfully. Let AI handle repetitive tasks: boilerplate, documentation, test scaffolding. Keep human judgment for architecture, security, and business logic.

Integrate AI tools into existing workflows through pre-commit hooks, CI pipelines, and IDE extensions. Maintain code quality standards. Verify AI output before committing.

The goal is not AI replacement of developers but AI amplification of developer capabilities. Teams that master this partnership ship faster with higher quality.

Further Reading

GitHub Copilot Documentation: IDE integration patterns and features
Cursor Documentation: Mastering AI-native editor workflows
OpenAI Prompt Engineering Guide: Strategies for effective AI interaction
Microsoft AI Productivity Studies: Research on how AI coding tools impact developer velocity
Martin Fowler on AI-Assisted Programming: Engineering practices for working with LLMs

How I Automate Compliance Reporting with Python and AI Tools

Veduis — Thu, 26 Mar 2026 09:48:19 +0000

I've seen compliance teams at small businesses spend 15-20 hours monthly on reporting tasks that could be automated. Data collection from multiple systems, formatting into required templates, generating evidence packages, and tracking policy acknowledgments consume time that could be spent on actual risk management and strategic compliance work.

In my experience, the combination of Python scripting and AI tools has made compliance automation accessible to organizations without dedicated development teams. Simple scripts handle data extraction and formatting, while AI assists with document analysis, policy drafting, and evidence summarization.

In this guide, I share practical automation examples for common compliance tasks, designed for implementation without extensive programming background.

Why I Automate Compliance Reporting

Manual compliance reporting creates multiple problems beyond time consumption. Here's what I've observed.

Consistency Issues:

Human data entry and formatting introduces variability. The same report generated by different team members may present information differently, creating confusion during audits.

Timeliness Challenges:

Manual processes often cannot keep pace with reporting requirements. Monthly reports slip into the following month; quarterly evidence packages are assembled under deadline pressure.

Scalability Limitations:

As businesses grow, compliance obligations expand. Manual processes that work for 50 employees become unsustainable at 200.

Error Risk:

Copy-paste errors, formula mistakes in spreadsheets, and overlooked data sources create compliance exposure that automation eliminates.

Getting Started with Python for Compliance

I find Python provides an accessible entry point for compliance automation. The language reads almost like English, and extensive libraries handle complex tasks with minimal code.

Setting Up the Environment

Installing Python:

Download Python from python.org. During installation, check "Add Python to PATH" to enable command-line access.

Essential Libraries:

Install libraries needed for compliance automation:

pip install pandas openpyxl python-docx requests schedule openai

Library	Purpose
pandas	Data manipulation and analysis
openpyxl	Excel file handling
python-docx	Word document generation
requests	API communication
schedule	Task scheduling
openai	AI integration

First Automation: Access Review Report

Access reviews require collecting user access data, comparing against approved access lists, and generating exception reports. This task typically takes 4-6 hours monthly when done manually.

My Automated Script:

import pandas as pd
from datetime import datetime

def generate_access_review():
    # Load current access from system export
    current_access = pd.read_csv('exports/current_user_access.csv')

    # Load approved access matrix
    approved_access = pd.read_csv('config/approved_access_matrix.csv')

    # Merge to find discrepancies
    merged = current_access.merge(
        approved_access, 
        on=['user_id', 'system', 'role'],
        how='outer',
        indicator=True
    )

    # Identify unauthorized access (in current but not approved)
    unauthorized = merged[merged['_merge'] == 'left_only']

    # Identify missing access (approved but not current)
    missing = merged[merged['_merge'] == 'right_only']

    # Generate report
    report_date = datetime.now().strftime('%Y-%m-%d')

    with pd.ExcelWriter(f'reports/access_review_{report_date}.xlsx') as writer:
        current_access.to_excel(writer, sheet_name='Current Access', index=False)
        unauthorized.to_excel(writer, sheet_name='Unauthorized Access', index=False)
        missing.to_excel(writer, sheet_name='Missing Access', index=False)

    # Also save latest for dashboard
    import shutil
    shutil.copy(f'reports/access_review_{report_date}.xlsx', 'reports/access_review_latest.xlsx')

    # Summary for email
    summary = f"""
    Access Review Summary - {report_date}

    Total Users Reviewed: {len(current_access['user_id'].unique())}
    Unauthorized Access Findings: {len(unauthorized)}
    Missing Access Findings: {len(missing)}

    Report saved to: reports/access_review_{report_date}.xlsx
    """

    return summary

if __name__ == '__main__':
    print(generate_access_review())

Running the Script:

python access_review.py

This 40-line script replaces hours of manual Excel manipulation. I customize the input files and field names to match specific system exports.

Common Compliance Automation Examples

The following examples address frequently automated compliance tasks I encounter.

Policy Acknowledgment Tracking

Track employee policy acknowledgments and generate reports of outstanding items.

import pandas as pd
from datetime import datetime, timedelta

def policy_acknowledgment_report():
    # Load acknowledgment data (from HR system export)
    acks = pd.read_csv('exports/policy_acknowledgments.csv')

    # Load active employee list
    employees = pd.read_csv('exports/active_employees.csv')

    # Load required policies
    policies = pd.read_csv('config/required_policies.csv')

    # Create expected acknowledgments (every employee x every policy)
    expected = employees.merge(policies, how='cross')

    # Merge with actual acknowledgments
    status = expected.merge(
        acks,
        on=['employee_id', 'policy_id'],
        how='left'
    )

    # Identify missing acknowledgments
    status['acknowledged'] = status['acknowledgment_date'].notna()
    missing = status[~status['acknowledged']].copy()

    # Calculate days overdue
    today = datetime.now()
    missing['days_overdue'] = missing.apply(
        lambda row: (today - pd.to_datetime(row['policy_effective_date'])).days,
        axis=1
    )

    # Group by department for management reporting
    by_department = missing.groupby('department').agg({
        'employee_id': 'count',
        'days_overdue': 'mean'
    }).rename(columns={
        'employee_id': 'missing_count',
        'days_overdue': 'avg_days_overdue'
    })

    # Generate outputs
    report_date = datetime.now().strftime('%Y-%m-%d')

    with pd.ExcelWriter(f'reports/policy_ack_{report_date}.xlsx') as writer:
        status.to_excel(writer, sheet_name='All Status', index=False)
        missing.to_excel(writer, sheet_name='Missing', index=False)
        by_department.to_excel(writer, sheet_name='By Department')

    # Also save latest for dashboard
    import shutil
    shutil.copy(f'reports/policy_ack_{report_date}.xlsx', 'reports/policy_ack_latest.xlsx')

    return f"Report generated: {len(missing)} missing acknowledgments across {len(by_department)} departments"

if __name__ == '__main__':
    print(policy_acknowledgment_report())

Vendor Risk Assessment Tracking

Monitor vendor assessment status and generate renewal reminders.

import pandas as pd
from datetime import datetime, timedelta

def vendor_assessment_status():
    # Load vendor data
    vendors = pd.read_csv('data/vendors.csv')

    # Load assessment records
    assessments = pd.read_csv('data/vendor_assessments.csv')

    # Get most recent assessment per vendor
    latest = assessments.sort_values('assessment_date').groupby('vendor_id').last()

    # Merge with vendor info
    status = vendors.merge(latest, on='vendor_id', how='left')

    # Calculate assessment age
    today = datetime.now()
    status['assessment_date'] = pd.to_datetime(status['assessment_date'])
    status['days_since_assessment'] = (today - status['assessment_date']).dt.days

    # Flag vendors needing reassessment (annual requirement)
    status['needs_reassessment'] = status['days_since_assessment'] > 365
    status['never_assessed'] = status['assessment_date'].isna()

    # Categorize by urgency
    def urgency(row):
        if row['never_assessed']:
            return 'Critical - Never Assessed'
        elif row['days_since_assessment'] > 400:
            return 'High - Overdue'
        elif row['days_since_assessment'] > 330:
            return 'Medium - Due Soon'
        else:
            return 'Low - Current'

    status['urgency'] = status.apply(urgency, axis=1)

    # Summary by risk tier
    summary = status.groupby(['vendor_risk_tier', 'urgency']).size().unstack(fill_value=0)

    # Generate report
    report_date = datetime.now().strftime('%Y-%m-%d')

    with pd.ExcelWriter(f'reports/vendor_status_{report_date}.xlsx') as writer:
        status.to_excel(writer, sheet_name='All Vendors', index=False)
        summary.to_excel(writer, sheet_name='Summary')
        status[status['urgency'] != 'Low - Current'].to_excel(
            writer, sheet_name='Action Required', index=False
        )

    # Also save latest for dashboard
    import shutil
    shutil.copy(f'reports/vendor_status_{report_date}.xlsx', 'reports/vendor_status_latest.xlsx')

    return summary

if __name__ == '__main__':
    print(vendor_assessment_status())

Security Training Compliance

Generate training compliance reports with completion rates by department.

import pandas as pd
from datetime import datetime

def training_compliance_report():
    # Load employee and training data
    employees = pd.read_csv('exports/employees.csv')
    training_records = pd.read_csv('exports/training_completions.csv')
    required_training = pd.read_csv('config/required_training.csv')

    # Calculate required training per employee based on role
    employee_requirements = employees.merge(
        required_training,
        on='job_role',
        how='left'
    )

    # Match with completion records
    compliance = employee_requirements.merge(
        training_records,
        on=['employee_id', 'training_id'],
        how='left'
    )

    # Determine compliance status
    compliance['completed'] = compliance['completion_date'].notna()

    # Check if completion is within validity period (annual training)
    today = datetime.now()
    compliance['completion_date'] = pd.to_datetime(compliance['completion_date'])
    compliance['is_current'] = (
        compliance['completed'] & 
        ((today - compliance['completion_date']).dt.days <= 365)
    )

    # Department summary
    dept_summary = compliance.groupby('department').agg({
        'employee_id': 'nunique',
        'is_current': ['sum', 'mean']
    })
    dept_summary.columns = ['employee_count', 'compliant_trainings', 'compliance_rate']
    dept_summary['compliance_rate'] = (dept_summary['compliance_rate'] * 100).round(1)

    # Non-compliant list for follow-up
    non_compliant = compliance[~compliance['is_current']].copy()[[
        'employee_id', 'employee_name', 'department', 
        'training_name', 'completion_date'
    ]]

    # Generate report
    report_date = datetime.now().strftime('%Y-%m-%d')

    with pd.ExcelWriter(f'reports/training_compliance_{report_date}.xlsx') as writer:
        dept_summary.to_excel(writer, sheet_name='Department Summary')
        non_compliant.to_excel(writer, sheet_name='Action Required', index=False)
        compliance.to_excel(writer, sheet_name='Full Detail', index=False)

    # Also save latest for dashboard
    import shutil
    shutil.copy(f'reports/training_compliance_{report_date}.xlsx', 'reports/training_compliance_latest.xlsx')

    overall_rate = compliance['is_current'].mean() * 100
    return f"Overall compliance rate: {overall_rate:.1f}%\nNon-compliant items: {len(non_compliant)}"

if __name__ == '__main__':
    print(training_compliance_report())

Integrating AI for Document Analysis

I find AI tools excel at tasks requiring interpretation and summarization that pure scripting cannot handle.

Policy Gap Analysis

I use AI to compare current policies against regulatory requirements.

import openai
from pathlib import Path

def analyze_policy_gaps(policy_file, requirement_file):
    # Read documents
    policy_text = Path(policy_file).read_text()
    requirements = Path(requirement_file).read_text()

    client = openai.OpenAI()

    response = client.chat.completions.create(
        model="gpt-4-turbo",
        messages=[
            {
                "role": "system",
                "content": """You are a compliance analyst reviewing policies against 
                regulatory requirements. Identify gaps where the policy does not 
                adequately address requirements. Be specific about which requirements 
                are unaddressed or insufficiently covered."""
            },
            {
                "role": "user",
                "content": f"""Review this policy against the requirements listed.

                POLICY:
                {policy_text}

                REQUIREMENTS:
                {requirements}

                Provide a structured analysis:
                1. Requirements fully addressed
                2. Requirements partially addressed (explain gaps)
                3. Requirements not addressed
                4. Recommendations for policy updates"""
            }
        ],
        temperature=0.3
    )

    return response.choices[0].message.content

# Example usage
if __name__ == '__main__':
    analysis = analyze_policy_gaps(
        'policies/data_protection_policy.txt',
        'requirements/gdpr_requirements.txt'
    )
    print(analysis)

Evidence Package Summarization

I summarize lengthy evidence documents for audit preparation.

import openai
from pathlib import Path

def summarize_evidence(evidence_folder, control_description):
    # Collect all evidence files
    evidence_files = list(Path(evidence_folder).glob('*'))

    evidence_content = []
    for file in evidence_files:
        if file.suffix in ['.txt', '.md', '.csv']:
            content = file.read_text()
            evidence_content.append(f"FILE: {file.name}\n{content[:2000]}")

    combined_evidence = "\n\n---\n\n".join(evidence_content)

    client = openai.OpenAI()

    response = client.chat.completions.create(
        model="gpt-4-turbo",
        messages=[
            {
                "role": "system",
                "content": """You are preparing audit evidence summaries. Create clear, 
                concise summaries that explain how the evidence demonstrates compliance 
                with the stated control. Use professional language suitable for auditors."""
            },
            {
                "role": "user",
                "content": f"""Summarize this evidence package for the following control:

                CONTROL: {control_description}

                EVIDENCE:
                {combined_evidence}

                Provide:
                1. Executive summary (2-3 sentences)
                2. Key evidence items and what they demonstrate
                3. Any gaps or weaknesses in the evidence
                4. Suggested additional evidence if needed"""
            }
        ],
        temperature=0.3
    )

    return response.choices[0].message.content

# Example usage
if __name__ == '__main__':
    summary = summarize_evidence(
        'evidence/AC-2_account_management/',
        'AC-2: The organization manages information system accounts'
    )
    print(summary)

Incident Classification

I automatically classify and route compliance incidents using AI.

import openai
import json

def classify_incident(incident_description):
    client = openai.OpenAI()

    response = client.chat.completions.create(
        model="gpt-4-turbo",
        messages=[
            {
                "role": "system",
                "content": """You are a compliance incident classifier. Analyze incident 
                descriptions and provide structured classification. Return JSON format only."""
            },
            {
                "role": "user",
                "content": f"""Classify this compliance incident:

                {incident_description}

                Return JSON with:
                - category: one of [data_breach, policy_violation, access_violation, 
                  vendor_incident, regulatory_inquiry, other]
                - severity: one of [critical, high, medium, low]
                - affected_regulations: list of potentially affected regulations 
                  (e.g., GDPR, HIPAA, SOX, PCI-DSS)
                - recommended_response_time: in hours
                - key_stakeholders: list of roles to notify
                - summary: one sentence summary"""
            }
        ],
        temperature=0.1,
        response_format={"type": "json_object"}
    )

    return json.loads(response.choices[0].message.content)

# Example usage
if __name__ == '__main__':
    incident = """
    Employee reported that they accidentally sent a spreadsheet containing 
    customer names, email addresses, and purchase history to an external 
    vendor not covered by our DPA. The spreadsheet contained approximately 
    500 customer records. The employee realized the mistake within 2 hours 
    and contacted the vendor to delete the file.
    """

    classification = classify_incident(incident)
    print(json.dumps(classification, indent=2))

Scheduling Automated Reports

I run compliance scripts automatically on schedule.

Using Python Schedule Library

import schedule
import time
from access_review import generate_access_review
from training_compliance import training_compliance_report

def run_daily_reports():
    print(f"Running daily compliance reports...")
    training_compliance_report()

def run_monthly_reports():
    print(f"Running monthly compliance reports...")
    generate_access_review()

# Schedule jobs
schedule.every().day.at("06:00").do(run_daily_reports)
schedule.every(30).days.at("06:00").do(run_monthly_reports)

# Run scheduler
while True:
    schedule.run_pending()
    time.sleep(60)

Using Windows Task Scheduler

For Windows systems, create a batch file:

@echo off
cd C:\compliance_automation
python access_review.py >> logs\access_review.log 2>&1

Schedule via Task Scheduler:

Open Task Scheduler
Create Basic Task
Set trigger (daily, weekly, monthly)
Action: Start a Program
Program: Path to batch file

Using Cron (Linux/Mac)

# Edit crontab
crontab -e

# Add scheduled jobs
# Daily training report at 6 AM
0 6 * * * cd /home/user/compliance && python training_compliance.py >> logs/training.log 2>&1

# Monthly access review on 1st at 6 AM
0 6 1 * * cd /home/user/compliance && python access_review.py >> logs/access.log 2>&1

Building a Compliance Dashboard

I aggregate automated reports into a simple dashboard.

import pandas as pd
from datetime import datetime
from pathlib import Path

def generate_compliance_dashboard():
    dashboard_data = {
        'metric': [],
        'value': [],
        'status': [],
        'last_updated': []
    }

    # Training compliance
    training_report = pd.read_excel(
        'reports/training_compliance_latest.xlsx',
        sheet_name='Department Summary'
    )
    overall_training = training_report['compliance_rate'].mean()
    dashboard_data['metric'].append('Training Compliance Rate')
    dashboard_data['value'].append(f'{overall_training:.1f}%')
    dashboard_data['status'].append('Green' if overall_training >= 95 else 'Yellow' if overall_training >= 85 else 'Red')
    dashboard_data['last_updated'].append(datetime.now().strftime('%Y-%m-%d'))

    # Access review
    access_report = pd.read_excel(
        'reports/access_review_latest.xlsx',
        sheet_name='Unauthorized Access'
    )
    unauthorized_count = len(access_report)
    dashboard_data['metric'].append('Unauthorized Access Findings')
    dashboard_data['value'].append(str(unauthorized_count))
    dashboard_data['status'].append('Green' if unauthorized_count == 0 else 'Yellow' if unauthorized_count < 5 else 'Red')
    dashboard_data['last_updated'].append(datetime.now().strftime('%Y-%m-%d'))

    # Vendor assessments
    vendor_report = pd.read_excel(
        'reports/vendor_status_latest.xlsx',
        sheet_name='Action Required'
    )
    overdue_vendors = len(vendor_report)
    dashboard_data['metric'].append('Overdue Vendor Assessments')
    dashboard_data['value'].append(str(overdue_vendors))
    dashboard_data['status'].append('Green' if overdue_vendors == 0 else 'Yellow' if overdue_vendors < 3 else 'Red')
    dashboard_data['last_updated'].append(datetime.now().strftime('%Y-%m-%d'))

    # Policy acknowledgments
    policy_report = pd.read_excel(
        'reports/policy_ack_latest.xlsx',
        sheet_name='Missing'
    )
    missing_acks = len(policy_report)
    dashboard_data['metric'].append('Missing Policy Acknowledgments')
    dashboard_data['value'].append(str(missing_acks))
    dashboard_data['status'].append('Green' if missing_acks == 0 else 'Yellow' if missing_acks < 10 else 'Red')
    dashboard_data['last_updated'].append(datetime.now().strftime('%Y-%m-%d'))

    # Create dashboard dataframe
    dashboard = pd.DataFrame(dashboard_data)

    # Generate HTML dashboard
    html = f"""
    <html>
    <head>
        <title>Compliance Dashboard</title>
        <style>
            body {{ font-family: Arial, sans-serif; margin: 40px; }}
            h1 {{ color: #333; }}
            table {{ border-collapse: collapse; width: 100%; }}
            th, td {{ border: 1px solid #ddd; padding: 12px; text-align: left; }}
            th {{ background-color: #4472C4; color: white; }}
            .Green {{ background-color: #92D050; }}
            .Yellow {{ background-color: #FFC000; }}
            .Red {{ background-color: #FF6B6B; }}
        </style>
    </head>
    <body>
        <h1>Compliance Dashboard</h1>
        <p>Generated: {datetime.now().strftime('%Y-%m-%d %H:%M')}</p>
        <table>
            <tr>
                <th>Metric</th>
                <th>Value</th>
                <th>Status</th>
                <th>Last Updated</th>
            </tr>
    """

    for _, row in dashboard.iterrows():
        html += f"""
            <tr>
                <td>{row['metric']}</td>
                <td>{row['value']}</td>
                <td class="{row['status']}">{row['status']}</td>
                <td>{row['last_updated']}</td>
            </tr>
        """

    html += """
        </table>
    </body>
    </html>
    """

    # Ensure dashboard directory exists
    dashboard_path = Path('dashboard')
    dashboard_path.mkdir(parents=True, exist_ok=True)

    dashboard_path.joinpath('compliance_dashboard.html').write_text(html)
    return dashboard

if __name__ == '__main__':
    print(generate_compliance_dashboard())

Implementation Roadmap

I recommend rolling out compliance automation incrementally.

Week 1-2: Foundation

Install Python and required libraries
Identify 2-3 highest-impact manual reports
Collect sample data files from source systems
Create folder structure for scripts and outputs

Week 3-4: First Automation

Build script for single report type
Test with real data
Refine output format based on stakeholder feedback
Document data sources and dependencies

Week 5-6: Expansion

Add second and third automated reports
Implement scheduling
Set up logging and error handling
Create basic dashboard

Week 7-8: AI Integration

Integrate AI for document analysis tasks
Test AI outputs for accuracy
Establish review process for AI-generated content
Document AI usage policies

Ongoing

Monitor automation reliability
Expand to additional report types
Refine based on audit feedback
Update as regulatory requirements change

Risk Considerations

I've found that automation introduces its own compliance considerations.

Data Handling:

Scripts may process sensitive data
Ensure appropriate access controls on automation systems
Log automation activities for audit trail

Output Accuracy:

Validate automated outputs initially
Maintain manual backup procedures
Document known limitations

AI Considerations:

Review AI outputs before distribution
Document AI usage in compliance processes
Consider data privacy implications of AI processing

Change Management:

Document automation logic
Version control scripts
Test changes before production deployment

In my experience, compliance automation transforms tedious reporting into reliable, consistent processes. The Python scripts and AI integrations I've shown here represent starting points that can be customized for specific regulatory requirements and organizational needs. Each hour I invest in automation returns many hours of reduced manual effort over subsequent reporting cycles.

DEV Community: Veduis

Hermes Agent: A Beginner's Guide to Getting It Running and Configured

Hermes Agent: A Beginner's Guide to Getting It Running and Configured

What Hermes Agent Actually Is

What You Need Before Starting

Step 1: Running the Installer

Step 2: Running the Setup Wizard

Understanding the Config Files

Step 3: Connecting Your API Key

OpenRouter (Recommended for Beginners)

Anthropic

OpenAI

Local Models via Ollama

Step 4: Picking a Model

Step 5: Starting Hermes

Classic CLI

Terminal UI (TUI) Mode

Step 6: Connecting to a Messaging Platform

Setting Up the Telegram Gateway

Keeping Hermes Running Persistently

Understanding Skills and Memory

Useful Commands to Know

Common Beginner Mistakes

A Good First Task to Try

Related Reading

Where to Go from Here

Conclusion

Multi-Region Architecture: Global Deployment, Data Residency, and Designing for Regional Failures

When Multi-Region Is Worth It

Architectural Patterns

Active-Passive

Active-Active

The Write Consistency Problem

Data Residency and Compliance

GDPR (EU)

Data Localization Laws

Traffic Routing

GeoDNS

Latency-Based Routing

Anycast

Deploying to Multiple Regions

Terraform with Multiple Providers

Testing Regional Failover

Cost Considerations

Conclusion

Data Consistency Models: From Strong Consistency to Eventual Consistency

The Consistency Spectrum

ACID: Strong Consistency in Practice

Atomicity

Consistency

Isolation

Durability

BASE: Accepting Weak Consistency

Basically Available

Soft State

Eventual Consistency

The CAP Theorem

The Practical Reality

Examples

PACELC: Extending CAP

Consistency Models in Detail

Linearizability (Strong Consistency)

Sequential Consistency

Causal Consistency

Session Consistency

Read-Your-Writes Consistency

Monotonic Reads

Eventual Consistency (Weakest)

Quorum-Based Consistency

Read and Write Quorums

Dynamo-Style Quorum (Sloppy Quorum)

Tunable Consistency

Consistency in Practice: Implementation Patterns

Optimistic Concurrency Control

Pessimistic Concurrency Control

Conflict-Free Replicated Data Types (CRDTs)

Vector Clocks

Read Repair and Anti-Entropy

Read Repair

Anti-Entropy Repair