DEV Community: Yevhen Shaforostov

Building an 84-Skill JIT Architecture for AI Agents (Without Context Degradation)

Yevhen Shaforostov — Mon, 31 Aug 2026 13:24:23 +0000

When building autonomous AI agents for large, real-world codebases (using Claude Code, LangGraph, or custom agent harnesses), developers almost immediately hit a wall: Context Window Decay.

As your system prompt accumulates business rules, database schemas, coding standards, and multi-file instructions (often climbing to 30k–50k+ tokens upfront), three critical problems arise:

Inference Latency Spikes: Every tool call and user query must process the massive static prompt overhead.
Attention Rot & Hallucinations: Large language models struggle to maintain strict adherence across 50,000 tokens of static rules, often hallucinating API parameters or ignoring negative constraints.
Escalating Token COGS: You pay for tens of thousands of unused instruction tokens on every single execution step.

To solve this in production, I designed a Just-in-Time (JIT) Modular Skill Architecture covering 84+ specialized production domains.

🏗️ The Architectural Pattern

Instead of dumping every instruction into a monolithic system prompt, we decouple instructions into lightweight domain bundles loaded strictly on demand.

How to Build a Production-Grade RAG Agent with Hybrid Search in 30 Minutes

Yevhen Shaforostov — Sat, 29 Aug 2026 16:46:20 +0000

Most Retrieval-Augmented Generation (RAG) tutorials stop at a naive vector lookup: embed text with OpenAI, store it in Pinecone or Chroma, and perform cosine similarity search.
In production environments, this naive approach quickly fails. Vector embeddings excel at semantic similarity, but they consistently stumble on exact keywords, acronyms, product SKUs, UUIDs, and domain-specific error codes.
To build an enterprise-grade RAG agent, you need Hybrid Search with Reciprocal Rank Fusion (RRF) combined with a deterministic multi-agent harness.

In this guide, we will construct a production RAG system in under 30 minutes using PostgreSQL (`pgvector`), Reciprocal Rank Fusion (RRF), and LangGraph.

1. Why Pure Vector Search Fails in Production

Consider a developer searching an enterprise knowledge base for:
"Fix CVE-2024-38077 Windows Netlogon RPC buffer overflow"

Dense Vector Search: Understands the general concept of "Windows vulnerabilities" and returns general security advisories, often missing the exact patch document.
Sparse Keyword Search (BM25): Finds the exact token "CVE-2024-38077", but misses related contextual documentation that uses synonyms like "Netlogon remote elevation vulnerability". ### The Solution: Hybrid Search + Reciprocal Rank Fusion (RRF) Hybrid search executes both retrieval pipelines concurrently and merges the candidate rankings using the mathematical RRF formula: $$\text{RRF Score}(d) = \sum_{m \in M} \frac{1}{60 + r_m(d)}$$ Where:
$r_m(d)$ is the rank of document $d$ in retrieval method $m$ (Dense vs Sparse).

- $60$ is a smoothing constant that prevents top-heavy outliers from dominating the result set.

2. PostgreSQL Schema Setup (`pgvector` + `tsvector`)

You don't need a separate vector database. PostgreSQL handles dense vector embeddings and sparse full-text search within a single atomic ACID transaction.


sql
-- Enable vector extension
CREATE EXTENSION IF NOT EXISTS vector;
-- Create enterprise knowledge documents table
CREATE TABLE enterprise_documents (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id VARCHAR(64) NOT NULL,
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    search_vector TSVECTOR GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || content)) STORED,
    embedding VECTOR(1536), -- Compatible with text-embedding-3-small
    metadata JSONB DEFAULT '{}'::jsonb,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Fast approximate nearest neighbor index (HNSW)
CREATE INDEX ON enterprise_documents USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
-- High-performance GIN index for sparse full-text search
CREATE INDEX ON enterprise_documents USING gin(search_vector);
3. The SQL Reciprocal Rank Fusion (RRF) Query
Here is the single SQL query that executes both dense and sparse retrieval in parallel and merges them via RRF:

sql
WITH dense_candidates AS (
    SELECT id, content, metadata,
           rank() OVER (ORDER BY embedding <=> $1) as d_rank
    FROM enterprise_documents
    WHERE tenant_id = $3
    LIMIT 50
),
sparse_candidates AS (
    SELECT id, content, metadata,
           rank() OVER (ORDER BY ts_rank_cd(search_vector, plainto_tsquery($2)) DESC) as s_rank
    FROM enterprise_documents
    WHERE tenant_id = $3 AND search_vector @@ plainto_tsquery($2)
    LIMIT 50
)
SELECT 
    COALESCE(d.id, s.id) as id,
    COALESCE(d.content, s.content) as content,
    COALESCE(d.metadata, s.metadata) as metadata,
    (COALESCE(1.0 / (60 + d.d_rank), 0.0) + COALESCE(1.0 / (60 + s.s_rank), 0.0)) as fusion_score
FROM dense_candidates d
FULL OUTER JOIN sparse_candidates s ON d.id = s.id
ORDER BY fusion_score DESC
LIMIT 10;
4. Connecting the Retrieval Engine to LangGraph
Now we wrap the hybrid search function inside an autonomous LangGraph Supervisor Agent that can analyze the retrieved context and verify facts before generating an answer.

typescript
import { StateGraph, END, START } from '@langchain/langgraph';
import { HybridSearchEngine } from './tools/hybridSearch';
export async function createRAGAgent() {
  const searchEngine = new HybridSearchEngine(process.env.DATABASE_URL!);
  const workflow = new StateGraph({
    // Define state channels
    channels: {
      query: { value: (x, y) => y ?? x, default: () => '' },
      retrievedContext: { value: (x, y) => y ?? x, default: () => [] },
      finalAnswer: { value: (x, y) => y ?? x, default: () => '' },
    }
  });
  // Step 1: Hybrid Retrieval Node
  workflow.addNode('retrieve', async (state) => {
    const embedding = await generateEmbedding(state.query);
    const results = await searchEngine.search(state.query, embedding, 5);
    return { retrievedContext: results };
  });
  // Step 2: Answer Generation Node (Claude 3.5 Sonnet)
  workflow.addNode('synthesize', async (state) => {
    const answer = await generateGroundedAnswer(state.query, state.retrievedContext);
    return { finalAnswer: answer };
  });
  workflow.addEdge(START, 'retrieve');
  workflow.addEdge('retrieve', 'synthesize');
  workflow.addEdge('synthesize', END);
  return workflow.compile();
}
5. Architectural Checklist for Production RAG
Before rolling this out to production users, enforce these 4 guardrails:

Strict Multi-Tenant Row-Level Security (RLS): Ensure tenant IDs are parameterized at the DB connection level.
Context Window Truncation: Dynamically budget token usage with tiktoken to prevent context overflow.
Cross-Encoder Re-ranking: For legal/medical data, add a secondary cross-encoder re-ranking pass (cohere.rerank or bge-reranker-large).
Hallucination Tripwires: Measure output faithfulness against retrieved <context> chunks using an automated LLM-as-a-Judge pass.
🚀 Complete Production Starter Kits & Resources
If you are building autonomous AI agents or production RAG systems, explore our battle-tested templates:

⭐️ Open-Source Claude Skills Starter Kit — Free on GitHub
📦 LangGraph Multi-Agent Production Starter Kit ($29) — Full TypeScript + Python code with Redis & pgvector memory
📋 Claude Code & Agent Prompt Templates Pack ($19) — 50 battle-tested production prompts
🛠️ Claude AI Engineering Skills Pack — 84 Skills ($49) — (Use coupon LAUNCH20 for 20% off)
How are you currently handling hybrid search and agent orchestration in your stack? Let's discuss in the comments!

5 Claude Code Skills That Saved Me 10 Hours a Week (With Real Code & Architecture)

Yevhen Shaforostov — Sat, 29 Aug 2026 16:19:43 +0000

When building software with Claude Code and autonomous agent harnesses, relying solely on generic conversational prompts quickly creates bottlenecks. As codebases scale, agents without structured instructions tend to make breaking changes, skip edge-case testing, or hallucinate non-existent API parameters.
To turn Claude Code into a production-ready engineering partner, I developed a modular architecture of custom agent skills. Each skill acts as a focused operational manual with strict constraints, schema validations, and verification loops.

Here are the 5 foundational skills that save me over 10 hours every week in production engineering—complete with real architectural blueprints and implementation code.

1. `tdd-workflow` — Enforcing Red-Green-Refactor Loops

Autonomous agents often rush to write application logic first, leading to subtle regressions and untracked edge cases. The tdd-workflow skill forces the agent to follow a strict 3-phase Red-Green-Refactor loop.

How the Workflow Operates:

Red Phase: Claude creates test files under tests/ and executes the test runner (vitest, pytest, or bun test) to confirm failure before writing any business logic.
Green Phase: The agent writes the minimal implementation code necessary to turn all tests green.
Refactor Phase: Performs AST-level cleanup, eliminates duplicate abstractions, and ensures 80%+ branch coverage.


typescript
// Example: Test-first contract generated by the tdd-workflow skill
import { describe, it, expect, vi } from 'vitest';
import { executeAgentTask } from '../src/agentRunner';
describe('executeAgentTask', () => {
  it('should enforce idempotency and reject duplicate task IDs', async () => {
    const taskId = 'task-uuid-1234';
    const payload = { type: 'RAG_QUERY', query: 'PostgreSQL HNSW tuning' };
    const firstRun = await executeAgentTask(taskId, payload);
    expect(firstRun.status).toBe('SUCCESS');
    // Duplicate execution with same idempotency key must be rejected
    await expect(executeAgentTask(taskId, payload)).rejects.toThrow('DuplicateTaskIdError');
  });
});
2. api-and-interface-design — Schema-First Contract Stability
Before introducing REST endpoints, tRPC routers, or MCP remote tools, this skill parses existing type definitions and Zod/Pydantic schemas. It prevents breaking public interface contracts and enforces strict runtime validation.

Strict Schema Definition:
typescript
import { z } from 'zod';
export const AgentExecutionRequestSchema = z.object({
  executionId: z.string().uuid(),
  tenantId: z.string().min(3),
  actionType: z.enum(['HYBRID_SEARCH', 'DATA_EXTRACT', 'DOCUMENT_OCR']),
  parameters: z.record(z.unknown()),
  idempotencyKey: z.string().min(16),
  maxBudgetUsd: z.number().positive().max(5.0),
});
export type AgentExecutionRequest = z.infer<typeof AgentExecutionRequestSchema>;
3. doubt-driven-development — Adversarial Self-Review
Before marking any task as complete or creating a pull request, this skill triggers an internal adversarial review pass. It stress-tests the code for silent failure modes:

Resource Leaks: Are database connections, file handles, and stream readers properly released in finally blocks?
Concurrency Hazards: Are there race conditions during state updates?
Security Boundaries: Does user input pass through strict sanitization before reaching SQL queries or shell executions?
4. hybrid-rag-vector-search — Production RAG Architecture
Standard vector search frequently misses exact keyword queries (SKUs, UUIDs, error codes, domain terminology). This skill provides the blueprint for combining dense semantic embeddings with sparse BM25 full-text search using Reciprocal Rank Fusion (RRF) inside PostgreSQL (pgvector).

sql
-- Hybrid Vector Search with Reciprocal Rank Fusion (RRF)
WITH dense_search AS (
  SELECT id, rank() OVER (ORDER BY embedding <=> $1) as r_rank
  FROM enterprise_documents
  LIMIT 50
),
sparse_search AS (
  SELECT id, rank() OVER (ORDER BY ts_rank_cd(search_vector, plainto_tsquery($2)) DESC) as k_rank
  FROM enterprise_documents
  WHERE search_vector @@ plainto_tsquery($2)
  LIMIT 50
)
SELECT 
  COALESCE(d.id, s.id) as id,
  COALESCE(1.0 / (60 + d.r_rank), 0.0) + COALESCE(1.0 / (60 + s.k_rank), 0.0) as fusion_score
FROM dense_search d
FULL OUTER JOIN sparse_search s ON d.id = s.id
ORDER BY fusion_score DESC
LIMIT 10;
5. observability-and-instrumentation — Real-Time Agent Telemetry
This skill automatically instruments every LLM execution step with structured JSON logging, token cost metering, and latency tracking.

Sample Structured Log Output:
json
{
  "timestamp": "2026-08-29T16:15:30.120Z",
  "level": "INFO",
  "agent_id": "langgraph-lead-qualifier",
  "step": "TOOL_EXECUTION",
  "tool_name": "pgvector_hybrid_search",
  "latency_ms": 42,
  "tokens_in": 1240,
  "tokens_out": 380,
  "cost_usd": 0.0048,
  "status": "SUCCESS"
}
🚀 Get Started (Open Source & Full Pack)
⭐️ Open-Source Starter Kit (5 Foundation Skills): Grab the open-source repository on GitHub at github.com/yevhens-hue/claude-skills-starter-kit
📦 Complete 84-Skill Production Engineering Pack: Access the full collection on Gumroad at shaforostov5.gumroad.com/l/njzfuo (Use discount code LAUNCH20 for 20% off).
What custom skills or architectural guardrails are you using in your daily agent workflows? Share your setup in the comments!

How I Architected 84 Custom Skills for Claude Code to Automate My Daily Engineering

Yevhen Shaforostov — Fri, 28 Aug 2026 22:28:35 +0000

I've been using Anthropic's Claude Code and autonomous coding agents in production daily as an AI Product Manager & Full-Stack AI Engineer at a B2B SaaS platform.

Like many engineers, my initial workflow consisted of pasting massive, 50-line system prompts into every new session:

"Remember to enforce strict TDD with 80%+ coverage..."
"When designing REST endpoints, enforce idempotency keys and RFC 7807 error schemas..."
"When evaluating AI agent failures, follow systematic failure injection rather than trial-and-error..."

This approach quickly broke down. Monolithic prompts lead to context window degradation, token bloat, and subtle hallucinations where the LLM forgets critical constraints halfway through a refactor.

To solve this, I designed a Modular Skill Architecture. Today, I want to break down how it works, the anatomy of a skill file, and how you can implement this in your own projects.

🧠 The Architecture: Just-in-Time Context Loading

Instead of loading all domain rules at once, we partition our engineering knowledge into isolated, self-contained Domain Skills stored in ~/.claude/skills/ (or ~/.gemini/config/skills/).

Each skill contains:

YAML Frontmatter: Defines the skill name, trigger keywords, and domain description.
Markdown Instruction Body: Specifies deterministic workflows, safety guardrails, and validation protocols.

When Claude detects you are tackling a specific domain (e.g., writing unit tests or debugging an agent), it dynamically indexes and injects only the necessary skill into active memory.