DEV Community: Stokry

In a world where AI can fake anything....

Stokry — Wed, 04 Feb 2026 08:01:39 +0000

In a world where AI can fake anything...

How do you prove what's real?

We're building the answer.

Coming soon.

Semantic Caching for RubyLLM: Cut Your AI Costs by 70%

Stokry — Tue, 03 Feb 2026 07:39:36 +0000

If you're using RubyLLM to build AI-powered applications in Ruby, you're already enjoying a clean, unified API across multiple providers. But there's one problem RubyLLM doesn't solve: every API call costs money.

Enter SemanticCache — a semantic caching layer that integrates seamlessly with RubyLLM to dramatically reduce your AI costs.

The Problem: Redundant API Calls

Consider a typical chatbot or AI assistant. Users often ask variations of the same question:

"What's the capital of France?"
"What is France's capital city?"
"Tell me the capital of France"
"Capital of France?"

Without caching, each of these triggers a separate API call. With traditional caching, you'd need an exact string match — which almost never happens with natural language.

Semantic caching solves this. It understands that these questions are semantically identical and returns the cached response instantly.

Why SemanticCache + RubyLLM?

1. Provider Flexibility

RubyLLM already gives you the freedom to switch between OpenAI, Gemini, Mistral, Ollama, Bedrock, and more. SemanticCache extends this flexibility to embeddings:

# Use Gemini for embeddings (cheaper) while using GPT-4 for completions
RubyLLM.configure do |config|
  config.gemini_api_key = ENV["GEMINI_API_KEY"]
  config.openai_api_key = ENV["OPENAI_API_KEY"]
end

SemanticCache.configure do |c|
  c.embedding_adapter = :ruby_llm
  c.embedding_model = "text-embedding-004"  # Gemini's embedding model
end

# Now cache expensive GPT-4 calls using cheap Gemini embeddings
cache = SemanticCache.new
response = cache.fetch("Explain quantum computing") do
  RubyLLM.chat(model: "gpt-4o", messages: [{ role: "user", content: "Explain quantum computing" }])
end

2. Local Embeddings with Ollama

Running Ollama locally? You can generate embeddings without any API costs:

RubyLLM.configure do |config|
  config.ollama_api_base = "http://localhost:11434/v1"
end

SemanticCache.configure do |c|
  c.embedding_adapter = :ruby_llm
  c.embedding_model = "nomic-embed-text"  # Free, local embeddings
end

This is perfect for:

Development environments (no API costs while testing)
Privacy-sensitive applications (embeddings never leave your server)
High-volume applications where embedding costs add up

3. Enterprise-Ready with AWS Bedrock

For enterprise deployments, you might need to keep everything within AWS:

RubyLLM.configure do |config|
  config.bedrock_region = "us-east-1"
end

SemanticCache.configure do |c|
  c.embedding_adapter = :ruby_llm
  c.embedding_model = "amazon.titan-embed-text-v1"
end

No data leaves your AWS environment, and you get the compliance benefits of Bedrock.

Real-World Impact

Let's do some quick math. Assume:

10,000 queries per day
Average query: 50 tokens
GPT-4o pricing: ~$5/1M input tokens, ~$15/1M output tokens
Average response: 200 tokens

Without caching:

Daily cost: ~$35/day
Monthly cost: ~$1,050/month

With SemanticCache (assuming 70% hit rate):

Daily cost: ~$10.50/day + minimal embedding costs
Monthly cost: ~$315/month
Savings: $735/month

And that's a conservative estimate. Applications with repetitive queries (FAQ bots, customer support, documentation assistants) often see 80-90% hit rates.

How It Works

Query comes in → SemanticCache generates an embedding vector using your configured RubyLLM provider
Similarity search → The cache searches for stored entries with high cosine similarity
Cache hit? → If similarity exceeds your threshold (default 0.85), return the cached response
Cache miss? → Execute the LLM call, store the result with its embedding for future queries

cache = SemanticCache.new(similarity_threshold: 0.85)

# First call - cache miss, calls the API
response = cache.fetch("What is Ruby?") do
  RubyLLM.chat(messages: [{ role: "user", content: "What is Ruby?" }])
end

# Second call - cache hit! Returns instantly
response = cache.fetch("Tell me about the Ruby programming language") do
  RubyLLM.chat(messages: [{ role: "user", content: "Tell me about Ruby" }])
end
# => Returns cached response, no API call made

Getting Started

Add both gems to your Gemfile:

gem "ruby_llm"
gem "semantic-cache"

Configure your providers:

# config/initializers/ruby_llm.rb
RubyLLM.configure do |config|
  config.openai_api_key = ENV["OPENAI_API_KEY"]
end

# config/initializers/semantic_cache.rb
SemanticCache.configure do |c|
  c.embedding_adapter = :ruby_llm
  c.embedding_model = "text-embedding-3-small"
  c.similarity_threshold = 0.85
  c.store = :redis  # For production
  c.store_options = { url: ENV["REDIS_URL"] }
end

Start caching:

cache = SemanticCache.new

response = cache.fetch(user_message, model: "gpt-4o") do
  RubyLLM.chat(
    model: "gpt-4o",
    messages: [{ role: "user", content: user_message }]
  )
end

# Check your savings
puts cache.savings_report
# => Total saved: $23.45 (156 cached calls)

Advanced Patterns

Tag-Based Invalidation

Group related cache entries for bulk invalidation:

# Cache with tags
cache.fetch("Ruby version?", tags: [:ruby, :versions]) do
  RubyLLM.chat(messages: [{ role: "user", content: "What's the latest Ruby version?" }])
end

# When Ruby 3.4 releases, invalidate all version-related caches
cache.invalidate(tags: [:versions])

TTL for Time-Sensitive Data

# News summaries should expire quickly
cache.fetch("Latest tech news?", ttl: 3600) do  # 1 hour
  RubyLLM.chat(messages: [{ role: "user", content: "Summarize today's tech news" }])
end

Per-User Namespacing

# Each user gets their own cache namespace
SemanticCache.with_cache(namespace: "user_#{current_user.id}") do
  # Cached responses are isolated per user
end

Conclusion

SemanticCache and RubyLLM are a natural fit. RubyLLM gives you provider flexibility for LLM calls; SemanticCache extends that flexibility to embeddings while dramatically cutting your costs.

The combination is particularly powerful because:

No vendor lock-in — Switch embedding providers without changing your caching logic
Cost optimization — Use cheaper providers for embeddings, premium providers for completions
Local development — Use Ollama for free local embeddings during development
Enterprise compliance — Keep everything within AWS using Bedrock

Stop paying for the same answers twice. Add SemanticCache to your RubyLLM project today.

Links:

Why Ruby is Excellent for AI Development

Stokry — Thu, 29 Jan 2026 08:55:53 +0000

Ruby gets overlooked in AI discussions, but developers who use it know it offers a unique combination of productivity and elegance that makes it an excellent choice for many AI projects.

The Hidden Token Efficiency Advantage

Here's something most developers miss: Ruby is ~40% more token-efficient than Python for equivalent code. In the LLM era where context windows and API costs matter, this is a massive advantage.

Real-world measurements:

Ruby class: ~83 tokens vs Python: ~140 tokens
Rails controller: ~134 tokens vs FastAPI endpoint: ~216 tokens
Rails model: ~90 tokens vs Django model: ~199 tokens
JavaScript consistently uses the most tokens

Why Ruby wins on token efficiency:

Less boilerplate - no self., no __init__, no def __init__(self)
Ruby blocks are more concise than Python comprehensions
Rails DSL is brutally efficient (validations, scopes, associations)
Symbols (:email) vs strings ('email')
Idiomatic Ruby is naturally more compact

What this means for AI/LLM development:

Fit ~40% more code in the same context window
Cheaper API calls (fewer input tokens)
Faster generation (fewer output tokens)
More context available for reasoning and examples

Expressiveness That Accelerates Experimentation

Ruby's "optimized for developer happiness" philosophy pays dividends in AI development where rapid iteration makes the difference. Clean syntax lets you focus on algorithms instead of boilerplate.

Production-Ready AI Stack

The ecosystem is mature. Rumale brings scikit-learn quality - SVMs, Random Forests, neural networks, all optimized with C extensions. TensorStream runs TensorFlow models directly from Ruby. Numo/NArray offers NumPy-like performance for numerical operations. LangChain.rb provides idiomatic Ruby access to LLMs. ruby-openai gem makes API integration trivial.

The Rails Advantage for AI Products

The biggest secret: Ruby on Rails is killer for AI applications. While Python devs cobble together Flask/FastAPI with a frontend framework, Rails developers ship complete AI apps with authentication, background jobs, and real-time updates in 30% less code. Hotwire/Turbo enables interactive AI interfaces without JavaScript complexity.

Real Production Examples

GitHub uses Ruby for ML-based code suggestions. Shopify runs AI recommendations in Rails apps processing billions of transactions. GitLab integrates AI features directly into their Rails monolith. Discourse uses Ruby for spam detection and content moderation AI.

Practical Code Example

require 'openai'
require 'vectra'

class AIAssistant
  def initialize
    @client = OpenAI::Client.new
    @db = Vectra.new(path: "embeddings.json")
  end

  def answer(question)
    embedding = get_embedding(question)
    context = @db.query(embedding, k: 3)

    @client.chat(
      parameters: {
        model: "gpt-4",
        messages: [
          {role: "system", content: build_context(context)},
          {role: "user", content: question}
        ]
      }
    )
  end

  private

  def get_embedding(text)
    response = @client.embeddings(
      parameters: { model: "text-embedding-3-small", input: text }
    )
    response.dig("data", 0, "embedding")
  end

  def build_context(results)
    results.map { |r| r[:metadata][:text] }.join("\n\n")
  end
end

Clean. Readable. Production-ready. And 40% fewer tokens than the Python equivalent.

Performance Reality Check

Yes, Python has faster numerical libraries. But most AI apps spend 90% of time waiting on API calls or GPU operations. Ruby's "slow" execution doesn't matter when OpenAI API latency is 500ms. For heavy compute, call out to C extensions or microservices.

The Underrated Benefits

Metaprogramming: DSLs for defining neural network architectures feel natural in Ruby. Concurrency: Ractors and async gems handle parallel API calls elegantly. Testing culture: RSpec makes testing AI behavior straightforward. Gem ecosystem: 175,000+ gems mean you're never starting from scratch. Token efficiency: Ship more functionality in fewer tokens.

When Ruby Makes Sense

Ruby excels for: AI-powered web apps, RAG applications, LLM orchestration, chatbots, content generation tools, AI automation scripts, and rapid prototyping.

Ruby struggles with: Training large models from scratch, real-time computer vision, and hardcore numerical optimization (use Python/Julia for these).

The Bottom Line

Ruby won't replace Python for research or deep learning. But for building and shipping AI products - especially web applications with AI features - Ruby offers a compelling developer experience that deserves serious consideration.

The token efficiency advantage alone makes Ruby worth evaluating for any project involving LLMs. When you're paying per token and fighting context window limits, 40% efficiency gains aren't just nice to have - they're a competitive advantage.

The AI gold rush needs builders who ship fast. Ruby gives you that superpower.

Rack-style middleware for vector databases in Ruby: vectra-client 1.1.0

Stokry — Thu, 15 Jan 2026 09:04:03 +0000

If you’re building serious AI features in Ruby (semantic search, RAG, recommendations), you quickly realize you don’t just need a client for your vector database — you need a production toolkit.

With vectra-client 1.1.0, the gem gets exactly that: Rack-style middleware for vector databases plus production patterns Ruby/Rails developers already know.

Why this matters

Vectra is aiming to be the most complete production toolkit in the Ruby vector DB ecosystem:

Production patterns: 7+ built-in patterns (batch upsert, health check, ActiveRecord integration, reindex, caching…)
Middleware: 5 built-in middlewares (the only gem with this approach)
Multi-provider: 5 providers behind a unified API (:pinecone, :qdrant, :weaviate, :pgvector, :memory)

The core idea: everything you already know from Rack/Faraday land, now applied to AI / vector workloads.

Positioning:

“The only Ruby gem with Rack-style middleware for vector databases — production patterns you already know, applied to AI workloads.”

What’s new in vectra-client 1.1.0?

1. Rack-style middleware stack for all operations

All public operations on Vectra::Client now go through a middleware stack:

upsert, query, fetch, update, delete
list_indexes, describe_index, stats
create_index, delete_index, list_namespaces

Which means you can:

add logging / retry / instrumentation globally
add per-client middleware (e.g. PII redaction only for a specific index)

require "vectra-client"

# Global middleware
Vectra::Client.use Vectra::Middleware::Logging
Vectra::Client.use Vectra::Middleware::Retry, max_attempts: 3
Vectra::Client.use Vectra::Middleware::CostTracker

client = Vectra::Client.new(
  provider: :qdrant,
  index: "products",
  namespace: "tenant-1",
  middleware: [
    Vectra::Middleware::PIIRedaction,
    Vectra::Middleware::Instrumentation
  ]
)

Every call (client.upsert, client.query, …) now flows through the same, consistent pipeline.

2. Built-in middleware (production-ready)

Version 1.1.0 ships with 5 ready-to-use middlewares:

Logging – structured logs for all operations + duration in ms
Retry – retry logic for rate limit / network / timeout errors
Instrumentation – hooks for metrics and APM (Datadog, New Relic, Prometheus…)
PIIRedaction – PII redaction in metadata before sending to the vector DB
CostTracker – simple cost estimation per operation (per provider, per vector)

The point: you can add observability and safety without forking the gem.

3. A client that understands “context”

Vectra::Client now has:

default index & namespace (via index: and namespace: on init)
helpers:

client.with_index("shadow-products") do |ctx|
  ctx.upsert(vectors: batch)
end

client.with_namespace("tenant-42") do |ctx|
  ctx.query(vector: embedding, top_k: 10)
end

client.with_index_and_namespace("products", "tenant-1") do |ctx|
  ctx.delete(ids: ["products_123"])
end

This makes multi-tenant, shadow indexes, migrations and A/B testing much easier without duplicating a ton of boilerplate.

Using this in a Rails app

Fastest path:

Add the gem:

gem "vectra-client"

Run the install generator:

rails generate vectra:install

Run the index generator for your model (e.g. Product):

rails generate vectra:index Product embedding dimension:1536 provider:qdrant

Include the generated concern and use has_vector + the reindex_vectors helper:

class Product < ApplicationRecord
  include ProductVector
end

Now you can:

automatically index embeddings after save
run semantic search via Product.vector_search(...)
reindex the entire table via Product.reindex_vectors

When does vectra-client make sense?

If you’re building:

e‑commerce search (semantic + filters)
blog / docs search (hybrid: keyword + vectors)
RAG chatbot pulling context from PostgreSQL + pgvector
multi-tenant SaaS with namespace isolation
recommendation systems (similar products / articles)

…then production features (middleware, health check, batch, reindex, provider-agnostic API) matter a lot more than a barebones “client”.

Wrap-up

The Ruby ecosystem finally gets a serious, production-grade toolkit for working with vector databases:

Rack-style middleware you already understand
production patterns mapped onto AI workloads
a multi-provider API that doesn’t lock you into a single vendor

If you want to see it in action, check out:

If you’d like an example for a specific stack (Sidekiq, Hanami, dry-rb, …), open an issue — this middleware layer is designed exactly for those kinds of extensions.

GitHub repo: stokry/vectra

Building Enterprise Vector Search in Rails (Part 2/3): Production Resilience & Monitoring

Stokry — Tue, 13 Jan 2026 14:03:54 +0000

*This is Part 2 of a 3-part series* on building production-ready vector search for enterprise SaaS.

Part 1: Architecture & Implementation - Multi-tenant document processing

*Part 2: Production Resilience & Monitoring* 👈 You are here

Part 3: Cost Optimization & Lessons Learned (Coming Friday)

*TL;DR*: Production-ready means more than just working code. This part covers circuit breakers, rate limiting, health checks, and monitoring that keep the system running through Black Friday traffic spikes and Qdrant outages.

The Black Friday Incident 🚨

*Date:* November 24, 2023, 2:47 PM EST

Our monitoring dashboard lit up red:


⚠️  Qdrant CPU: 98%

⚠️  Memory: 95%

⚠️  Query latency: 12,000ms (P95)

🔥 Error rate: 45%

*What happened:* A major client launched their compliance platform to 5,000 users simultaneously. Search traffic spiked from 800 req/min → 4,200 req/min. Our Qdrant cluster couldn't handle it.

*Without circuit breakers:* We would have:

Hammered the failing Qdrant cluster
Exhausted connection pools
Brought down the entire Rails app
100% outage for all 150 clients

*With circuit breakers:*

Detected 5 failures in 30 seconds
Opened the circuit
Served cached results (42% hit rate)
Fell back to PostgreSQL full-text search
*99.2% of searches still worked*
MTTR: 12 minutes

Let me show you how we built this resilience.

Part 1: Circuit Breaker Pattern

The Implementation


# lib/circuit_breaker_registry.rb

class CircuitBreakerRegistry

@breakers = Concurrent::Map.new



def self.for_tenant(tenant_id)

@breakers.compute_if_absent("tenant_#{tenant_id}") do

Vectra::CircuitBreaker.new(

name: "tenant_#{tenant_id}",

failure_threshold: 5,  # Open after 5 failures

recovery_timeout: 60,  # Try again after 60s

success_threshold: 3 # Close after 3 successes

)

end

end



def self.stats

@breakers.each_pair.map do |name, breaker|

[name, breaker.stats]

end.to_h

end



def self.reset_all!

@breakers.each_value(&:reset!)

end

end

Using Circuit Breakers in Vector Search


# app/services/vector_indexing_service.rb

class VectorIndexingService

def search(tenant_id:, query:, filters: {}, limit: 20)

query_embedding = EmbeddingService.new.generate(query)

namespace = namespace_for_tenant(tenant_id)



# Get circuit breaker for this tenant

breaker = CircuitBreakerRegistry.for_tenant(tenant_id)



# Execute with fallback

breaker.call(fallback: -> { fallback_search(tenant_id, query, limit) }) do

@client.query(

index: 'compliance_documents',

vector: query_embedding,

top_k: limit,

namespace: namespace,

filter: filters,

include_metadata: true

)

end

end



private



def fallback_search(tenant_id, query, limit)

# Fallback to PostgreSQL full-text search

Document.where(tenant_id: tenant_id)

.where("to_tsvector('english', title || ' ' || content) @@ plainto_tsquery(?)", query)

.limit(limit)

.map { |doc| convert_to_vector_result(doc) }

end



def convert_to_vector_result(doc)

# Convert Document to QueryResult format

OpenStruct.new(

id: doc.id,

score: 0.5,  # Fallback score

metadata: {

'document_id' => doc.id,

'tenant_id' => doc.tenant_id,

'title' => doc.title,

'chunk_text' => doc.content[0..500]

}

)

end

end

Circuit Breaker States


┌──────────┐

│  CLOSED  │  ← Normal operation (requests pass through)

└──────────┘

│

│ (5 failures detected)

↓

┌──────────┐

│ OPEN │  ← Requests fail immediately, use fallback

└──────────┘

│

│ (60 seconds elapsed)

↓

┌──────────┐

│ HALF_OPEN│  ← Limited requests allowed to test recovery

└──────────┘

│

│ (3 successes) → Back to CLOSED

│ (1 failure) → Back to OPEN

Monitoring Circuit Breaker States


# app/controllers/admin/circuit_breakers_controller.rb

module Admin

class CircuitBreakersController < AdminController

def index

@breakers = CircuitBreakerRegistry.stats



render json: {

breakers: @breakers.map do |name, stats|

{

name: name,

state: stats[:state],

failures: stats[:failures],

successes: stats[:successes],

last_failure_at: stats[:last_failure_at],

opened_at: stats[:opened_at]

}

end,

summary: {

total: @breakers.size,

open: @breakers.count { |_, s| s[:state] == :open },

half_open: @breakers.count { |_, s| s[:state] == :half_open }

}

}

end



def reset

CircuitBreakerRegistry.reset_all!

render json: { message: 'All circuit breakers reset' }

end

end

end

Rate Limiting (Per-Tenant)

Why Per-Tenant Rate Limiting?

*Problem:* A single tenant running a poorly-configured script could DOS the entire platform.

*Solution:* Isolate rate limits per tenant.


# lib/rate_limiter_registry.rb

class RateLimiterRegistry

@limiters = Concurrent::Map.new



def self.for_tenant(tenant_id)

@limiters.compute_if_absent("tenant_#{tenant_id}") do

# 500 requests per minute per tenant

Vectra::RateLimiter.new(

requests_per_second: 8.33,  # 500/60

burst_size: 20

)

end

end



def self.stats

@limiters.each_pair.map do |name, limiter|

[name, limiter.stats]

end.to_h

end

end

Using Rate Limiting


# app/services/vector_indexing_service.rb

def search(tenant_id:, query:, filters: {}, limit: 20)

# Get rate limiter for this tenant

limiter = RateLimiterRegistry.for_tenant(tenant_id)



# Acquire rate limit token

limiter.acquire do

# Get circuit breaker

breaker = CircuitBreakerRegistry.for_tenant(tenant_id)



# Execute search with resilience

breaker.call(fallback: -> { fallback_search(tenant_id, query, limit) }) do

perform_vector_search(query, filters, limit, tenant_id)

end

end

rescue Vectra::RateLimitError => e

# Return 429 Too Many Requests

raise SearchRateLimitExceeded.new(

tenant_id: tenant_id,

retry_after: limiter.retry_after

)

end

Rate Limit Headers


# app/controllers/api/v1/search_controller.rb

def create

limiter = RateLimiterRegistry.for_tenant(current_tenant.id)



# Add rate limit headers

response.headers['X-RateLimit-Limit'] = '500'

response.headers['X-RateLimit-Remaining'] = limiter.remaining.to_s

response.headers['X-RateLimit-Reset'] = limiter.reset_at.to_i.to_s



# ... search logic ...

rescue SearchRateLimitExceeded => e

response.headers['Retry-After'] = e.retry_after.to_s

render json: {

error: 'Rate limit exceeded',

retry_after: e.retry_after

}, status: :too_many_requests

end

Health Checks

Comprehensive Health Check Endpoint


# app/controllers/health_controller.rb

class HealthController < ApplicationController

skip_before_action :authenticate_user!



def show

health = {

status: 'healthy',

timestamp: Time.current.iso8601,

version: Vectra::VERSION,

services: {}

}



# Check Vector DB

begin

vectra_client = Vectra.qdrant(host: ENV['QDRANT_HOST'])

vectra_health = vectra_client.health_check



health[:services][:vector_db] = {

status: vectra_health.healthy? ? 'up' : 'down',

latency_ms: vectra_health.latency_ms,

provider: vectra_health.provider,

indexes: vectra_health.indexes_available

}

rescue StandardError => e

health[:services][:vector_db] = {

status: 'down',

error: e.message

}

health[:status] = 'degraded'

end



# Check Database

begin

start = Time.current

ActiveRecord::Base.connection.execute('SELECT 1')

db_latency = ((Time.current - start) * 1000).round(2)



health[:services][:database] = {

status: 'up',

latency_ms: db_latency

}

rescue StandardError => e

health[:services][:database] = {

status: 'down',

error: e.message

}

health[:status] = 'unhealthy'

end



# Check Redis

begin

start = Time.current

Rails.cache.redis.ping

redis_latency = ((Time.current - start) * 1000).round(2)



health[:services][:redis] = {

status: 'up',

latency_ms: redis_latency

}

rescue StandardError => e

health[:services][:redis] = {

status: 'down',

error: e.message

}

health[:status] = 'degraded'

end



# Check Embedding Service

begin

start = Time.current

EmbeddingService.new.generate('test')

embedding_latency = ((Time.current - start) * 1000).round(2)



health[:services][:embeddings] = {

status: 'up',

latency_ms: embedding_latency

}

rescue StandardError => e

health[:services][:embeddings] = {

status: 'down',

error: e.message

}

health[:status] = 'degraded'

end



# Check Sidekiq

begin

queue_size = Sidekiq::Queue.new('document_processing').size



health[:services][:sidekiq] = {

status: queue_size < 1000 ? 'up' : 'degraded',

queue_size: queue_size

}

rescue StandardError => e

health[:services][:sidekiq] = {

status: 'down',

error: e.message

}

end



status_code = case health[:status]

when 'healthy' then 200

when 'degraded' then 200

when 'unhealthy' then 503

end



render json: health, status: status_code

end

end

Kubernetes Liveness & Readiness Probes


# kubernetes/deployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

name: vector-search-api

spec:

template:

spec:

containers:

- name: rails

image: vector-search-api:latest

ports:

- containerPort: 3000



# Liveness probe (restart if unhealthy)

livenessProbe:

httpGet:

path: /health

port: 3000

initialDelaySeconds: 30

periodSeconds: 10

timeoutSeconds: 5

failureThreshold: 3



# Readiness probe (remove from load balancer if not ready)

readinessProbe:

httpGet:

path: /health

port: 3000

initialDelaySeconds: 10

periodSeconds: 5

timeoutSeconds: 3

failureThreshold: 2

Prometheus Metrics & Monitoring

Setting Up Prometheus


# config/initializers/prometheus.rb

require 'prometheus/client'

require 'prometheus/client/formats/text'



# Global prometheus registry

PROMETHEUS = Prometheus::Client.registry



# Vector search metrics

VECTOR_SEARCH_DURATION = PROMETHEUS.histogram(

:vector_search_duration_seconds,

docstring: 'Vector search duration in seconds',

labels: [:tenant_id, :status],

buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]

)



VECTOR_SEARCH_TOTAL = PROMETHEUS.counter(

:vector_search_total,

docstring: 'Total vector searches',

labels: [:tenant_id, :status]

)



VECTOR_SEARCH_CACHE_HIT = PROMETHEUS.counter(

:vector_search_cache_hit_total,

docstring: 'Cache hits for vector searches',

labels: [:tenant_id]

)



# Circuit breaker metrics

CIRCUIT_BREAKER_STATE = PROMETHEUS.gauge(

:circuit_breaker_state,

docstring: 'Circuit breaker state (0=closed, 1=open, 2=half_open)',

labels: [:tenant_id]

)



CIRCUIT_BREAKER_FAILURES = PROMETHEUS.counter(

:circuit_breaker_failures_total,

docstring: 'Total circuit breaker failures',

labels: [:tenant_id]

)



# Document processing metrics

DOCUMENT_PROCESSING_DURATION = PROMETHEUS.histogram(

:document_processing_duration_seconds,

docstring: 'Document processing duration in seconds',

labels: [:status],

buckets: [0.1, 0.5, 1, 2, 5, 10, 30, 60, 120, 300]

)



DOCUMENT_PROCESSING_TOTAL = PROMETHEUS.counter(

:document_processing_total,

docstring: 'Total documents processed',

labels: [:status]

)



# Sidekiq queue metrics

SIDEKIQ_QUEUE_SIZE = PROMETHEUS.gauge(

:sidekiq_queue_size,

docstring: 'Current Sidekiq queue size',

labels: [:queue]

)

Instrumenting Vector Search


# app/services/vector_indexing_service.rb

def search(tenant_id:, query:, filters: {}, limit: 20)

start_time = Time.current



begin

# Check cache first

cache_key = generate_cache_key(query, filters, limit)

cached_result = Rails.cache.read(cache_key)



if cached_result

VECTOR_SEARCH_CACHE_HIT.increment(labels: { tenant_id: tenant_id })

return cached_result

end



# Perform search with resilience

limiter = RateLimiterRegistry.for_tenant(tenant_id)

breaker = CircuitBreakerRegistry.for_tenant(tenant_id)



result = limiter.acquire do

breaker.call(fallback: -> { fallback_search(tenant_id, query, limit) }) do

perform_vector_search(query, filters, limit, tenant_id)

end

end



# Cache result

Rails.cache.write(cache_key, result, expires_in: 1.hour)



# Record success

duration = Time.current - start_time

VECTOR_SEARCH_DURATION.observe(duration, labels: { tenant_id: tenant_id, status: 'success' })

VECTOR_SEARCH_TOTAL.increment(labels: { tenant_id: tenant_id, status: 'success' })



result

rescue StandardError => e

# Record failure

duration = Time.current - start_time

VECTOR_SEARCH_DURATION.observe(duration, labels: { tenant_id: tenant_id, status: 'error' })

VECTOR_SEARCH_TOTAL.increment(labels: { tenant_id: tenant_id, status: 'error' })



raise

end

end

Metrics Endpoint


# config/routes.rb

Rails.application.routes.draw do

get '/metrics', to: 'metrics#show'

end



# app/controllers/metrics_controller.rb

class MetricsController < ApplicationController

skip_before_action :authenticate_user!



def show

# Update circuit breaker metrics

CircuitBreakerRegistry.stats.each do |name, stats|

state_value = case stats[:state]

when :closed then 0

when :open then 1

when :half_open then 2

end



CIRCUIT_BREAKER_STATE.set(state_value, labels: { tenant_id: name })

end



# Update Sidekiq queue metrics

Sidekiq::Queue.all.each do |queue|

SIDEKIQ_QUEUE_SIZE.set(queue.size, labels: { queue: queue.name })

end



# Return Prometheus format

render plain: Prometheus::Client::Formats::Text.marshal(PROMETHEUS),

content_type: Prometheus::Client::Formats::Text::CONTENT_TYPE

end

end

Grafana Dashboards

Key Panels

*1. Search Performance*


# P50 latency

histogram_quantile(0.50,

rate(vector_search_duration_seconds_bucket[5m]))



# P95 latency

histogram_quantile(0.95,

rate(vector_search_duration_seconds_bucket[5m]))



# P99 latency

histogram_quantile(0.99,

rate(vector_search_duration_seconds_bucket[5m]))

*2. Throughput*


# Requests per second

rate(vector_search_total[1m])



# By tenant

sum(rate(vector_search_total[5m])) by (tenant_id)

*3. Cache Hit Rate*


# Cache hit rate percentage

100 * (

rate(vector_search_cache_hit_total[5m]) /

rate(vector_search_total[5m])

)

*4. Circuit Breaker Status*


# Count of open circuits

count(circuit_breaker_state == 1)



# Circuit state by tenant

circuit_breaker_state

*5. Error Rate*


# Error percentage

100 * (

rate(vector_search_total{status="error"}[5m]) /

rate(vector_search_total[5m])

)

Dashboard JSON

See full dashboard configuration: examples/grafana-dashboard.json in the Vectra repo.

Production Metrics (Real Numbers)

After implementing resilience patterns:


Search Latency:

- P50: 45ms (cached: 3ms)

- P95: 120ms (cached: 8ms)

- P99: 250ms



Availability:

- Uptime: 99.94% (exceeding 99.9% SLA)

- MTTR: 8 minutes average

- Longest incident: 45 minutes (Qdrant upgrade)



Circuit Breaker Stats:

- Opened: 12 times in 2024

- Average recovery time: 2.5 minutes

- Prevented outages: 8 incidents



Cache Performance:

- Hit rate: 42%

- Memory usage: 2.1GB

- Evictions: 145/day



Rate Limiting:

- Triggered: 234 times/month

- Top abuser: 1,847 requests in 1 minute

- No successful DOS attacks

What We've Covered

✅ *Circuit Breakers* - Prevented 8 outages in 2024

✅ *Rate Limiting* - Per-tenant isolation, no DOS attacks

✅ *Health Checks* - Kubernetes-ready monitoring

✅ *Prometheus Metrics* - Real-time observability

✅ *Grafana Dashboards* - Beautiful visualizations

Coming in Part 3: Cost Optimization 💰

We've built a resilient system. Now let's talk money.

In *Part 3* (Friday), we'll cover:

*Infrastructure Cost Breakdown* - $600/month vs $3,200/month (81% savings)
*Cache Optimization* - 42% hit rate = 70% latency reduction
*Chunking Strategy* - Why 512 tokens with 128 overlap beats 1000 tokens
*Self-Hosting Embeddings* - $200/month savings vs OpenAI
*5 Key Lessons Learned* - What worked, what didn't

*Spoiler:* The biggest savings came from caching, not from switching providers.

Resources

*Vectra Gem*: github.com/stokry/vectra
*Grafana Dashboard*: examples/grafana-dashboard.json
*Prometheus Exporter*: examples/prometheus-exporter.rb

Questions about production resilience? Share your own war stories in the comments!

Building Enterprise Vector Search in Rails (Part 1/3): Architecture & Multi-Tenant Implementation

Stokry — Sun, 11 Jan 2026 15:38:35 +0000

This is Part 1 of a 3-part series on building production-ready vector search for enterprise SaaS.

Part 1: Architecture & Implementation 👈 You are here

Part 2: Production Resilience & Monitoring

Part 3: Cost Optimization & Lessons Learned (Coming Friday)

TL;DR: Deep dive into building an enterprise SaaS platform that processes 2M+ compliance documents monthly using vector search. This part covers the architecture, Rails implementation, and multi-tenant isolation patterns.

The Business Problem

The scenario: An enterprise SaaS platform serving Fortune 500 companies managing regulatory compliance - banks, healthcare, and pharma companies dealing with complex regulatory requirements.

The pain point: Organizations receive thousands of regulatory documents monthly (SEC filings, FDA guidelines, ISO standards, GDPR updates). Compliance teams spend 60+ hours/week manually searching through PDFs to find relevant sections.

The challenge: Build AI-powered semantic search that understands regulatory language and returns precise results across millions of documents.

The constraints:

150+ enterprise clients (multi-tenant architecture)
2.1M documents total, growing 50k/month
Average document: 200 pages, 500KB
SOC2 + GDPR compliant (audit logs, data isolation)
99.9% uptime SLA ($10k/hour penalties)
Budget: $2,500/month for infrastructure

The Architecture: What We Built

High-Level Overview

Key Components:

Rails API - Multi-tenant document processing
Vectra - Unified vector DB client (provider-agnostic)
Qdrant - Self-hosted vector database (cost + GDPR compliance)
Sidekiq - Background job processing
Sentence-Transformers - Self-hosted embedding generation

Multi-Tenant Document Processing

The Challenge: Processing PDFs at Scale

Input: Client uploads a 300-page PDF (SEC 10-K filing)

Requirements:

Extract text from PDF
Split into searchable chunks (with overlap for context)
Generate embeddings for each chunk
Index with tenant isolation
Track processing status
Audit trail for compliance

The Implementation

1. Document Model (Rails)

# app/models/document.rb
class Document < ApplicationRecord
  belongs_to :tenant
  has_many :document_chunks, dependent: :destroy

  # Store embeddings as binary
  serialize :metadata, JSON

  enum status: {
    pending: 0,
    processing: 1,
    indexed: 2,
    failed: 3
  }

  # Validations
  validates :title, :file_url, presence: true
  validates :tenant_id, presence: true

  # Callbacks
  after_create :schedule_processing

  def schedule_processing
    DocumentProcessingJob.perform_later(id)
  end
end

Why this model?

tenant_id ensures every document belongs to a tenant
status enum tracks processing pipeline
document_chunks stores split chunks with embeddings
Background job keeps upload fast (< 300ms)

2. Chunk Splitting Strategy

Problem: You can't embed an entire 300-page document. You need to split it intelligently.

Solution: Sliding window with overlap:

# app/services/document_chunker.rb
class DocumentChunker
  CHUNK_SIZE = 512        # tokens (~400 words)
  CHUNK_OVERLAP = 128     # tokens for context continuity

  def initialize(document)
    @document = document
    @text = extract_text_from_pdf(document.file_url)
  end

  def split
    # Tokenize text
    tokens = tokenize(@text)
    chunks = []

    # Sliding window
    position = 0
    while position < tokens.length
      chunk_tokens = tokens[position...(position + CHUNK_SIZE)]

      # Convert back to text
      chunk_text = detokenize(chunk_tokens)

      chunks << {
        text: chunk_text,
        position: position,
        page_start: calculate_page(position),
        page_end: calculate_page(position + CHUNK_SIZE)
      }

      # Slide window with overlap
      position += (CHUNK_SIZE - CHUNK_OVERLAP)
    end

    chunks
  end

  private

  def extract_text_from_pdf(file_url)
    # Using pdf-reader gem
    pdf = PDF::Reader.new(file_url)
    pdf.pages.map(&:text).join("\n")
  end

  def tokenize(text)
    # Simple whitespace tokenization (use tiktoken for production)
    text.split(/\s+/)
  end

  def calculate_page(token_position)
    # Average 300 tokens per page
    (token_position / 300.0).ceil
  end
end

Why overlap? Without it, search misses context at chunk boundaries. With 128-token overlap, we improved precision from 0.73 → 0.89.

3. Background Processing Job

# app/jobs/document_processing_job.rb
class DocumentProcessingJob < ApplicationJob
  queue_as :document_processing

  # Retry with exponential backoff
  retry_on StandardError, wait: :exponentially_longer, attempts: 5

  def perform(document_id)
    document = Document.find(document_id)
    document.update!(status: :processing)

    # Step 1: Split document into chunks
    chunker = DocumentChunker.new(document)
    chunks = chunker.split

    # Step 2: Generate embeddings for each chunk
    embedder = EmbeddingService.new
    chunk_records = []

    chunks.each_with_index do |chunk, idx|
      embedding = embedder.generate(chunk[:text])

      chunk_record = DocumentChunk.create!(
        document: document,
        text: chunk[:text],
        position: chunk[:position],
        page_start: chunk[:page_start],
        page_end: chunk[:page_end],
        embedding: embedding
      )

      chunk_records << chunk_record
    end

    # Step 3: Batch index to vector DB (tenant-isolated)
    VectorIndexingService.new.index_document(document, chunk_records)

    # Step 4: Mark as indexed
    document.update!(
      status: :indexed,
      chunk_count: chunks.size,
      indexed_at: Time.current
    )

    # Step 5: Audit log
    AuditLog.create!(
      tenant_id: document.tenant_id,
      event_type: 'document_indexed',
      resource_type: 'Document',
      resource_id: document.id,
      metadata: {
        chunk_count: chunks.size,
        file_size: document.file_size,
        duration_ms: (Time.current - document.created_at) * 1000
      }
    )

  rescue StandardError => e
    document.update!(status: :failed, error_message: e.message)
    Sentry.capture_exception(e, extra: { document_id: document.id })
    raise
  end
end

Performance:

Processes 15 documents/minute
Average processing time: 4 seconds per document
Failure rate: 0.3% (mostly PDF parsing issues)

4. Embedding Service (Self-Hosted)

Why self-hosted? GDPR compliance - we can't send client data to OpenAI.

# app/services/embedding_service.rb
class EmbeddingService
  EMBEDDING_ENDPOINT = ENV.fetch('EMBEDDING_SERVICE_URL', 'http://localhost:8080')
  MODEL = 'all-mpnet-base-v2'  # 768 dimensions

  def generate(text)
    # Add retry logic
    response = Faraday.new(url: EMBEDDING_ENDPOINT) do |f|
      f.request :json
      f.response :json
      f.adapter Faraday.default_adapter
      f.options.timeout = 30
      f.options.open_timeout = 10
    end.post('/embeddings', {
      text: text,
      model: MODEL
    })

    raise "Embedding failed: #{response.status}" unless response.success?

    response.body['embedding']
  rescue Faraday::Error => e
    Rails.logger.error("Embedding service error: #{e.message}")
    raise
  end
end

Cost savings: Self-hosting saves $200/month vs OpenAI API (2M embeddings/month).

Vector Search with Multi-Tenant Isolation

The Challenge: Tenant Data Isolation

Critical requirement: Client A must NEVER see Client B's documents.

Approach: Qdrant namespaces + application-level verification (defense in depth)

# app/services/vector_indexing_service.rb
class VectorIndexingService
  def initialize
    @client = build_client
  end

  # Index entire document (batched)
  def index_document(document, chunk_records)
    vectors = chunk_records.map do |chunk|
      {
        id: vector_id(chunk),
        values: chunk.embedding,
        metadata: {
          document_id: document.id,
          tenant_id: document.tenant_id,
          title: document.title,
          page_start: chunk.page_start,
          page_end: chunk.page_end,
          chunk_text: chunk.text[0..500], # Preview only
          indexed_at: Time.current.iso8601
        }
      }
    end

    # Batch upsert with tenant namespace
    batch = Vectra::Batch.new(@client)
    result = batch.upsert_async(
      index: 'compliance_documents',
      vectors: vectors,
      namespace: namespace_for_tenant(document.tenant_id),
      concurrency: 4
    )

    Rails.logger.info(
      "Indexed document #{document.id} for tenant #{document.tenant_id}: " \
      "#{result[:success]} chunks"
    )

    result
  end

  # Search within tenant (isolated)
  def search(tenant_id:, query:, filters: {}, limit: 20)
    # Generate query embedding
    query_embedding = EmbeddingService.new.generate(query)

    # Ensure tenant isolation
    namespace = namespace_for_tenant(tenant_id)

    # Query vector DB
    results = @client.query(
      index: 'compliance_documents',
      vector: query_embedding,
      top_k: limit,
      namespace: namespace,  # CRITICAL: tenant isolation
      filter: filters,
      include_metadata: true
    )

    # Verify tenant_id in results (defense in depth)
    verified_results = results.select do |match|
      match.metadata['tenant_id'] == tenant_id
    end

    # Log potential security issue
    if verified_results.size != results.size
      SecurityAlert.create!(
        severity: 'critical',
        message: "Tenant isolation breach detected",
        details: {
          tenant_id: tenant_id,
          expected: results.size,
          verified: verified_results.size
        }
      )
    end

    verified_results
  end

  private

  def build_client
    # Cached client with resilience
    base_client = Vectra.qdrant(
      host: ENV.fetch('QDRANT_HOST'),
      api_key: ENV['QDRANT_API_KEY'],
      timeout: 10,
      max_retries: 3
    )

    # Add caching layer
    cache = Vectra::Cache.new(
      ttl: 3600,      # 1 hour for search results
      max_size: 5000  # Top 5000 queries cached
    )

    Vectra::CachedClient.new(base_client, cache: cache)
  end

  def namespace_for_tenant(tenant_id)
    "tenant_#{tenant_id}"
  end

  def vector_id(chunk)
    "chunk_#{chunk.id}"
  end
end

Security Best Practice: Never Trust, Always Verify

# NEVER trust the namespace alone
results = client.query(namespace: "tenant_#{tenant_id}", ...)

# ALWAYS verify in application
verified = results.select { |r| r.metadata['tenant_id'] == tenant_id }

if verified.size != results.size
  # SECURITY BREACH - alert immediately
  SecurityAlert.critical!("Tenant isolation breach detected")
end

Search API with Enterprise Features

The Controller

# app/controllers/api/v1/search_controller.rb
module Api
  module V1
    class SearchController < ApiController
      before_action :authenticate_user!
      before_action :rate_limit_check

      # POST /api/v1/search
      def create
        query = params.require(:query)

        # Validate query
        if query.blank? || query.length < 3
          return render json: { error: 'Query too short' }, status: :bad_request
        end

        # Build filters from params
        filters = build_filters(params[:filters])

        # Perform search (with timing)
        start_time = Time.current

        results = VectorIndexingService.new.search(
          tenant_id: current_tenant.id,
          query: query,
          filters: filters,
          limit: params[:limit] || 20
        )

        duration_ms = ((Time.current - start_time) * 1000).round(2)

        # Hydrate results (load Document records)
        documents = hydrate_results(results)

        # Log search for analytics
        SearchLog.create!(
          tenant_id: current_tenant.id,
          user_id: current_user.id,
          query: query,
          result_count: results.size,
          duration_ms: duration_ms,
          filters: filters
        )

        # Audit log for compliance
        AuditLog.create!(
          tenant_id: current_tenant.id,
          user_id: current_user.id,
          event_type: 'document_search',
          metadata: {
            query: query,
            result_count: results.size,
            duration_ms: duration_ms
          }
        )

        render json: {
          results: documents.map { |doc| DocumentSerializer.new(doc).as_json },
          metadata: {
            total: results.size,
            duration_ms: duration_ms,
            cached: cache_hit?(results)
          }
        }
      rescue Vectra::Error => e
        # Handle vector DB errors gracefully
        Rails.logger.error("Vector search error: #{e.message}")
        Sentry.capture_exception(e)

        # Fallback to SQL search
        fallback_results = fallback_search(query)

        render json: {
          results: fallback_results,
          metadata: {
            fallback: true,
            error: 'Vector search unavailable'
          }
        }, status: :partial_content
      end

      private

      def build_filters(filter_params)
        return {} unless filter_params.present?

        filters = {}
        filters[:document_type] = filter_params[:document_type] if filter_params[:document_type]
        filters[:year] = filter_params[:year].to_i if filter_params[:year]
        filters[:regulatory_body] = filter_params[:regulatory_body] if filter_params[:regulatory_body]
        filters
      end

      def hydrate_results(results)
        # Extract document IDs from vector search results
        document_ids = results.map { |r| r.metadata['document_id'] }.uniq

        # Load documents from DB
        documents = Document.where(id: document_ids, tenant_id: current_tenant.id)
                           .index_by(&:id)

        # Attach scores to documents
        results.map do |match|
          doc = documents[match.metadata['document_id']]
          next unless doc

          doc.instance_variable_set(:@search_score, match.score)
          doc.instance_variable_set(:@matched_chunk, match.metadata['chunk_text'])
          doc.instance_variable_set(:@matched_pages, "#{match.metadata['page_start']}-#{match.metadata['page_end']}")

          doc.define_singleton_method(:search_score) { @search_score }
          doc.define_singleton_method(:matched_chunk) { @matched_chunk }
          doc.define_singleton_method(:matched_pages) { @matched_pages }

          doc
        end.compact
      end

      def rate_limit_check
        # 100 searches per hour per user
        key = "search_rate_limit:#{current_user.id}"
        count = Rails.cache.increment(key, 1, expires_in: 1.hour)

        if count > 100
          render json: { error: 'Rate limit exceeded' }, status: :too_many_requests
        end
      end
    end
  end
end

API Response Example:

{
  "results": [
    {
      "id": 1234,
      "title": "SEC 10-K Filing 2024",
      "search_score": 0.89,
      "matched_chunk": "The company faces significant regulatory risks...",
      "matched_pages": "45-46"
    }
  ],
  "metadata": {
    "total": 10,
    "duration_ms": 45,
    "cached": false
  }
}

What We've Built So Far

✅ Multi-tenant document processing pipeline

PDF → Text extraction
Intelligent chunking with overlap
Self-hosted embeddings (GDPR compliant)
Background job processing

✅ Secure vector search

Qdrant namespace isolation
Application-level verification
Audit logging
Graceful fallback to SQL

✅ Production-ready API

Fast response times (45ms P50)
Comprehensive error handling
Rate limiting
Search analytics

Coming in Part 2: Production Resilience 🛡️

Building the search feature is only half the battle. In Part 2 (Wednesday), we'll cover:

Circuit Breakers - How they saved us during Black Friday when Qdrant overloaded
Rate Limiting - Per-tenant throttling to prevent abuse
Health Checks - Kubernetes-ready monitoring
Prometheus Metrics - Real-time observability
Grafana Dashboards - Visualizing search performance

Real incident story: Our Qdrant cluster hit 98% CPU during a traffic spike. Without circuit breakers, we would have had a complete outage. Instead, 99.2% of searches still worked using cached results and PostgreSQL fallback.

Resources

Vectra Gem: github.com/stokry/vectra
Documentation: vectra-docs.netlify.app
Example Code: examples/comprehensive_demo.rb

Questions about the architecture or implementation? Drop a comment below!

Vectra — The Unified Vector Database Client for Ruby

Stokry — Fri, 09 Jan 2026 17:59:04 +0000

If you've ever built AI applications in Ruby — semantic search, RAG, recommendation engines, or embedding queries — you know how quickly it becomes tedious to support multiple vector databases. Each provider has its own API, client, and quirks.

Vectra solves this problem by providing a single, unified API for all popular vector DBs, freeing you from vendor lock-in. Write your code once and switch backends effortlessly — Pinecone, Qdrant, Weaviate, and even PostgreSQL with pgvector.

GitHub Repository

🧠 Why Vectra?

Vector databases are becoming a standard in modern AI applications, yet the Ruby ecosystem lacks a consistent way to work with them. Vectra changes that with:

✨ Provider-agnostic API

One set of methods: upsert, query, delete, and more — no need to learn each vendor's SDK. Documentation

⚙️ Support for multiple databases

Pinecone
Qdrant
Weaviate
pgvector All through the same client. GitHub

💪 Production-ready

Retry logic, configurable backoff, observability through metrics, and ready-to-use classes for production workloads. Docs

📦 Rails Integration

Use has_vector DSL for ActiveRecord models — embedding fields can be automatically indexed and searched. GitHub

📚 Well-documented gem

Guides, YARD documentation, and examples for each provider. Documentation

Example usage:

require 'vectra'

# Initialize client for any provider
client = Vectra::Client.new(
  provider: :pinecone,
  api_key: ENV['PINECONE_API_KEY'],
  environment: 'us-west-4'
)

# Upsert an embedding
client.upsert(
  vectors: [
    { id: 'doc-1', values: [0.1,0.2,0.3], metadata: { title: 'Hello' } }
  ]
)

# Query for similar vectors
results = client.query(vector: [0.1,0.2,0.3], top_k: 5)
results.each { |m| puts "#{m.id}: #{m.score}" }

# Delete
client.delete(ids: ['doc-1'])

The same API works without changes if you switch the client to Qdrant, Weaviate, or pgvector.

GitHub

💡 When Vectra is Useful

Building applications with semantic search
Implementing RAG (retrieval-augmented generation) with embeddings
Avoiding vendor lock-in across vector databases
Having a consistent Ruby API surface for all vector databases

📦 Where to Find It

GitHub: [https://github.com/stokry/vectra
Documentation: [https://vectra-docs.netlify.app/

If you’re working with embeddings and vector search in Ruby (especially in Rails apps), Vectra can save you hours of frustration and make your architecture future-proof — letting you switch backends without rewriting code

I Found a Copy of My Blog Post Online — Here’s the Tool I Built to Track Every Plagiarized Version

Stokry — Fri, 05 Dec 2025 07:58:55 +0000

Have you ever googled a paragraph from your own article…

…and found 5 other websites publishing it as if they wrote it?

That happened to me.

More than once.

After dealing with DMCA requests, unanswered emails, and plagiarism detectors that give you a “score” but no actual proof, I decided to build something of my own.

Today I’m sharing the tool I built — and use daily — to find duplicated content across the web and see exactly who copied what.

Why I Built It

I wanted three things that existing tools weren’t giving me:

Actual sources — not vague percentages
Visual diff — sentence-by-sentence similarities
Evidence — something I could send to a hosting provider or editor

And I wanted it to be instant: paste text → get proof.

That’s how Nooth.dev was born.

How It Works (Demo)

You paste any piece of content — a blog post, newsletter, documentation, product description — and Nooth.dev searches the web for matches.

Within a few seconds, you get:

A list of websites containing similar or identical content
A similarity score
A split-screen diff view (left = your original, right = the found source)
Highlighted overlaps so you can see the exact copied fragments
Domain metadata (age, authority, region)

You can try it here (3 free analyses/day):

👉 nooth.dev

No login required.

A Real Example: 74% Similarity Match

A few days ago I tested one of my older tutorials.

Nooth found a website hosting a barely modified version of it at 74% similarity.

The diff view clearly showed block-by-block copying.

That was the moment I realized: okay — this thing is genuinely useful.

What Makes It Different

Most plagiarism tools give you:

❌ No real source

❌ No detailed diff

❌ No transparency

❌ No usable proof

Nooth is more like a forensic tool:

It doesn’t just say “67% copied”
It shows exactly which sentences match
And from which site

If you publish anything online — especially technical content — this helps you see where your work ends up.

Free Tier (Current MVP Stage)

Right now, during early access:

3 free analyses per day (IP-based, no login)
The first results are fully visible
Additional matches are behind the upcoming registration

Everything is fully functional — just limited to prevent abuse.

Registration + unlimited mode is coming soon.

Want to Test It?

Try the live tool here:

https://nooth.dev

If you find an interesting match, I’d love to hear — feel free to DM me on X or leave a comment.

What’s Next

In the next weeks, I plan to add:

User accounts
Saved reports
Email alerts when new copies appear
Automatic DMCA template
API access

I’ll share weekly progress updates here on Dev.to.

I Built an AI That Finds Every Article Similar to Your Text — Here’s How It Works

Stokry — Tue, 02 Dec 2025 12:34:19 +0000

Most search engines will show you pages that mention the same keywords.

But what if you want to find articles that express the same ideas, even if they’re written completely differently?

That’s the problem that pushed me into building something new.

A few months ago, I needed to verify whether parts of my content were being reused online. Google wasn’t helpful — keyword search missed half of the semantically similar pages. Rephrased paragraphs were completely invisible.

So I built my own engine.

Today, I’m excited to introduce Nooth, an AI-powered system that takes any text you enter and searches the web for content with similar themes, ideas, and semantic meaning.

Drop in a paragraph → get URLs, similarity percentages, and deep analytical insights.

Here’s how it works under the hood.

🚀 The Idea

Traditional search relies on keyword overlap.

If two texts don’t share the same words, the engine assumes they’re unrelated.

But humans don’t think that way.

We recognize similarity through meaning — the structure of ideas, not characters.

So the goal was simple:

Build an engine that understands what your text is about, then find everything on the internet that expresses the same ideas.

That meant two pieces were crucial:

A way to measure meaning
A way to scan the web, extract clean text, and compare it

🧠 Step 1: Turning Text Into Meaning (Embeddings)

At the core of Nooth is a semantic model that converts text into embeddings — numerical vectors that represent meaning.

Example:

“How to speed up your Node.js API”

and

“Optimizing performance in a JavaScript backend”

These look different, but in vector space, their embeddings are neighbors.

Every time you paste text into Nooth:

it cleans and normalizes it
creates sentence-level embeddings
composes them into a unified “semantic fingerprint”

That fingerprint becomes the reference point for the search.

🌐 Step 2: Finding Content Across the Web

Nooth crawls and indexes content using:

lightweight scrapers
structured metadata
boilerplate removal
language detection
canonical URL resolution
and chunking for long-form content

The goal isn’t to store the entire HTML — only the meaningful part:

titles, headings, paragraphs, and contextual metadata.

Each chunk gets its own embedding.

That allows highly precise, paragraph-level similarity detection.

📊 Step 3: Computing Semantic Similarity

Once all candidates are scraped, Nooth performs:

cosine similarity
conceptual overlap scoring
syntactic variance weighting
partial-match scoring (for rewritten content)

This combination is what lets Nooth detect even heavily rephrased articles.

It doesn’t just ask:

“Do these texts share words?”

It asks:

“Do these texts express the same thing?”

🔍 Step 4: Presenting Clear Evidence

The final result looks like this (example flow):

You paste a paragraph
Engine breaks it into meaning blocks
It fetches semantically closest content across the web
You get:

-   URL

-   domain

-   similarity percentage

-   highlighted overlapping ideas

-   reasoning on _why_ the match is close

This makes it useful for:

discovering who’s copying your content
researching related topics
finding competitors
understanding what else exists on the same theme
generating new ideas based on existing clusters

🧭 Why I Built It

I didn’t want another Google-like keyword search.

I wanted a lens into the semantic web — the world of ideas, not strings.

Whether you’re a researcher, writer, builder, or SEO nerd, sometimes you need to find content that’s conceptually similar, not textually identical.

Nooth solves that by combining:

modern embeddings
optimized scraping
semantic search
scoring heuristics
and clean UI on top of everything

It shows you what’s related on an idea level, not just word level.

🧪 Try It Out

If you want to test it:

👉 Paste any text.

👉 See what the web holds that’s semantically connected.

👉 Explore clusters of related ideas you didn’t even know existed.

https://nooth.dev/

🔮 What’s Next

I’m currently working on:

deeper plagiarism detection
topic extraction
content clustering
semantic timelines
alerts when new similar content appears
full API for developers

If any of this sounds interesting, I’d love feedback.

Thanks for reading — and welcome to the world of semantic discovery.

Django Has Never Let Me Down

Stokry — Wed, 13 Aug 2025 12:06:12 +0000

I’ve been working with Django for years, and honestly — it has never once let me down.

It doesn’t matter if it’s a huge, complex project with a million moving parts, a small personal app, or just a CMS for a simple website, blog, or magazine — Django delivers every single time.

Big or Small, It Just Works

Whether I’m building a full-fledged web application or using Django purely as an API backend, the experience is the same:

I know exactly what I need to do and where to do it. The structure is predictable, the patterns are clear, and there’s a kind of quiet confidence that comes with using it.

Help Is Always There

The Django community is one of the friendliest and most resourceful I’ve ever encountered.

If you get stuck, there’s always an answer online — whether it’s in the official docs, Stack Overflow, or countless blog posts and tutorials.

Secure by Default

Security is not something you have to “bolt on” later with Django.

It’s there from the start — CSRF protection, SQL injection prevention, secure authentication, and a track record of taking security seriously.

Easy Deployment

Deploying Django is straightforward.

Whether it’s on a small VPS, a cloud service, or containerized in Docker, it’s a smooth process. No magic tricks required.

Why Look Anywhere Else?

I genuinely don’t see a reason to switch to another framework. For me, Django is the perfect mix of:

Stability
Security
Predictability
Flexibility

It’s the framework I trust, the one I know inside out, and the one that keeps delivering — no matter the scale or complexity of the project.

If I had to sum it up:

Django is my go-to, and I’m not planning to change that anytime soon.

We Built a Django App Serving 10k Mobile Users, 2k Admins, and 30 Companies – Here’s What Worked

Stokry — Mon, 11 Aug 2025 10:55:03 +0000

We started with a single monolithic Django app that was buckling under API traffic and ERP sync jobs.

Here's how we turned it into a lean, scalable platform.

The Scope

Our platform now:

Serves 10,000+ active mobile app users daily
Powers an admin portal for 2,000+ staff
Runs 30 different company logics with their own ERP integrations

We needed a setup that could scale independently, keep costs under control, and securely isolate tenants.

The Architecture That Worked

1. API/Web separated from Workers

We split the API/web layer from background workers.

Web pods scale on traffic spikes, workers scale on ERP sync queue depth.

This means heavy data processing never slows down API responses.

2. Multi-tenant PostgreSQL (RLS)

We chose Row-Level Security for tenant isolation.

It’s secure, easier to manage than schema-per-tenant in our case, and allows shared migrations.

3. Celery + Redis

Our ERP syncs run hourly per tenant. Celery workers handle:

Heavy data imports
Parsing and bulk upserts
Notifications

4. API Gateway

We placed an API gateway in front of Django for:

Auth
Rate limits
Request shaping

5. S3 + CDN

For media and assets. Redis is also used for:

Session store
Caching hot queries

6. Observability

Sentry for errors
Prometheus + Grafana for metrics
ELK stack for logs

7. Zero-downtime deploys

Feature flags let us roll out new ERP parsers without breaking production.

Results After 6 Months

~30% lower cloud spend compared to monolithic scaling
5–7× faster ERP syncs thanks to bulk upserts & isolated worker scaling
Zero cross-tenant data leaks
New company onboarding in < 2 days

Key Lessons Learned

Separate workloads early – Scaling API and workers independently saves money and avoids performance bottlenecks.
Invest in observability – Catching issues early is cheaper than firefighting later.
Multi-tenancy design is critical – Choose your isolation model carefully; it’s hard to change later.
Feature flags are invaluable for safe rollouts.

Curious how others have scaled Django for multi-tenant apps — what’s your go-to trick?

Need a hand with Django? Drop by djangofixer.com

I Launched DjangoFixer — Here’s Why

Stokry — Thu, 07 Aug 2025 11:22:14 +0000

Over the past few years, I’ve been working with Django on a daily basis — both in teams and solo. And no matter how experienced the developer is, problems still happen:

Async views breaking in production
Strange deployment errors out of nowhere
API endpoints returning 500s
Migrations gone rogue
“Can you just take a look?” messages at 11pm 😅

After getting countless messages from fellow devs and friends like:

“Hey, I’ve got this Django bug — can you fix it?”

I started to realize: there’s a pattern here.

People don’t always need a full-time Django consultant.

They just want a fast, no-BS fix for their current problem — without having to hire a whole agency or dig through StackOverflow for hours.

So I built DjangoFixer.com ⚡

DjangoFixer is a small service I launched to help developers and teams fix Django problems quickly and professionally.

Think of it as a Django emergency room — focused only on:

Bug fixing
Deployment troubleshooting
Async + API issues
Performance slowdowns
DB migrations, and more

I don’t do everything. I just fix Django issues — fast.

No delays, no confusion, no vague consulting contracts.

Why this makes sense (at least to me):

🧠 Specialization means I get better at these problems every week.
🛠️ Most teams don’t need long-term help — they just need this one thing fixed.
⏱️ Turnaround time matters. When stuff is broken, you don’t want to wait 2 weeks for an agency call.

Want to try it out?

Check it out here → DjangoFixer.com

If you’re dealing with Django bugs that slow you down — I’d be happy to help.

And if you’ve built something similar or are running a microservice business, let’s connect! 🙌