DEV Community: Mārtiņš Veiss

Weekly Update: ✨ feat(backend): lightweight inference mode — bypass RAG/mem

Mārtiņš Veiss — Mon, 01 Jun 2026 05:00:12 +0000

Weekly Update: ✨ feat(backend): lightweight inference mode — bypass RAG/memory for trivial tier (MVA-1992) (#9160)

This week we shipped:

🔧 feat(backend): lightweight inference mode — bypass RAG/memor
🔧 feat(plugins): add capability approval dialog and audit log
🔧 fix(ci): sync Ansible slm_agent role + fix chromadb hardened

Contributors: +2

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-05-25T05:00:04.234278Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

Running AI in regulated environments: how AutoBot keeps your documents on-premise

Mārtiņš Veiss — Thu, 28 May 2026 06:51:48 +0000

Most AI productivity tools are asking you to trust a third party with your data. For a solo dev building side projects, that trade-off is fine. For a law firm, a hospital system, or a fintech company — it isn't.

This post is for the second group. I want to walk through exactly how AutoBot handles data in a regulated environment, where the risks actually sit, and what you need to configure to deploy it safely.

The problem with cloud AI in regulated industries

When you send a prompt to GPT-4, Claude, or Gemini, your text crosses the network. That's obvious. What's less obvious is what else goes with it when you're using most AI platforms:

The documents you've uploaded to provide context
The "system prompt" that describes your business or patient workflows
Metadata about what you're working on and when

For HIPAA-covered entities, PHI (Protected Health Information) cannot be processed by a business associate without a signed BAA. Most consumer AI products don't offer BAAs. The ones that do cost enterprise pricing and require legal review cycles.

GDPR's article 28 has similar requirements for data processors. SOC 2 Type II audits will ask where your data goes. ISO 27001 requires you to document and control it.

None of this means you can't use AI. It means you need to choose carefully where your data goes.

AutoBot's data model

AutoBot separates two things that most platforms conflate: the knowledge base and the brain.

The knowledge base is the documents you upload — your patient intake forms, your case files, your customer contracts, your internal codebase. In AutoBot, this data never leaves your machine. The RAG engine (Retrieval-Augmented Generation) indexes your documents locally into ChromaDB, a vector database running on your own hardware. When you ask a question, the relevant chunks are retrieved locally — no external API call has happened yet.

The brain is the LLM that synthesizes the retrieved context into an answer. This is where you have a choice:

Run it locally via Ollama or any OpenAI-compatible server — prompts never leave your network
Route to a cloud model (GPT-4, Claude) — only the synthesized prompt goes out, not your documents

The line is: the brain phones home. Your documents don't.

This is meaningfully different from uploading files to a cloud AI assistant. In AutoBot, a search for "what does our standard NDA say about IP assignment" retrieves the relevant clause locally and sends only the question + retrieved text to the LLM. Your full NDA document never leaves your hardware.

Network isolation in practice

Data model is one layer. Network configuration is another.

AutoBot runs on Docker Compose. The default configuration is suitable for a development environment — it exposes ports on 0.0.0.0, which means anything on your network can reach it. For production in a regulated environment, you need to tighten this.

1. Bind to localhost, not all interfaces

In your docker-compose.override.yml:

services:
  frontend:
    ports:
      - "127.0.0.1:3000:3000"
  backend:
    ports:
      - "127.0.0.1:8000:8000"

This means AutoBot is only reachable from the host machine itself. Put a reverse proxy (nginx, Caddy) in front of it that handles TLS and access control.

2. Internal service network

Keep internal services off the host network entirely:

networks:
  autobot-internal:
    internal: true

services:
  chromadb:
    networks:
      - autobot-internal
  redis:
    networks:
      - autobot-internal
  backend:
    networks:
      - autobot-internal
      - default  # only backend needs external access for LLM calls

ChromaDB and Redis should never be reachable outside the Docker network.

3. Resource limits

One question I keep seeing (shoutout to @clawnewsai.bsky.social for surfacing this): does AutoBot enforce resource limits by default?

Currently no — the deploy.resources stanza in docker-compose.yml is left to the operator. For production, add explicit limits to prevent one runaway process from starving the host:

services:
  backend:
    deploy:
      resources:
        limits:
          cpus: "2.0"
          memory: 4G
        reservations:
          cpus: "0.5"
          memory: 1G

Real-world numbers vary with load. A good starting point for a team of 5-10 users: 4 vCPUs and 8GB RAM for the full stack on a dedicated host.

What goes where: the data flow audit

For compliance documentation, here's exactly what leaves your network in each configuration:

Mode	Documents	Prompts	Answers
Full local (Ollama)	Never	Never	Never
Hybrid (OpenAI/Claude for LLM)	Never	Yes — to LLM provider	Returned from LLM
Cloud-only (no local Ollama)	Never	Yes — to LLM provider	Returned from LLM

In all cases: your documents stay on your hardware. The ChromaDB vector store is local. The retrieval step is local.

If you're running Ollama on the same host, nothing crosses the network boundary at all. This is the right configuration for HIPAA-covered environments until you have a BAA with your LLM provider.

Getting started in a restricted environment

git clone https://github.com/mrveiss/AutoBot-AI.git
cd AutoBot-AI

# Copy and edit the environment file
cp .env.example .env

# For full local operation: install Ollama first
# https://ollama.com/download
ollama pull llama3.2

# Start AutoBot
docker compose up -d

Open http://localhost:3000. Connect Ollama as your LLM provider. Upload your first document.

From that point, nothing has left your machine.

Compliance checklist before production

This is not legal advice. Get a qualified compliance officer to sign off before processing regulated data. But here's the technical baseline:

[ ] Reverse proxy with TLS in front of AutoBot
[ ] Ports bound to 127.0.0.1, not 0.0.0.0
[ ] ChromaDB and Redis on internal Docker network only
[ ] Resource limits set per service
[ ] Ollama running locally if zero data egress is required
[ ] Audit logging enabled on the reverse proxy layer
[ ] Host firewall rules (ufw / firewalld) blocking unexpected inbound
[ ] Regular backups of ChromaDB volume (your knowledge base)
[ ] BAA executed with any cloud LLM provider you route to

The DevOps guide at dev.to/mrveiss/self-hosting-autobot-a-devops-deep-dive covers the reverse proxy and firewall setup in detail.

The overhead question

I keep getting asked: what's the overhead for a small team?

For a team of 5-10: a single machine with 16GB RAM and a modern CPU runs the full stack comfortably in CPU-only mode. GPU is optional — it dramatically accelerates local inference but isn't required. The DevOps guide has sizing tables for different workload profiles.

The operational overhead is similar to running any self-hosted application — you own the uptime, you manage the updates, you back up the data. For regulated industries, that overhead is already priced in. You're not adding new burden; you're moving existing compliance obligations to infrastructure you control.

The bottom line

AutoBot doesn't solve your compliance program. But it removes the data-egress problem from the AI layer entirely.

If you're in a regulated industry and you've been waiting on cloud AI because you can't figure out the data residency question — the answer is to not send the data in the first place.

Your knowledge base stays on your machine. You pick the brain. If you pick a local brain, nothing leaves your network.

That's the architecture. The compliance framework on top is yours to build. But the foundation is solid.

AutoBot is open source at github.com/mrveiss/AutoBot-AI. Full documentation, Docker deployment guides, and community discussions are there. If you're working through a regulated-environment deployment and hit a configuration question, open a discussion — the community has seen most of the edge cases.

Weekly Update: ✨ fix(llc): address LLC API/model gaps (#8479 #8478 #8476 #8

Mārtiņš Veiss — Mon, 25 May 2026 05:00:10 +0000

Weekly Update: ✨ fix(llc): address LLC API/model gaps (#8479 #8478 #8476 #8474 #8462 #8461 #8493)

This week we shipped:

🔧 fix(llc): address LLC API/model gaps (#8479 #8478 #8476 #847
🔧 fix(llc): fix 18 LLC functional bugs from sprint discovery (
🔧 fix(llc): restore missing enums, auth, migrations, SAST, CI

Contributors: +2

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-05-18T05:00:06.602657Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

Inside AutoBot's Frontend: A Developer Walkthrough

Mārtiņš Veiss — Mon, 11 May 2026 11:32:51 +0000

AutoBot is the open-source, self-hosted AI automation platform where your data never leaves your server.

GitHub: mrveiss/AutoBot-AI

What you see when you open AutoBot

AutoBot's chat interface greets you with a familiar two-pane layout: a conversation sidebar on the left and an active chat panel on the right. Behind that simplicity lives a rich UI built from about 40 focused Vue single-file components.

Chat UI

The core chat flow is:

ChatView.vue
  └── ChatInterface.vue
        ├── ChatSidebar.vue        ← conversation list + search
        ├── ChatHeader.vue         ← model selector, settings toggle
        ├── ChatMessages.vue       ← scrolling message feed
        │     └── MessageItem.vue  ← per-message bubble + citations
        ├── ChatInput.vue          ← textarea, attachments, send button
        ├── ChatTabs.vue           ← switch between Chat / Browser / Docs
        └── CitationsDisplay.vue   ← inline source links from RAG

The ChatTabs.vue component is the pivot point: it lets you jump between a raw conversation, an embedded browser session (for web research), and a documentation search sidebar — all within the same view.

Knowledge Base UI

AutoBot's Knowledge Base is where the "your data" part of Your data. Your AI. lives. The KnowledgeView.vue brings together:

KnowledgeBrowser — file-tree style explorer of all ingested documents
KnowledgeSearch — full-text + vector search with KBSearchResultPanel rendering scored results
KnowledgeGraph / KnowledgeGraph3D — D3-powered entity graph so you can see how concepts connect
KnowledgeUpload — drag-and-drop ingestion with real-time vectorization progress (VectorizationProgressModal)
KnowledgeMaintenance — deduplication, cleanup stats, and orphan management

The pipeline inside KnowledgeView fans out to more than 30 sub-components, but each one has a narrow responsibility. If you add a new panel, you're usually only touching one file.

Component architecture in `autobot-frontend/`

autobot-frontend/
├── src/
│   ├── components/       # feature-scoped component trees
│   │   ├── chat/
│   │   ├── knowledge/
│   │   ├── agents/
│   │   ├── browser/
│   │   ├── charts/
│   │   └── base/         # shared primitives (buttons, modals, …)
│   ├── views/            # route-level pages (one per route)
│   ├── stores/           # Pinia stores (useChatStore, useKnowledgeStore, …)
│   ├── composables/      # shared reactive logic
│   ├── design-system/    # tokens.ts — canonical token catalog
│   ├── router/           # Vue Router config
│   └── styles/           # global CSS + Tailwind @theme block
├── cypress/              # end-to-end tests
└── package.json          # Vue 3 + Vite + Tailwind CSS 4 + TypeScript

Tech stack at a glance

Layer	Choice
Framework	Vue 3 (Composition API)
Language	TypeScript
Build	Vite
State	Pinia
Styling	Tailwind CSS 4 (`@theme` tokens)
Testing	Vitest (unit) + Cypress/Playwright (e2e)
Storybook	Component stories live in `src/stories/`

Design tokens

All colors, spacings, and radii flow from src/design-system/tokens.ts. This file is the single source of truth for token names; actual values live in src/assets/tailwind.css under the @theme block.

// tokens.ts (abridged)
export const SEMANTIC_COLORS = [
  { name: 'autobot-primary',   cls: 'bg-autobot-primary text-white' },
  { name: 'autobot-secondary', cls: 'bg-autobot-secondary text-white' },
  { name: 'autobot-success',   cls: 'bg-autobot-success text-white' },
  // …
]

Adding a new brand color means two edits: tailwind.css for the value, tokens.ts to register the name. That's it.

How to contribute to the UI

Get the repo running

git clone https://github.com/mrveiss/AutoBot-AI.git
cd AutoBot-AI/autobot-frontend
npm install
npm run dev

The dev server starts at http://localhost:5173. You don't need a running backend to work on visual components — the Storybook stories in src/stories/ cover most UI primitives in isolation.

Explore Storybook

npm run storybook

DesignTokens.stories.ts gives you the full token palette in one page. If you want to see a component in isolation before wiring it up to real data, stories are the right place to start.

Find good first issues

The fastest path to a first contribution is the good first issue + area: frontend label combination. These are scoped to single components or small style fixes — no need to understand the full stack before opening a PR.

Common entry points:

Accessibility — the ACCESSIBILITY_IMPROVEMENTS.md doc tracks open a11y work across the chat and KB UIs.
Design token gaps — new palette entries or missing dark-mode mappings in tailwind.css.
Storybook coverage — components in src/components/base/ that don't have a story yet.
Unit tests — src/components/**/__tests__/ has gaps; Vitest tests are welcome.

Testing approach

Unit tests (npm run test:unit) — use Vitest + Vue Test Utils. Keep tests in __tests__/ next to the component.
E2E (npm run test:e2e:dev) — Cypress tests live in cypress/. Run against the Vite dev server.
Type-check — npx vue-tsc --noEmit -p tsconfig.app.json. The repo has a tracked baseline of ~248 type errors (legacy debt); PRs should not add errors — see the CI check in .github/workflows/frontend-typecheck-regression.yml.

PR checklist

npm run lint passes
npm run test:unit passes (or new tests added for the changed component)
No new type errors vs. the baseline
Storybook story updated/added if you touched a base/ component

Your data. Your AI.

AutoBot's frontend reflects the same philosophy as the project: everything runs locally, nothing is sent to a third party, and every part of the stack is open for you to inspect, extend, or replace.

If this post helped you find your way around the codebase, the best next step is to open an issue or pick one that's already waiting.

Links

Self-Hosting AutoBot: A DevOps Deep Dive into Docker Compose, Model Sizing, and Production Ops

Mārtiņš Veiss — Mon, 11 May 2026 11:31:40 +0000

You've seen the demos. You want to run AutoBot on your own hardware, your own data, under your own control. Good instinct. Here's the full operational picture — Docker Compose internals, how to match LLM models to your GPU or CPU, and the production habits that keep things stable long-term.

Why Self-Host?

AutoBot's tagline is "Your data. Your AI." That's not marketing copy — it's an architectural choice. When you self-host:

Conversations never leave your network
You choose which models run (open-weight, cloud API, or a mix)
Upgrade timing is yours to control
No per-seat pricing surprises

The trade-off is operational responsibility. This post is about making that trade-off comfortable.

Docker Compose Deep Dive

AutoBot ships with a docker-compose.yml that wires together several services. Let's walk through each layer.

Services Overview

services:
  backend:
    build: ./backend
    ports: ["8000:8000"]
    depends_on: [chromadb, redis]
    environment:
      - OLLAMA_HOST=http://ollama:11434
      - CHROMA_HOST=chromadb
      - REDIS_URL=redis://redis:6379

  frontend:
    build: ./frontend
    ports: ["3000:3000"]
    depends_on: [backend]

  chromadb:
    image: chromadb/chroma:latest
    volumes:
      - chroma_data:/chroma/chroma

  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    command: redis-server --appendonly yes

  ollama:
    image: ollama/ollama:latest
    volumes:
      - ollama_models:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  chroma_data:
  redis_data:
  ollama_models:

What Each Service Does

backend — FastAPI application. Handles chat sessions, RAG retrieval, fleet management. The OLLAMA_HOST env var points it at your local model server; swap this for an OpenAI-compatible URL to use a cloud LLM instead.

frontend — Next.js UI. Talks only to the backend on port 8000. Stateless — you can restart it without losing anything.

chromadb — Vector database for knowledge bases. Your embedded documents live here. The chroma_data volume is critical — back it up.

redis — Session state and task queues. With --appendonly yes, Redis persists to disk. Losing this volume means losing active session context (but not your knowledge bases).

ollama — Local LLM inference server. Holds downloaded model weights in ollama_models. Models are large (4–70 GB each); this volume is expensive to rebuild.

Networking

All services communicate on a default Docker bridge network. The service names (chromadb, redis, ollama) resolve as hostnames inside the network — that's why the backend config uses http://ollama:11434 rather than localhost.

For a production deployment, consider an explicit network definition:

networks:
  autobot_net:
    driver: bridge

services:
  backend:
    networks: [autobot_net]
  # ... same for all services

This lets you add an Nginx reverse proxy or Traefik on the same network without exposing internal ports.

Model Sizing to Hardware

This is where most self-hosting guides go wrong — they talk about VPS pricing instead of the actual constraint: inference throughput vs. memory bandwidth.

The Rule of Thumb

A model running entirely in VRAM is fast. A model that spills to RAM (or worse, disk) is slow. Plan your setup so your primary model fits in VRAM with room for the OS and other processes.

Hardware	VRAM	Practical Model Ceiling
RTX 3060	12 GB	Llama 3 8B (Q4), Mistral 7B
RTX 3090 / 4090	24 GB	Llama 3 70B (Q4 at the edge), Llama 3 8B (full precision)
2× A100 80 GB	160 GB	Llama 3 70B (full), most open-weight frontier models
CPU only (32 GB RAM)	—	Llama 3 8B (Q4, slow) — workable for low-traffic RAG

Local Ollama vs. Cloud LLM Trade-offs

AutoBot supports both. Here's how to think about the choice:

Local Ollama (default)

Zero per-token cost
Private by definition
Latency depends on your hardware
Best for: high-volume internal tools, sensitive data, experimentation

Cloud LLM (OpenAI, Anthropic, etc.)

Pay per token
Faster for large models you can't run locally
Data leaves your network (check your provider's retention policy)
Best for: production apps that need frontier model quality without buying GPUs

The OLLAMA_HOST env var makes switching simple. Point it at https://api.openai.com/v1 (with an OpenAI-compatible wrapper) to route through a cloud provider without touching application code.

Practical Model Recommendations

For a RAG-heavy knowledge base workload (most AutoBot deployments): a quantized 8B model (Llama 3.1 8B Q4_K_M) hits the sweet spot — fast enough for real-time chat, accurate enough for document retrieval, fits comfortably on a single consumer GPU.

For a multi-agent fleet workload: consider running a smaller model (3B–7B) per agent node and reserving a larger model for orchestration decisions. AutoBot's fleet manager is built to handle per-agent model config.

Production Tips

Backups

The three volumes that matter:

# ChromaDB — your knowledge bases
docker run --rm \
  -v autobot_chroma_data:/source \
  -v /backup:/backup \
  alpine tar czf /backup/chroma-$(date +%Y%m%d).tar.gz -C /source .

# Redis — session state
docker exec autobot-redis-1 redis-cli BGSAVE
docker cp autobot-redis-1:/data/dump.rdb /backup/redis-$(date +%Y%m%d).rdb

# Ollama models — large, but painful to re-download
docker run --rm \
  -v autobot_ollama_models:/source \
  -v /backup:/backup \
  alpine tar czf /backup/ollama-$(date +%Y%m%d).tar.gz -C /source .

Run chroma and redis backups daily. Ollama models only change when you pull new ones — back up on change, not on schedule.

Upgrades

# Pull latest images
docker compose pull

# Recreate containers (zero-downtime if you add a load balancer)
docker compose up -d --no-deps --build backend frontend

# Full restart (brief downtime)
docker compose down && docker compose up -d

Pin image tags in production (chromadb/chroma:0.5.3 not latest) so upgrades are deliberate, not automatic.

Monitoring

AutoBot's backend exposes a /health endpoint. Wire it into your monitoring stack:

# Simple cron healthcheck
*/5 * * * * curl -sf http://localhost:8000/health || notify-oncall

For metrics, the backend emits structured logs to stdout. Forward them to Loki, Datadog, or whatever you already use:

  backend:
    logging:
      driver: "json-file"
      options:
        max-size: "50m"
        max-file: "5"

Watch for these signals:

ChromaDB query latency > 2s — index fragmentation or under-resourced container
Redis memory approaching limit — set maxmemory and a sensible eviction policy (allkeys-lru)
Ollama inference time spiking — model being swapped to RAM; consider reducing context length or switching to a smaller quantization

What's Next

Self-hosting is the start, not the finish. Once you're running in production, the interesting work is building knowledge bases, connecting data sources, and wiring up agents for your specific workflows.

If you want to help make AutoBot better at the infrastructure layer, there are open issues tagged for DevOps contributors:

→ Good first issues — DevOps label on AutoBot-AI

If AutoBot is saving you money or time on your infra, consider supporting development:

→ Ko-fi: ko-fi.com/mrveiss

Questions, corrections, or war stories from your own deployment — drop them in the comments.

AutoBot's RAG Pipeline Internals — A Python Developer's Guide

Mārtiņš Veiss — Mon, 11 May 2026 11:30:31 +0000

If you've been watching the local-AI space lately, you've probably seen OpenClaw land 100k GitHub stars on the back of autonomous agents that build their own tools, their own social networks, and — if you're not careful — their own threat models.

AutoBot takes a different approach: you stay in control. Your data never leaves your machine. Your AI runs on your hardware. And the knowledge base — the thing that makes your local AI actually useful — is something you can read, extend, and contribute to.

This post is for Python developers who want to understand exactly how that knowledge base works, how to feed it your own codebase, and where to plug in if you want to help build it.

The Stack at a Glance

AutoBot's RAG pipeline is built on three components:

Layer	Technology	Role
Embedding model	Ollama (configurable)	Text → vectors
Vector store	ChromaDB	Similarity search
Retrieval + generation	LlamaIndex	Query → answer

All of it runs locally. No API calls. No data leaving your machine.

The main module lives at autobot-backend/knowledge/. The legacy knowledge_base.py at the backend root is a thin re-export shim — all real logic is in the knowledge/ package.

End-to-End Pipeline Walk-Through

1. Document Ingestion

Entry point: knowledge/documents.py — DocumentsMixin.add_document()

# knowledge/documents.py
async def add_document(
    self,
    content: str,
    metadata: Dict[str, Any] = None,
    doc_id: Optional[str] = None,
) -> Dict[str, Any]:
    """Add a document to the knowledge base with async processing."""

When you drop a file into AutoBot, this is what happens:

Content arrives — plain text, Markdown, or PDF.
Chunking — the document is split into overlapping chunks so context is preserved at retrieval time.
Embedding — each chunk is converted to a 768-dimensional float vector by the configured Ollama model.
Storage — vectors + original text land in ChromaDB, keyed by a stable document ID.

The embedding call goes through knowledge/embedding_cache.py (EmbeddingCache), which deduplicates repeated content and tracks usage via api/analytics_embedding_patterns.py (Issue #285). Cache hits skip the Ollama round-trip entirely — useful when you re-index after editing a doc.

2. Index Configuration

Entry point: knowledge/index.py — IndexMixin

ChromaDB uses HNSW (Hierarchical Navigable Small World) for approximate nearest-neighbour search. AutoBot exposes the tuning parameters directly:

# knowledge/index.py
def _get_hnsw_metadata(self) -> Dict[str, Any]:
    return {
        "hnsw:space": self.hnsw_space,           # distance metric (cosine by default)
        "hnsw:construction_ef": self.hnsw_construction_ef,
        "hnsw:search_ef": self.hnsw_search_ef,
        "hnsw:M": self.hnsw_m,
    }

The current defaults are tuned for collections with 545k+ vectors (Issue #72). If you're running on modest hardware with a small KB, you can tighten hnsw:M to reduce memory pressure.

All ChromaDB calls are wrapped with asyncio.to_thread() (Issue #369) to keep the FastAPI event loop unblocked — something to be aware of if you're adding new index operations.

3. Query → Answer

Entry point: knowledge/base.py — KnowledgeBaseCore

On query:

The question is embedded with the same Ollama model used at ingestion (same vector space = valid similarity).
HNSW search finds the top-k most similar chunks.
The chunks are passed to LlamaIndex as context alongside the query.
LlamaIndex sends the augmented prompt to the local Ollama LLM.
The answer references your documents, not generic training data.

# knowledge/base.py — core wiring
from llama_index.core import Settings, VectorStoreIndex
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.llms.ollama import Ollama
from llama_index.vector_stores.chroma import ChromaVectorStore

4. Advanced Retrieval

Entry point: autobot-backend/advanced_rag_optimizer.py

For complex queries, AutoBot can upgrade from plain vector search to a hybrid pipeline:

Hybrid scoring — blends semantic similarity (HNSW cosine) with BM25 keyword score via knowledge/search_components/reranking.py.
Query expansion — reformulates the question to improve recall on technical vocabulary mismatches.
MAP-Elites diversification — ensures results span multiple knowledge categories rather than returning near-duplicate chunks.
GPU acceleration — utils/semantic_chunker_gpu.py uses RTX 4070 / OpenVINO where available.

The SearchResult dataclass in advanced_rag_optimizer.py carries both the raw content and all four score dimensions (semantic_score, keyword_score, hybrid_score, rerank_score) — useful if you want to instrument retrieval quality.

5. Background Vectorization

Entry point: autobot-backend/background_vectorization.py — BackgroundVectorizer

When you add new facts or documents while AutoBot is running, BackgroundVectorizer picks them up asynchronously via FastAPI background tasks. You don't have to trigger a full re-index — the KB stays live.

Feeding Your Codebase to the Knowledge Base

AutoBot has a dedicated CodeEmbeddingGenerator (autobot-backend/code_embedding_generator.py) that uses CodeBERT instead of a generic text embedding model. Code has different semantics than prose — function names, types, and structure matter — and CodeBERT is trained on code.

# code_embedding_generator.py
@dataclass
class CodeEmbeddingResult:
    embedding: np.ndarray       # 768-dim CodeBERT vector
    device_used: str            # 'npu', 'cuda', or 'cpu'
    processing_time_ms: float
    model_name: str
    cache_hit: bool

To index your codebase:

Option 1 — Via the chat UI

You: Index the ./src directory into the knowledge base
AutoBot: ✓ Scanning ./src...
         Indexed 847 functions across 63 files
         Embedding device: NPU (OpenVINO)
         Ready for semantic code search

Option 2 — Via the connector system

The knowledge/connectors/ directory has a registry (registry.py) and a scheduler (scheduler.py). You can register a file-server connector pointing at your repo root and let AutoBot watch for changes:

# knowledge/connectors/file_server.py
# Register your source directory as a watched connector
connector = FileServerConnector(
    root_path="/path/to/your/repo",
    watch=True,
    file_extensions=[".py", ".md", ".yaml"],
)

Option 3 — Notion, web, database

Connectors also exist for Notion (notion.py), web crawl (web_crawler.py), audio (audio_connector.py), and database (database.py). The base class is knowledge/connectors/base.py — implement fetch() and register via registry.py.

Where to Plug In: Contributing to the KB Engine

Here are the cleanest entry points for first contributions:

`knowledge/documents.py` — DocumentsMixin

Good for: adding new file format support (EPUB, HTML, DOCX), improving chunking strategy.

The add_document() and related methods are well-isolated. A chunking improvement here applies to every ingestion path.

`knowledge/connectors/` — Connector Registry

Good for: adding new data sources (GitHub issues, Jira, Slack export).

Implement the BaseConnector interface and register in registry.py. Look at notion.py for a reference implementation with authentication handling.

`advanced_rag_optimizer.py` — Hybrid Search

Good for: retrieval quality improvements, new reranking strategies, better query expansion.

The SearchResult + QueryContext dataclasses are clean — adding a new scoring dimension means extending the dataclass and wiring it into compute_blended_score() in knowledge/search_components/reranking.py.

`knowledge/index.py` — HNSW Tuning

Good for: performance work on large vector collections, memory footprint reduction.

The HNSW parameter exposure is deliberately simple. There's room for adaptive tuning based on collection size and hardware profile.

`background_vectorization.py` — BackgroundVectorizer

Good for: incremental sync improvements, smarter deduplication, conflict resolution when a connector and a manual upload touch the same document.

Running the KB Locally

# Clone and start the full stack
git clone https://github.com/mrveiss/AutoBot-AI
cd AutoBot-AI
docker compose up -d

# Or use the installer script
curl -fsSL https://raw.githubusercontent.com/mrveiss/AutoBot-AI/Dev_new_gui/install.sh | bash

The knowledge base stores vectors in ./data/chromadb/ by default. It persists across container restarts.

To run just the backend in dev mode:

cd autobot-backend
pip install -r requirements.txt
uvicorn app_factory:create_app --factory --reload --port 8000

Where to Go Next

If you want to contribute to the Python side:

Good first issues (Python label): github.com/mrveiss/AutoBot-AI/labels/python
All good first issues: github.com/mrveiss/AutoBot-AI/labels/good%20first%20issue
Contributing guide: CONTRIBUTING.md
GitHub Discussions: github.com/mrveiss/AutoBot-AI/discussions

If this article saved you an hour of reading source code, you can buy me a coffee on Ko-fi — it goes directly toward hardware time for the project.

AutoBot is free, open source, and runs entirely on your hardware. The RAG pipeline is the core of what makes a local AI assistant actually useful — and it's a great place to dig in.

Your data. Your AI.

→ github.com/mrveiss/AutoBot-AI

OpenClaw and AutoBot: two different visions for local AI

Mārtiņš Veiss — Mon, 11 May 2026 11:21:25 +0000

OpenClaw hit 100,000 GitHub stars in two months. Its agents built their own social network. PCWorld and TechCrunch ran pieces on the risks. If you've been anywhere near AI Twitter this week, you've seen the wave.

I've been building AutoBot for three years. People keep asking me the same question: is this your competitor?

It's not. We're solving different problems.

This piece is for the developers I keep meeting who are excited by OpenClaw and unsettled by it at the same time. There's a real reason for that feeling — and there's room in the local-AI world for both projects to exist.

Two philosophies, one ecosystem

OpenClaw is about agent autonomy. You give the agent goals, system access, and time. It figures out the rest. The whole point is that you're not in the loop for every step.

AutoBot is about data sovereignty. You feed it your docs, your codebase, your business knowledge. It answers questions, drafts copy, helps you code — but it does what you ask, when you ask, on your machine.

Different problems. Different trade-offs. Both legitimate.

The PCWorld and TechCrunch coverage didn't say OpenClaw was bad. It said autonomous agents with system-level permissions are a category of risk we don't have great answers for yet. That's true. It's also the price of admission for what OpenClaw is trying to do, and a lot of people will pay it gladly.

Some won't. Those are the people I want to talk to.

What "Your data. Your AI." actually means

The line we built AutoBot around is Your data. Your AI. Here's what that resolves to in code:

Your data stays on your machine. The knowledge base — the documents you upload, the codebase you index, the business processes you paste in — never leaves your hardware. There is no cloud component. There is no telemetry pipe. If your machine is offline, AutoBot is offline.

You pick the brain. Want to run fully local? Plug in Ollama, LM Studio, llama.cpp — anything with an OpenAI-compatible endpoint. Want GPT-4 or Claude for the heavy lifting? Connect your API key. Your prompts go to that model, but your knowledge base documents don't.

The brain phones home. Your documents don't. That's the line.

You decide what it does. AutoBot doesn't run on a schedule. It doesn't take actions while you sleep. It doesn't have system access beyond what its container can see. The trade-off: you have to ask. The benefit: nothing happens that you didn't ask for.

When you'd pick which

I'm not going to pretend AutoBot is the answer for everything. Picking the right tool matters more than picking a side.

OpenClaw fits when:

You want long-running, multi-step automation
You're comfortable scoping permissions and accepting agent risk
The win is the agent doing things without you in the loop

AutoBot fits when:

Your data can't leave your network (regulated industries, proprietary code, client work)
You want an AI that knows your domain, not a generic model
You want to keep the human in the loop — the AI is a tool, not a coworker
You need something you can deploy once and run forever

There are people who will run both. AutoBot for the knowledge base and chat layer over their own data. OpenClaw for autonomous tasks where they've scoped the risk. That's a legitimate stack.

What we actually built

Because I keep getting asked: AutoBot is a self-hosted AI platform. The chat interface gets the attention but the knowledge base is the product.

RAG engine that turns your raw files into a searchable AI layer that knows your domain
Pluggable LLM — local via Ollama, or any OpenAI-compatible endpoint
Fleet management for running AutoBot across multiple machines
Docker Compose deploy — one command, full stack

It's open source. It's actively developed. The roadmap is public. Community PRs are welcome and tagged with skill-based good-first-issue labels for Python, frontend, and DevOps contributors.

Try it in five minutes

git clone https://github.com/mrveiss/AutoBot-AI.git
cd AutoBot-AI
docker compose up -d

Open http://localhost:3000. Connect your LLM. Feed it your first document.

That's it. You're running your own AI.

If the OpenClaw moment got you thinking harder about where your data lives and who controls your AI — even if you stay on OpenClaw — that's a good thing for the ecosystem. We need more people asking those questions.

If the answer you land on is I want the AI but I want to stay in control, AutoBot is here for that.

Star us on GitHub · Join the discussions · Sponsor the project

Weekly Update: ✨ docs: refresh stale status/changelog and canonical TaskSta

Mārtiņš Veiss — Mon, 11 May 2026 05:00:06 +0000

Weekly Update: ✨ docs: refresh stale status/changelog and canonical TaskStatus examples (#7498)

This week we shipped:

🔧 docs: refresh stale status/changelog and canonical TaskStatu
🔧 feat(web_fetch): add WebFetcher.fetch_raw_html public API (c
🔧 fix(multimodal): LSP exception contract — VisionProcessor +

Contributors: +1

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-05-04T05:00:03.384055Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

Weekly Update: ✨ fix(backend): replace bare singleton aliases with get_*()

Mārtiņš Veiss — Mon, 27 Apr 2026 05:00:10 +0000

Weekly Update: ✨ fix(backend): replace bare singleton aliases with get_*() pattern at all call sites (#6196) (#6217)

This week we shipped:

🔧 fix(backend): replace bare singleton aliases with get_*() pa
🔧 fix(api): move @with_error_handling below @router.* in workf
🔧 fix(deploy): add Ansible task to clear stale .pyc files afte

Contributors: +2

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-04-20T05:00:04.313917Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

Why We Built AutoBot: The WordPress of AI

Mārtiņš Veiss — Wed, 15 Apr 2026 19:39:01 +0000

Three years ago I started building AutoBot because I was tired of renting my own intelligence.

Every AI tool I used followed the same playbook: send your data to our servers, pay monthly, accept our terms, trust us not to read your prompts. The model was the product. You were the user. Your data was the inventory.

I wanted something different. I wanted an AI that felt like mine.

Your data. Your AI.

That's the line we built everything around.

WordPress gave everyone a website. Before WordPress, having a web presence meant renting space on someone else's platform, playing by their rules, losing your content if they shut down. WordPress flipped that. You install it, you own it, you extend it, you run it forever.

AutoBot is that for AI.

Self-hosted. Open source. Yours to extend. The AI platform that belongs to you — not to us.

Feed it your world

The core feature isn't the chat interface. It's the knowledge base.

Drop in your docs. Upload your codebase. Paste your business processes. AutoBot's RAG engine turns your raw files into a searchable, queryable AI layer that actually knows your domain — not just what some model was trained on two years ago.

Feed it your docs. Your codebase. Your business.
It learns what you know. It stays where you are.

This is the part that changes things. An AI that knows your codebase gives better answers than a generic model. An AI that's read your legal documents is more useful than one guessing at your jurisdiction. An AI trained on your patient intake forms is more reliable than one pattern-matching across the internet.

An AI that's actually about you.

Pick your brain

AutoBot is not opinionated about which LLM powers it. That's your call.

Want to run fully local? Plug in Ollama. LM Studio. llama.cpp. Anything with an OpenAI-compatible endpoint.

Prefer GPT-4 or Claude for the heavy lifting? Connect your API key. Your data stays on your machine — your prompts go to the model, but your knowledge base documents don't.

That distinction matters. The brain phones home. Your documents don't.

You shouldn't have to ask permission

The thing that finally broke me on cloud AI wasn't the pricing. It was the 2 AM email.

"We're updating our terms of service effective next month. By continuing to use the service you agree to..."

Your data never leaves your machine — whatever brain powers it.
No rate limits on your knowledge. No vendor changing the rules on you overnight.

Deploy once. Run it your way. Forever.

Most AI is rented. AutoBot is yours.

You decide what it does. You decide where it runs. You decide who sees it.

No subscription. No surveillance. No one reading your prompts.
Install it. Own it. Run it forever.

For developers who've been burned by API deprecations, pricing pivots, and terms changes — this is for you.

For law firms, medical startups, and anyone in a regulated industry — your data never touches our servers. No cloud vendor to breach. No third party holding your keys. What stays on your machine stays yours — full stop. You control the perimeter. We just give you the tools.

Get started in 5 minutes

git clone https://github.com/mrveiss/AutoBot-AI.git
cd AutoBot-AI
docker compose up -d

Open http://localhost:3000. Connect your LLM. Feed your first document.

That's it. You're running your own AI.

AutoBot is open source. If this resonates, star us on GitHub, join the community discussions, or sponsor the project.

Weekly Update: ✨ docs(claude.md): add codebase-as-source-of-truth rule

Mārtiņš Veiss — Mon, 13 Apr 2026 05:00:06 +0000

Weekly Update: ✨ docs(claude.md): add codebase-as-source-of-truth rule

This week we shipped:

🔧 docs(claude.md): add codebase-as-source-of-truth rule
🔧 Merge branch 'main' into Dev_new_gui
🔧 fix(devops): ensure log directory/files have correct ownersh

Contributors: +2

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-04-06T05:00:03.370813Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

Weekly Update: ✨ WIP: preserve work from issue-3291 (#4241)

Mārtiņš Veiss — Sun, 12 Apr 2026 19:33:43 +0000

Weekly Update: ✨ WIP: preserve work from issue-3291 (#4241)

This week we shipped:

🔧 WIP: preserve work from issue-3291 (#4241)
🔧 WIP: preserve work from issue-3290 (#4240)
🔧 WIP: preserve work from issue-3281 (#4239)

Contributors: +2

→ Full changelog: https://github.com/mrveiss/AutoBot-AI/commits/Dev_new_gui?since=2026-04-05T16:46:53.357011Z
→ Discuss on GitHub: https://github.com/mrveiss/AutoBot-AI/discussions

DEV Community: Mārtiņš Veiss

Weekly Update: ✨ feat(backend): lightweight inference mode — bypass RAG/mem

Weekly Update: ✨ feat(backend): lightweight inference mode — bypass RAG/memory for trivial tier (MVA-1992) (#9160)

Running AI in regulated environments: how AutoBot keeps your documents on-premise

The problem with cloud AI in regulated industries

AutoBot's data model

Network isolation in practice

1. Bind to localhost, not all interfaces

2. Internal service network

3. Resource limits

What goes where: the data flow audit

Getting started in a restricted environment

Compliance checklist before production

The overhead question

The bottom line

Weekly Update: ✨ fix(llc): address LLC API/model gaps (#8479 #8478 #8476 #8

Weekly Update: ✨ fix(llc): address LLC API/model gaps (#8479 #8478 #8476 #8474 #8462 #8461 #8493)

Inside AutoBot's Frontend: A Developer Walkthrough

What you see when you open AutoBot

Chat UI

Knowledge Base UI

Component architecture in autobot-frontend/

Design tokens

How to contribute to the UI

Get the repo running

Explore Storybook

Find good first issues

Testing approach

PR checklist

Your data. Your AI.

Self-Hosting AutoBot: A DevOps Deep Dive into Docker Compose, Model Sizing, and Production Ops

Why Self-Host?

Docker Compose Deep Dive

Services Overview

What Each Service Does

Networking

Model Sizing to Hardware

The Rule of Thumb

Local Ollama vs. Cloud LLM Trade-offs

Practical Model Recommendations

Production Tips

Backups

Upgrades

Monitoring

What's Next

AutoBot's RAG Pipeline Internals — A Python Developer's Guide

The Stack at a Glance

End-to-End Pipeline Walk-Through

1. Document Ingestion

2. Index Configuration

3. Query → Answer

4. Advanced Retrieval

5. Background Vectorization

Feeding Your Codebase to the Knowledge Base

Where to Plug In: Contributing to the KB Engine

knowledge/documents.py — DocumentsMixin

knowledge/connectors/ — Connector Registry

advanced_rag_optimizer.py — Hybrid Search

knowledge/index.py — HNSW Tuning

background_vectorization.py — BackgroundVectorizer

Running the KB Locally

Where to Go Next

OpenClaw and AutoBot: two different visions for local AI

Two philosophies, one ecosystem

What "Your data. Your AI." actually means

When you'd pick which

What we actually built

Try it in five minutes

Weekly Update: ✨ docs: refresh stale status/changelog and canonical TaskSta

Weekly Update: ✨ docs: refresh stale status/changelog and canonical TaskStatus examples (#7498)

Weekly Update: ✨ fix(backend): replace bare singleton aliases with get_*()

Weekly Update: ✨ fix(backend): replace bare singleton aliases with get_*() pattern at all call sites (#6196) (#6217)

Why We Built AutoBot: The WordPress of AI

Your data. Your AI.

Feed it your world

Pick your brain

You shouldn't have to ask permission

Most AI is rented. AutoBot is yours.

Get started in 5 minutes

Weekly Update: ✨ docs(claude.md): add codebase-as-source-of-truth rule

Component architecture in `autobot-frontend/`

`knowledge/documents.py` — DocumentsMixin

`knowledge/connectors/` — Connector Registry

`advanced_rag_optimizer.py` — Hybrid Search

`knowledge/index.py` — HNSW Tuning

`background_vectorization.py` — BackgroundVectorizer