Hamdy ELbatal

Posted on Oct 12

Building Local RAG Workflows in Laravel with NativeRAG

#laravel #php #ai #rag

AI features are becoming normal in web applications, but a lot of teams still hesitate when the first step requires sending private application data to an external API.

That is the problem NativeRAG tries to solve for Laravel developers.

NativeRAG is a Laravel package for building small to medium retrieval-augmented generation workflows using local models through Ollama or LM Studio. It gives you chat completions, embeddings, vector search, streaming responses, model indexing, and conversation persistence without needing a separate vector database to get started.

The goal is simple: keep the Laravel developer experience familiar while making local AI workflows easier to wire into a real application.

Why Local RAG?

Retrieval-augmented generation usually has a few moving parts:

a chat model
an embedding model
document chunking
vector storage
similarity search
prompt construction
conversation history

In many setups, those pieces quickly turn into multiple services. That can be the right architecture at scale, but it is not always what you want on day one.

Sometimes you just need to index Eloquent models, search relevant chunks, and ask a local model a question using that context.

NativeRAG keeps those pieces inside Laravel as much as possible.

What NativeRAG Provides

The package currently supports:

Ollama and LM Studio drivers
chat completions
embedding generation
server-sent event streaming
database-backed embedding storage
SQLite and PHP-side cosine similarity search
Eloquent model indexing through an Embeddable trait
persistent conversations
count-based and token-based history pruning
optional encrypted message storage
Artisan commands for indexing and health checks

It is not trying to replace a dedicated vector database for every use case. Instead, it gives Laravel applications a practical starting point for local RAG features.

Installation

Install the package with Composer:

composer require hamzi/nativerag

Then publish the configuration and migrations:

php artisan vendor:publish --tag="nativerag-config"
php artisan vendor:publish --tag="nativerag-migrations"
php artisan migrate

The package stores embeddings and conversations in your application database.

Configuration

You can use Ollama as the default driver:

NATIVE_RAG_DRIVER=ollama

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_CHAT_MODEL=llama3
OLLAMA_EMBEDDING_MODEL=nomic-embed-text

Or LM Studio:

NATIVE_RAG_DRIVER=lmstudio

LMSTUDIO_BASE_URL=http://localhost:1234
LMSTUDIO_CHAT_MODEL=meta-llama-3-8b-instruct
LMSTUDIO_EMBEDDING_MODEL=nomic-embed-text

You can also tune chunking and search behavior:

NATIVE_RAG_CHUNK_SIZE=1000
NATIVE_RAG_CHUNK_OVERLAP=200
NATIVE_RAG_MIN_SCORE=0.35

Chat Completions

The simplest use case is sending messages to a local chat model:

use Hamzi\NativeRag\Facades\NativeRag;

$response = NativeRag::chat([
    [
        'role' => 'system',
        'content' => 'You are a concise Laravel assistant.',
    ],
    [
        'role' => 'user',
        'content' => 'Explain service containers in simple terms.',
    ],
]);

echo $response->content;

This keeps the API close to common chat message formats while still using Laravel's facade and manager patterns under the hood.

Indexing Eloquent Models

One of the most useful parts of the package is the Embeddable trait.

You can make a model searchable by implementing EmbeddableContract and returning the text that should be embedded:

namespace App\Models;

use Hamzi\NativeRag\Contracts\EmbeddableContract;
use Hamzi\NativeRag\Traits\Embeddable;
use Illuminate\Database\Eloquent\Model;

class Article extends Model implements EmbeddableContract
{
    use Embeddable;

    public function toEmbeddableString(): string
    {
        return "Title: {$this->title}\n\nContent: {$this->content}";
    }
}

When the model is saved, NativeRAG can chunk the text, generate embeddings, and store them in the embeddings table.

This is especially useful for documentation, knowledge bases, product content, internal notes, support articles, and other text-heavy records.

Searching Indexed Content

Once content is indexed, you can search using either a plain text query or a precomputed embedding vector:

use Hamzi\NativeRag\Facades\NativeRag;

$results = NativeRag::search(
    query: 'How do I run database migrations?',
    limit: 5,
    minScore: 0.4,
);

foreach ($results as $chunk) {
    echo $chunk->chunk_content;
}

For many Laravel applications, this is enough to start building contextual AI features without introducing more infrastructure.

Building A RAG Prompt

A common flow looks like this:

Receive a user question.
Search indexed chunks.
Combine the matching chunks into context.
Send the context and question to the model.

Example:

use Hamzi\NativeRag\Facades\NativeRag;
use Hamzi\NativeRag\Services\PromptCompiler;

$question = 'How can users reset their password?';

$chunks = NativeRag::search($question, limit: 4);

$context = $chunks
    ->pluck('chunk_content')
    ->implode("\n\n---\n\n");

$prompt = app(PromptCompiler::class)->buildRagPrompt(
    systemInstruction: 'Answer using only the provided context.',
    context: $context,
    query: $question,
);

$response = NativeRag::chat([
    ['role' => 'user', 'content' => $prompt],
]);

echo $response->content;

This pattern is intentionally straightforward. You can replace the prompt, filtering, scoring threshold, or context formatting based on your application's needs.

Streaming Responses

For UI experiences, waiting for a full response can feel slow. NativeRAG supports streaming through server-sent events:

use Hamzi\NativeRag\Facades\NativeRag;
use Illuminate\Support\Facades\Route;

Route::post('/ai/stream', function () {
    return NativeRag::stream([
        [
            'role' => 'user',
            'content' => request('message'),
        ],
    ]);
});

That makes it easier to build chat interfaces with Livewire, Alpine.js, Vue, React, or any frontend that can consume an event stream.

Conversation Memory

NativeRAG also includes conversation models for persistent chat history:

use Hamzi\NativeRag\Models\NativeRagConversation;

$conversation = NativeRagConversation::create([
    'name' => 'Support session',
]);

$conversation->addSystemMessage('You are a helpful support assistant.');

$reply = $conversation->ask('How do I publish the config file?');

echo $reply->content;

The conversation stores user and assistant messages, then sends the current history to the selected driver.

History can be pruned by message count or an approximate token budget. System messages can also be preserved so the assistant keeps its instructions.

Queueing Embedding Sync

Embedding generation can take time, especially when indexing larger records. NativeRAG can dispatch embedding sync to Laravel queues:

NATIVE_RAG_QUEUE_INDEXING=true
NATIVE_RAG_QUEUE_CONNECTION=redis
NATIVE_RAG_QUEUE_NAME=default

That lets normal application requests stay responsive while workers handle chunking and embedding generation in the background.

Privacy And Tradeoffs

Running models locally gives you more control over where prompts, documents, and embeddings live. That is useful for internal tools and privacy-sensitive applications.

But local RAG also has tradeoffs:

local models need enough CPU, GPU, and memory
embedding quality depends on the selected model
database-backed vector search is convenient, but not a replacement for specialized vector infrastructure at high scale
prompt design still matters
evaluation is still your responsibility

The package gives you the Laravel plumbing. You still need to choose good models, test outputs, and decide what quality bar makes sense for your product.

Where This Package Fits

NativeRAG is a good fit when you want to:

prototype AI features inside an existing Laravel app
build internal assistants over application data
index Eloquent models without standing up another service
keep model calls local through Ollama or LM Studio
experiment with RAG before committing to heavier infrastructure

It may not be the right fit if you already need distributed vector indexing, advanced ranking pipelines, or large-scale retrieval over millions of chunks.

Final Thoughts

What I like about this approach is that it keeps the first version boring in the best way.

You install a Laravel package, publish config and migrations, add a trait to a model, and start searching your own data. The architecture can evolve later, but the early developer experience stays simple.

For many teams, that is exactly the right place to start.

Repository:

https://github.com/hamdyelbatal122/nativerag

Package:

composer require hamzi/nativerag

DEV Community