** Building Document Q&A with Laravel Embeddings and Vector Stores
**
If you've shipped a Laravel app before, you already know how to query a database. Building a document Q&A system is not fundamentally different — you're still querying for relevant records, except the "similarity" between a question and a chunk of text is measured as a cosine distance between floating-point vectors, not a WHERE title = ? clause. The Laravel AI SDK (v0.8.1, stable since Laravel 13) adds two things that make this tractable without leaving your Eloquent comfort zone: a toEmbeddings() method on Stringable and a whereVectorSimilarTo() scope on your models.
This walkthrough builds a fully functional document Q&A feature — upload a document, chunk it, embed each chunk, store it in PostgreSQL with pgvector, then answer user questions grounded in the document text. For a broader look at everything the SDK covers (agents, MCP, multi-agent pipelines, audio, images), see the Laravel AI SDK: Complete Guide to Building AI Applications.
Prerequisites
- PHP 8.3+
- Laravel 12.x or 13.x
-
laravel/aiv0.8.x installed - PostgreSQL with the
pgvectorextension enabled (CREATE EXTENSION vector;) - An embedding provider configured — OpenAI (
text-embedding-3-small) is used throughout, but VoyageAI and Jina work via the same API
Not compatible with Laravel 11 or PHP 8.2. The
toEmbeddings()helper lives inilluminate/support ^12.62|^13.15.
Step 1 — Database Schema for Chunks
Create a migration for your document chunks table. The embedding column is a pgvector vector type with 1536 dimensions (matching text-embedding-3-small). If you switch to text-embedding-3-large, change it to 3072.
// database/migrations/2026_08_01_000001_create_document_chunks_table.php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
use Illuminate\Support\Facades\DB;
return new class extends Migration
{
public function up(): void
{
DB::statement('CREATE EXTENSION IF NOT EXISTS vector');
Schema::create('document_chunks', function (Blueprint $table) {
$table->id();
$table->foreignId('document_id')->constrained()->cascadeOnDelete();
$table->integer('chunk_index');
$table->text('content');
$table->timestamps();
});
// pgvector column — Blueprint doesn't know this type natively
DB::statement(
'ALTER TABLE document_chunks ADD COLUMN embedding vector(1536)'
);
// IVFFlat index for approximate nearest-neighbour search
DB::statement(
'CREATE INDEX ON document_chunks USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100)'
);
}
public function down(): void
{
Schema::dropIfExists('document_chunks');
}
};
The IVFFlat index with lists = 100 is a reasonable starting point for up to ~1 million chunks. For datasets below 10,000 rows, skip the index — a sequential scan with vector_cosine_ops is faster at small scale.
Step 2 — The DocumentChunk Model
// app/Models/DocumentChunk.php
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\BelongsTo;
class DocumentChunk extends Model
{
protected $fillable = ['document_id', 'chunk_index', 'content', 'embedding'];
// Cast the stored vector string back to an array when reading
protected $casts = [
'embedding' => 'array',
];
public function document(): BelongsTo
{
return $this->belongsTo(Document::class);
}
}
Step 3 — Chunking and Embedding on Upload
Text chunking strategy matters more than people expect. Splitting on paragraph boundaries preserves context better than a fixed character count. A 500-token target with 50-token overlap is a solid default for general documents.
// app/Services/DocumentEmbeddingService.php
namespace App\Services;
use App\Models\Document;
use App\Models\DocumentChunk;
use Illuminate\Support\Str;
class DocumentEmbeddingService
{
// Chunk a plain-text document into overlapping segments
public function chunk(string $text, int $targetChars = 1500, int $overlapChars = 150): array
{
$paragraphs = preg_split('/\n{2,}/', trim($text));
$chunks = [];
$current = '';
foreach ($paragraphs as $paragraph) {
if (strlen($current) + strlen($paragraph) > $targetChars && $current !== '') {
$chunks[] = trim($current);
// Carry overlap forward
$current = substr($current, -$overlapChars) . "\n\n" . $paragraph;
} else {
$current .= ($current === '' ? '' : "\n\n") . $paragraph;
}
}
if ($current !== '') {
$chunks[] = trim($current);
}
return $chunks;
}
// Generate and persist embeddings for all chunks of a document
public function embedAndStore(Document $document, string $text): void
{
$chunks = $this->chunk($text);
foreach ($chunks as $index => $chunkText) {
// toEmbeddings() calls the configured embedding provider
$vector = Str::of($chunkText)->toEmbeddings();
DocumentChunk::create([
'document_id' => $document->id,
'chunk_index' => $index,
'content' => $chunkText,
// pgvector expects the array serialised as '[0.1,0.2,...]'
'embedding' => '[' . implode(',', $vector) . ']',
]);
}
}
}
Queuing this as a job is non-negotiable for documents longer than a few paragraphs. Each toEmbeddings() call is an HTTP round trip to the embedding API — embedding 50 chunks synchronously will block the request thread for several seconds.
// app/Jobs/EmbedDocumentJob.php
namespace App\Jobs;
use App\Models\Document;
use App\Services\DocumentEmbeddingService;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
class EmbedDocumentJob implements ShouldQueue
{
use Queueable;
public int $timeout = 300; // embedding 100 chunks can take 3–4 minutes
public int $tries = 2;
public function __construct(
public readonly Document $document,
public readonly string $text,
) {}
public function handle(DocumentEmbeddingService $service): void
{
$service->embedAndStore($this->document, $this->text);
}
}
Dispatch it from your controller after storing the uploaded file:
EmbedDocumentJob::dispatch($document, $extractedText)->onQueue('ai');
Run your AI queue worker with an extended timeout:
php artisan queue:work --queue=ai --timeout=360
Step 4 — Answering Questions (Retrieval + Generation)
The retrieval step embeds the user's question with the same model used during ingestion, finds the top-k most similar chunks, then injects them as context into a generation prompt.
// app/Services/DocumentQAService.php
namespace App\Services;
use App\Models\Document;
use App\Models\DocumentChunk;
use Illuminate\Support\Facades\AI;
use Illuminate\Support\Str;
class DocumentQAService
{
public function answer(Document $document, string $question): string
{
// 1. Embed the question using the same model
$queryVector = Str::of($question)->toEmbeddings();
$serialised = '[' . implode(',', $queryVector) . ']';
// 2. Retrieve the 5 most relevant chunks
$chunks = DocumentChunk::whereVectorSimilarTo('embedding', $serialised)
->where('document_id', $document->id)
->limit(5)
->get();
if ($chunks->isEmpty()) {
return 'No relevant content found in this document for your question.';
}
// 3. Build context block
$context = $chunks
->pluck('content')
->map(fn ($c, $i) => "[Passage " . ($i + 1) . "]\n$c")
->join("\n\n---\n\n");
// 4. Generate a grounded answer
$prompt = <<<PROMPT
Answer the question below using ONLY the passages provided.
If the answer is not contained in the passages, say "I don't know based on this document."
Do not fabricate information.
QUESTION:
{$question}
PASSAGES:
{$context}
PROMPT;
return AI::text()
->using('gpt-4o-mini')
->prompt($prompt)
->generate()
->text();
}
}
The instruction "answer using ONLY the passages provided" is the most important guard against hallucination in a RAG system. Without it, the model will freely blend its pretraining knowledge with the retrieved text.
Step 5 — Testing Without API Calls
The Laravel AI SDK ships built-in fakes. Use them so your tests do not hit live APIs:
// tests/Feature/DocumentQATest.php
use Laravel\AI\Fakes\TextFake;
use Illuminate\Support\Facades\AI;
it('returns a grounded answer for a known question', function () {
AI::fake([
'text' => new TextFake('The document mentions Q3 revenue was $4.2M.'),
]);
$document = Document::factory()->create();
DocumentChunk::factory()->for($document)->create([
'content' => 'Q3 revenue was $4.2M, up 18% year-on-year.',
'embedding' => '[' . implode(',', array_fill(0, 1536, 0.01)) . ']',
]);
$service = app(DocumentQAService::class);
$answer = $service->answer($document, 'What was Q3 revenue?');
expect($answer)->toContain('$4.2M');
});
For the embedding side, stub Str::of()->toEmbeddings() to return a zeroed vector — the important thing to test is that your chunking and similarity query logic run without errors, not that the embedding model returns sensible distances.
Common Mistakes and Limitations
Mismatched embedding models. Embedding at ingestion with text-embedding-3-small and querying with text-embedding-3-large produces nonsense similarity scores. Store the model name alongside each chunk if you ever plan to re-embed with a different model.
Skipping the IVFFlat index. At 10,000+ rows, a full table scan with vector_cosine_ops becomes measurably slow. Create the index before you go live, not after performance degrades.
Trusting whereVectorSimilarTo without a similarity threshold. The scope returns the nearest neighbours regardless of how dissimilar they are. A question about pricing asked against a legal contract will return the least-bad chunks, not meaningful ones. Add a raw cosine similarity filter:
// Pseudocode — adjust column reference for your driver
DocumentChunk::whereVectorSimilarTo('embedding', $serialised)
->whereRaw('1 - (embedding <=> ?) > 0.75', [$serialised])
->limit(5)
->get();
Chunk boundaries splitting key facts. A sentence like "Revenue grew from $3.5M to $4.2M" split across two chunks causes both halves to retrieve below threshold. Use overlap (as in Step 3) and prefer semantic paragraph splits over fixed byte offsets.
pgvector vs an external vector DB. pgvector suits collections up to a few million chunks. Beyond that, purpose-built stores (Pinecone, Weaviate, Qdrant) handle indexing and filtering better at the cost of added operational complexity.
Not scoping retrieval to the authenticated user's documents. Always include a where('document_id', $document->id) or equivalent ownership constraint. An embedding similarity query without a scope filter is a data-leakage bug.
Tradeoffs at a Glance
| Decision | Simple option | Scalable option |
|---|---|---|
| Vector storage | pgvector in Postgres | External vector DB |
| Chunking | Fixed character split | Semantic paragraph split with overlap |
| Embedding model |
text-embedding-3-small (cheaper) |
text-embedding-3-large (more accurate) |
| Embedding on upload | Synchronous (small docs only) | Queued job with --queue=ai
|
| Answer grounding | Prompt instruction only | Prompt instruction + similarity threshold |
Verifying the System Works
After uploading a test document, run these checks before considering the feature production-ready:
- Confirm the job completed: check
failed_jobsorphp artisan queue:monitor ai. - Count stored chunks:
DocumentChunk::where('document_id', $id)->count()— must be non-zero. - Verify embeddings are non-null:
DocumentChunk::whereNull('embedding')->count()— must be zero. - Ask a question whose answer appears verbatim in the document; assert the key phrase appears in the response.
- Ask a question not covered by the document; assert the fallback message is returned, not a hallucination.
This pipeline — chunk, embed, store, retrieve, generate — is the foundation of every production RAG system. The Laravel AI SDK makes the glue code thin enough that the interesting engineering decisions (chunking strategy, similarity thresholds, retrieval scope) remain visible rather than hidden behind abstractions.
If you need Laravel development in Mumbai, Mumbai Web Designer builds production-grade Laravel applications.
Top comments (0)