How I Built a Secure, 3,072-Dim AI Document Indexer Using Next.js & Supabase.

dhritich20baruah — Wed, 20 May 2026 03:20:19 +0000

Building a production-ready RAG (Retrieval-Augmented Generation) application from scratch is a very difficult and time consuming.

You first get a new idea for a project or business but before you can write a single line of your actual core business logic, you find yourself spending weeks fighting with multimodal file parsing, configuring vector extensions, designing complex database architecture, and securing multi-tenant storage.

The idea behind building DocuIntel was to solve that exact infrastructure headache. I wanted a template where I could just drop in my API keys, run a single setup script, and have a fully functioning, secure AI document portal ready to go.

Here is a deep dive into the architecture, some of the technical hurdles I ran into, and how I solved them.

The Multimodal Architecture
DocuIntel needs to process a wide variety of inputs—PDFs, Word docs, images (JPG/PNG), and even audio transcriptions (MP3s).

At first I was routing different file types through different third party parsing libraries like I was using tesseract OCR for extracting text from from images and pdf-parser for PDF files.
But instead of doing that, I offloaded the heavy lifting directly to Gemini 2.0 Flash. Gemini’s native multimodal capabilities handle OCR and layout analysis beautifully.

To ensure consistent, high-quality search results, I hardcoded a rigorous master SYSTEM_PROMPT inside the backend processing route (/api/process/route.ts). It forces the AI to behave like a clean data architect:

const SYSTEM_PROMPT = `
  You are an expert document parser for DocuIntel. 
  Your goal is to extract text with 100% accuracy for semantic search indexing.

  RULES:
  1. PRESERVE STRUCTURE: Use Markdown for headings, subheadings, and lists.
  2. TABLES: Convert all data tables into clean Markdown table format.
  3. NO CHAT: Do not say "Here is the text" or "I have processed the file." 
  4. NOISE REDUCTION: Ignore headers, footers, and page numbers.
  5. SMART METADATA: At the very end of your output, add a section called '---METADATA---' 
     and list 5-10 key entities or topics (e.g., 'Company: Acme Corp', 'Date: 2024-01-01').
`;

Why the Automated Metadata Tag Matters?
By forcing Gemini to extract smart metadata tags right into the text, the vector embedding model (gemini-embedding-2) captures high-value concepts. If a user searches for a specific company or date, the semantic search will surface the document even if that detail only appeared once in fine print.

Navigating the New Supabase Security Standards
Supabase has shifted to a "Secure by Default" model which will come into force by 30th May 2026.

Previously, creating a table in the public schema automatically exposed it to the Data API. Now, new tables require explicit grants, or your frontend client libraries (supabase-js) will get a 42501 permission error—even if Row-Level Security (RLS) is enabled.

To future-proof the setup, my SQL initialization script applies explicit grants directly to the authenticated user roles right after creating the tables and vector search RPC functions:

-- Create the documents table with 3,072-dimensional vector support
create table if not exists public.documents (
  id uuid primary key default uuid_generate_v4(),
  file_name text not null,
  file_url text not null,
  content text,               
  user_id uuid references auth.users(id),
  user_email text not null,   
  embedding vector(3072),     -- Optimized for high-res gemini-embedding-2
  created_at timestamp with time zone default timezone('utc'::text, now())
);

-- EXPLICIT GRANTS (Fixes 42501 Permission Errors)
grant select, insert, update, delete on table public.documents to authenticated;
grant all on table public.documents to service_role;

Hardened Multi-Tenant Storage Policies
Since the app deals with private user documents i.e. User A must never be able to discover or access User B's files, I have structured the Supabase Storage bucket so that every uploaded file is dynamically sandboxed into a folder named exactly after the user's unique authenticated ID (auth.uid()).

Here are the folder-level RLS storage policies that enforce that rule on every single request:

-- Restrict file uploads to the user's own UID folder
create policy "Users can upload their own documents"
on storage.objects for insert to authenticated
with check (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);

-- Restrict file reading to the user's own UID folder
create policy "Users can view their own documents"
on storage.objects for select to authenticated
using (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);

Keeping the Frontend Clean & Readable
On the frontend (Next.js 14 + TypeScript + Tailwind CSS), I wanted a highly scalable UI. A minor but common mess I see in boilerplate projects is massive, nested ternary operators in the JSX to handle dynamic file icons.

To keep things clean and performant, I grouped the file extension arrays and used standard JavaScript .includes() methods to dynamically assign Lucide React icons using a single class utility string:

Wrapping Up:
Building these layers from scratch took extensive testing, debugging environment variables, and reading through updated security documentation. But once it's configured, it works like magic: a user uploads an asset, Gemini reads and indexes it with a 3,072-dimension vector embedding, and you can instantly query your documents conceptually rather than just matching keywords.

If you are planning to build an AI document product for a client or launching your own micro-SaaS, you don't have to spend your weekend configuring this infrastructure from zero.

I've packaged this exact production-ready foundation—complete with the 1-click database initialization script, Next.js frontend, and pre-configured API routes—into a developer-friendly boilerplate.

https://dhritiman.gumroad.com/l/docuintel

DEV Community: dhritich20baruah

How I Built a Secure, 3,072-Dim AI Document Indexer Using Next.js & Supabase.