DEV Community: Abayomi Olatunji

Building an AI Assistant with Ollama and Next.js – Part 3 (RAG with LangChain, Pinecone and Ollama)

Abayomi Olatunji — Sun, 01 Jun 2025 18:21:47 +0000

🚨 This is Part 3 of the “Building an AI Assistant with Ollama and Next.js” series.

👉 Check out Part 1 here

👉 Check out Part 2 here

🤖 Introduction

In the previous parts, we covered how to set up an AI assistant locally using Ollama, Next.js, and different package integrations. In this part, we’re diving deeper into building a Knowledge-Based AI assistant using RAG (Retrieval-Augmented Generation) with LangChain, Ollama, and Pinecone.

We’ll walk through how to:

Load and preprocess documents
Split and embed them into vector space
Store the embeddings in Pinecone
Query these vectors for smart retrieval

🔧 Tools Used

Next.js
TailwindCSS
Cursor IDE
Ollama
LangChain
Pinecone Vector Database
PDF-Parse, Mammoth.js for document reading

📘 What is RAG?

RAG stands for Retrieval-Augmented Generation. It’s a hybrid AI approach that improves response accuracy by combining:

Retrieval: Searches for relevant documents or chunks from a knowledge base.
Generation: Uses a language model (like Gemma or LLaMA) to generate natural responses based on the retrieved content.

🔁 Flow Summary

Load files (PDF, DOCX, TXT)
Split them into readable chunks
Embed those chunks into vector representations
Store them in Pinecone
Query Pinecone using user input and generate context-aware answers

You can read more on this on the LangChain documentation: https://js.langchain.com/docs/tutorials/rag/

🧩 Key Packages and Docs for further reading

Package	Use	Docs
`langchain`	Framework for chaining LLMs with tools	Docs
`@pinecone-database/pinecone`	Pinecone client	Docs
`@langchain/pinecone`	LangChain-Pinecone integration	Docs
`@langchain/community/embeddings/ollama`	Ollama embeddings for LangChain	Docs
`pdf-parse`, `mammoth`	For loading and reading PDFs, DOCX, and TXT	pdf-parse, mammoth

🧰 Tool Setup Overview

🔧 1. Setting Up Pinecone

Create an account on Pinecone
Create an Index with the following settings:
- Name: e.g., database_name
- Vector Type: Dense
- Dimension: 1024 (must match mxbai-embed-large)
- Metric: Cosine
- Environment: us-east-1-aws

You can select existing embedding models available on the setup options, I choose to use the custom setting so that it aligns with the model I'm using on the project. i.e mxbai-embed-large

🛠 2. Configure `.env`

Add these to your .env.local:

PINECONE_API_KEY=your-api-key
PINECONE_INDEX_NAME=database_name
PINECONE_ENVIRONMENT=us-east-1-aws
OLLAMA_MODEL=gemma3:1b

🚀 3. Launch Ollama and Embedding Model

Make sure Ollama is installed and run this model in your terminal (you can use any LLM model of your choice):

ollama run gemma3:1b

Install embedding model with:

ollama pull mxbai-embed-large

LangChain will reference this locally using Ollama via:

new OllamaEmbeddings({
  model: 'mxbai-embed-large',
  baseUrl: 'http://localhost:11434'
});

Note: you can check more models in https://js.langchain.com/docs/integrations/chat/ and https://ollama.com/search. Also, you explore other embedding models in https://js.langchain.com/docs/integrations/text_embedding/

🧪 How It Works – Step by Step

Here, I will be giving a breakdown of what we're trying to achieve, followed by the code snippet to use.

Step 1: Upload and Process Document

User uploads .pdf, .docx, or .txt.
We load the file using langchain loaders.
The text is split into chunks using RecursiveCharacterTextSplitter.
Chunks are returned as an array of LangChain Document objects.

Step 2: Embed and Store in Pinecone

Chunks are embedded via OllamaEmbeddings using mxbai-embed-large.
Vectors are stored in the Pinecone vector index under a namespace.

Step 3: Query for Context

When a user types a question, we run a vector similarity search.
Relevant chunks are retrieved from Pinecone.
Chunks are combined into a context block.
The context is injected into the prompt as a system message for the LLM.

utils/documentProcessing.ts

import { OllamaEmbeddings } from '@langchain/community/embeddings/ollama';
import { Document } from '@langchain/core/documents';
import { PineconeStore } from '@langchain/pinecone';
import { Pinecone } from '@pinecone-database/pinecone';
import { DocxLoader } from 'langchain/document_loaders/fs/docx';
import { PDFLoader } from 'langchain/document_loaders/fs/pdf';
import { TextLoader } from 'langchain/document_loaders/fs/text';
import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';

const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY! });
const embeddings = new OllamaEmbeddings({ model: 'mxbai-embed-large', baseUrl: 'http://localhost:11434' });
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200 });

export async function processDocument(file: File | Blob, fileName: string): Promise<Document[]> {
  let documents: Document[];
  if (fileName.endsWith('.pdf')) documents = await new PDFLoader(file).load();
  else if (fileName.endsWith('.docx')) documents = await new DocxLoader(file).load();
  else if (fileName.endsWith('.txt')) documents = await new TextLoader(file).load();
  else throw new Error('Unsupported file type');

  return await textSplitter.splitDocuments(documents);
}

export async function storeDocuments(documents: Document[]): Promise<void> {
  const pineconeIndex = pinecone.Index(process.env.PINECONE_INDEX_NAME!);
  await PineconeStore.fromDocuments(documents, embeddings, {
    pineconeIndex,
    maxConcurrency: 5,
    namespace: 'your_namespace', //optional
  });
}

export async function queryDocuments(query: string): Promise<Document[]> {
  const pineconeIndex = pinecone.Index(process.env.PINECONE_INDEX_NAME!);
  const vectorStore = await PineconeStore.fromExistingIndex(embeddings, {
    pineconeIndex,
    maxConcurrency: 5,
    namespace: 'your_namespace', //optional
  });

  return await vectorStore.similaritySearch(query, 4);
}

api/chat/upload/route.ts

import { processDocument, storeDocuments } from '@/utils/documentProcessing';
import { NextResponse } from 'next/server';

export async function POST(req: Request) {
  const formData = await req.formData();
  const file = formData.get('file') as File;
  if (!file) return NextResponse.json({ error: 'No file provided' }, { status: 400 });

  const documents = await processDocument(file, file.name);
  await storeDocuments(documents);

  return NextResponse.json({
    message: 'Document processed and stored successfully',
    fileName: file.name,
    documentCount: documents.length
  });
}

api/chat/route.ts

import { queryDocuments } from '@/utils/documentProcessing';
import { Message, streamText } from 'ai';
import { NextRequest } from 'next/server';
import { createOllama } from 'ollama-ai-provider';

const ollama = createOllama();
const MODEL_NAME = process.env.OLLAMA_MODEL || 'gemma3:1b';

export async function POST(req: NextRequest) {
  const { messages } = await req.json();
  const lastMessage = messages[messages.length - 1];
  const relevantDocs = await queryDocuments(lastMessage.content);

  const context = relevantDocs.map((doc) => doc.pageContent).join('\n\n');
  const systemMessage: Message = {
    id: 'system',
    role: 'system',
    content: `You are a helpful AI assistant with access to a knowledge base. 
    Use the following context to answer the user's questions:\n\n${context}`,
  };

  const promptMessages = [systemMessage, ...messages];
  const result = await streamText({
    model: ollama(MODEL_NAME),
    messages: promptMessages
  });

  return result.toDataStreamResponse();
}

For the UI part, here are the code snippet

ChatInput.tsx

'use client'
interface ChatInputProps {
  input: string;
  handleInputChange: (e: React.ChangeEvent<HTMLTextAreaElement>) => void;
  handleSubmit: (e: React.FormEvent<HTMLFormElement>) => void;
  isLoading: boolean;
}

export default function ChatInput({ input, handleInputChange, handleSubmit, isLoading }: ChatInputProps) {

  return (
    <form onSubmit={handleSubmit} className="flex gap-4">
      <textarea
        value={input}
        onChange={handleInputChange}
        placeholder="Ask a question about the documents..."
        className="flex-1 p-4 border border-gray-200 dark:border-gray-700 rounded-xl 
          bg-white dark:bg-gray-800 
          placeholder-gray-400 dark:placeholder-gray-500
          focus:outline-none focus:ring-2 focus:ring-blue-500 dark:focus:ring-blue-400
          resize-none min-h-[50px] max-h-32
          text-gray-700 dark:text-gray-200"
        rows={1}
        required
        disabled={isLoading}
      />
      <button
        type="submit"
        disabled={isLoading}
        className={`px-6 py-2 rounded-xl font-medium transition-all duration-200
          ${isLoading 
            ? 'bg-gray-100 dark:bg-gray-700 text-gray-400 dark:text-gray-500 cursor-not-allowed'
            : 'bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white shadow-sm hover:shadow'
          }`}
      >
        {isLoading ? (
          <span className="flex items-center gap-2">
            <svg className="animate-spin h-4 w-4" viewBox="0 0 24 24">
              <circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" fill="none"/>
              <path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"/>
            </svg>
            Processing
          </span>
        ) : 'Send'}
      </button>
    </form>
  );
}

ChatMessage.tsx

'use client'
import { Message } from 'ai';
import ReactMarkdown from 'react-markdown';

interface ChatMessageProps {
  message: Message;
}

export default function ChatMessage({ message }: ChatMessageProps) {
  return (
    <div
      className={`flex items-start gap-4 p-6 rounded-2xl shadow-sm transition-colors ${
        message.role === 'assistant'
          ? 'bg-white dark:bg-gray-800 border border-gray-100 dark:border-gray-700'
          : 'bg-blue-50 dark:bg-blue-900/30 border border-blue-100 dark:border-blue-800'
      }`}
    >
      <div className={`w-8 h-8 rounded-full flex items-center justify-center flex-shrink-0 ${
        message.role === 'assistant'
          ? 'bg-purple-100 text-purple-600 dark:bg-purple-900 dark:text-purple-300'
          : 'bg-blue-100 text-blue-600 dark:bg-blue-900 dark:text-blue-300'
      }`}>
        {message.role === 'assistant' ? '🤖' : '👤'}
      </div>
      <div className="flex-1 min-w-0">
        <div className="font-medium text-sm mb-2 text-gray-700 dark:text-gray-300">
          {message.role === 'assistant' ? 'AI Assistant' : 'You'}
        </div>
        <div className="prose dark:prose-invert prose-sm max-w-none">
          <ReactMarkdown>{message.content}</ReactMarkdown>
        </div>
      </div>
    </div>
  );
}

FileUpload.tsx

"use client"
import React, { useState } from 'react';

export default function FileUpload() {
  const [isUploading, setIsUploading] = useState(false);
  const [message, setMessage] = useState('');
  const [error, setError] = useState('');

  const handleFileUpload = async (e: React.ChangeEvent<HTMLInputElement>) => {
    const file = e.target.files?.[0];
    if (!file) return;

    // Reset states
    setMessage('');
    setError('');
    setIsUploading(true);

    try {
      const formData = new FormData();
      formData.append('file', file);

      const response = await fetch('/api/chat/upload', {
        method: 'POST',
        body: formData,
      });

      const data = await response.json();

      if (!response.ok) {
        throw new Error(data.error || 'Error uploading file');
      }

      setMessage(`Successfully uploaded ${file.name}`);
    } catch (err) {
      setError(err instanceof Error ? err.message : 'Error uploading file');
    } finally {
      setIsUploading(false);
    }
  };

  return (
    <div className="mb-6">
      <div className="flex flex-col sm:flex-row items-center gap-4">
        <label
          className={`flex items-center gap-2 px-6 py-3 rounded-xl border-2 border-dashed
            transition-all duration-200 cursor-pointer
            ${isUploading 
              ? 'border-gray-300 bg-gray-50 dark:border-gray-700 dark:bg-gray-800/50'
              : 'border-blue-300 hover:border-blue-400 hover:bg-blue-50 dark:border-blue-700 dark:hover:border-blue-600 dark:hover:bg-blue-900/30'
            }`}
        >
          <svg 
            className={`w-5 h-5 ${isUploading ? 'text-gray-400' : 'text-blue-500'}`} 
            fill="none" 
            stroke="currentColor" 
            viewBox="0 0 24 24"
          >
            <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-8l-4-4m0 0L8 8m4-4v12" />
          </svg>
          <span className={`font-medium ${isUploading ? 'text-gray-400' : 'text-blue-500'}`}>
            {isUploading ? 'Uploading...' : 'Upload Document'}
          </span>
          <input
            type="file"
            className="hidden"
            accept=".pdf,.docx"
            onChange={handleFileUpload}
            disabled={isUploading}
          />
        </label>
        <span className="text-sm text-gray-500 dark:text-gray-400 flex items-center gap-2">
          <svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
            <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
          </svg>
          Supported: PDF, DOCX
        </span>
      </div>

      {message && (
        <div className="mt-4 p-4 bg-green-50 dark:bg-green-900/30 rounded-xl border border-green-100 dark:border-green-800">
          <p className="text-sm text-green-600 dark:text-green-400 flex items-center gap-2">
            <svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
              <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
            </svg>
            {message}
          </p>
        </div>
      )}
      {error && (
        <div className="mt-4 p-4 bg-red-50 dark:bg-red-900/30 rounded-xl border border-red-100 dark:border-red-800">
          <p className="text-sm text-red-600 dark:text-red-400 flex items-center gap-2">
            <svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
              <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 8v4m0 4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
            </svg>
            {error}
          </p>
        </div>
      )}
    </div>
  );
}

ChatPage.tsx

"use client"
import { useChat } from 'ai/react';

import ChatInput from './ChatInput';
import ChatMessage from './ChatMessage';
import FileUpload from './FileUpload';

export default function ChatPage() {
  const { input, messages, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat',
    onError: (error) => {
      console.error('Chat error:', error);
      alert('Error: ' + error.message);
    }
  });

  return (
    <div className="flex flex-col h-screen bg-gray-50 dark:bg-gray-900">
      <div className="flex-1 max-w-5xl mx-auto w-full p-4 md:p-6 lg:p-8">
        <div className="flex-1 overflow-y-auto mb-4 space-y-6">
          <h1 className="text-3xl font-bold text-gray-900 dark:text-white text-center mb-8">
            RAG-Powered Knowledge Base Chat
          </h1>
          <div className="bg-white dark:bg-gray-800 rounded-xl shadow-lg p-6">
            <FileUpload />
          </div>
          <div className="space-y-6">
            {messages.map((message) => (
              <ChatMessage key={message.id} message={message} />
            ))}
          </div>
        </div>
        <div className="sticky bottom-0 bg-white dark:bg-gray-800 rounded-xl shadow-lg p-4">
          <ChatInput 
            input={input} 
            handleInputChange={handleInputChange} 
            handleSubmit={handleSubmit} 
            isLoading={isLoading}
          />
        </div>
      </div>
    </div>
  );
}

Viola! You're reading to run your code

npm run dev

Click the Upload document button to upload the document you want to store. Once the upload is successful, your Pinecone dashboard will look like this:

With the document loaded, you can ask your AI Assistant questions relating to the content in the document and you will get the correct response. Here is a screenshot of my test:

Happy Coding 😎! Feel free to share your experience and feedbacks too. Cheers!

Building an AI Assistant with Ollama and Next.js - Part 2 (Using Packages)

Abayomi Olatunji — Thu, 29 May 2025 03:35:41 +0000

💡 Missed Part 1? Start with the basics and learn how to build an AI assistant using Ollama locally in a Next.js app:

👉 Building an AI Assistant with Ollama and Next.js - Part 1

🧠 Let's go — Part 2

In Part 1, we set up a local AI assistant using Ollama, Next.js, and the Gemma 3:1B model with minimal setup.

In this article, we’ll explore two powerful and flexible methods to integrate Ollama directly into your Next.js project using JavaScript libraries.

We'll walk through:

Installing the necessary packages
How each method works
Benefits and differences between them

🛠 Tools Used

Next.js – App framework for building fast React apps
TailwindCSS – Styling made simple and responsive
Cursor IDE – Developer-friendly coding environment
Ollama – Local model runner
Gemma 3:1B Model – Lightweight, open-source LLM
Ollama.js https://github.com/ollama/ollama-js
AI SDK https://www.npmjs.com/package/ai
ollama-ai-provider https://www.npmjs.com/package/ollama-ai-provider
react-markdown - https://www.npmjs.com/package/react-markdown

🚀 Getting Started

Make sure you already have Ollama and the model installed. Run this in your terminal:

ollama run gemma3:1b

📥 You can get the model from: https://ollama.com/library

📦 Method 1 – Using ollama-js

The ollama-js package is a lightweight Node client for interacting with the Ollama server directly from your code.

📌 Install:

npm install ollama

📁 API Route in Next.js

// app/api/chat/route.js

import ollama from 'ollama';

export async function POST(req) {
  const { message } = await req.json();
  const response = await ollama.chat({
    model: 'gemma3:1b',
    messages: [{ role: 'user', content: message }],
  });

  return Response.json(response);
}

Using the existing UI implementation from Part 1, this method will work perfectly with it.

✅ Benefits:

Minimal setup
Direct control over the model and requests
Great for full-stack or custom workflows

⚡ Method 2 – Using `ai-sdk` + `ollama-ai-provider` + `react-markdown` (Preferred)

This method uses AI SDK, which abstracts a lot of complexity and provides a seamless experience, especially for frontend-focused applications.

📌 Install the packages first:

npm install ai ollama-ai-provider react-markdown

🧠 Usage Overview:

// app/api/chat/route.ts

import { streamText } from 'ai';
import { NextRequest } from 'next/server';
import { createOllama } from 'ollama-ai-provider';

export const runtime = 'edge';

// Create Ollama provider with configuration
const ollamaProvider = createOllama();

// Configure the model name
const MODEL_NAME = process.env.OLLAMA_MODEL || 'gemma3:1b';

export async function POST(req: NextRequest) {
  try {
    const { messages } = await req.json();

    if (!messages || !Array.isArray(messages) || messages.length === 0) {
      return new Response('Invalid messages format', { status: 400 });
    }

    // Add system message if not present
    const messagesWithSystem = messages[0]?.role !== 'system' 
      ? [
          { 
            role: 'system', 
            content: 'You are a helpful AI assistant powered by Ollama. You help users with their questions and tasks.'
          },
          ...messages
        ]
      : messages;

    const result = await streamText({
      model: ollamaProvider(MODEL_NAME),
      messages: messagesWithSystem,
    });

    return result.toDataStreamResponse();
  } catch (error) {
    console.error('Chat API error:', error);
    return new Response(
      JSON.stringify({ error: 'Failed to process chat request' }), 
      { status: 500 }
    );
  }
}

// ChatInput.tsx

import { useEffect, useRef } from 'react';

interface ChatInput2Props {
  input: string;
  handleInputChange: (e: React.ChangeEvent<HTMLTextAreaElement>) => void;
  handleSubmit: (e: React.FormEvent) => void;
  isLoading: boolean;
}

export default function ChatInput2({
  input,
  handleInputChange,
  handleSubmit,
  isLoading
}: ChatInput2Props) {
  const textareaRef = useRef<HTMLTextAreaElement>(null);

  useEffect(() => {
    if (textareaRef.current) {
      textareaRef.current.style.height = 'auto';
      textareaRef.current.style.height = `${textareaRef.current.scrollHeight}px`;
    }
  }, [input]);

  return (
    <form onSubmit={handleSubmit} className="flex items-end gap-4 border-t border-gray-700 bg-gray-800 p-4 sticky bottom-0">
      <div className="relative flex-1">
        <textarea
          ref={textareaRef}
          className="w-full resize-none rounded-xl border border-gray-600 bg-gray-700 p-4 pr-12 text-gray-100 placeholder-gray-400 focus:outline-none focus:ring-2 focus:ring-blue-500 max-h-[200px] min-h-[56px]"
          rows={1}
          placeholder="Type your message..."
          value={input}
          onChange={handleInputChange}
          disabled={isLoading}
        />
        <button
          type="submit"
          disabled={isLoading || !input.trim()}
          className="absolute bottom-2 right-2 rounded-lg bg-blue-600 p-2 text-white hover:bg-blue-700 disabled:opacity-50 disabled:hover:bg-blue-600"
        >
          <svg
            xmlns="http://www.w3.org/2000/svg"
            fill="none"
            viewBox="0 0 24 24"
            strokeWidth={2}
            stroke="currentColor"
            className="w-5 h-5"
          >
            <path
              strokeLinecap="round"
              strokeLinejoin="round"
              d="M6 12L3.269 3.126A59.768 59.768 0 0121.485 12 59.77 59.77 0 013.27 20.876L5.999 12zm0 0h7.5"
            />
          </svg>
        </button>
      </div>
    </form>
  );
}

// ChatMessage.tsx

import ReactMarkdown from 'react-markdown';

interface ChatMessage2Props {
  role: 'user' | 'assistant' | 'system';
  content: string;
}

export default function ChatMessage2({ role, content }: ChatMessage2Props) {
  return (
    <div
      className={`flex ${
        role === 'user' ? 'justify-end' : 'justify-start'
      } mb-4`}
    >
      <div
        className={`max-w-[80%] rounded-xl p-4 shadow-md ${
          role === 'user'
            ? 'bg-blue-600 text-gray-100'
            : 'bg-gray-700 text-gray-100 border border-gray-600'
        }`}
      >
        <ReactMarkdown
          components={{
            p: ({ children }) => <p className="mb-2 last:mb-0">{children}</p>,
            code: ({ children }) => (
              <code
                className={`block p-2 rounded my-2 ${
                  role === 'user'
                    ? 'bg-blue-700 text-gray-100'
                    : 'bg-gray-800 text-gray-100'
                }`}
              >
                {children}
              </code>
            ),
            ul: ({ children }) => (
              <ul className="list-disc list-inside mb-2 text-gray-100">{children}</ul>
            ),
            ol: ({ children }) => (
              <ol className="list-decimal list-inside mb-2 text-gray-100">{children}</ol>
            ),
          }}
        >
          {content}
        </ReactMarkdown>
      </div>
    </div>
  );
}

// ChatPage.tsx

"use client"
import { useChat } from 'ai/react';
import ChatInput2 from './ChatInput2';
import ChatMessage2 from './ChatMessage2';

type MessageRole = 'system' | 'user' | 'assistant';

function normalizeRole(role: string): MessageRole {
  if (role === 'system' || role === 'user' || role === 'assistant') {
    return role as MessageRole;
  }
  return 'assistant';
}

export default function Chat2Page() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat2',
    initialMessages: [
      {
        id: 'system-1',
        role: 'system',
        content: 'You are a helpful AI assistant powered by Ollama. You can help users with various tasks and answer their questions.',
      },
    ],
  });

  return (
    <div className="container mx-auto max-w-4xl p-4 h-[calc(100vh-2rem)] bg-gray-900">
      <div className="mb-4">
        <h1 className="text-3xl font-bold text-gray-100">AI Chat Assistant v2</h1>
        <p className="text-gray-400">
          Powered by Ollama with markdown support and streaming responses
        </p>
      </div>

      <div className="flex flex-col h-[calc(100%-8rem)]">
        <div className="flex-1 overflow-y-auto rounded-xl border border-gray-700 bg-gray-800 p-4 mb-4">
          {messages.map((message) => (
            <ChatMessage2
              key={message.id}
              role={normalizeRole(message.role)}
              content={message.content}
            />
          ))}
          {messages.length === 1 && (
            <div className="flex h-full items-center justify-center text-gray-500">
              Start a conversation by typing a message below
            </div>
          )}
        </div>

        <ChatInput2
          input={input}
          handleInputChange={handleInputChange}
          handleSubmit={handleSubmit}
          isLoading={isLoading}
        />
      </div>
    </div>
  );
}

// app/chat/page.tsx

import ChatPage from '@/modules/chat/ChatPage';

export default function Chat() {
  return <ChatPage />;
}

✅ Benefits:

Built-in support for streaming responses (for real-time UX)
Works smoothly with React Server Components
Clean abstraction that improves maintainability
Easy markdown rendering with react-markdown

📸 Outcome

🧠 Summary: Which Method Should You Use?

Feature	`ollama-js`	`ai-sdk` + `ollama-ai-provider`
Setup Simplicity	✅ Simple	✅ Moderate
Streaming Support	❌ Manual	✅ Built-in
Frontend Friendly	❌ More Backend Focused	✅ Tailored for React
Markdown Rendering	❌ Manual	✅ Easy via `react-markdown`
Recommended For	Custom/Low-level Projects	Production-ready AI UI

👋 What's Next?

While this is another method to build your AI assistant locally,
👉 Click the next article to learn how to build this using LangChain and Ollama for more advanced AI workflows.

Happy Coding....

Building an AI Assistant with Ollama and Next.js - Part 1

Abayomi Olatunji — Wed, 28 May 2025 23:48:06 +0000

Introduction 🧠💬

Artificial Intelligence (AI) is reshaping how we interact with digital tools, and building your own local AI assistant has never been easier. In this guide, I’ll walk you through how I built a simple AI assistant using Next.js, TailwindCSS, and Ollama, running the Gemma 3:1B model

Note: You can run any model of your choice from the available models on https://ollama.com/models; however you should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.).

Whether you're a beginner or just looking for a lightweight and privacy-friendly AI implementation, you’ll find this guide approachable and relatable. No cloud APIs. No subscriptions. Just local magic.

🧰 Tools Used

Before we dive in, here are the key tools used in this project:

Next.js – Our React framework of choice, with app router support.
TailwindCSS – For fast and beautiful styling.
Cursor IDE – A modern coding environment tailored for AI-assisted development. (I will create an article to help you setup your IDE, use rules and also work with MCPs)
Ollama – A simple way to run open-source large language models locally. Download and use directly from your terminal.
Gemma 3:1B model – A lightweight model great for running on most modern laptops. Using this just for testing purposes. Others includes Llama, DeepSeek, and Mistral, etc.

📦 Step 1: Set Up Your Next.js App

Let’s start by creating a new Next.js project. Open your terminal and run:

npx create-next-app@latest ollama-assistant --app
cd ollama-assistant
cursor .

Installing Next.js now allows you to set up Tailwind, TypeScript, and other configurations from the installation process.

Then, run the app on your locals using:

npm run dev

🤖 Step 2: Install and Run Ollama with Gemma

Ollama makes it super simple to run models locally. Head over to https://ollama.com/download and install it for your OS.

Once installed, open your terminal and run:

ollama run gemma3:1b

This will download and start the Gemma 3:1B model locally. Once the download is complete, the model will launch, and you can start chatting with it on the terminal, as shown in the screenshot below.

⚙️ Step 3: Connect Your App to Ollama

Next, we’ll add a simple API route that communicates with the local Ollama server. Ollama provides REST API endpoints that allow you to interact with the downloaded models. Once the terminal is running, it exposes an endpoint at http://localhost:11434/api, where you can send your HTTP requests.

app/api/chat/route.ts

import { NextResponse } from 'next/server';

export async function POST(req: Request) {
  try {
    const { message } = await req.json();

    // Make request to Ollama API
    const response = await fetch('http://localhost:11434/api/generate', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: 'gemma3:1b',
        prompt: message,
        stream: false,
      }),
    });

    const data = await response.json();

    return NextResponse.json({
      response: data.response,
    });
  } catch (error) {
    console.error('Error:', error);
    return NextResponse.json(
      { error: 'Failed to process the request' },
      { status: 500 }
    );
  }
}

This endpoint accepts a message from the frontend and returns the model’s response.

🖼️ Step 4: Build the Chat Interface

Let’s create a simple UI where users can type and get responses from our assistant.

These files will be in a modules folder named chat. (you can use your preferred project folder structure).

ChatInput.tsx

import React, { useState } from 'react';

interface ChatInputProps {
  onSendMessage: (message: string) => void;
}

const ChatInput: React.FC<ChatInputProps> = ({ onSendMessage }) => {
  const [message, setMessage] = useState('');

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (message.trim()) {
      onSendMessage(message);
      setMessage('');
    }
  };

  return (
    <form onSubmit={handleSubmit} className="border-t border-gray-200 dark:border-gray-700 p-4">
      <div className="flex items-center gap-2">
        <input
          type="text"
          value={message}
          onChange={(e) => setMessage(e.target.value)}
          placeholder="Type your message..."
          className="flex-1 rounded-lg border border-gray-300 dark:border-gray-600 p-2 
                   bg-white dark:bg-gray-800 text-gray-900 dark:text-gray-100"
        />
        <button
          type="submit"
          className="bg-blue-600 text-white px-4 py-2 rounded-lg hover:bg-blue-700 
                   transition-colors duration-200"
        >
          Send
        </button>
      </div>
    </form>
  );
};

export default ChatInput;

ChatMessage.tsx

import React from 'react';

interface ChatMessageProps {
  message: string;
  isUser: boolean;
}

const ChatMessage: React.FC<ChatMessageProps> = ({ message, isUser }) => {
  return (
    <div className={`flex ${isUser ? 'justify-end' : 'justify-start'} mb-4`}>
      <div
        className={`${
          isUser
            ? 'bg-blue-600 text-white rounded-l-lg rounded-tr-lg'
            : 'bg-gray-200 dark:bg-gray-700 text-gray-800 dark:text-gray-200 rounded-r-lg rounded-tl-lg'
        } px-4 py-2 max-w-[80%]`}
      >
        <p className="text-sm">{message}</p>
      </div>
    </div>
  );
};

export default ChatMessage;

ChatPage.tsx 

"use client"
import React, { useEffect, useRef, useState } from 'react';
import ChatInput from './ChatInput';
import ChatMessage from './ChatMessage';

interface Message {
  text: string;
  isUser: boolean;
}

const ChatPage: React.FC = () => {
  const [messages, setMessages] = useState<Message[]>([]);
  const [isLoading, setIsLoading] = useState(false);
  const messagesEndRef = useRef<HTMLDivElement>(null);

  const scrollToBottom = () => {
    messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
  };

  useEffect(() => {
    scrollToBottom();
  }, [messages]);

  const handleSendMessage = async (message: string) => {
    // Add user message
    setMessages(prev => [...prev, { text: message, isUser: true }]);
    setIsLoading(true);

    try {
      const response = await fetch('/api/chat', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ message }),
      });

      const data = await response.json();

      // Add AI response
      setMessages(prev => [...prev, { text: data.response, isUser: false }]);
    } catch (error) {
      console.error('Error:', error);
      setMessages(prev => [...prev, { text: 'Sorry, there was an error processing your request.', isUser: false }]);
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <div className="flex flex-col h-screen max-w-2xl mx-auto">
      <div className="bg-white dark:bg-gray-800 shadow-lg rounded-lg m-4 flex-1 flex flex-col overflow-hidden">
        <div className="p-4 border-b border-gray-200 dark:border-gray-700">
          <h1 className="text-xl font-semibold text-gray-800 dark:text-white">AI Assistant</h1>
        </div>

        <div className="flex-1 overflow-y-auto p-4">
          {messages.map((msg, index) => (
            <ChatMessage key={index} message={msg.text} isUser={msg.isUser} />
          ))}
          {isLoading && (
            <div className="flex justify-start mb-4">
              <div className="bg-gray-200 dark:bg-gray-700 rounded-lg px-4 py-2">
                <div className="animate-pulse flex space-x-2">
                  <div className="w-2 h-2 bg-gray-400 rounded-full"></div>
                  <div className="w-2 h-2 bg-gray-400 rounded-full"></div>
                  <div className="w-2 h-2 bg-gray-400 rounded-full"></div>
                </div>
              </div>
            </div>
          )}
          <div ref={messagesEndRef} />
        </div>

        <ChatInput onSendMessage={handleSendMessage} />
      </div>
    </div>
  );
};

export default ChatPage;

Then, the page/tsx in the app folder will have;

import ChatPage from '@/modules/ChatPage';

export default function Chat() {
  return <ChatPage />;
}

Here is how the user interface look after running npm run dev in the terminal

Viola! You can continue to build on this.

🧠 How It Works

Here’s a quick summary of what’s happening:

You enter a message in the text area.
The app sends your message to the /api/chat endpoint.
The endpoint forwards it to Ollama’s local API (localhost:11434).
Ollama responds with the model’s reply.
The frontend displays the AI response instantly.

💡 Why Use Ollama?

✅ Privacy – Everything runs locally. No data leaves your device.
✅ Speed – No network latency when calling the model.
✅ Cost-effective – No token limits or monthly subscriptions.

✨ What's Next?

While this method connects to the Ollama server externally (using the terminal), in the next article, I’ll show you how to build your assistant using the ollamajs package directly in your codebase for even tighter integration. Stay tuned!

📩 Feel free to drop a comment if you have any questions or need help setting things up!

Deploying NestJS Application using Vercel and Supabase

Abayomi Olatunji — Fri, 25 Oct 2024 12:58:52 +0000

Understand that deploying to Vercel is quite easy, however, there are some setups you need to take into consideration during deployment.

Prerequisites

NestJS Project connected and working properly locally in the development environment with PostgreSQL
Vercel Account for deployment
Supabase Account (we will be setting up our PostgreSQL Database here)

Let's start with the Supabase setup considering that your NestJS app is ready for deployment.

Supabase Account

Supabase is an opensource firebase alternative with full support and seamless configuration of your PostgreSQL database, and it also provides additional features such as authentication, storage, etc.

Set up a new account on Supabase and create a new project in the account.

Once the setup is completed, click on the connect button on the Home page. This will show you different options for connecting the DB to your project

Test the connection on your locals with the credentials provided to make sure everything is working perfectly well.

NOTE: Make sure the credentials are not exposed and stored in your .env file (I believe you know this already 😉)

Next, Let's set up our Vercel account and deploy the project

Vercel

Typically, Vercel is known mostly to be used for front-end app deployment, however, it can also be used to deploy backend projects.

PS: Use a suited service provider instead if you're working on a medium to large-scale project for your backend deployments.

On your Vercel account, create a new project and connect to your Git repository. Import your .env file and click the Deploy button.

Voila, that's it 🎉🎉🎉.
...

Common Issues likely encountered

# Error: No Output Directory named "public"

This is a common error because Vercel needs to know your output directory during the build process. To fix this, simply add a versel.json file and copy this:

{
  "version": 2,

  "builds": [
    {
      "src": "src/main.ts",
      "use": "@vercel/node"
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "src/main.ts",
      "methods": ["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    }
  ]
}

Run deployment again and that's all

...

# Error: This Serverless Function has crashed

In my case, it was because of a module not found error

...
There are several ways to fix this problem:

Method 1 (Replace all your imports with relative path)

From

import { UsersService } from 'src/users/users.service';

import { UsersService } from '../users/users.service';

...

Method 2 (Modify your vercel.json file and .gitignore file)

I eventually went with this method because I didn't need to confine my app to using only relative path imports.

So, modify the vercel.json to this

{
  "version": 2,

  "builds": [
    {
      "src": "dist/main.js",
      "use": "@vercel/node"
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "dist/main.js",
      "methods": ["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"]
    }
  ]
}

Go to your .gitignore file and remove /dist.

Run a new deployment and that's all.

Happy coding! 😎

How to fix Nextjs image not loading on production

Abayomi Olatunji — Wed, 25 Sep 2024 08:32:05 +0000

Hello devs,
I recently faced an issue around images not loading on production but working perfectly well on locals.

Accessing the image on production gives this error message

"url" parameter is valid but upstream response is invalid

For the above error, irrespective of the version you are running on, kindly install sharp and that is all!

npm i sharp

yarn add sharp

NextJS API working on locals but not working on production

Abayomi Olatunji — Mon, 01 Nov 2021 10:25:05 +0000

Hey, I am writing this post to share the experience I had dealing with NextJS API not working on production (that is, returning a 404 Bad Request).

A 400 Bad Request simply means that the server cannot process a request due to client error and this errors could be wrong URL or issues in the service use in the request.

For this particular use-case, the problem was related to the environment variables.

The environment variables store in the .env.local file wasn't working after deploying to Vercel and the solution is to also setup your variables on Vercel.

Let me show you a walkthrough;

Adding the variables needed in your project in .env file, you can check Environment Variable for more information of that. (NOTE: Make sure you add it to .gitignore so you don't expose it).

SAMPLE ENV VARIABLE

DB_USER=james

After deploying the app on Vercel, navigate to Settings

Click on Environment Variables

From there, you can add your environment variable and your web app will work as expected.

I hope this is helpful to you.

❤️❤️❤️

DEV Community: Abayomi Olatunji

Building an AI Assistant with Ollama and Next.js – Part 3 (RAG with LangChain, Pinecone and Ollama)

🤖 Introduction

🔧 Tools Used

📘 What is RAG?

🔁 Flow Summary

🧩 Key Packages and Docs for further reading

🧰 Tool Setup Overview

🔧 1. Setting Up Pinecone

🛠 2. Configure .env

🚀 3. Launch Ollama and Embedding Model

🧪 How It Works – Step by Step

Step 1: Upload and Process Document

Step 2: Embed and Store in Pinecone

Step 3: Query for Context

Building an AI Assistant with Ollama and Next.js - Part 2 (Using Packages)

🧠 Let's go — Part 2

🛠 Tools Used

🚀 Getting Started

📦 Method 1 – Using ollama-js

📌 Install:

📁 API Route in Next.js

✅ Benefits:

⚡ Method 2 – Using ai-sdk + ollama-ai-provider + react-markdown (Preferred)

📌 Install the packages first:

🧠 Usage Overview:

✅ Benefits:

📸 Outcome

🧠 Summary: Which Method Should You Use?

👋 What's Next?

Building an AI Assistant with Ollama and Next.js - Part 1

Introduction 🧠💬

🧰 Tools Used

📦 Step 1: Set Up Your Next.js App

🤖 Step 2: Install and Run Ollama with Gemma

⚙️ Step 3: Connect Your App to Ollama

🖼️ Step 4: Build the Chat Interface

🧠 How It Works

💡 Why Use Ollama?

✨ What's Next?

Deploying NestJS Application using Vercel and Supabase

Prerequisites

Supabase Account

Vercel

Common Issues likely encountered

# Error: No Output Directory named "public"

# Error: This Serverless Function has crashed

Method 1 (Replace all your imports with relative path)

Method 2 (Modify your vercel.json file and .gitignore file)

How to fix Nextjs image not loading on production

NextJS API working on locals but not working on production

🛠 2. Configure `.env`

⚡ Method 2 – Using `ai-sdk` + `ollama-ai-provider` + `react-markdown` (Preferred)