DEV Community: ELI

How to Build Your Own AI Platform with Ollama Cloud Models

ELI — Mon, 24 Nov 2025 23:22:35 +0000

Learn how to create your own AI chat application using Ollama Cloud's powerful models and Chainlit's intuitive framework.

Introduction

While Ollama provides the cognitive engine, Chainlit provides the interface. As LLM applications move beyond simple chatbot interactions into complex, multi-step agentic workflows, traditional dashboarding tools like Streamlit have proven insufficient. Chainlit has emerged as the purpose-built framework for this new paradigm.

What You'll Need

Understanding the Tech Stack

What is Ollama Cloud?

Managed version of Ollama - no local setup required
Access to popular models (deepseek-v3.1, gpt-oss, qwen3-vl, etc.)
API-based, scalable, and easy to integrate

What is Chainlit?

Python framework for building chat interfaces
Pre-built UI components
Easy integration with AI models
Real-time updates and streaming

AI-Powered Chat Application with Ollama Cloud and Chainlit

A sophisticated conversational AI application built with Chainlit and Ollama, featuring multi-modal document processing, MCP (Model Context Protocol) tool integration, and persistent chat sessions. Supports voice input, file analysis (PDF, DOCX, images), and seamless LLM model switching through customizable chat profiles.

Key Features

MCP Integration: Connect external tools and services via Model Context Protocol
Multi-format Document Processing : PDF, DOCX, TXT, and image analysis
Voice Input: Audio transcription with ElevenLabs integration
Multiple Chat Profiles: Switch between different LLM models and configurations
Persistent Sessions: Resume conversations with SQLAlchemy + Azure Blob storage
OAuth Authentication: Secure user management
Modern UI: Clean, responsive interface with custom themes
Tech Stack: Python, Chainlit, Ollama, SQLAlchemy, Azure Storage, ElevenLabs, PyMuPDF, OCR

Perfect for building intelligent document analysis tools, customer support bots, or educational AI assistants with enterprise-grade persistence and tool integration capabilities.

Bellow, you'll find the final home page

All my code can be found in my GitHub repository

Getting Started: Installation & Setup

1.Clone the Repository

git clone https://github.com/EliAbdiel/ollama-chat-application.git

cd ollama-chat-application

2.Install dependencies

uv sync

3.Create and fill .env

# Windows
copy .env.example .env

# Linux/Mac
cp .env.example .env

4.Run the app

# Linux/Mac
source .venv/bin/activate

# Windows (CMD)
.venv\Scripts\activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

chainlit run mani.py -w

How the App Works

The Ollama Chat Application is a Chainlit-based conversational AI interface that processes text, files, and voice input through local or cloud-hosted Ollama models with MCP tool integration and persistent storage.

Open the UI in your browser

The Ollama Chat Application uses Chainlit's built-in OAuth authentication system with support for GitHub and Auth0 providers.

Model Selection

Chat profiles in the Ollama Chat Application provide a user-friendly interface for selecting different LLM models while abstracting the underlying model identifiers used by the Ollama API.

Document Processing

Document processing in the Ollama Chat Application handles file uploads, extracts content from various formats (PDF, DOCX, images), and integrates the extracted text into the conversation flow.

MCP Integration

The MCP (Model Context Protocol) system in the Ollama Chat Application enables external tool integration, allowing the LLM to execute functions from connected MCP servers and incorporate results into conversations.

MCP Connection is Established

GitHub MCP Server

Conclusion

The combination of Ollama's Cloud Models with Chainlit's dynamic, event-driven interface marks a significant step forward in modern AI application development. Together, they deliver a unified workflow that blends the privacy and flexibility of local execution with the power, scalability, and reliability of cloud-hosted LLMs. Chainlit provides the rich, developer-friendly UI layer needed to rapidly iterate, while Ollama's cloud infrastructure ensures consistent performance and elastic scaling for real-world deployment.

Resources

A Beginner's Guide to Ollama Cloud Models

ELI — Sun, 19 Oct 2025 22:35:08 +0000

Ollama's cloud models are a new feature that allows users to run large language models without needing a powerful local GPU. These models are automatically offloaded to Ollama's cloud service, providing the same capabilities as local models while enabling the use of larger models that would typically not fit on a personal computer.

Ollama currently supports the following cloud models

deepseek-v3.1:671b-cloud
gpt-oss:20b-cloud
gpt-oss:120b-cloud
kimi-k2:1t-cloud
qwen3-coder:480b-cloud
glm-4.6:cloud
qwen3-vl:235b-cloud
Browse the latest additions ollama's cloud models

Cloud API Access

Cloud models can also be accessed directly on ollama.com API. In this mode, ollama acts as a remote Ollama host.

For direct access to ollama cloud api, first create an API key.
Then, set the OLLAMA_API_KEY environment variable to your API key.

export OLLAMA_API_KEY=your_api_key

Generating a response

First, install dependencies. Run the following in your terminal or notebook cell:

print("Installing ollama")
!pip install -qU ollama
print("Installing pymupdf")
!pip install -qU pymupdf
print("Installing IPython")
!pip install -qU IPython
print("Intalling markdown")
!pip install -qU markdown
print("Intalling sqlalchemy")
!pip install -qU sqlalchemy
print("Installing langchain")
!pip install -qU langchain
print("Installing langgraph")
!pip install -qU langgraph
print("Installing python docx")
!pip install -qU python-docx
print("Installing pytesseract")
!pip install -qU pytesseract
print("Installing python magic")
!pip install -qU python-magic
print("Installing python dotenv")
!pip install -qU python-dotenv
print("Installing langchain core")
!pip install -qU langchain-core
print("Installing email-validator")
!pip install -qU pydantic[email]
print("Installing langchain ollama")
!pip install -qU langchain-ollama
print("Installing langchain community")
!pip install -qU langchain-community
print("Installed packages successfully!")

This notebook runs in Google Colab.

Basic Usage

This code snippet connects to the Ollama cloud API using your API key, sends a question to a specific language model (gpt-oss:120b-cloud), and then prints the model's answer as it's generated

try:
    import os
    from google.colab import userdata
except Exception as e:
    print(e)

else:
    userdata_ollama = None
    try:
        userdata_ollama = userdata.get('OLLAMA_API_KEY')
        userdata_ollama_url = userdata.get('OLLAMA_BASE_URL')
    except:
        pass

    if userdata_ollama:
        print("OLLAMA_API_KEY found in Secrets\n")
        os.environ['OLLAMA_API_KEY'] = userdata_ollama
    if userdata_ollama_url:
        print("OLLAMA_BASE_URL found in Secrets\n")
        os.environ['OLLAMA_BASE_URL'] = userdata_ollama_url

    ollama_api_key = os.getenv('OLLAMA_API_KEY')
    ollama_base_url = os.getenv('OLLAMA_BASE_URL')
finally:
    pass

Custom Client

A custom client can be created by instantiating Client or AsyncClient from ollama

All extra keyword arguments are passed into the httpx.Client

from httpx import ConnectError
from ollama import Client, ResponseError

try:
    client = Client(
        host=ollama_base_url,
        headers={
            'Authorization': f'Bearer {ollama_api_key}'
        }
    )
    # List models available via the API
    response = client.list()
except ConnectError as e:
    msg = (
        "Failed to connect to Ollama. Please check that Ollama is downloaded, "
        "running and accessible. https://ollama.com/download"
    )
    raise ValueError(msg) from e
except ResponseError as e:
    msg = (
        "Received an error from the Ollama API. "
        "Please check your Ollama server logs."
    )
    raise ValueError(msg) from e
except Exception as e:
    msg = (
        "An unexpected error occurred while trying to connect to Ollama. "
        "Please check your Ollama server logs."
    )
    raise ValueError(msg) from e

else:
    model_names = [model["model"] for model in response["models"]]

    print(f"Models available via the API: {len(model_names)}\n")
    for name in model_names:
        print(name)
finally:
    pass

messages = [{
    'role': 'user',
    'content': 'How are you?',
},]

for part in client.chat('gpt-oss:120b-cloud', messages=messages, stream=True):
    print(part.message.content, end='', flush=True)

Capabilities

Ollama's cloud models offer advanced capabilities beyond basic text generation, tailored for developers and AI practitioners. Key features include tool calling (for integrating external functions), thinking traces (to reveal the model's reasoning), streaming (for real-time responses), structured outputs (to enforce reliable JSON schemas), and vision (for multimodal image understanding). Together, these enable the development of robust, scalable, and production ready AI applications with enhanced control and insight.

Tool Calling

Ollama offers support for tool calling, also referred to as function calling. This feature empowers a language model to utilize external tools or functions and integrate the outcomes of these tools into its responses.

Supported models

deepseek-v3.1:671b-cloud
gpt-oss:20b-cloud
gpt-oss:120b-cloud
Browse the latest additions models to utilize external tools

try:
    from ollama import (
        Client, ResponseError,
        web_search, web_fetch
    )
    from datetime import datetime
    from httpx import ConnectError
except ImportError as e:
    print(e)

else:
    client = Client(
        host=ollama_base_url,
        headers={
            'Authorization': f'Bearer {ollama_api_key}'
        }
    )
finally:
    pass

available_tools = {'web_fetch': web_fetch, 'web_search': web_search}

current_time = datetime.now().strftime("%A, %B %d, %Y at %I:%M %p")

SYSTEM_PROMPT = f"""
You are an intelligent assistant designed to provide accurate, relevant, and up-to-date information.

You have access to:

- web_search: to discover relevant and current sources
- web_fetch: to retrieve and analyze the contents of specific URLs

System Context:

The current time is '{current_time}'. Always prioritize the topic with current time, which should be used to find up-to-date information.
""".strip()

def ollama_web_search(message):
    """
    Ollama web search can be used to augment models with the latest information to reduce hallucinations and improve accuracy.

    Args:
        - message (str): The user's question or query.
    Returns:
        - str: The model's response to the user's question.
    Example:
         ollama_web_search("What are Ollama's Cloud Models?")
         "Ollama's cloud models are a new feature that allow users to run large language models without needing a powerful local GPU.
    """

    try:
        messages = [
            {'role': 'system', 'content': SYSTEM_PROMPT},
            {'role': 'user', 'content': message}
        ]

        tool_calls = []
        max_iterations = 0
        iterations = 3

        while max_iterations < iterations:
            try:
                result = client.chat(
                    model='gemma4:31b-cloud',
                    messages=messages,
                    tools=[web_fetch, web_search],
                    think=True,
                    # stream=True,
                    options={'temperature': 0},
                )
            except ConnectError as e:
                msg = (
                    "Failed to connect to Ollama. Please check that Ollama is downloaded, "
                    "running and accessible. https://ollama.com/download"
                )
                raise ValueError(msg) from e
            except ResponseError as e:
                msg = (
                    "Received an error from the Ollama API. "
                    "Please check your Ollama server logs."
                )
                raise ValueError(msg) from e
            except Exception as e:
                msg = (
                    "An unexpected error occurred while trying to connect to Ollama. "
                    "Please check your Ollama server logs."
                )
                raise ValueError(e)


            if result.message.tool_calls:
                tool_calls.extend(result.message.tool_calls)
            if result.message.thinking:
                print("THINKING...\n\n")
                print(f"{result.message.thinking}\n\n")
            if result.message.content:
                print("ANSWER...\n\n")
                print(f"{result.message.content}\n")

            messages.append(result.message)

            if tool_calls:
                tool_executed = False
                for tool_call in tool_calls:
                    function_to_call = available_tools.get(tool_call.function.name)
                    if function_to_call:
                        tool_args = tool_call.function.arguments
                        tool_result = function_to_call(**tool_args)
                        # print(f"Executing tool: {tool_call.function.name}, with args: {tool_args}\n")
                        messages.append({
                            'role': 'tool',
                            'content': str(tool_result)[:2000 * 4],
                            'tool_name': tool_call.function.name
                        })
                        tool_executed = True
                    else:
                        # print(f'Tool {tool_call.function.name} not found')
                        messages.append({
                            'role': 'tool',
                            'content': f'Tool {tool_call.function.name} not found',
                            'tool_name': tool_call.function.name
                        })
                        tool_executed = True
                if not tool_executed:
                    break
            max_iterations += 1
            # No tool calls, conversation is complete
    except Exception as e:
        print(f'Error: {e}')

try:
    ollama_web_search("What are Ollama Cloud Models?")
except Exception as e:
    print(e)

Thinking

Thinking-capable models can generate a distinct "thinking trace" that details their reasoning process, separate from the final output. This feature allows for auditing the model's steps, visualizing its thought process in user interfaces, or concealing the trace when only the final answer is required.

Supported models

deepseek-v3.1:671b-cloud
gpt-oss:20b-cloud
gpt-oss:120b-cloud
Browse the latest additions under thinking models

try:
  message = """Solve this expression and explain each algebraic step in plain English: $\frac{\frac{1}{x} + \frac{1}{x + 1}}{\frac{1}{x} - \frac{1}{x + 1}}$"""

  think_messages = [{'role': 'user', 'content': message}]

  thinking_result = client.chat(model='deepseek-v3.1:671b', messages=think_messages, think=True, stream=True)

  in_thinking = False

  for chunk in thinking_result:
    if chunk.message.thinking and not in_thinking:
      in_thinking = True
      print('Thinking:\n', end='')

    if chunk.message.thinking:
      print(chunk.message.thinking, end='')
    elif chunk.message.content:
      if in_thinking:
        print('\n\nAnswer:\n', end='')
        in_thinking = False
      print(chunk.message.content, end='')
except Exception as e:
  print(f'Error: {e}')

Streaming

Streaming lets you display text as the model generates it, rather than waiting for the full response. It's on by default in the REST API but off by default in SDKs, you must set stream=True to enable it there

Key Streaming Concepts

Chatting: Receive and render partial assistant messages in real time, as each chunk arrives.
Thinking: Some models include a thinking field in chunks, allowing you to optionally show the model's reasoning before the final answer.
Tool calling: Tool calls may appear incrementally in the stream; you can detect them, run the tools, and send the results back into the conversation.

query = """
Explain me this code snippet:

from ollama import chat

messages = [
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
]

response = chat('gemma3', messages=messages)
print(response['message']['content'])
"""

message = [{'role': 'user', 'content': query}]
for stream in client.chat(model='qwen3-coder:480b', messages=message, stream=True):
  print(stream.message.content, end='', flush=True)

Structured Outputs

Structured outputs let you enforce a specific JSON schema on model responses, ensuring reliable extraction of structured data, consistent replies, or formatted image descriptions

You can enable this in two ways:

Generic JSON: Set the format parameter to json to ensure the output is valid JSON.
Specific Schema: Provide a detailed JSON schema (using tools like Pydantic in Python or Zod in JavaScript) to the format parameter. This forces the model to return data matching your exact structure.

from pydantic import BaseModel, EmailStr, HttpUrl, Field
from typing import Optional

class Product(BaseModel):
    name: str = Field(..., description='Name of the product')
    description: str = Field(..., description='Description of the product')
    price: float = Field(0, description='Price of the product (e.g. 199.99)')
    currency: str = Field('USD', description='Currency of the price')
    category: Optional[str] = None # Category is not always present
    in_stock: bool = Field(..., description='Whether the product is in stock')

class ProviderDetails(BaseModel):
    name: str = Field(..., description='Name of the provider')
    description: str = Field(..., description='Description of the provider')
    email: EmailStr = Field(..., description='Email of the provider')
    phone: Optional[str] = None
    website: HttpUrl = Field(..., description='Website of the provider')
    address: str = Field(..., description='Address of the provider')

class Provider(BaseModel):
    provider: ProviderDetails = Field(None, description='Details of the provider')
    products: list[Product] = Field(default_factory=list, description='List of products provided by the provider')

def extract_provider_and_products(text: str) -> Provider:
    """
    Takes a text and extracts Provider and Product information
    structured according to the defined Pydantic models.
    """
    prompt = f"""
    Analyze the following text and extract the information about the Provider and their Products into a JSON format.
    The JSON must contain a top-level key 'provider' which holds an object with the fields:
    'name', 'description', 'email', 'phone', 'website', and 'address.
    It must also contain a top-level key 'products' which is an list of objects. Each product object must have the fields:
    'name', 'description', 'price' (as a number, e.g., 199.99, not $199.99), 'currency', 'category', and 'in_stock' (as a boolean, true/false).

    Text: {text}

    Ensure the output conforms strictly to the JSON format provided.
    If a field is not present, omit it (do not return null).
    Only return the JSON, do not include any markdown or other text.
    """

    response = client.chat(
        model='gpt-oss:120b', # Replace with your chosen model
        messages=[{'role': 'user', 'content': prompt}],
        format=Provider.model_json_schema(), # Use the schema to enforce structure
        options={'temperature': 0},  # Set temperature to 0 for more deterministic output
    )

    # Validate the model response against the Pydantic schema
    provider_data = Provider.model_validate_json(response.message.content)
    return provider_data


supplier_text = """
Meet TechSolutions Inc., a leading provider of innovative software development tools and services.
We are headquartered in Austin, Texas, and have been delivering cutting-edge solutions to businesses
worldwide for the past eight years. You can reach us at contact@techsolutions.com or call (555) 987-6543.
Visit our website: https://www.techsolutions.com

Our current product catalog includes:
- CodeMaster Pro IDE: Advanced integrated development environment, $199.99/user/year, category software, currently in stock.
- BugFinder Suite: Comprehensive debugging toolkit, $49.99/month, category software, currently in stock.
- CloudDeploy Platform: Automated deployment service, $29.99/month per project, category software, currently in stock.
"""

try:
  extracted_data = extract_provider_and_products(supplier_text)
  print(extracted_data.model_dump_json(indent=2))
except Exception as e:
  print(f'Error during extraction or validation: {e}')

Vision

Vision models are AI models that can process both images and text to perform tasks like describing, classifying, and answering questions about visual content.

Supported models

qwen3-vl:235b-cloud
Browse the latest additions under vision models

Request (with images)

To submit images to vision models such as qwen3-vl, llava or bakllava, provide a list of base64-encoded images.

import base64
from IPython.display import Image, display

image_path = '/content/image.jpg'
image_base64 = base64.b64encode(open(image_path, 'rb').read()).decode('utf-8')

messages = [
  {
    'role': 'user',
    'content': 'What is in this image?',
    'images': [image_base64]
  }
]

try:
  result = client.chat(model='qwen3-vl:235b', messages=messages, stream=True)
  for part in result:
    print(part.message.content, end='', flush=True)
  display(Image(data=image_path, width=300))
except Exception as e:
  print(f'Error: {e}')

Advanced use cases

The following projects demonstrate how Ollama Cloud models can support practical AI workflows: Document Comprehension extracts and summarizes content from PDFs, DOCX files, and images; Code Assistant generates, runs, and self-corrects Python code; and Text-to-SQL Agent translates natural-language questions into safe, executable queries against a database.

Document Comprehension

LLMs can't see files directly; they need raw text. The following code defines a DocumentProcessor class designed to extract and process content from various document types, including PDFs, DOCX files, and images.

Functionality:

Initialization (__init__)

The constructor __init__ initializes an Ollama client. It uses ollama_base_url and ollama_api_key (likely loaded from environment variables or Colab secrets) to connect to the Ollama service, which is used for AI-powered text processing.
PDF Text Extraction (_extract_text_from_pdf_bytes)

This method takes raw bytes of a PDF file. It uses the fitz library (PyMuPDF) to open the PDF, iterate through its pages, and extract all text. The extracted text is then truncated to the first 8000 characters and passed to the _clean_text method for further processing.
DOCX Text Extraction (_extract_text_from_docx_bytes)

Similar to PDF extraction, this method handles raw DOCX file bytes. It uses the python-docx library to open the document and concatenate text from all paragraphs. The extracted text is also truncated and sent to _clean_text.
Image Content Extraction (_extract_content_from_image_bytes)

For images, this method takes image bytes, base64-encodes them, and then sends them to a vision-capable Ollama model (qwen3-vl:235b). The prompt asks the model to extract all visible text and provide a concise summary. The model's response is then passed to _clean_text.
Text Cleaning and Summarization (_clean_text)

This is a crucial method that takes the extracted text (from PDF, DOCX, or image) and sends it to another Ollama model (gpt-oss:120b-cloud). It uses a detailed system prompt to instruct the model to create a comprehensive briefing document, including an executive summary and a detailed examination of themes, structured with headings and bullet points. This effectively cleans and summarizes the input text.
Document Processing Dispatcher (process_document)

This public method acts as a dispatcher. It takes a filename and file bytes, checks the file extension (.pdf, .docx, .jpg, .png), and calls the appropriate private extraction method (_extract_text_from_pdf_bytes, _extract_text_from_docx_bytes, or _extract_content_from_image_bytes). If the file type is not supported, it raises a ValueError.

The DocumentProcessor class provides a unified interface to extract and summarize information from different document formats using Ollama's language and vision models.

import fitz
import base64
import markdown
from io import BytesIO
from pathlib import Path
from docx import Document
from ollama import Client, ChatResponse
from IPython.display import display, HTML
from typing import List, Dict, Optional, AsyncIterator

class DocumentProcessor:

    def __init__(self) -> None:
        self.client = Client(host=ollama_base_url, headers={'Authorization': f'Bearer {ollama_api_key}'})

    def _extract_text_from_pdf_bytes(self, pdf_bytes: bytes) -> str:
        """
        Extract text from PDF bytes using PyMuPDF.

        Args:
            pdf_bytes (bytes): The raw bytes of the PDF file.

        Returns:
            str: The extracted text from the PDF pages.
        """
        doc = fitz.open(stream=pdf_bytes, filetype='pdf')
        text = ''
        for page in doc:
            text += page.get_text()
        doc.close()
        return self._clean_text(text[:8000])

    def _extract_text_from_docx_bytes(self, docx_bytes: bytes) -> str:
        """
        Extract text from DOCX bytes using python-docx.

        Args:
        docx_bytes (bytes): The raw bytes of the DOCX file.

        Returns:
        str: The extracted text from the DOCX paragraphs.

        """
        doc = Document(BytesIO(docx_bytes))
        text = ''
        for paragraph in doc.paragraphs:
            text += paragraph.text + "\n"
        return self._clean_text(text[:8000].strip())

    def _extract_content_from_image_bytes(self, image_bytes: bytes) -> str:
        """
        Extract content from an image using Ollama.

        Args:
            image_bytes (bytes): The raw bytes of the image.

        Returns:
            str: The extracted content from the image.
        """
        image_base64 = base64.b64encode(image_bytes).decode('utf-8')
        messages = [{
            'role': 'user',
            'content': 'Extract all visible text, and then generate a clear, and concise summary.',
            'images': [image_base64]
        }]

        try:
            result = self.client.chat(model='qwen3-vl:235b', messages=messages, think=True)
            return self._clean_text(result.message.content)
        except Exception as e:
            print(f'Error: {e}')

    def _clean_text(self, text: str) -> str:
        """
        Clean the extracted text by removing unwanted characters.

        Args:
            text (str): The extracted text.

        Returns:
            str: The cleaned text.
        """
        try:
            system_prompt = """
            You are an expert in information compression.
            Your task is to make the given text more concise while preserving all key information.
            Aim to reduce the word count by 25% without losing important content.
            Ensure the OUTPUT conforms strictly to the markdown format. Only return the markdown.
            """.strip()

            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": text}
            ]
            result = self.client.chat(model='gemma4:31b-cloud', messages=messages)
            return result.message.content
        except Exception as e:
            print(f'Error: {e}')
            return None

    def process_document(self, filename: str, file_bytes: bytes) -> str:
        """
        Process a document (PDF, DOCX, or IMAGE) and extract text.

        Args:
            filename (str): The name of the file.
            file_bytes (bytes): The raw bytes of the file.
        """
        if filename.lower().endswith(".pdf"):
            return self._extract_text_from_pdf_bytes(file_bytes)
        elif filename.lower().endswith(".docx"):
            return self._extract_text_from_docx_bytes(file_bytes)
        elif filename.lower().endswith(".jpg") or filename.lower().endswith(".png"):
            return self._extract_content_from_image_bytes(file_bytes)
        else:
            raise ValueError(f"Unsupported file extension for file: {filename}")

try:
    file = Path('/content/linux_cheatsheet.pdf')

    with open(file, 'rb') as f:
        file_bytes = f.read()

    document = DocumentProcessor()
    extracted_content = document.process_document(file.name, file_bytes)

    if not extracted_content:
        raise ValueError("No content extracted from the file.")

    html = markdown.markdown(extracted_content)
    display(HTML(html))

except Exception as e:
    print(f'Error: {e}')

Code Assistant

To build this code assistant, we'll use Python, Ollama Cloud Models, and the Python Subprocess module to run code.

All the tools are free and open source.

Create the Prompt Templates

This step is very important. Even small mistakes in prompts can cause endless debugging loops.

Adding the instruction Return ONLY executable Python code makes the system much more reliable.

SYSTEM_PROMPT = """
You are an expert Python engineer.

Your task is to:
1. Write clean Python code
2. Fix bugs when errors appear
3. Return ONLY executable Python code
4. Do not include markdown
5. Ensure the code runs correctly
""".strip()

DEBUG_PROMPT = """
The following Python code failed.

CODE:
{code}

ERROR:
{error}

Fix the issue and return corrected executable Python code only.
""".strip()

Build the Code Executor

This function runs the generated Python code and captures:

Standard output
Runtime errors
Timeout failures

In real-world systems, this part is usually separated using Sandboxes for security.

import subprocess

def run_code(file_path):
    """
    Run Python code from a file.

    Args:
        file_path (str): The path to the Python file.

    Returns:
        dict: A dictionary containing the execution result.
    """
    try:
        result = subprocess.run(
            ["python", file_path],
            capture_output=True,
            text=True,
            timeout=10
        )

        if result.returncode == 0:
            return {
                "success": True,
                "output": result.stdout
            }

        return {
            "success": False,
            "error": result.stderr
        }

    except Exception as e:
        return {
            "success": False,
            "error": str(e)
        }

Build the Self-Correcting Workflow

This code contains the main logic. The assistant keeps generating code, running it, checking for errors, fixing them, and trying again.

try:
    from httpx import ConnectError
    from ollama import Client, ResponseError
except ImportError as e:
    print(e)

else:
    client = Client(host=ollama_base_url, headers={'Authorization': f'Bearer {ollama_api_key}'})
finally:
    pass

def generate_code(prompt):
    """
    Generate Python code using Ollama.

    Args:
        prompt (str): The prompt for code generation.

    Returns:
        str: The generated Python code.
    """
    try:
        result = client.chat(model='gemma4:31b-cloud', messages=[{'role': 'user', 'content': prompt}])
        return result.message.content
    except ConnectError as e:
        msg = f"""
        Failed to connect to Ollama. Please check that Ollama is downloaded,
        running and accessible. https://ollama.com/download
        --------------------------------------------------------------------
        {e}
        """.strip()
        raise ValueError(msg) from e
    except ResponseError as e:
        msg = f"""
        Received an error from the Ollama API.
        Please check your Ollama server logs.
        --------------------------------------------------------------------
        {e}
        """.strip()
        raise ValueError(msg) from e
    except Exception as e:
        msg = f"""
        An unexpected error occurred while trying to connect to Ollama.
        Please check your Ollama server logs.
        --------------------------------------------------------------------
        {e}
        """.strip()
        raise ValueError(msg) from e

user_task = """
Create a Python script that:
1. Reads this CSV file, '/content/Salaries.csv'
2. Check data types of each column
3. Identify missing values in each column
4. Check and remove duplicate rows
5. Get a summary of numerical columns
""".strip()

code_prompt = f"""
{SYSTEM_PROMPT}

TASK:
{user_task}
""".strip()

try:
    generated_code = generate_code(code_prompt)
except ValueError as er:
    print(er)
    exit(1)

MAX_RETRIES = 5
generated_code_path = "/content/generated_code.py"

for attempt in range(MAX_RETRIES):

    with open(generated_code_path, "w") as f:
        f.write(generated_code)

    result = run_code(generated_code_path)

    if result["success"]:
        print("\nFINAL WORKING CODE:\n")
        print(generated_code)

        print("\nOUTPUT:\n")
        print(result["output"])

        break

    print(f"\nAttempt {attempt + 1} failed...")
    print(result["error"])

    debug_prompt = DEBUG_PROMPT.format(
        code=generated_code,
        error=result["error"]
    )

    generated_code = generate_code(debug_prompt)

else:
    print("Assistant failed after maximum retries.")


FINAL WORKING CODE:

import pandas as pd

try:
    # 1. Read the CSV file
    df = pd.read_csv('/content/Salaries.csv')

    # 2. Check data types of each column
    print("--- Data Types ---")
    print(df.dtypes)
    print("\n")

    # 3. Identify missing values in each column
    print("--- Missing Values ---")
    print(df.isnull().sum())
    print("\n")

    # 4. Check and remove duplicate rows
    duplicates_count = df.duplicated().sum()
    print(f"Number of duplicate rows found: {duplicates_count}")
    df = df.drop_duplicates()
    print("Duplicates removed.")
    print("\n")

    # 5. Get a summary of numerical columns
    print("--- Summary of Numerical Columns ---")
    print(df.describe())

except FileNotFoundError:
    print("Error: The file '/content/Salaries.csv' was not found.")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

OUTPUT:

--- Data Types ---
Id                    int64
EmployeeName         object
JobTitle             object
BasePay             float64
OvertimePay         float64
OtherPay            float64
Benefits            float64
TotalPay            float64
TotalPayBenefits    float64
Year                  int64
Notes               float64
Agency               object
Status              float64
dtype: object


--- Missing Values ---
Id                       0
EmployeeName             0
JobTitle                 0
BasePay                609
OvertimePay              4
OtherPay                 4
Benefits             36163
TotalPay                 0
TotalPayBenefits         0
Year                     0
Notes               148654
Agency                   0
Status              148654
dtype: int64


Number of duplicate rows found: 0
Duplicates removed.


--- Summary of Numerical Columns ---
                  Id        BasePay  ...  Notes  Status
count  148654.000000  148045.000000  ...    0.0     0.0
mean    74327.500000   66325.448840  ...    NaN     NaN
std     42912.857795   42764.635495  ...    NaN     NaN
min         1.000000    -166.010000  ...    NaN     NaN
25%     37164.250000   33588.200000  ...    NaN     NaN
50%     74327.500000   65007.450000  ...    NaN     NaN
75%    111490.750000   94691.050000  ...    NaN     NaN
max    148654.000000  319275.010000  ...    NaN     NaN

[8 rows x 10 columns]

Text-to-SQL Agent

This code demonstrates how to use the text-to-sql agent built with LangChain's create_agent.

The agent can answer natural language questions about the Northwind database (it describes a fictional store). You can ask it a question in plain language, such as "What is the average price of products in each category?" It then checks your database schema and creates a correct SQL query for you.

Creating the Database

We need some data to work with. We'll write a Python script to make a sample SQLite database.

import os
import sqlite3
import warnings

warnings.filterwarnings(action='ignore')

northwind_script = '/content/northwind_script.txt'
northwind_db = '/content/northwind.db'

if os.path.exists(northwind_script) and os.path.getsize(northwind_script) > 0:
    with open(northwind_script, "r", encoding="utf-8") as f:
        northwind = f.read()

try:
    # Connect to a database file (it will be created if it doesn't exist)
    conn = sqlite3.connect(northwind_db)

    # Create a cursor object to execute SQL commands
    cursor = conn.cursor()

    # Create a sample table
    cursor.executescript(northwind.strip())
except sqlite3.Error as e:
    print(f"An error occurred while connecting to SQLite: {e}")
except Exception as e:
    print(e)

else:
    cursor.execute("""
    SELECT name
    FROM sqlite_master
    WHERE type = 'table'
    ORDER BY name
    """.strip())

    tables = cursor.fetchall()

    print('Available tables:\n')
    for table in tables:
        print(table[0])

    cursor.execute("SELECT * FROM Customers LIMIT 5;")

    customers = cursor.fetchall()

    print(f"\nSample output:\n")
    for c in customers:
        print(c)
finally:
    # Commit changes and close the connection
    conn.commit()
    conn.close()
    print("\nDatabase connection closed safely.")

Available tables:

Categories
Customers
Employees
OrderDetails
Orders
Products
Shippers
Suppliers
sqlite_sequence

Sample output:

(1, 'Alfreds Futterkiste', 'Maria Anders', 'Obere Str. 57', 'Berlin', '12209', 'Germany')
(2, 'Ana Trujillo Emparedados y helados', 'Ana Trujillo', 'Avda. de la Constitución 2222', 'México D.F.', '5021', 'Mexico')
(3, 'Antonio Moreno Taquería', 'Antonio Moreno', 'Mataderos 2312', 'México D.F.', '5023', 'Mexico')
(4, 'Around the Horn', 'Thomas Hardy', '120 Hanover Sq.', 'London', 'WA1 1DP', 'UK')
(5, 'Berglunds snabbköp', 'Christina Berglund', 'Berguvsvägen 8', 'Luleå', 'S-958 22', 'Sweden')

Database connection closed safely.

Select an LLM

Set up your language model with the right parameters for your use case. Select a model that supports tool-calling

from langchain_ollama import ChatOllama
from langchain_core.rate_limiters import InMemoryRateLimiter

rate_limiter = InMemoryRateLimiter(
    requests_per_second=1/15,  # <-- Super slow! We can only make a request once every 10 seconds!!
    check_every_n_seconds=0.1,  # Wake up every 100 ms to check whether allowed to make a request,
    max_bucket_size=10,  # Controls the maximum burst size.
)

model = ChatOllama(
    model='gemma4:31b-cloud',
    base_url=ollama_base_url,
    temperature=0,
    client_kwargs={
        "headers": {
            "Authorization": f"Bearer {ollama_api_key}"
        }
    },
    rate_limiter=rate_limiter
)

Add tools for database interactions

We can implement database tools as thin wrappers using the @tool decorator from langchain.tools

Below are minimal tools for demonstration purposes. They are not intended to be secure or for production use.

from langchain.tools import tool

@tool
def sql_db_list_tables() -> str:
    """
    Input is an empty string, output is a comma-separated list of tables in the database.

    Args:
        None. This tool does not require any input.

    Return:
        A comma-separated string containing the names of all tables
        available in the database.
    """
    try:
        con = sqlite3.connect(northwind_db)
        cursor = con.cursor()
        cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
    except sqlite3.Error as e:
        return f"Error: {e}"
    else:
        tables = [row[0] for row in cursor.fetchall() if not row[0].startswith("sqlite_")]
        return ", ".join(tables)
    finally:
        con.close()

@tool
def sql_db_schema(table_names: str) -> str:
    """
    Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables.
    Be sure that the tables actually exist by calling sql_db_list_tables first!
    Example Input: table1, table2, table3

    Args:
        table_names:
            A comma-separated list of table names.
            Example: "Customers, Orders, Products"

    Return:
        A string containing the schema definitions and sample rows for
        each requested table.
    """
    try:
        con = sqlite3.connect(northwind_db)
        cursor = con.cursor()
        cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
        valid_tables = {row[0] for row in cursor.fetchall() if not row[0].startswith("sqlite_")}
        results = []
        for table in table_names.split(","):
            table = table.strip()
            if table not in valid_tables:
                results.append(f"Error: table_names {{{table!r}}} not found in database")
                continue
            cursor.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name=?;", (table,))
            schema_row = cursor.fetchone()
            if schema_row:
                results.append(schema_row[0])
                try:
                    quoted_table = '"' + table.replace('"', '""') + '"'
                    cursor.execute(f"SELECT * FROM {quoted_table} LIMIT 3;")
                    rows = cursor.fetchall()
                    if rows:
                        col_names = [description[0] for description in cursor.description]
                        results.append(
                            f"/*\n3 rows from {table} table:\n"
                            + "\t".join(col_names)
                            + "\n"
                            + "\n".join("\t".join(str(x) for x in row) for row in rows)
                            + "\n*/"
                        )
                except Exception as e:
                    results.append(f"Error fetching sample rows: {e}")
        return "\n\n".join(results)
    finally:
        con.close()

@tool
def sql_db_query(query: str) -> str:
    """
    Input to this tool is a detailed and correct SQL query, output is a result from the database.
    If the query is not correct, an error message will be returned.
    If an error is returned, rewrite the query, check the query, and try again.
    If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.

    Args:
        query:
            A detailed, syntactically correct SQL query to execute.

    Return:
        The result returned by the database, including rows or an
        appropriate error message when execution fails.
    """
    try:
        con = sqlite3.connect(northwind_db)
        cursor = con.cursor()
        cursor.execute(query)
    except sqlite3.Error as e:
        return f"Error: {e}"
    else:
        res = cursor.fetchall()
        return str(res)
    finally:
        con.close()

@tool
def sql_db_query_checker(query: str) -> str:
    """
    Use this tool to double check if your query is correct before executing it.
    Always use this tool before executing a query with sql_db_query!

    Args:
        query:
            The SQL query that should be validated.

    Return:
        A message indicating whether the SQL query is valid.
        If the query is invalid, the return value should contain the
        error and, when possible, guidance for correcting the query.
    """
    trigger_prompt = """
    {query}
    Double check the sqlite query above for common mistakes, including:
    - Using NOT IN with NULL values
    - Using UNION when UNION ALL should have been used
    - Using BETWEEN for exclusive ranges
    - Data type mismatch in predicates
    - Properly quoting identifiers
    - Using the correct number of arguments for functions
    - Casting to the correct data type
    - Using the proper columns for joins

    If there are any of the above mistakes, rewrite the query. If there are no mistakes, just reproduce the original query.

    Output the final SQL query only.

    SQL Query: """.format(query=query).strip()

    response = model.invoke(trigger_prompt)
    return response.text.strip()

tools = [
    sql_db_list_tables,
    sql_db_schema,
    sql_db_query,
    sql_db_query_checker
]

Create the agent

Use create_agent to build a ReAct agent with minimal code. The agent will interpret the request and generate a SQL command, which the tools will execute. If the command has an error, the error message is returned to the model. The model can then examine the original request and the new error message and generate a new command.

from langchain.agents import create_agent
from langchain.messages import HumanMessage
from langchain_core.messages import ChatMessage

system_prompt = """
You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct {dialect} query to run,
then look at the results of the query and return the answer. Unless the user
specifies a specific number of examples they wish to obtain, always limit your
query to at most {top_k} results.

You can order the results by a relevant column to return the most interesting
examples in the database. Never query for all the columns from a specific table,
only ask for the relevant columns given the question.

You MUST double check your query before executing it. If you get an error while
executing a query, rewrite the query and try again.

DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the
database.

To start you should ALWAYS look at the tables in the database to see what you
can query. Do NOT skip this step.

Then you should query the schema of the most relevant tables.
""".format(
    dialect="sqlite",
    top_k=5,
).strip()

agent = create_agent(
    model=model,
    tools=tools,
    system_prompt=system_prompt,
)

Run the agent

Run the agent on a sample query and observe its behavior:

How many customers are in each country?
Which customer spent the most money?
What is the average price of products in each category?
Show each employee with their number of orders and total sales.

Let's create a helper function to run queries.

def run_agent(question):
    """Ask the agent a question about the database"""
    messages = [
        ChatMessage(role="control", content="thinking"),
        HumanMessage(question),
    ]
    return agent.stream_events(
        {"messages": messages},
        version="v3",
    )

stream = run_agent("What are the 10 most expensive products?")

for kind, item in stream.interleave("messages", "tool_calls"):
    if kind == "messages":
        for token in item.text:
            print(token, end="", flush=True)
    elif kind == "tool_calls":
        print(f"\nTool call: {item.tool_name}({item.input})")
        for delta in item.output_deltas:
            print(delta, end="", flush=True)
        print(f"\nTool result: {item.output}")


Tool call: sql_db_list_tables({})

Tool result: content='Categories, Customers, Employees, Shippers, Suppliers, Products, Orders, OrderDetails' name='sql_db_list_tables' tool_call_id='cda6478d-c0f9-46be-86ac-ed315cdc662d'

Tool call: sql_db_schema({'table_names': 'Products'})

Tool result: content='CREATE TABLE Products(\n    ProductID INTEGER PRIMARY KEY AUTOINCREMENT,\n    ProductName TEXT,\n    SupplierID INTEGER,\n    CategoryID INTEGER,\n    Unit TEXT,\n    Price NUMERIC DEFAULT 0,\n\tFOREIGN KEY (CategoryID) REFERENCES Categories (CategoryID),\n\tFOREIGN KEY (SupplierID) REFERENCES Suppliers (SupplierID)\n)\n\n/*\n3 rows from Products table:\nProductID\tProductName\tSupplierID\tCategoryID\tUnit\tPrice\n1\tChais\t1\t1\t10 boxes x 20 bags\t18\n2\tChang\t1\t1\t24 - 12 oz bottles\t19\n3\tAniseed Syrup\t1\t2\t12 - 550 ml bottles\t10\n*/' name='sql_db_schema' tool_call_id='3bd168bc-65c1-42c1-bdbc-08a6ca7232ee'

Tool call: sql_db_query_checker({'query': 'SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10'})

Tool result: content='SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10' name='sql_db_query_checker' id='b541b0a4-e8ce-4146-b7a7-43c2517efcf3' tool_call_id='dc123a83-81a0-4137-990b-28854e3eba7e'
SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10
Tool call: sql_db_query({'query': 'SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10'})

Tool result: content='[(\'Côte de Blaye\', 263.5), (\'Thüringer Rostbratwurst\', 123.79), (\'Mishi Kobe Niku\', 97), ("Sir Rodney\'s Marmalade", 81), (\'Carnarvon Tigers\', 62.5), (\'Raclette Courdavault\', 55), (\'Manjimup Dried Apples\', 53), (\'Tarte au sucre\', 49.3), (\'Ipoh Coffee\', 46), (\'Rössle Sauerkraut\', 45.6)]' name='sql_db_query' tool_call_id='2ca7fd9c-2d45-400d-9ddd-c7108431f174'

The 10 most expensive products are:

1. **Côte de Blaye**: 263.5
2. **Thüringer Rostbratwurst**: 123.79
3. **Mishi Kobe Niku**: 97.0
4. **Sir Rodney's Marmalade**: 81.0
5. **Carnarvon Tigers**: 62.5
6. **Raclette Courdavault**: 55.0
7. **Manjimup Dried Apples**: 53.0
8. **Tarte au sucre**: 49.3
9. **Ipoh Coffee**: 46.0
10. **Rössle Sauerkraut**: 45.6

Example: Aggregation with GROUP BY

stream = run_agent('What is the total sales amount for each product?')

print(stream.output['messages'][-1].text)

The total sales amount for the top 5 products are as follows:

1. **Côte de Blaye**: 62,976.50
2. **Thüringer Rostbratwurst**: 20,796.72
3. **Raclette Courdavault**: 19,030.00
4. **Tarte au sucre**: 16,022.50
5. **Camembert Pierrot**: 14,620.00

Example: Complex Query with Multiple JOINs

stream = run_agent('Which customer has spent the most money?')

print(stream.output['messages'][-1].text)

 The customer who has spent the most money is **Ernst Handel**, with a total expenditure of approximately **$35,631.21**.

Conclution

Ollama's Cloud Models represent a significant leap forward in making powerful, large-scale AI accessible to developers without the need for high-end local hardware. By offloading computation to the cloud, users can seamlessly run massive models, such as deepseek-v3.1:671b-cloud, gpt-oss:120b-cloud, and qwen3-vl:235b-cloud, that would otherwise be impractical on personal machines.

These cloud models deliver a rich set of production ready capabilities:

Tool calling enables dynamic interaction with external APIs and services.
Thinking traces provide transparent, step-by-step reasoning for auditability and enhanced UX.
Streaming supports real-time, low-latency responses ideal for chat and interactive applications.
Structured outputs guarantee reliable, schema-compliant JSON—critical for data extraction, automation, and integration.
Vision support unlocks multimodal understanding, allowing models to interpret images alongside text.

With straightforward API access, environment-based authentication, and full compatibility with popular developer tools, Ollama Cloud lowers the barrier to building sophisticated AI applications. Whether you're prototyping, automating workflows, or deploying enterprise solutions, Ollama's cloud offering combines flexibility, control, and scalability bringing the future of open, on-demand AI within reach.