Ollama's cloud models are a new feature that allows users to run large language models without needing a powerful local GPU. These models are automatically offloaded to Ollama's cloud service, providing the same capabilities as local models while enabling the use of larger models that would typically not fit on a personal computer.
Ollama currently supports the following cloud models
deepseek-v3.1:671b-cloudgpt-oss:20b-cloudgpt-oss:120b-cloudkimi-k2:1t-cloudqwen3-coder:480b-cloudglm-4.6:cloudqwen3-vl:235b-cloud- Browse the latest additions ollama's cloud models
Cloud API Access
Cloud models can also be accessed directly on ollama.com API. In this mode, ollama acts as a remote Ollama host.
For direct access to ollama cloud api, first create an API key.
Then, set the OLLAMA_API_KEY environment variable to your API key.
export OLLAMA_API_KEY=your_api_key
Generating a response
First, install dependencies. Run the following in your terminal or notebook cell:
print("Installing ollama")
!pip install -qU ollama
print("Installing pymupdf")
!pip install -qU pymupdf
print("Installing IPython")
!pip install -qU IPython
print("Intalling markdown")
!pip install -qU markdown
print("Intalling sqlalchemy")
!pip install -qU sqlalchemy
print("Installing langchain")
!pip install -qU langchain
print("Installing langgraph")
!pip install -qU langgraph
print("Installing python docx")
!pip install -qU python-docx
print("Installing pytesseract")
!pip install -qU pytesseract
print("Installing python magic")
!pip install -qU python-magic
print("Installing python dotenv")
!pip install -qU python-dotenv
print("Installing langchain core")
!pip install -qU langchain-core
print("Installing email-validator")
!pip install -qU pydantic[email]
print("Installing langchain ollama")
!pip install -qU langchain-ollama
print("Installing langchain community")
!pip install -qU langchain-community
print("Installed packages successfully!")
This notebook runs in Google Colab.
Basic Usage
This code snippet connects to the Ollama cloud API using your API key, sends a question to a specific language model (gpt-oss:120b-cloud), and then prints the model's answer as it's generated
try:
import os
from google.colab import userdata
except Exception as e:
print(e)
else:
userdata_ollama = None
try:
userdata_ollama = userdata.get('OLLAMA_API_KEY')
userdata_ollama_url = userdata.get('OLLAMA_BASE_URL')
except:
pass
if userdata_ollama:
print("OLLAMA_API_KEY found in Secrets\n")
os.environ['OLLAMA_API_KEY'] = userdata_ollama
if userdata_ollama_url:
print("OLLAMA_BASE_URL found in Secrets\n")
os.environ['OLLAMA_BASE_URL'] = userdata_ollama_url
ollama_api_key = os.getenv('OLLAMA_API_KEY')
ollama_base_url = os.getenv('OLLAMA_BASE_URL')
finally:
pass
Custom Client
A custom client can be created by instantiating Client or AsyncClient from ollama
- All extra keyword arguments are passed into the httpx.Client
from httpx import ConnectError
from ollama import Client, ResponseError
try:
client = Client(
host=ollama_base_url,
headers={
'Authorization': f'Bearer {ollama_api_key}'
}
)
# List models available via the API
response = client.list()
except ConnectError as e:
msg = (
"Failed to connect to Ollama. Please check that Ollama is downloaded, "
"running and accessible. https://ollama.com/download"
)
raise ValueError(msg) from e
except ResponseError as e:
msg = (
"Received an error from the Ollama API. "
"Please check your Ollama server logs."
)
raise ValueError(msg) from e
except Exception as e:
msg = (
"An unexpected error occurred while trying to connect to Ollama. "
"Please check your Ollama server logs."
)
raise ValueError(msg) from e
else:
model_names = [model["model"] for model in response["models"]]
print(f"Models available via the API: {len(model_names)}\n")
for name in model_names:
print(name)
finally:
pass
messages = [{
'role': 'user',
'content': 'How are you?',
},]
for part in client.chat('gpt-oss:120b-cloud', messages=messages, stream=True):
print(part.message.content, end='', flush=True)
Capabilities
Ollama's cloud models offer advanced capabilities beyond basic text generation, tailored for developers and AI practitioners. Key features include tool calling (for integrating external functions), thinking traces (to reveal the model's reasoning), streaming (for real-time responses), structured outputs (to enforce reliable JSON schemas), and vision (for multimodal image understanding). Together, these enable the development of robust, scalable, and production ready AI applications with enhanced control and insight.
Tool Calling
Ollama offers support for tool calling, also referred to as function calling. This feature empowers a language model to utilize external tools or functions and integrate the outcomes of these tools into its responses.
Supported models
deepseek-v3.1:671b-cloudgpt-oss:20b-cloudgpt-oss:120b-cloud- Browse the latest additions models to utilize external tools
try:
from ollama import (
Client, ResponseError,
web_search, web_fetch
)
from datetime import datetime
from httpx import ConnectError
except ImportError as e:
print(e)
else:
client = Client(
host=ollama_base_url,
headers={
'Authorization': f'Bearer {ollama_api_key}'
}
)
finally:
pass
available_tools = {'web_fetch': web_fetch, 'web_search': web_search}
current_time = datetime.now().strftime("%A, %B %d, %Y at %I:%M %p")
SYSTEM_PROMPT = f"""
You are an intelligent assistant designed to provide accurate, relevant, and up-to-date information.
You have access to:
- web_search: to discover relevant and current sources
- web_fetch: to retrieve and analyze the contents of specific URLs
System Context:
The current time is '{current_time}'. Always prioritize the topic with current time, which should be used to find up-to-date information.
""".strip()
def ollama_web_search(message):
"""
Ollama web search can be used to augment models with the latest information to reduce hallucinations and improve accuracy.
Args:
- message (str): The user's question or query.
Returns:
- str: The model's response to the user's question.
Example:
ollama_web_search("What are Ollama's Cloud Models?")
"Ollama's cloud models are a new feature that allow users to run large language models without needing a powerful local GPU.
"""
try:
messages = [
{'role': 'system', 'content': SYSTEM_PROMPT},
{'role': 'user', 'content': message}
]
tool_calls = []
max_iterations = 0
iterations = 3
while max_iterations < iterations:
try:
result = client.chat(
model='gemma4:31b-cloud',
messages=messages,
tools=[web_fetch, web_search],
think=True,
# stream=True,
options={'temperature': 0},
)
except ConnectError as e:
msg = (
"Failed to connect to Ollama. Please check that Ollama is downloaded, "
"running and accessible. https://ollama.com/download"
)
raise ValueError(msg) from e
except ResponseError as e:
msg = (
"Received an error from the Ollama API. "
"Please check your Ollama server logs."
)
raise ValueError(msg) from e
except Exception as e:
msg = (
"An unexpected error occurred while trying to connect to Ollama. "
"Please check your Ollama server logs."
)
raise ValueError(e)
if result.message.tool_calls:
tool_calls.extend(result.message.tool_calls)
if result.message.thinking:
print("THINKING...\n\n")
print(f"{result.message.thinking}\n\n")
if result.message.content:
print("ANSWER...\n\n")
print(f"{result.message.content}\n")
messages.append(result.message)
if tool_calls:
tool_executed = False
for tool_call in tool_calls:
function_to_call = available_tools.get(tool_call.function.name)
if function_to_call:
tool_args = tool_call.function.arguments
tool_result = function_to_call(**tool_args)
# print(f"Executing tool: {tool_call.function.name}, with args: {tool_args}\n")
messages.append({
'role': 'tool',
'content': str(tool_result)[:2000 * 4],
'tool_name': tool_call.function.name
})
tool_executed = True
else:
# print(f'Tool {tool_call.function.name} not found')
messages.append({
'role': 'tool',
'content': f'Tool {tool_call.function.name} not found',
'tool_name': tool_call.function.name
})
tool_executed = True
if not tool_executed:
break
max_iterations += 1
# No tool calls, conversation is complete
except Exception as e:
print(f'Error: {e}')
try:
ollama_web_search("What are Ollama Cloud Models?")
except Exception as e:
print(e)
Thinking
Thinking-capable models can generate a distinct "thinking trace" that details their reasoning process, separate from the final output. This feature allows for auditing the model's steps, visualizing its thought process in user interfaces, or concealing the trace when only the final answer is required.
Supported models
deepseek-v3.1:671b-cloudgpt-oss:20b-cloudgpt-oss:120b-cloud- Browse the latest additions under thinking models
try:
message = """Solve this expression and explain each algebraic step in plain English: $\frac{\frac{1}{x} + \frac{1}{x + 1}}{\frac{1}{x} - \frac{1}{x + 1}}$"""
think_messages = [{'role': 'user', 'content': message}]
thinking_result = client.chat(model='deepseek-v3.1:671b', messages=think_messages, think=True, stream=True)
in_thinking = False
for chunk in thinking_result:
if chunk.message.thinking and not in_thinking:
in_thinking = True
print('Thinking:\n', end='')
if chunk.message.thinking:
print(chunk.message.thinking, end='')
elif chunk.message.content:
if in_thinking:
print('\n\nAnswer:\n', end='')
in_thinking = False
print(chunk.message.content, end='')
except Exception as e:
print(f'Error: {e}')
Streaming
Streaming lets you display text as the model generates it, rather than waiting for the full response. It's on by default in the REST API but off by default in SDKs, you must set stream=True to enable it there
Key Streaming Concepts
Chatting: Receive and render partial assistant messages in real time, as each chunk arrives.
Thinking: Some models include a
thinkingfield in chunks, allowing you to optionally show the model's reasoning before the final answer.Tool calling: Tool calls may appear incrementally in the stream; you can detect them, run the tools, and send the results back into the conversation.
query = """
Explain me this code snippet:
from ollama import chat
messages = [
{
'role': 'user',
'content': 'Why is the sky blue?',
},
]
response = chat('gemma3', messages=messages)
print(response['message']['content'])
"""
message = [{'role': 'user', 'content': query}]
for stream in client.chat(model='qwen3-coder:480b', messages=message, stream=True):
print(stream.message.content, end='', flush=True)
Structured Outputs
Structured outputs let you enforce a specific JSON schema on model responses, ensuring reliable extraction of structured data, consistent replies, or formatted image descriptions
You can enable this in two ways:
Generic JSON: Set the
formatparameter tojsonto ensure the output is valid JSON.Specific Schema: Provide a detailed JSON schema (using tools like
Pydanticin Python orZodin JavaScript) to the format parameter. This forces the model to return data matching your exact structure.
from pydantic import BaseModel, EmailStr, HttpUrl, Field
from typing import Optional
class Product(BaseModel):
name: str = Field(..., description='Name of the product')
description: str = Field(..., description='Description of the product')
price: float = Field(0, description='Price of the product (e.g. 199.99)')
currency: str = Field('USD', description='Currency of the price')
category: Optional[str] = None # Category is not always present
in_stock: bool = Field(..., description='Whether the product is in stock')
class ProviderDetails(BaseModel):
name: str = Field(..., description='Name of the provider')
description: str = Field(..., description='Description of the provider')
email: EmailStr = Field(..., description='Email of the provider')
phone: Optional[str] = None
website: HttpUrl = Field(..., description='Website of the provider')
address: str = Field(..., description='Address of the provider')
class Provider(BaseModel):
provider: ProviderDetails = Field(None, description='Details of the provider')
products: list[Product] = Field(default_factory=list, description='List of products provided by the provider')
def extract_provider_and_products(text: str) -> Provider:
"""
Takes a text and extracts Provider and Product information
structured according to the defined Pydantic models.
"""
prompt = f"""
Analyze the following text and extract the information about the Provider and their Products into a JSON format.
The JSON must contain a top-level key 'provider' which holds an object with the fields:
'name', 'description', 'email', 'phone', 'website', and 'address.
It must also contain a top-level key 'products' which is an list of objects. Each product object must have the fields:
'name', 'description', 'price' (as a number, e.g., 199.99, not $199.99), 'currency', 'category', and 'in_stock' (as a boolean, true/false).
Text: {text}
Ensure the output conforms strictly to the JSON format provided.
If a field is not present, omit it (do not return null).
Only return the JSON, do not include any markdown or other text.
"""
response = client.chat(
model='gpt-oss:120b', # Replace with your chosen model
messages=[{'role': 'user', 'content': prompt}],
format=Provider.model_json_schema(), # Use the schema to enforce structure
options={'temperature': 0}, # Set temperature to 0 for more deterministic output
)
# Validate the model response against the Pydantic schema
provider_data = Provider.model_validate_json(response.message.content)
return provider_data
supplier_text = """
Meet TechSolutions Inc., a leading provider of innovative software development tools and services.
We are headquartered in Austin, Texas, and have been delivering cutting-edge solutions to businesses
worldwide for the past eight years. You can reach us at contact@techsolutions.com or call (555) 987-6543.
Visit our website: https://www.techsolutions.com
Our current product catalog includes:
- CodeMaster Pro IDE: Advanced integrated development environment, $199.99/user/year, category software, currently in stock.
- BugFinder Suite: Comprehensive debugging toolkit, $49.99/month, category software, currently in stock.
- CloudDeploy Platform: Automated deployment service, $29.99/month per project, category software, currently in stock.
"""
try:
extracted_data = extract_provider_and_products(supplier_text)
print(extracted_data.model_dump_json(indent=2))
except Exception as e:
print(f'Error during extraction or validation: {e}')
Vision
Vision models are AI models that can process both images and text to perform tasks like describing, classifying, and answering questions about visual content.
Supported models
qwen3-vl:235b-cloud- Browse the latest additions under vision models
Request (with images)
To submit images to vision models such as qwen3-vl, llava or bakllava, provide a list of base64-encoded images.
import base64
from IPython.display import Image, display
image_path = '/content/image.jpg'
image_base64 = base64.b64encode(open(image_path, 'rb').read()).decode('utf-8')
messages = [
{
'role': 'user',
'content': 'What is in this image?',
'images': [image_base64]
}
]
try:
result = client.chat(model='qwen3-vl:235b', messages=messages, stream=True)
for part in result:
print(part.message.content, end='', flush=True)
display(Image(data=image_path, width=300))
except Exception as e:
print(f'Error: {e}')
Advanced use cases
The following projects demonstrate how Ollama Cloud models can support practical AI workflows: Document Comprehension extracts and summarizes content from PDFs, DOCX files, and images; Code Assistant generates, runs, and self-corrects Python code; and Text-to-SQL Agent translates natural-language questions into safe, executable queries against a database.
Document Comprehension
LLMs can't see files directly; they need raw text. The following code defines a DocumentProcessor class designed to extract and process content from various document types, including PDFs, DOCX files, and images.
Functionality:
-
Initialization (
__init__)The constructor
__init__initializes an Ollama client. It usesollama_base_urlandollama_api_key(likely loaded from environment variables or Colab secrets) to connect to the Ollama service, which is used for AI-powered text processing. -
PDF Text Extraction (
_extract_text_from_pdf_bytes)This method takes raw bytes of a PDF file. It uses the fitz library (PyMuPDF) to open the PDF, iterate through its pages, and extract all text. The extracted text is then truncated to the first 8000 characters and passed to the
_clean_textmethod for further processing. -
DOCX Text Extraction (
_extract_text_from_docx_bytes)Similar to PDF extraction, this method handles raw DOCX file bytes. It uses the python-docx library to open the document and concatenate text from all paragraphs. The extracted text is also truncated and sent to
_clean_text. -
Image Content Extraction (
_extract_content_from_image_bytes)For images, this method takes image bytes, base64-encodes them, and then sends them to a vision-capable Ollama model (qwen3-vl:235b). The prompt asks the model to extract all visible text and provide a concise summary. The model's response is then passed to _clean_text.
-
Text Cleaning and Summarization (
_clean_text)This is a crucial method that takes the extracted text (from PDF, DOCX, or image) and sends it to another Ollama model (gpt-oss:120b-cloud). It uses a detailed system prompt to instruct the model to create a comprehensive briefing document, including an executive summary and a detailed examination of themes, structured with headings and bullet points. This effectively cleans and summarizes the input text.
-
Document Processing Dispatcher (
process_document)This public method acts as a dispatcher. It takes a filename and file bytes, checks the file extension (.pdf, .docx, .jpg, .png), and calls the appropriate private extraction method (
_extract_text_from_pdf_bytes,_extract_text_from_docx_bytes, or_extract_content_from_image_bytes). If the file type is not supported, it raises a ValueError.
The DocumentProcessor class provides a unified interface to extract and summarize information from different document formats using Ollama's language and vision models.
import fitz
import base64
import markdown
from io import BytesIO
from pathlib import Path
from docx import Document
from ollama import Client, ChatResponse
from IPython.display import display, HTML
from typing import List, Dict, Optional, AsyncIterator
class DocumentProcessor:
def __init__(self) -> None:
self.client = Client(host=ollama_base_url, headers={'Authorization': f'Bearer {ollama_api_key}'})
def _extract_text_from_pdf_bytes(self, pdf_bytes: bytes) -> str:
"""
Extract text from PDF bytes using PyMuPDF.
Args:
pdf_bytes (bytes): The raw bytes of the PDF file.
Returns:
str: The extracted text from the PDF pages.
"""
doc = fitz.open(stream=pdf_bytes, filetype='pdf')
text = ''
for page in doc:
text += page.get_text()
doc.close()
return self._clean_text(text[:8000])
def _extract_text_from_docx_bytes(self, docx_bytes: bytes) -> str:
"""
Extract text from DOCX bytes using python-docx.
Args:
docx_bytes (bytes): The raw bytes of the DOCX file.
Returns:
str: The extracted text from the DOCX paragraphs.
"""
doc = Document(BytesIO(docx_bytes))
text = ''
for paragraph in doc.paragraphs:
text += paragraph.text + "\n"
return self._clean_text(text[:8000].strip())
def _extract_content_from_image_bytes(self, image_bytes: bytes) -> str:
"""
Extract content from an image using Ollama.
Args:
image_bytes (bytes): The raw bytes of the image.
Returns:
str: The extracted content from the image.
"""
image_base64 = base64.b64encode(image_bytes).decode('utf-8')
messages = [{
'role': 'user',
'content': 'Extract all visible text, and then generate a clear, and concise summary.',
'images': [image_base64]
}]
try:
result = self.client.chat(model='qwen3-vl:235b', messages=messages, think=True)
return self._clean_text(result.message.content)
except Exception as e:
print(f'Error: {e}')
def _clean_text(self, text: str) -> str:
"""
Clean the extracted text by removing unwanted characters.
Args:
text (str): The extracted text.
Returns:
str: The cleaned text.
"""
try:
system_prompt = """
You are an expert in information compression.
Your task is to make the given text more concise while preserving all key information.
Aim to reduce the word count by 25% without losing important content.
Ensure the OUTPUT conforms strictly to the markdown format. Only return the markdown.
""".strip()
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": text}
]
result = self.client.chat(model='gemma4:31b-cloud', messages=messages)
return result.message.content
except Exception as e:
print(f'Error: {e}')
return None
def process_document(self, filename: str, file_bytes: bytes) -> str:
"""
Process a document (PDF, DOCX, or IMAGE) and extract text.
Args:
filename (str): The name of the file.
file_bytes (bytes): The raw bytes of the file.
"""
if filename.lower().endswith(".pdf"):
return self._extract_text_from_pdf_bytes(file_bytes)
elif filename.lower().endswith(".docx"):
return self._extract_text_from_docx_bytes(file_bytes)
elif filename.lower().endswith(".jpg") or filename.lower().endswith(".png"):
return self._extract_content_from_image_bytes(file_bytes)
else:
raise ValueError(f"Unsupported file extension for file: {filename}")
try:
file = Path('/content/linux_cheatsheet.pdf')
with open(file, 'rb') as f:
file_bytes = f.read()
document = DocumentProcessor()
extracted_content = document.process_document(file.name, file_bytes)
if not extracted_content:
raise ValueError("No content extracted from the file.")
html = markdown.markdown(extracted_content)
display(HTML(html))
except Exception as e:
print(f'Error: {e}')
Code Assistant
To build this code assistant, we'll use Python, Ollama Cloud Models, and the Python Subprocess module to run code.
All the tools are free and open source.
Create the Prompt Templates
This step is very important. Even small mistakes in prompts can cause endless debugging loops.
Adding the instruction Return ONLY executable Python code makes the system much more reliable.
SYSTEM_PROMPT = """
You are an expert Python engineer.
Your task is to:
1. Write clean Python code
2. Fix bugs when errors appear
3. Return ONLY executable Python code
4. Do not include markdown
5. Ensure the code runs correctly
""".strip()
DEBUG_PROMPT = """
The following Python code failed.
CODE:
{code}
ERROR:
{error}
Fix the issue and return corrected executable Python code only.
""".strip()
Build the Code Executor
This function runs the generated Python code and captures:
- Standard output
- Runtime errors
- Timeout failures
In real-world systems, this part is usually separated using Sandboxes for security.
import subprocess
def run_code(file_path):
"""
Run Python code from a file.
Args:
file_path (str): The path to the Python file.
Returns:
dict: A dictionary containing the execution result.
"""
try:
result = subprocess.run(
["python", file_path],
capture_output=True,
text=True,
timeout=10
)
if result.returncode == 0:
return {
"success": True,
"output": result.stdout
}
return {
"success": False,
"error": result.stderr
}
except Exception as e:
return {
"success": False,
"error": str(e)
}
Build the Self-Correcting Workflow
This code contains the main logic. The assistant keeps generating code, running it, checking for errors, fixing them, and trying again.
try:
from httpx import ConnectError
from ollama import Client, ResponseError
except ImportError as e:
print(e)
else:
client = Client(host=ollama_base_url, headers={'Authorization': f'Bearer {ollama_api_key}'})
finally:
pass
def generate_code(prompt):
"""
Generate Python code using Ollama.
Args:
prompt (str): The prompt for code generation.
Returns:
str: The generated Python code.
"""
try:
result = client.chat(model='gemma4:31b-cloud', messages=[{'role': 'user', 'content': prompt}])
return result.message.content
except ConnectError as e:
msg = f"""
Failed to connect to Ollama. Please check that Ollama is downloaded,
running and accessible. https://ollama.com/download
--------------------------------------------------------------------
{e}
""".strip()
raise ValueError(msg) from e
except ResponseError as e:
msg = f"""
Received an error from the Ollama API.
Please check your Ollama server logs.
--------------------------------------------------------------------
{e}
""".strip()
raise ValueError(msg) from e
except Exception as e:
msg = f"""
An unexpected error occurred while trying to connect to Ollama.
Please check your Ollama server logs.
--------------------------------------------------------------------
{e}
""".strip()
raise ValueError(msg) from e
user_task = """
Create a Python script that:
1. Reads this CSV file, '/content/Salaries.csv'
2. Check data types of each column
3. Identify missing values in each column
4. Check and remove duplicate rows
5. Get a summary of numerical columns
""".strip()
code_prompt = f"""
{SYSTEM_PROMPT}
TASK:
{user_task}
""".strip()
try:
generated_code = generate_code(code_prompt)
except ValueError as er:
print(er)
exit(1)
MAX_RETRIES = 5
generated_code_path = "/content/generated_code.py"
for attempt in range(MAX_RETRIES):
with open(generated_code_path, "w") as f:
f.write(generated_code)
result = run_code(generated_code_path)
if result["success"]:
print("\nFINAL WORKING CODE:\n")
print(generated_code)
print("\nOUTPUT:\n")
print(result["output"])
break
print(f"\nAttempt {attempt + 1} failed...")
print(result["error"])
debug_prompt = DEBUG_PROMPT.format(
code=generated_code,
error=result["error"]
)
generated_code = generate_code(debug_prompt)
else:
print("Assistant failed after maximum retries.")
FINAL WORKING CODE:
import pandas as pd
try:
# 1. Read the CSV file
df = pd.read_csv('/content/Salaries.csv')
# 2. Check data types of each column
print("--- Data Types ---")
print(df.dtypes)
print("\n")
# 3. Identify missing values in each column
print("--- Missing Values ---")
print(df.isnull().sum())
print("\n")
# 4. Check and remove duplicate rows
duplicates_count = df.duplicated().sum()
print(f"Number of duplicate rows found: {duplicates_count}")
df = df.drop_duplicates()
print("Duplicates removed.")
print("\n")
# 5. Get a summary of numerical columns
print("--- Summary of Numerical Columns ---")
print(df.describe())
except FileNotFoundError:
print("Error: The file '/content/Salaries.csv' was not found.")
except Exception as e:
print(f"An unexpected error occurred: {e}")
OUTPUT:
--- Data Types ---
Id int64
EmployeeName object
JobTitle object
BasePay float64
OvertimePay float64
OtherPay float64
Benefits float64
TotalPay float64
TotalPayBenefits float64
Year int64
Notes float64
Agency object
Status float64
dtype: object
--- Missing Values ---
Id 0
EmployeeName 0
JobTitle 0
BasePay 609
OvertimePay 4
OtherPay 4
Benefits 36163
TotalPay 0
TotalPayBenefits 0
Year 0
Notes 148654
Agency 0
Status 148654
dtype: int64
Number of duplicate rows found: 0
Duplicates removed.
--- Summary of Numerical Columns ---
Id BasePay ... Notes Status
count 148654.000000 148045.000000 ... 0.0 0.0
mean 74327.500000 66325.448840 ... NaN NaN
std 42912.857795 42764.635495 ... NaN NaN
min 1.000000 -166.010000 ... NaN NaN
25% 37164.250000 33588.200000 ... NaN NaN
50% 74327.500000 65007.450000 ... NaN NaN
75% 111490.750000 94691.050000 ... NaN NaN
max 148654.000000 319275.010000 ... NaN NaN
[8 rows x 10 columns]
Text-to-SQL Agent
This code demonstrates how to use the text-to-sql agent built with LangChain's create_agent.
The agent can answer natural language questions about the Northwind database (it describes a fictional store). You can ask it a question in plain language, such as "What is the average price of products in each category?" It then checks your database schema and creates a correct SQL query for you.
Creating the Database
We need some data to work with. We'll write a Python script to make a sample SQLite database.
import os
import sqlite3
import warnings
warnings.filterwarnings(action='ignore')
northwind_script = '/content/northwind_script.txt'
northwind_db = '/content/northwind.db'
if os.path.exists(northwind_script) and os.path.getsize(northwind_script) > 0:
with open(northwind_script, "r", encoding="utf-8") as f:
northwind = f.read()
try:
# Connect to a database file (it will be created if it doesn't exist)
conn = sqlite3.connect(northwind_db)
# Create a cursor object to execute SQL commands
cursor = conn.cursor()
# Create a sample table
cursor.executescript(northwind.strip())
except sqlite3.Error as e:
print(f"An error occurred while connecting to SQLite: {e}")
except Exception as e:
print(e)
else:
cursor.execute("""
SELECT name
FROM sqlite_master
WHERE type = 'table'
ORDER BY name
""".strip())
tables = cursor.fetchall()
print('Available tables:\n')
for table in tables:
print(table[0])
cursor.execute("SELECT * FROM Customers LIMIT 5;")
customers = cursor.fetchall()
print(f"\nSample output:\n")
for c in customers:
print(c)
finally:
# Commit changes and close the connection
conn.commit()
conn.close()
print("\nDatabase connection closed safely.")
Available tables:
Categories
Customers
Employees
OrderDetails
Orders
Products
Shippers
Suppliers
sqlite_sequence
Sample output:
(1, 'Alfreds Futterkiste', 'Maria Anders', 'Obere Str. 57', 'Berlin', '12209', 'Germany')
(2, 'Ana Trujillo Emparedados y helados', 'Ana Trujillo', 'Avda. de la Constitución 2222', 'México D.F.', '5021', 'Mexico')
(3, 'Antonio Moreno Taquería', 'Antonio Moreno', 'Mataderos 2312', 'México D.F.', '5023', 'Mexico')
(4, 'Around the Horn', 'Thomas Hardy', '120 Hanover Sq.', 'London', 'WA1 1DP', 'UK')
(5, 'Berglunds snabbköp', 'Christina Berglund', 'Berguvsvägen 8', 'Luleå', 'S-958 22', 'Sweden')
Database connection closed safely.
Select an LLM
Set up your language model with the right parameters for your use case. Select a model that supports tool-calling
from langchain_ollama import ChatOllama
from langchain_core.rate_limiters import InMemoryRateLimiter
rate_limiter = InMemoryRateLimiter(
requests_per_second=1/15, # <-- Super slow! We can only make a request once every 10 seconds!!
check_every_n_seconds=0.1, # Wake up every 100 ms to check whether allowed to make a request,
max_bucket_size=10, # Controls the maximum burst size.
)
model = ChatOllama(
model='gemma4:31b-cloud',
base_url=ollama_base_url,
temperature=0,
client_kwargs={
"headers": {
"Authorization": f"Bearer {ollama_api_key}"
}
},
rate_limiter=rate_limiter
)
Add tools for database interactions
We can implement database tools as thin wrappers using the @tool decorator from langchain.tools
Below are minimal tools for demonstration purposes. They are not intended to be secure or for production use.
from langchain.tools import tool
@tool
def sql_db_list_tables() -> str:
"""
Input is an empty string, output is a comma-separated list of tables in the database.
Args:
None. This tool does not require any input.
Return:
A comma-separated string containing the names of all tables
available in the database.
"""
try:
con = sqlite3.connect(northwind_db)
cursor = con.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
except sqlite3.Error as e:
return f"Error: {e}"
else:
tables = [row[0] for row in cursor.fetchall() if not row[0].startswith("sqlite_")]
return ", ".join(tables)
finally:
con.close()
@tool
def sql_db_schema(table_names: str) -> str:
"""
Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables.
Be sure that the tables actually exist by calling sql_db_list_tables first!
Example Input: table1, table2, table3
Args:
table_names:
A comma-separated list of table names.
Example: "Customers, Orders, Products"
Return:
A string containing the schema definitions and sample rows for
each requested table.
"""
try:
con = sqlite3.connect(northwind_db)
cursor = con.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
valid_tables = {row[0] for row in cursor.fetchall() if not row[0].startswith("sqlite_")}
results = []
for table in table_names.split(","):
table = table.strip()
if table not in valid_tables:
results.append(f"Error: table_names {{{table!r}}} not found in database")
continue
cursor.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name=?;", (table,))
schema_row = cursor.fetchone()
if schema_row:
results.append(schema_row[0])
try:
quoted_table = '"' + table.replace('"', '""') + '"'
cursor.execute(f"SELECT * FROM {quoted_table} LIMIT 3;")
rows = cursor.fetchall()
if rows:
col_names = [description[0] for description in cursor.description]
results.append(
f"/*\n3 rows from {table} table:\n"
+ "\t".join(col_names)
+ "\n"
+ "\n".join("\t".join(str(x) for x in row) for row in rows)
+ "\n*/"
)
except Exception as e:
results.append(f"Error fetching sample rows: {e}")
return "\n\n".join(results)
finally:
con.close()
@tool
def sql_db_query(query: str) -> str:
"""
Input to this tool is a detailed and correct SQL query, output is a result from the database.
If the query is not correct, an error message will be returned.
If an error is returned, rewrite the query, check the query, and try again.
If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.
Args:
query:
A detailed, syntactically correct SQL query to execute.
Return:
The result returned by the database, including rows or an
appropriate error message when execution fails.
"""
try:
con = sqlite3.connect(northwind_db)
cursor = con.cursor()
cursor.execute(query)
except sqlite3.Error as e:
return f"Error: {e}"
else:
res = cursor.fetchall()
return str(res)
finally:
con.close()
@tool
def sql_db_query_checker(query: str) -> str:
"""
Use this tool to double check if your query is correct before executing it.
Always use this tool before executing a query with sql_db_query!
Args:
query:
The SQL query that should be validated.
Return:
A message indicating whether the SQL query is valid.
If the query is invalid, the return value should contain the
error and, when possible, guidance for correcting the query.
"""
trigger_prompt = """
{query}
Double check the sqlite query above for common mistakes, including:
- Using NOT IN with NULL values
- Using UNION when UNION ALL should have been used
- Using BETWEEN for exclusive ranges
- Data type mismatch in predicates
- Properly quoting identifiers
- Using the correct number of arguments for functions
- Casting to the correct data type
- Using the proper columns for joins
If there are any of the above mistakes, rewrite the query. If there are no mistakes, just reproduce the original query.
Output the final SQL query only.
SQL Query: """.format(query=query).strip()
response = model.invoke(trigger_prompt)
return response.text.strip()
tools = [
sql_db_list_tables,
sql_db_schema,
sql_db_query,
sql_db_query_checker
]
Create the agent
Use create_agent to build a ReAct agent with minimal code. The agent will interpret the request and generate a SQL command, which the tools will execute. If the command has an error, the error message is returned to the model. The model can then examine the original request and the new error message and generate a new command.
from langchain.agents import create_agent
from langchain.messages import HumanMessage
from langchain_core.messages import ChatMessage
system_prompt = """
You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct {dialect} query to run,
then look at the results of the query and return the answer. Unless the user
specifies a specific number of examples they wish to obtain, always limit your
query to at most {top_k} results.
You can order the results by a relevant column to return the most interesting
examples in the database. Never query for all the columns from a specific table,
only ask for the relevant columns given the question.
You MUST double check your query before executing it. If you get an error while
executing a query, rewrite the query and try again.
DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the
database.
To start you should ALWAYS look at the tables in the database to see what you
can query. Do NOT skip this step.
Then you should query the schema of the most relevant tables.
""".format(
dialect="sqlite",
top_k=5,
).strip()
agent = create_agent(
model=model,
tools=tools,
system_prompt=system_prompt,
)
Run the agent
Run the agent on a sample query and observe its behavior:
- How many customers are in each country?
- Which customer spent the most money?
- What is the average price of products in each category?
- Show each employee with their number of orders and total sales.
Let's create a helper function to run queries.
def run_agent(question):
"""Ask the agent a question about the database"""
messages = [
ChatMessage(role="control", content="thinking"),
HumanMessage(question),
]
return agent.stream_events(
{"messages": messages},
version="v3",
)
stream = run_agent("What are the 10 most expensive products?")
for kind, item in stream.interleave("messages", "tool_calls"):
if kind == "messages":
for token in item.text:
print(token, end="", flush=True)
elif kind == "tool_calls":
print(f"\nTool call: {item.tool_name}({item.input})")
for delta in item.output_deltas:
print(delta, end="", flush=True)
print(f"\nTool result: {item.output}")
Tool call: sql_db_list_tables({})
Tool result: content='Categories, Customers, Employees, Shippers, Suppliers, Products, Orders, OrderDetails' name='sql_db_list_tables' tool_call_id='cda6478d-c0f9-46be-86ac-ed315cdc662d'
Tool call: sql_db_schema({'table_names': 'Products'})
Tool result: content='CREATE TABLE Products(\n ProductID INTEGER PRIMARY KEY AUTOINCREMENT,\n ProductName TEXT,\n SupplierID INTEGER,\n CategoryID INTEGER,\n Unit TEXT,\n Price NUMERIC DEFAULT 0,\n\tFOREIGN KEY (CategoryID) REFERENCES Categories (CategoryID),\n\tFOREIGN KEY (SupplierID) REFERENCES Suppliers (SupplierID)\n)\n\n/*\n3 rows from Products table:\nProductID\tProductName\tSupplierID\tCategoryID\tUnit\tPrice\n1\tChais\t1\t1\t10 boxes x 20 bags\t18\n2\tChang\t1\t1\t24 - 12 oz bottles\t19\n3\tAniseed Syrup\t1\t2\t12 - 550 ml bottles\t10\n*/' name='sql_db_schema' tool_call_id='3bd168bc-65c1-42c1-bdbc-08a6ca7232ee'
Tool call: sql_db_query_checker({'query': 'SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10'})
Tool result: content='SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10' name='sql_db_query_checker' id='b541b0a4-e8ce-4146-b7a7-43c2517efcf3' tool_call_id='dc123a83-81a0-4137-990b-28854e3eba7e'
SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10
Tool call: sql_db_query({'query': 'SELECT ProductName, Price FROM Products ORDER BY Price DESC LIMIT 10'})
Tool result: content='[(\'Côte de Blaye\', 263.5), (\'Thüringer Rostbratwurst\', 123.79), (\'Mishi Kobe Niku\', 97), ("Sir Rodney\'s Marmalade", 81), (\'Carnarvon Tigers\', 62.5), (\'Raclette Courdavault\', 55), (\'Manjimup Dried Apples\', 53), (\'Tarte au sucre\', 49.3), (\'Ipoh Coffee\', 46), (\'Rössle Sauerkraut\', 45.6)]' name='sql_db_query' tool_call_id='2ca7fd9c-2d45-400d-9ddd-c7108431f174'
The 10 most expensive products are:
1. **Côte de Blaye**: 263.5
2. **Thüringer Rostbratwurst**: 123.79
3. **Mishi Kobe Niku**: 97.0
4. **Sir Rodney's Marmalade**: 81.0
5. **Carnarvon Tigers**: 62.5
6. **Raclette Courdavault**: 55.0
7. **Manjimup Dried Apples**: 53.0
8. **Tarte au sucre**: 49.3
9. **Ipoh Coffee**: 46.0
10. **Rössle Sauerkraut**: 45.6
Example: Aggregation with GROUP BY
stream = run_agent('What is the total sales amount for each product?')
print(stream.output['messages'][-1].text)
The total sales amount for the top 5 products are as follows:
1. **Côte de Blaye**: 62,976.50
2. **Thüringer Rostbratwurst**: 20,796.72
3. **Raclette Courdavault**: 19,030.00
4. **Tarte au sucre**: 16,022.50
5. **Camembert Pierrot**: 14,620.00
Example: Complex Query with Multiple JOINs
stream = run_agent('Which customer has spent the most money?')
print(stream.output['messages'][-1].text)
The customer who has spent the most money is **Ernst Handel**, with a total expenditure of approximately **$35,631.21**.
Conclution
Ollama's Cloud Models represent a significant leap forward in making powerful, large-scale AI accessible to developers without the need for high-end local hardware. By offloading computation to the cloud, users can seamlessly run massive models, such as deepseek-v3.1:671b-cloud, gpt-oss:120b-cloud, and qwen3-vl:235b-cloud, that would otherwise be impractical on personal machines.
These cloud models deliver a rich set of production ready capabilities:
-
Tool callingenables dynamic interaction with external APIs and services. -
Thinkingtraces provide transparent, step-by-step reasoning for auditability and enhanced UX. -
Streamingsupports real-time, low-latency responses ideal for chat and interactive applications. -
Structured outputsguarantee reliable, schema-compliant JSON—critical for data extraction, automation, and integration. -
Visionsupport unlocks multimodal understanding, allowing models to interpret images alongside text.
With straightforward API access, environment-based authentication, and full compatibility with popular developer tools, Ollama Cloud lowers the barrier to building sophisticated AI applications. Whether you're prototyping, automating workflows, or deploying enterprise solutions, Ollama's cloud offering combines flexibility, control, and scalability bringing the future of open, on-demand AI within reach.
Top comments (0)