OCI Generative AI and LangChain: Building Enterprise AI Applications with Oracle

Integrating Oracle Cloud Infrastructure (OCI) Generative AI with LangChain unlocks powerful capabilities for building enterprise-grade AI applications. This comprehensive guide explores the official integration, demonstrates practical implementations, and shows how Oracle Database 23ai complements LangChain workflows for complete end-to-end AI solutions.

What is LangChain?

LangChain is a powerful, open-source framework for developing applications powered by large language models (LLMs). It provides essential tools for managing workflows, maintaining context, and integrating with external systems, streamlining the development of AI-powered applications.

Key Capabilities:

LangChain acts as a bridge, connecting LLMs with other data sources and tools, dramatically simplifying application development. The framework enables developers to:

• Build chatbots and conversational AI
• Create autonomous agents
• Implement Retrieval-Augmented Generation (RAG) workflows
• Manage prompt templates and chains
• Maintain conversation memory
• Integrate with vector databases and external APIs

Language Support:

LangChain primarily supports Python but also has libraries for JavaScript, TypeScript, and Java (through LangChain4j).

OCI Generative AI Integration with LangChain

Oracle Cloud Infrastructure Generative AI has an official integration with LangChain, providing native support through wrapper classes that make OCI's LLMs and embedding models easily accessible within LangChain applications.

The Official langchain-oci Package

As of 2025, Oracle maintains the official langchain-oci package that replaces the earlier community integrations.

Important Migration Note: The OCI integrations available in langchain-community are now deprecated. All future development, bug fixes, and feature enhancements are hosted in the new langchain-oci package.

Installation:

pip install langchain-oci

This package provides:

• ChatOCIGenAI: Chat models from OCI Generative AI
• OCIGenAI: Text generation from OCI Generative AI
• OCIGenAIEmbeddings: Embedding models from OCI Generative AI
• ChatOCIModelDeployment: Integration with OCI Data Science model endpoints
• Oracle AI Vector Search: Native vector store integration

Available Model Classes

1. ChatOCIGenAI - Conversational AI

The primary class for building chat applications with OCI Generative AI models.

from langchain_oci import ChatOCIGenAI

llm = ChatOCIGenAI(
    model_id="meta.llama-4-maverick-17b-128e-instruct-fp8",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_COMPARTMENT_ID",
    model_kwargs={"temperature": 0, "max_tokens": 500},
    auth_profile="MY_AUTH_PROFILE",
    is_stream=True
)

response = llm.invoke("Sing a ballad of LangChain.")
print(response.content)

Supported Features:

• Multi-turn conversations with context retention
• Streaming responses for real-time output
• Structured output generation
• Tool calling and function execution
• Parallel tool calling (Llama 4+ models only)

2. OCIGenAI - Text Generation

For simpler text generation tasks without conversational context.

from langchain_oci import OCIGenAI

llm = OCIGenAI()
response = llm.invoke("The meaning of life is")
print(response)

3. OCIGenAIEmbeddings - Vector Representations

Convert text into semantic embeddings for similarity search and RAG applications.

from langchain_oci import OCIGenAIEmbeddings

embeddings = OCIGenAIEmbeddings(
    model_id="cohere.embed-v4.0",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="compartment_id"
)

# Single query embedding
query = "This is a query in English."
response = embeddings.embed_query(query)

# Multiple document embeddings
documents = ["This is a sample document", "and here is another one"]
response = embeddings.embed_documents(documents)

Authentication Methods:

LangChain OCI integration supports standard SDK authentication:

• API Key (default)
• Security Token
• Instance Principal
• Resource Principal

Advanced Features

Structured Output:

ChatOCIGenAI supports structured output generation using function calling, JSON schema, or JSON mode.

from langchain_oci import ChatOCIGenAI

llm = ChatOCIGenAI(
    model_id="meta.llama-4-maverick-17b-128e-instruct-fp8",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_COMPARTMENT_ID"
)

# Define your structured output schema
structured_llm = llm.with_structured_output(YourSchema)

Parallel Tool Calling:

Llama 4+ models support parallel tool calling, allowing multiple tools to execute simultaneously.

llm_with_tools = llm.bind_tools(
    [get_weather, calculate_tip, get_population],
    parallel_tool_calls=True  # Tools can execute simultaneously
)

Note: Llama 3.x (including 3.3) and Cohere models will raise an error if parallel tool calling is used.

LangChain Prompt Templates

LangChain prompts can be created using two types of prompt templates, each optimized for different use cases.

1. String Prompt Template

Outputs a simple string, ideal for basic text generation tasks.

from langchain_core.prompts import PromptTemplate

template = """
You are a helpful assistant. Translate the following text to French:
{text}
"""

prompt = PromptTemplate.from_template(template)

2. Chat Prompt Template

Supports a list of messages with different roles (system, user, assistant), perfect for conversational AI.

from langchain_core.prompts import ChatPromptTemplate

template = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant specialized in {domain}."),
    ("user", "{question}")
])

prompt = template.invoke({
    "domain": "Italian cuisine",
    "question": "What are the best ingredients for carbonara?"
})

Message Types:

• System messages: Set behavior and context
• User messages: User queries and inputs
• Assistant messages: Previous AI responses (for context)
• Function messages: Tool/function call results

Creating Chains with LCEL

LangChain Expression Language (LCEL) enables declarative chain creation, offering a more concise and composable approach than traditional Python classes.

Declarative Chains with LCEL

LCEL uses the pipe operator (|) to chain components together.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_oci import ChatOCIGenAI

# Define components
prompt = ChatPromptTemplate.from_template(
    "If the text is a question, first translate the question and then "
    "answer the question in French.: {text}"
)

llm = ChatOCIGenAI(
    model_id="meta.llama-4-scout-17b-16e-instruct",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="<sandbox-compartment>",
    model_kwargs={"temperature": 0, "max_tokens": 500}
)

output_parser = StrOutputParser()

# Create chain using LCEL
chain = prompt | llm | output_parser

# Execute chain
result = chain.invoke({"text": "What are the four seasons in a year?"})
print(result)

Output:

La traduction de la question est : Quelles sont les quatre saisons dans une année?
Les quatre saisons dans une année sont : le printemps, l'été, l'automne et l'hiver.

Traditional Python Classes

Alternatively, use Python classes like LLMChain for explicit chain construction.

from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain_oci import ChatOCIGenAI

llm = ChatOCIGenAI(model_id="meta.llama-4-scout-17b-16e-instruct", ...)
prompt = PromptTemplate.from_template("Translate to French: {text}")

chain = LLMChain(llm=llm, prompt=prompt)
result = chain.run(text="Hello, world!")

LCEL vs Python Classes:

• LCEL: More concise, composable, modern approach (recommended)
• Python Classes: More explicit, traditional, verbose

LangChain Memory: Maintaining Conversation Context

LangChain memory provides the ability to store information about past interactions, crucial for building conversational applications.

Memory Types

Conversation Buffer Memory:
Stores the entire conversation history.

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory()
memory.save_context({"input": "Hi"}, {"output": "Hello! How can I help?"})
memory.save_context({"input": "What's the weather?"}, {"output": "It's sunny!"})

print(memory.load_memory_variables({}))

Conversation Buffer Window Memory:
Maintains only the last K interactions to manage token limits.

from langchain.memory import ConversationBufferWindowMemory

memory = ConversationBufferWindowMemory(k=2)  # Keep last 2 exchanges

Conversation Summary Memory:
Uses an LLM to summarize conversation history, maintaining context while reducing tokens.

from langchain.memory import ConversationSummaryMemory

memory = ConversationSummaryMemory(llm=llm)

Oracle Database 23ai as a Vector Store

Oracle Database 23ai can be used as a powerful vector store to generate and manage embeddings, seamlessly integrating with LangChain for RAG applications.

Why Oracle Database 23ai for Vector Storage?

Oracle Database 23ai supports native AI Vector Search, enabling:

• Semantic similarity search on text, documents, images, and unstructured data
• Combined relational and vector queries in a single system
• Pre-filtering, in-filtering, and post-filtering for enhanced search capabilities
• Multiple distance metrics: Cosine, Euclidean, dot product
• HNSW indexing for high-performance similarity search
• Integration with existing enterprise data without data duplication

LangChain Oracle Vector Store Integration

Oracle AI Vector Search integrates with LangChain at various levels, providing document loaders, text splitters, embeddings, and vector stores.

Installation:

pip install langchain-oracle

Basic Vector Store Usage:

from langchain_community.vectorstores import OracleVS
from langchain_oracle.embeddings import OCIGenAIEmbeddings
from langchain.schema import Document
import oracledb

# Connect to Oracle Database 23ai
conn = oracledb.connect(
    user="username",
    password="password",
    dsn="localhost/FREEPDB1"
)

# Create embedding model
embeddings = OCIGenAIEmbeddings(
    model_id="cohere.embed-v4.0",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="compartment_id"
)

# Prepare documents
documents = [
    Document(
        page_content="LangChain is a framework for LLM applications",
        metadata={"id": "doc1", "source": "documentation"}
    ),
    Document(
        page_content="Oracle Database 23ai supports vector search natively",
        metadata={"id": "doc2", "source": "announcement"}
    )
]

# Create vector store
vector_store = OracleVS.from_documents(
    documents,
    embeddings,
    client=conn,
    table_name="LANGCHAIN_VECTORS",
    distance_strategy=DistanceStrategy.COSINE
)

Creating HNSW Index for Performance

import oraclevs

oraclevs.create_index(
    conn,
    vector_store,
    params={
        "idx_name": "hnsw_idx1",
        "idx_type": "HNSW",
        "accuracy": 97,  # Target accuracy percentage
        "parallel": 8     # Number of parallel workers
    }
)

Similarity Search

# Perform similarity search
query = "What is LangChain?"
results = vector_store.similarity_search(query, k=3)

for doc in results:
    print(f"Content: {doc.page_content}")
    print(f"Metadata: {doc.metadata}\n")

Accessing OCI Generative AI from Oracle Database

Oracle Database 23ai can directly access OCI Generative AI services through built-in APIs and Select AI capabilities.

Using DBMS_CLOUD for REST API Access

You can access OCI Generative AI service using the DBMS_CLOUD REST API directly from SQL.

-- Create credential for OCI authentication
BEGIN
  DBMS_CLOUD.CREATE_CREDENTIAL(
    credential_name => 'OCI_CRED',
    username => 'ocid1.user.oc1...',
    password => '<your-auth-token>'
  );
END;
/

-- Call OCI Generative AI endpoint
DECLARE
  l_response CLOB;
BEGIN
  l_response := DBMS_CLOUD.SEND_REQUEST(
    credential_name => 'OCI_CRED',
    uri => 'https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/...',
    method => 'POST',
    body => '{
      "model": "cohere.command-r-plus",
      "prompt": "Explain quantum computing",
      "max_tokens": 500
    }'
  );

  DBMS_OUTPUT.PUT_LINE(l_response);
END;
/

Select AI: Natural Language to SQL

Select AI enables SQL access to generative AI using Large Language Models (LLMs) and embedding models, including support for natural language to SQL query generation and retrieval-augmented generation.

How Select AI Works

Select AI translates natural language questions into SQL queries that the database understands, then uses LLMs trained to infer user intent to interpret queries and return desired answers.

Key Capabilities:

Select AI automatically builds and maintains a vector index using metadata for applicable objects in your schema, using semantic similarity search to choose the most relevant objects based on user prompts.

• Natural language to SQL (NL2SQL / text-to-SQL) query generation
• Automatic metadata augmentation with table/column comments, constraints, annotations
• Semantic similarity search to select relevant schema objects
• RAG support for context-augmented responses
• Multi-dialect support including Oracle SQL and SQLite

Setting Up Select AI

-- Create credential for LLM provider
BEGIN
  DBMS_CLOUD.CREATE_CREDENTIAL(
    credential_name => 'COHERE_CRED',
    username => 'COHERE',
    password => '<your-api-key>'
  );
END;
/

-- Create AI profile
BEGIN
  DBMS_CLOUD_AI.CREATE_PROFILE(
    profile_name => 'COHERE',
    attributes => '{
      "provider": "cohere",
      "credential_name": "COHERE_CRED",
      "model": "cohere.command-a-03-2025",
      "object_list": [{"owner": "ADB_USER"}],
      "max_tokens": 512,
      "temperature": 0.5,
      "comments": true,
      "constraints": true
    }'
  );
END;
/

Using Select AI

Run SQL from Natural Language:

SELECT AI What are the top 5 products by revenue this quarter?;

Show Generated SQL:

SELECT AI showsql What are the top 5 products by revenue this quarter?;

Explain SQL in Natural Language:

SELECT AI explainsql SELECT * FROM sales WHERE region = 'West';

Chat Mode:

SELECT AI chat Who won the 2024 World Series?;

Select AI with RAG

Select AI supports RAG using Oracle AI Vector Search, integrating vector database content to deliver more accurate results from LLMs through semantic similarity search.

Select AI RAG automates the orchestration steps of transforming prompts into vectors and performing semantic similarity search on vector database content, making it easy to use LLMs with your private data using natural language prompts.

Complete RAG Application Example

Here's a complete example combining OCI Generative AI, LangChain, and Oracle Database 23ai:

from langchain_oci import ChatOCIGenAI, OCIGenAIEmbeddings
from langchain_community.vectorstores import OracleVS
from langchain_core.prompts import ChatPromptTemplate
from langchain.schema import Document
from langchain_core.output_parsers import StrOutputParser
import oracledb

# 1. Connect to Oracle Database 23ai
conn = oracledb.connect(
    user="username",
    password="password",
    dsn="localhost/FREEPDB1"
)

# 2. Initialize embedding model
embeddings = OCIGenAIEmbeddings(
    model_id="cohere.embed-v4.0",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="compartment_id"
)

# 3. Load documents into vector store
documents = [
    Document(page_content="Your document content here", metadata={"source": "doc1"}),
    # ... more documents
]

vector_store = OracleVS.from_documents(
    documents,
    embeddings,
    client=conn,
    table_name="KNOWLEDGE_BASE"
)

# 4. Initialize LLM
llm = ChatOCIGenAI(
    model_id="meta.llama-4-scout-17b-16e-instruct",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="compartment_id",
    model_kwargs={"temperature": 0.3, "max_tokens": 1000}
)

# 5. Create RAG chain
template = """
Answer the question based on the following context:

Context: {context}

Question: {question}

Answer:
"""

prompt = ChatPromptTemplate.from_template(template)

# 6. Query function
def ask_question(question: str):
    # Retrieve relevant documents
    docs = vector_store.similarity_search(question, k=3)
    context = "\n\n".join([doc.page_content for doc in docs])

    # Create and execute chain
    chain = prompt | llm | StrOutputParser()
    response = chain.invoke({"context": context, "question": question})

    return response

# 7. Use the RAG system
answer = ask_question("What are the benefits of Oracle Database 23ai?")
print(answer)

Best Practices for OCI + LangChain Applications

1. Choose the Right Components

• ChatOCIGenAI for conversational applications
• OCIGenAI for simple text generation
• OCIGenAIEmbeddings for semantic search and RAG
• Oracle Database 23ai as vector store for enterprise data

2. Optimize Embedding Storage

• Use appropriate distance metrics (cosine for text similarity)
• Create HNSW indexes for large datasets
• Leverage metadata filtering for refined searches
• Batch document processing for efficiency

3. Manage Memory Effectively

• Use ConversationBufferWindowMemory for long conversations
• Implement ConversationSummaryMemory for token efficiency
• Clear memory when starting new conversation threads

4. Leverage LCEL for Composability

• Build modular, reusable chains
• Combine multiple chains for complex workflows
• Use parallel execution where appropriate

5. Implement Proper Error Handling

try:
    response = chain.invoke(input_data)
except Exception as e:
    logging.error(f"Chain execution failed: {e}")
    # Implement fallback behavior

6. Monitor and Optimize Costs

• Cache frequently accessed embeddings
• Use appropriate model sizes (Scout vs Maverick)
• Implement rate limiting for user requests
• Monitor token usage across applications

The integration of OCI Generative AI with LangChain provides a powerful, enterprise-ready platform for building AI applications. Key takeaways:

Official Integration:

• langchain-oci package provides native support for OCI Generative AI
• Supports chat models, text generation, and embeddings
• Compatible with OCI Data Science model deployments

Prompt Engineering:

• String and Chat prompt templates for different use cases
• LCEL for declarative chain creation
• Memory management for conversational context

Oracle Database 23ai:

• Native vector store with LangChain integration
• Combines relational and vector search in one system
• Select AI for natural language to SQL generation

Complete AI Stack:
By combining OCI Generative AI (LLMs + embeddings), LangChain (orchestration), and Oracle Database 23ai (vector store + Select AI), you can build sophisticated RAG applications, conversational agents, and enterprise AI solutions with security, scalability, and performance.