DEV Community

NeuML profile image David Mezzetti
David Mezzetti for NeuML

Posted on

OpenCode as a txtai LLM

#ai #llm #rag #vectordatabase

OpenCode is an open source AI coding agent. It's rapidly growing in popularity as an open alternative to Claude Code.

OpenCode shows the power of permissive open source. It's rapidly undergoing mass adoption and as of January 2026 sits at 81K+ GitHub ⭐'s. OpenCode supports running as a local server via opencode serve.

This article will explore how TxtAI integrates this as a LLM pipeline.

Install dependencies

Install txtai and all dependencies.

%%capture
!pip install txtai[api,pipeline-llm]

# Install OpenCode if not already installed and run serve
!curl -fsSL https://opencode.ai/install | bash
!opencode serve &
Enter fullscreen mode Exit fullscreen mode

Create a OpenCode LLM

Now that everything is installed, let's test it out!

from IPython.display import Markdown, display

from txtai import LLM

def generate(request):
    display(Markdown(llm(request)))

# Connect to OpenCode server and use default LLM provider
llm = LLM("opencode")

generate("Show a Python Hello World example")
Enter fullscreen mode Exit fullscreen mode 
print("Hello, World!")
Enter fullscreen mode Exit fullscreen mode

OK, now let's try this for a more advanced use case.

generate("Show a simple and basic TxtAI RAG example")
Enter fullscreen mode Exit fullscreen mode 
from txtai import Embeddings, LLM

# Create embeddings model
embeddings = Embeddings(path="sentence-transformers/all-MiniLM-L6-v2")

# Create LLM
llm = LLM("huggingface/Qwen/Qwen2.5-1.5B-Instruct")

# Sample documents
documents = [
    "Python is a programming language created by Guido van Rossum",
    "Machine learning is a subset of artificial intelligence",
    "Deep learning uses neural networks with multiple layers"
]

# Build the index
embeddings.index([(i, doc) for i, doc in enumerate(documents)])

# RAG function
def rag_query(question):
    # Search for relevant documents
    results = embeddings.search(question, 2)

    # Build context from retrieved documents
    context = " ".join([documents[result[0]] for result in results])

    # Generate response using LLM with context
    prompt = f"Context: {context}\n\nQuestion: {question}\n\nAnswer:"
    return llm(prompt)

# Example usage
question = "What is Python?"
answer = rag_query(question)
print(f"Question: {question}")
print(f"Answer: {answer}")
Enter fullscreen mode Exit fullscreen mode

Interesting! It's not limited to Python though.

generate("Show a x86 assembly hello world")
Enter fullscreen mode Exit fullscreen mode 
section .data
    hello db 'Hello, World!', 10    ; String with newline
    hello_len equ $ - hello         ; Length of string

section .text
    global _start

_start:
    ; sys_write system call
    mov eax, 4                      ; sys_write
    mov ebx, 1                      ; stdout
    mov ecx, hello                  ; message
    mov edx, hello_len              ; message length
    int 0x80                        ; kernel interrupt

    ; sys_exit system call
    mov eax, 1                      ; sys_exit
    mov ebx, 0                      ; exit code 0
    int 0x80                        ; kernel interrupt
Enter fullscreen mode Exit fullscreen mode

Compile and run:

nasm -f elf32 hello.asm -o hello.o
ld -m elf_i386 hello.o -o hello
./hello
Enter fullscreen mode Exit fullscreen mode 
generate("""
Fix the following error in the code below and explain what's wrong.

Error:
AttributeError: 'NoneType' object has no attribute 'startswith'

Code:
a = None
a.startswith("hello")
""")
Enter fullscreen mode Exit fullscreen mode 
a = None
if a is not None:
    a.startswith("hello")
else:
    print("a is None, cannot call startswith")
Enter fullscreen mode Exit fullscreen mode

What's wrong: The variable a is None, which is a NoneType object. The startswith() method only exists for strings, not for NoneType. You need to check if a is not None before calling string methods on it.

OpenAI-compatible Endpoint

Now that OpenCode is integrated into the TxtAI ecosystem, it opens a lot of different opportunities. Let's say we want to host a local OpenAI compatible endpoint, no problem!

Write the following file.

config.yml 

# Enable OpenAI compat endpoint
openai: True

llm:
  path: opencode
Enter fullscreen mode Exit fullscreen mode

and start the following process.

CONFIG=config.yml uvicorn "txtai.api:app"
Enter fullscreen mode Exit fullscreen mode 
from IPython.display import Markdown, display

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="api-key",
)

response = client.chat.completions.create(
    messages=[{
        "role": "user",
        "content": "Show a Python factorial function",
    }],
    model="opencode",
)

display(Markdown((response.choices[0].message.content)))
Enter fullscreen mode Exit fullscreen mode 
def factorial(n):
    if n < 0:
        raise ValueError("Factorial is not defined for negative numbers")
    if n == 0 or n == 1:
        return 1
    result = 1
    for i in range(2, n + 1):
        result *= i
    return result

# Example usage:
# print(factorial(5))  # Output: 120
Enter fullscreen mode Exit fullscreen mode

Just like before we get an answer! This time via the OpenAI client. Fun times.

Wrapping up

This article showed how to connect TxtAI and OpenCode. This will lead to some interesting integrations. One such example is ncoder, an AI coding agent that integrates with Jupyter Notebooks. Stay tuned for more!

Top comments (0)