OrkaJS: The TypeScript Framework That Makes LLM Development Actually Simple

If you've tried building an LLM-powered application recently, you know the pain. The ecosystem is fragmented, frameworks are complex, and you're often locked into specific providers. After months of frustration, I built OrkaJS to solve these problems.

The Problem

Let me paint a picture. You want to add RAG (Retrieval-Augmented Generation) to your app. Simple, right?

Reality check:

• Learn a framework-specific API (steep learning curve)
• Get locked into one LLM provider
• Manually handle retries, caching, and error handling
• Write custom code for each vector database
• Spend weeks on infrastructure instead of features

What you actually want:

const result = await orka.ask({
  question: 'How do I reset my password?',
  knowledge: 'support-docs'
});

That's it. That's OrkaJS.

What is OrkaJS?

OrkaJS is a progressive TypeScript framework for building production-ready LLM systems. It's designed with three core principles:

1. Universal Compatibility - Works with ANY JS/TS framework (Next.js SSR, Express, NestJS, Fastify, AdonisJS, KoaJS, Hono...)
2. Pluggable Architecture - Swap LLMs or vector databases in 3 lines of code
3. Production-Ready - Resilience, caching, evaluation, and observability built-in

Quick Start (Literally 2 Minutes)

Installation

npm install orkajs

Basic RAG in 10 Lines

import { createOrka, OpenAIAdapter, MemoryVectorAdapter } from 'orkajs';

const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  vectorDB: new MemoryVectorAdapter(),
});

// Create knowledge base
await orka.knowledge.create({
  name: 'docs',
  source: ['OrkaJS is a TypeScript framework for LLM systems.'],
});

// Ask with RAG
const result = await orka.ask({
  knowledge: 'docs',
  question: 'What is OrkaJS?',
});

console.log(result.answer);

That's it. No complex configuration, no boilerplate, no vendor lock-in.

Why OrkaJS is Different

1. Works Everywhere

Same code, any framework:

// Next.js API Route
export async function POST(req: Request) {
  const { question } = await req.json();
  const answer = await orka.ask({ question, knowledge: 'docs' });
  return Response.json({ answer });
}

// Express.js
app.post('/ask', async (req, res) => {
  const { question } = req.body;
  const answer = await orka.ask({ question, knowledge: 'docs' });
  res.json({ answer });
});

// NestJS
@Post('ask')
async ask(@Body() body: AskDto) {
  return this.orka.ask({ question: body.question, knowledge: 'docs' });
}

2. Swap Providers in Seconds

// Development: Use Ollama locally
const orka = createOrka({
  llm: new OllamaAdapter({ model: 'llama2' }),
  vectorDB: new MemoryVectorAdapter(),
});

// Production: Switch to OpenAI + Pinecone
const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  vectorDB: new PineconeAdapter({ 
    apiKey: process.env.PINECONE_API_KEY,
    indexHost: process.env.PINECONE_HOST 
  }),
});

Same API. Zero code changes.

3. Production Patterns Built-In

Automatic Retries with Exponential Backoff

import { ResilientLLM } from 'orkajs/resilience';

const llm = new ResilientLLM(
  new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  { maxRetries: 3, backoffMs: 1000 }
);

Multi-Provider Fallback

import { FallbackLLM } from 'orkajs/resilience';

const llm = new FallbackLLM([
  new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  new AnthropicAdapter({ apiKey: process.env.ANTHROPIC_API_KEY }),
  new OllamaAdapter({ model: 'llama2' }), // Local fallback
]);

Intelligent Caching

import { CachedLLM } from 'orkajs/cache';
import { RedisCache } from 'orkajs/cache/redis';

const llm = new CachedLLM(
  new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  new RedisCache({ url: process.env.REDIS_URL })
);

Advanced Features

🤖 Intelligent Agents

OrkaJS includes multiple agent architectures:

ReAct Agent (Reasoning + Acting):

import { ReActAgent } from 'orkajs/agent/react';

const agent = new ReActAgent({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  tools: [searchTool, calculatorTool],
  maxIterations: 5,
});

const result = await agent.run('What is the weather in Paris and what is 25 * 4?');

Plan & Execute Agent:

import { PlanAndExecuteAgent } from 'orkajs/agent/plan-and-execute';

const agent = new PlanAndExecuteAgent({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  tools: [webSearchTool, calculatorTool, weatherTool],
});

const result = await agent.run('Research competitors and create a pricing strategy');

🔄 Multi-Model Orchestration

Router (route queries to specialized models):

import { RouterLLM } from 'orkajs/orchestration';

const router = new RouterLLM({
  router: new OpenAIAdapter({ model: 'gpt-4o-mini' }), // Fast router
  routes: [
    {
      name: 'code',
      llm: new OpenAIAdapter({ model: 'gpt-4o' }),
      condition: (prompt) => prompt.includes('code') || prompt.includes('function'),
    },
    {
      name: 'creative',
      llm: new AnthropicAdapter({ model: 'claude-3-5-sonnet-20241022' }),
      condition: (prompt) => prompt.includes('story') || prompt.includes('creative'),
    },
  ],
  fallback: new OpenAIAdapter({ model: 'gpt-4o-mini' }),
});

Consensus (multiple models vote):

import { ConsensusLLM } from 'orkajs/orchestration';

const consensus = new ConsensusLLM({
  llms: [
    new OpenAIAdapter({ model: 'gpt-4o' }),
    new AnthropicAdapter({ model: 'claude-3-5-sonnet-20241022' }),
    new MistralAdapter({ model: 'mistral-large-latest' }),
  ],
  strategy: 'majority', // or 'unanimous'
});

📊 Built-in Evaluation

import { TestRunner } from 'orkajs/evaluation';

const runner = new TestRunner(orka);

await runner.test('RAG Quality', async () => {
  const result = await orka.ask({
    question: 'How do I reset my password?',
    knowledge: 'support-docs',
    includeContext: true,
  });

  runner.assert.relevance(result.answer, result.context, 0.8);
  runner.assert.faithfulness(result.answer, result.context, 0.9);
  runner.assert.noHallucination(result.answer, result.context);
});

await runner.run();

🔍 Advanced Retrievers

import { MultiQueryRetriever } from 'orkajs/retrievers';

const retriever = new MultiQueryRetriever({
  vectorDB: new PineconeAdapter({ /* config */ }),
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }),
  numQueries: 3, // Generate 3 variations of the query
});

const chunks = await retriever.retrieve('How to reset password?');
// Generates: "reset password", "change password", "password recovery"
// Returns combined unique results

Real-World Example: Customer Support Bot

Here's a complete example of a production-ready support bot:

import { createOrka, OpenAIAdapter, PineconeAdapter } from 'orkajs';
import { ResilientLLM } from 'orkajs/resilience';
import { CachedLLM } from 'orkajs/cache';
import { RedisCache } from 'orkajs/cache/redis';

// 1. Setup with production patterns
const llm = new CachedLLM(
  new ResilientLLM(
    new OpenAIAdapter({ 
      apiKey: process.env.OPENAI_API_KEY,
      model: 'gpt-4o-mini',
    }),
    { maxRetries: 3, backoffMs: 1000 }
  ),
  new RedisCache({ url: process.env.REDIS_URL })
);

const orka = createOrka({
  llm,
  vectorDB: new PineconeAdapter({
    apiKey: process.env.PINECONE_API_KEY,
    indexHost: process.env.PINECONE_HOST,
  }),
});

// 2. Index your documentation
await orka.knowledge.create({
  name: 'support-docs',
  source: { path: './docs' }, // Auto-loads all files
  chunkSize: 1000,
  chunkOverlap: 200,
});

// 3. Handle user questions
export async function handleSupportQuestion(question: string) {
  const { answer, context } = await orka.ask({
    question,
    knowledge: 'support-docs',
    systemPrompt: 'You are a helpful support agent. Be concise and friendly.',
    topK: 5,
    includeContext: true,
  });

  return {
    answer,
    sources: context.map(c => ({
      content: c.content.slice(0, 100),
      score: c.score,
    })),
  };
}

Framework Comparison

Feature	OrkaJS	LangChain	LlamaIndex
Learning Curve	⭐⭐⭐⭐⭐ Low	⭐⭐ High	⭐⭐⭐ Medium
TypeScript-First	✅ Native	⚠️ Port	⚠️ Python-first
Framework Agnostic	✅ All JS/TS	⚠️ Limited	⚠️ Limited
Pluggable Providers	✅ 3 lines	⚠️ Complex	⚠️ Complex
Production Patterns	✅ Built-in	❌ Manual	❌ Manual
Tree-shaking	✅ Optimized	❌ No	❌ No
Bundle Size	🟢 Small	🔴 Large	🔴 Large

Architecture Philosophy

OrkaJS is built on three pillars:

1. Intent-Based API

Focus on what you want, not how to do it:

// ✅ OrkaJS - Intent-based
await orka.ask({ question, knowledge: 'docs' });

// ❌ Other frameworks - Implementation-based
const vectorStore = await loadVectorStore();
const retriever = vectorStore.asRetriever();
const chain = RetrievalQAChain.fromLLM(llm, retriever);
const result = await chain.call({ query: question });

2. Adapter Pattern

Every external dependency is an adapter implementing a simple interface:

interface LLMAdapter {
  generate(prompt: string, options?: GenerateOptions): Promise<string>;
  embed(texts: string[]): Promise<number[][]>;
}

interface VectorDBAdapter {
  upsert(chunks: Chunk[]): Promise<void>;
  search(query: number[], topK: number): Promise<ChunkResult[]>;
}

Swap implementations without changing your code.

3. Progressive Enhancement

Start simple, add complexity only when needed:

// Day 1: Simple RAG
const answer = await orka.ask({ question, knowledge: 'docs' });

// Week 2: Add caching
const llm = new CachedLLM(baseAdapter, cache);

// Month 3: Add multi-provider fallback
const llm = new FallbackLLM([primary, secondary, local]);

// Month 6: Add custom evaluation
await runner.test('Quality', async () => { /* tests */ });

Check out the Quick Start Guide and build your first LLM-powered feature in 10 minutes.