Lean AI in LangBear

#ai #genai #langbear

LangBear is fully bootstrapped. This means no investors' money to burn, no big budgets. All funding has been coming from my main job's income. This is why I made a decision early on to stay AI provider-agnostic as much as possible to have the flexibility to migrate to cheaper options.

At first, I tried LangChain.js. It was too heavy and was burdensome to maintain. Too many dependencies. The API wasn't stable and oftentimes broke when I had to bump the version for security fixes.

I decided to write a very lightweight interface that would satisfy all use cases of the app.

I present it to you in all its glory as it is today:

import type z from "zod/v3";

export type AIProvider = {

  generateText<T extends z.ZodTypeAny>(
    prompt: string,
    schema: T,
    options?: AIProviderGenerateTextOptions,
  ): Promise<z.infer<T>>;

  transcribeAudio(filePath: string): Promise<string>;
};

export type AIProviderThinkingConfig =
  | {
      // 0 disables thinking, -1 lets the provider choose automatically
      thinkingBudget: 0 | -1;
      thinkingLevel?: never;
    }
  | {
      thinkingBudget?: never;
      thinkingLevel: "MINIMAL" | "LOW" | "MEDIUM" | "HIGH";
    };

export type AIProviderGenerateTextOptions = {
  model?: string;
  thinkingConfig?: AIProviderThinkingConfig;
};

In the code, you would use it like this:

function translateText(
  aiProvider: AIProvider,
  model: string,
  text: string,
  targetLang: string,
): Promise<string> {
  const { translatedText } = await aiProvider.generateText(
    getTranslateTextPrompt(text, targetLang),
    z.object({ translatedText: z.string() }),
    {
      model,
    },
  );

  return translatedText;
}

await translateText(geminiProvider, "gemini-3.5-flash", "hej", "en")
// hello

Implementation of this interface is trivial. Here's a partial implementation for the Gemini provider for generateText function:

async function generateText<T extends z.ZodTypeAny>(
  prompt: string,
  schema: T,
  { geminiApiKey, model, thinkingConfig }: GenerateTextOptions,
): Promise<z.infer<T>> {
  const gemini = new GoogleGenAI({
    apiKey: geminiApiKey,
  });

  const config: GenerateContentConfig = {
    responseMimeType: "application/json",
    responseJsonSchema: zodToJsonSchema(schema),
  };
  const geminiThinkingConfig = toGeminiThinkingConfig(thinkingConfig);

  if (geminiThinkingConfig) {
    config.thinkingConfig = geminiThinkingConfig;
  }

  const response = await gemini.models.generateContent({
    model,
    contents: prompt,
    config,
  });

  if (!response.text) {
    throw new Error("no content");
  }

  return schema.parse(JSON.parse(response.text));
}