Conversation between two LLMs

I wanted to make two LLMs talk to each other. So I built an application that generates a conversation between two "personas".

Intro

Having two LLMs talk to each other is straightforward, and you can emulate this by generating an output on let's say ChatGPT, copying it and pasting it in Gemini. One fun fact is that a conversation between two LLMs never ends. LLMs are programmed to always generate a response based on an input, even if the input is short or repetitive. Another fun fact is that regardless of the starting point of the conversation, most times the conversation ends with the two LLMs saying goodbye to each other, for as long as you let them.

Context

Once I had the conversation between the two LLMs, I tried finding a good use case, and I think "Word of the day" is that, so I built an app that shows a different word each day, with its definition, an example and a conversation generated by the two LLms.

The conversation can be generated by one LLM, but for educational purposes I'm using two.

Initially I tried to get a random question out of the LLMs, but the output was very poor. Also, I tried to generate the definition using the LLMs, but since LLMs are probabilistic, there's a good chance of generating wrong definitions. Finally, because the website shows multiple languages, I tried to do the translation using the LLMs but in the end I decided not to, because of the same risks of getting wrong translations.

So, I left deterministic:

• Word of the Day
• Definition
• Translations

Non-deterministic:

• Conversation

Word of the Day

Architecture

Components

Nextjs

Nextjs has two main purposes:

• API endpoint that orchestrates the update

export async function POST(request) {
  const wordOfTheDay = await getWordOfTheDayFromMerriam();

  const conversation = await generateConversationWithOllama(wordOfTheDay, "en");
  const english = {
    wordOfTheDay,
    conversation,
  };

  const wordOfTheDayES = await getTranslationFor(wordOfTheDay, "es");
  const conversationES = await generateConversationWithOllama(
    wordOfTheDayES,
    "es"
  );

  const spanish = {
    wordOfTheDay: wordOfTheDayES,
    conversation: conversationES,
  };

  const data = {
    en: english,
    es: spanish,
  };

  await saveConversationNetlifyStore(data);

  await hitNetlifyWebhook();

  return Response.json({ success: true });
}

• Build the web app

export default async function Page() {
  const { wordOfTheDay, conversation } =
    await getConversationFromNetlifyStore();

  return (
    <main>
      <h1>{wordOfTheDay.word}</h1>
      <p>{wordOfTheDay.definition}</p>

      <div>
        {conversation.map((text, index) => (
          <p index={index}>{text}</p>
        ))}
      </div>
    </main>
  );
}

Note: There's more code behind, but this is a good reference for how I built it.

1. Merriam Webster

The first thing is to get the word of the day. Luckily Merriam Webster is accessible so the site can be fetched and the needed information can be extracted using regex. Since this is done once a day, the extra load sent to this site is minimal.

import { XMLParser } from "fast-xml-parser";

export async function getWordOfTheDayFromMerriam() {
  const response = await fetch(
    "https://www.merriam-webster.com/wotd/feed/rss2"
  );

  const xmlString = await response.text();

  const parser = new XMLParser({ ignoreAttributes: false });
  const result = parser.parse(xmlString);

  const description = result.rss.channel.item[0].description;

  const wordMatch = description.match(/is:\s*(\w+)/i);
  const word = wordMatch?.[1] ?? "";

  const definitionMatch = description.match(
    /means\s+"(.+?)"\s*(?:\/\/|See the entry)/is
  );
  const definition = definitionMatch?.[1] ?? "";

  const exampleRegex = /\/\/\s*(.*?)(?=\s*(?:\/\/|See the entry|>))/g;
  const examples = [...description.matchAll(exampleRegex)].map((m) =>
    m[1].trim()
  );

  return {
    word,
    definition,
    examples,
  };
}

2. Ollama

I'm running Ollama locally and I downloaded the models.

$ ollama pull qwen2.5:3b

$ ollama pull gemma2:9b

$ ollama list
NAME          ID              SIZE      MODIFIED
qwen2.5:3b    357c53fb659c    1.9 GB    8 weeks ago
gemma2:9b     ff02c3702f32    5.4 GB    8 weeks ago

No specific reason for why I picked these two models, other than I wanted to run the LLMs locally, so I picked two that were OK for the task (generating a conversation) considering the hardware on my machine. Worth mentioning that running better LLMs requires more resources, so the days when it runs on my machine but not on the server are gone. It's now very expensive to have the hardware to run the latest LLMs.

import { Ollama } from "ollama";

let historyA = [];
let historyB = [];

const ollama = new Ollama({ host: process.env.OLLAMA_API_URL });

export async function generateConversationWithOllama(wordOfTheDay, lang) {
  const systemPrompt = `
  You are a 20-year-old student. You're friendly and easy to talk to. You talk the way you'd text a friend — casual, warm, to the point.

  Chat in ${lang} only. This is a short conversation so keep it natural but don't drag it out.

  Keep each reply to 1–2 short sentences. React naturally and sometimes ask something back.

  Today's word: "${wordOfTheDay.word}"
  What it means: "${wordOfTheDay.definition}"

  Strict formatting rules:
  - Plain text only. No emojis, no emoticons, no symbols like :) or ^^.
  - No tags, labels, role markers, or special tokens of any kind.
  - Output nothing except the message itself.`;

  const response = await ollama.chat({
    model: "gemma2:9b",
    messages: [{ role: "system", content: systemPrompt }],
    stream: false,
    options: { temperature: 0.4, num_predict: 80 },
  });

  const responseA = response.message.content;

  historyA.push({ role: "assistant", content: responseA });
  historyB.push({ role: "user", content: responseA });

  return responseA.trim();
}

3. Translation

Initially I wanted the LLM to generate the translations, but the LLMs I'm using are not the best at it, and I wanted to run the LLMs locally. So, for this part I'm using @google-cloud/translate which by default doesn't use LLMs.

import { Translate } from "@google-cloud/translate/build/src/v2/index.js";

const translate = new Translate({
  key: process.env.GOOGLE_TRANSLATION_KEY,
});

export async function getTranslationFor(wordOfTheDay, targetLanguage) {
  const [translations] = await translate.translate(
    wordOfTheDay.word,
    targetLanguage
  );

  return translations;
}

4. Text to Speech

Once the text is generated and translated, the next thing is to get audio so that the conversation can be listened to. For this part I'm using Python, because the community around machine learning is bigger, and while there are packages that run on Node.js, I would need extra steps (adapter, bridge) to make it work, so it's better to use Python. I also wanted to run it locally, so I set up an api endpoint that receives a string and returns audio.

For TTS I'm using coqui/XTTS-v2 which is a cool model that produces pretty decent audio, it supports multiple languages and has multiple voices.

import os, io
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse

from TTS.api import TTS

tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2", gpu=False)

app = FastAPI(title="Local XTTS v2 Voice Server")

@app.post("/v1/tts")
async def generate_tts(request: TTSRequest):
    text = request.text
    language = request.lang
    temp_filename = "output/api_output.wav"

    try:
        # Synthesize audio file locally
        tts.tts_to_file(
            text=text,
            file_path=temp_filename,
            language=language,
            speaker="Damien Black"
        )

        # Read the file data into an in-memory byte buffer
        with open(temp_filename, "rb") as f:
            audio_buffer = io.BytesIO(f.read())

        # Clean up the file off your Mac's disk storage immediately
        if os.path.exists(temp_filename):
            os.remove(temp_filename)

        # Rewind buffer pointer and stream the raw binary stream data back to the client
        audio_buffer.seek(0)

        return StreamingResponse(audio_buffer, media_type="audio/wav")

    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

5. Update Cache

Netlify provides @netlify/blobs which is their own object storage system. Once the "payload" (texts plus audios) is ready, it's saved into Netlify. This way it will be available for the next step.

import { getStore } from "@netlify/blobs";

const store = getStore("my-store", {
  siteID: process.env.MY_SITE_ID,
  token: process.env.MY_SITE_TOKEN,
});

export async function saveConversationNetlifyStore(value) {
  await store.setJSON("data", value);
}

6. Deploy site

Netlify offers a hook that, when hit, triggers a new deploy. As part of the deploy, Netlify runs npm run build, in this case Nextjs generates a new version of the site and updates the CDN with these new files. As part of the build, the blob values previously saved are used, and this is how the new HTML gets the latest content.

async function hitNetlifyWebhook() {
  await fetch(`https://api.netlify.com/build_hooks/${process.env.MY_HOOK_ID}`, {
    method: "POST",
    body: JSON.stringify({}),
  });
}

Conclusion

Running LLMs locally is limited by the resources of your machine, and it's interesting because the race for better hardware started a while ago, but now there's also a race for better LLMs and big LLMs are being produced. Some companies are working on efficiency, and are generating smaller LLMs while keeping the same quality. The future is uncertain, and there are hardware limitations, even for data centers. Data centers already consume a significant percentage of electricity and water, so at what point is it enough?

This was a good exercise to get familiar with how to run models locally while understanding its limitations. However, it's hard to compete against the big companies, so in production, because of the importance of accuracy, I would most likely end up using their services.