DEV Community

Cover image for Building a Streaming Chatbot with Node.js
Marce
Marce

Posted on

Building a Streaming Chatbot with Node.js

#tutorial #javascript #ai #node

Most backend tutorials show synchronous calls: the API generates the full response and only then prints it, after several seconds of waiting. That’s not how real chat products work — in interfaces like ChatGPT or Claude, the response starts appearing word by word, before it’s even complete. That’s called streaming, and it’s what we’re going to build today.

What you’ll learn

  • Rendering LLM responses token by token in the console
  • Understanding the streaming architecture (real-time data events, similar to SSE)
  • A reusable pattern that works the same way with OpenAI, Anthropic, and Groq

Requirements

  • Node.js 18 or higher
  • npm
  • A free API key, no credit card required — we’ll use Groq

Step 1: Project setup

mkdir llm-chatbot-tutorial
cd llm-chatbot-tutorial
npm init -y
...
Enter fullscreen mode Exit fullscreen mode

Install the dependencies. We use the openai package because Groq’s API is compatible with its format — the same SDK, just pointed at a different URL:

npm install openai dotenv
Enter fullscreen mode Exit fullscreen mode

Step 2: Protect your credentials from the start

Create .gitignore :

node_modules/
.env
Enter fullscreen mode Exit fullscreen mode

Create .env.example (this one does get committed):

GROQ_API_KEY=your_api_key_here
Enter fullscreen mode Exit fullscreen mode

And your real .env (this one is never committed):

GROQ_API_KEY=your_api_key_here
Enter fullscreen mode Exit fullscreen mode

Step 3: The chatbot code
Create chatbot.js :

require("dotenv").config();
const OpenAI = require("openai");

const client = new OpenAI({
  apiKey: process.env.GROQ_API_KEY,
  baseURL: "https://api.groq.com/openai/v1",
});

// Check console.groq.com/docs/models for the current model list —
// providers rotate models periodically.
const MODEL = "openai/gpt-oss-120b";

async function chat(userMessage) {
  console.log(`\n📝 User: ${userMessage}\n`);
  console.log("🤖 Assistant: ");

  try {
    const stream = await client.chat.completions.create({
      model: MODEL,
      stream: true,
      messages: [
        { role: "user", content: userMessage },
      ],
    });

    for await (const chunk of stream) {
      const text = chunk.choices[0]?.delta?.content || "";
      process.stdout.write(text);
    }

    console.log("\n");
  } catch (error) {
    console.error("Error:", error.message);
  }
}

async function main() {
  console.log("=== Streaming Chatbot with Node.js ===\n");
  await chat("What is the capital of Argentina?");
  await chat("Explain what Node.js is in two sentences.");
  await chat("Give me 3 tips for learning to code.");
  console.log("\n✅ Done!");
}

main();
Enter fullscreen mode Exit fullscreen mode

What stream: true actually does
Without streaming, the API waits until the full response is generated and only then returns it as a single block. With stream: true , the server instead sends the response as a sequence of small fragments ( delta objects) as the model generates each token.
The for await loop processes each fragment as soon as it arrives. We use
process.stdout.write(text) instead of console.log(text) on purpose: console.log adds a line break after every call, which would break the live-typing visual effect.
This pattern stays the same across providers — only the SDK import and a couple of field names change. Once you understand this shape, you can swap providers with minimal effort.

Step 4: Run it

node chatbot.js
Enter fullscreen mode Exit fullscreen mode

You should see the three questions answered one after another, with text streaming in real time —not appearing all at once.

*If you hit an error: *- Check that .env is in the project root - Confirm the variable is named exactly GROQ_API_KEY - Verify the configured model is still available at console.groq.com/docs/models

Step 5: Customize it

Change the questions — edit the chat(...) calls inside main() .
Add a system prompt to control the assistant’s tone or role:

messages: [
  { role: "system", content: "You are a concise, friendly coding tutor." },
  { role: "user", content: userMessage },
],
Enter fullscreen mode Exit fullscreen mode

Swap the model — just change the MODEL constant. Always check the provider’s current model list before publishing or deploying, since free-tier models get deprecated and replaced over time.

Conclusion
You now have a working, streaming LLM chatbot in under 60 lines of Node.js — and, more importantly, you understand the streaming pattern that powers every modern AI chat interface. This is the foundation for anything more advanced: a web-based chat UI, a Slack bot, or a backend service that talks to an LLM.
Next in this series: containerizing this app with Docker and deploying it to the cloud.

Top comments (0)