DEV Community

Abhishek Gupta
Abhishek Gupta

Posted on

🚀 Streaming AI Responses with Groq SDK in Node.js (Llama 3.3 70B)

Artificial Intelligence feels much more natural when responses appear word by word instead of waiting for the complete answer.

Groq provides one of the fastest inference APIs available today, making streaming responses incredibly smooth.

In this article, we'll build a simple streaming chatbot using the Groq SDK and the Llama 3.3 70B Versatile model.

📦 Installation

Install the official SDK.

npm install groq-sdk
Enter fullscreen mode Exit fullscreen mode

🔑 Configure API Key

Create a .env file.

GROQ_API_KEY=your_api_key_here
Enter fullscreen mode Exit fullscreen mode

Install dotenv if you haven't already.

npm install dotenv
Enter fullscreen mode Exit fullscreen mode

📝 Complete Example 

import { Groq } from "groq-sdk";
import dotenv from "dotenv/config";

const groq = new Groq({
    apiKey: process.env.GROQ_API_KEY,
});

const chatCompletion = await groq.chat.completions.create({
    model: "llama-3.3-70b-versatile",
    messages: [
        {
            role: "user",
            content: "Explain Artificial Intelligence in simple words."
        }
    ],
    temperature: 1,
    max_completion_tokens: 1024,
    top_p: 1,
    stream: true,
});

for await (const chunk of chatCompletion) {
    process.stdout.write(
        chunk.choices[0]?.delta?.content || ""
    );
}
Enter fullscreen mode Exit fullscreen mode

📖 Understanding the Code

1. Import the SDK 

import { Groq } from "groq-sdk";
Enter fullscreen mode Exit fullscreen mode

This imports the official Groq client.

2. Create the Client 

const groq = new Groq({
    apiKey: process.env.GROQ_API_KEY,
});
Enter fullscreen mode Exit fullscreen mode

The SDK authenticates using your API key.

3. Send a Chat Request 

const chatCompletion = await groq.chat.completions.create({
    model: "llama-3.3-70b-versatile",
    messages: [
        {
            role: "user",
            content: "Explain Artificial Intelligence in simple words."
        }
    ],
    stream: true,
});
Enter fullscreen mode Exit fullscreen mode

Here we specify:

  • Model
  • User message
  • Streaming enabled

4. Read the Stream 

for await (const chunk of chatCompletion) {
    process.stdout.write(
        chunk.choices[0]?.delta?.content || ""
    );
}
Enter fullscreen mode Exit fullscreen mode

Instead of waiting for the whole response, each token is printed immediately.

Output looks like this:

Artificial Intelligence (AI) is...
...technology that allows...
...computers to learn...
...and solve problems...
Enter fullscreen mode Exit fullscreen mode

This creates a ChatGPT-like typing effect.

⚙️ Parameters Explained

Parameter Description
model AI model to use
messages Conversation history
temperature Controls creativity
top_p Controls token sampling
max_completion_tokens Maximum output length
stream Enables real-time streaming

💡 Why Use Streaming?

Streaming provides several advantages:

  • ⚡ Faster perceived response time
  • 😊 Better user experience
  • 💬 ChatGPT-like interface
  • 📉 Reduced waiting time
  • 🔄 Real-time updates

For chat applications, streaming is almost always the preferred approach.

🏗️ Perfect For

This pattern works great for building:

  • AI Chatbots
  • Customer Support Assistants
  • Coding Assistants
  • AI Terminal Applications
  • Documentation Assistants
  • AI Agents
  • Personal Assistants

🚀 Example Output 

User:
What is Machine Learning?

AI:
Machine Learning is a branch of Artificial Intelligence that enables computers to learn patterns from data without being explicitly programmed...
Enter fullscreen mode Exit fullscreen mode

The text appears progressively as the model generates it.

Top comments (0)