King coder

Posted on Aug 16

🌟 Today I Learned: Basics of Generative AI (Gen AI) with Groq Cloud and Prompting

🤖 What is Generative AI?

Generative AI (Gen AI) is a type of artificial intelligence that can create new content like:

Text (e.g., writing emails, summaries, answers)
Images (e.g., AI-generated art)
Code (e.g., coding assistants like GitHub Copilot)
Music, videos, and more

🎯 In simple words: You give the AI a prompt (a message or question), and it gives you a new response or result.

🛠️ How Does Gen AI Work?

Here’s a basic process:

Training:

The AI is trained using massive datasets (books, articles, code, images). It learns patterns, language, and structure.

Understanding Prompts:

You give it a prompt like “Explain gravity,” and it uses its knowledge to give a relevant answer.

Generating Response:

It predicts the most likely next word/token — one at a time — to form a complete response.

🧱 What Are Tokens?

A token is a small piece of text — a word or part of a word.

AI doesn’t understand full sentences; it understands tokens.

Example:

"Hello" might be one token.
"unbelievable" might be split into "un", "believ", "able".

🔢 Token limits affect how long your prompts and responses can be.

⚠️ AI Has a Limitation: It's Trained on Old Data

Generative AI models are trained on existing data from the internet (books, websites, codebases), but they don’t know what’s happening right now.

❌ Ask: “What happened in the 2024 elections?”

AI might say: “I don't know,” because it wasn’t trained on that.

🧰 Why Do We Add Tools to AI?

Since the AI cannot access live data or the internet by default, we can add tools to give it more power.

✅ What tools allow AI to do:

Use Case	Without Tool	With Tool
Live Weather	Can’t answer	Fetches data from weather API
Stock Prices	Might be outdated	Gets real-time prices
Access Your Files	Can’t read local files	Tool connects to file system
Use Google Search	Doesn’t know how	Tool sends search query and returns result
Custom Code Execution	Not possible	Tool runs code and sends output

🚀 Using Groq Cloud to Run Gen AI Models

Groq Cloud is a platform that allows you to run powerful open-source generative AI models — instantly and at lightning-fast speeds ⚡. It removes the need for expensive hardware or complex local setups, making AI more accessible to everyone.

Instead of running models on your own machine or renting expensive GPUs, you can use Groq's cloud infrastructure to send prompts and receive AI-generated responses in real time.

🌟 Models Available on Groq

Groq supports a range of cutting-edge open-source models, including:

LLaMA 3 — Created by Meta, this model offers high-quality responses and strong reasoning capabilities, suitable for chat, summarization, and more.
Mixtral — A high-performance, multilingual model capable of handling complex prompts in multiple languages with strong accuracy.
Gemma — A lightweight and efficient model designed for fast performance with minimal resources, ideal for quick, low-latency tasks.
Falcon — An open-source large language model known for speed and versatility in many NLP tasks.
Mistral — A powerful, efficient model designed for state-of-the-art performance on various language understanding tasks.
OpenAssistant — A community-driven model designed to assist with conversational AI needs.

…and many more, constantly updated and expanded to provide the latest in generative AI capabilities.

💡 Why Use Groq Cloud and APIs for AI?

Training and building large AI models from scratch is extremely resource-intensive. Models today often have millions or even billions of parameters, which require vast amounts of:

Computational power (high-end GPUs, TPUs, or specialized AI hardware)
Time (weeks or months of training)
Energy (a huge electricity footprint)
Expertise (data scientists, machine learning engineers)

Because of this, most developers, startups, and even many large companies do not train AI models themselves. Instead, they use APIs provided by platforms like Groq Cloud to access these powerful models on-demand.

🔎 Other Examples of AI APIs and Platforms

Grok by Salesforce — Offers AI models integrated with business applications and workflows.
OpenAI API — Provides access to GPT models like ChatGPT for conversational AI and content generation.
Cohere — Focuses on natural language processing APIs for tasks like classification and search.

These platforms allow users to integrate AI capabilities without needing to manage or train models themselves.

🧑‍🏫 What About Training Your Own AI Model?

Some organizations and researchers do train and create their own AI models, especially when they need custom functionality or have unique data requirements. However, training large models requires:

Massive computational resources (clusters of GPUs or specialized hardware)
Access to huge datasets
Considerable time and expertise
Ongoing maintenance and fine-tuning

For many use cases, it’s far more practical and cost-effective to use APIs to access pre-trained models hosted on the cloud, rather than building and maintaining models from scratch.

🔧 How Groq Cloud Works

Sign Up for Groq Cloud: Create an account on Groq Cloud.
Get API Access: Obtain an API key to authenticate your requests.
Choose a Model: Select from models like LLaMA 3, Mixtral, or Gemma.
Send a Prompt: Make an HTTP request to the Groq API with your prompt.
Receive a Response: Instantly get back a generated response from the selected model.

🧪 Example Use Case

Summarize an article using LLaMA 3:

curl -X POST https://api.groq.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3-70b",
    "messages": [
      {
        "role": "user",
        "content": "Summarize the key points of the article on climate change."
      }
    ]
  }'

🤔 Why Do We Use the Groq SDK/Library?

When you want to use Groq’s AI models, you can either:

Call their API (like sending a request on the internet) yourself, or
Use the Groq SDK — a special tool (library) made to help you talk to Groq more easily.

Why use the Groq SDK?

Makes your work easier: You don’t have to write all the complicated code to connect and talk to Groq.
Automatically handles login: The SDK takes care of sending your secret API key safely.
Helps with errors: If something goes wrong, it gives you clear messages.
Simple commands: You can ask the AI in just a few lines of code.
Keeps updated: The SDK is updated when Groq changes their API, so you don’t have to worry.

🚀 How Does the Groq SDK Work?

When you tell the SDK to “create a chat completion,” it sends your request behind the scenes to Groq’s servers.
It sends your prompt (the message you give the AI) and the model name.
It waits for Groq’s AI to answer.
Then it gives you the AI’s response in a way that’s easy to use in your code.

You don’t have to do all the hard work of sending and receiving data — the SDK does it for you.

📜 Simple Example Using Groq SDK (JavaScript)

import Groq from "groq-sdk";
import dotenv from "dotenv";

dotenv.config();

const groq = new Groq({ apiKey: process.env.GROQ_API_KEY });

async function main() {
  const response = await groq.chat.completions.create({
    model: "llama3-70b-8192",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "Explain Generative AI simply." },
    ],
  });

  console.log("AI says:", response.choices[0]?.message?.content);
}

main();

Building a Simple Web Search Tool with Groq AI

Imagine you want to find the current weather in Ranchi using AI. You write this code to ask Groq’s AI:


import Groq from "groq-sdk";
import dotenv from "dotenv";
dotenv.config();

const groq = new Groq({ apiKey: process.env.GROQ_API_KEY });

async function main() {
  const completion = await groq.chat.completions.create({
    model: "llama3-70b-8192",
    messages: [
      {
        role: "system",
        content: "You are Rohit, a smart AI assistant. Give answers in paragraphs, easy to understand.",
      },
      {
        role: "user",
        content: "What is the current weather in Ranchi?",
      },
    ],
  });

  console.log(completion.choices[0]?.message?.content || "");
}

main();

You send a prompt to the AI asking about the weather in Ranchi.
The AI responds based on its training data (which only includes information up to a certain date).
So, it gives you general weather info but cannot provide live, current weather data.

Example AI answer:

“I don’t have real-time access to the internet, but here’s what I know about Ranchi’s climate... For real-time data, check websites like AccuWeather.”

❓ Why Does This Happen?

AI models like this are trained on past data, so they don’t know about things happening right now.
They cannot browse the internet or fetch live info by default.

🧰 What Are “Tools” in AI?

In the world of AI, Tools are special functions that an AI can use when it needs extra capabilities — like searching the internet, doing calculations, reading uploaded files, generating images, or running code.

Think of Tools as the AI's superpowers. On their own, AI models like ChatGPT are trained on a lot of information, but they don’t always know real-time data, can’t access private files, or perform live web lookups unless you give them the ability to use tools.

🧠 Simple Analogy

Imagine the AI is like a really smart assistant. It knows a lot but sometimes needs to grab a calculator, browse the web, or look at a document.

You ask:

❓ “What’s the current gold price in India?”

The AI responds:

🤔 “Hmm, I don’t know the latest price. Let me use my ‘web search’ tool.”

✅ It uses the tool → 🔍 gets live data → 🧾 gives you the correct answer.

🛠️ Common Types of Tools

Tool Name	What It Does
🌐 `Web Search`	Looks up real-time information from the internet
🧮 `Python`	Runs code, does math, makes charts, or analyzes data
📂 `File Reader`	Reads and processes uploaded files (PDFs, CSVs, etc.)
🖼️ `Image Generator`	Creates images from text prompts
📊 `Data Analysis`	Summarizes or visualizes structured data

🚀 Why Are Tools Useful?

🕒 Get Real-Time Answers (e.g., weather, stock prices, news)
📊 Analyze Files or Data (e.g., spreadsheets, reports)
✨ Generate Visuals (e.g., mockups, charts, illustrations)
🔄 Automate Tasks (e.g., code execution, format conversion)

✅ What We're Building

We are building an AI Assistant that can:

🧑 Talk with the user in simple language

🧠 Think: “Do I already know the answer?”

❌ If not, use a tool (like search)

🔄 Use the result of the tool to answer fully

🔁 How It Works — Real Life Example

👤 User: “What is the latest news about Chandrayaan?”

🤖 AI: “Hmm… I don’t have up-to-date info. Let me use my webSearch tool.”

🌐 AI uses Tavily to search the internet

✅ AI gets live info and replies:

“The latest update is...”

🧱 How We Build This — Step-by-Step Breakdown

Let’s understand the code, tools, and flow in very clear steps.

🔹 1. Install Libraries (Tools We Use)

Open your terminal and type:


npm init -y
npm install groq-sdk @tavily/core dotenv readline

🔧 What Are These?

Library	What It Does
`groq-sdk`	Connects to Groq AI (like ChatGPT)
`@tavily/core`	Searches the internet (real-time)
`dotenv`	Stores API keys safely
`readline`	Lets you type in the terminal

🔹 2. Add API Keys in .env

Create a .env file and add your keys:


GROQ_API_KEY=your_groq_api_key
TAVILY_API_KEY=your_tavily_api_key

This keeps your keys safe and private.

🔹 3. The Core Concept: Tools and Tool Use

A tool is just a function that the AI can call.

You define it like this:

{
  type: "function",
  function: {
    name: "webSearch",
    description: "Search real-time information from the internet.",
    parameters: {
      type: "object",
      properties: {
        query: {
          type: "string",
          description: "The search query to use on the internet.",
        },
      },
      required: ["query"],
    },
  },
}

Then you tell Groq:

"AI, here's a tool called webSearch. Use it when you need live info."

🔹 4. Full Working Code (Explained Line-by-Line)

import Groq from "groq-sdk";
import { tavily } from "@tavily/core";
import dotenv from "dotenv";
import readline from "readline";
import { readableStreamAsyncIterable } from "groq-sdk/lib/streaming.mjs";
dotenv.config();
const groq = new Groq({ apiKey: process.env.GROK_API_KEY });
const tvly = tavily({ apiKey: process.env.TAVILY_API_KEY });

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const systemPrompt = {
  role: "system",
  content: `You are Rohit, a Smart AI assistant. Make sure to give answers in paragraphs and in a way that is easy to understand for the user. Provide answer in the Indian Context.



You have Access to the Following Tools:
1. webSearch({ query: string }): This tool allows you to search the latest and real-time data from the Internet. You can use this tool to find information that is not available in your training data or to get the latest updates on a topic.
`,
};



async function main() {
  let messages = [systemPrompt];

  while (true) {
    const userInput = await ask("Ask your question :- ");

    messages.push({ role: "user", content: userInput });

    while(true) {
      const completion = await groq.chat.completions.create({
        messages: messages,
        tools: [
          {
            type: "function",
            function: {
              name: "webSearch",
              description: "Search the latest and Real time data from the Internet.",
              parameters: {
                type: "object",
                properties: {
                  query: {
                    type: "string",
                    description: "The search query to Perform Search from the Internet.",
                  },
                },
                required: ["query"],
              },
            },
          },
        ],
        tool_choice: "auto",
        model: "llama3-70b-8192",
      });

      messages.push(completion.choices[0].message);

      if(completion.choices[0].message.content) {
        console.log("\nAI Answer:\n" + completion.choices[0].message.content);
      } else {
        console.log("/n Tool calls made by the AI.");
      }

      const toolCalls = completion.choices[0].message.tool_calls;

      if (!toolCalls || toolCalls.length === 0) {
        console.log("No tool calls made by the AI.");
        break;
      }

      for (const call of toolCalls) {
        if (call.function.name === "webSearch") {
          const args = JSON.parse(call.function.arguments);
          const toolResult = await webSearch({ query: args.query });
          messages.push({
            role: "tool",
            tool_call_id: call.id, // REQUIRED for Groq API!
            name: "webSearch",
            content: toolResult,
          });

          // Call Groq again with updated messages to get the final answer
          const followUp = await groq.chat.completions.create({
            messages: messages,
            model: "llama3-70b-8192",
          });

          // Log the final AI answer after tool result
          const finalContent = followUp.choices[0].message.content;
          if (finalContent) {
            console.log("\nAI Final Answer:\n" + finalContent);
          } else {
            console.log("\nAI Final Answer: (No content returned)");
          }
        } else {
          console.log("Unknown tool call:", call);
        }
      }
    }

  }
}

function ask(query) {
  return new Promise((resolve) => rl.question(query, resolve));
}

async function webSearch({ query }) {
  const response = await tvly.search(query, {
    searchDepth: "advanced", // Must be a string 
    searchType: "web",
    searchLanguage: "en",
  });

  // Join all results into a single string
  const finalResult = response.results
    .map((result) => Array.isArray(result.content) ? result.content.join("\n") : result.content)
    .join("\n\n");

  return finalResult;
}



main();

✅ 1. Importing Required Packages

You start by importing libraries:

groq-sdk: This lets your app talk to Groq’s AI models (like LLaMA 3).
@tavily/core: This connects your app to Tavily, which does real-time web searches.
dotenv: Allows your app to load secret keys (like API keys) from a .env file.
readline: Lets your app take input from the user in the terminal (like a chatbot).
readableStreamAsyncIterable: Used for reading streamed responses, though it’s not actively used here.

🔐 2. Load Environment Variables

You load the .env file using dotenv. This gives access to your API keys securely instead of writing them in the code.

🔌 3. Create Groq and Tavily Clients

You create instances of both Groq and Tavily using the API keys. These instances allow you to send requests to both platforms.

🧾 4. Set Up Terminal Interface

You use readline to make a simple question-and-answer interface in the terminal where the user can type their questions.

🤖 5. Define System Prompt for the AI

You tell the AI how to behave:

Act like "Rohit", a smart assistant.
Answer in paragraph form, in simple, Indian-friendly language.
Also mention the tools available to it — in this case, the webSearch tool.

🔁 6. Start the Chat Loop

You start an infinite loop where the user can keep asking questions.

First, the user types a question.
The question gets added to the message history (a chat-like structure).

💬 7. AI Processes the Question

Now the Groq AI is called with the current message history and given access to tools (like webSearch).

You set tool_choice: "auto", which means:

The AI can decide for itself whether to answer directly or use a tool if it doesn’t know something.

🧠 8. Store AI’s First Response

You take the AI’s response and add it to the message history.

If the AI replied with text, you print it to the user.
If the AI didn't give an answer, it probably tried to call a tool instead.

🧰 9. Check If Any Tool Was Used

You check whether the AI tried to call any tools (like webSearch).

If no tool was called, the loop ends and waits for the next user question.
If a tool was called, you move to the next step.

🌐 10. Handle the webSearch Tool Call

For each tool call:

You check if the AI called the webSearch tool.
If yes, you take the search query given by the AI.
You use Tavily to run the search using that query.
Tavily sends back search results from the real internet.

🔄 11. Send Tool Result Back to AI

Once the tool returns the data:

You send the search result back to Groq as a tool response.
Then, you ask Groq again to complete the answer — now that it has access to real-time data.
Groq replies with the final, full answer using both its training and the tool's result.
You print this final answer to the user.

📦 12. If Tool Call is Unknown (Fallback)

You also have a safety check in case the AI tries to call a tool that your app doesn’t support. In this case, it logs that an unknown tool was requested.