10x Faster LLM Inference?

Groq API Python Tutorial: Free LLM Inference 10x Faster Than ChatGPT

Groq offers free LLM inference that's 10x faster than OpenAI. Here's how to use it.

Why Groq?

• ⚡ 500+ tokens/second (vs ~50 for GPT-4)
• 🆓 Free tier: 14,400 requests/day
• 🔑 Get key: console.groq.com

Install

pip install httpx

Basic Chat

import httpx
import os

GROQ_KEY = os.getenv("GROQ_API_KEY")  # Get free at console.groq.com

def chat(prompt: str, model: str = "llama-3.3-70b-versatile") -> str:
    with httpx.Client() as client:
        r = client.post(
            "https://api.groq.com/openai/v1/chat/completions",
            headers={"Authorization": f"Bearer {GROQ_KEY}"},
            json={
                "model": model,
                "messages": [{"role": "user", "content": prompt}],
                "max_tokens": 1000,
                "temperature": 0.7,
            },
            timeout=30,
        )
        return r.json()["choices"][0]["message"]["content"]

# Test it
response = chat("Explain Python decorators in 3 sentences")
print(response)

Async Version (For APIs)

import httpx
import asyncio

async def async_chat(prompt: str) -> str:
    async with httpx.AsyncClient() as client:
        r = await client.post(
            "https://api.groq.com/openai/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.getenv('GROQ_API_KEY')}"},
            json={"model": "llama-3.1-8b-instant", "messages": [{"role": "user", "content": prompt}]},
            timeout=30,
        )
        return r.json()["choices"][0]["message"]["content"]

# Run multiple prompts concurrently
async def main():
    prompts = ["What is Python?", "Explain async/await", "Best Python libraries 2024"]
    results = await asyncio.gather(*[async_chat(p) for p in prompts])
    for prompt, result in zip(prompts, results):
        print(f"Q: {prompt}\nA: {result[:100]}...\n")

asyncio.run(main())

Available Models

Model	Speed	Context	Best For
llama-3.3-70b-versatile	Fast	128K	General tasks
llama-3.1-8b-instant	Fastest	128K	Simple tasks
gemma2-9b-it	Fast	8K	Code generation
mixtral-8x7b-32768	Medium	32K	Complex reasoning

Rate Limits (Free Tier)

• 14,400 requests/day
• 1,000 requests/minute
• 6,000 tokens/minute

That's essentially unlimited for most projects!

---

Follow me for more AI Python tutorials! 🐍

Follow for more Python content!

---

💡 Related: **Content Creator Ultimate Bundle (Save 33%)* — $30*

---

If you found this useful, you might like Python Interview Prep Guide — a practical resource that takes things a step further. At $24.99 it's a solid investment for your toolkit.

---

喜欢这篇文章？关注获取更多Python自动化内容！