DEV Community

AIGB Team
AIGB Team

Posted on

Running a Global Telegram Community with Free AI Tools

#telegram #ai #community

Telegram is a favorite playground for developers, DevOps engineers, and crypto‑enthusiasts because it offers fast, low‑latency messaging and a powerful bot API. The challenge isn’t getting people into a group; it’s keeping the conversation healthy, inclusive, and productive as the community scales across time zones and languages.

In this article I’ll walk through a complete, low‑cost workflow that turns a plain Telegram group into a self‑moderating, multilingual hub. All the tooling I use is free (or has a generous free tier), and the only paid component is the optional upgrade to a paid LLM provider if you exceed the free quota.

TL;DR – Use an open‑source LLM (e.g., Llama‑3‑8B), host it on a free tier of a cloud provider, connect it to a Telegram bot via the Bot API, and add a few smart middleware functions (semantic moderation, language routing, “WeChat‑style” group buying). The result is a community that feels alive, not just a spam‑filter.

1. Why AI‑Driven Moderation Beats Keyword Filters

Traditional Telegram bots rely on regular expressions or keyword blacklists. They work for obvious spam (e.g., repeated “FREE NFT” links) but stumble when users:

  • Ask nuanced technical questions in code blocks
  • Slip jokes or idioms in languages other than English
  • Post legitimate advertisements that violate a simple keyword rule

A small language model can read the surrounding context and decide whether a message is actually harmful. The benefits are immediate:

Situation Keyword Filter LLM‑Based Filter
“I’m new, can anyone help me set up a Docker compose?” ✅ (passes) ✅ (passes)
“Buy cheap VPN at http://spam.site” (Russian) ❌ (misses Russian) ✅ (detects spam)
“😂😂😂” (spammy emoji flood) ❓ (hard to pattern) ✅ (recognizes flood)
“Let’s schedule a dev‑talk on Saturday” ✅ (passes) ✅ (passes)

The model acts like a junior moderator: it flags ambiguous messages for a human, auto‑deletes the obvious spam, and can even respond with helpful resources when it understands the intent.

2. Choosing a Free LLM

If you’re comfortable with a little setup, the best free option is a quantized open‑source model hosted on a cloud service that offers a free tier (e.g., Oracle Cloud Free Tier, Google Cloud’s “Always Free” VM, or Fly.io). Here’s a quick checklist:

  1. Model size – 7‑8 B parameters fit comfortably in a 16 GB VM. Llama‑3‑8B‑Chat or Mistral‑7B‑Instruct are good choices.
  2. Quantization – Use ggml 4‑bit quantization to drop RAM usage to ~4 GB.
  3. Inference framework – llama.cpp provides a single‑binary that runs on CPU only, no GPU required.

All of these pieces are open source and can be launched with a single docker run command, which we’ll see next.

3. Deploying the Model for Free

Below is a minimal Docker command that pulls a pre‑quantized Llama‑3‑8B image from Docker Hub and exposes it on port 8080. The container will automatically download the model files on first start, so make sure the VM has at least 10 GB of outbound storage.

docker run -d \
  --name llm \
  -p 8080:80 \
  -v /home/ubuntu/llama_models:/models \
  ghcr.io/ggerganov/llama.cpp:latest \
  -m /models/llama3-8b-chat.ggmlv3.q4_0.bin \
  -c 2048 -ngl 0 -t 4 -b 8 -s 42 \
  --host 0.0.0.0 --port 80
Enter fullscreen mode Exit fullscreen mode

Explanation

  • -c 2048 – context length; enough for most moderation prompts.
  • -t 4 – use four CPU threads (adjust to your VM’s vCPU count).
  • -b 8 – batch size, keeps latency low for short messages.

You can test the endpoint with curl:

curl -X POST http://localhost:8080/completions \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Is the following message spam?\\nUser: \"Check out my cheap VPN at http://spam.site\"\\nAnswer:","max_tokens":10}'
Enter fullscreen mode Exit fullscreen mode

If the response is "Yes" you’ve got a working inference server.

4. Connecting the LLM to Telegram

Telegram bots are just HTTP webhooks. The flow looks like this:

  1. A user sends a message → Telegram forwards it to your webhook URL.
  2. Your webhook script extracts the text, calls the LLM endpoint, parses the answer.
  3. Based on the answer, the script either deletes the message, replies with a hint, or forwards it to a human moderator queue.

Below is a compact Python snippet (uses python-telegram-bot v20) that implements this loop. Save it as bot.py and run it on the same VM as the LLM container.

import os
import json
import aiohttp
from telegram import Update, ChatAction
from telegram.ext import ApplicationBuilder, ContextTypes, MessageHandler, filters

TOKEN = os.getenv("TELEGRAM_BOT_TOKEN")
LLM_URL = "http://localhost:8080/completions"

async def moderate(update: Update, ctx: ContextTypes.DEFAULT_TYPE):
    text = update.message.text
    # Build LLM prompt
    prompt = f"Is the following Telegram message spam or toxic? Answer with Yes or No.\nMessage: \"{text}\""
    async with aiohttp.ClientSession() as session:
        async with session.post(
            LLM_URL,
            json={"prompt": prompt, "max_tokens": 5},
            timeout=5,
        ) as resp:
            data = await resp.json()
    verdict = data["choices"][0]["text"].strip().lower()
    if verdict == "yes":
        await ctx.bot.delete_message(chat_id=update.effective_chat.id,
                                     message_id=update.message.message_id)
        await ctx.bot.send_message(chat_id=update.effective_chat.id,
                                   text="🚫 Message removed – looks like spam.")
    else:
        # Optional: add a friendly reaction
        await ctx.bot.send_chat_action(chat_id=update.effective_chat.id,
                                       action=ChatAction.TYPING)

app = ApplicationBuilder().token(TOKEN).build()
app.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, moderate))

if __name__ == "__main__":
    app.run_polling()
Enter fullscreen mode Exit fullscreen mode

Key points

  • The bot runs in polling mode for simplicity; switch to a webhook if you have a public HTTPS endpoint.
  • The LLM prompt is deliberately short – keep latency under 500 ms for a smooth user experience.
  • You can extend the if verdict == "yes" branch to route messages to a private moderator group instead of deleting them outright.

Deploy the script with a systemd service or pm2 so it survives reboots.

5. Going Multilingual – The 14‑Language Trick

Many global Telegram groups have members speaking English, Spanish, Russian, Mandarin, Arabic, and more. Instead of training separate models, you can keep a language‑routing map that selects a prompt template based on detected language.

A free language detection library such as fasttext (via fasttext-langdetect) works offline:

from fasttext import load_model
lang_model = load_model("lid.176.ftz")

def detect_lang(text: str) -> str:
    return lang_model.predict(text)[0][0].replace("__label__", "")
Enter fullscreen mode Exit fullscreen mode

Add a small dictionary that maps language codes to LLM prompts. For example:

PROMPTS = {
    "en": "Is this message spam? Answer Yes or No.\nMessage: \"{msg}\"",
    "es": "¿Este mensaje es spam? Responde Sí o No.\nMensaje: \"{msg}\"",
    "ru": "Это спам? Ответьте Да или Нет.\nСообщение: \"{msg}\"",
    # add the rest up to 14 languages...
}
Enter fullscreen mode Exit fullscreen mode

When a message arrives, run detect_lang, pick the right template, and feed it to the model. The LLM will stay in the same language for its answer, which you can then translate back to English for logging if needed.

6. Adding “WeChat‑Style” Group Buying (Jiele)

One of the most engaging community rituals is a collective “group buy” or event RSVP where members vote with a simple emoji reaction. You can emulate WeChat’s “jiele” (接龙) pattern using inline keyboards:

from telegram import InlineKeyboardButton, InlineKeyboardMarkup

async def start_jiele(update: Update, ctx: ContextTypes.DEFAULT_TYPE):
    keyboard = [
        [InlineKeyboardButton("🟢 Yes, I’m in", callback_data="jiele_yes"),
         InlineKeyboardButton("⚪️ No, thanks", callback_data="jiele_no")]
    ]
    await ctx.bot.send_message(
        chat_id=update.effective_chat.id,
        text="🚀 Community Hackathon – Who wants to join?\nReact below:",
        reply_markup=InlineKeyboardMarkup(keyboard)
    )
Enter fullscreen mode Exit fullscreen mode

When users click a button, store their user ID in a tiny SQLite DB. Periodically (e.g., every hour) the bot can post a summary:

🟢 23 members have joined
⚪️ 12 members declined
Enter fullscreen mode Exit fullscreen mode

Because the bot is already connected to the LLM, you can even let it answer follow‑up questions like “What’s the agenda?” automatically.

7. Monitoring & Continuous Improvement

Running AI in a live community means you need visibility into false positives, latency spikes, and usage quotas.

  • Metrics collection – Export moderation_success, moderation_false_positive, and average_latency to Prometheus (or a simple CSV file).
  • Human‑in‑the‑loop – Create a private moderator channel where the bot posts messages it wasn’t sure about (e.g., confidence < 0.7). Moderators can reply with “allow” or “remove,” and you can feed that feedback back into a fine‑tuning dataset later.
  • Cost guardrails – Most

Top comments (0)