How I Built an Autonomous AI Tool Engine with MCP, Llama 3.3, and Automated Hugging Face Pipelines

Vishal Singh — Fri, 31 Jul 2026 14:40:33 +0000

Most AI tool directories suffer from two issues: they are statically hardcoded, and they break when you search by intent instead of exact keywords.

I built AI Tool Hunter V2 to solve this—turning a basic directory into an automated discovery engine, open-source dataset pipeline, and Model Context Protocol (MCP) server.

Here is a complete breakdown of the system architecture and the engineering trade-offs made along the way.

1. The Autonomous Ingestion Pipeline
Rather than manually curating links, a daily GitHub Actions workflow (harvest.yml) triggers an ingestion script:

Live Web Scan: Tavily API queries web sources for newly launched software.
Intent & Metadata Extraction: Raw payload is passed to Llama 3.3-70B via Groq. The model parses pricing structures, feature sets, and assigns intent-based categories.
Deduplication & Storage: Clean records are verified against existing database entries and stored in Supabase (PostgreSQL with RLS).

2. The Open-Source Data Flywheel
To share this corpus with the open-source community, the daily cron pipeline doesn't stop at database insertion.
It formats the updated database into a structured corpus and uses huggingface_hub to push a nightly commit directly to Hugging Face (AI-Tools-Corpus-2026).

3. Exposing the System via MCP (Model Context Protocol)
Modern developer tools live in the IDE. I built a dedicated MCP server hosted on Render so external LLMs (in Cursor, VS Code, or Claude Desktop) can call the search index natively.

Tech Stack

Frontend: Next.js 16, TypeScript, Tailwind CSS
Backend & Auth: Supabase (Auth + RLS), Python, Render
AI / Data: Groq (Llama 3.3-70B), Tavily API, Hugging Face Datasets
DevOps: GitHub Actions, Model Context Protocol (MCP)

Links & Open Source:

Live App: (https://ai-tool-hunter-eight.vercel.app/)
GitHub Repo: (https://github.com/VishalsWorkspace/AI-TOOL-HUNTER)
Hugging Face Corpus: (https://huggingface.co/datasets/Vishalsworkspace/AI-Tools-Corpus-2026)

DEV Community: Vishal Singh

How I Built an Autonomous AI Tool Engine with MCP, Llama 3.3, and Automated Hugging Face Pipelines