DEV Community

Jotham Ntekim
Jotham Ntekim

Posted on • Edited on

CineAgent: Bridging the Web and Conversational AI with a Concurrent Go ETL

Search engines are only as good as the data you feed them. For the Algolia Agent Studio Challenge, I wanted to build more than just a chatbot; I wanted to build a living pipeline.

Introducing CineAgent, a voice-powered cinema concierge grounded by Trawl—a custom-built, high-concurrency ETL engine written in Go.


🚀 The Vision: Beyond Static Data

Most AI agents are limited by their training data or static JSON uploads. CineAgent is different. It uses a "Trawler" approach to bridge the gap between unstructured web content and conversational discovery.

The project solves three specific problems:

  1. Data Freshness: Trawl (the backend) can scrape and index thousands of records concurrently.
  2. Grounding: The AI Agent is strictly grounded in a curated index of 250+ cinematic masterpieces.
  3. The "Empty State" Problem: In standard search, "No Results" is a dead end. In CineAgent, "No Results" is the beginning of a conversation.

🛠 The Tech Stack

  • The Engine (Trawl): Built with Go, utilizing Goroutines and Channels for high-concurrency web scraping and batch indexing via the Algolia Go API Client v5.
  • The Brain: Algolia Agent Studio, configured as a "Film Discovery Expert" to handle complex, natural language queries.
  • The Face: A React frontend styled with Chakra UI v3 for a premium, immersive dark-mode experience.
  • The Interaction: Web Speech API for hands-free discovery and the Algolia InstantSearch Chat Widget (Beta).

💡 Key Features & Originality

1. The Trawl ETL Pipeline (Go)

While most participants might upload a static file, I built an ETL (Extract, Transform, Load) engine. Trawl follows a concurrent worker-pool pattern. It scrapes metadata (Title, Year, Rating, High-res posters), normalizes it into Go structs, and pipes it into Algolia in optimized batches.

2. The Hybrid "Voice-to-Agent" UI

I implemented a "NoResultsBoundary" logic. If the local search grid can't find a specific niche movie, the UI doesn't just show an error. It proactively invites the user to "Ask CineAgent AI," handing off the search query to the conversational agent.

3. Voice-First Discovery

Users can click the microphone icon to search. If the voice input is a short keyword (e.g., "Inception"), the movie grid updates. If the input is a complex question (e.g., "What is the best sci-fi movie from the 90s?"), CineAgent opens the conversational chat automatically.


📺 Demo & Visuals

Live URL: trawl-ui.vercel.app


![Image Description: A sleek dark-mode movie library with a floating AI Assistant bubble.]


🧠 Lessons Learned

Integrating a Beta widget like the Algolia Chat component into a modern React 18 / Chakra UI v3 environment was a challenge in state management. I learned that isolating the AI handshake from the main search lifecycle is key to maintaining a responsive, "hang-free" user experience.


🔗 Links

  • GitHub Repository:

Frontend (UI)
github.com/ntekim/trawl-ui
Backend
github.com/ntekim/trawl

  • Algolia Agent ID: 4ddf0636-38b2-4578-a98f-8715efde81ca

algoliachallenge #devchallenge #agents #webdev #golang #react

Top comments (0)