📺 Find the Anime: A Semantic AI Anime Search Tool

Yuki Lun — Sun, 20 Apr 2025 12:24:09 +0000

(Note: This project was created as part of the Gen AI Intensive Course Capstone 2025Q1.)

🔎 Use Case: Smarter Anime Discovery with Gen AI

Anime lovers know the struggle: sometimes you're dying to rewatch a show, but all you remember is that it had a shy girl, a magical sword, and maybe a floating cat. Or maybe you’ve just finished an amazing series and want something just like it — another story with the same themes, genre, or a specific kind of character.

Unfortunately, traditional search tools like Google or anime databases often fall short when the query is fuzzy, emotional, or based on vibes. They rely on exact titles, tags, or genre filters — great if you know what you're looking for, but not so great when you're working from memory or mood.

That’s where "Find the Anime" comes in.

What it solves:

Memory-based discovery: For users who remember only fragments of the story, tone, or visual elements (e.g. “a robot cat from the future”).
Interest-based discovery: For users who want to explore new anime with similar themes, genres, or character archetypes (e.g. “dark psychological thriller with a lone genius protagonist”).

How Gen AI helps:
Instead of relying on rigid keyword matching, this tool uses semantic search — a technique that goes beyond literal keyword matching and instead focuses on understanding the meaning and context of a query. This approach, powered by large language models (LLMs) (AI models trained on huge amounts of text that understand and generate human language), along with image understanding (the AI's ability to "see" and interpret what's in an image), vector embeddings (where words or images are converted into numerical representations that capture their meaning, allowing computers to compare them), and grounded explanations (which uses external sources like Google Search to make the AI's answers more accurate and based on real-world information), can interpret vague or descriptive queries and return contextually relevant results — along with the reasoning behind them.

🛠️ How Gen AI Solves the Problem: Implementation Highlights

My solution uses Google’s Gemini to connect what the user is looking for with the right anime. Here’s a quick look at the main parts and techniques I used:

Data Acquisition

I gathered anime data using the Jikan API, which provides metadata like titles, genres, and synopses.

Data Structuring with Gemini

The raw data from the API was often unstructured and difficult to work with. To structure this data, I used Gemini's JSON mode to convert it into a clean, strongly typed format.

class Anime(typing.TypedDict):
    mal_id: int
    title: str
    genres: list[str]

response = client.models.generate_content(
    model='gemini-2.0-flash',
    contents=prompt,
    config=types.GenerateContentConfig(
        temperature=0.1,
        response_mime_type="application/json",
        response_schema=Anime,
    ),
)

Character Image Analysis with Gemini Vision

To capture character appearances, I used Gemini Vision to analyze the images and extract relevant features:

image = requests.get(image_url)

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=[prompt, types.Part.from_bytes(data=image.content, mime_type="image/jpeg")]
)

Embedding Generation

The core of semantic search is converting text (anime descriptions and character appearances) into meaningful numerical representations. Here's the code for my custom embedding function:

 class GeminiEmbeddingFunction(EmbeddingFunction):
    document_mode = True

    def __call__(self, input: Documents) -> Embeddings:
        embedding_task = "retrieval_document" if self.document_mode else "retrieval_query"
        response = client.models.embed_content(
            model="models/text-embedding-004",
            contents=input,
            config=types.EmbedContentConfig(task_type=embedding_task),
        )
        return [e.values for e in response.embeddings]

Semantic Search with Chroma

I used Chroma, a vector database, to store the anime embeddings. Chroma allows for efficient similarity search, enabling the system to quickly retrieve the anime that are most relevant to a user's query.

```python
results = anime_db.query(query_texts=[query], n_results=3)
```

Relevance Evaluation with Gemini and Google Grounding:

After retrieving potential matches, I used Gemini to evaluate how relevant each anime was to the user's query. I also enabled Google Grounding for more accurate, real-world-aware reasoning.

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())],
    ),
)

User Interface with Gradio

I built a simple yet effective front end using Gradio, allowing users to input queries and view the results with styled anime cards.

💭 Limitations and Future Improvements

While this project demonstrates the power of generative AI for anime search, it's important to acknowledge its limitations:

Data Dependence: The accuracy of the search relies heavily on the quality and completeness of the anime data.
Image Variability: Character appearance matching can be affected by variations in image quality and style.
Model Subjectivity: The relevance evaluation process introduces some degree of subjectivity, as it depends on Gemini's reasoning.
Limited Data Scope: The current system was trained and evaluated on a relatively small dataset, which may limit its ability to generalize to the entire anime catalog.

To further enhance this tool, future work could explore these directions:

Enhanced Data: Incorporating a larger and more detailed dataset of anime information, including more granular details about plot, characters, and themes.
Image Embeddings: Using image embeddings and allowing users to upload images for character-based search, leading to more precise and intuitive search results.
Voice Input: Adding voice-based query input to allow users to search using spoken words, making the tool more accessible and convenient.
Chatbot Interface: Transforming the UI into a conversational chatbot to provide a more natural and interactive search experience, with guided recommendations.
Multi-Model Evaluation: Employing multiple Gemini models to evaluate the relevance of search results, combining their outputs to reduce bias and subjectivity.

🎉 Conclusion

This project demonstrates the transformative potential of generative AI for anime search and discovery. By using Gemini’s ability to understand language, analyze images, and give grounded explanations, we can create a more intuitive, efficient, and personalized way to find the perfect anime.

🚀 Whether you're an anime fan or an AI enthusiast, try the tool out and let me know what anime it recommends for you!

👉 Check out the full code and data in my Kaggle notebook: Find the Anime: A Semantic AI Anime Search Tool

DEV Community: Yuki Lun