QA Bug Triage Pipeline: From App Reviews to Searchable Bug Reports

#testautomation #llm #qaautomation #artificialintelligen

A simple Python project that turns messy user reviews into structured QA bug reports using an LLM and RAG.

📖 Full guide: blog.aiqualitylab.org

Why this project

Product teams get lots of feedback, but most of it is noisy and unstructured. This project helps QA teams convert that feedback into consistent bug records that are easy to search and summarize.

Photo by Guille B on Unsplash

What it does

Collects reviews from Google Play

Routes review text (bug report vs non-bug)

Generates structured JSON bug reports with an LLM

Stores bugs in ChromaDB for semantic retrieval

Adds BM25 keyword matching for hybrid search

Produces short AI summaries for triage

Lets you clear the stored bugs from the UI

Quick start

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
python app.py

Then open the local Gradio URL.

API key

This app uses BYOK (Bring Your Own Key):

Paste your OpenAI API key in the UI

The key is masked

Do not commit keys to source control

Main files

app.py: Gradio app flows

collect.py: review collection

triage.py: routing and structured triage logic

rag.py: storage and hybrid retrieval

eval/eval.py: evaluation script