DEV Community

AI and QA Leaders profile image Let's Automate 🛡️
Let's Automate 🛡️ for AI and QA Leaders

Posted on • Originally published at Medium

Build Your Own AI QA Assistant — Automate Test Understanding with LangChain & OpenAI

#ai #testing #aqe #python

Stop digging through test cases, requirement docs, and logs manually. Build your own QA Assistant that can answer your questions using natural language — powered by OpenAI and LangChain.

👋 Why This?

As QA engineers, we often deal with:

  • Test cases in JSON or Jira/Xray
  • BDD specs in .feature files
  • Markdown requirements
  • Logs in .txt and .html
  • Allure reports and PDFs

...scattered across tools and folders.

The problem? Searching across all these files is slow, repetitive, and error-prone.

The solution? An AI-powered bot that understands all your test artifacts and answers your questions — like a real assistant.

🧠 What This Bot Does

It allows you to:

✅ Ask things like:

  • "What are the test steps in TC-002?"
  • "Summarize login test cases."
  • "What does the PDF say about edge cases?"

📁 It reads from:

  • .json → structured test cases
  • .feature → BDD scenarios
  • .md, .txt, .html → requirements, logs, reports
  • .pdf, .docx → spec files

🔎 Then it:

  • Breaks the content into searchable chunks
  • Creates semantic embeddings
  • Stores it in a FAISS vector DB
  • Uses OpenAI to generate responses in natural language

🛠️ Tech Stack

  • LangChain — to connect everything
  • OpenAI — to generate QA-friendly responses
  • FAISS — to index content for fast semantic search
  • Python — because we like speed and simplicity

🚀 How to Use It

1. 🔄 Clone the Repo 

git clone https://github.com/aiqualitylab/AI-QA-Assistant-Bot.git
cd AI-QA-Assistant-Bot
Enter fullscreen mode Exit fullscreen mode

2. 📦 Install Dependencies 

pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

3. 🔐 Set Your OpenAI Key

Create a .env file:

OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Enter fullscreen mode Exit fullscreen mode

🔐 You can get your API key from: https://platform.openai.com/account/api-keys

4. 🚀 Run the Assistant 

python bot.py
Enter fullscreen mode Exit fullscreen mode

If it's your first run, it will index the documents. After that, it reuses the saved index.

You'll see:

🤖 QA Assistant Ready! Ask anything (type 'exit' to quit')
Enter fullscreen mode Exit fullscreen mode

Start asking questions — and get instant, AI-powered answers.

💬 Example Use Cases

  • 🔍 "What's in TC-003?"
  • 📝 "Summarize all test cases about checkout"
  • 🐞 "What do the logs say about payment errors?"
  • 📋 "Give me test steps for login with invalid credentials"

🛠️ Behind the Scenes

Under the hood:

  • Document loaders handle multiple formats using langchain_community.loaders
  • Recursive chunking splits large text into overlapping pieces
  • OpenAIEmbeddings converts chunks into vectors
  • FAISS indexes and searches them semantically
  • RetrievalQA runs your question against the best-matching documents

💡 Why It Matters

QA engineers shouldn't have to:

  • Ctrl+F through hundreds of lines
  • Switch between Jira, Confluence, Allure, Git
  • Manually correlate logs and test cases

This assistant lets you work smarter, not harder.

🧪 Demo Available

Try it out locally on your own test artifacts. Clone the repo 👉 https://github.com/aiqualitylab/AI-QA-Assistant-Bot

Have questions or suggestions? Drop a comment below or open an issue on GitHub!

GitHub logo aiqualitylab / AI-QA-Assistant-Bot

QA Assistant that can answer your questions using natural language — powered by OpenAI and LangChain.

🤖 QA Assistant Bot

AI-powered assistant for QA engineers
Uses OpenAI + LangChain to answer questions from test cases, feature files, logs, specs, and more — all from natural language.

🧠 Purpose

This bot helps QA engineers:

  • ✅ Ask natural questions like:
    "What are the steps in TC-001?"
    "List test cases for login."

  • 📄 Understand test specs and logs quickly

  • 🔍 Search across multiple formats (JSON, Markdown, PDF, etc.)

  • 🧪 Automate documentation understanding and reduce manual effort

📁 Supported File Types

It can read and index:

  • .json — Test cases
  • .feature — BDD specs
  • .md — Requirements
  • .txt — Logs
  • .html — Reports
  • .pdf — Test documents
  • .docx — Word specs

Place files inside the data/ folder.

⚙️ Installation

✅ 1. Clone the repo

git clone https://github.com/aiqualitylab/AI-QA-Assistant-Bot
cd qa-bot
Enter fullscreen mode Exit fullscreen mode

✅ 2. Set up Python environment

Make sure Python 3.10+ is installed. Then:

pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

✅ 3. Create .env

…
View on GitHub

Top comments (0)