DEV Community: Abdullah Javed

Integrating YOLO Fire Detection with FastAPI and Next.js 🔥

Abdullah Javed — Tue, 27 Oct 2026 11:42:55 +0000

A practical walkthrough of setting up FlameGuard, connecting a custom YOLO model, and debugging a full-stack Computer Vision application.

Introduction

I recently explored FlameGuard, an open-source fire detection application built with YOLO, FastAPI, and Next.js.

I wanted to understand how a trained object detection model works inside a complete web application.

I used the original FlameGuard repository and adapted it to work with my own trained fire detection model.

Here's a breakdown of the implementation and what I learned.

Tech Stack

Technology	Purpose
Python	Backend development
YOLO	Fire detection
OpenCV	Image processing
FastAPI	REST API
SQLite	Detection history
Next.js	Frontend
React	User interface
Conda	Dependency management

1. Project Setup

Clone the repository:

git clone https://github.com/nohsangwoo/FlameGuard.git
cd FlameGuard

The main project structure is:

FlameGuard/
├── backend/
│   ├── app/
│   └── environment.yml
├── frontend/
└── README.md

Create the Python environment:

cd backend
conda env create -f environment.yml
conda activate flameguard

2. Integrating the YOLO Model

I already had a trained fire detection model named best.pt.

The model contains one detection class:

{0: "fire"}

I placed the model in the backend's assets directory and configured the detection endpoint to use it.

from pathlib import Path
from ultralytics import YOLO

APP_DIR = Path(__file__).resolve().parents[2]

MODEL_PATH = APP_DIR / "assets" / "best.pt"

model = YOLO(str(MODEL_PATH))

Loading the model outside the API handler avoids repeatedly initializing it for each request.

3. Running Fire Detection

The prediction process uses the YOLO model to analyze an uploaded image.

results = model.predict(
    source=str(temp_file_path),
    conf=0.25,
    imgsz=640
)

result = results[0]

We can extract detection information:

for box in result.boxes:
    class_id = int(box.cls.item())
    class_name = model.names[class_id]
    confidence = float(box.conf.item())
    bbox = box.xyxy[0].tolist()

    print(class_name, confidence, bbox)

Each detection includes a predicted class, confidence score, and bounding-box coordinates.

4. Creating Annotated Images

Ultralytics provides a convenient method for drawing predicted bounding boxes:

annotated_img = result.plot()

The annotated image can then be saved using OpenCV:

import cv2

cv2.imwrite(
    str(log_file_path),
    annotated_img
)

These images can be used by the frontend to display detection results.

5. FastAPI Integration

The backend exposes a prediction endpoint:

POST /predict_fire

It receives an uploaded image, processes it with YOLO, and returns a structured detection response.

For example, one of my tests returned:

{
  "message": "fire detected",
  "file_name": "frame.jpg",
  "detections": [
    {
      "class_name": "fire",
      "confidence": 0.419
    }
  ]
}

The API can also store detection information in SQLite for later retrieval.

6. Debugging Detection Images

One issue I encountered was that the backend returned successful predictions, but detection images were missing.

The logs showed:

POST /predict_fire 200 OK
GET /log/image.jpg 404 Not Found

I investigated the image-saving and static-file directory paths.

Using pathlib helped keep the paths consistent:

from pathlib import Path

APP_DIR = Path(__file__).resolve().parents[2]

LOG_DIR = APP_DIR / "log"
LOG_DIR.mkdir(parents=True, exist_ok=True)

This was a useful lesson in handling generated files inside FastAPI applications.

7. Running the Backend

From the backend directory:

python -m uvicorn app.main:app --reload

The backend runs at:

http://localhost:8000

API documentation:

http://localhost:8000/docs

8. Running the Next.js Frontend

Open another terminal:

cd frontend
pnpm install
pnpm run dev

The frontend runs at:

http://localhost:3000

The interface communicates with the FastAPI backend to process images and retrieve detection results.

9. Handling Camera Errors

During testing, I encountered a browser error related to:

navigator.mediaDevices.enumerateDevices()

A simple feature check helps prevent the application from crashing when the browser camera API is unavailable:

if (!navigator.mediaDevices?.enumerateDevices) {
  console.warn("Camera API unavailable");
  return;
}

const devices =
  await navigator.mediaDevices.enumerateDevices();

Browser permissions and secure-context requirements are important when working with camera-based applications.

10. Lessons Learned

This project helped me better understand:

YOLO inference and detection results.
FastAPI integration with Computer Vision models.
Image annotation and file management.
SQLite detection logging.
Next.js and backend communication.
Debugging missing images and browser camera errors.
Organizing a full-stack AI application.

Future Improvements

I plan to explore better detection accuracy, improve frontend interactions, test more challenging images, and make the application more reliable for continuous monitoring.

The current project is a learning experiment rather than a production-ready fire safety system.

Conclusion

Working on FlameGuard was a valuable opportunity to connect Computer Vision concepts with full-stack application development.

It also reminded me that a trained AI model is only one part of a complete application.

There is much more to explore, and I look forward to improving my understanding through more practical projects.

Original project: FlameGuard by nohsangwoo

GitHub: github.com/abdullahjave

Thanks for reading!

Connecting AI Agents to Scientific Tools with MCP: My OSI Hackathon 2026

Abdullah Javed — Sat, 24 Oct 2026 19:19:55 +0000

Connecting AI Agents to Scientific Tools with MCP: My OSI Hackathon 2026 Experience

Building reproducible AI-agent workflows with Python, Claude Code, GitHub Copilot CLI, and scientific MCP tools.

Introduction

What happens when we connect AI coding agents to real scientific tools?

That was one of the questions I explored while contributing to KnowledgeIsPower, an open-source project developed during the OSI Hackathon 2026 (October 21–22).

Our team, led by Amit Shenoy, worked on a problem in single-cell proteomics: estimating missing protein measurements using biological knowledge and data-driven methods.

We developed scp-graphfs-mcp, a scientific Model Context Protocol (MCP) server that enables AI agents to interact with protein-complex data, knowledge graphs, and imputation tools.

My contribution focused on S5: MCP Client Integration, Testing, and Documentation.

In this article, I'll walk through the project architecture, how I connected AI agents to the MCP server, and what I learned from running a real scientific workflow.

1. The Problem We Were Solving

Single-cell proteomics helps researchers measure proteins in individual cells.

However, mass-spectrometry experiments often produce incomplete measurements. Depending on the experiment, only around 20–60% of a cell's proteins may be quantified.

Researchers use imputation to estimate missing values.

Traditional approaches, including k-nearest neighbors (kNN), rely on patterns already present in the dataset.

Our team explored whether biological knowledge, particularly relationships between proteins, could improve these estimates.

The project ultimately compared 22 protein knowledge representations and investigated methods for transferring protein co-regulation information from external datasets.

2. What Is MCP?

The Model Context Protocol (MCP) provides a standardized way for AI applications to connect with external tools and data sources.

An MCP server exposes tools, while an MCP-compatible client allows an AI agent to discover and call them.

For our project, the architecture looked like this:

             AI Agent / MCP Client
                       |
           +-----------+-----------+
           |                       |
      Claude Code            GitHub Copilot CLI
           |                       |
           +-----------+-----------+
                       |
                MCP Protocol
                 (STDIO)
                       |
               scp-graphfs-mcp
                       |
             Scientific MCP Tools
                       |
       Protein Complexes / Knowledge Graphs
                       |
           Proteomics Dataset Analysis

Instead of writing a separate integration for every AI client, we could expose scientific capabilities through one MCP server.

3. The Six Scientific MCP Tools

Our server exposed six tools:

MCP Tool	Purpose
`get_complexes`	Retrieve protein-complex information
`explain_protein`	Explore biological information about a protein
`build_prior`	Construct knowledge-based priors
`propagate_scores`	Propagate information through graph relationships
`impute_with_prior`	Impute missing values using a biological prior
`select_panel`	Support protein measurement-panel selection

These tools made it possible to move from a natural-language research question to structured scientific operations.

4. Setting Up the Development Environment

I worked with the project's Python environment and MCP server.

After cloning the repository, I created a virtual environment and installed the development dependencies.

On Windows PowerShell:

git clone https://github.com/xX-its-amit-Xx/knowledgeispower.git
cd knowledgeispower

git switch claude/happy-hamilton-kdexid

python -m venv .venv
.\.venv\Scripts\Activate.ps1

python -m pip install -e ".[dev]"

The repository's active development branch was important because it contained the implementation used for the hackathon.

I also ran the project's offline tests and checked that the MCP server exposed its expected tools.

During my local testing, the offline suite reported:

19 passed, 27 xfailed

This was a snapshot of my development environment, not the final project's complete test count.

5. Connecting Claude Code to the MCP Server

One of my tasks was configuring Claude Code as an MCP client.

I installed the Claude Code CLI and registered the local scientific server using STDIO transport.

The following example uses a Windows virtual environment:

claude mcp add --transport stdio --scope local scp-graphfs -- ".\.venv\Scripts\scp-graphfs-mcp.exe"

For a local installation, use the actual executable path from your environment. An absolute path is generally more reliable.

To inspect the configuration:

claude mcp list
claude mcp get scp-graphfs

I successfully configured the connection and verified that Claude Code recognized the MCP server.

However, I couldn't complete a full Claude Code AI session myself because I didn't have the required Claude access.

The connection setup and verification were completed; a real Claude Code session transcript was not part of my individual submission.

6. Connecting GitHub Copilot CLI

I also configured GitHub Copilot CLI to communicate with scp-graphfs-mcp.

After installing and authenticating Copilot CLI, I used its MCP configuration interface to register the server.

The configuration included:

Server name: scp-graphfs
Transport: STDIO
Command: <absolute-path-to-scp-graphfs-mcp.exe>
Environment variables: None required
Tools: *

After configuration, the MCP server reported a connected status.

All six scientific tools were available to the Copilot agent.

This allowed me to run a real scientific analysis through an AI-agent workflow.

7. Running a Real Scientific Experiment

I used GitHub Copilot CLI to investigate the 26S proteasome, a protein complex involved in protein degradation.

The research question was:

Which subunits of the 26S proteasome are measured in data/datasets/petrosius2023_aml_sc.h5ad, and how many of their missing values can be filled from partners?

The workflow involved retrieving the protein complex, examining the dataset, preparing a 26S-only subset, and using the MCP imputation tool.

Step 1: Retrieve the Protein Complex

The agent called the MCP tool with parameters equivalent to:

{
  "id": "PSMB5",
  "sources": ["complexportal"]
}

The tool identified:

Complex Portal ID: CPX-5993
Complex: 26S proteasome
Total subunits: 33
Measured subunits: 31

Two subunits, PSMB3 and SEM1, were not measured in the dataset.

Step 2: Prepare the Dataset

The experiment used the Petrosius 2023 single-cell proteomics dataset.

The agent-assisted workflow prepared a temporary .h5ad subset containing the measured 26S proteasome proteins.

This subset was important because the following imputation counts applied specifically to the 26S proteasome experiment.

Step 3: Impute Missing Measurements

The agent called impute_with_prior using the Complex Portal biological prior.

The resulting metrics were:

Metric	Value
Total missing values	638
Imputed from protein partners	607
kNN fallback imputations	31
Partner-based coverage	95.1%

The experiment showed that the MCP tool could use protein-complex relationships to fill many missing entries in this particular subset.

Important: The 95.1% figure describes the fraction of missing entries handled through partner-based imputation. It is not an accuracy score or evidence that this method outperformed kNN.

The complete real-session transcript was preserved in the repository.

8. What Did the Team's Research Show?

Alongside MCP integration, the team conducted a broader scientific benchmark.

Finding 1: Curated Biological Knowledge Was Useful but Insufficient

Figure 1. Comparison of 22 biological knowledge representations. Curated physical complexes ranked highest among the priors, but none independently beat the plain kNN baseline. Source: KnowledgeIsPower team, OSI Hackathon 2026.

The top curated knowledge sources included:

Complex Portal
PDB co-structure
CORUM

These represented meaningful biological relationships, but none of the individual priors passed the project's development gate against the strongest data-only methods.

Finding 2: Reference-Based Protein Imputation Improved the Final Holdout Results

Figure 2. Untouched Brunner 2022 holdout experiment. The reference-based method reduced in-complex RMSE compared with the strongest data-only baseline. Source: KnowledgeIsPower team, OSI Hackathon 2026.

The final reported comparison showed:

Baseline	MCAR improvement	MNAR improvement
kNN	6.0%	8.9%
kNN + low-rank	2.4%	5.4%
Scrambled reference	2.5%	4.3%

All six reported paired-bootstrap 95% confidence intervals were below zero.

However, an earlier holdout produced weaker results against the strongest data-only baseline, so replication remained an important limitation.

The key research lesson was that learning protein relationships from external biological data could be more effective than relying on curated protein-complex knowledge alone.

9. My Open-Source Contribution

My work focused on practical MCP integration and reproducible documentation.

I completed:

Claude Code CLI installation, MCP registration, and connection verification.
GitHub Copilot CLI authentication and MCP integration.
Verification of all six scientific MCP tools.
A real agent-assisted proteomics experiment.
Local development setup and offline testing.
A reproducible GitHub Copilot CLI setup guide.
Export and redaction of the real agent-session transcript.
Submission of the documentation through merged Pull Request #8.

My merged contribution is available here:

https://github.com/xX-its-amit-Xx/knowledgeispower/pull/8

The scientific benchmark results discussed above were produced by the broader KnowledgeIsPower team.

10. What I Learned

This project gave me hands-on experience with several technologies and practices.

MCP integration: How AI agents can communicate with specialized tools using a standard protocol.

AI-assisted scientific workflows: How an agent can combine natural-language questions with structured scientific operations.

Python and bioinformatics: Working with proteomics datasets, protein identifiers, and scientific tools.

Testing and reproducibility: Why setup instructions, tool outputs, and real session transcripts matter.

Open-source collaboration: Contributing to an existing repository, following project specifications, and submitting pull requests.

Coming from a computer vision and machine learning background, I found it especially interesting to see how AI agents could support research outside my usual domain.

11. Final Thoughts

The OSI Hackathon 2026 was an opportunity to explore the intersection of AI agents, scientific computing, biological knowledge, and open-source software.

Our team built a functional MCP-based research system and reported both positive findings and important limitations.

My contribution helped demonstrate how an external AI coding agent could connect to the scientific server and execute a reproducible analysis.

I'm grateful to our team lead Amit Shenoy and my teammates Ratish Panda, Eeshan Khan, Fatima Zahra, and Asiya Ali for the collaboration.

I look forward to working on more projects involving AI agents, MCP, computer vision, and scientific research.

Project Links

Official OSI Hackathon 2026:
https://scientificintelligence.org/campaigns/osi-hackathon-2026

KnowledgeIsPower Submission:
https://scientificintelligence.org/campaigns/osi-hackathon-2026/submissions/190

Project Repository:
https://github.com/xX-its-amit-Xx/knowledgeispower

My Merged Pull Request:
https://github.com/xX-its-amit-Xx/knowledgeispower/pull/8

My GitHub:
https://github.com/abdullahjave

Written by Abdullah Javed — Computer Science student, AI and Computer Vision enthusiast, and open-source contributor.