Medical Disease/Injury Analysis App using Gemini API

In recent years, the integration of AI with healthcare has brought transformative potential in medical diagnostics. With the ability to analyze complex medical images, AI-driven applications can assist doctors and specialists in identifying diseases and recommending potential treatments. In this blog post, we’ll explore how to build a Streamlit-based Medical Disease/Injury Analysis App that leverages Google's Gemini API to provide intelligent image analysis.

This app is designed to showcase the capabilities of AI in processing medical images and generating insightful analyses, using the Generative AI from Google. The app can be used to upload medical images, analyze them, and provide structured findings based on AI's interpretation, all while emphasizing the importance of consulting with a medical professional before making decisions.

Key Features of the App

Medical Image Uploading: Users can upload medical images (JPEG/PNG) for analysis.
AI-Powered Image Analysis: The app uses the Google Gemini API to analyze medical images and generate insights.
Recommendations: Based on the AI’s analysis, the app can suggest potential further steps like treatments or tests.
Safety-First: Includes filters to block harmful content such as hate speech, harassment, and explicit materials.
Easy-to-Use Interface: Built using Streamlit for an intuitive and responsive UI.
Prerequisites
API Access: You’ll need access to Google’s Generative AI API (Gemini).
Python Setup: Ensure you have Python installed on your system.
Streamlit and Other Dependencies: Install Streamlit and other necessary libraries like Google’s Generative AI client using requirements.txt.

google.generativeai
protobuf==3.20.*

Step-by-Step Breakdown

1. Configuring the Gemini API To begin, we configure the Gemini API with an API key. This key is essential for authentication and communicating with Google’s model.

import google.generativeai as genai
from api_key import api_key

genai.configure(api_key=api_key)

The API uses a predefined system prompt that guides the AI to behave like a medical professional, ensuring that it provides a structured and informative analysis of the medical images uploaded.

1. Setting Up the System Prompt The system prompt is the "script" that instructs the AI on how to behave. In this case, the AI behaves as a medical professional tasked with analyzing medical images.

system_prompt = """
WE KNOW YOU ARE AN AI BUT RIGHT NOW YOU BEHAVE LIKE A MEDICAL PROFESSIONAL...
"""

This prompt ensures that the AI provides detailed image analysis while following medical guidelines, documenting findings, and suggesting appropriate actions or treatments.

1. Implementing the Streamlit App The app is built using Streamlit, a Python library that makes it easy to create web applications. The app interface allows users to upload a medical image (JPEG/PNG) and generates a detailed AI-powered analysis upon submission.

Setting Up the User Interface

import streamlit as st
from pathlib import Path

st.set_page_config(page_title="VitalImageAnalytics", page_icon=":robot:")
st.image("OIG2.2.jpeg")
st.title(" 🧑‍⚕️Vital❤️Image📷 Analytics📊🧑‍⚕️ ")
st.subheader("An application that can help users to identify medical images")
uploaded_file = st.file_uploader("Upload the medical image for analysis", type=['png', 'jpeg', 'jpg'])
submit_button = st.button("Generate the analysis")

This snippet configures the Streamlit app to display a title, a subheader, and an image uploader for users to input their medical images.

Image Upload and Analysis
When the user uploads an image and clicks the "Generate the analysis" button, the app passes the image to the Gemini API for analysis.

if submit_button:
    image_data = uploaded_file.getvalue()
    st.image(image_data)
    image_parts = [{"mime_type": "image/jpeg", "data": image_data}]

    prompt_parts = [image_parts[0], system_prompt]
    response = model.generate_content(prompt_parts)

    if response:
        st.title("Here is the analysis based on your image: ")
        st.write(response.text)

The AI processes the image and generates a structured response, following the system prompt’s guidelines.

1. Gemini API Configuration and Safety Settings The Gemini API model is configured with specific settings for optimal performance, including temperature, top-k sampling, and maximum output tokens. The app also enforces safety settings to filter out inappropriate content.

generation_config = {
  "temperature": 0.4,
  "top_p": 1,
  "top_k": 32,
  "max_output_tokens": 4096,
}

safety_settings = [
  {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_MEDIUM_AND_ABOVE"},
  {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_MEDIUM_AND_ABOVE"},
  {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_MEDIUM_AND_ABOVE"},
  {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_MEDIUM_AND_ABOVE"},
]

model = genai.GenerativeModel(model_name="gemini-1.5-flash",
                              generation_config=generation_config,
                              safety_settings=safety_settings)

These settings help ensure that the analysis remains safe and appropriate for the medical context.

Running the Application

Once all the components are set up, you can run the app locally by executing the following command:

streamlit run app.py

This will launch the Streamlit app in your browser, where you can upload medical images and get detailed AI-generated analysis reports.

Conclusion

This project demonstrates how AI can assist in the medical field by analyzing images and providing recommendations. However, it’s essential to note that this app is not meant for real-world medical diagnosis. Its primary goal is to showcase the potential of AI in healthcare and how generative models like Gemini can be utilized for medical image analysis.

For any clinical decision-making, it's critical to always consult with a licensed medical professional.

By integrating powerful AI tools with intuitive platforms like Streamlit, developers can create innovative applications that make complex tasks—such as medical image analysis—more accessible and automated.

Note: If you'd like to build this app yourself, don't forget to set up the api_key.py file with your own Gemini API key for proper functionality.

Happy coding!