Use Gemini Pro Asynchronously in Python

When your prompt is too large and the LLM starts to hallucinate, or when the data you want from the LLM is too extensive to be handled in one response, asynchronous calling can help you get the desired output. In this brief blog, I will teach you how to call Gemini Pro asynchronously to achieve the best results.

let's go

First create project in new directory then install following Package

pip install asyncio
pip install python-dotenv
pip install aiohttp

Go to this following link to get gemini api key
https://aistudio.google.com/app/apikey

in the above picture you can see that *create API key * click on it and get your api key

Now, create a file named .env in your project directory and add your key like this:
API_KEY = "here you copy paste the API key "

Next, create a file named main.py in your project directory.

Let's start coding the** main.py** file. First, import all necessary libraries and retrieve the API key from the .env file.

import asyncio
import os
from dotenv import load_dotenv
import aiohttp

load_dotenv()
API_KEY = os.getenv("API_KEY")

Now create a async function which return all list of all prompts

async def prompts() -> list:
    heros = """
    Give me the list of all 10 top highest win rate Dota 2 hero 2023
    """

    players = """
    Top players in the game Dota 2 in 2023
    """

    team = """
    Give me the name the name of all team who got directe invite in   TI 2023 dota 2.
    """

    return [heros, players, team ]

Now we are going to send an asynchronous POST request to the Google Generative Language API to generate content based on a given prompt. To do this, we will first set the endpoint we are going to access, then define the headers, and finally create the payload with all the necessary parameters for the endpoint. After that, we will send an asynchronous call to the endpoint. We are accessing response in json.
Note: Session (aiohttp.ClientSession): An instance of the aiohttp ClientSession class.

async def fetch_ai_response(session, prompt):
    url = f"https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key={API_KEY}"
    headers = {
        "Content-Type": "application/json"
    }
    payload = {
        "contents": [
            {
                "parts": [
                    {
                        "text": prompt
                    }
                ]
            }
        ]
    }
    async with session.post(url, headers=headers, json=payload) as response:
        result = await response.json()
        # Extract text from the response
        try:
            content = result['candidates'][0]['content']['parts'][0]['text']
            return content
        except (KeyError, IndexError) as e:
            # Log the error and response for debugging
            print(f"Error parsing response: {e}")
            print(f"Unexpected response format: {result}")
            return "Error: Unexpected response format"

following function takes a list of prompts and uses the fetch_ai_response function to retrieve the AI response for each prompt.
The function then uses asyncio.gather to run the fetch_ai_response function in parallel for each prompt.
The results are returned as a list of responses.

async def test_questions_from_ai() -> list:
    prompts_list = await prompts()
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_ai_response(session, prompt) for prompt in prompts_list]
        results = await asyncio.gather(*tasks)
    return results

Now calling test_questions_from_ai function Asynchronously

if __name__ == "__main__":
    responses = asyncio.run(test_questions_from_ai())
    for inx, response in enumerate(responses):
        print(f"Response: {inx} ", response)

now run following command to run see the response

python main.py

check the code on following repo on github
https://github.com/GoAndPyMasters/asyncgemini

here is my github profile

https://github.com/MuhammadNizamani

If you need any help contact with on linkedin
https://www.linkedin.com/in/muhammad-ishaque-nizamani-109a13194/