DEV Community

Evan Lin
Evan Lin

Posted on • Originally published at evanlin.com on

Using Vertex AI instead of Gemini in LangChain (Python)

#gemini #cloud #python #ai 
title: [Python] Switching from Gemini to Vertex AI in LangChain
published: false
date: 2025-03-04 00:00:00 UTC
tags: 
canonical_url: https://www.evanlin.com/til-gemini-vertex-ai/
---

![image-20250305221818691](https://www.evanlin.com/images/2022/image-20250305221818691.png)

# Background

Previously, I provided many examples of building a LINE Bot using LangChain. However, if you want a more stable backend and want to use more AI-related features, then Vertex AI is a great choice.  Next, I will start introducing the entire migration process and explain the areas that need to be introduced, as well as potential problems.

### Sample Code:

[https://github.com/kkdai/linebot-gemini-python](https://github.com/kkdai/linebot-gemini-python)

(With this code, you can quickly deploy to GCP Cloud Run)

## Building a LINE Bot with LangChain and Gemini to Vertex AI

First, here's a simple example code for building a LINE Bot with LangChain + Gemini:

Code related to Webhook processing:
Enter fullscreen mode Exit fullscreen mode 
for event in events:
    if not isinstance(event, MessageEvent):
        continue

    if (event.message.type == "text"):
        # Process text message using LangChain
        msg = event.message.text
        response = generate_text_with_langchain(f'{msg}, reply in zh-TW:')
        reply_msg = TextSendMessage(text=response)
        await line_bot_api.reply_message(
            event.reply_token,
            reply_msg
        )
Enter fullscreen mode Exit fullscreen mode 

Next, let's explain the content of `generate_text_with_langchain`:
Enter fullscreen mode Exit fullscreen mode

Initialize LangChain with Gemini

os.environ["GOOGLE_API_KEY"] = gemini_key

....

def generate_text_with_langchain(prompt):
"""
Generate a text completion using LangChain with Gemini model.
"""
# Create a chat prompt template with system instructions
prompt_template = ChatPromptTemplate.from_messages([
SystemMessage(
content="You are a helpful assistant that responds in Traditional Chinese (zh-TW)."),
HumanMessage(content=prompt)
])

# Format the prompt and call the model
formatted_prompt = prompt_template.format_messages()
response = text_model.invoke(formatted_prompt)

return response.content
Enter fullscreen mode Exit fullscreen mode 

This is a snippet of code for building a LINE Bot using LangChain and Gemini.  [Complete code](https://github.com/kkdai/linebot-gemini-python/blob/a746ad144bf4e0a760e5d6b1d361b5a7745bbadb/main.py).

## What is Vertex AI? What are its special features?

![Typora 2025-03-05 22.17.07](https://www.evanlin.com/images/2022/Typora%202025-03-05%2022.17.07.png)

Vertex AI is a comprehensive machine learning platform provided by Google Cloud, designed to simplify model training, deployment, and management. It is particularly suitable for developers who need enterprise-grade solutions. Unlike directly using the Gemini API, Vertex AI offers the following unique features:

1.  Integration of Gemini Models and More Choices Vertex AI supports the Gemini series models (such as gemini-pro and gemini-1.5-flash), as well as other models (such as PaLM and Codey), giving you more choices to meet different needs.
2.  Hosted Runtime and Auto-Scaling Through Vertex AI's Reasoning Engine, you can easily deploy LangChain applications and enjoy auto-scaling, security, and monitoring features without managing servers yourself.
3.  Multi-Modal Support Vertex AI's Gemini models support multi-modal input (text, images, and even videos), which is very useful for building feature-rich LINE Bots (such as processing image messages).
4.  Enterprise-Grade Security and Compliance Vertex AI provides IAM (Identity and Access Management), data encryption, and regional expansion capabilities to ensure that applications meet enterprise needs.
5.  Tool Integration and Function Calling Vertex AI supports Function Calling, allowing models to interact with external APIs or tools, such as weather queries or database searches, enhancing the practicality of LINE Bots.

## Starting to Migrate Gemini to Vertex AI

First, you will need two main parameters:

-   \[Project\_ID]: Your GCP Project ID
-   \[Location]: The region of Vertex AI! The region of Vertex AI! The region of Vertex AI! (Important, said three times)

Next, here is a good [Colab](https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_flash.ipynb) that you can use to learn more.

The main code is as follows:
Enter fullscreen mode Exit fullscreen mode

import os
from google import genai
from google.genai.types import (
FunctionDeclaration,
GenerateContentConfig,
GoogleSearch,
HarmBlockThreshold,
HarmCategory,
MediaResolution,
Part,
Retrieval,
SafetySetting,
Tool,
ToolCodeExecution,
VertexAISearch,
)

PROJECT_ID = "" # @param {type: "string", placeholder: "[your-project-id]", isTemplate: true}
if not PROJECT_ID or PROJECT_ID == "[your-project-id]":
PROJECT_ID = str(os.environ.get("GOOGLE_CLOUD_PROJECT"))

LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")

client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)

設定 model

MODEL_ID = "gemini-2.0-flash-001" # @param {type: "string"}

產生輸出

response = client.models.generate_content(
model=MODEL_ID, contents="What's the largest planet in our solar system?"
)

display(Markdown(response.text))


## So, how do you migrate from LangChain + Gemini to LangChain + VertexAI?

According to the LangChain documentation, the `VertexAI` in `langchain_google_vertexai` is mainly used for Google Vertex AI. The following points need attention:

-   If placed on GCP Cloud Run, you do not need to include the JSON content of the Services Account.
-   If not placed on GCP, you must put the JSON content into the `GOOGLE_APPLICATION_CREDENTIALS` system parameter.

If built on GCP, the following is the most basic way to start.
Enter fullscreen mode Exit fullscreen mode

Create LangChain Vertex AI model instances

For Vertex AI, we use "gemini-2.0-flash" instead of "gemini-2.0-flash-lite"

text_model = ChatVertexAI(
model_name="gemini-2.0-flash-001",
project=google_project_id,
location=google_location,
max_output_tokens=1024
)


The code to call Vertex AI through LangChain is as follows:
Enter fullscreen mode Exit fullscreen mode

def generate_text_with_langchain(prompt):
"""
Generate a text completion using LangChain with Vertex AI model.
"""
# Create a chat prompt template with system instructions
prompt_template = ChatPromptTemplate.from_messages([
SystemMessage(
content="You are a helpful assistant that responds in Traditional Chinese (zh-TW)."),
HumanMessage(content=prompt)
])

# Format the prompt and call the model
formatted_prompt = prompt_template.format_messages()
response = text_model.invoke(formatted_prompt)

return response.content
Enter fullscreen mode Exit fullscreen mode 

That's it. This time, we only focus on converting the text part of LangChain Gemini to use Vertex AI. However, because the use of images on Vertex AI will be through Google Cloud Storage, we will not focus on this part for now. If you want to know more, you can

### Problems to note:

1.
#### How to solve the following error?
Enter fullscreen mode Exit fullscreen mode

details = "Publisher Model projects/PROJECT_ID/locations/asia-east1/publishers/google/models/gemini-2.0-flash not found."


This problem occurs mainly because if you choose `asia-east1` as your Vertex AI region. It currently does not support the gemini-2.0 related models.

You need to change it to `us-central1` to call it correctly.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)