DEV Community: xbill

MCP Development with Amazon Elastic Beanstalk (EBS)

xbill — Fri, 08 May 2026 21:56:56 +0000

Leveraging Gemini CLI and the underlying Gemini LLM to build Model Context Protocol (MCP) AI applications with Python from a local development environment deployed to the EBS service on AWS.

Yet another Python MCP Demo?

Yes — thanks for asking.

Python has traditionally been the main coding language for ML and AI tools. The goal of this article is to provide a minimal viable basic working MCP stdio server that can be run locally without any unneeded extra code or extensions.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lightsail-python-aws$ python --version
Python 3.13.12

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~$ gemini

 ▝▜▄ Gemini CLI v0.40.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

The AWS Cli tools and Lightsail extensions need current version of Docker. If your environment does not provide a recent docker tool- the Docker Version Manager can be used to downlaod the latest supported Docker:

Install

Amazon Elastic Bean Stalk

AWS Elastic Beanstalk is a Platform-as-a-Service (PaaS) used for deploying and scaling web applications and services into the Amazon Web Services (AWS) Cloud. [1, 2]

It simplifies the development process by allowing you to upload your application code while the service automatically manages the complex infrastructure details.

More details are here:

Web App Deployment - AWS Elastic Beanstalk - AWS

The EBS console looks similar to this:

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Elastic Beanstalk. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The simplest transport that the SDK supports is the stdio (stdio/stdout) transport — which connects a locally running process. Both the MCP client and MCP Server must be running in the same environment.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

mcp.run(
        transport="http",
        host="0.0.0.0",
        port=port,
    )

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-ebs-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-ebs-python-aws$ aws login --remote

xbill@penguin:~/gemini-cli-aws/mcp-ebs-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.

Run the deploy version on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-ebs-python-aws$ make deploy
Creating Lightsail instance mcp-vps-python-aws...
Instance already exists or creation in progress.
Waiting for instance mcp-vps-python-aws to reach 'running' state...
Instance is running.

You can validate the final result by checking the messages:

  Environment Status: mcp-server-eb-env
   * Status: Ready
   * Health: Green
   * Platform: Docker running on 64bit Amazon Linux 2/4.8.2
   * Deployed Version: app-260508_165941877951
   * CNAME: mcp-server-eb-env.eba-ce3smmqd.us-east-1.elasticbeanstalk.com

  Recent Events
  The environment was recently created and updated successfully:
   * 20:56:10: createEnvironment started.
   * 20:59:32: Initial instance deployment completed.
   * 20:59:38: Successfully launched environment.
   * 21:00:03: Subsequent instance deployment completed successfully.

You can then get the endpoint:

 > make endpoint

  Retrieving Application Endpoint: Retrieving the public endpoint URL of the Elastic Beanstalk environment.

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ Shell make endpoint │
│ │
│ http://mcp-server-eb-env.eba-ce3smmqd.us-east-1.elasticbeanstalk.com │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The public endpoint for your Elastic Beanstalk environment is:

  http://mcp-server-eb-env.eba-ce3smmqd.us-east-1.elasticbeanstalk.com

The service will be visible in the AWS console:

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

{
  "mcpServers": {
    "mcp-ebs-python-aws": {
      "httpUrl": "http://mcp-server-eb-env.eba-ce3smmqd.us-east-1.elasticbeanstalk.com/mcp/"
    }
  }
}

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

xbill@penguin:~/gemini-cli-aws/mcp-ebs-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.41.2
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

🟢 mcp-ebs-python-aws - Ready (1 tool)
  Tools:
  - mcp_mcp-ebs-python-aws_greet

Then the remote MCP tool in EBS can be called:

 > mcp_mcp-ebs-python-aws_greet Hello Beanstalk!

  Topic: Executing the greet tool to verify the MCP server functionality.

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greet (mcp-ebs-python-aws MCP Server) {"param":"Hello Beanstalk!"} │
│ │
│ Hello, Hello Beanstalk!! │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ Hello, Hello Beanstalk!!

Gemini CLI can also summarize the deployment:

> summarize the deployment

✦ The mcp-ebs-python-aws project is a Python-based Model Context Protocol (MCP) server deployed to AWS Elastic Beanstalk (EB) using Docker.

  Deployment Architecture
   * Server Framework: Built with FastMCP and Starlette, providing an asynchronous foundation for MCP tools and standard HTTP endpoints.
   * Infrastructure: Uses the Elastic Beanstalk Docker platform, which automatically handles capacity provisioning, load balancing, and
     auto-scaling.
   * Compute: Defaults to t3.micro instances in the us-east-1 region.

  Deployment Workflow (make deploy)
  The deployment is automated via a Makefile and deploy.sh script:
   1. Validation: Runs unit tests using pytest to ensure code quality before deployment.
   2. Initialization: Uses the EB CLI (eb init) to configure the project for the AWS Docker platform.
   3. Provisioning: Creates the environment (eb create) if it doesn't exist, setting up the required AWS resources (ALB, ASG, EC2).
   4. Updates: Deploys code changes (eb deploy), which triggers a Docker build and rolling update on the EB instances.

  Key Endpoints
  Once deployed, the environment provides:
   * MCP SSE Transport: http://<eb-cname>/mcp
   * Health Check: http://<eb-cname>/health (used by EB for monitoring)
   * Root Info: http://<eb-cname>/

  Management Commands
   * make status: Displays the current health of the EB environment and recent deployment events.
   * make endpoint: Retrieves the public URL of the deployed server.
   * make aws-destroy: Terminates the environment and all associated AWS resources to stop billing.

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon EBS was validated with a incremental step by step approach.

This MCP server was then deployed to Amazon Elastic Bean Stalk. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

Deploying a Rust MCP Server to Amazon LightSail

xbill — Fri, 08 May 2026 18:01:57 +0000

The rmcp crate and standard Rust libraries are used to build a basic MCP Server in Rust. This MCP Server is then built and deployed to AWS LightSail and validated locally with Gemini CLI

More MCP Demos?

Yes sir. Ferris the Crab demands it.

Why not just use Python?

Python has traditionally been the main coding language for ML and AI tools. One of the strengths of the MCP protocol is that the actual implementation details are independent of the development language. The reality is that not every project is coded in Python- and MCP allows you to use the latest AI appt roaches with other coding languages.

What is this Tutorial Trying to Do?

Building on previous tutorials, the goal is to extend a Rust MCP server with basic support for deployment to AWS.

What is Rust?

Rust is a high performance, memory safe, compiled language:

Rust

Rust provides memory safe operations beyond C/C++ and also can provide exceptional performance gains as it is compiled directly to native binaries.

So is this the real Slim Shady?

So what is different about this lab compared to all the others out there?

This is one of the first deep dives into deploying a Rust based MCP server hosted on AWS. The Amazon LightSail service was targeted for ease of setup and deployment.

Rust Setup

Instructions to install Rust are available here:

Getting started

For a Linux like environment the command looks like this:

curl — proto ‘=https’ — tlsv1.2 -sSf https://sh.rustup.rs | sh

Rust also depends on a working C compiler and OpenSSL setup. For a Debian 12 system — install the basic tools for development:

sudo apt install build-essential
sudo apt install libssl-dev
sudo apt install pkg-config
sudo apt-get install libudev-dev
sudo apt install make
sudo apt install git

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

▝▜▄ Gemini CLI v0.33.1
    ▝▜▄
   ▗▟▀ Logged in with Google /auth
  ▝▀ Gemini Code Assist Standard /upgrade no sandbox (see /docs) /model Auto (Gemini 3) | 239.8 MB

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

You can version check the tool after installation:

xbill@penguin:~/gemini-cli-aws/mcp-lightsai-rust-aws$ aws --version
aws-cli/2.34.43 Python/3.14.4 Linux/6.6.99-09128-g14e87a8a9b71 exe/x86_64.debian.12

Amazon Lightsail

Amazon Lightsail is an easy-to-use virtual private server (VPS) provider and cloud platform designed by AWS for simpler workloads, offering developers pre-configured compute, storage, and networking for a low, predictable monthly price. It is ideal for hosting small websites, simple web apps, or creating development environments.

More information is available on the official site here:

Amazon's Simple Cloud Server | Amazon Lightsail

And this is the direct URL to the console:

https://lightsail.aws.amazon.com/ls/webapp/home/containers

The Lightsail console will look similar to:

Setup the Basic Environment

At this point you should have a working Rust environment and a working Gemini CLI installation. All of the relevant code examples and documentation is available in GitHub.

The next step is to clone the GitHub repository to your local environment:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ aws login --remote
Browser will not be automatically opened.

xbill@penguin:~/gemini-cli-aws/mcp-ligthsail-rust-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.

Finally install the packages and dependencies:

~/gemini-cli-aws/mcp-lightsail-rust-aws 
make install

Build The Rust MCP Server

Some background information on building and configuring a Rust MCP server is here:

Building a Secure HTTP Transport MCP Server with Rust, and Gemini CLI

The mcp-lightsail-rust subdirectory has the complete Rust MCP server in one subdirectory.

Minimal System Information Tool Build

The first step is to build the basic tool directly with Rust. This allows the tool to be debugged and tested locally before adding the MCP layer.

First build the tool locally:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make
Building the Rust project...
   Compiling mcp-lightsail-rust-aws v1.0.0 (/home/xbill/gemini-cli-aws/mcp-lightsail-rust-aws)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.67s
xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$

then lint check the code:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make lint
Linting code...
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.61s
xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$

and run local tests:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make test
Running tests...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.22s
     Running unittests src/main.rs (target/debug/deps/mcp_lightsail_rust_aws-926af5aee1927183)

running 1 test
test tests::test_greeting ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The last step is to build the production version:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make release
Building Release...
   Compiling mcp-lightsail-rust-aws v1.0.0 (/home/xbill/gemini-cli-aws/mcp-lightsail-rust-aws)
    Finished `release` profile [optimized] target(s) in 43.31s
xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$

The MCP server can be started locally:

xbill@penguin:~/mcp-adk-rust/mcp-cloudrun-rust$ make start
Building Release...
    Finished `release` profile [optimized] target(s) in 0.14s
Starting the MCP server...
Server started with PID 1569

Then Gemini CLI is used as a MCP client:

🟢 local-rust - Ready (1 tool)
  Tools:
  - mcp_local-rust_greeting

The MCP tool can then be tested:

> mcp_local-rust_greeting local

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Action Required │
│ │
│ ? greeting (local-rust MCP Server) {"message":"local"} │
│ │
│ MCP Server: local-rust │
│ Tool: greeting │
│ │
│ MCP Tool Details: │
│ (press Ctrl+O to expand MCP tool details) │
│ Allow execution of MCP tool "greeting" from server "local-rust"? │
│ │
│ 1. Allow once │
│ 2. Allow tool for this session │
│ 3. Allow all server tools for this session │
│ ● 4. Allow tool for all future sessions ~/.gemini/policies/auto-saved.toml │
│ 5. No, suggest changes (esc) │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The mcp_local-rust_greeting tool was executed with the message "local", returning:
  "Hello World MCP! local"

Deploy To LightSail

A basic Dockerfile is used to build an image for deployment:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make deploy
Building the Docker image...
[+] Building 5.3s (15/15) FINISHED docker:default
 => [internal] load build definition from Dockerfile 0.0s

Get the Endpoint:

xbill@penguin:~/gemini-cli-aws/mcp-lightsail-rust-aws$ make endpoint
https://mcp-lightsail-rust-aws.6wpv8vensby5c.us-east-1.cs.amazonlightsail.com/

Check Gemini MCP settings:

{
    "mcpServers": {
    "aws-lightsail-rust": {
      "httpUrl": "https://mcp-lightsail-rust-aws.6wpv8vensby5c.us-east-1.cs.amazonlightsail.com/mcp"
    },
    "local-rust": {
      "httpUrl": "http://127.0.0.1:8080/mcp"
    }
  }
}

The service will be visible on the LightSail console:

Final Test

Start up Gemini CLI and check the MCP server status:

🟢 aws-lightsail-rust - Ready (1 tool)
  Tools:
  - mcp_aws-lightsail-rust_greeting

> mcp_aws-lightsail-rust_greeting Hello LightSail!

  Executing Lightsail Greeting: Executing the greeting tool for Amazon Lightsail Rust MCP server.

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greeting (aws-lightsail-rust MCP Server) {"message":"Hello LightSail!"} │
│ │
│ Hello World MCP! Hello LightSail! │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The mcp_aws-lightsail-rust_greeting tool returned:

  Hello World MCP! Hello LightSail!

Summary

A complete HTTP transport MCP server was built using Rust. This application was tested locally with Gemini CLI. Then, the entire solution was deployed to AWS LightSail. The remote MCP server was validated with Gemini CLI locally.

Gemma-4-31B on v6e-4 TPU Benchmarks

xbill — Fri, 08 May 2026 16:57:09 +0000

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

model: Gemma-4-31B

🚀 Gemma 4 TPU v6e-4 Performance Report

📋 Deployment Overview

Model: google/gemma-4-31B-it
Hardware: Cloud TPU v6e-4 (Trillium)
Runtime: v2-alpha-tpuv6e (Flex-start)
TPU Location: southamerica-east1-c
Serving Engine: vLLM (v0.20.2rc1.dev111+g8eb401134)

📊 Performance Summary (C1 - C1024)

Peak Prefill Throughput: 463,345 tokens/sec
Avg TTFT (~1.6k tokens): 2.597 seconds
Avg TTFT (16k tokens): 4.775 seconds

📈 Concurrency Scaling Matrix (Mean per Concurrency)

concurrency	avg_ttft	prefill_tps
1	0.546599	14778.3
2	0.562068	28121.7
4	0.595823	51869.1
8	0.679816	88055.5
16	0.872466	133697
32	1.16488	191631
64	1.55596	261802
128	2.15464	328909
256	3.55723	352654
512	7.59987	318854
1024	21.005	240170

🔍 Key Findings

Efficiency Saturated: Maximum throughput was achieved at concurrency 256, reaching 463,345 tok/s.
Trillium Scalability: The TPU v6e-4 architecture handled 1024 concurrent requests without memory exhaustion, maintaining throughput stability even under extreme queueing.
Responsive Context: Even at 16k tokens, the TTFT remained under 1 second for low concurrencies (C1-C8).

💸 Cost Efficiency

Estimated Hourly Cost: ~.40 (Flex-start rate for v6e-4)
Throughput Efficiency: ~308,000,000 tokens per dollar at peak saturation.

Report generated by Gemini CLI on 2026-05-08.

⚖️ Competitive Analysis: Dense (31B) vs. MoE (26B A4B)

Metric	Gemma 4 31B (Dense)	Gemma 4 26B (MoE)	Winner
Model Architecture	Dense (31B parameters)	Sparse (26B Total / 3.8B Active)	MoE (Efficiency)
Peak Throughput (TPU v6e-4)	463,345 tok/s	~457,000 tok/s	Dense (Slightly)
Interactive Latency (TTFT)	0.314s (at C1/128t)	< 1.200s (Interactive)	Dense (Low Load)
Active Compute cost	31B params / token	3.8B params / token	MoE (7.5x lower)
Max Context Window	64K (Tested to 16K)	256K (Shared KV Cache)	MoE

Analysis Summary

Throughput Parity: Our benchmarks show that the 31B Dense model actually matches or slightly exceeds the peak throughput of the 26B MoE model on the same TPU v6e-4 hardware. This indicates exceptional hardware-software co-optimization for dense matrix operations in the Trillium architecture.
Compute Efficiency: While throughput is similar, the MoE model is 7.5x more compute-efficient per token generated (activating only 3.8B parameters). In a multi-tenant environment, the MoE model would likely sustain higher concurrent user counts before hitting power or thermal limits.
Latency Advantage: The Dense model demonstrates superior snappiness for low-load interactive tasks, with a TTFT of 0.314s, which is significantly below the MoE target of 1.2s.
Context Scaling: The MoE model's Shared KV Cache allows it to scale to 256K tokens, whereas our Dense stack is currently optimized for high-throughput within the 16K-64K range.

Gemma-4-26B on v6e-4 TPU Benchmarks

xbill — Thu, 07 May 2026 20:22:23 +0000

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

model: google/gemma-4-26B-A4B-it

gemini "run the context benchmark with concurrency from 1, 2, 4, 8, 16, 32,64, 128, 256, 512, 1024 up to 16384 tokens"

Extreme Stress Test at 1024 Concurrency: Execute an extreme-load benchmark sweep up to concurrency 1024 to find the breaking point.

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ Shell Execute an extreme-load multi-dimensional benchmark sweep up to concurrency 1024. Steps reduced to 2 (0, 8192, 16384) to manage t… │
│ │
│ ... first 89 lines hidden (Ctrl+O to show) ... │
│ | 128 | 256 | 1.14308 | 18554.9 | │
│ | 8192 | 256 | 3.15647 | 419771 | │
│ | 16384 | 256 | 5.58021 | 457101 | │
│ | 128 | 512 | 3.80498 | 11249.3 | │
│ | 8192 | 512 | 6.69551 | 374758 | │
│ | 16384 | 512 | 10.7299 | 443335 | │
│ | 128 | 1024 | 14.7126 | 5878.47 | │
│ | 8192 | 1024 | 19.6215 | 267038 | │
│ | 16384 | 1024 | 24.6752 | 388684 | │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The extreme-load benchmark sweep up to 1024 concurrency is complete. We have successfully identified the breaking point of the current TPU v6e
vLLM stack.

📊 Extreme Load Performance Summary

┌─────────────┬──────────────────┬──────────────┬───────────────────┐
│ concurrency │ context (tokens) │ avg_ttft (s) │ batch prefill_tps │
├─────────────┼──────────────────┼──────────────┼───────────────────┤
│ 128 │ 16,384 │ 3.30s │ 439,833 │
│ 256 │ 16,384 │ 5.58s │ 457,101 │
│ 512 │ 16,384 │ 10.73s │ 443,335 │
│ 1024 │ 16,384 │ 24.68s │ 388,684 │
└─────────────┴──────────────────┴──────────────┴───────────────────┘
📉 Breaking Point Analysis

Saturation Point: The aggregate throughput peaked at 457,101 tokens/second at 256 concurrency. Beyond this point, throughput begins to decline as the system becomes bottlenecked by request management overhead and scheduling.
The "Knee" of the Curve:
- Throughput Decline: Moving from 256 to 1024 concurrency resulted in a 15% drop in aggregate throughput (457k to 388k).
- Latency Explosion: The Time to First Token (TTFT) grew exponentially, jumping from ~5.6s at concurrency 256 to nearly 25 seconds at concurrency 1024.
Small Context Collapse: At 1024 concurrency, even short 128-token requests took 14.7 seconds to start generating, compared to just 0.3s at low concurrency. This indicates the server's internal queues are completely saturated.
Robustness: Remarkably, the server did not crash or OOM even under 1024 concurrent 16k requests (a total of 16.7 million tokens in flight). This speaks to the robustness of the vLLM-TPU implementation and the underlying Trillium hardware.

🏁 Final Recommendation
For this google/gemma-4-26B-A4B-it deployment on an 4-chip TPU v6e pod:

Optimal High-Throughput: Target 128-256 concurrency. This yields the highest efficiency (~440k-457k tps) with acceptable latency (3s-6s).
Optimal Interactive: Target 1-16 concurrency. This keeps TTFT under 1.2s while still processing up to 200k tokens per second.
Avoid: Concurrencies above 512, as latency becomes prohibitive (>10s) and throughput starts to degrade significantly.

✦ The visualization of our extreme stress test (up to 1024 concurrency) provides a clear picture of the TPU v6e's performance boundaries:

📈 Visual Summary

Avg TTFT (s) vs. Context Length
The plot shows several distinct curves corresponding to the different concurrency levels.
- Low-Middle Curves: For concurrencies 1–128, the lines remain flat and clustered near the bottom (sub-4s), indicating the system is well within its operational limits.
- Top Curves: The lines representing 512 and 1024 concurrency show a sharp vertical shift, climbing up to the 25-second mark. This illustrates the "queue wait" effect, where requests spend significantly more time waiting for compute resources than being processed.
Prefill Throughput (tok/s) vs. Context Length
This plot displays the aggregate system efficiency:
- Saturation Peak: You can see the density of the curves increasing as aggregate throughput approaches 450,000–475,000 tokens/second.
- Performance Inversion: At the highest concurrency (1024), the throughput curve actually dips below the 128/256 curves. This visual "crossover" confirms that the system has moved past its efficiency peak and is now incurring a penalty for managing too many simultaneous requests.

🏁 Final Project State
The benchmark suite is now a robust, multi-dimensional performance analysis engine:

Enhanced context_benchmark.py: Supports deep sweeps of prompt length × concurrency.
Integrated MCP Tool: The run_context_benchmark tool allows triggering these complex stress tests via a single agent command.
Advanced Visualization: plot_context_benchmark.py handles high-density data and provides ASCII visualization for immediate analysis.
Optimized Reporting: Automatically generates Markdown, CSV, and JSON outputs for every run.

The project is fully prepared for production capacity planning and further infrastructure tuning.

xbill@penguin:~/aisprintapr2026/gemma4-benchmark$ python plot_context_benchmark.py

--- Context Length vs. Avg TTFT (s) ---
25 +----------------------------------------------------------------------+
| ****** |
| ****FT (s) **** |
| ******* |
20 | ****** |
| ******* |
| ******** |
| ******** |
15 | ******** |
|*********** |
| ********************** |
| ********************* |
| *********** |
10 | *********** |
| *********** |
| *********** |
| ********** **************** |
5 | ********** *************************************** |
|***************** **************** ***************** |
| ********************************************************* |
|***************************************************************** |
0 +----------------------------------------------------------------------+
0 2000 4000 6000 8000 10000 12000 14000 16000 18000

--- Context Length vs. Prefill Throughput (tok/s) ---
500000 +------------------------------------------------------------------+
| |
450000 | ******************** |
| ******************************** |
400000 | ** ************************ |
| ** ********* ****** **** |
350000 | ** *** *************** |
| * *** **************** **** |
300000 | ***** ********* ****** **** |
| ***** ** ******** ***** ****** |
250000 | ***** *** ********* ***************** |
| ***** ** *** **** ********** ***** |
| **** *** ** *********** ****** ****** |
200000 | **** ** ************ ***** ************ |
| ******* *********** ******************** |
150000 | ****** ********* *********** ******* ******** |
| ***** *************** ************************* |
100000 | ****************** ********************** ******** |
| ********************************* **************** |
50000 |************************************** **************** |
|************************************************************* |
0 +------------------------------------------------------------------+
0 2000 4000 6000 8000 10000 12000 14000 16000 18000

Building ADK Agents with Azure Cloud Shell

xbill — Thu, 07 May 2026 17:14:45 +0000

This article covers installing and Running Gemini CLI cross-cloud. The Azure Cloud Shell provides a basic Linux environment that can support running Gemini CLI.

Azure Cloud Shell

Azure Cloud Shell is a browser-based, authenticated terminal for managing Azure resources, accessible directly through the Azure portal, shell.azure.com, or the mobile app. It provides a preconfigured environment with Bash or PowerShell, eliminating local installation of tools. It runs on a temporary, secure, Linux-based host.

The full background information on the Cloud Shell is here:

https://azure.microsoft.com/en-us/get-started/azure-portal/cloud-shell

More information is available here:

Get started with Azure Cloud Shell

Why would I want Gemini CLI on Azure? Isn’t that a Google Thing?

Yes- Gemini CLI leverages the Google Cloud console and Gemini models but it is also open source and platform agnostic. Many applications are already cross-cloud so this enables familiar tools to be run natively on Microsoft Azure.

Activating Azure CloudShell

The Azure CloudShell comes preinstalled on all Azure Accounts.

To access the shell — click the Terminal Icon in the upper right hand corner of the console:

Then activate the full shell:

Note- the disk space in the environment is limited to 5GB- but this is sufficient for Gemini CLI, Google Cloud Tools, and some customization.

What is this project trying to Do?

This project aims to configure an ADK agent to use an Azure Foundry model. The ADK provides agent support in standard Python code.

Lite LLM

LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-compatible interface to over 100 LLMs (Anthropic, Gemini, Azure, Bedrock, Ollama). It simplifies API management by allowing users to call any model using the same format, providing features like spend tracking, model fallbacks, key management, and load balancing.

More info is here:

GitHub - BerriAI/litellm: Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

Azure Foundry

Azure AI Foundry (formerly Azure AI Studio) is a unified platform-as-a-service for developing, customizing, and managing AI agents and applications. It serves as an AI “factory” combining model catalogs (including OpenAI and Hugging Face), low-code tools, and enterprise-grade security for building generative AI apps. [1, 2, 3, 4]

More details are available here:

https://azure.microsoft.com/en-us/products/ai-foundry

Why would I want Gemini CLI with Azure? Isn’t that a Google Thing?

Azure CLI

The Azure Command-Line Interface (CLI) is a cross-platform tool used to connect to Azure and execute administrative commands on Azure resources. It allows you to manage services like virtual machines, databases, and networking through a terminal using interactive prompts or scripts. [1, 2, 3]

More details are available here:

What is the Azure Developer CLI?

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~/aisprintapr2026/tpu-vllm-devops-agent$ gemini

▝▜▄ Gemini CLI v0.39.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

Agent Development Kit

The Google Agent Development Kit (ADK) is an open-source, Python-based framework designed to streamline the creation, deployment, and orchestration of sophisticated, multi-agent AI systems. It treats agent development like software engineering, offering modularity, state management, and built-in tools (like Google Search) to build autonomous agents.

The ADK can be installed from here:

Agent Development Kit (ADK)

Where do I start?

The strategy for starting multi agent development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, and ADK agent is built, debugged, and tested locally.

Setup the Basic Environment

At this point you should have a working Python environment and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-azure

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd adk-litellm
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Azure Setup

The Azure CLI provides the basic tools for working with the cloud based services:

Check the installation:

william [~/gemini-cli-azure/adk-litellm-azure]$ az --version
azure-cli 2.85.0 *

core 2.85.0 *
telemetry 1.1.0

Extensions:
ml 2.42.0
ssh 2.0.6

Dependencies:
msal 1.35.1
azure-mgmt-resource 24.0.0

Python location '/usr/bin/python3.12'
Config directory '/home/william/.azure'
Extensions directory '/home/william/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.12/site-packages/azure-cli-extensions'

Python (Linux) 3.12.9 (main, Mar 26 2026, 23:21:55) [GCC 13.2.0]

Legal docs and information: aka.ms/AzureCliLegal

You have 2 update(s) available. They will be updated with the next build of Cloud Shell.
william [~/gemini-cli-azure/adk-litellm-azure]$

Then run the Azure setup scripts:

./setup-azure-phi.sh

./get-azure-phi-creds.sh

The Azure console should look something like this:

With the resource group looking like this:

Running the Python Code

First- switch the directory with the Python version of the MCP sample code. Run the release version on the local system:

cd adk-litellm$ make install
Processing ./.

Verify The ADK Installation

First- verify the ADK is installed:

william [~/gemini-cli-azure/adk-litellm-azure]$ adk --version
adk, version 1.32.0
william [~/gemini-cli-azure/adk-litellm-azure]$

To check the ADK setup, run the ADK CLI locally with the hello Agent. This is a basic Hello World style agent. No external tools or MCP calls are used in the Agent code- which allows the ADK installation to be checked:

william [~/gemini-cli-azure/adk-litellm-azure]$ adk run agents/azure
Log setup complete: /tmp/agents_log/agent.20260505_192533.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
/home/william/.local/lib/python3.12/site-packages/google/adk/cli/cli.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/william/.local/lib/python3.12/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
Running agent azure_agent, type exit to exit.
[user]: what model are you
19:25:39 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: I am Phi, an AI developed by Microsoft, created to help assist you with providing information, answering questions, and helping out with related tasks. How can I assist you today?



  
  
  Phi Models on Azure


Microsoft Phi models are a family of highly capable, cost-effective Small Language Models (SLMs) designed for low-latency generative AI, available on Azure AI Foundry. They support text, image, and audio inputs (Phi-4-multimodal) and are ideal for memory-constrained environments. Key models include Phi-4, Phi-3.5-MoE, and Phi-3-mini/medium, optimized for high-performance reasoning tasks.

More details are here:



  
  
  Deep Dive with a Phi Model


Once the connection has been tested. You can drill into the model:



xbill@penguin:~/gemini-cli-azure/adk-litellm-azure$ adk run agents/azure
Log setup complete: /tmp/agents_log/agent.20260505_134754.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/cli.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
Running agent azure_agent, type exit to exit.
[user]: what model are you
13:48:00 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: I am Phi, an AI developed by Microsoft, designed to help you with a wide range of queries and tasks. How can I assist you today?



  
  
  One Last Code Review with Gemini CLI


Gemini CLI was used to summarize the project:



> code review adk agent for phi model                                                                                                            

✦ I've reviewed the Azure agent implementation in agents/azure/agent.py. Overall, the implementation correctly utilizes the ADK (Agent Development
  Kit) and integrates with Azure AI Foundry via LiteLLM.

  🔍 Code Review Findings

  1. ADK Best Practices
   * Missing description: While the name is correctly defined, the description parameter is missing. In ADK, the description is critical for
     multi-agent systems, as it helps routers and other agents understand when to delegate tasks to this specific agent.
   * Minimal instruction: The current instruction is very brief. ADK best practices suggest using Markdown for clarity and being more specific
     about the agent's persona and constraints.
   * Short request_timeout: The timeout is set to 10 seconds. For LLM inferences, especially via serverless endpoints which may have occasional
     cold starts or latency, 30 seconds is generally more reliable.

  2. Azure AI Foundry Integration
   * Model String: The model string azure_ai/phi-4-mini is correctly formatted for LiteLLM to recognize the Azure AI Foundry provider.
   * Environment Variables: The code correctly leverages AZURE_MODEL with a fallback. It assumes AZURE_AI_API_KEY and AZURE_AI_API_BASE are set in
     the environment, which matches the setup-azure-phi.sh automation script.

  3. Code Quality
   * Imports: Clean and follows standard ADK patterns.
   * Encapsulation: Using root_agent as the export in agent.py (and re-exporting in __init__.py) is the standard pattern for ADK's discovery
     mechanism.







  
  
  Summary


The strategy for using the ADK with Azure LLMs was validated with a incremental step by step approach.

First- the basic environment tools were configured. Then the required Python libraries were installed. LiteLLM was used to proxy the Google ADK to a Model hosted on Azure. Finally, some tests were run from the CLI and the Web interface. Gemini CLI was used to validate the approach.

MCP Development with Amazon Bedrock AgentCore

xbill — Thu, 07 May 2026 15:33:42 +0000

Leveraging Gemini CLI and the underlying Gemini LLM to build Model Context Protocol (MCP) AI applications with Python from a local development environment deployed to the Amazon Bedrock AgentCore service on AWS.

Is this yet another Python MCP Demo?

Yes it is.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lms-python-aws$ python --version
Python 3.13.13

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.41.2
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

Install

To check the version of Docker:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ dvm --version
Docker Version Manager version 1.0.3 (e59ba7b)

Amazon Bedrock AgentCore

Amazon Bedrock AgentCore is a fully managed AWS platform designed for building, deploying, and operating AI agents securely at enterprise scale. It removes the “undifferentiated heavy lifting” of infrastructure management, allowing developers to focus on agent logic while providing enterprise-grade security and observability.

Full details are here:

Amazon Bedrock AgentCore - AWS

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

You can version check the tool after installation:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ aws --version
aws-cli/2.34.44 Python/3.14.4 Linux/6.6.99-09128-g14e87a8a9b71 exe/x86_64.debian.12
xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Bedrock Agent Core. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

# Initialize FastMCP server
mcp = FastMCP(
    "hello_world_server", host="0.0.0.0", port=port, stateless_http=True
)

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-bedrock-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ aws login --remote
Browser will not be automatically opened.

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.
xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$

Run the install target on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make install
Installing dependencies...
cd app/hello_world_server && pip install -r requirements.txt

Then Lint:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make lint
Running linting (flake8)...
flake8 app/hello_world_server app/testagent --exclude .venv
xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$

and test:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make test
Running tests (pytest)...
PYTHONPATH=app/hello_world_server:app/testagent pytest app/hello_world_server app/testagent
=============================================================== test session starts ===============================================================
platform linux -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/xbill
configfile: pyproject.toml
plugins: anyio-4.11.0, asyncio-1.3.0, langsmith-0.8.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 3 items                                                                                                                                 

app/hello_world_server/tests/test_main.py ... [100%]

================================================================ 3 passed in 0.39s ================================================================
xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$

Then deploy:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make deploy
Deploying to AgentCore...
agentcore deploy

  AgentCore Deploy

  Project: McpBedrockPythonAws
  Target: us-east-1:106059658660

  [done] Validate project
  [done] Check dependencies
            Build CDK project
            Synthesize CloudFormation
            Check stack status
            Computing diff changes...
            Publish assets
            Deploy to AWS

  Log: agentcore/.cli/logs/deploy/deploy-20260507-105552.log

  Esc back · Ctrl+C quit

Once the everything is deployed:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make status
Showing AgentCore status...
agentcore status
AgentCore Status (target: default, us-east-1)

Gateways
  hello-world-gateway: Deployed (1 target) (mcpbedrockpythonaws-hello-world-gateway-3fielpjqo4)

You can then get the endpoint:

xbill@penguin:~/gemini-cli-aws/mcp-bedrock-python-aws$ make endpoint
Retrieving gateway endpoints...
https://mcpbedrockpythonaws-hello-world-gateway-3fielpjqo4.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

   "mcp-bedrock-remote": {
      "httpUrl": "https://mcpbedrockpythonaws-hello-world-gateway-3fielpjqo4.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp"
    }

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

gemini

/mcp list

🟢 mcp-bedrock-remote - Ready (2 tools)
  Tools:
  - mcp_mcp-bedrock-remote_hello-world-target___greet
  - mcp_mcp-bedrock-remote_x_amz_bedrock_agentcore_search

Then the remote MCP tool in AWS Bedrock can be called:

 > greet Amazon Bedrock Agentcore!

  Topic: Greeting Amazon Bedrock Agentcore using the hello-world-target greet tool.

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ hello-world-target___greet (mcp-bedrock-remote MCP Server) {"param":"Amazon Bedrock Agentcore"} │
│ │
│ Amazon Bedrock Agentcore │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ Hello, Amazon Bedrock Agentcore!

Project Review

Gemini CLI can also summarize the deployment:

✦ I have completed the documentation updates for the entire project.

  Final Summary of Changes:

   1. Root Documentation (GEMINI.md & README.md):
       * New Features: Documented the /health endpoint and the add_numbers local tool.
       * Hybrid Architecture: Explained how the testagent combines discovered MCP tools with local tools.
       * Authentication: Added instructions for save-aws-creds.sh to facilitate Makefile-based deployments.
       * Utility Commands: Documented make endpoint for retrieving the gateway URL.

   2. Server Documentation (app/hello_world_server/README.md):
       * Completely rewrote the previously empty README to include implementation details (FastMCP, Streamable HTTP), tool definitions, and local
         execution instructions.

   3. Agent Documentation (app/testagent/README.md):
       * Updated local development instructions to emphasize the no-venv rule, aligning with the project's managed container approach.

   4. Verification:
       * Ran make test to ensure code integrity (3/3 tests passed).
       * Verified the agentcore.json configuration matches the documented endpoints.

  The project is now fully documented, providing clear guidance for both developers and AI assistants.

  Project Documentation Complete: Final documentation review and cleanup completed.

and a code review:

✦ Based on my review of the codebase, project configuration, and infrastructure code, here is a project review with best practices and recommended
  improvements for your Model Context Protocol (MCP) ecosystem on Amazon Bedrock AgentCore.

  1. Architecture & Design Patterns
   * Gateway-Target Decoupling: You have correctly implemented the "Gateway" pattern using AgentCore. This decouples your Strands-based agent from
     the Python-based tool provider, allowing them to scale and be updated independently.
   * Stateless Tool Runtimes: Your use of FastMCP(..., stateless_http=True) and transport="streamable-http" in server.py is perfectly aligned with
     the requirements for managed runtimes in Amazon Bedrock AgentCore.
   * Dynamic Tool Discovery: The implementation in app/testagent/mcp_client/client.py using injected environment variables for gateway URLs is the
     idiomatic way to handle discovery within the AgentCore ecosystem.

  2. Code Quality & Implementation
   * Structured Logging: Using python-json-logger is excellent. It ensures that logs from your managed runtimes are easily searchable in
     CloudWatch.
       * Improvement: In server.py, transition from configuring the root logger to using a named logger: logger = logging.getLogger( __name__ ). This
         avoids conflicts with library loggers.
   * Robust Streaming in Agents: In app/testagent/agent.py, the invoke entrypoint currently only yields event["data"] if it is a string.
       * Best Practice: Update this to handle other event types (like tool calls or usage metadata) or use a more robust streaming utility from the
         strands SDK to ensure the agent's full reasoning process is captured or correctly filtered.

  3. Infrastructure & Security (CDK)
   * IAM Least Privilege: Your CDK code in agentcore/cdk/lib/cdk-stack.ts uses a wildcard for invocation permissions: resources:
     [arn:aws:bedrock-agentcore:${this.region}:${this.account}:runtime/*].
       * Best Practice: For production, narrow this scope to only the specific runtime ARNs that the Gateway needs to access.
   * Auth Patches: The patch in your CDK to enable GATEWAY_IAM_ROLE for SigV4 authentication is a sophisticated and correct way to secure the
     "internal" communication between the Gateway and your MCP Server.

  4. Dependency Management & DX
   * Unified Tooling: You have a uv.lock and pyproject.toml files, which are modern and efficient. However, your Makefile still uses pip install -r
     requirements.txt.
       * Recommendation: Transition the Makefile and installation workflows fully to uv (e.g., uv sync or uv pip install .). This ensures
         deterministic builds and faster setup.
   * Testing Strategy:
       * Mocking: Your current agent tests are limited because they require a live model. Use pytest-mock to mock the BedrockModel and MCPClient
         responses. This allows your CI/CD pipeline to verify agent logic without AWS credentials.
       * Protocol Testing: In app/hello_world_server/tests/test_server.py, add a test that sends a raw JSON-RPC payload to the / endpoint to verify
         the MCP protocol layer, ensuring your server handles the protocol correctly, not just the tool logic.

  5. Observability
   * OTEL Integration: You have enableOtel: true in agentcore.json. This is a top-tier best practice. Ensure you are viewing these traces in AWS
     X-Ray or Amazon CloudWatch ServiceLens to identify latency bottlenecks in your tool calls.

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon Bedrock AgentCore was validated with a incremental step by step approach.

A minimal HTTP transport MCP Server was started from Python source code and validated with Gemini CLI running as a MCP client in the same local environment.

This MCP server was then deployed to Amazon Bedrock AgentCore. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

MCP Development with Amazon Lambda Managed Instances

xbill — Thu, 07 May 2026 13:08:51 +0000

Leveraging Gemini CLI and the underlying Gemini LLM to build Model Context Protocol (MCP) AI applications with Python from a local development environment deployed to the Lambda Managed Instance (LMI) service on AWS.

Aren’t There a Gazillion Python MCP Demos?

Yes there are.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ python --version
Python 3.13.13

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

gemini

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lambda-python-aws$ gemini

▝▜▄ Gemini CLI v0.33.1
    ▝▜▄
   ▗▟▀ Logged in with Google /auth
  ▝▀ Gemini Code Assist Standard /upgrade no sandbox (see /docs) /model Auto (Gemini 3) | 239.8 MB

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

Install

To check the version of Docker:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ dvm --version
Docker Version Manager version 1.0.3 (e59ba7b)

Amazon Lamba Managed Instances

Lambda Managed Instances (LMI) allow running AWS Lambda functions on dedicated EC2 instances, blending serverless simplicity with EC2 performance flexibility. They feature fully managed infrastructure (patching, scaling) with higher throughput, no cold starts, and cost-efficient scaling for predictable, high-volume, memory-intensive, or specialized hardware workloads

Full details are here:

Lambda Managed Instances

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

You can version check the tool after installation:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ aws --version
aws-cli/2.34.43 Python/3.14.4 Linux/6.6.99-09128-g14e87a8a9b71 exe/x86_64.debian.12
xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Lambda Instances. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

mcp.run(
        transport="http",
        host="0.0.0.0",
        port=port,
    )

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-lambdami-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ aws login --remote
Browser will not be automatically opened.

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.
xbill@penguin:~/gemini-cli-aws/mcp-lmi-python-aws$

Run the deploy version on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.
xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make deploy
Step 0: Running Tests...
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Running tests...
============================================================== test session starts ===============================================================
platform linux -- Python 3.13.13, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/xbill
configfile: pyproject.toml
plugins: anyio-4.11.0, asyncio-1.3.0, langsmith-0.8.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 3 items                                                                                                                                

tests/test_main.py ... [100%]

=============================================================== 3 passed in 0.74s ================================================================
make[1]: Leaving directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Step 1: Building and Pushing Docker Image to ECR...
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Checking if ECR repository exists...
Logging in to Amazon ECR...

You can validate the final result by checking the messages:

  Step 7: Publishing to LATEST.PUBLISHED for Managed Instances support...
Step 8: Finalizing API Gateway...
Deployment complete!
API Endpoint URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com
MCP HTTP URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/mcp
Health Check URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/health

Once the everything is deployed:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make status
Function Status (mcp-lambda-python-aws):
---------------------------------------------------------------------------------
| GetFunction |
+------------------------------+-------------------------+----------------------+
| LastModified | Name | Status |
+------------------------------+-------------------------+----------------------+
| 2026-05-06T16:33:26.000+0000| mcp-lambda-python-aws | ActiveNonInvocable |
+------------------------------+-------------------------+----------------------+
Prod Alias Status:
---------------------
| GetAlias |
+-------+-----------+
| Name | Version |
+-------+-----------+
| prod | 28 |
+-------+-----------+

You can then get the endpoint:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make endpoint
https://3x10n3uguc.execute-api.us-east-1.amazonaws.com

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

{
  "mcpServers": {
    "mcp-lambdami-python-aws": {
      "httpUrl": "https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/mcp/"
    }
  }
}

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.41.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

/mcp list

🟢 mcp-lambdami-python-aws - Ready (1 tool)
  Tools:
  - mcp_mcp-lambdami-python-aws_greet

Then the remote MCP tool in Lambda Managed Instances can be called:

> greet Lambda Managed Instances!

  Researching MCP Server Implementation: Reading main.py to understand the greet tool implementation.

  ✓ ReadFile main.py
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greet (mcp-lambdami-python-aws MCP Server) {"param":"Lambda Managed Instances"} │
│ │
│ Hello, Lambda Managed Instances! │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ Hello, Lambda Managed Instances!

Gemini CLI can also summarize the deployment:

✦ The Lambda Managed Instances (LMI) deployment in this project is a hybrid setup that combines CloudFormation for infrastructure and AWS CLI for
  specialized Lambda configurations. It is designed to provide high-performance, dedicated compute for the MCP server with zero cold starts and
  multi-concurrency.

  1. Core Architecture
   * Compute: Lambda functions run on dedicated EC2 capacity managed by AWS but living in your VPC.
   * Networking: A VPC with Public and Private subnets. The Lambda resides in the Private subnet and uses a NAT Gateway for outbound internet
     access (required for external tool calls).
   * Inbound: API Gateway (HTTP API) acts as the entry point, routing traffic to a Lambda Alias (prod).

  2. Infrastructure (CloudFormation)
  The template.yaml defines the foundational components:
   * VPC & NAT Gateway: Essential for providing the dedicated instances with a secure environment and internet connectivity.
   * IAM Roles:
       * Execution Role: Standard Lambda permissions + VPC access.
       * Operator Role: Allows the Lambda service to manage EC2 resources (Scaling, Patching) on your behalf via the
         AWSLambdaManagedEC2ResourceOperator policy.
   * API Gateway: Configured with a $default route pointing to the prod alias of the Lambda.

  3. Managed Instances Configuration (deploy.sh)
  Because LMI features are often newer or require specific CLI sequences, deploy.sh handles the following:
   * Capacity Provider: A named resource (mcp-lambda-python-aws-cp) that defines:
       * Instance Requirements: Architectures (x86_64) and Scaling (Max vCPU count).
       * VPC Placement: Subnets and Security Groups.
   * Lambda Function: Created with a capacity-provider-config that links it to the Capacity Provider.
       * Memory/vCPU: Set to 2048MB (2GB) minimum to meet LMI requirements.
       * Package Type: Image (Container-based using the project's Dockerfile).

  4. Deployment Workflow
   1. Build & Push: Docker image is built and pushed to Amazon ECR.
   2. Infra Deploy: CloudFormation creates the VPC, IAM roles, and API Gateway.
   3. CP Setup: The script ensures the Capacity Provider exists and is configured.
   4. Function Update/Create: The Lambda function is deployed or updated with the LMI configuration.
   5. Versioning:
       * A new version is published.
       * The prod alias is updated to point to this version.
       * LATEST_PUBLISHED support: The function is published to the special LATEST_PUBLISHED state, which is a requirement for LMI to manage
         instances effectively.

  5. Key Operational Characteristics
   * Multi-concurrency: Unlike standard Lambda, LMI handles multiple concurrent requests per instance (default 8), making it ideal for the
     persistent connections used in MCP SSE.
   * Stateless HTTP: The server uses stateless_http=True in FastMCP, ensuring compatibility with the Lambda lifecycle.
   * Warm-up Period: Functions may show as ActiveNonInvocable initially while the Managed Instances are being provisioned and warmed up.

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon Lambda Managed Instances was validated with a incremental step by step approach.

A minimal HTTP transport MCP Server was started from Python source code and validated with Gemini CLI running as a MCP client in the same local environment.

This MCP server was then wrapped in a Docker container and deployed to Amazon Lambda Instances. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

MCP Development with Amazon Lambda Managed Instances

xbill — Wed, 06 May 2026 16:52:50 +0000

Aren’t There a Gazillion Python MCP Demos?

Yes there are.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ python --version
Python 3.13.13

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

gemini

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lambda-python-aws$ gemini

▝▜▄ Gemini CLI v0.33.1
    ▝▜▄
   ▗▟▀ Logged in with Google /auth
  ▝▀ Gemini Code Assist Standard /upgrade no sandbox (see /docs) /model Auto (Gemini 3) | 239.8 MB

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

Install

To check the version of Docker:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ dvm --version
Docker Version Manager version 1.0.3 (e59ba7b)

Amazon Lamba Managed Instances

Full details are here:

Lambda Managed Instances

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

You can version check the tool after installation:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ aws --version
aws-cli/2.34.43 Python/3.14.4 Linux/6.6.99-09128-g14e87a8a9b71 exe/x86_64.debian.12
xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Lambda Instances. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

mcp.run(
        transport="http",
        host="0.0.0.0",
        port=port,
    )

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-lambdami-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ aws login --remote
Browser will not be automatically opened.

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.
xbill@penguin:~/gemini-cli-aws/mcp-lmi-python-aws$

Run the deploy version on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.
xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make deploy
Step 0: Running Tests...
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Running tests...
============================================================== test session starts ===============================================================
platform linux -- Python 3.13.13, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/xbill
configfile: pyproject.toml
plugins: anyio-4.11.0, asyncio-1.3.0, langsmith-0.8.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 3 items                                                                                                                                

tests/test_main.py ... [100%]

=============================================================== 3 passed in 0.74s ================================================================
make[1]: Leaving directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Step 1: Building and Pushing Docker Image to ECR...
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lambdami-python-aws'
Checking if ECR repository exists...
Logging in to Amazon ECR...

You can validate the final result by checking the messages:

  Step 7: Publishing to LATEST.PUBLISHED for Managed Instances support...
Step 8: Finalizing API Gateway...
Deployment complete!
API Endpoint URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com
MCP HTTP URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/mcp
Health Check URL: https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/health

Once the everything is deployed:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make status
Function Status (mcp-lambda-python-aws):
---------------------------------------------------------------------------------
| GetFunction |
+------------------------------+-------------------------+----------------------+
| LastModified | Name | Status |
+------------------------------+-------------------------+----------------------+
| 2026-05-06T16:33:26.000+0000| mcp-lambda-python-aws | ActiveNonInvocable |
+------------------------------+-------------------------+----------------------+
Prod Alias Status:
---------------------
| GetAlias |
+-------+-----------+
| Name | Version |
+-------+-----------+
| prod | 28 |
+-------+-----------+

You can then get the endpoint:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ make endpoint
https://3x10n3uguc.execute-api.us-east-1.amazonaws.com

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

{
  "mcpServers": {
    "mcp-lambdami-python-aws": {
      "httpUrl": "https://3x10n3uguc.execute-api.us-east-1.amazonaws.com/mcp/"
    }
  }
}

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

xbill@penguin:~/gemini-cli-aws/mcp-lambdami-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.41.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

/mcp list

🟢 mcp-lambdami-python-aws - Ready (1 tool)
  Tools:
  - mcp_mcp-lambdami-python-aws_greet

Then the remote MCP tool in Lambda Managed Instances can be called:

> greet Lambda Managed Instances!

  Researching MCP Server Implementation: Reading main.py to understand the greet tool implementation.

  ✓ ReadFile main.py
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greet (mcp-lambdami-python-aws MCP Server) {"param":"Lambda Managed Instances"} │
│ │
│ Hello, Lambda Managed Instances! │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ Hello, Lambda Managed Instances!

Gemini CLI can also summarize the deployment:

✦ The Lambda Managed Instances (LMI) deployment in this project is a hybrid setup that combines CloudFormation for infrastructure and AWS CLI for
  specialized Lambda configurations. It is designed to provide high-performance, dedicated compute for the MCP server with zero cold starts and
  multi-concurrency.

  1. Core Architecture
   * Compute: Lambda functions run on dedicated EC2 capacity managed by AWS but living in your VPC.
   * Networking: A VPC with Public and Private subnets. The Lambda resides in the Private subnet and uses a NAT Gateway for outbound internet
     access (required for external tool calls).
   * Inbound: API Gateway (HTTP API) acts as the entry point, routing traffic to a Lambda Alias (prod).

  2. Infrastructure (CloudFormation)
  The template.yaml defines the foundational components:
   * VPC & NAT Gateway: Essential for providing the dedicated instances with a secure environment and internet connectivity.
   * IAM Roles:
       * Execution Role: Standard Lambda permissions + VPC access.
       * Operator Role: Allows the Lambda service to manage EC2 resources (Scaling, Patching) on your behalf via the
         AWSLambdaManagedEC2ResourceOperator policy.
   * API Gateway: Configured with a $default route pointing to the prod alias of the Lambda.

  3. Managed Instances Configuration (deploy.sh)
  Because LMI features are often newer or require specific CLI sequences, deploy.sh handles the following:
   * Capacity Provider: A named resource (mcp-lambda-python-aws-cp) that defines:
       * Instance Requirements: Architectures (x86_64) and Scaling (Max vCPU count).
       * VPC Placement: Subnets and Security Groups.
   * Lambda Function: Created with a capacity-provider-config that links it to the Capacity Provider.
       * Memory/vCPU: Set to 2048MB (2GB) minimum to meet LMI requirements.
       * Package Type: Image (Container-based using the project's Dockerfile).

  4. Deployment Workflow
   1. Build & Push: Docker image is built and pushed to Amazon ECR.
   2. Infra Deploy: CloudFormation creates the VPC, IAM roles, and API Gateway.
   3. CP Setup: The script ensures the Capacity Provider exists and is configured.
   4. Function Update/Create: The Lambda function is deployed or updated with the LMI configuration.
   5. Versioning:
       * A new version is published.
       * The prod alias is updated to point to this version.
       * LATEST_PUBLISHED support: The function is published to the special LATEST_PUBLISHED state, which is a requirement for LMI to manage
         instances effectively.

  5. Key Operational Characteristics
   * Multi-concurrency: Unlike standard Lambda, LMI handles multiple concurrent requests per instance (default 8), making it ideal for the
     persistent connections used in MCP SSE.
   * Stateless HTTP: The server uses stateless_http=True in FastMCP, ensuring compatibility with the Lambda lifecycle.
   * Warm-up Period: Functions may show as ActiveNonInvocable initially while the Managed Instances are being provisioned and warmed up.

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon Lambda Managed Instances was validated with a incremental step by step approach.

A minimal HTTP transport MCP Server was started from Python source code and validated with Gemini CLI running as a MCP client in the same local environment.

This MCP server was then wrapped in a Docker container and deployed to Amazon Lambda Instances. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

Building ADK Agents with Phi Models on Azure Foundry

xbill — Wed, 06 May 2026 14:32:51 +0000

This article provides a step by step deployment guide for using Azure foundry models with ADK Agents. The small Phi-4 model hosted on Azure Foundry is configured with LiteLLM for use from the ADK Agent.

What is this project trying to Do?

This project aims to configure an ADK agent to use an Azure Foundry model. The ADK provides agent support in standard Python code.

Lite LLM

More info is here:

Azure Foundry

More details are available here:

https://azure.microsoft.com/en-us/products/ai-foundry

Why would I want Gemini CLI with Azure? Isn’t that a Google Thing?

Azure CLI

More details are available here:

What is the Azure Developer CLI?

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~/aisprintapr2026/tpu-vllm-devops-agent$ gemini

▝▜▄ Gemini CLI v0.39.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

Agent Development Kit

The ADK can be installed from here:

Agent Development Kit (ADK)

Agent Skills

Gemini CLI can be customized to work with ADK agents. Both an Agent Development MCP server, and specific Agent skills are available.

More details are here:

Agent Development Kit (ADK)

The ADK documentation is available as a MCP server:

> /mcp list
Configured MCP servers:
🟢 adk-docs-mcp (from adk-docs-ext) - Ready (2 tools)
  Tools:
  - mcp_adk-docs-mcp_fetch_docs
  - mcp_adk-docs-mcp_list_doc_sources

Where do I start?

The strategy for starting multi agent development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, and ADK agent is built, debugged, and tested locally.

Setup the Basic Environment

At this point you should have a working Python environment and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-azure

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd adk-litellm
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Azure Setup

The Azure CLI provides the basic tools for working with the cloud based services:

Check the installation:

xbill@penguin:~/gemini-cli-azure$ az --version
azure-cli 2.86.0

core 2.86.0
telemetry 1.1.0

Dependencies:
msal 1.35.1
azure-mgmt-resource 24.0.0

Python location '/opt/az/bin/python3'
Config directory '/home/xbill/.azure'
Extensions directory '/home/xbill/.azure/cliextensions'

Python (Linux) 3.13.13 (main, May 1 2026, 00:24:43) [GCC 12.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Once the tools are installed — login to the console:

az login

Then run the Azure setup scripts:

./setup-azure-phi.sh

./get-azure-phi-creds.sh

The Azure console should look something like this:

Running the Python Code

First- switch the directory with the Python version of the MCP sample code. Run the release version on the local system:

cd adk-litellm$ make install
Processing ./.

Verify The ADK Installation

First- verify the ADK is installed:

xbill@penguin:~/gemini-cli-aws/adk-litellm$ adk --version
adk, version 1.32.0

xbill@penguin:~/gemini-cli-azure/adk-litellm-azure$ adk run agents/azure
Log setup complete: /tmp/agents_log/agent.20260505_133501.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/cli.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
Running agent azure_agent, type exit to exit.

13:35:04 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

13:35:14 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: Hello! How can I assist you today?
[user]: what are your model details
13:35:23 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: Hello! I am Phi, developed by Microsoft, and I'm here to help you with information, answer questions, and assist with a wide range of topics. How may I be of service to you?



  
  
  Test The ADK Web Interface


This step tests the ADK agent interactions with a browser:



xbill@penguin:~/gemini-cli-azure/adk-litellm-azure/agents$ adk web --host 0.0.0.0
2026-05-05 13:39:05,375 - INFO - service_factory.py:266 - Using in-memory memory service
2026-05-05 13:39:05,375 - INFO - local_storage.py:84 - Using per-agent session storage rooted at /home/xbill/gemini-cli-azure/adk-litellm-azure/agents
2026-05-05 13:39:05,376 - INFO - local_storage.py:110 - Using file artifact service at /home/xbill/gemini-cli-azure/adk-litellm-azure/agents/.adk/artifacts
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/fast_api.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
INFO: Started server process [15609]
INFO: Waiting for application startup.

+-----------------------------------------------------------------------------+
| ADK Web Server started |
| |
| For local testing, access at http://0.0.0.0:8000. |
+-----------------------------------------------------------------------------+






Then use the web interface — either on the local interface 127.0.0.1 or the catch-all web interface 0.0.0.0 -depending on your environment:



Special note for Google Cloud Shell Deployments- add a CORS allow_origins configuration exemption to allow the ADK agent to run:




adk web --host 0.0.0.0 --allow_origins 'regex:.*'







  
  
  Phi Models on Azure


Microsoft Phi models are a family of highly capable, cost-effective Small Language Models (SLMs) designed for low-latency generative AI, available on Azure AI Foundry. They support text, image, and audio inputs (Phi-4-multimodal) and are ideal for memory-constrained environments. Key models include Phi-4, Phi-3.5-MoE, and Phi-3-mini/medium, optimized for high-performance reasoning tasks.

More details are here:




  
  
  Deep Dive with a Phi Model


Once the connection has been tested. You can drill into the model:




xbill@penguin:~/gemini-cli-azure/adk-litellm-azure$ adk run agents/azure
Log setup complete: /tmp/agents_log/agent.20260505_134754.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/cli.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
Running agent azure_agent, type exit to exit.
[user]: what model are you
13:48:00 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: I am Phi, an AI developed by Microsoft, designed to help you with a wide range of queries and tasks. How can I assist you today?



and deep model details:



10:29:49 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= phi-4-mini; provider = azure_ai
[azure_agent]: My model knowledge was last updated in October 2023. Please note that I can't access or retrieve real-time data, so my information may not include events or developments that occurred after this date. For the most current and accurate information, I recommend referring to the latest sources.



  
  
  One Last Code Review with Gemini CLI


Gemini CLI was used to summarize the project:



> code review adk agent for phi model                                                                                                            

✦ I've reviewed the Azure agent implementation in agents/azure/agent.py. Overall, the implementation correctly utilizes the ADK (Agent Development
  Kit) and integrates with Azure AI Foundry via LiteLLM.

  🔍 Code Review Findings

  1. ADK Best Practices
   * Missing description: While the name is correctly defined, the description parameter is missing. In ADK, the description is critical for
     multi-agent systems, as it helps routers and other agents understand when to delegate tasks to this specific agent.
   * Minimal instruction: The current instruction is very brief. ADK best practices suggest using Markdown for clarity and being more specific
     about the agent's persona and constraints.
   * Short request_timeout: The timeout is set to 10 seconds. For LLM inferences, especially via serverless endpoints which may have occasional
     cold starts or latency, 30 seconds is generally more reliable.

  2. Azure AI Foundry Integration
   * Model String: The model string azure_ai/phi-4-mini is correctly formatted for LiteLLM to recognize the Azure AI Foundry provider.
   * Environment Variables: The code correctly leverages AZURE_MODEL with a fallback. It assumes AZURE_AI_API_KEY and AZURE_AI_API_BASE are set in
     the environment, which matches the setup-azure-phi.sh automation script.

  3. Code Quality
   * Imports: Clean and follows standard ADK patterns.
   * Encapsulation: Using root_agent as the export in agent.py (and re-exporting in __init__.py) is the standard pattern for ADK's discovery
     mechanism.








  
  
  Summary


The strategy for using the ADK with Azure LLMs was validated with a incremental step by step approach.

First- the basic environment tools were configured. Then the required Python libraries were installed. LiteLLM was used to proxy the Google ADK to a Model hosted on Azure. Finally, some tests were run from the CLI and the Web interface. Gemini CLI was used to validate the approach.

MCP Development with Amazon Lightsail Instances

xbill — Tue, 05 May 2026 19:36:06 +0000

Leveraging Gemini CLI and the underlying Gemini LLM to build Model Context Protocol (MCP) AI applications with Python from a local development environment deployed to the Lightsail container service on AWS.

Yet another Python MCP Demo?

Yes — thanks for asking.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lightsail-python-aws$ python --version
Python 3.13.12

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~$ gemini

 ▝▜▄ Gemini CLI v0.40.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

Install

Amazon Lightsail

Amazon Lightsail provides a hosted container server. More information is available on the official site here:

Amazon's Simple Cloud Server | Amazon Lightsail

And this is the direct URL to the console:

https://lightsail.aws.amazon.com/ls/webapp/home/containers

Amazon Lightsail also provides managed Instances:

Virtual private server instances in Lightsail

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

Lightsail Control Plugin

The Lightsail Plugin allows the AWS CLI tools to interact with Lightsail. The full GitHub repo is available here:

GitHub - aws/lightsailctl: Amazon Lightsail CLI Extensions

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Lightsail Instances. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

mcp.run(
        transport="http",
        host="0.0.0.0",
        port=port,
    )

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-lightmi-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ aws login --remote

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.

Run the deploy version on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ make deploy
Creating Lightsail instance mcp-vps-python-aws...
Instance already exists or creation in progress.
Waiting for instance mcp-vps-python-aws to reach 'running' state...
Instance is running.
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lightmi-python-aws' 0.0s

You can validate the final result by checking the messages:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ make status
Checking AWS Lightsail container service status for mcp-lightsail-python-aws...
No container service found.
Checking AWS Lightsail instance status for mcp-vps-python-aws...
---------------------------------------------------
| GetInstance |
+---------------------+----------------+----------+
| Name | PublicIp | State |
+---------------------+----------------+----------+
| mcp-vps-python-aws | 13.218.126.2 | running |
+---------------------+----------------+----------+

You can then get the endpoint:

  Running make endpoint now returns the functional URL:
  http://13.218.126.2:8080

 > make endpoint

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ Shell make endpoint │
│ │
│ http://13.218.126.2:8080 │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ http://13.218.126.2:8080

The service will be visible in the AWS console:

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

{
  "mcpServers": {
    "aws-lightsailmi-python": {
      "httpUrl": "http://13.218.126.2:8080/mcp"
    }
  }
}

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.40.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

🟢 aws-lightsailmi-python - Ready (1 tool)
  Tools:
  - mcp_aws-lightsailmi-python_greet

Then the remote MCP tool in Lightsail can be called:

 > greet hello lightsail instances

  Greeting Lightsail Instances: Greeting the Lightsail instances using the greet tool.

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greet (aws-lightsailmi-python MCP Server) {"param":"hello lightsail instances"} │
│ │
│ hello lightsail instances │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The greet tool returned: "hello lightsail instances"

Gemini CLI can also summarize the deployment:

> summarize the deployment

✦ The project is currently deployed to an Amazon Lightsail Managed Instance (VPS).

  Current Status
   * Instance Name: mcp-vps-python-aws
   * Public IP: 13.218.126.2
   * State: Running
   * Container Service: None active (Container deployment is supported but not currently in use).

  Deployment Architecture
   * Host OS: Amazon Linux 2023.
   * Runtime: Python MCP server running as a systemd service (mcp-server).
   * Port: 8080 (HTTP).
   * Automation: Orchestrated via Makefile (e.g., make instance-deploy, make instance-sync).

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon Lightsail Instances was validated with a incremental step by step approach.

This MCP server was then deployed to Amazon Lightsail Instances. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

MCP Development with Amazon Lightsail Instances

xbill — Tue, 05 May 2026 16:38:08 +0000

Yet another Python MCP Demo?

Yes — thanks for asking.

What Is Python?

Python is an interpreted language that allows for rapid development and testing and has deep libraries for working with ML and AI:

Welcome to Python.org

Python Version Management

One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.

The pyenv tool enables deploying consistent versions of Python:

GitHub - pyenv/pyenv: Simple Python version management

As of writing — the mainstream python version is 3.13. To validate your current Python:

admin@ip-172-31-70-211:~/gemini-cli-aws/mcp-lightsail-python-aws$ python --version
Python 3.13.12

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~$ gemini

 ▝▜▄ Gemini CLI v0.40.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

The most common MCP Python deployment path uses the FASTMCP library:

Welcome to FastMCP - FastMCP

Docker Version Management

Install

Amazon Lightsail

Amazon Lightsail provides a hosted container server. More information is available on the official site here:

Amazon's Simple Cloud Server | Amazon Lightsail

And this is the direct URL to the console:

https://lightsail.aws.amazon.com/ls/webapp/home/containers

Amazon Lightsail also provides managed Instances:

Virtual private server instances in Lightsail

AWS CLI

The AWS CLI provides a command line tool to directly access AWS services from your current environment. Full details on the CLI are available here:

Install Docker, AWS CLI, and the Lightsail Control plugin for containers

Lightsail Control Plugin

The Lightsail Plugin allows the AWS CLI tools to interact with Lightsail. The full GitHub repo is available here:

GitHub - aws/lightsailctl: Amazon Lightsail CLI Extensions

Where do I start?

The strategy for starting MCP development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, a minimal Hello World Style Python MCP Server is built with HTTP transport. This server is validated with Gemini CLI in the local environment.

This setup validates the connection from Gemini CLI to the local process via MCP. The MCP client (Gemini CLI) and the Python MCP server both run in the same local environment.

Next- the MCP server is wrapped in a container with docker and deployed to Amazon Lightsail Instances. This remote deployment is validated with Gemini CLI running as a MCP client.

Setup the Basic Environment

At this point you should have a working Python interpreter and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd gemini-cli-aws
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

cd gemini-cli-aws
source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

Hello World with HTTP Transport

One of the key features that the standard MCP libraries provide is abstracting various transport methods.

The high level MCP tool implementation is the same no matter what low level transport channel/method that the MCP Client uses to connect to a MCP Server.

The HTTP transport allows the MCP client and server to run in the same environment or distributed over the Internet.

The connection over HTTP will look similar to this:

mcp.run(
        transport="http",
        host="0.0.0.0",
        port=port,
    )

Running the Python Code

First- switch the directory with the Python MCP sample code:

cd ~/gemini-cli-aws/mcp-lightmi-python-aws

Refresh the AWS credentials:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ aws login --remote

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.

Run the deploy version on the local system:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ make deploy
Creating Lightsail instance mcp-vps-python-aws...
Instance already exists or creation in progress.
Waiting for instance mcp-vps-python-aws to reach 'running' state...
Instance is running.
make[1]: Entering directory '/home/xbill/gemini-cli-aws/mcp-lightmi-python-aws' 0.0s

You can validate the final result by checking the messages:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ make status
Checking AWS Lightsail container service status for mcp-lightsail-python-aws...
No container service found.
Checking AWS Lightsail instance status for mcp-vps-python-aws...
---------------------------------------------------
| GetInstance |
+---------------------+----------------+----------+
| Name | PublicIp | State |
+---------------------+----------------+----------+
| mcp-vps-python-aws | 13.218.126.2 | running |
+---------------------+----------------+----------+

You can then get the endpoint:

  Running make endpoint now returns the functional URL:
  http://13.218.126.2:8080

 > make endpoint

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ Shell make endpoint │
│ │
│ http://13.218.126.2:8080 │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ http://13.218.126.2:8080

The service will be visible in the AWS console:

Gemini CLI settings.json

Once you have the deployed endpoint — update the Gemini CLI MCP settings:

{
  "mcpServers": {
    "aws-lightsailmi-python": {
      "httpUrl": "http://13.218.126.2:8080/mcp"
    }
  }
}

Remote MCP Server Testing

Restart Gemini CLI and check for the new MCP tools:

xbill@penguin:~/gemini-cli-aws/mcp-lightmi-python-aws$ gemini

 ▝▜▄ Gemini CLI v0.40.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

🟢 aws-lightsailmi-python - Ready (1 tool)
  Tools:
  - mcp_aws-lightsailmi-python_greet

Then the remote MCP tool in Lightsail can be called:

 > greet hello lightsail instances

  Greeting Lightsail Instances: Greeting the Lightsail instances using the greet tool.

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ greet (aws-lightsailmi-python MCP Server) {"param":"hello lightsail instances"} │
│ │
│ hello lightsail instances │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✦ The greet tool returned: "hello lightsail instances"

Gemini CLI can also summarize the deployment:

> summarize the deployment

✦ The project is currently deployed to an Amazon Lightsail Managed Instance (VPS).

  Current Status
   * Instance Name: mcp-vps-python-aws
   * Public IP: 13.218.126.2
   * State: Running
   * Container Service: None active (Container deployment is supported but not currently in use).

  Deployment Architecture
   * Host OS: Amazon Linux 2023.
   * Runtime: Python MCP server running as a systemd service (mcp-server).
   * Port: 8080 (HTTP).
   * Automation: Orchestrated via Makefile (e.g., make instance-deploy, make instance-sync).

Summary

The strategy for using Python for MCP development with Gemini CLI on Amazon Lightsail Instances was validated with a incremental step by step approach.

This MCP server was then deployed to Amazon Lightsail Instances. The local copy of Gemini CLI was used as a MCP client to validate the connection.

This approach can be extended to more complex deployments using other MCP transports and Cloud based options.

ADK Agents with Amazon Bedrock Models

xbill — Tue, 05 May 2026 13:16:49 +0000

This article provides a step by step deployment guide for using Amazon Bedrock models with ADK Agents.

What is this project trying to Do?

This project aims to configure an ADK agent to use an Amazon Bedrock model.

Lite LLM

More info is here:

Gemini CLI

If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:

npm install -g @google/gemini-cli

Testing the Gemini CLI Environment

Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:

xbill@penguin:~/aisprintapr2026/tpu-vllm-devops-agent$ gemini

 ▝▜▄ Gemini CLI v0.39.1
   ▝▜▄
  ▗▟▀ Signed in with Google /auth
 ▝▀ Plan: Gemini Code Assist Standard /upgrade

Node Version Management

Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:

GitHub - nvm-sh/nvm: Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

Python MCP Documentation

The official GitHub Repo provides samples and documentation for getting started:

GitHub - modelcontextprotocol/python-sdk: The official Python SDK for Model Context Protocol servers and clients

Agent Development Kit

The ADK can be installed from here:

Agent Development Kit (ADK)

Agent Skills

Gemini CLI can be customized to work with ADK agents. Both an Agent Development MCP server, and specific Agent skills are available.

More details are here:

Agent Development Kit (ADK)

To get the Agent Skills in Gemini CLI:

> /skills list

and the ADK documentation:

> /mcp list
Configured MCP servers:
🟢 adk-docs-mcp (from adk-docs-ext) - Ready (2 tools)
  Tools:
  - mcp_adk-docs-mcp_fetch_docs
  - mcp_adk-docs-mcp_list_doc_sources

Where do I start?

The strategy for starting multi agent development is a incremental step by step approach.

First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.

Then, and ADK agent is built, debugged, and tested locally.

Setup the Basic Environment

At this point you should have a working Python environment and a working Gemini CLI installation. The next step is to clone the GitHub samples repository with support scripts:

cd ~
git clone https://github.com/xbill9/gemini-cli-aws

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

cd adk-litellm
source init.sh

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

source set_env.sh

Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.

AWS Setup

The AWS CLI provides the basic tools for working with the AWS services:

AWS CLI

Check the AWS installation:

xbill@penguin:~/gemini-cli-aws/adk-litellm$ aws --version
aws-cli/2.34.41 Python/3.14.4 Linux/6.6.99-09128-g14e87a8a9b71 exe/x86_64.debian.12
xbill@penguin:~/gemini-cli-aws/adk-litellm$

Once the tools are installed — login to the AWS console:

aws login --remote

Then run the sample script to save the credentials:

xbill@penguin:~/gemini-cli-aws/adk-litellm$ source save-aws-creds.sh 
Exporting AWS credentials...
Successfully saved credentials to .aws_creds
The Makefile will now automatically use these for deployments.

Running the Python Code

First- switch the directory with the Python version of the MCP sample code. Run the release version on the local system:

cd adk-litellm$ make install
Processing ./.

Verify The ADK Installation

First- verify the ADK is installed:

xbill@penguin:~/gemini-cli-aws/adk-litellm$ adk --version
adk, version 1.32.0

xbill@penguin:~/gemini-cli-aws/adk-litellm$ adk run agents/bedrock
Log setup complete: /tmp/agents_log/agent.20260504_210146.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/cli.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
Running agent bedrock_agent, type exit to exit.

21:01:50 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= amazon.nova-micro-v1:0; provider = bedrock
[bedrock_agent]: Hello! How can I assist you today? Whether you need help with product recommendations, information about services, or anything else, I'm here to help. Just let me know what you need!

Test The ADK Web Interface

This step tests the ADK agent interactions with a browser:

xbill@penguin:~/gemini-cli-aws/adk-litellm/agents$ adk web --host 0.0.0.0
2026-05-04 21:02:34,837 - INFO - service_factory.py:266 - Using in-memory memory service
2026-05-04 21:02:34,837 - INFO - local_storage.py:84 - Using per-agent session storage rooted at /home/xbill/gemini-cli-aws/adk-litellm/agents
2026-05-04 21:02:34,837 - INFO - local_storage.py:110 - Using file artifact service at /home/xbill/gemini-cli-aws/adk-litellm/agents/.adk/artifacts
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/fast_api.py:204: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  super(). __init__ ()
INFO: Started server process [19403]
INFO: Waiting for application startup.

+-----------------------------------------------------------------------------+
| ADK Web Server started |
| |
| For local testing, access at http://0.0.0.0:8000. |
+-----------------------------------------------------------------------------+

INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Then use the web interface — either on the local interface 127.0.0.1 or the catch-all web interface 0.0.0.0 -depending on your environment:

Special note for Google Cloud Shell Deployments- add a CORS allow_origins configuration exemption to allow the ADK agent to run:

adk web --host 0.0.0.0 --allow_origins 'regex:.*'

Deep Dive with a Bedrock Model

Once the connection has been tested. You can drill into the bedrock model:

[user]: what model are you
21:04:32 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= amazon.nova-micro-v1:0; provider = bedrock
[bedrock_agent]: As an AI model, I am part of the Amazon Bedrock family of AI services. Specifically, I am based on advanced deep learning architectures designed for natural language understanding and generation. While I can't disclose the exact model name due to proprietary reasons, rest assured that I leverage cutting-edge technology to provide you with the best possible assistance and information.

If you need help with anything specific, feel free to ask, and I'll do my best to assist you!



and deep model details:



[user]: what is your knowledge cut-off date
21:05:28 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= amazon.nova-micro-v1:0; provider = bedrock
[bedrock_agent]: My knowledge was last updated in October 2023. This means I can provide information and insights based on events, data, and developments up until that time. If you have any questions or need information within that timeframe, I'll do my best to assist you! If you have any more specific questions or need the latest updates, you might want to check the most recent sources or news.



and Bedrock details:



[user]: what other bedrock models are there
21:06:15 - LiteLLM:INFO: utils.py:4011 - 
LiteLLM completion() model= amazon.nova-micro-v1:0; provider = bedrock
[bedrock_agent]: Amazon Bedrock is a platform that offers access to various foundation models from different vendors. These models can be used for a wide range of applications, including natural language processing, computer vision, and more. As of my last update in October 2023, specific details about all the models available through Amazon Bedrock could vary because the platform is continually evolving with new model integrations.

However, some common types of foundation models you might find on Amazon Bedrock include:

1. **Natural Language Processing (NLP) Models**: These are used for tasks like text generation, translation, summarization, and question-answering.
2. **Computer Vision Models** : These are used for image and video analysis, including object detection, image classification, and optical character recognition (OCR).
3. **General Purpose Foundation Models** : These are versatile models that can be fine-tuned for specific tasks across different domains.

For the most current and detailed information about the specific models available on Amazon Bedrock, I recommend visiting the [official Amazon Bedrock website](https://aws.amazon.com/bedrock/) or checking the AWS documentation and services catalog. These resources will provide the latest updates on available models, their capabilities, and how to integrate them into your applications.



  
  
  One Last Code Review with Gemini CLI


Gemini CLI was used to summarize the project:



✦ I have completed the code review for agent.py. My analysis is based on the latest ADK 1.32.0 standards and LiteLLM best practices.

  Code Review: agent.py

  Summary
  The current implementation is a clean and syntactically correct minimal scaffold for an ADK agent using Amazon Bedrock. It correctly leverages
  the LiteLlm connector and defines a root_agent as required by the ADK runtime.

  ---

  Detailed Feedback

  1. Model Configuration & Fallbacks
   * Positive: Using os.getenv for BEDROCK_MODEL is excellent for 12-factor app compliance.
   * Positive: The fallback bedrock/amazon.nova-micro-v1:0 is a modern and efficient choice.
   * Tip: Ensure AWS_REGION and credentials are set in the execution environment, as LiteLLM relies on these for Bedrock connectivity.

  2. Agent Identity (name and description)
   * Observation: The agent lacks a description field.
   * Recommendation: While optional, a description is highly recommended for multi-agent systems. It allows the ADK runtime and other agents to
     "discover" this agent's purpose.
   * Action: Add description="A general-purpose assistant powered by Amazon Bedrock via LiteLLM.".

  3. Instructions (instruction)
   * Observation: The instruction "You are a helpful assistant powered by Amazon Bedrock." is very generic.
   * Recommendation: A strong ADK instruction should define a persona, goals, and constraints.
   * Risk: Hardcoding "Amazon Bedrock" in the instruction might lead to confusing output if the model is later changed via environment variables to
     another provider (e.g., openai/gpt-4o).
   * Action: Use a more descriptive or provider-agnostic instruction.

  4. Tooling & Extensibility
   * Observation: The agent currently has no tools and a placeholder comment # ... other agent parameters.
   * Recommendation: To make the agent functional beyond simple chat, you should define and register tools (functions). Even if not immediately
     needed, adding an empty tools=[] list makes the intended extension point clear.

  5. Library Versions & Security
   * Note: I verified your environment is running google-adk==1.32.0 and litellm==1.83.14. This is safe and avoids the recent LiteLLM security
     advisory (v1.82.7/8).








  
  
  Summary


The strategy for using the ADK with Amazon Bedrock LLMs was validated with a incremental step by step approach.

First- the basic environment tools were configured. Then the required Python libraries were installed. LiteLLM was used to proxy the Google ADK to a Model hosted on Amazon Bedrock. Finally, some tests were run from the CLI and the Web interface. Gemini CLI was used to validate the approach.