MCP Server Part 2 : SageMaker Studio, Kubernetes manifest, Load Balancer and MCP Client for Fraud Detection

Hello everyone. I want to continue writing about fraud detection MCP server. In this blog tutorial, I want to create MCP server on Amazon Elastic Kubernetes Service (EKS) using ECR fraud detection image that already created before. Amazon EKS is a fully managed Kubernetes service that build, run and scale Kubernetes applications such as generative AI, agentic AI and physical AI.

REQUIREMENTS :

1. MCP Server Part 1 blog tutorial, you can see this link
2. AWS account, you can sign up/sign in here
3. kubectl for apply Kubernetes manifest for vLLM service and MCP server load balancer, you can see this link
4. FastMCP for native MCP client, you can see this link
5. (Optional) Langchain MCP adapter for MCP client, you can see this link

KUBERNETES MANIFEST - vLLM :

After all AWS service that created are available, update EKS Auto Mode cluster by run this shell script.

cd ..

cd manifests

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity \
  --query Account \
  --output text)

aws eks update-kubeconfig --region us-west-2 --name vllm-mcp-server

aws iam list-roles \
  --query "Roles[?contains(RoleName, 'SageMaker')].[RoleName,Arn]" \
  --output table

The above shell script means :

• export AWS_ACCOUNT_ID = specify AWS account ID (do not write hardcoded account ID).

• aws eks update-kubeconfig = configure/authenticate kubectl to connect to the EKS cluster.

• aws iam list-roles = display information about IAM role in AWS account. Example :
  
  

aws eks create-access-entry \
  --cluster-name vllm-mcp-server \
  --principal-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/service-role/AmazonSageMaker-ExecutionRole-xxxxx \
  --type STANDARD \
  --region us-west-2

aws eks associate-access-policy \
  --cluster-name vllm-mcp-server \
  --principal-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/service-role/AmazonSageMaker-ExecutionRole-xxxxx \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
  --access-scope type=cluster \
  --region us-west-2

The above shell script means :

• aws eks create-access-entry = allow IAM principal (principal-arn) to access the EKS cluster.

• aws eks associate-access-policy = associate access policy to access entry.

Run this Kubernetes manifest of this vLLM service.

cd vllm
kubectl apply -f ebs.yaml
kubectl apply -f pvc.yaml
kubectl apply -f nodepool.yaml
kubectl apply -f envsubst < deployment.yaml | kubectl apply -f -
kubectl apply -f service.yaml

The above Kubernetes manifest means:

• ebs.yaml = create Storage Class using Amazon Elastic Block Store (EBS) that automatically provisioned when application request persistent storage. In this use case, EBS is used to keep the Gemma 4 model and Hugging Face cache persistent.

• pvc.yaml = create persistent volume claim associated to storage class that was created.

• nodepool.yaml = create Node Pool using Karpenter to manage compute resources in Kubernetes cluster. Karpenter is open-source Kubernetes cluster autoscaler. In this use case, use EC2 g6.2xlarge GPU instance available after request to AWS Services Quota.

• deployment.yaml = create Kubernetes deployment gemma-4-vllm that deploy Amazon ECR vLLM server image ${AWS_ACCOUNT_ID}.dkr.ecr.us-west-2.amazonaws.com/vllm-gemma-4-eks:0.25.0-eks, port 8000, need 1 NVIDIA GPU, serve Gemma 4 then download Gemma 4 to EBS PVC.

• service.yaml = create Kubernetes service gemma-4-vllm for vLLM server. Check app.py file in mcp-server folder then see this row. This Kubernetes service is wrapped with OpenAI chat completions.
  

VLLM_URL = "http://gemma-4-vllm:8000/v1/chat/completions"

To check vLLM server is running or not, write kubectl get pods and view the vLLM pods until status is Running.

KUBERNETES MANIFEST - MCP SERVER :

Before deploying the Kubernetes manifest for the MCP server, create and configure an EKS Pod Identity for access to Amazon DynamoDB.

cd ..

cd mcp-server

cat > dynamodb-mcp-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:*"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam create-policy \
    --policy-name DynamoDBMCPPolicy \
    --policy-document file://dynamodb-mcp-policy.json
rm -f dynamodb-mcp-policy.json

cat > trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "pods.eks.amazonaws.com"
      },
      "Action": [
        "sts:AssumeRole",
        "sts:TagSession"
      ]
    }
  ]
}
EOF

aws iam create-role \
    --role-name MCPServerPodRole \
    --assume-role-policy-document file://trust-policy.json
rm -f trust-policy.json

The above shell script means :

• aws iam create-policy = create IAM policy for DynamoDB.

• aws iam create-role = create IAM role for MCP Server Pod which EKS pod can use to talk with DynamoDB.
  

aws iam attach-role-policy \
    --role-name MCPServerPodRole \
    --policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/DynamoDBMCPPolicy

aws eks create-pod-identity-association \
    --cluster-name vllm-mcp-server \
    --namespace default \
    --service-account mcp-server-sa \
    --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/MCPServerPodRole

The above shell script means :

• aws iam attach-role-policy = attach IAM policy to IAM role.

• aws eks create-pod-identity-association = create EKS pod identity association between service account in EKS cluster and IAM role with EKS Pod Identity.

Run this Kubernetes manifest of this MCP server ingress/load balancer.

kubectl apply -f serviceaccount.yaml
envsubst < deployment.yaml | kubectl apply -f -
kubectl apply -f service.yaml
kubectl apply -f ingressclassparams.yaml
kubectl apply -f ingressclass.yaml
kubectl apply -f ingress.yaml

The above Kubernetes manifest means :

• serviceaccount.yaml = create Kubernetes Service Account mcp-server-sa to a Pod needs to access AWS service (for this use case, Amazon DynamoDB). The service account must be associated to an AWS IAM role that has permissions to access the AWS service.

• deployment.yaml = create Kubernetes deployment mcp-server-gemma that associate with service account mcp-server-sa that already created, deploy Amazon ECR MCP server image ${AWS_ACCOUNT_ID}.dkr.ecr.us-west-2.amazonaws.com/mcp-server-gemma-4:latest and port 8080.

• service.yaml = create Kubernetes service mcp-server-gemma for MCP server application.

The documentation for Ingressclassparams, ingressclass, and ingress can be found at this link.

• ingressclassparams.yaml = create Application Load Balancer configuration that load balancer scheme as internet-facing.

• ingressclass.yaml = create Application Load Balancer controller that following ingressClassparams parameter.

• ingress.yaml = associate path and port on the Application Load Balancer with workloads in your cluster.

LOAD BALANCER :

To check MCP server is running or not, write kubectl get pods and view the MCP server pods until status is Running.

But if find mcp-server-gemma pod error, write kubectl rollout restart deployment mcp-server-gemma and automatically creating a new pod for mcp-server-gemma.

Then go to EC2 -> Load Balancing -> Load Balancers -> checklist Load Balancer -> Details -> Copy DNS name to MCP client code in mcp-client folder.

MCP CLIENT :

To access the MCP server, you need an MCP client using native MCP client from FastMCP or MCP client from Langchain.

cd ..

cd ..

cd mcp-client

Open mcp-with-langchain.py file in mcp-client folder then look at this line of code.

client = Client("http://ALB_DNS_name/mcp")

Replace ALB_DNS_name to DNS name of load balancer that already created and run the shell script below, which means installing Langchain MCP adapter and running MCP with Langchain client code.

pip install langchain-mcp-adapters

python mcp-with-langchain.py

Open mcp-without-langchain.py file in mcp-client folder look at this line of code.

ALB_URL = "http://ALB_DNS_name/mcp"

Replace ALB_DNS_name to DNS name of load balancer that already created then run MCP without Langchain client code.

python mcp-without-langchain.py

Good! Finally connect MCP client to MCP server is successful.

NOTE : Delete all AWS services such as EKS, VPC, DynamoDB and other services because if all AWS services are not deleted, the cost will be very high (EKS cluster plane and EC2 GPU price is 24/7 or always running) by running this shell script.

cd ..

cd terraform

terraform destroy --auto-approve

terraform destroy is destroy/delete all AWS service resources based all Terraform file. If not write --auto-approve, you must write/enter "yes" every want to apply Terraform file.

NOTE : In SageMaker Studio JupyterLab, click "Stop space" and delete your SageMaker Studio domain.

CONCLUSION :

• Create, update/restart, delete Kubernetes resources with kubectl only without using EKS console.
• To connect MCP client to MCP server can using native MCP client (FastMCP) or MCP client (Langchain MCP adapter).

DOCUMENTATION :

• Amazon Elastic Kubernetes Service (EKS) documentation
• kubectl, FastMCP and Langchain MCP adapter documentation is same as above requirements.

GITHUB REPOSITORY :

https://github.com/budionosanai/google-gemma-eks-vllm-dynamodb-fastmcp-fraud-detection

Thank you,
Budi