DEV Community

Budiono Santoso
Budiono Santoso

Posted on

MCP Server Part 1 : SageMaker Studio, vLLM, Gemma 4 and Terraform for Fraud Detection

#aws #eks #gemma #terraform

Hello everyone. I want to create fraud detection MCP server using Amazon Elastic Kubernetes Service (Amazon EKS). Amazon EKS is a fully managed Kubernetes service that build, run and scale Kubernetes applications such as generative AI, agentic AI and physical AI.

In this blog tutorial, I using SageMaker Studio as my online IDE, Amazon ECR to containerize vLLM server and MCP server, Amazon EKS for vLLM service and MCP server load balancer, Amazon VPC for EKS networking and Amazon DynamoDB for transaction data.

REQUIREMENTS :

  1. AWS account, you can sign up/sign in here
  2. vLLM image for inference and serving, you can see this link
  3. Gemma 4 model for open-source LLM, you can see this link
  4. Terraform AWS provider for Infrastructure as Code, you can see this link
  5. FastMCP for create MCP server, you can see this link

STEP-BY-STEP :

  1. Open Amazon SageMaker AI like this screenshot then click "Set up for a single user" for create SageMaker Studio.
    SageMaker AI

  2. Wait until SageMaker Studio is ready. After SageMaker Studio is ready, click "Open Studio".
    Loading
    Click open studio

  3. This is what SageMaker Studio looks like. Click JupyterLab logo top left corner then click "Create JupyterLab space".
    SageMaker Studio
    JupyterLab space
    Create space

  4. After space is created, click "Run space" then wait until show "Open JupyterLab" is available and status is Running.
    Run space
    Open JupyterLab

  5. Open SageMaker AI (not SageMaker Studio) console then click your Quick setup domain. Click "User profiles" -> click "default-..." then copy execution role for create SageMaker Endpoint step. For app configuration, click "Enable Docker on this domain" to can pull vLLM image and push to Amazon ECR.
    User and app details

  6. Click "Open JupyterLab" that automatically open a new tab. For this use case, Terraform can automatically create and delete AWS services such as EKS, VPC and DynamoDB with just one line of code. In the Launcher, click Terminal to install Terraform and kubectl, among other things.

Install Terraform with run this shell script.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv bash)"

brew install gcc

brew tap hashicorp/tap

brew install hashicorp/tap/terraform

terraform --version
Enter fullscreen mode Exit fullscreen mode

Install kubectl with run this shell script.

curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.35.3/2026-04-08/bin/linux/amd64/kubectl

chmod +x ./kubectl

mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$HOME/bin:$PATH

kubectl version --client
Enter fullscreen mode Exit fullscreen mode

Search "0.25.0-gpu-py312-cu130-ubuntu22.04-ec2" in image tags of vLLM because vLLM 0.25.0 version already support Gemma 4 and use EC2/EKS version. Pull the vLLM image and push it to Amazon ECR with one shell script. In the terminal, run this shell script.

cd scripts
chmod +x vllm-to-ecr.sh
./vllm-to-ecr.sh
Enter fullscreen mode Exit fullscreen mode

Open and check Amazon ECR private repository "vllm-gemma-4-eks".
vLLM Gemma 4 on EKS

This ECR vLLM server image will be used for the next step using Amazon Elastic Kubernetes Service (Auto Mode on EC2) in next blog tutorial.

Open AWS IAM console then click Roles, search your IAM execution role that already copy before and add some policies because SageMaker Studio as a IDE needs to connect to AWS services. But what happens if you don't add some policies? Yes, access to AWS services is denied.
IAM role
Trust relationship

Create MCP server using FastMCP, create app.py file, Dockerfile and requirements.txt file in ONE folder (available on GitHub, "mcp-server" folder) then containerize this MCP server folder using Amazon ECR.

Then create Amazon ECR private repository "mcp-server-gemma-4" with run this shell script.

cd ..

cd mcp-server

aws ecr create-repository \
    --repository-name "mcp-server-gemma-4" \
    --image-scanning-configuration scanOnPush=false \
    --image-tag-mutability MUTABLE \
    --region "us-west-2" 2>/dev/null || echo "Repository already exists, skipping creation."
Enter fullscreen mode Exit fullscreen mode

MCP server image

However, when want to build MCP server and push to Amazon ECR private repository, display error like this screenshot.
Docker build error

To fix this error, use SageMaker Docker Build. SageMaker Docker Build is CLI tool for building Docker images in SageMaker Studio using AWS CodeBuild.

Install SageMaker Docker Build with run this shell script.

pip install sagemaker-studio-image-build
Enter fullscreen mode Exit fullscreen mode

After SageMaker Docker Build is installed, run this shell script to build and push to ECR and wait until finished.

sm-docker build . --repository mcp-server-gemma-4:latest
Enter fullscreen mode Exit fullscreen mode

MCP server latest version

This ECR fraud detection image will be used for next step using Amazon Elastic Kubernetes Service (Auto Mode on EC2) in next blog tutorial.

Then go to AWS Service Quotas to request EC2 G instance quota, then search "Amazon EC2" then click "View quotas".
Service Quota

Write "On-demand G" then click "Request increase at account level".
On-demand G

However, you can view EC2 G6 NVIDIA GPU instance and then look at the vCPU to fill in the number. In my use case, enter "8" in the Increase quota value and click "Request".
Increase number

In my case, my request was automatically "Case Opened" that means you need explain what you need for this GPU instance. So why must request an EC2 GPU instance quota? If don't request an EC2 GPU instance quota, will receive an error during the EC2 GPU instance creation process such as your EC2 GPU instance are 0 and you need to request a Service Quota.

Then create Terraform files such as main.tf, eks.tf, vpc.tf and dynamodb.tf in ONE folder (available on GitHub, "terraform" folder).

  • main.tf for Terraform AWS version, AWS region and Terraform VPC module
  • eks.tf for create EKS Auto Mode cluster, IAM role for EKS node and IAM role for EKS cluster.
  • vpc.tf for create EKS networking using Terraform VPC module that make simple configuration.
  • dynamodb.tf for create DynamoDB table. 
main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "6.45.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "6.6.0"
}
Enter fullscreen mode Exit fullscreen mode 
eks.tf

resource "aws_eks_cluster" "cluster" {
  name     = "vllm-mcp-server"
  role_arn = aws_iam_role.cluster.arn
  version  = "1.35"

  access_config {
    authentication_mode = "API"
  }

  bootstrap_self_managed_addons = false

  compute_config {
    enabled       = true
    node_pools    = ["system"]
    node_role_arn = aws_iam_role.node.arn
  }

  kubernetes_network_config {
    elastic_load_balancing {
      enabled = true
    }
  }

  storage_config {
    block_storage {
      enabled = true
    }
  }

  vpc_config {
    subnet_ids              = module.vpc-eks.private_subnets
    endpoint_private_access = true
    endpoint_public_access  = true
  }

  depends_on = [
    aws_iam_role_policy_attachment.cluster_AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSComputePolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSBlockStoragePolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSLoadBalancingPolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSNetworkingPolicy
  ]
}

resource "aws_iam_role" "node" {
  name = "eks-auto-node-iam"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = ["sts:AssumeRole"]
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "node_AmazonEKSWorkerNodeMinimalPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodeMinimalPolicy"
  role       = aws_iam_role.node.name
}

resource "aws_iam_role_policy_attachment" "node_AmazonEC2ContainerRegistryPullOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPullOnly"
  role       = aws_iam_role.node.name
}

resource "aws_iam_role" "cluster" {
  name = "eks-cluster-iam"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = [
          "sts:AssumeRole",
          "sts:TagSession"
        ]
        Effect = "Allow"
        Principal = {
          Service = "eks.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSComputePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSComputePolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSBlockStoragePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSLoadBalancingPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSLoadBalancingPolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSNetworkingPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSNetworkingPolicy"
  role       = aws_iam_role.cluster.name
}
Enter fullscreen mode Exit fullscreen mode 
vpc.tf

module "vpc-eks" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "6.6.0"

  name = "eks-gemma-vllm-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-west-2a", "us-west-2b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]

  enable_nat_gateway     = true
  single_nat_gateway     = true
  enable_vpn_gateway     = false
  one_nat_gateway_per_az = false

  public_subnet_tags = {
    "kubernetes.io/role/elb" = "1"
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = "1"
  }
}
Enter fullscreen mode Exit fullscreen mode 
dynamodb.tf

resource "aws_dynamodb_table" "transactions_table" {
  name           = "Transactions"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "userId"
  range_key      = "transactionId"

  attribute {
    name = "userId"
    type = "S"
  }

  attribute {
    name = "transactionId"
    type = "S"
  }

  attribute {
    name = "location"
    type = "S"
  }

  attribute {
    name = "timestamp"
    type = "S"
  }

  global_secondary_index {
    name               = "LocationTimestampIndex"
    key_schema {
        attribute_name = "location"
        key_type       = "HASH"
    }
    key_schema {
        attribute_name = "timestamp"
        key_type       = "RANGE"
    }
    write_capacity     = 5
    read_capacity      = 5
    projection_type    = "ALL" 
  }
}
Enter fullscreen mode Exit fullscreen mode

After 4 Terraform files is available, write and run this Terraform script.

terraform init
Enter fullscreen mode Exit fullscreen mode

terraform init is initialize Terraform latest version in main.tf

terraform plan
Enter fullscreen mode Exit fullscreen mode

terraform plan is check make sure your Terraform file is true and right format. If false and wrong format, you must check your Terraform file again.

terraform apply --auto-approve
Enter fullscreen mode Exit fullscreen mode

terraform apply is apply/create all AWS service resources based all Terraform file. If not write --auto-approve, you must write/enter "yes" every want to apply Terraform file.

Wait until all AWS service that created to become available. The result of this Terraform process are shown in screenshot below.

The name of this EKS cluster is "vllm-mcp-server" that means vLLM server is a Kubernetes service and MCP server is a ingress/load balancer within a single Kubernetes cluster. The Kubernetes version is 1.35
EKS cluster

EKS cluster

This EKS cluster show EKS Auto Mode is enabled, also IAM role for cluster and node is available. EKS Auto Mode means AWS manage infrastructure such as compute autoscaling, pod and service networking, application load balancing (in EKS without Auto Mode, must install application load balancing using Helm or eksctl), block storage and etc.
EKS cluster overview

This EKS cluster is connected to VPC networking that already created. To view VPC resources such as subnets, security groups and etc, click the VPC name.
EKS - VPC
VPC

NAT gateway is a Network Address Translation (NAT) service. Instance in private subnet can connect to service outside your VPC but external service can not connect with those instance. In this use case, NAT gateway is used to download Gemma 4 model from HuggingFace to Elastic Block Store (EBS) Persistent Volume Claim (PVC) that will be explained in next blog tutorial.
NAT Gateway

Security group control the traffic is allowed to reach and leave the resources that it is associated with. For example, after you associate a security group with an EC2 instance, it controls the inbound and outbound traffic for the instance. In this use case, this security group is associated with EKS cluster.
Security Group

Make sure DynamoDB "Transactions" table is available.
DynamoDB table

Generate data and upload data to DynamoDB "Transactions" table with write this shell script.


cd ..

cd scripts

python generate-data.py
Enter fullscreen mode Exit fullscreen mode

Data in DynamoDB

I wrote the second part of the MCP server tutorial blog is Kubernetes manifest, Load Balancer and MCP Client are here.

CONCLUSION :

  • SageMaker Studio can be used as a online IDE. If you have a laptop with limited specifications, I recommended this online IDE.
  • vLLM can be used for inference and serving on ECR.
  • Gemma 4 can running in Kubernetes service.
  • Terraform help create automatically EKS, VPC and DynamoDB resources faster and also destroy/delete all resources faster.
  • FastMCP help create MCP server for fraud detection.

DOCUMENTATION :

GITHUB REPOSITORY :

https://github.com/budionosanai/google-gemma-eks-vllm-dynamodb-fastmcp-fraud-detection

Thank you,
Budi

Top comments (0)