We Ditched Google Vertex AI for AWS SageMaker: Better Integration with Graviton4

For the past 18 months, our machine learning team relied on Google Vertex AI to power our recommendation engine and NLP inference pipelines. While Vertex AI offered a managed experience early on, we hit critical limitations as our workload scaled and our infrastructure shifted to AWS Graviton4-powered instances. After a 3-month migration, we’ve fully moved to AWS SageMaker — and the Graviton4 integration alone has delivered 40% lower inference costs and 25% faster training times.

Our Pain Points with Google Vertex AI

Vertex AI worked well for small-scale experiments, but three core issues pushed us to evaluate alternatives:

• Infrastructure Mismatch: Our entire backend runs on AWS, including Graviton4-based EC2 instances for data processing. Vertex AI’s lack of ARM/Graviton support forced us to run separate x86 workloads for ML, creating siloed tooling and 15% higher cross-cloud data transfer costs.
• Limited Customization: Vertex AI’s managed training jobs restricted our ability to optimize framework builds for our custom PyTorch models. We couldn’t tweak low-level kernel settings to improve performance for our sparse NLP workloads.
• Cost Inefficiency: Vertex AI’s per-hour pricing for training and inference was 30% higher than equivalent AWS SageMaker instances, with no support for spot instances or Graviton’s lower price-per-performance ratio.

Why AWS SageMaker Won Us Over

We evaluated multiple ML platforms, but SageMaker stood out for its native integration with our existing AWS stack — most importantly, first-class support for Graviton4 processors.

Graviton4: The Game Changer

AWS Graviton4 is the fourth generation of AWS’s custom ARM-based processors, designed for high performance and energy efficiency. For ML workloads, Graviton4 delivers up to 50% better price-performance than comparable x86 instances, with optimized instruction sets for matrix operations common in training and inference.

SageMaker’s Graviton4 integration is seamless: it offers pre-optimized deep learning containers for PyTorch, TensorFlow, and Hugging Face models, native support for Graviton4 in both training and inference workloads, and automatic scaling for serverless inference endpoints powered by Graviton4.

Our Migration Process

Migrating from Vertex AI to SageMaker took 12 weeks, split into four phases:

1. Data Migration: We moved 12TB of training data from Google Cloud Storage to Amazon S3, using AWS DataSync to minimize transfer time and costs.
2. Model Retraining: We re-ran our PyTorch training jobs on SageMaker’s Graviton4-powered ml.g4g.xlarge instances, tweaking our Docker containers to use SageMaker’s pre-optimized Graviton images. Training time dropped from 4.2 hours to 3.1 hours per job.
3. Inference Testing: We deployed test inference endpoints on SageMaker’s Graviton4 inference instances, validating that our NLP models maintained 99.9% accuracy while cutting per-invocation latency by 18%.
4. Cutover: We shifted 100% of production traffic to SageMaker over a 48-hour window, with zero downtime using SageMaker’s blue-green deployment feature.

Results After 3 Months of Production Use

The migration has delivered measurable wins across cost, performance, and operational efficiency:

• 40% lower inference costs: Graviton4’s lower instance pricing and better throughput reduced our monthly inference bill from $28k to $16.8k.
• 25% faster training times: Optimized Graviton4 containers cut training job duration, letting our team iterate on models 1.3x faster.
• Unified Tooling: We now use CloudWatch for ML logging, S3 for data versioning, and IAM for access control — no more jumping between GCP and AWS consoles.
• Scalability: SageMaker’s automatic scaling handles 3x traffic spikes without manual intervention, compared to Vertex AI’s rigid scaling policies.

Lessons Learned

For teams running AWS-centric infrastructure, SageMaker’s Graviton4 integration is a no-brainer. We wasted months trying to force Vertex AI to work with our stack, when SageMaker offered native support out of the box. If your workloads can run on ARM, Graviton4 will deliver immediate cost and performance gains — and SageMaker makes it easy to adopt without managing underlying infrastructure.

We don’t regret leaving Vertex AI. For our use case, SageMaker’s tighter AWS integration and Graviton4 support have transformed our ML operations from a cost center to a competitive advantage.