DEV Community: aimldl Account

Setting Up Managed Lustre Storage on Google Cloud (1/3)

aimldl Account — Thu, 12 Nov 2026 02:33:37 +0000

Building Managed Lustre Storage on Google Cloud (1/3)

Overview
Official Documentation Highlights
Hands-On: Running Commands and Troubleshooting

1. Overview

1-1 What Is Google Cloud Managed Lustre?

Google Cloud Managed Lustre is a fully managed parallel file system designed for large-scale high-performance computing (HPC) and AI/ML workloads. Powered by the industry-standard Lustre file system (built on DDN EXAScaler) and delivered as a cloud-native service, it provides petabyte-scale capacity and hundreds of GB/s of throughput out of the box—without the operational burden of managing complex storage infrastructure.

1-2 Google Cloud High-Performance Storage Options and Lustre Selection Criteria

Google Cloud provides several high-performance storage options tailored to different workload characteristics:

Local SSD: Directly attached to the host VM to deliver the lowest latency and highest IOPS. However, the data is ephemeral (temporary) and cannot be shared across multiple VMs.
Managed Lustre: Distributes data across numerous storage servers to provide high-throughput parallel I/O.
Filestore (High Scale/Enterprise): A fully managed Network File System (NFS) ideal for general-purpose file sharing. While its high-performance tiers deliver strong throughput, it faces scalability limits when thousands of clients access the system simultaneously.
Hyperdisk (Extreme Throughput): Next-generation block storage providing high IOPS and throughput, primarily allocated to single VMs with limited shared access capabilities.
Cloud Storage FUSE: Mounts nearly infinitely scalable object storage like a local file system via GCS FUSE. While it offers high throughput, its metadata operations and latency are typically slower than dedicated HPC file systems.

Storage Option	Primary Use Case	Key Performance Indicators
Local SSD	Maximum-performance scratch storage for individual VMs (ephemeral data)	Ultra-low latency, highest IOPS per VM
Managed Lustre	Workloads demanding extreme parallel and sequential I/O (HPC, AI/ML training, media rendering, simulations)	Low latency, hundreds of GB/s throughput
Filestore (High Scale)	Enterprise workloads, GKE persistent volumes, shared file systems	High availability, high IOPS, solid throughput (lower than Lustre)
Persistent Disk (Extreme/Hyperdisk)	Persistent block storage requiring high IOPS	Flexible performance tuning, persistence, reliability
Cloud Storage FUSE	Cost-effective AI/ML data ingestion, checkpoint storage, unlimited-capacity data lakes	Infinite scalability, high sequential read throughput (higher latency for metadata operations and random I/O compared to Lustre)

1-3. When Should You Choose Lustre?

With multiple high-performance storage options available, deciding when to choose Lustre comes down to specific architectural demands. Managed Lustre is the ideal choice when your workload requires:

Extreme Concurrency: Hundreds to thousands of clients (VMs or containers) reading or writing to the exact same file system simultaneously.
Maximum Aggregate Throughput: Tens to hundreds of GB/s of evenly distributed throughput that exceeds the limits of Filestore or individual persistent disks.
Strict POSIX Compliance: Seamless migration of existing HPC applications to the cloud without rewriting file I/O operations.

1.4 Use Case: Vertex AI Training Cluster and Managed Lustre

Vertex AI Training Cluster (VTC) is a product within Vertex AI, Google Cloud's fully managed service, that allows you to provision and operate clusters dedicated to machine learning training. Because it is fully managed, it removes the operational overhead on internal AI infrastructure teams typically needed to deploy and maintain large-scale HPC clusters. You can easily provision clusters using AI accelerator resources, such as GPUs and TPUs. For example, you can deploy and run a GPU cluster with thousands or tens of thousands of data center GPUs without taking on ongoing infrastructure maintenance burdens or their associated operational costs.

Large-scale HPC clusters represent one of the primary use cases requiring high-performance storage. In these environments, Managed Lustre serves as a critical storage tier to drive overall cluster efficiency:

Maximizing GPU/TPU Utilization (Eliminating I/O Bottlenecks): High read throughput ensures training data feeds continuously into GPUs/TPUs, preventing expensive accelerators from sitting idle waiting on data loads.
High-Speed Checkpointing: Large language model (LLM) training on distributed accelerators requires frequent checkpointing to save model states. These checkpoints trigger massive write bursts. Managed Lustre's parallel write capabilities drastically cut checkpoint duration, improving overall training efficiency.
Streamlined Configuration: Specifying Managed Lustre as the storage option when creating a VTC connects the storage directly to the cluster's VPC via Private Service Access, enabling immediate access to high-performance storage without complex manual mounting steps.

For more details on VTC, which launched in Public Preview in Q4 2025, refer to the Vertex AI Training cluster overview. For information on storage options in VTC, see Vertex AI > Storage.

Next: Building Managed Lustre Storage on Google Cloud (2/3)

Setting Up Managed Lustre Storage on Google Cloud

aimldl Account — Thu, 12 Nov 2026 01:04:14 +0000

Building Managed Lustre Storage on Google Cloud

Table of Contents

Overview
Excerpts from Official Documentation
Hands-on: Command Execution and Troubleshooting
Overview

1-1 What Is Google Cloud Managed Lustre?

Google Cloud Managed Lustre is a fully managed parallel file system engineered specifically for massive High Performance Computing (HPC) and AI/ML workloads. By offering the industry-standard Lustre file system (powered by DDN EXAScaler) as a cloud-native service, it delivers petabyte-scale storage and hundreds of GB/s of throughput out of the box—freeing you from the complexities of managing underlying storage infrastructure.

1-2 Google Cloud High-Performance Storage Options and When to Choose Lustre

Google Cloud provides multiple high-performance storage options tailored to the demands of different workloads:

Local SSD: Directly attached to VMs to deliver the lowest latency and highest IOPS, but the data is volatile (ephemeral) and cannot be shared across multiple VMs.

Managed Lustre: Distributes data across numerous storage servers to execute parallel I/O.

Filestore (High Scale/Enterprise): A fully managed Network File System (NFS) ideal for general file sharing. While its high-performance tiers offer solid throughput, it can hit scalability bottlenecks when thousands of clients access data concurrently in parallel.

Hyperdisk (Extreme Throughput): Next-generation block storage providing high IOPS and throughput, but it is primarily attached to a single VM with limited multi-VM sharing options.

Cloud Storage FUSE: Near-infinitely scalable object storage accessed like a file system via GCS FUSE. While it provides high throughput, metadata operations and latency may lag behind dedicated HPC file systems.

Storage Option

Primary Use Cases

Key Performance Metrics

Local SSD

Maximum performance temporary storage for individual VMs (volatile data)

Ultra-low latency,

Highest IOPS per VM

Managed Lustre

HPC, AI/ML training, media rendering, simulation, etc.

Workloads demanding extreme parallel and sequential I/O performance

Low latency,

Hundreds of GB/s throughput

Filestore

(High Scale)

Enterprise workloads, GKE persistent volumes, shared file systems

High availability,

High IOPS and solid throughput

(Lower compared to Lustre)

Persistent Disk

(Extreme/Hyperdisk)

Persistent block storage requiring high IOPS

Flexible performance tuning, persistence, reliability

Cloud Storage FUSE

AI/ML data loading (cost-effective), checkpoint storage,

Data lakes requiring virtually unlimited capacity

Virtually unlimited scalability, high sequential read throughput (though metadata operations and random I/O latency are higher than Lustre)

1-3. When Should You Choose Lustre?

With multiple high-performance storage options available, you might wonder: "When should I choose Lustre?" Managed Lustre is the ideal choice in the following scenarios:

Extreme Concurrency: Hundreds to thousands of clients (VMs, containers) must read or write to the same file system simultaneously.

Maximum Throughput Needs: You require tens to hundreds of GB/s of evenly distributed throughput that exceeds the limits of Filestore or individual disks.

Strict POSIX Compliance: You need to migrate legacy HPC applications directly to the cloud without modifying application code.

1.4 Use Case: Vertex AI Training Cluster and Managed Lustre

Vertex AI Training Cluster (VTC) allows you to provision and manage machine learning training clusters directly inside Google Cloud's fully managed Vertex AI platform. Because it is a fully managed service, it eliminates the operational overhead of requiring a dedicated internal AI infrastructure team to build and maintain large-scale HPC clusters. You can easily spin up and operate HPC clusters powered by AI accelerators such as GPUs and TPUs. For instance, you can run a cluster with thousands or tens of thousands of data center GPUs without shouldering infrastructure maintenance overhead and its associated costs.

Large-scale HPC clusters represent a prime use case for high-performance storage. In these environments, Managed Lustre is strongly recommended as the core storage option to maximize overall cluster efficiency:

Maximizing GPU/TPU Utilization (Eliminating I/O Bottlenecks): To prevent expensive GPUs and TPUs from sitting idle while waiting for data to load, Managed Lustre feeds training data to accelerators at ultra-high read throughput.

High-Speed Checkpointing: Training large language models (LLMs) requiring large-scale AI accelerators (e.g., GPUs) involves frequent checkpointing to save model states. Checkpointing creates massive burst write traffic; Managed Lustre's parallel write performance significantly cuts write times, boosting overall training throughput.

Configuration: When provisioning a VTC, selecting Managed Lustre as the storage option connects it directly to the cluster's internal VPC via Private Service Access, allowing you to use high-performance storage immediately without complex manual mount configurations.

For more information on VTC, which launched in Public Preview in Q4 2025, refer to the Vertex AI Training Cluster Overview; for details on VTC storage options, see Vertex AI > Storage.

Next: Building Managed Lustre Storage on Google Cloud (2/3)