DEV Community

Frank Zhang
Frank Zhang

Posted on Originally published at docs.opshome.run

Why VMware iSCSI Does Not Get Faster with LACP: NFS, MPIO and 10GbE Explained

VMware storage performance depends on the complete I/O path, not only the disks. A fast NAS, NVMe cache, or a large LACP bundle cannot compensate for a storage path that still constrains each iSCSI flow to one physical link.

Topics: VMware vSphere, ESXi, iSCSI, NFS, LACP, MPIO, 10GbE, storage architecture

Key distinction

LACP increases aggregate network capacity across multiple flows. MPIO creates and manages multiple storage paths. They solve different problems, and an LACP bundle does not replace VMware iSCSI multipathing.

1. Storage performance is an end-to-end system

In a VMware vSphere environment, storage performance is not determined only by the media inside the host or NAS. The final result depends on every layer between the virtual machine and the storage backend.

Virtual Machine
      ↓
Storage Protocol
      ↓
Network
      ↓
Storage Path Design
      ↓
NAS/SAN Controller
      ↓
Cache
      ↓
RAID / Storage Media

This article compares three common VMware storage models:

  • Local VMFS storage
  • NFS datastore
  • iSCSI datastore

It also explains how 8×1GbE LACP, 10GbE, iSCSI MPIO, and NVMe SSD cache affect different parts of that path.

2. VMware storage types compared

Local VMFS storage

Local storage uses disks installed directly in the ESXi host.

VM
 ↓
VMDK
 ↓
VMFS
 ↓
Local SSD / NVMe / RAID

Advantages:

  • Lowest latency
  • Highest single-host performance
  • Simple configuration

Limitations:

  • Storage is bound to one ESXi host
  • Shared-storage capability is limited
  • It is not ideal for multi-host HA clusters

NFS datastore

NFS provides file-based storage from a NAS.

ESXi
 ↓
NFS Client
 ↓
Network
 ↓
NAS File System

Advantages:

  • Simple deployment
  • Easy capacity expansion
  • Shared access between ESXi hosts
  • Good compatibility with NAS platforms

NFS is often preferred for homelabs, small VMware clusters, and straightforward shared-storage requirements.

iSCSI datastore

iSCSI provides block storage to ESXi.

ESXi
 ↓
iSCSI Initiator
 ↓
LUN
 ↓
VMFS
 ↓
Virtual Machines

Advantages:

  • Block-level storage
  • VMFS support
  • Mature multipathing capability
  • Enterprise SAN-style architecture

Limitations:

  • More complex configuration
  • Requires a correct path design
Feature Local VMFS NFS iSCSI
Storage type Block File Block
Shared storage No Yes Yes
VMFS Yes No Yes
Configuration Easy Easy Complex
HA / vMotion suitability Limited Good Good
MPIO No Not traditional block MPIO Yes
Enterprise SAN model Low Medium High

3. Why 8×1GbE LACP is not one 8Gbps link

A common assumption is that combining eight 1GbE interfaces creates a single 8Gbps connection. LACP does not work that way. It distributes network flows across the physical members of a bundle.

Flow A → NIC1 → 1Gbps
Flow B → NIC2 → 1Gbps
Flow C → NIC3 → 1Gbps

Total aggregate bandwidth can increase when there are multiple suitable flows, but each member remains a 1GbE interface.

8×1GbE LACP provides multiple 1GbE paths. It does not create one 8GbE interface for a single flow.

The 10GbE advantage

A 10GbE interface provides up to 10Gbps on one physical path. This is materially different from distributing traffic across multiple 1GbE members.

Feature 8×1GbE LACP 10GbE
Aggregate bandwidth High across multiple flows Higher
Single-flow performance Limited by a member link Much stronger
Cabling Complex Simpler
Port usage High Low
Management More complex Easier

For storage vMotion, VM cloning, backup, and large sequential I/O, 10GbE usually provides a better operational experience.

4. What NVMe SSD cache can and cannot improve

Modern NAS platforms often use NVMe SSD cache in front of HDD RAID.

VMware
  ↓
NFS / iSCSI
  ↓
Network
  ↓
NAS Controller
  ↓
NVMe Cache
  ↓
HDD RAID

NVMe cache can improve random IOPS, storage latency, small-block workloads, and concurrency across multiple virtual machines. It is particularly useful for VM operating system disks, databases, logs, and random read/write workloads.

It cannot bypass a network bottleneck. If the NAS can process more than 1GB/s internally but VMware reaches it through 1GbE, actual throughput remains constrained to roughly the capacity of that network path.

5. LACP and MPIO operate at different layers

Technology Primary purpose What it handles
LACP Network link aggregation Ethernet links, network flows, and link redundancy
MPIO Storage path multipathing iSCSI paths, storage I/O distribution, failover, and load balancing

LACP ≠ MPIO.

A working LACP bundle can provide network redundancy and aggregate capacity while the VMware storage stack still sees only one usable iSCSI path.

6. Correct iSCSI MPIO design

An LACP connection between ESXi, the switch, and the NAS does not by itself create multiple VMware storage paths.

ESXi
  |
LACP
  |
Switch
  |
NAS
  |
iSCSI

A multipath design presents independent paths to the storage stack:

vmnic1 → vmk1 → iSCSI Path 1
vmnic2 → vmk2 → iSCSI Path 2
vmnic3 → vmk3 → iSCSI Path 3

Multiple Paths
      ↓
VMware NMP
      ↓
Round Robin

Each path becomes visible to VMware Native Multipathing, allowing the configured path-selection policy to distribute I/O and provide failover.

7. Case study: 8×1GbE LACP iSCSI remained near 1GbE

Initial design

A VMware server and NAS both used multiple 1GbE interfaces in the following design:

ESXi
8×1GbE
  ↓
LACP
  ↓
Switch
  ↓
LACP
  ↓
NAS
  ↓
iSCSI

The expected result was 8Gbps of aggregate storage bandwidth. Actual performance remained close to a single 1GbE link.

Root cause

The problem was not a bad cable, a failed LACP bundle, the NAS, or the disks. The issue was architectural: the iSCSI session was still subject to flow distribution over individual LACP members.

iSCSI Flow
    ↓
LACP Hash
    ↓
One 1GbE Member
    ↓
≈100MB/s

The environment had network aggregation, but it did not have storage multipathing.

8. Recommended architectures

When existing hardware is limited to 1GbE

Separate normal workload traffic from storage traffic, and use independent interfaces for independent iSCSI paths.

Work network:
2×1GbE → LACP

Storage network:
NIC1 → iSCSI Path 1
NIC2 → iSCSI Path 2
NIC3 → iSCSI Path 3
NIC4 → iSCSI Path 4
           ↓
          MPIO

For a new deployment

Use independent 10GbE storage paths, ideally through separate switching paths where the availability design requires it.

10GbE Path A → Switch A → Storage
10GbE Path B → Switch B → Storage

MPIO + independent storage paths

9. Final recommendations

Scenario Recommended design
Single ESXi host Local SSD or NVMe
Simple VMware shared storage NFS + 10GbE
NAS with HDD and VM workloads NVMe cache + 10GbE
iSCSI with existing 1GbE Independent NICs + MPIO
Enterprise iSCSI 10GbE + MPIO
New VMware cluster Dual 10GbE storage paths

Conclusion

LACP improves network aggregation. MPIO improves storage path utilization. They solve different problems.

A faster NAS with NVMe cache cannot overcome a poor network design, and a large number of 1GbE ports does not automatically become one high-speed iSCSI path.

Predictable VMware storage performance requires the storage protocol, network design, independent paths, multipathing policy, and storage backend to be designed as one system.

References

  1. Article basis: VMware vSphere storage architecture and operational troubleshooting notes covering VMFS, NFS, iSCSI, LACP, MPIO, 10GbE, and NVMe cache.
  2. Related platform: VMware vSphere / ESXi.
  3. More articles: Technical Articles.

Top comments (0)