Comparing AWS Glue Job's Default Mode and VPC Mode

Original Japanese article: Glue JobのDefaultモードとVPCモードを比較する

Introduction

I'm Aki, an AWS Community Builder (@jitepengin).

When you use AWS Glue, it's pretty common to run Glue Jobs in default mode, without a VPC at all.

If your job only needs to reach AWS-managed services like S3 or the Glue Data Catalog, default mode gives you a simpler setup. On the other hand, if you need a more secure environment, or your requirements call for it, you might reach for VPC mode instead.

I'd always had a vague sense that "VPC mode is probably slower," but I'd never actually measured how much of a difference there is between Default mode and VPC mode.

In this article, I'll use a simple pipeline — reading from S3 (Raw) and writing into S3 Tables — as a test case, and actually run both connection modes to compare them.

Test Environment

I'm reusing the resources from a previous article.

Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables

• Table bucket: penguin-rest-test
• Namespace: analytics
• Table: daily_sales (two columns: sales_date, amount)
• Raw data location: s3://penguin-raw-test/daily_sales/ (CSV)
• Glue Job: PySpark, G.1X worker, a simple job that reads CSV from S3 and writes it into S3 Tables as Iceberg

Since this article focuses on network reachability (Default vs. VPC mode), I'm skipping the authorization layer (IAM/Lake Formation) setup here. I'm assuming the table bucket and namespace are already registered, and that the three-tier Catalog/Database/Table grants are already in place, following the steps from a previous article.

Test Architecture

Default Mode

A very simple setup. The job runs on AWS-managed infrastructure without touching your own VPC, so you don't need to think about subnets or security groups.

Reachable destinations are limited to AWS-provided service endpoints, but for a setup like this one — S3, Glue Data Catalog, and S3 Tables only — that's more than enough.

VPC Mode

The Glue Job creates an ENI inside a private subnet at runtime and reaches each service through a VPC endpoint. A Glue Connection is used to create an ENI (Elastic Network Interface) in the specified subnet, and all traffic goes through it.

You'd choose this mode when you need reachability to VPC-internal resources like RDS or Redshift, or when you want fine-grained traffic control via security groups and NACLs — in short, when you have stricter security requirements.

Access to the Glue Data Catalog is officially documented as going "through a local proxy by default." More precisely, this means the traffic is proxied through AWS's Glue-managed VPC to reach the Glue API — which implies there needs to be a network path between that Glue-managed VPC and your own VPC in the first place. In a private subnet setup with no NAT Gateway or Internet Gateway, that path simply doesn't exist, so the local proxy can't do its job and the connection times out. In that case, you need to create an Interface VPC Endpoint for com.amazonaws.<region>.glue. I actually confirmed this in testing — without a Glue endpoint in place, Iceberg's GlueCatalog implementation timed out.

A Note on ENIs: There Are Two Kinds

One thing worth calling out here is that there are two ENIs with very different characteristics involved:

• The Interface VPC Endpoint's ENI: a persistent resource you provision ahead of time. It isn't recreated on every job run — it's billed continuously by the hour as a fixed cost.
• The Glue Job's own ENI: created inside the subnet on every job run, via the Glue Connection. The network initialization this involves is the extra element unique to VPC mode.

Sample Code for Testing

The Glue Job I'm using here is a simple PySpark script that reads a CSV from S3 (Raw) and writes it into an Iceberg table on S3 Tables. The job code itself is identical between Default and VPC mode — the only thing that changes is the network path.

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

args = getResolvedOptions(sys.argv, ["JOB_NAME"])

sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)

# Read the CSV from S3 (Raw)
raw_df = spark.read.option("header", "true").csv(
    "s3://penguin-raw-test/daily_sales/"
)

raw_df = raw_df.selectExpr(
    "CAST(sales_date AS date) AS sales_date",
    "CAST(amount AS long) AS amount",
)

# Write into the S3 Tables Iceberg table (Glue-integrated)
# The catalog is configured ahead of time in Spark to point through s3tablescatalog
raw_df.writeTo("glue_catalog.analytics.daily_sales").append()

job.commit()

On the Spark configuration side (Job Parameters, or %%configure in a notebook), I'm passing --conf settings to point the Iceberg catalog at the Glue catalog via s3tablescatalog.

--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog
--conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
--conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
--conf spark.sql.catalog.glue_catalog.glue.id=123456789012:s3tablescatalog/penguin-rest-test
--conf spark.sql.catalog.glue_catalog.glue.account-id=123456789012
--conf spark.sql.catalog.glue_catalog.glue.region=ap-northeast-1
--conf spark.sql.catalog.glue_catalog.warehouse=s3://penguin-rest-test/
--conf spark.sql.defaultCatalog=glue_catalog

For a notebook session, you pass this as JSON via the %%configure magic:

%%configure
{
  "--conf": "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions --conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog --conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO --conf spark.sql.catalog.glue_catalog.glue.id=123456789012:s3tablescatalog/penguin-rest-test --conf spark.sql.catalog.glue_catalog.glue.account-id=123456789012 --conf spark.sql.catalog.glue_catalog.glue.region=ap-northeast-1 --conf spark.sql.catalog.glue_catalog.warehouse=s3://penguin-rest-test/ --conf spark.sql.defaultCatalog=glue_catalog",
  "--datalake-formats": "iceberg"
}

To run in VPC mode, you first need to create a Glue Connection. The ConnectionType is set to NETWORK. Unlike connection types for data stores such as JDBC, this one simply grants reachability to a subnet inside your VPC.

aws glue create-connection \
  --connection-input '{
    "Name": "penguin-vpc-connection",
    "ConnectionType": "NETWORK",
    "PhysicalConnectionRequirements": {
      "SubnetId": "subnet-xxxxxxxxxxxxxxxxx",
      "SecurityGroupIdList": ["sg-xxxxxxxxxxxxxxxxx"],
      "AvailabilityZone": "ap-northeast-1a"
    }
  }' \
  --region ap-northeast-1

On the Glue Job side, you just add this connection name under Connections to switch it into VPC mode. For the Default mode job, you leave Connections unset and use it as-is.

For measuring execution time, I use ExecutionTime, StartedOn, and CompletedOn from get_job_run. StartedOn is defined in the official API reference only as "the time at which this job run was started" — it doesn't specify whether that's the transition into the STARTING state or the RUNNING state.

ExecutionTime, on the other hand, is officially defined as "the amount of time the job run consumed resources," which means it can include internal Glue processing time — such as SparkContext initialization — in addition to the actual ETL processing done by user code.

If you want to isolate pure ETL processing time, you'll need to separately check the CloudWatch metric glue.driver.aggregate.elapsedTime (ETL elapsed time that excludes bootstrap time). For even finer-grained measurement, tracking JobRunState transitions (STARTING, RUNNING, SUCCEEDED) via EventBridge would be a more precise approach.

Execution Time Comparison Results

For both Default and VPC mode, I pulled the full job run history via get-job-runs and compared total execution time (StartedOn to CompletedOn), ExecutionTime, and overhead (total time minus ExecutionTime). After excluding values that deviated significantly from the rest of the same-mode results (discussed below), I ended up with 15 samples for each mode.

Mode	Total time (mean / min / max / stddev)	ExecutionTime (mean / min / max / stddev)	Overhead (mean / min / max / stddev)	n
Default	80.2s / 62.5s / 110.3s / 11.6s	73.1s / 58s / 98s / 9.7s	7.1s / 4.5s / 12.3s / 2.4s	15
VPC	97.9s / 69.0s / 133.5s / 19.0s	87.7s / 64s / 109s / 14.1s	10.1s / 5.0s / 24.5s / 6.6s	15

The mean differences were +17.7s for total time, +14.7s for ExecutionTime, and +3.0s for overhead — VPC mode came out larger on every metric. Even accounting for the standard deviations (11.6s for Default, 19.0s for VPC on total time), VPC mode was consistently higher across the sample range I collected.

The gap in standard deviation itself is also worth noting: VPC mode's stddev is about 1.6x Default's (19.0s vs. 11.6s). In other words, VPC mode isn't just slower on average — its execution time is also more variable. This suggests that ENI creation and Interface VPC Endpoint API calls may be introducing latency that fluctuates from run to run.

The excluded outliers were two Default runs (150s and 146s ExecutionTime) and one VPC run (177s). All three deviated clearly from the rest of the samples within their own mode, so I excluded them from the statistics. Concretely, Default's normal values topped out at 98s, while the outliers were 146–150s (roughly 1.5–1.6x); VPC's normal values topped out at 109s, while its outlier was 177s (roughly 1.6x). Both sets of outliers are clearly separated from the rest of their mode's values.

Going in, I'd expected ExecutionTime to be roughly the same between the two modes, with only the ENI-creation overhead being larger for VPC mode. What I actually found was that the overhead gap (+3.0s) supports the direction of that hypothesis, but doesn't fully explain the result on its own. The absolute gap is larger on the ExecutionTime side (+14.7s) — so "the difference is just overhead" doesn't hold up.

Why the Gap Isn't Fully Explained by Overhead Alone

As for why ENI creation isn't showing up as as large an overhead as I'd expected, I have a few hypotheses:

• Hypothesis 1: ExecutionTime is defined as "the time the job run spent consuming resources." It's possible that ENI creation is counted within the ExecutionTime window itself, rather than in the StartedOn-to-ExecutionTime-start gap I'm calling "overhead." Given that the overhead gap is only +3.0s while the total-time gap is +17.7s, it would make sense if most of the ENI-creation cost is being attributed to ExecutionTime rather than overhead.

• Hypothesis 2: Now that access to the Glue Data Catalog goes through an Interface VPC Endpoint instead of the local proxy, the catalog API calls themselves might be taking longer. That said, Interface VPC Endpoint API latency is generally on the order of single-digit to double-digit milliseconds, which feels too small to account for a 14.7-second gap on its own. It's conceivable that repeated catalog API calls add up, but I don't think this is strong enough evidence to call it the primary cause.

• Hypothesis 3: A combination of factors specific to VPC mode — network initialization for the Glue execution environment, DNS resolution, and Interface VPC Endpoint-based service access — could be contributing together.

None of these has solid supporting evidence; the only things I can say for certain are the facts drawn from the measurements themselves. Across 15 samples per mode, VPC mode ran about 17.7 seconds longer on average in total, with roughly 14.7 seconds of that on the ExecutionTime side and about 3.0 seconds on the overhead side. Every hypothesis here is speculation drawn from that data — including whether ENI creation is even included in the ExecutionTime measurement window — and none of it is backed by official documentation. Measuring more granularly within the ExecutionTime window could help confirm or rule these out.

Choosing Between Default and VPC Mode

Cost Comparison

What VPC mode adds is the hourly cost of Interface VPC Endpoints (for Glue and for S3 Tables). Access to CloudWatch Logs continues to go through the local proxy even with --disable-proxy-v2 enabled, so no additional Interface VPC Endpoint is needed there. Access to the Glue Data Catalog, however, does need one — in a private subnet setup without a NAT Gateway or Internet Gateway, the local proxy has no path to the Glue-managed VPC and simply doesn't work (more on this below).

I checked the actual rate in the AWS Pricing Calculator for the Tokyo region (Amazon VPC → AWS PrivateLink), and an Interface VPC Endpoint comes out to $0.014 per endpoint, per AZ, per hour.

Glue endpoint (Interface):        1 AZ × $0.014/hour
S3 Tables endpoint (Interface):   1 AZ × $0.014/hour

2 endpoints × 1 AZ × $0.014/hour × 730 hours/month
= ~$20.44/month (single-AZ setup)

For a multi-AZ (2 AZ) setup:
2 endpoints × 2 AZ × $0.014/hour × 730 hours/month
= ~$40.88/month

On top of that, there's a data processing charge based on the volume passing through each endpoint (for the first PB per month, $0.01/GB). For a small data movement like this S3 Raw → S3 Tables job, that charge is negligible — most of the monthly cost comes from the hourly endpoint charge.

Default mode doesn't carry any of these fixed costs, so for a setup where everything is reachable via AWS public services, you're looking at roughly a $20–40/month cost difference (single-AZ to multi-AZ).

Item	Default mode	VPC mode
Interface VPC Endpoint (Glue)	Not needed	Needed (per AZ)
Interface VPC Endpoint (S3 Tables)	Not needed	Needed (per AZ)
Gateway VPC Endpoint (S3)	Not needed	Needed (free)
CloudWatch Logs traffic	Via AWS-managed network	Via local proxy (no endpoint needed)
Job startup overhead	None	ENI creation (on every run)

One thing worth noting: S3 Tables routes different kinds of operations through different endpoints. Table bucket, namespace, and table creation/deletion operations go through the dedicated S3 Tables endpoint (s3tables.<region>.amazonaws.com), while object-level operations — reading and writing data files and metadata files — go through the S3 endpoint (s3.<region>.amazonaws.com). This s3tables endpoint can't be substituted with the S3 Gateway Endpoint; it needs to be created separately as an Interface endpoint. AWS's own documentation recommends creating two separate VPC endpoints — one for S3, one for S3 Tables — when accessing S3 Tables, which is exactly what this setup does.

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-id \
  --service-name com.amazonaws.ap-northeast-1.s3tables \
  --subnet-ids subnet-1 subnet-2 \
  --vpc-endpoint-type Interface \
  --ip-address-type dualstack \
  --dns-options "DnsRecordIpType=dualstack" \
  --security-group-ids sg-id \
  --region ap-northeast-1

For CloudWatch Logs, the documentation states that even with --disable-proxy-v2 enabled, traffic continues going through the local proxy — so no additional Interface VPC Endpoint is needed there.

Access to the Glue Data Catalog is a different story, though. The local proxy mechanism relies on proxying requests through the Glue-managed VPC to reach the Glue API — and in a private subnet setup without a NAT Gateway or Internet Gateway, there's no path between your VPC and that Glue-managed VPC, so the proxy simply doesn't function. I ran into this directly during testing: connections to glue.<region>.amazonaws.com timed out, and creating an Interface VPC Endpoint for Glue (com.amazonaws.<region>.glue) resolved it.

As for STS, in a basic setup like this one — without Lake Formation credential vending — I found no evidence that it's needed. So for a setup like this, the conclusion is that you need two Interface VPC Endpoints: one for Glue, one for S3 Tables.

Thinking in Terms of Responsibility

• Default mode: hands off responsibility for network control to AWS. You can focus purely on IAM permissions and job logic.
• VPC mode: you own network control yourself, all the way down to subnet design, security groups, and endpoint placement.

This mirrors a structure I laid out in an earlier article about where authorization responsibility sits across IAM, Lake Formation, and s3tables IAM actions. That same lens — where does the responsibility live — applies just as well to Glue's network configuration.

Thinking in Terms of Security Requirements

VPC mode becomes necessary mainly in two situations:

• You need reachability to VPC-internal resources like RDS, Redshift, or on-prem systems
• You have an explicit requirement to keep your traffic fully private end-to-end (audit or compliance requirements, for example)

Conversely, if everything you're reaching is an AWS public service — S3, Glue Data Catalog, S3 Tables — there's not much of a reason to actively reach for VPC mode.

For a setup like this one, where everything is contained within S3, Glue Data Catalog, and S3 Tables, I'd treat Default mode as the first thing to reach for.

That said, this isn't really a "VPC is safer" vs. "Default is simpler" kind of choice. What you're actually deciding is how much of the responsibility for network control you want to hold onto yourself, within whatever scope is genuinely necessary. If you have a clear reason to own that responsibility — reachability into VPC-internal resources, an audit requirement — go with VPC mode. Otherwise, it's reasonable to hand it off to AWS.

Other Notes

• Security groups need a self-referencing rule: to allow communication between the Glue Spark driver and executors, the security group you specify needs an inbound rule that allows all TCP traffic from itself as the source. Without this, the job can start but internal communication will fail.
• Overly restrictive outbound rules can break ENI creation: the security group needs to allow HTTPS to the Glue and S3 Tables Interface VPC Endpoints, so make sure port 443/TCP is open to those endpoints. It's safer to start permissive and tighten gradually.
• A Glue endpoint is required in a private subnet without NAT: without a NAT Gateway or Internet Gateway, the local proxy for Glue Data Catalog access won't work, and connections to glue.<region>.amazonaws.com will time out. Creating an Interface VPC Endpoint for com.amazonaws.<region>.glue resolves this.
• VPC DNS settings: to run in VPC mode, both enableDnsHostnames and enableDnsSupport need to be true on the VPC. If they're not, Glue will fail to resolve hostnames — worth checking ahead of time.
• Verifying the Gateway Endpoint path: VPC Flow Logs alone aren't enough to confidently confirm that traffic is going through the Gateway Endpoint. Since Gateway endpoints resolve via prefix lists in the route table, checking the route table configuration directly is more reliable. In this environment's route table, I found an active route with a destination of pl-xxxxxxxx (the S3 prefix list) and a target pointing to the Gateway VPC Endpoint I'd created — confirming that S3-bound traffic does resolve through that endpoint.

Conclusion

In this article, I used an S3 (Raw) → S3 Tables pipeline as a test case to compare AWS Glue Job's Default mode and VPC mode with actual measurements.

To summarize:

• Default mode reaches AWS public services over AWS-managed networking, without touching your own VPC.
• VPC mode creates an ENI in a subnet via a Glue Connection and routes traffic through it. It's the right choice when you need reachability to VPC-internal resources or full control over your network path.
• Under the conditions tested here (G.1X, a simple S3 → S3 Tables write, 15 samples per mode), VPC mode ran about 17.7 seconds longer on average in total execution time. Of that, roughly 14.7 seconds showed up on the ExecutionTime side and about 3.0 seconds on the overhead side — meaning the gap was larger on the ExecutionTime side than my original hypothesis (that the difference would come entirely from ENI-creation overhead) predicted, and overhead alone doesn't explain it.
• VPC mode's cost has two distinct components: the ongoing hourly charge for Interface VPC Endpoints (fixed cost), and network-initialization latency incurred on every job run.
• In terms of responsibility, Default mode is the simpler choice that hands network control to AWS, while VPC mode means taking on that control yourself, in exchange for meeting security requirements or reaching resources inside your VPC.

Network architecture decisions tend to stick around once you make them — there's rarely a natural point to revisit them later. Having actual numbers on hand for how much of a difference a choice like this makes should be useful the next time a similar decision comes up.

I hope this article is useful to anyone weighing network configuration options for their Glue Jobs.