DEV Community

Cover image for AWS CDK 100 Drill Exercises #008: CloudFront VPC Origin — Reaching an Internal ALB Directly, and Designing an Incident-Response Escape Hatch
AWS Community Builders profile image TOMOAKI ishihara
TOMOAKI ishihara for AWS Community Builders

Posted on

AWS CDK 100 Drill Exercises #008: CloudFront VPC Origin — Reaching an Internal ALB Directly, and Designing an Incident-Response Escape Hatch

#aws #cdk #100drillexercises #cloudfront

Level 300

Introduction

This is the 8th installment of "AWS CDK 100 Drill Exercises." See here for an overview of the series.

This time, we place two very different origins behind a single Amazon CloudFront distribution: a private S3 bucket serving a static site, and a private, internal ALB that CloudFront can reach only through a VPC Origin. No public IP, no NAT Gateway — CloudFront reaches the ALB directly in a private subnet.

And we don't stop at a "working architecture" — we also build in an incident-response mechanism based on a real AWS outage.

To keep this sample simple, the CloudFront⇔ALB communication uses HTTP (port 80) only. In production, you'd want to attach an ACM certificate to the ALB, add an HTTPS listener, and switch CloudFront's origin protocol policy to HTTPS_ONLY (or match-viewer).

What you'll learn in this article

  • How to serve content without ever making an origin public, using VPC Origin + Origin Access Control (OAC)
  • Automatic failover from a dynamic origin to a static one via Origin Group
  • How to allow-list IP addresses with AWS WAF
  • Recording denied requests to CloudWatch Logs via CloudFront Function + standard logging v2
  • How to deal with the constraint that some CloudFront APIs and metrics are pinned to us-east-1
  • A way of thinking about designing "what to do when it breaks" as a CDK parameter, assuming failure up front

📁 Code repository: GitHub

Architecture Overview

Architecture Overview

Feature Benefit
VPC Origin CloudFront reaches a private ALB directly, with no public IP and no NAT Gateway
Origin Group failover Shows a static fallback page instead of a raw 502/503 when the ALB is unhealthy
Edge IP allow-listing Disallowed IPs are rejected at the edge before ever reaching the ALB or S3
Geo restriction Restricts delivery to only the explicitly allowed countries
Incident-response escape hatch Routes around a VPC Origin-side outage with just a parameter switch — no code changes

Data Flow 

Viewer
  │  HTTPS (min TLS 1.3, geo-restricted to JP/US/GB/CA/AU/NZ/IE)
  ▼
CloudFront distribution
  ├─ WAF (Web ACL, always active): managed rules + rejects IPs not on the allow list
  ├─ CloudFront Function (optional, disabled by default): rejects IPs not on the allow list
  │
  ├─ Default behavior "/*"
  │     └─ S3 origin (OAC) ────────► WebsiteBucket (private)
  │
  └─ Behavior "/alb/*"
      └─ Origin Group
          ├─ Primary: VPC Origin ──► Internal ALB (private subnet)
          └─ Fallback (403/404/5xx): S3 origin (OAC) ─► ErrorBucket (private)
Enter fullscreen mode Exit fullscreen mode

The ALB's security group allows inbound traffic only on port 80 from the CloudFront-managed prefix list. CloudFront itself is reachable from the internet, but the ALB is not directly accessible.

Key Components

Component Design Points
ALB (internal) internetFacing: false, placed in a private-isolated subnet. The VPC Origin's target
ALB (Public, created only when publicAlbFailover is enabled) internetFacing: true, public subnet. A dedicated security group also allows only the CloudFront-managed prefix list
WebsiteBucket / ErrorBucket Private S3 buckets, readable only by CloudFront via OAC
CloudFront distribution Min TLS 1.3, geo restriction, dedicated access log bucket
CloudfrontWafStack (pinned to us-east-1) Managed rules (Core/KnownBadInputs) + IP allow list (before/after rules selectable), defaultAction is block, logs delivered directly to S3
CloudFront Function Viewer IP allow list (optional, created only when allowedCloudFunctionIps is set), evaluated on viewer-request
CloudfrontLogDeliveryStack (pinned to us-east-1, created only when the CloudFront Function is used) Delivers denied-request logs to CloudWatch Logs
CloudfrontMonitoringStack (pinned to us-east-1) 5xx error rate alarm + SNS notification

Implementation Highlights

1. An internal ALB reachable only from CloudFront

The ALB's security group opens port 80 only to the CloudFront-managed prefix list.

albSecurityGroup.addIngressRule(
  ec2.Peer.prefixList(props.cloudfrontManagedPrefixList),
  ec2.Port.tcp(80),
  'Allow inbound HTTP traffic from CloudFront managed prefix list'
);

const alb = new elbv2.ApplicationLoadBalancer(this, 'Alb', {
  vpc: this.vpc.vpc,
  internetFacing: false,
  securityGroup: albSecurityGroup,
  vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_ISOLATED },
});
Enter fullscreen mode Exit fullscreen mode

The prefix list ID is region-specific. Check it beforehand.

aws ec2 describe-managed-prefix-lists \
  --filters "Name=prefix-list-name,Values=com.amazonaws.global.cloudfront.origin-facing"

2. Automatic failover with VPC Origin and Origin Group

CloudFront reaches the internal ALB directly via a VPC Origin. If the ALB stops responding, CloudFront automatically fails over to the static page in S3.

const originGroup = new cloudfront_origins.OriginGroup({
  primaryOrigin: cloudfront_origins.VpcOrigin.withApplicationLoadBalancer(alb, {
    httpPort: 80,
  }),
  fallbackOrigin: cloudfront_origins.S3BucketOrigin.withOriginAccessControl(errorBucket),
  fallbackStatusCodes: [403, 404, 500, 502, 503, 504],
});

distribution.addBehavior('/alb/*', originGroup, {
  viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
  // Without this, the default CACHING_OPTIMIZED policy applies (1-day default TTL). The ALB's
  // fixed responses carry no Cache-Control header at all, so once a successful response gets
  // cached at the edge, it keeps being served for up to a day regardless of the origin's or
  // Origin Group's actual state.
  cachePolicy: cloudfront.CachePolicy.CACHING_DISABLED,
});
Enter fullscreen mode Exit fullscreen mode

Without this explicit cache policy, you'll run into a confusing situation while testing failover: "the origin has switched, but the browser/edge keeps serving a stale cached success response, so nothing looks different." /alb/* exists to show you the ALB's live response, so disabling caching outright is the straightforward choice here.

3. IP allow-listing and managed rules with WAF

Edge access control is handled first by a WAFv2 Web ACL attached to the whole distribution (CloudfrontWafStack). Since defaultAction is block, the effective behavior is that everything except allow-listed IPs is rejected.

const wafAcl = new wafv2.CfnWebACL(this, 'WafAcl', {
  scope: 'CLOUDFRONT',
  defaultAction: { block: {} },
  rules: [
    // Allow before managed rules (only created if IPs are actually specified. priority 1)
    { name: 'AllowSpecificIPsBeforeRules', priority: 1, action: { allow: {} }, ... },
    { name: 'CoreRuleSet', priority: 2, overrideAction: { none: {} }, /* AWSManagedRulesCommonRuleSet */ },
    { name: 'KnownBadInputsRuleSet', priority: 3, overrideAction: { none: {} }, /* AWSManagedRulesKnownBadInputsRuleSet */ },
    // Allow after managed rules (enabled by default. priority 100)
    { name: 'AllowSpecificIPsAfterRules', priority: 100, action: { allow: {} }, /* IPSetReferenceStatement */ },
  ],
});
Enter fullscreen mode Exit fullscreen mode

In the current wiring, the deploying operator's IP (getMyGlobalIp()) is passed as allowedIpsAfterRules, and is allowed after the managed rules are evaluated (priority 100). In other words, even an allow-listed IP is first checked against the Core/KnownBadInputs managed rules for known attack patterns before being allowed through. The allowedIpsBeforeRules option (allow before rule evaluation) is there for cases where you want to verify behavior before the WAF rules take effect.

WAF logs are delivered directly to an S3 bucket whose name must carry the aws-waf-logs- prefix (CfnLoggingConfiguration). The authorization and cookie headers are masked via redactedFields before being recorded.

4. Edge control via CloudFront Function, and incident investigation via standard logging v2

Separately from WAF, a CloudFront Function-based allow list is also implemented for cases where you want finer, per-behavior control. When allowedCloudFunctionIps is set, every viewer request is evaluated and IPs not on the list are rejected. Denied requests are recorded as custom data via cf.logCustomData().

function handler(event) {
  var request = event.request;
  var allowedIps = ['203.0.113.10']; // baked in at synth time
  var clientIp = event.viewer.ip; // viewer IP accessor for the JS_2_0 runtime
  if (!allowedIps.includes(clientIp)) {
    cf.logCustomData(JSON.stringify({ clientIp: clientIp, allowedIps: allowedIps }));
    return { statusCode: 403, statusDescription: 'Forbidden', body: 'Access denied' };
  }
  return request;
}
Enter fullscreen mode Exit fullscreen mode

The CloudFront Function is positioned as an optional feature you enable as needed.

5. The incident-response escape hatch (publicAlbFailover)

This is the point I most want to get across in this article.

On July 16, 2026, AWS CloudFront experienced a multi-hour outage in which 5xx errors spiked for customers using VPC Origins. The root cause was not in customers' own stacks — it was in the VPC Origin connectivity layer itself.

We identified the root cause of the issue as an internal constraint on the fleet that manages connections to private VPC origins. When this constraint was reached, the system responsible for distributing routing configuration to our network processors failed to load the updated configuration data correctly, affecting routing of VPC Origin connections.

AWS's guidance at the time was: "customers who are able to should temporarily switch away from the VPC Origin connection, and switch back once it's resolved."

The problem is that trying to do this by manually rewriting CloudFormation while the incident is ongoing makes both the decision-making and the work slower. So this stack prepares the switch ahead of time, as a single publicAlbFailover parameter.

// The internal ALB (for VPC Origin) is always created. A separate public ALB is created only
// when publicAlbFailover is enabled.
const publicAlb = publicAlbFailoverEnabled
  ? new elbv2.ApplicationLoadBalancer(this, 'PublicAlb', {
      internetFacing: true,
      securityGroup: publicAlbSecurityGroup,
      vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
    })
  : undefined;

// The VPC Origin used as the primaryOrigin in normal mode
const vpcOriginAlb = cloudfront_origins.VpcOrigin.withApplicationLoadBalancer(alb, {
  httpPort: 80,
  protocolPolicy: cloudfront.OriginProtocolPolicy.HTTP_ONLY,
});

// Escape hatch enabled: reach the separate public ALB as a plain public HTTP origin
const publicAlbOrigin = publicAlbFailoverEnabled
  ? new cloudfront_origins.HttpOrigin(publicAlb!.loadBalancerDnsName, {
      httpPort: 80,
      protocolPolicy: cloudfront.OriginProtocolPolicy.HTTP_ONLY,
    })
  : undefined;

// The fallback is always the S3 error page; only the primary swaps to the public ALB when enabled
const originGroup = new cloudfront_origins.OriginGroup({
  primaryOrigin: publicAlbFailoverEnabled ? publicAlbOrigin! : vpcOriginAlb,
  fallbackOrigin: cloudfront_origins.S3BucketOrigin.withOriginAccessControl(errorBucket),
  fallbackStatusCodes: [403, 404, 500, 502, 503, 504],
});
Enter fullscreen mode Exit fullscreen mode

There are 4 design points here.

  • Traffic still only ever flows through CloudFront. Enabling this makes the newly-created public ALB internet-facing, but its security group — just like the internal ALB's — still only allows the CloudFront-managed prefix list, so it's never directly reachable by clients. The client-facing URL doesn't change either.
  • Setting publicAlbFailover.enabled: true makes cloudfrontManagedPrefixList a required field. Without it, there'd be no way to restrict the public ALB to CloudFront only, so it throws at synth time.
  • The prefix list alone isn't enough. The CloudFront managed prefix list is a shared IP range used by every CloudFront customer's distribution, so if someone else's CloudFront distribution pointed at this public ALB's DNS name as an origin, it would still pass the security group check. So, following AWS's official recommended procedure, we generate a secret value at synth time and send it as a CloudFront origin custom header, with the ALB listener rejecting (via its default action, 403) any request that doesn't carry that header.
  • It switches over in "minutes, not instantly." We originally tried a design where keeping the VPC Origin bound as the Origin Group's fallback would avoid recreating it. But CDK auto-derives an origin's logical ID from its "position" — which includes whether it's the OriginGroup's primary or fallback role — so simply swapping that role changes the VPC Origin's own logical ID, and CloudFormation ends up deleting and recreating it anyway (up to ~15 minutes). (I cover this CDK logical-ID auto-numbering mechanism in depth in my own article, "AWS CDK Logical ID Deep Dive: How Adding One CloudFront Origin Broke My Entire AWS CDK Deployment".) You can pin the logical ID with overrideLogicalId to avoid that, but then you trade it for a different constraint: CloudFront's UpdateVpcOrigin API rejects any in-place property change to the origin while it's still associated with a distribution. This sample favors simplicity and accepts that the VPC Origin gets recreated.

If you need genuinely zero-downtime, instant switching in production, consider the CloudFront continuous deployment approach (a staging distribution with header/weight-based gradual traffic shifting) or CloudFront Functions + KeyValueStore-based weighted routing, both covered in Migrate Amazon CloudFront public origins to private VPC origins. Both keep both origins alive at all times and shift traffic gradually by ratio — a step up in complexity from this article's Origin Group swap, but the right choice when you truly need zero-downtime switching.

Using it is simple.

// parameters/dev-params.ts
publicAlbFailover: {
    enabled: true, // false → true
},
Enter fullscreen mode Exit fullscreen mode 
npm run deploy:all
Enter fullscreen mode Exit fullscreen mode

Once AWS resolves the issue, just set enabled: false and redeploy.

6. 5xx error rate alarm (also pinned to us-east-1)

CloudFront's request/error metrics (the AWS/CloudFront namespace) are only published to us-east-1, regardless of the distribution's own region. A CloudWatch Alarm can only evaluate a metric in its own region, so this alarm is also broken out into its own stack, CloudfrontMonitoringStack, deployed to us-east-1.

const alarm = new cloudwatch.Alarm(this, 'CloudFront5xxErrorRateAlarm', {
  metric: new cloudwatch.Metric({
    namespace: 'AWS/CloudFront',
    metricName: '5xxErrorRate',
    dimensionsMap: { DistributionId: props.distributionId },
    period: cdk.Duration.minutes(5),
    statistic: cloudwatch.Stats.AVERAGE,
  }),
  threshold: 5, // 5%
  evaluationPeriods: 3,
  datapointsToAlarm: 3, // sustained for 15 minutes before treating it as a real incident
  comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
  treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});
Enter fullscreen mode Exit fullscreen mode

This alarm watches exactly the metric AWS itself called out during the 2026-07-16 outage (the distribution's overall 5xx error rate). In other words, it's designed to plug directly into an operational flow: "alarm fires" → "check the AWS Health Dashboard" → "if it's a VPC Origin-side problem, enable publicAlbFailover."

Deploy & Verify 

export PROJECT=your-project
export ENV=dev

npm run bootstrap   # first time only
npm run deploy:all
Enter fullscreen mode Exit fullscreen mode 
# Served from S3
curl https://<distribution-domain>/

# Served from the ALB via the Origin Group
curl https://<distribution-domain>/alb/
Enter fullscreen mode Exit fullscreen mode

Cost Estimate

💰 Rough monthly estimate (Tokyo region, low traffic)

Service Usage Rough monthly cost
Application Load Balancer Always on ~$25.00
CloudFront Low request volume ~$1–2
AWS WAF Web ACL + 3 rules + low request volume ~$8.00
S3 (3 buckets) A few MB of static content + logs < $0.10
CloudWatch Alarm + SNS 5xx error rate alarm notifications < $0.10

Total: roughly $23–25/month

The ALB is billed hourly regardless of traffic. Don't forget to run npm run destroy:all to clean up after you're done experimenting.

If cdk destroy / npm run destroy:all gets stuck — a non-empty S3 bucket, a leftover ENI tied to the VPC Origin, or some other resource CloudFormation refuses to delete on its own — delstack, an OSS tool by AWS DevTools Hero k.goto (@365_step_tech), is worth having on hand. It forcibly cleans up the kind of resources that commonly block stack deletion.

Summary

What we learned from this pattern:

  1. VPC Origin: CloudFront can reach a fully private ALB directly, with no public IP and no NAT Gateway
  2. Origin Group: Automatic failover from a dynamic origin to a static one when a server error occurs
  3. Defense in depth at the edge: Geo restriction, WAF, and CloudFront Function give you access control before traffic ever reaches the origin
  4. CloudFront's us-east-1 constraint: Both the log delivery API and metrics are pinned to us-east-1, regardless of the distribution's own region
  5. Bake incident response into code: Given a real outage, prepare "how to route around it" as a parameter ahead of time rather than as manual work
  6. The prefix list isn't a silver bullet: Since it's a shared IP range across every customer, an internet-facing origin should also verify a secret custom header
  7. Be explicit about cache policy on dynamic paths: Leaving the default CACHING_OPTIMIZED in place means a successful response from an origin that never sends Cache-Control gets cached for up to a day, masking your failover test results
  8. CDK's origin logical-ID auto-numbering also depends on "role": Just swapping which origin is primary vs. fallback changes that origin's logical ID, so it can be recreated on every switch. Genuinely zero-downtime switching needs continuous deployment or CloudFront Functions + KeyValueStore-based weighted routing

References

Let's keep learning practical AWS CDK patterns through the 100 drill exercises!
If you found this helpful, please ⭐ the repository!

📌 You can see the entire code in my GitHub repository.

Top comments (0)