DEV Community: kazuho cryer-shinozuka

Implementing Feature Flags in AWS CDK: A Contributor's Guide

kazuho cryer-shinozuka — Tue, 09 Dec 2025 14:13:22 +0000

Overview

When introducing breaking changes in CDK, implementing feature flags is mandatory.

Since there isn't much information about feature flags, I created this article with two goals:

Summarize the implementation method for contributors
Explain the role and purpose of feature flags for CDK users who are unfamiliar with them

If you're in the latter category, feel free to skip the implementation steps section.

Introduction

In 2023, AWS Network Load Balancer (NLB) finally introduced the long-awaited security group (SG) support.

This was a feature that most NLB users had been eagerly waiting for, and with its introduction, there's no longer any benefit to using the legacy "NLB without SG."

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-security-groups.html

AWS CDK also enabled SG configuration immediately after the release, but to avoid breaking changes, the default behavior was still to create NLBs without SGs.
However, once you create an NLB without SG, it's impossible to add SG configuration later. (The NLB will be replaced!)

declare const vpc: ec2.IVpc;

// Creates NLB without SG
new elbv2.NetworkLoadBalancer(this, 'Nlb', {
  vpc,
});

// Creates NLB with SG
new elbv2.NetworkLoadBalancer(this, 'Nlb', {
  vpc,
  securityGroups: [sg] ← Having to specify SG every time is tedious
});

// Desired behavior: NLB with SG is created by default
new elbv2.NetworkLoadBalancer(this, 'Nlb', {
  vpc,
  // securityGroups: [sg] ← No explicit SG configuration needed
});

To improve this situation, I submitted a PR using feature flags to make implicit SG enablement the default behavior. It took about six months to merge, but it was successfully released in v2.221.1.

From here, I'll summarize the know-how for implementing feature flags in CDK using the PR creation process as an example.

What Are Feature Flags?

Feature flags in CDK are a mechanism for gradually introducing breaking changes. New projects get the new behavior by default, while existing projects maintain the old behavior and can explicitly enable the flag to use the new feature.

For a detailed explanation, I'll defer to this excellent existing article. (Sorry for in Japanese)

https://www.ogis-ri.co.jp/otc/hiroba/technical/cdk-concepts/part8.html

Existing Implementation

Let's first review the implementation before the modification.

export class NetworkLoadBalancer extends BaseLoadBalancer implements INetworkLoadBalancer {
  ...
  this.connections = new ec2.Connections({
    securityGroups: props.securityGroups,
  });
  ...
}

This code might seem a bit confusing at first...

The NetworkLoadBalancer class inherits IConnectable via INetworkLoadBalancer, allowing you to configure security group rules through connections.

Many of you are probably familiar with allowTo/From() methods like the following. The IConnectable interface makes this possible.

declare const instance: ec2.Instance;

const nlb = new elbv2.NetworkLoadBalancer(this, 'Nlb');
nlb.connections.allowTo(instance, ec2.Port.tcp(8080));

Classes that inherit IConnectable need to store an ec2.Connections instance with security groups as arguments in this.connections. This is what the CDK implementation above is doing.

  this.connections = new ec2.Connections({
    securityGroups: props.securityGroups,
  });

When you pass undefined to securityGroups of ec2.Connections, no security group is created, resulting in a legacy NLB without any security group attached.
In other words, with the existing implementation, unless you explicitly pass security groups to props.securityGroups, you couldn't create an NLB with security group configuration.

The Naive Approach

How should we make security groups created by default?
The simplest approach would be to create a security group within the L2 construct if props.securityGroups is undefined.

export class NetworkLoadBalancer extends BaseLoadBalancer implements INetworkLoadBalancer {
  ...
  this.connections = new ec2.Connections({
    // Create new SG if props.securityGroups is undefined
    securityGroups: props.securityGroups ?? new ec2.SecurityGroup({...}),
  });
  ...
}

However, if we actually released this implementation, a very serious problem would occur.

For example, suppose you had already created legacy NLBs without SGs using CDK. If you upgrade aws-cdk-lib and deploy in this state, all NLBs would be changed to NLBs with SG configuration by default, causing all existing legacy NLBs to be recreated. NLB recreation involves changes to access URL FQDNs and IPs, so it's easy to imagine the chaos of production systems going down one after another.

To avoid this problem, let's implement a feature flag to achieve:

(i) No changes for existing users
(ii) Provide NLBs with security group configuration for new users

Implementation Steps

1. Define the Feature Flag

First, add the flag to packages/aws-cdk-lib/cx-api/lib/features.ts. This file contains a list of feature flags defined as constants.

export const NETWORK_LOAD_BALANCER_WITH_SECURITY_GROUP_BY_DEFAULT = '@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault';

For the feature flag name, just write out what changes the flag causes in lowerCamelCase.
Don't worry about the length - define it freely like @aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions or @aws-cdk/s3-notifications:addS3TrustKeyPolicyForSnsSubscriptions.

Also, add the feature flag definition to FLAGS in features.ts. It should be appended at the bottom, otherwise you'll get a rosetta error (unverified).

export const FLAGS: Record<string, FlagInfo> = {

  ...(existing feature flag definitions),

  //////////////////////////////////////////////////////////////////////
  [NETWORK_LOAD_BALANCER_WITH_SECURITY_GROUP_BY_DEFAULT]: {
    // Choose from ApiDefault, BugFix, VisibleContext, Temporary. Almost always ApiDefault or BugFix.
    type: FlagType.ApiDefault,
    // Summary
    summary: 'When enabled, Network Load Balancer will be created with a security group by default.',
    // Detailed description
    detailsMd: `
      When this feature flag is enabled, Network Load Balancer will be created with a security group by default.
    `,
    // When it was introduced. V2NEXT is fine. It will be replaced by sed during release.
    introducedIn: { v2: 'V2NEXT' },
    // Recommended value
    // This value is set in cdk.json when creating a new project (cdk init)
    recommendedValue: true,
    // Default value when not set in cdk.json
    // This value is used when existing users upgrade aws-cdk-lib, so set a value that reproduces existing behavior
    // If not set, defaults to false.
    unconfiguredBehavesLike: { v2: false },
    // How to achieve the old behavior
    compatibilityWithOldBehaviorMd: 'Disable the feature flag to create Network Load Balancer without a security group by default.',
  },

Feature flags are recommended to have true for new behavior and false for old behavior.
If you follow this guideline, you only need to set recommendedValue: true, and unconfiguredBehavesLike can be left undefined.

https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md#feature-flags

(I find the mysterious /////// section dividers a bit quirky)

2. Add Feature Flag Documentation

This is the part I still don't fully understand the correct way to write.

First, there are two choices for documentation: README.md or FEATURE_FLAGS.md. The official documentation recommends the former, but the majority of merged PRs use the latter. Sometimes both are updated in the same PR.

Furthermore, there are two candidate files for FEATURE_FLAGS.md, and it's unclear which one to write in.

As mentioned later, there are multiple merge records with modifications only to packages/aws-cdk-lib/cx-api/FEATURE_FLAGS.md, so that should be sufficient.

If modifying README.md

Add to packages/aws-cdk-lib/cx-api/README.md.

* `@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault`

When this feature flag is enabled, Network Load Balancer will be created with a security group by default.

_cdk.json_

{
  "context": {
    "@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault": true
  }
}

If modifying FEATURE_FLAGS.md

Add to packages/aws-cdk-lib/cx-api/FEATURE_FLAGS.md or packages/@aws-cdk/cx-api/FEATURE_FLAGS.md.
Personally, I only modify the former.

First, add a summary to the overview table.

| Flag | Summary | Since | Type |
| ----- | ----- | ----- | ----- |
| [@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault](#aws-cdkaws-elasticloadbalancingv2networkloadbalancerwithsecuritygroupbydefault) | When enabled, Network Load Balancer will be created with a security group by default. | V2NEXT | new default |

Next, add the recommended value to the cdk.json example list.

{
  "context": {
    ... (existing flag list),
    "@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault": true,
}

Finally, add the feature flag summary.

### @aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault

*When enabled, Network Load Balancer will be created with a security group by default.*

Flag type: New default behavior

When this feature flag is enabled, Network Load Balancer will be created with a security group by default.


| Since | Unset behaves like | Recommended value |
| ----- | ----- | ----- |
| (not in v1) |  |  |
| V2NEXT | `false` | `true` |

**Compatibility with old behavior:** Disable the feature flag to create Network Load Balancer without a security group by default.

For "Since", write the version when it was introduced, but at PR time, V2NEXT is fine. It will be replaced with the actual version during release.

3. Implementation in the Construct

This is the main part. Check the feature flag in the actual L2 construct and branch the behavior based on its value.
Make it perform the new behavior when true and the old behavior when false.

import { FeatureFlags } from '../../core';

constructor(scope: Construct, id: string, props: NetworkLoadBalancerProps) {
  // Get the feature flags configured for this CDK App
  const enableDefaultSg = FeatureFlags.of(this).isEnabled(
    cxapi.NETWORK_LOAD_BALANCER_WITH_SECURITY_GROUP_BY_DEFAULT
  );

  // Create security group by default if flag is enabled (new behavior)
  if (enableDefaultSg) {
    this.connections = new ec2.Connections({
      securityGroups: props.securityGroups ?? new ec2.SecurityGroup({...}),
    });
  // Old behavior if flag is disabled
  } else {
    this.connections = new ec2.Connections({
      securityGroups: props.securityGroups,
    });
  }
}

The implementation itself is quite simple.

4. Create Unit Tests

Test the CloudFormation template content for both feature flag true/false states.

In typical cases, unit tests for disabled feature flag (existing behavior) should already be implemented, so add unit tests with the feature flag enabled.

      test('creates NLB with auto-generated security group', () => {
        app = new cdk.App({
          postCliContext: { // Inject feature flag into the App here
            '@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault': true,
          },

         // GIVEN
        // Define as a Stack under the App with feature flag enabled
        const stack = new cdk.Stack(app);
        const vpc = new ec2.Vpc(stack, 'Stack');

        // WHEN
        new elbv2.NetworkLoadBalancer(stack, 'LB', {
          vpc,
          internetFacing: true,
        });

        // THEN
        const template = Template.fromStack(stack);
      template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
          Scheme: 'internet-facing',
          SecurityGroups: [
            {
              'Fn::GetAtt': [
                Match.stringLikeRegexp('LBSecurityGroup.*'),
                'GroupId',
              ],
            },
          ],
          Type: 'network',
        });

        template.hasResourceProperties('AWS::EC2::SecurityGroup', {
          GroupDescription: Match.stringLikeRegexp('Automatically created Security Group for ELB.*'),
          VpcId: { Ref: Match.stringLikeRegexp('Stack.*') },
          SecurityGroupEgress: [{
            CidrIp: '255.255.255.255/32',
            Description: 'Disallow all traffic',
            FromPort: 252,
            IpProtocol: 'icmp',
            ToPort: 86,
          }],
        });
      });

By enabling the feature flag, we can confirm that a security group is now configured on the NLB by default.

5. Create Integration Tests (integ test)

Create integ tests to verify actual deployment behavior.
Similar to unit tests, in most cases integ tests for disabled feature flag (old behavior) already exist, so add integ tests with the feature flag enabled.

const app = new App({
  // Set feature flag as context when creating CDK App
  postCliContext: {
    '@aws-cdk/aws-elasticloadbalancingv2:networkLoadBalancerWithSecurityGroupByDefault': true, // Enable feature flag
  },
});
const stack = new Stack(app, 'NetworkLoadBalancerSecurityGroupFlagStack');

new NetworkLoadBalancer(stack, 'NLB', {
  vpc: vpc,
  internetFacing: true,
});

new IntegTest(app, 'NetworkLoadBalancerSecurityGroupFlag', {
  testCases: [stack],
});

In the actual PR, I implemented assertions to verify security group configuration.
The necessity of assertions in integ tests varies depending on the modification, so implement appropriate integ tests as needed.

Summary

With the above steps, you can introduce a feature flag to CDK.
I hope you now understand the very important role feature flags play in maintaining backward compatibility, though they're something general CDK users don't often need to think about. (Which is actually a testament to CDK's good design)

However, there's rarely a contribution guide for implementing such minor features.
So I put extra effort into writing this article.

If you've ever thought "Ugh, this CDK behavior is not good... I want to fix it my way...", please use this as a reference to submit a PR.

Appendix

The Position of Feature Flags

As mentioned in the article above, feature flags are originally meant to be a last resort, so they should not be added carelessly.

However, it's true that many changes cannot be achieved without feature flags, and I feel that they will inevitably continue to increase at a certain pace.
Personally, I think that CDK often has deprecated default behaviors due to backward compatibility concerns, and such cases should be actively fixed by introducing feature flags.

Currently, I'm proposing modifications to make default values align with modern times for CloudFront Functions runtime settings and IP address types for Lambda function URLs as CloudFront origins. Hopefully they'll be merged within a year...

Feature Flags in Alpha Modules

CDK alpha modules before GA allow breaking changes, so feature flag implementation is not required.
However, you need to clearly state in the PR body that it's a breaking change.

BREAKING CHANGE: Corrected LogRetention IDs for DatabaseCluster. Previously, regardless of the log type, the string 'objectObject' was always included, but after the correction, the log type is now included.

Let's submit PRs that aggressively change behaviors before GA to create more polished modules.

CDK construct for a SaaS that performs data analysis on S3 using DuckDB

kazuho cryer-shinozuka — Sat, 11 Jan 2025 03:04:01 +0000

Introduction

Are you familiar with DuckDB? It can be described as a fast analytical database that operates with a single file, similar to SQLite. I believe the following blog will convey its greatness.

https://www.linkedin.com/pulse/analysing-aws-application-load-balancer-logs-duckdb-unleashing/

By setting up AWS CLI credentials and simply starting DuckDB, you can directly load files on S3 and perform various analyses with very simple queries.

SELECT * FROM read_csv_auto('s3://your-bucket-name/your-file.csv');

However, considering real-world use cases, there may be many situations where issuing AWS credentials to analysts is not ideal.

You want to delegate log analysis to operators, but issuing CLI credentials for them involves a cumbersome approval process.
You want to limit or update the target buckets for analysis, but adjusting IAM policies every time is a hassle.

With this in mind, I thought it would be convenient to run DuckDB on the backend and allow users to freely execute SQL from a frontend with an authentication interface. To address this, I published a CDK construct (cloud-duck).

https://constructs.dev/packages/cloud-duck

In this article, I would like to introduce its features, usage, and the trial-and-error process during development.

Features

cloud-duck offers the following features:

Provides an environment for easily analyzing diverse data on S3.
Access is restricted to authenticated users only.
Allows each user to save analysis results independently.
Extremely easy to deploy using CDK.

Architecture

The frontend is served via S3 + CloudFront.
Users are managed using Cognito User Pool.
API calls are restricted to authenticated users only.
DuckDB runs on Lambda, with read permissions granted to access S3 within the same account.
DuckDB files are persistently saved to S3 whenever a CREATE TABLE operation is performed.

Example Usage

Upon logging in, you will see an SQL input form and a result output form.

Let’s immediately load s3://target-bucket/testdata.csv and extract some data.

You can also save the load results as a table using CREATE TABLE {table_name} AS SELECT ....

Let’s run a query on the test_data table we just created.

The query successfully runs without directly accessing the original data stored in S3.

Data Storage Per User

Query results are stored independently for each logged-in user. This means you can create tables freely without worrying about other users.

Costs

The primary cost comes from the compute usage of the Lambda function running DuckDB. By default, a 1GB memory x86 Lambda function is used. Assuming an average query takes 10 seconds and 100 queries are executed daily, the cost is approximately $0.5 per month.

Additional costs include S3 storage for large tables generated by DuckDB. However, unless used heavily, the monthly cost should remain within a few dollars.

The serverless approach truly shines here.

Setup Instructions

Now, let’s walk through how to deploy cloud-duck using AWS CDK.

Deployment

Since cloud-duck is provided as a CDK construct, it must be deployed using AWS CDK.

For setting up CDK itself, refer to the official documentation. Following the steps up to STEP 5 is sufficient.

Once your CDK application is ready, install cloud-duck and update the Stack definition as follows.

npm install cloud-duck

import * as cdk from 'aws-cdk-lib';
import { CloudDuck } from 'cloud-duck';
import { Construct } from 'constructs';

export class CloudDuckStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create CloudDuck instance
    new CloudDuck(this, 'CloudDuck');
  }
}

Let’s proceed with the deployment.

$ npx cdk deploy

✨  Synthesis time: 5.39s

CloudDuckStack: start: Building 6b82c08c411ad583faa859a28107837b81c3dc67035e3a7bebd7f45fc243e8f0:current_account-current_region
...

IAM Statement Changes

// continue by entering "y"
Do you wish to deploy these changes (y/n)? y
CloudDuckStack: deploying... [1/1]
CloudDuckStack: creating CloudFormation changeset...

 ✅  CloudDuckStack

✨  Deployment time: 646.04s

Outputs:
CloudDuckStack.CloudDuckApiEndpoint38BFF0BB = https://nm5qodr9jl.execute-api.us-east-1.amazonaws.com/api/
// Cognito User Pool ID
CloudDuckStack.CloudDuckCognitoUserPoolId2D258434 = us-east-1_D8Wk6DLME
// Frontend URL
CloudDuckStack.CloudDuckDistributionUrl84FC8296 = https://d28c2qhdxa1so0.cloudfront.net
Stack ARN:
arn:aws:cloudformation:us-east-1:123456789012:stack/CloudDuckStack/75019580-c39b-11ef-b8b8-12856c43826d

✨  Total time: 651.44s

The deployment will be completed in approximately 10 minutes.

Add Users to Cognito User Pool

Add users to the Cognito User Pool who should be granted access. Use the User Pool ID that was output during the deployment process and pass it as the --user-pool-id option.

aws cognito-idp admin-create-user \
--user-pool-id {user-pool-id} \
--username "naonao@example.com" \
--user-attributes Name=email,Value="naonao@example.com" Name=email_verified,Value=true \
--message-action SUPPRESS \
--temporary-password Password1!

Access to the frontend

Access the CloudFront URL displayed during deployment in your browser.

You will be directed to the sign-in screen. Log in using the temporary password that was set when adding the user.

Once successful, you will be redirected to a password change screen. Enter a new password to proceed.

You will then be redirected to the query execution screen.

Note

As of version 0.0.13, there may be an issue where the Submit Query button does not respond immediately after logging in. A simple page reload will resolve the issue, so please try that. I plan to address this in future updates.

Development Trials and Errors

Now, I'd like to share some of the trial and error I encountered during development.

Running DuckDB in the Browser

One of the key attractions of DuckDB is its fast query execution after the initial data load. Initially, I thought it would be ideal to run DuckDB in the browser to make the most of this feature. However, since regular DuckDB is built as a binary dependent on the CPU architecture, running it in the browser requires using the webassembly version, duckdb-wasm.

Of course, there were pioneers before me, and this wonderful Japanese blog was incredibly helpful in guiding me.

https://zenn.dev/shiguredo/articles/duckdb-wasm-s3-parquet-opfs

I also tried creating a Cognito Identity Pool and issuing temporary AWS credentials to the frontend, then saving the S3 data to OPSF and querying duckdb-wasm.

However, I found that duckdb-wasm is limited in functionality compared to the main DuckDB, especially since the AWS extension cannot be used, making it difficult to easily query S3. When I first started the project, I was captivated by the power of DuckDB in cases like the ALB log analysis queries, where the AWS extension truly shines. Writing complex data retrieval queries seemed to undermine the beauty of DuckDB, so I decided to switch to running DuckDB on the backend.

CREATE TABLE alb_log_202411 AS
SELECT *
-- Just with this, it automatically consolidates a bunch of gzip-compressed log files into a DuckDB table.
FROM read_csv(
   's3://[YOUR_S3_BUCKET_NAME]/AWSLogs/[YOUR_ACCOUNT_ID]/elasticloadbalancing/[YOUR_REGION]/2024/11/**/*.log.gz',
    columns={
        'type': 'VARCHAR',
       ...

Passing deploy-time values to the frontend after deployment

The frontend operates as a Single Page Application (SPA), but to integrate authentication and make API calls, the following information needs to be configured:

Cognito User Pool ID
Cognito Application Client ID
API Endpoint

However, these values are finalized only after the deployment to each environment, so they cannot be hardcoded into the frontend code in advance. There are two ways to resolve this:

Output the deploy-time values as a JSON file via S3 Bucket Deployment, and dynamically import them in the frontend.
- https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3_deployment.Source.html#static-jsonwbrdataobjectkey-obj
Build the frontend after deployment is complete.
- deploy-time-build construct

There’s no issue with either method as both allow for a one-time deployment, but this time, we chose the latter to avoid modifying the frontend.

Both methods can be resolved instantly thanks to using CDK, but without it, the process would be quite cumbersome, which makes me appreciate CDK even more.

Persistence of DuckDB Files

By default, DuckDB stores data in memory, allowing for fast performance. However, it is also possible to persist the data to a file.

const duckdb = require('duckdb');
// in memory
const db = new duckdb.Database(':memory:');
// persist data to a file
const persistedDb = new duckdb.Database('/tmp/db.duckdb');

Since the application is running on Lambda, it is essential to persist the data to a file. Also, since the saved files would be stored on the ephemeral storage (/tmp) of Lambda and would be lost after execution, it is necessary to save them to S3 every time.

So, I initially created the following Lambda code:

Download the DuckDB file from S3 (if it exists).
Generate a connection, execute a query, and close the connection. Upload the DuckDB file back to S3.

However, when I actually ran it, for some reason, the result of the CREATE TABLE was not saved.

$ CREATE TABLE test_table AS SELECT ...
// Succesfully completed and show the number of columns of the created table
{
  count: 1234
}

$ SELECT * FROM test_table;
// The test_table I created does not exist.
test_table does not exist.

The cause was that it was also necessary to save the intermediate file, duckdb.wal, which is automatically generated by DuckDB. Often, the .wal file is not deleted when connection.close() is called, and in those cases, it was necessary to generate a new connection while both the DuckDB file and the .wal file were present at the same time.

In the end, I am running the Lambda function with the following steps:

(If present) Download the DuckDB file and the intermediate file.
Generate connection, execute query, and close connection.
Sync the DuckDB file and the intermediate file between Lambda and S3:
- If there was an intermediate file after the previous query, but there is no intermediate file for the current query, delete the intermediate file from S3 as well.

This part is a bit cumbersome, and I’m considering whether mounting EFS to store the files might have been a better approach.

Including Lambda and Frontend Code in CDK Constructs

Deploying Lambda Code

To avoid unnecessary processing for the user when using NodejsFunction in a construct, I decided to build the Lambda code during the release process and specify the pre-built JS files using the lambda.Functionclass.

As detailed in the mentioned presentation, the Lambda code is built using the prependExec in .projenrc.ts. Once built, I simply specify the pre-built .js files in the lambda.Function constructor.

// generate .js files at `lambda/duckdb/build`
project.projectBuild.compileTask.prependExec('npm ci && npm run build', {
  cwd: 'lambda/duckdb',
});

 const duckdbHandler = new lambda.Function(this, 'DuckDbHandler', {
      runtime: lambda.Runtime.NODEJS_20_X,
      // specify lambda/duckdb/build/index.js
      code: lambda.Code.fromAsset(path.join(__dirname, '../../../lambda/duckdb/build')),
      handler: 'index.handler',
    });

Folder Structure

The CDK construct itself is created using projen's AwsCdkConstruct. It provides a convenient template, including GitHub Actions for npm package publishing, which has been very helpful.

Initially, I set up frontend and lambda directories under the src directory (for some reason), but the uploaded npm package ended up being an empty shell, without any of the code from frontend or lambda.

It seems that projen has some filtering logic that excludes these files.

Ultimately, by organizing the project with /frontend, /lambda, and /src directories at the root, it began working properly.

Bad Example

.
└── src
    ├── bin
    ├── frontend
    ├── lambda
    └── lib

Good Example

.
├── frontend
├── lambda
└── src
    ├── bin
    └── lib

Finally

This concludes the introduction of a CDK construct that allows you to quickly leverage DuckDB. I hadn’t seen a CDK construct that includes frontend code before, so I believe this could be a valuable contribution.

I would be very happy if those who haven’t used CDK yet could use this as a starting point to get into it! I’m also open to bug reports, feature requests, and suggestions for areas where implementation may have been lacking.

https://github.com/badmintoncryer/cloud-duck

Maximizing the Use of EC2 Instance Connect Endpoint with CDK

kazuho cryer-shinozuka — Mon, 06 May 2024 09:13:59 +0000

Introduction

Are you using the EC2 Instance Connect Endpoint (EIC Endpoint)? It's a groundbreaking service that allows you to eliminate bastion hosts for free.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connect-with-ec2-instance-connect-endpoint.html

With CDK, it has become easy to set this up, so let me introduce you to it.

Spoiler

You can establish administrative communications not only to EC2 but also to RDS (Aurora). It's incredibly convenient, so I highly recommend giving it a try.

Preparation

While there is no official L2 construct for the EIC Endpoint, there is an L2 available in the community-driven Construct library called open-constructs. Therefore, we will set up a CDK project, install this, and use it.

npx cdk init --language=typescript
npm install @open-constructs/aws-cdk

Usecase

We will place an EC2 instance in a private subnet and try to manage it via SSH through the EIC Endpoint.

Implementation example with CDK

export class TempCdkProjectStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // NAT Gateways and Internet Gateways are not necessary. Please remove them as appropriate.
    const vpc = new ec2.Vpc(this, 'VPC')

    // EC2 instance
    const insntance = new ec2.Instance(this, 'Instance', {
      vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.MICRO),
      machineImage: new ec2.AmazonLinuxImage({
        generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2023,
      })
    });

    // L2 Construct for EIC Endpoint
    const eicEndpoint = new ocf.aws_ec2.InstanceConnectEndpoint(this, 'InstanceConnectEndpoint', {
      vpc,
    });

    // Opening a hole in the Security Group from EIC Endpoint to EC2 Instance
    eicEndpoint.connections.allowTo(insntance, ec2.Port.tcp(22));

    // Output of Instance ID
    new cdk.CfnOutput(this, 'InstanceId', {
      value: insntance.instanceId,
    });
  }
}

Connection method

After deploying the above CDK code, execute the following command. Please modify the instance ID to the one output during the cdk deploy process.

$ aws ec2-instance-connect ssh --instance-id i-12345example --connection-type eice

The authenticity of host '10.0.0.1 (<no hostip for proxy command>)' can't be established.
ED25519 key fingerprint is SHA256:abcdefg.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.1' (ED25519) to the list of known hosts.
   ,     #_
   ~\_  ####_        Amazon Linux 2023
  ~~  \_#####\
  ~~     \###|
  ~~       \#/ ___   https://aws.amazon.com/linux/amazon-linux-2023
   ~~       V~' '->
    ~~~         /
      ~~._.   _/
         _/ _/
       _/m/'
[ec2-user@ip-10-0-0-1 ~]$

It connected smoothly! It's fantastic.

Administrative communication to RDS (Aurora)

Next, we will place Aurora (MySQL) in a private subnet and attempt to manage it via the EIC Endpoint.
Although the default port for MySQL is 3306, which is not supported by the EIC Endpoint, we plan to circumvent this by changing the port that the DB listens on to 3389, a daring workaround.

Implementation example with CDK

export class TempCdkProjectStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = new ec2.Vpc(this, 'VPC')

    // Aurora cluster
    const auroraCluster = new rds.DatabaseCluster(this, 'Aurora', {
      engine: rds.DatabaseClusterEngine.auroraMysql({
        version: rds.AuroraMysqlEngineVersion.VER_3_06_0,
      }),
      // 【Important】 Change port to 3389 to use EIC Endpoint
      port: 3389,
      // Please use rds.Credentials.fromGeneratedSecret('hogeuser')
      credentials: rds.Credentials.fromPassword('admin', cdk.SecretValue.unsafePlainText('testPass')),
      vpc,
      writer: rds.ClusterInstance.serverlessV2('writer'),
    });

    const eicEndpoint = new ocf.aws_ec2.InstanceConnectEndpoint(this, 'InstanceConnectEndpoint', {
      vpc,
    });

    // Opening a hole in the Security Group from EIC Endpoint to EC2 Instance
    eicEndpoint.connections.allowTo(auroraCluster, ec2.Port.tcp(3389));

    // DB Endpoint (Use to get private IP)
    new cdk.CfnOutput(this, 'AuroraEndpoint', {
      value: auroraCluster.clusterEndpoint.hostname,
    });

    new cdk.CfnOutput(this, 'EicEndpointId', {
      value: eicEndpoint.instanceConnectEndpointId,
    });
  }
}

Connection method

First, obtain the private IP address of the database.

$ nslookup hogehoge.cluster-cvekcubvryhp.ap-northeast-1.rds.amazonaws.com
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
 hogehoge.cluster-cvekcubvryhp.ap-northeast-1.rds.amazonaws.com   canonical name =  hogehoge.cluster-cvekcubvryhp.ap-northeast-1.rds.amazonaws.com.
Name:    hogehoge.cluster-cvekcubvryhp.ap-northeast-1.rds.amazonaws.com
Address: 10.0.198.63

Next, set up an SSH tunnel. Please specify the parameters as follows:

Parameter	value
--private-ip-address	Private IP of the DB
--instance-connect-endpoint-id	EIC Endpoint ID
--local-port	Local listening port (Anything is OK)
--remote-port	DB listening port (3389)

$ aws ec2-instance-connect open-tunnel --instance-connect-endpoint-id eice-hogehoge --private-ip-address 10.0.198.63 --local-port 3306 --remote-port 3389
Listening for connections on port 3306.

The tunnel has been successfully established! With the above command still running, try connecting to localhost:3306 using any suitable DB client tool.

This time, we'll connect using Sequel Ace. The credentials are as specified in the CDK: username: admin, password: testPass.

Great to hear it worked! It’s fantastic, isn’t it?

This is something you can't do with Session Manager, so I think it's a major point of differentiation.

Precautions for use

Permissions when accessing via EIC Endpoint

Users need specific permissions for access via the EIC Endpoint. Please refer to the official documentation for precise information.

By leveraging this, you can grant access permissions to specific users only. Thanks to IAM, authorization management becomes a breeze!

Is database access included in the use cases for EIC Endpoint?

I'm not sure. It's probably a gray area.

Initially, when the EIC Endpoint service was released, all ports were open, but soon after, all ports except 22 and 3389 were closed. Therefore, I believe AWS wants us to use it only for SSH and RDP.

Currently, the EIC Endpoint only determines whether to allow or deny communication based on port numbers (Layer 4). I don't think they go as far as inspecting the payload at Layer 7 to make these decisions, but is it technically possible to do so...?? I'd appreciate it if someone with more expertise could clarify...

So, while it works, please use it at your own risk.

Can you also access other VPC resources?

It should work as long as you have the private IP and port number within the VPC determined.

At last

Actually, I created a PR for this L2 construct!

There are only two L2 constructs in open-constructs so far, so there are endless opportunities for contribution waiting. I encourage everyone to give it a try!