DEV Community

Cover image for AWS CDK Python
Deepak Porwal
Deepak Porwal

Posted on

AWS CDK Python

Basics:

cdk

Resources and Identifiers

  • Any piece of Infrastructure that is created via CDK is a Resource.
  • Every Resource has an identifier which will help you to refrence it later.
  • A Resource in the CDK maps to a Resource in CloudFormation
  • The Resource Identifierwill also be the Identifierwith CloudFormation.

Constructs

  • A Construct is a logical grouping of one or more Resources.
  • Constructs are the building blocks with the CDK.
  • Constructs can be programmatically customized.
  • Constructs enable customizable reuse within an Organization.

We can also create another resource out of the construct.
Eg. we need to have a Resource (security group) and an Identifier with (WebSG)

To deploy all of the above resources, called Stack.

Stack

  • A stack is the unit of deployment within CDK.
  • A stack in the CDK corresponds to a stack in Cloudformation.
  • Stacks share cloudformation Stack limitations.
  • Every stack has an Environment that specifies account and region.
  • Environments can be either explicity or implicitly define.

App

  • A CDK App is the root of the context tree for a CDK Project.
  • An App can contain one or more stack.
  • Each stack within an App can contain its own environment.

CDK Workflow

  1. Init --> Project is created using the command line tool
  2. Bootstrap --> Create needed AWS Resources for CDK Enviroment.
  3. Synth --> Generate Cloudformation template from Code.
  4. Deploy --> Template are launched by Cloudformation
  5. Update --> CDK Project is updated with new Infrastructure.
  6. Diff --> Update Against deployed stack are Identified
  7. Again Deploy.

Why use the AWS CDK

Let's look at the power of the AWS CDK. Here is some code in an AWS CDK
project to create an Amazon ECS service with AWS Fargate launch type (this is
the code we use in the Creating an AWS Fargate service using the AWS CDK).

class MyEcsConstructStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        vpc = ec2.Vpc(self, "MyVpc", max_azs=3) # default is all AZs in region
        cluster = ecs.Cluster(self, "MyCluster", vpc=vpc)

        ecs_patterns.ApplicationLoadBalancedFargateService(
            self,"MyFargateService",
            cluster=cluster, # Required
            cpu=512, # Default is 256
            desired_count=6, # Default is 1
                                task_image_options=ecs_patterns.ApplicationLoadBalancedTaskImageOptions(
            image=ecs.ContainerImage.from_registry("amazon/amazon-ecssample")),
            memory_limit_mib=2048, # Default is 512
            public_load_balancer=True) # Default is False
Enter fullscreen mode Exit fullscreen mode

This class produces an AWS CloudFormation template of more than 500
lines.

Other advantages of the AWS CDK include:

  • Use logic (if statements, for-loops, etc) when defining your infrastructure
  • Use object-oriented techniques to create a model of your system
  • Define high level abstractions, share them, and publish them to your team, company, or community
  • Organize your project into logical modules
  • Share and reuse your infrastructure as a library
  • Testing your infrastructure code using industry-standard protocols
  • Use your existing code review workflow
  • Code completion within your IDE

Developing with the AWS CDK

The AWS CDK Toolkit is a command line tool for interacting with CDK apps. It enables developers to synthesize artifacts such as AWS CloudFormation templates, deploy stacks to development AWS accounts, and diff against a deployed stack to understand the impact of a code change.

The AWS Construct Library includes a module for each AWS service with constructs that offer rich APIs that encapsulate the details of how to create resources for an Amazon or AWS service. The aim of the AWS Construct Library is to reduce the complexity and glue logic required when integrating various AWS services to achieve your goals on AWS.

Note:
There is no charge for using the AWS CDK, but you might incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage. Use the AWS Pricing Calculator to estimate charges for the use of various AWS resources.

Getting Started:

Create the App:

Each AWS CDK app should be in its own directory, with its own local module dependencies. Create a new directory for your app. Starting in your home directory, or another directory if you prefer, issue the following commands.

mkdir hello-cdk
cd hello-cdk
Enter fullscreen mode Exit fullscreen mode

Important
Be sure to use the name hello-cdk for your project directory, exactly as shown here. The AWS CDK project template uses the directory name to name things in the generated code, so if you use a different name, some of the code in this tutorial won't work.

Now initialize the app using the cdk init command, specifying the desired template ("app") and programming language.

cdk init TEMPLATE --language LANGUAGE
Enter fullscreen mode Exit fullscreen mode

i.e

cdk init app --language python
Enter fullscreen mode Exit fullscreen mode

![[Pasted image 20220405151619.png]]

After the app has been created, also enter the following two commands to activate the app's Python virtual environment and install its dependencies.

source .venv/Scripts/activate
python -m pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Tip -->
If you don't specify a template, the default is "app," which is the one we wanted anyway, so technically you can leave it out and save four keystrokes. If you have Git installed, each project you create using cdk init is also initialized as a Git repository. We'll ignore that for now, but it's there when you need it.

Here in Python Building is not required.

List the stacks in the app
Just to verify everything is working correctly, list the stacks in your app.

cdk ls
Enter fullscreen mode Exit fullscreen mode

If you don't see HelloCdkStack, make sure you named your app's directory hello-cdk.

Add an Amazon S3 bucket

At this point, your app doesn't do anything useful because the stack doesn't define any resources. Let's define an Amazon S3 bucket. Install the Amazon S3 package from the AWS Construct Library.

pip install aws-cdk.aws-s3
Enter fullscreen mode Exit fullscreen mode

Replace the first import statement in hello_cdk_stack.py in the hello_cdk directory with the following code.

from aws_cdk import (aws_s3 as s3,core)
#Replace the comment with the following code.
bucket = s3.Bucket(self,"MyFirstBucket",versioned=True,)
Enter fullscreen mode Exit fullscreen mode

Bucket is the first construct we've seen, so let's take a closer look. Like all constructs, the Bucket class takes three parameters.

scope:
Tells the bucket that the stack is its parent: it is defined within the scope of the stack. You can define constructs inside of constructs, creating a hierarchy (tree).
Id:
The logical ID of the Bucket within your AWS CDK app. This (plus a hash based on the bucket's location within the stack) uniquely identifies the bucket across deployments so the AWS CDK can update it if you change how it's defined in your app. Buckets can also have a name, which is separate from this ID (it's the bucketName property).
props:
A bundle of values that define properties of the bucket. Here we've defined only one roperty: versioned, which enables versioning for the files in the bucket.
All constructs take these same three arguments, so it's easy to stay oriented as you learn about new ones. And as you might expect, you can subclass any construct to extend it to suit your needs, or just to change its defaults.

Tip -->
If all a construct's props are optional, you can omit the third parameter entirely.

Synthesize an AWS CloudFormation template

Synthesize an AWS CloudFormation template for the app, as follows.

cdk synth
Enter fullscreen mode Exit fullscreen mode

If your app contained more than one stack, you'd need to specify which stack(s) to synthesize. But since it only contains one, the Toolkit knows you must mean that one.

Tip -->
If you received an error like "--app" is required..., it's probably because you are running the command from a subdirectory. Navigate to the main app directory and try again.

The cdk synth command executes your app, which causes the resources defined in it to be translated to an AWS CloudFormation template. The output of cdk synth is a YAML-format AWS CloudFormation template, which looks something like this.

Resources:
  MyFirstBucketB8884501:
    Type: AWS::S3::Bucket
    Properties:
      VersioningConfiguration:
        Status: Enabled
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
    Metadata:
      aws:cdk:path: HelloCdkStack/MyFirstBucket/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/yXIQQ5AMBBA0bPYt0NZ2XIB4QBSVTHKNNFpRMTdEav/8nNQJWSJPoI0o5MrDnB1rI0T7+pDAVcVjbMs6ol+3R9bG3zcjf1cexqR0dMtmpNnT2kBJSiVLAFR7pEYNwvt3wd3SI6vcAAAAA==
    Metadata:
      aws:cdk:path: HelloCdkStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
Enter fullscreen mode Exit fullscreen mode

Even if you aren't very familiar with AWS CloudFormation, you should be able to find the definition for an AWS::S3::Bucket and see how the versioning configuration was translated.

Note:
Every generated template contains a AWS::CDK::Metadata resource by default. The AWS CDK team uses this metadata to gain insight into how the AWS CDK is used, so we can continue to improve it. For details, including how to opt out of version reporting, see Version reporting. The cdk synth generates a perfectly valid AWS CloudFormation template. You could take it and deploy it using the AWS CloudFormation console. But the AWS CDK Toolkit also has that feature built-in.

Deploying the stack

To deploy the stack using AWS CloudFormation, issue:

cdk deploy
Enter fullscreen mode Exit fullscreen mode

As with cdk synth, you don't need to specify the name of the stack since there's only one in the app. It is optional (though good practice) to synthesize before deploying. The AWS
CDK synthesizes your stack before each deployment. If your code changes have security implications, you'll see a summary of these, and be asked to confirm them before deployment proceeds. cdk deploy displays progress information as your stack is deployed. When it's done, the command prompt reappears. You can go to the AWS CloudFormation console and see that it now lists HelloCdkStack. You'll also find
MyFirstBucket in the Amazon S3 console.
You've deployed your first stack using the AWS CDK—congratulations! But
that's not all there is to the AWS CDK.

Modifying the app

The AWS CDK can update your deployed resources after you modify your app. Let's make a little change to our bucket. We want to be able to delete the bucket automatically when we delete the stack, so we'll change the RemovalPolicy.

Update:

# hello_cdk/hello_cdk_stack.py
bucket = s3.Bucket(self,
    "MyFirstBucket",
    versioned=True,
    removal_policy=core.RemovalPolicy.DESTROY)
Enter fullscreen mode Exit fullscreen mode

Now we'll use the cdk diff command to see the differences between what's already been deployed, and the code we just changed.

cdk diff
Enter fullscreen mode Exit fullscreen mode

The AWS CDK Toolkit queries your AWS account for the current AWS CloudFormation template for the hello-cdk stack, and compares it with the template it synthesized from your app. The Resources section of the output should look like the following.

[~] AWS::S3::Bucket MyFirstBucket MyFirstBucketB8884501
├─ [~] DeletionPolicy
│ ├─ [-] Retain
│ └─ [+] Delete
└─ [~] UpdateReplacePolicy
├─ [-] Retain
└─ [+] Delete

As you can see, the diff indicates that the DeletionPolicy property of the bucket is now set to Delete, enabling the bucket to be deleted when its stack is deleted. The UpdateReplacePolicy is also changed.
Don't be confused by the difference in name. The AWS CDK calls it RemovalPolicy because its meaning is slightly different from AWS CloudFormation's DeletionPolicy: the AWS CDK default is to retain the bucket when the stack is deleted, while AWS CloudFormation's default is to delete it. See Removal policies for further details.
You can also see that the bucket isn't going to be replaced, but will be updated instead.
Now let's deploy.

cdk deploy
Enter fullscreen mode Exit fullscreen mode

Enter y to approve the changes and deploy the updated stack. The Toolkit updates the bucket configuration as you requested.

Destroying the app's resources

Now that you're done with the quick tour, destroy your app's resources to avoid incurring any costs from the bucket you created, as follows.

cdk destroy
Enter fullscreen mode Exit fullscreen mode

Enter y to approve the changes and delete any stack resources.

Note
This wouldn't have worked if we hadn't changed the bucket's RemovalPolicy
just a minute ago!
If cdk destroy fails, it probably means you put something in your Amazon S3 bucket. AWS CloudFormation won't delete buckets with files in them. Delete the files and try again.

Working with the AWS CDK in Python

Python is a fully-supported client language for the AWS CDK and is considered stable. Working with the AWS CDK in Python uses familiar tools, including the standard Python implementation (CPython), virtual environments with virtualenv, and the Python package installer pip. The modules comprising the AWS Construct Library are distributed via pypi.org. The Python version of the AWS CDK even uses Python-style identifiers (for example, snake_case method names).
You can use any editor or IDE; many AWS CDK developers use Visual Studio Code (or its open-source equivalent VSCodium), which has good support for Python via an official extension. The IDLE editor included with Python will suffice to get started. The Python modules for the AWS CDK do have type hints, which are useful for a linting tool or an IDE that supports type validation.

Prerequisites

To work with the AWS CDK, you must have an AWS account and credentials and have installed Node.js and the AWS CDK Toolkit. See AWS CDK Prerequisites.
Python AWS CDK applications require Python 3.6 or later. If you don't already have it installed, download a compatible version for your platform at python.org. If you run Linux, your system may have come with a compatible version, or you may install it using your distro's package manager (yum, apt, etc.).
Mac users may be interested in Homebrew, a Linux-style package manager for Mac OS X.
The Python package installer, pip, and virtual environment manager, virtualenv, are also required. Windows installations of compatible Python versions include these tools. On Linux, pip and virtualenv may be provided as separate packages in your package manager. Alternatively, you may install them with the following commands:

python -m ensurepip --upgrade
python -m pip install --upgrade pip
python -m pip install --upgrade virtualenv
Enter fullscreen mode Exit fullscreen mode

If you encounter a permission error, run the above commands with the --user flag so that the modules are installed in your user directory, or use sudo to obtain the permissions to install the modules system-wide.

Note
It is common for Linux distros to use the executable name python3 for Python 3.x, and have python refer to a Python 2.x installation. You can adjust the command used to run your application by editing cdk.json in the project's main directory.

Creating a project

You create a new AWS CDK project by invoking cdk init in an empty directory.

mkdir my-project
cd my-project
cdk init app --language python
Enter fullscreen mode Exit fullscreen mode

cdk init uses the name of the project folder to name various elements of the project, including classes, subfolders, and files. After initializing the project, activate the project's virtual environment. This allows the project's dependencies to be installed locally in the project folder, instead of globally.

source .env/bin/activate
Enter fullscreen mode Exit fullscreen mode

Note:
You may recognize this as the Mac/Linux command to activate a virtual environment. The Python templates include a batch file, source.bat, that allows the same command to be used on Windows. The traditional Windows command, .env\Scripts\activate.bat, works, too. Then install the app's standard dependencies:

python -m pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Important
Activate the project's virtual environment whenever you start working on it. Otherwise, you won't have access to the modules installed there, and modules you install will go in the Python global module directory (or will result in a permission error).

Managing AWS construct library modules

Use the Python package installer, pip, to install and update AWS Construct Library modules for use by your apps, as well as other packages you need. pip also installs the dependencies for those modules automatically. To run pip without needing it installed in a special directory, invoke it as:

python -m pip PIP-COMMAND
Enter fullscreen mode Exit fullscreen mode

AWS Construct Library modules are named like aws-cdk.SERVICE-NAME. For
example, the command below installs the modules for Amazon S3 and AWS Lambda.

python -m pip install aws-cdk.aws-s3 aws-cdk.aws-lambda
Enter fullscreen mode Exit fullscreen mode

Similar names are used for importing AWS Construct Library modules into your Python code (just replace the hyphens with underscores).

import aws_cdk.aws_s3 as s3
import aws_cdk.aws_lambda as lam
Enter fullscreen mode Exit fullscreen mode

After installing a module, update your project's requirements.txt file, which lists your project's dependencies. It is best to do this manually rather than using pip freeze. pip freeze captures the current versions of all modules installed in your Python virtual environment, which can be useful when bundling up a project to be run elsewhere.
Usually, though, your requirements.txt should list only top-level dependencies (modules that your app depends on directly) and not the dependencies of those modules. This strategy makes updating your dependencies simpler. Here is what your requirements.txt file might look like if you have installed the Amazon S3 and AWS Lambda modules as shown earlier.

aws-cdk.aws-s3==X.YY.ZZ
aws-cdk.aws-lambda==X.YY.ZZ
Enter fullscreen mode Exit fullscreen mode

You can edit requirements.txt to allow upgrades; simply replace the == preceding a version number with ~= to allow upgrades to a higher compatible version, or remove the version requirement entirely to specify the latest available version of the module.
With requirements.txt edited appropriately to allow upgrades, issue this command to upgrade your project's installed modules at any time:

pip install --upgrade -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Note
All AWS Construct Library modules used in your project must be the same version.

AWS CDK idioms in Python

Props

All AWS Construct Library classes are instantiated using three arguments: the scope in which the construct is being defined (its parent in the construct tree), a name, and props, a bundle of key/value pairs that the construct uses to configure the resources it creates. Other classes and methods also use the "bundle of attributes" pattern for arguments.

In Python, props are expressed as keyword arguments. If an argument contains nested data structures, these are expressed using a class which takes its own keyword arguments at instantiation. The same pattern is applied to other method calls that take a single structured argument.

For example, in a Amazon S3 bucket's add_lifecycle_rule method, the transitions property is a list of Transition instances.

bucket.add_lifecycle_rule(
    transitions=[
        Transition(
            storage_class=StorageClass.GLACIER,
            transition_after=Duration.days(10))
        ]
)
Enter fullscreen mode Exit fullscreen mode

When extending a class or overriding a method, you may want to accept additional arguments for your own purposes that are not understood by the parent class. In this case you should accept the arguments you don't care about using the kwargs idiom, and use keyword-only arguments to accept the arguments you're interested in. When calling the parent's constructor or the overridden method, pass only the arguments it is expecting (often just kwargs). Passing arguments that the parent class or method doesn't expect results in an error.
Future releases of the AWS CDK may coincidentally add a new property with a name you used for your own property. This won't cause any technical issues for users of your construct or method (since your property isn't passed "up the chain," the parent class or overridden method will simply use a default value) but it may cause confusion. You can avoid this potential problem by naming your properties so they clearly belong to your construct. If there are many new properties, bundle them into an appropriately-named class and pass it as a single keyword argument.***

Top comments (0)

Read next

arjun98k profile image

The Future of Rust Programming and My Experience with Rust-Based Tools

arjun -

guru_prasanna_01 profile image

Python Day-26 List comprehension-Exercises

Guru prasanna -

arber profile image

Proposal: Standard Communication API Channels for AI Agents (AI Generated)

Arber -

talonx profile image

How to Configure a Remote Data Store for Prometheus

Hrish B -