DEV Community: Miguel A. Calles

Six Costly Findings in AWS Serverless Computing

Miguel A. Calles — Tue, 23 Jan 2024 16:00:00 +0000

I've worked in serverless computing for over five years and wanted to share some lessons learned.

By the term "costly," I don't mean necessarily dollars or how expensive it was. "Costly" could mean difficulties in troubleshooting, difficulties in architecture, and just pain points I've discovered over the years.

There are many more, but this post focuses on the costly findings that have stood out.

An overview of serverless

Read the following post for an overview of serverless.

https://medium.com/serverless-is-cool/a-super-quick-overview-of-serverless-52b1e5cf9e2f

Input injection

Until just recently, OWASP Top Ten had input injection as the top risk. It is still the top risk in the Serverless Top Ten.

The OWASP Top Ten API Security Risks has broken object-level authorization as the highest risk. This means that if I ask an API for data, it will give it to me. Then, if I ask it for another user's data, it'll give me that, too, even if I'm still logged in with my account. Since I'm not meant to get someone else's data, it seems appropriate that it would be a higher risk than input injection.

Input injection is still high on the list, and it would be important to address that first.

In the scenario above, we designed a Lambda function to get an update from the database to get an event from a DynamoDB table whenever the table data changes.

The Lambda function then sends a clear cache command to the API Gateway.

This workflow allowed the API Gateway to cache data and its cache to be cleared when there was new data. The DynamoDB table data stayed the same most of the time, and caching would reduce the number of read transactions on the table.

Unfortunately, this resulted in an accidental input injection attack. Somehow, the DynamoDB stream would resend old events. Some of these events were old. Furthermore, multiple old events were sent in a short period.

The Lambda function was designed to clear the API Gateway cache when it received an event. The numerous events resulted in multiple commands to the API. Shortly after, the API reached its rate limit and stopped responding to requests.

We experienced a self-induced denial of service attack!

We reworked our Lambda function when we realized our users could not get data because of this DoS event.

The Lambda function would validate each input rather than assuming it was "safe" to process. It would check the timestamp and only process events created within a few minutes. Furthermore, it would keep track of the last time it cleared the API cache. It would only clear the cache no sooner than five minutes than the previous time.

The image above shows some input injection risks and mitigations to consider.

Over-privileged policies

I joined an existing serverless project. I discovered all the Lambda functions within a Serverless Framework service used the default IAM role instead of having one least privileged role per function.

The above shows the example of a role with an over-privileged IAM policy.

I emulated an injection attack where I would send a malicious command to the AWS CLI. (The Lambda function code allowed me to send Linux commands to the operating system with the AWS CLI installed.) I was able to modify other parts of the AWS infrastructure.

An attacker could have issued a malicious command to IAM and started an account takeover.

We created an IAM role for each function and assigned a unique IAM policy that had the least privileges to allow the function to execute successfully.

I attempted the same malicious exploit, and the function threw an error.

The image above shows some over-provisioning risks and mitigations to consider.

Failures and service outages

It is easy to assume that AWS services will never go down or experience degradation. Designing for the "happy path" is the most logical approach. But, it is essential to consider the "negative path" where things go wrong.

One day, we experienced degradation in CloudWatch rules. (Since then, it was rebranded to EventBridge.) Some of our functions failed to work for hours.

Some functions were designed for the scheduled events to get third-party data to store it in our DynamoDB tables. The third-party data source only provided new data every six minutes. If we failed to save the data within that time, we would never have access to it again.

We lost data for hours. That affected our user experience and billing.

To remediate this, we started monitoring the health of the AWS services. We would be notified when there was an outage.

We made sure all functions exited gracefully by using try-catch blocks. We could log errors that could be monitored. If our monitoring noticed an increase in logged errors, that would be an indicator to investigate.

We explored having multi-region scheduled events. For example, if the North Virginia region was down, we could still get scheduled events from the Ohio region.

The image above shows some failure and service outage risks and mitigations to consider.

Cost engineering

We found our AWS bill was costly. For almost a year, we accepted it as the "cost of moving" to the cloud.

We decided to investigate whether we were making the best use of the AWS cloud and being as efficient as possible.

We observed our DynamoDB traffic patterns and realized they were irregular. We realized we would save money by moving to on-demand capacity even though it was more expensive than the provisioned capacity.

We realized we had old log and S3 files that were no longer needed. We took advantage of lifecycle rules to delete old, unnecessary data.

An initial version of an app used Kinesis, but we found it was more cost-effective to use SQS.

The image above shows some cost risks and mitigations to consider.

Encryption

We wanted to be super secure and enabled customer-managed KMS keys for all our data. We found our AWS bill increased considerably.

We discovered the AWS-managed keys were less expensive than the customer-managed keys (CMK). We did not have security requirements that required us to use CMKs. We moved all our encryption to AWS-managed keys where possible.

Better security did not provide business value in this case.

The image above shows some encryption risks and mitigations to consider.

Spaghetti

We sometimes created "serverless spaghetti" designs.

Much like "spaghetti code" that is difficult to trace and has complex dependencies, the same can happen in a serverless design.

The image above shows an unnecessarily complex design in an attempt to implement a "pure" and "clean" microservice design.

We reviewed our design and attempted to simplify it. The above shows how we simplified the previous design example. Getting the data directly from the table was cleaner than going through a microservice.

The image above shows some serverless spaghetti risks and mitigations to consider.

Watch the LayerOne presentation

Before you go

Here are other posts you might enjoy.

AWS CDK Serverless Cookbook: A Step-by-step Guide to Build a Serverless App in the Cloud

Using Docker for AWS CDK development: Have a consistent development environment for your team on any OS

Speed up AWS CDK deploys up to 80%: The ts-node package needed the speed boost

Build Robust Cloud Architectures: Applying Military Design Principles to Cloud Applications

Introduction to Cloud Computing Security: An excerpt from the Serverless Security book

We migrated from Serverless Framework to AWS CDK, and this is what we learned: Lessons from the AWS CDK Day 2023 presentation

Miguel A. Calles — Mon, 08 Jan 2024 17:39:00 +0000

I will be sharing what my team and I learned when we migrated one of our applications from the Serverless Framework to AWS CDK.

Introduction to serverless

What is serverless?

Serverless is a different way of developing. Rather than having a dedicated server (e.g., an EC2 instance) where you have to install the packages, configure runtimes, and cyber harden it.

With serverless, you can rely on services to do what you need (e.g., compute, database, and storage).

It doesn't mean you don't have servers. It means you rely more on your provider's services or functionality (e.g., AWS) provides.

Types of serverless services

There are different types of serverless services.

The most commonly known are for functions or code execution. That would be Lambda.

APIs (that would be API Gateway).

Databases (like DynamoDB).

Object storage (like S3).

There are also other things you can do with orchestration. You can use CloudWatch events, CICD with CodePipeline, CodeBuild, AI, and ML with SageMaker.

You can integrate all these services together depending on your architecture.

Why choose serverless?

There are different reasons, but it comes down to allowing you to be more agile and focus more on building and developing rather than maintaining, securing, and configuring.

It comes in handy when you're building prototypes, have a small team, or want to provide the most value with the least amount of maintenance effort.

Serverless computing, designs, and architectures are not silver bullets. Sometimes, dedicated servers are more beneficial in some designs. These are just some common reasons why you would want to use it.

An example of a serverless architecture on AWS

This is an example of a serverless architecture built on AWS.

We have a serverless website that runs a single-page application that a browser can use to run that web app.
You can use the CloudFront service to deploy a content delivery network that provides caching at edge routers. That way, the web application loads much faster when a user visits it.

You can use the S3 service for website hosting. You can connect the CloudFront and the bucket so any HTML, CSS, or JavaScript file and image file that is loaded from the bucket gets cached via the CloudFront service.

You can also build up your DNS (domain name service) on Route 53 or use an external domain name service. If you use Route 53, then the automatic deployment of any DNS records is managed through the AWS processes.

The backend can also be serverless by using an API gateway to send HTTP events to Lambda functions. A lambda function will run its code. It can get data from a database (like a DynamoDB table) if needed.

The web app will communicate with your backend, and then if you have a mobile app, that too can make API calls to the API gateway.

Using the Serverless Framework

History

To build our application, we previously used the Serverless Framework to build a serverless web app, an API, and its backend. We also built a mobile app that used its own serverless API.

We chose the framework because the team was highly experienced with serverless development and the Serverless Framework.

The architecture

This application had a web app and a mobile app. The web app had its own dedicated API, and so did the mobile app. They shared common backends, databases, and Lambda functions.

Serverless Framework services

The serverless framework is designed for services. When you use the Serverless Framework CLI create command, it creates a new service.

You run the create command in different folders if you want multiple services. This follows the microservices concept, which is very popular.

There were four key services: the platform, which is the core functionality of our application (i.e., the backend); the account service (i.e., the identity provider) that authenticates users; the APIs for the mobile app and the web app to get data from the backend; the web app since it could be standalone.

Platform service

The platform service allowed us to set up the hosted zone, which is where the DNS is set up.

We created our SSL certificates using the certificate manager.

Our Lambda functions provided the code to power our APIs and to do the initial setup when we did onboarding or set up the application in a new AWS account.

We used IAM for the security policies.

We used EventBridge for the background processes.

We also had a core set of DynamoDB tables shared across the application's different parts.

We used various plugins to do different things. You will notice that there was a common set of plugins that will show up in all services.

We built a reusable Serverless Framework service for use in future services and projects.

Accounts service

In the accounts service, we used Cognito. We used one user pool specifically for the web app and another one for the mobile app.

The web app had one type of user. The mobile app had another type of user. Each type of user had different roles and functionality. Therefore, we felt keeping them separate with different user pools was important.

There was also some data that we wanted to keep specific to the account that we chose not to store in Cognito. As a result, we had account DynamoDB tables.

APIs service

We used GraphQL for the API instead of using a RESTful API. We had dedicated APIs for the web app and mobile app.

Web app service

The web app is a serverless web app. It used an S3 bucket, CloudFront distribution, and any related certificates from the platform service.

Findings

We had pretty good success. We were able to deploy a minimum viable product pretty quickly.

We built a common library. We were using a mono repo with a common library folder and any common code shared across the different services.

We found that the deployment order could be tricky sometimes. Some plugins require you to run a pre-deploy command. Others required you to do a post-deploy command. Sometimes, it had dependencies on other plugins. Other times, one service relied on another service's resource that had not been created yet.

YAML was great, but as the file got bigger, reading it was a little confusing sometimes.

The most "annoying" part was creating the IAM policies. There were some plugins that were great. They worked well in Serverless Framework version 1, but there were breaking changes in version 2. Some plugins stopped working in version 3. Using the native CloudFormation way of creating IAM policies was the most reliable, but can be tricky for some.

Regardless, the Serverless Framework was still awesome to build and deploy. We created and deployed our MVP within months.

Moving to AWS CDK

As a team, in 2021, we decided to try out this AWS CDK. It was at version 1 at the time and at least over 100 in the minor version.

We had heard successes with CDK. We decided to experiment with CDK for future projects. We were building MVPs and trying out different product ideas internally.

We tried CDK on one and found that it was a different way of thinking than the Serverless Framework. We had to think about apps and not microservices. Each Serverless Framework service was a CloudFormation stack, but a CDK app could consist of multiple stacks. Both CDK and Serverless Framework use CloudFormation in the underlying deployment method, but their organization is different.

We also found using SSM parameters was much easier to work with in CDK than in Serverless Framework.

In Serverless Framework, we used variables inside of the YAML code. We had to use a plugin to help interpret those variables since native support was limited.

Another nice finding was building reusable components. CDK has level one (L1) and level two (L2) constructs that are ready to be used to deploy certain resources on services. You can also combine level two and level one constructs to build a level three construct. L3 constructs provide a lot better reuse.

Custom level 3 constructs

These are some of the L3 constructs we built.

We created a construct for the hosted zone and certificate so we could create the DNS and any required certificates.

We created a function and log group construct. We found that log groups are automatically created the first time a function is invoked. By creating the log group, we can configure its retention settings and other settings. Doing this will help lower costs by deleting old logs.

We created an L3 construct for the web app app. We also used an existing L3 construct for deploying the web app.

We created a CICI pipeline to deploy our CDK apps. We would deploy applications in multiple AWS accounts, and they were the same app, so having a consistent pipeline was important.

Serverless Framework services to CDK apps

We needed to think about how we wanted to reorganize our Serverless Framework apps to CDK apps.

Web app CDK app

The web app service was the one service that had the least dependencies. We could deploy it by itself without a custom domain and have no external dependencies.

Backend CDK app

We found most of our services and their resources could be in a single backend CDK app since they are related to each other.

We started with databases since we had a Serverless Framework service with just database tables. We created a database stack.

We moved all the account services to a mobile app authentication stack and a web app authentication stack.

We also moved the API services to a web app API stack and a mobile API stack. We specified the database and authentication stacks as dependencies. Doing this allows CDK to determine the deployment order automatically.

We took a new approach to the pre-deploy and post-deploy commands. In Serverless Framework, we used plugins used in the CLI. For CDK, we created a setup stack that we would run as part of the deployment order. It would run the setup for the first deployment. It would not run again after that since there were no changes to any of the CloudFormation resources.

What did we learn from migrating to AWS CDK from Serverless Framework?

More stuff comes "out of the box"

We were able to use a lot of constructs that just came out of the box with CDK. And we could use these constructs to create L3 constructs.

We did not need any plugins. Plugins were essential in Serverless Framework but were not required in CDK.

Serverless Framework provides plugins for creating hosted zones and certificates, but this came native to CDK.

Building Lambda function packages also came native to CDK.

Better organization

We were able to organize our CDK apps better instead of having four main services. And we eliminated the need for pre-deploy and post-deploy commands for each service. We tried to put all the dependencies in their respective stacks. For those we couldn't, we specified which stacks depended on other stacks.

CDK also has a synthesis that is able to catch issues like circular dependencies before you try to deploy.

Working with SSM was easier, and we did not need dotenv files or plugins to work with environment variables.

Simpler deployments and better security

We also found it easier to work with deployments and security.

We only needed one CDK deploy command for each CDK app versus having pre-deploy, deploy, and post-deploy commands for each Serverless Framework service.

Creating IAM roles could be as simple as using a grant function.

CICD deployment was simpler. We created a build spec for each CDK app and used our L3 construct to create the pipeline.

Watch the CDK Day 2023 presentation

Before you go

Here are other posts you might enjoy.

https://medium.com/chapter-by-chapter/aws-cdk-serverless-cookbook-ebook-1d4d4e0488c

https://miguelacallesmba.medium.com/using-docker-for-aws-cdk-development-7054086deb3d

https://aws.plainenglish.io/speed-up-aws-cdk-deploys-up-to-80-c47afad1c18

https://medium.com/illumination/build-robust-cloud-architectures-d512fb9eae1a

https://medium.com/serverless-is-cool/introduction-to-cloud-computing-security-a37b6c4bc4f1

Setting Up our AWS Account and CDK Environment

Miguel A. Calles — Sat, 23 Sep 2023 16:14:01 +0000

This is chapter 1 of the AWS CDK Serverless Cookbook.

We will focus on setting up our AWS account and development environment to create our first CDK environment. We assume you already have access to an AWS account or can create one. We will use Docker[1] Desktop to help eliminate differences in Windows, Linux, and MacOS computing environments. We will also use GitHub as a git hosting provider for source control.

Creating a New Directory

We will create a new directory named cookbook where we put all our code. Create the cookbook directory in any location you prefer. We will reference files starting with the directory name. For example, we will reference a readme file inside that directory as the cookbook/README.md path.

Using GitHub and GitHub Desktop

Setting up an account with GitHub is not necessary. We will use GitHub to allow the reader to download the code used in this book.

The GitHub repository that will host the code from this book may be found at https://github.com/miguel-a-calles-mba/aws-cdk-serverless-cookbook.

The cookbook directory is the top-level directory in our repository. For example, cookbook/README.md will be README.md inside the repository. When you clone the repository onto your machine, the directory will likely be called aws-cdk-serverless-cookbook. If it makes it easier, rename the directory to cookbook so there is a one-to-one match.

Feel free to install GitHub Desktop to help with cloning the git repository and managing your own git respository.

Setting up Docker Desktop

We will use Docker Desktop to create a Linux container. A container is like a mini-computer that runs on our computers. The container runs an operating system with minimal dependencies and no graphical user interface. Running the same container configuration on a Windows, Mac, and Linux computer should provide the same environment. I have noticed differences in containers running 32-bit, 64-bit, and ARM architectures for some containers, but I have yet to find issues for the containers I used for CDK development.

To install Docker Desktop, visit https://www.docker.com/, download the version of Docker Desktop that will work on your computer, and follow the installation instructions. You will be able to run the docker command in your terminal application after successfully installing Docker Desktop.

We will use the docker command to control our containers. Specifically, we will use the Docker Compose feature in Docker Desktop to make it easier to work with containers. We will use the docker compose command.

Creating Our Container

Let’s create a file named cookbook/docker-compose.yml and add the following code.

version: "3"
services:
  nodejs:
    image: "cimg/node:18.15"
    user: "circleci"
    working_dir: "/home/circleci"
    volumes:
      - "./:/home/circleci"
    command: "bash"

This configuration file lets us download a container image and run a Docker container.

We will run the following command to enter our container in our terminal application.

docker compose run --rm nodejs bash
# or
docker compose run --rm nodejs

We will see our terminal change to something like the prompt below.

circleci@a1b2c3d4e5f6:/home/nodejs$

We now have a Linux terminal we will use for the rest of the book.

(Note: The --rm option in the docker compose command will delete the container after exiting it. This will help keep our computer clean.) Learn more about this development approach from the post below.

User Docker container to run Node.js and global npm packages | Better Programming

Protect your system from vulnerabilities

betterprogramming.pub

Setting Up the AWS Account

AWS accounts are free, but it requires a credit card to register. AWS also offers free tiers for many services. Most services we will use in this book are eligible for the free tier. Some services are only eligible for the free tier for the first 12 months after creating an AWS account. It is up to the reader to determine whether to use an existing or new AWS account.

Setting Up Identity and Access Management (IAM)

We will need IAM permissions to deploy our CDK app. We can create a user using the AWS IAM service or AWS Identity Center. The AWS Identity Center provides a more secure way to manage users and their IAM permission. For simplicity, we will create an IAM user and IAM policy it will use.

To go to the AWS IAM service:

Log into the AWS console
Search for the IAM service and select it

To create an IAM policy:

Go to Access management > Policies
Click the “Create policy” button
Click the “JSON” tab
Paste the JSON code below into the policy editor
Click the “Next” button
Click the “Create policy” button

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "cdk",
      "Effect": "Allow",
      "Action": [
        "acm:*",
        "apigateway:*",
        "cloudformation:*",
        "cloudwatch:*",
        "dynamodb:*",
        "ecr:*",
        "events:*",
        "iam:*",
        "lambda:*",
        "logs:*",
        "s3:*",
        "s3-object-lambda:*",
        "ssm:*",
        "route53:*"
      ],
      "Resource": "*"
    }
  ]
}

The IAM policy above gives us full permission to create and update the services used in our serverless application. (Note: The CDK bootstrap command needs “ecr” permission.) We will update the IAM policy as needed in the following chapters.

To create an IAM user:

Go to Access management > Users
Click the “Add users” button
Set the username to “cookbook”
Click the “Next” button
Check “Attach policies directly”
Check the “cookbook” policy name in the “Permissions policies” section
Click the “Next” button
Click the “Create user” button

To create the IAM access key:

Click on the “cookbook” IAM user
Click the “Security credentials” tab
Click the “Create access key” button in the “Access keys” section
Check “Command Line Interface (CLI)”
Click the “Next” button
Check the “I understand the above recommendation and want to proceed to create an access key.”
Click the “Next” button
Click the “Create access key” button
Click the “Download .csv file” button
Click the “Done” button

We will paste the access key values into our container’s terminal similar to the example below.

export AWS_ACCESS_KEY_ID="A...Y"
export AWS_SECRET_ACCESS_KEY="7...B"

The CDK app will use the access key to deploy the app with the permissions from the IAM policy.

As a good security practice, deactivate the access key when not using it and activate it when you need to use it.

To activate/deactivate the IAM access key:

Go to the IAM user
Click on the “Security credentials” tab
Go to the “Access keys” section
Click the “Actions” dropdown
Select “Activate” or “Deactivate”

Feel free to learn about the AWS Identity Center and create a user with the IAM policy above. This would be far more secure than using an IAM access key. Try experimenting with the AWS Cloud Shell available in the AWS console.

Creating Our CDK app

CDK supports JavaScript, TypeScript, Python, Java, C#, and Go. TypeScript provides the ease of JavaScript coding with the power of data types like Java. Also, the CDK framework is written in TypeScript. We will use TypeScript to create our CDK app for these two reasons.

Creating the CDK app

In the container’s terminal, run the following commands to create CDK app:

# paste the AWS access key information
mkdir ~/cdk-app
cd ~/cdk-app
npx cdk init --language typescript
# follow any on screen prompts

We now have our CDK app.

Bootstrapping the AWS account

We must bootstrap the AWS account to allow CDK to deploy the app. Bootstrapping creates a CloudFormation stack that CDK will use.

In the container’s terminal, run the following command to run the CDK bootstrap command to prepare the AWS account:

# paste the AWS access key information
npm run cdk bootstrap

Our AWS account now has the “CDKToolkit” CloudFormation stack, and we can deploy our CDK app in the next chapter; see Figure 1–1.

Figure 1–1. The CDKToolkit stack exists in the CloudFormation service.

Chapter Review

We created the directory to store all our code and shared the location of this book's GitHub source code repository. We downloaded and set up Docker Desktop to use a container when deploying our CDK app. We set up our AWS account to have an IAM user and access key that our CDK app will use. We used the access key to set up our AWS account to deploy CDK apps.

Next, we will describe the CDK app and prepare it to start building our application. The next section is coming soon.

Describing the CDK Application | Chapter by Chapter

Miguel A. Calles ・ Jun 24, 2023 ・
Medium

Before you go

Subscribe to my mailing list to get new chapters delivered to your email.

Go to the “AWS CDK Serverless Cookbook” table of contents.

AWS CDK Serverless Cookbook: A Step-by-step Guide to Building a Serverless App in the Cloud

Miguel A. Calles ・ May 28

#aws #serverless #cloud #cdk

Endnotes

[1] Docker is a registered trademark of Docker, Inc.

An Overview of AWS, Serverless Computing, and CDK

Miguel A. Calles — Sat, 23 Sep 2023 15:56:55 +0000

This is the "Introduction" chapter of the AWS CDK Serverless Cookbook.

This book will provide a step-by-step guide to creating a serverless application using Amazon Web Services. We will create a serverless website with databases, application programming interfaces, and processes to analyze data.

An Overview of Amazon Web Services

Amazon Web Services1 is a cloud computing service by Amazon. Cloud computing is a model where computing and data services (e.g., servers and storage) are hosted by another company and accessed over the Internet. Traditional models used on-premise hosting, where all servers and data storage are hosted inside the buildings where organizations run their operations. Some organizations have a hybrid model using a mix of on-premise and cloud-based solutions. This book will focus on cloud computing even though creating a serverless solution on-premise is possible.

AWS became popular because it offered low-cost cloud-based storage and virtual servers in various geographical locations worldwide. Developers could create and delete web servers, databases, and storage quickly and efficiently. As their applications grew, they could easily scale their resources. As demand increased, they could add more servers to meet the demand. As demand decreased, they could remove unused or underutilized servers. AWS simplified development compared to the limitations of physical space, utilities, support staff, and procurement in the on-premise model.

In 2014, AWS introduced its AWS Lambda. Lambda allowed developers to run code without having to provision a server. The introduction of Lambda introduced “serverless computing.”

An Overview of Serverless Computing

The term “serverless” has evolved over the years. The initial definition with the launch of AWS Lambda was focused on computing (i.e., running code) with less focus on servers. Whether or not Lambda uses servers to run the service is irrelevant. Developers did not need to worry about creating a server, installing the dependencies and packages, keeping it up to date, and hardening security settings. Their main focus was uploading code to the Lambda service to create a function and having it execute some business logic.

Over the years, serverless has been associated with certain key benefits:

Faster development
On-demand services
Improved elasticity
Increased scalability
Simplified maintenance
Reduced attack surfaces

These benefits came from the event-driven architectures typically used for serverless applications. An event-driven architecture assumes that an event drives system activity. For example, a computer will send an HTTP request to an Application Programming Interface (API). The API will forward the HTTP event to a Lambda function. The function will execute the code and send a response to the API. The API will forward the response to the computer. In this example, the API and Lambda functions will have no activity until it gets an HTTP request.

Microservices architectures can be used in serverless computing. This design paradigm allows us to divide resources into logical groups. For example, we can have an API and Lambda functions for creating, reading, updating, and deleting user data. The application will make API calls to the microservices to process relevant data.

Serverless applications can suffer from overly complex architectures that are difficult to maintain. For example, an HTTP event is sent to an API. The API forwards the request to a Lambda function. The Lambda function sends an event to a queue and a response to the API. Another Lambda function polls the queue and uses its data to write data to a database. The database sends an event to another Lambda function that writes data to an S3 bucket. The S3 bucket sends an event to another Lambda function that updates a value in a parameter store. We can easily fall into these confusing design traps when we develop a serverless application ad-hoc.

We will focus more on building on our application than architecture design in this book. This book is meant to be a book for practitioners who want to learn to make a serverless application step-by-step. We will keep the serverless application design as simple as possible. We will use the Keep It Super Simple (KISS) principle.

Read a more detailed introduction to cloud computing and security from the Serverless Security book.

Introduction to Cloud Computing Security | Serverless is Cool

Miguel A. Calles ・ Jun 6, 2023 ・
Medium

An Overview of Infrastructure as Code

Over time, cloud developers realized creating cloud resources and deploying the application systematically was essential. Programmatic deployments were already typical in software, but early cloud computing lacked this capability. AWS users had to configure cloud resources and applications using the web-based AWS console.

In 2010, AWS launched AWS CloudFormation, allowing users to write their cloud configurations in JSON and YAML file formats. A user would use the AWS command line interface (CLI) or software development kit (SDK) to request CloudFormation to create resources using the CloudFormation template. Teams could now use “code” to deploy their cloud infrastructure and maintain their cloud configuration in source code repositories.

In 2015, the Serverless Framework[2] was launched to simplify the creation of CloudFormation templates for serverless computing. The framework provided a simple custom YAML template, making creating and deploying serverless designs easy.

In 2018, AWS created the AWS Cloud Development Kit (CDK). Rather than writing CloudFormation templates, developers can create their cloud resources using JavaScript, TypeScript, Python, Java, C#, and Go. Unlike the Serverless Framework, CDK allows writing code to deploy any AWS resource that can be deployed using CloudFormation. The Serverless Framework can deploy CloudFormation templates for non-serverless cloud resources. Both frameworks have their benefits and drawbacks. Please read the post below for more details. We will create our cloud infrastructure with code with the AWS CDK framework.

Serverless vs. AWS CDK: Five Major Differences | JavaScript in Plain English

The Serverless Framework is the most popular framework for building serverless applications, but does how it compare to the new AWS CDK?

javascript.plainenglish.io

An Overview of the AWS Cloud Development Kit

The AWS CDK framework allows us to create infrastructure as code with CDK apps. A CDK app is used to deploy a group of AWS cloud resources. A CDK app is structured into the following components:

The executable bin file that runs the application
Stacks that have a group of cloud resources
Cloud resources defined by constructs

A CDK stack will create a CloudFormation stack. Thus a CDK app allows us to develop multiple CloudFormation stacks using the CDK app code.

CDK provides constructs that configure cloud resources. There are three types of constructs:

Level 1 constructs allow us to configure a CloudFormation resource directly. The L1 construct provides the same properties as the CloudFormation template.
Level 2 constructs are an abstraction of L1 constructs to configure a group of CloudFormation resources to deploy a logical group. For example, the aws_lambda package provides the Function construct that will create a Lambda function and the CloudFormation resources it needs. AWS writes the L2 constructs.
Level 3 constructs are the next level of abstraction, allowing us to group L2 constructs. For example, we can create an L3 construct to create a Lambda function, the related CloudWatch log group, and add custom tags. We create constructs or get them from Construct Hub.

The concepts will become more apparent as we build our serverless application.

Chapter Review

We quickly reviewed AWS, serverless computing, the infrastructure as code methodology, and AWS CDK. As we build our application, the following chapters will expand on relevant cloud computing and serverless concepts. In the next chapter, we will set up our environment to create our first CDK application.

Next, we will set up our AWS account and development environment to create our application.

Setting Up our AWS Account and CDK Environment

Miguel A. Calles ・ Sep 23

#aws #cdk #serverless #books

Before you go

Subscribe to my mailing list to get new chapters delivered to your email.

Go to the “AWS CDK Serverless Cookbook” table of contents.

AWS CDK Serverless Cookbook: A Step-by-step Guide to Building a Serverless App in the Cloud

Miguel A. Calles ・ May 28

#aws #serverless #cloud #cdk

Endnotes

[1] Amazon, Amazon Web Services and AWS are registered trademarks of Amazon Web Services, Inc.

[2] Serverless Framework is a registered trademark of Serverless, Inc.

Using Docker for AWS CDK development

Miguel A. Calles — Sun, 16 Jul 2023 16:14:18 +0000

Have a consistent development environment for your team on any OS

Support me by reading this post on Medium.

Introduction

In early 2022, I was starting a new project on AWS. I was using AWS CDK for about one year at this time. In a previous project, the team members used a mix of macOS and Linux computers. It was only a matter of time before a Windows user joined the team. Having a consistent development environment was beneficial. Based on previous experiments with Docker, I decided to use a Docker container as that development environment.

https://betterprogramming.pub/stop-installing-node-js-and-global-npm-packages-use-docker-instead-42597990db13

About one year later, in early 2023, I worked on a different CDK project. I started the project using macOS. About one month later, the team leader gave me a Windows laptop for me to use going forward. It was super simple to move my development environment. I installed Docker and git, cloned the git repository, ran the Docker command, and I was able to start developing right away. Using Docker containers saved me a lot of time.

Here is how I set up my Docker container

In my git repository, I created a docker-compose.yml file where I added the following code:

version: '3'
services:
  nodejs:
    # used for local development
    image: 'cimg/node:18.15'
    user: 'circleci'
    working_dir: '/home/cdk'
    volumes:
      - './:/home/cdk'
    command: bash

In the terminal, I ran the following command to start the container.

docker compose run --rm nodejs

Now I have a terminal running within the container.

I go to my AWS CDK app folder.

cd my-cdk-app

I paste my AWS credentials into the terminal. I can now run typical CDK commands.

npm run cdk synth
npm run cdk deploy
cdk synth
cdk deploy
# or however you have your cdk command execution set up

Conclusion

The setup was pretty simple. Now we have a consistent development environment regardless of the operating system. Leave a comment and let me know your experience.

Before you go

Here are other posts you might enjoy.

AWS CDK Serverless Cookbook: A Step-by-step Guide to Build a Serverless App in the Cloud

Speed up AWS CDK deploys up to 80%: The ts-node package needed the speed boost

Build Robust Cloud Architectures: Applying Military Design Principles to Cloud Applications

Introduction to Cloud Computing Security: An excerpt from the Serverless Security book

Speed up AWS CDK deploys up to 80%

Miguel A. Calles — Sun, 16 Jul 2023 14:21:24 +0000

TL;DR

The ts-node package needed the speed boost

Support me by reading this post on Medium.

Introduction

The Amazon Web Services Cloud Development Kit is an infrastructure-as-code solution. CDK allows us to write code (in JavaScript, TypeScript, Python, for example) to programmatically deploy resources in our AWS account.

CDK is written in TypeScript and uses the ts-node package to deploy. When we run the cdk command in the terminal, the npm package uses the ts-node package to transpile the CDK code before running it.

The issue

I was finding it was taking “forever” when running a cdk deploy or cdk synth command. When I was troubleshooting or developing code, I wanted faster deploys. I was using the cdk watch command to hotswap changes during development. This was a “little” faster but still felt super slow. I wanted to find out why these CDK commands were taking so long.

This was the solution

I will share the solution upfront. The rest of the post will share my findings.

I installed a new ts-node transpiler using npm.

npm i -D @swc/core @swc/helpers regenerator-runtime

I updated my tsconfig.json file to tell ts-node to use the new SWC transpiler. I added the ts-node property and the swc property under it.

{
  "ts-node": {
    "swc": true
  }
}

I noticed a significant improvement when running cdk and npx ts-node commands in the terminal.

Keep in mind this note from the SWC website:

SWC only transpiles the code and doesn’t perform type checking. Therefore, it’s recommended that you continue to use tsc for detecting any type errors.

How I found the solution

I realized that ts-node could be the issue. I write command line scripts with TypeScript. I use the npx ts-node myfile.ts command to execute them. They took a while to execute but did not think much about it. When I was getting frustrated by the CDK command start times, I started to realize the TypeScript script execution time was slow too.

I decided to investigate if they were speed issues with the ts-node package.

I found a GitHub issue where others expressed their concerns with the slowness of ts-node.

https://github.com/TypeStrong/ts-node/issues/1070

The discussion taught me about the SWC transpiler that can speed up the time ts-node uses.

https://typestrong.org/ts-node/docs/swc/

I installed the SWC packages and updated my tsconfig.json file. I noticed a significant improvement when using cdk deploy, cdk synth, cdk watch, and npx ts-node commands in the terminal. Development time was much faster now.

I updated my CI/CD pipeline that deploys multiple CDK apps and runs various TypeScript command line scripts. My CI/CD pipeline execution time dropped from 345 seconds to 74 seconds when I added the ts-node SWC setting. Wow! That’s over 75%.

I needed to share this finding with the CDK community. I submitted a GitHub pull request to add the SWC settings to the TypeScript app template.

https://github.com/aws/aws-cdk/pull/25089

If the pull request is approved and merged, new CDK apps created by the cdk init command will get this speed boost.

Empirical testing

Let’s do some testing to see the effects of the SWC transpiler.

Let’s create a folder for our CDK apps. We will use the ~/cdkapps folder. Note: My setup uses Docker rather than installing Node.js directly onto my machine.

cd ~
mkdir cdkapps

The first test

We will create a new CDK app.

cd ~/cdkapps
mkdir test1-cdk
cd test1-cdk
npx cdk init app --language typescript

Let’s test the time it takes to synthesize an empty app.

time npm run cdk synth

The command sent this output:

real 0m38.681s
user 0m46.195s
sys 0m1.769s

It took 46 seconds to synthesize a CDK app.

Now let’s test with the SWC transpiler. (The command assumes the steps to add the SWC transpiler were already done.)

time npm run cdk synth

The command sent this output:

real 0m9.562s
user 0m9.105s
sys 0m0.703s

It took 9 seconds to synthesize a CDK app when using SWC. That’s an 80% improvement!

The second test

Now let’s create a new CDK app with one TypeScript Lambda function.

We will create a new CDK app.

cd ~/cdkapps
mkdir test2-cdk
cd test2-cdk
npx cdk init app --language typescript

Let’s add the packages to help with adding a Lambda function.

npm i -D @types/aws-lambda

Let’s create the Lambda function.

touch function.ts

Let’s create the function code.

/* ~/cdkapps/test2-cdk/function.ts */
import { Handler } from 'aws-lambda';

export const handler: Handler<Event> = async function(event, _context) {
  return;
}

Let’s create the function resource.

/* ~/cdkapps/test2-cdk/lib/test2-cdk-stack.ts */
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as path from 'path';

export class Test2CdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new cdk.aws_lambda_nodejs.NodejsFunction(this, 'function', {
      functionName: 'function',
      handler: 'handler',
      runtime: cdk.aws_lambda.Runtime.NODEJS_18_X,
      entry: path.resolve('__dirname', '..', 'function.ts'),
    });
  }
}

Let’s test the time it takes to synthesize an app on a bare TypeScript Lambda function.

time npm run cdk synth

The command sent this output:

real 0m49.314s
user 0m58.874s
sys 0m3.818s

It took 58 seconds to synthesize a CDK app.

Now let’s test with the SWC transpiler. (The command assumes the steps to add the SWC transpiler were already done.)

time npm run cdk synth

The command sent this output:

real 0m18.985s
user 0m18.625s
sys 0m2.186s

It took 18 seconds to synthesize a CDK app when using SWC. That’s a 67% improvement.

Conclusion

Consider adding the SWC transpiler for ts-node to your AWS CDK apps. The time results will vary depending on your setup.

Before you go

Here are other posts you might enjoy.

AWS CDK Serverless Cookbook: A Step-by-step Guide to Build a Serverless App in the Cloud

Stop Installing Node.js and Global Npm Packages, Use Docker Instead: Protect your system from vulnerabilities

Cloudflare v. AWS: A Comparison of Their Serverless Offerings: They both have their benefits and downsides

How I Caused an Amazon API Gateway Denial of Service: The DoS attack was caused by a Lambda function

Photo by PAUL SMITH on Unsplash

AWS CDK Serverless Cookbook: A Step-by-step Guide to Building a Serverless App in the Cloud

Miguel A. Calles — Sun, 28 May 2023 13:54:11 +0000

Foreward

I started developing serverless applications on Amazon Web Services in 2018. The team used the Serverless Framework heavily. The framework made it simple to begin developing serverless applications on AWS. Over the years, I wanted greater control of provisioning and updating cloud resources. In 2020, my team and I experimented with the AWS CDK framework to deploy our applications. It was a different experience and required more lines of code than the same setup in the Serverless Framework. An AWS CDK developer needed to understand cloud provisioning better to use it effectively. I found the CDK refreshing and wrote a blog post sharing five significant differences between the two frameworks.

Serverless vs. AWS CDK: Five Major Differences | JavaScript in Plain English

The Serverless Framework is the most popular framework for building serverless applications, but does how it compare to the new AWS CDK?

javascript.plainenglish.io

Both frameworks were at version 1 at the time. We were not locked into any AWS deployment framework. As a team, we decided to continue using AWS CDK from then on. After using CDK in multiple projects, we found it allowed us to create patterns that we could reuse. We put standard deployment methods or cloud resources into reusable constructs. Spinning up new projects thus became faster and more efficient. Furthermore, we were able to make upgrades to configurations by simply updating the constructs. For example, when AWS announced deprecations to software runtimes, we would update the runtime in the construct, and all projects would benefit.

I wrote this book to share my experience using AWS CDK to build cloud-native serverless applications. I hope you find my experience over the past few years beneficial in helping you start building a serverless app with CDK.

You may be wondering why I am writing a book on Medium. These posts explain why.

Would you read a book on Medium if it were posted one chapter at a time? | by Miguel A. Calles | ILLUMINATION | Medium

Miguel A. Calles ・ May 12, 2023 ・
Medium

Writing a tech book on Medium, part 2 | by Miguel A. Calles | ILLUMINATION | Medium

Miguel A. Calles ・ May 7, 2023 ・
Medium

Introduction: An Overview of AWS, Serverless Computing, and CDK

A quick overview of Amazon Web Services, serverless computing, infrastructure-as-code, and AWS CDK.

An Overview of AWS, Serverless Computing, and CDK

Miguel A. Calles ・ Sep 23

#aws #cdk #serverless #books

Chapter 1: Setting Up our AWS Account and CDK Environment

We will set up our AWS account and development environment to create our application.

Setting Up our AWS Account and CDK Environment

Miguel A. Calles ・ Sep 23

#aws #cdk #serverless #books

Chapter 2: Describing the CDK Application and Serverless Architecture

We will review and modify the CDK app structure.

https://medium.com/chapter-by-chapter/describing-the-cdk-application-914efa958770

Chapter 3: Describing the Serverless Architecture

We will describe the serverless website we will build using CDK and the cloud resources needed to make it functional and practical.

Coming soon…

Chapter 4: Launching a Website

We will create a website and make it accessible on the Internet.

Coming soon…

Chapter 5: Creating an API

We will build an application programming interface the website will call to get information.

Coming soon…

Chapter 6: Adding a Database

We will create a serverless database and have the API read its data.

Coming soon…

Chapter 7: Adding Authentication

We will update the website to allow users to sign up, log in, and access protected pages.

Coming soon…

Chapter 8: Saving Data

We will allow logged-in users to save data to the website.

Coming soon…

Chapter 9: Analyzing Data

We will create a repeatable process to analyze the data.

Coming soon…

Chapter 10: Conclusion

We will discuss next steps and other things to consider.

Coming soon…

Before you go

Subscribe to my mailing list to get new chapters delivered to your email.

https://miguelacallesmba.medium.com/subscribe

Is your dream to write a book but you are worried you do not have what it takes? You can start writing one chapter at a time. Submit your drafts to the Chapter by Chapter publication and start realizing your dream.

https://medium.com/chapter-by-chapter/how-to-start-writing-your-book-with-chapter-by-chapter-1ec05495dcf4

Be careful when using .env files

Miguel A. Calles — Mon, 26 Sep 2022 16:19:11 +0000

Not even within 24 hours, a recently launched website was being sniffed for secrets. A bot was searching for a .env file where some projects store secrets. Fortunately, this website was protected by Cloudflare.

What can we learn from this?

The .env file should be treated as a potential source of a cyber breach. We should be cautious about what data we store there.

Should we put secrets in the `.env` file?

No, when possible.

Where should we store and use secrets?

Store secrets in a secrets manager or database. Limit the use of secrets to the backend code (i.e., the code not used in the frontend web application code).

How can we protect the `.env` file?

We should use a Web Application Firewall to stop external HTTP requests from reading this file. The file should only be read by the application code.

Originally published on Medium

When to use ?? and || in JavaScript and TypeScript

Miguel A. Calles — Sat, 17 Sep 2022 00:26:17 +0000

In this short post, I want to share a simple way to remember when to use ?? and || when checking values in JavaScript and TypeScript code.

Note: You can read this post on Medium too

When to use `??` when checking a value?

The ?? is an or operator where the left side must be empty before returning the right side.

The question mark ? is like asking yourself, “Does the value exist?”

var ?? varMustBeEmptyToReturnThisVar

Examples

Suppose you get an object where some properties are optional.

const obj0 = { required: 0 };
const { optional } = obj0;
console.log(optional ?? 'no optional property exists');
// returns 'no optional property exists'

Be careful when the property returns a falsy value.

const obj1 = { required: '' };
const { required } = obj1;
console.log(required ?? 'no required property exists');
// returns ''

When to use `||` when checking a value?

The || is an or operator where the left side must be falsy before returning the right side.

The vertical bar | is like putting up a barrier where only truthy values can stay on the left side.

var || varMustBeFalsyToReturnThisVar

Example

Suppose you get an object where some properties are falsy.

const obj2 = { required: '' };
const { required } = obj2;
console.log(required || 'required property is falsy');
// returns 'required property is falsy'

Conclusion

I sometimes misuse the ?? and || and thought I should write this reminder to myself. Hopefully that helps you too.

Before you go

These are other articles you might enjoy.

Stop Installing Node.js and Global Npm Packages, Use Docker Instead

Five scary Linux commands you should not run

How to Remove Sensitive Data and Plaintext Secrets From GitHub

How to use CryptoJS and Cookies to Work with Secrets in Postman

Photo by Towfiqu barbhuiya on Unsplash

Present like a boss with transcription and analytics based on Deepgram

Miguel A. Calles — Fri, 01 Apr 2022 00:14:46 +0000

Introduction

I learned about Deepgram from the hackathon on DEV. I had not heard of them prior. After reviewing their docs, it seemed to make speech to text much easier to adopt than some other services. Other services seem complex to build upon and building simple uses would take a long time. Deepgram seems like it could shorten development of simple use cases.

My Deepgram Use-Case

Have you ever heard of someone giving a talk or a presentation and been impressed with their ability to present? You may have noticed they did not use filler words like "umm", "so", "and", etc.

Think about the last time you heard a presentation and the speaker used these filler words frequently. Did you find the presentation hard to follow? Did the filler words seem to get louder and louder?

Organizations like Toastmasters count how many filler words people use in their presentations. What if you could practice a presentation with an app and it would give you stats on filler words?

What if the app could give you a score on sentiment? What if it could tell you how well your presentation would be received?

Dive into Details

The app would Deepgram's text to speech. Either the prerecorded or live streaming options would work.

Initially, the app would count filler words. The app would give statistics per presentation and over time. Whenever the number of filler words drops by every 10%, the presenter would get a badge. Another badge would get a special badge when they present without any filler words.

Based on this Deepgram tutorial, the app could provide statistics on total pause time per presentation and even pauses between words. The app could suggest optimal pause lengths and frequency for a good presentation.

Another feature would be add give statistics similar to reading level. Once the presentation is text form, the text could be analyzed for reading level. The average reading level would be provided and specifics for each sentence. The assumption is that the reading level would indicate what type of audience would be able to understand the presentation. A high reading level would indicate that the presentation is meant for expects, while a lower reading level would indicate a general audience. A presentation with the lowest reading level might be best appropriate for children.

The apps analytics will evolve based on presentation best practices and user feedback.

Conclusion

Deepgram's text to speech API seems to make transcriptions and analyzing the speech much easier than other platforms. Their capability can be used to help individuals improve their presentation and speaking skills by providing meaningful statistics and recommendations.

Photo by Teemu Paananen on Unsplash

If you build it, you should break it

Miguel A. Calles — Wed, 02 Mar 2022 16:57:01 +0000

You may have heard, “If you build it, they will come.”

Have you heard, “If you build it, I will break it?”

This is a skill I developed over my career to build better systems. It is a skill that took me a while to learn I had.

Shortly before I graduated from college, I decided to leave the material science industry because I had a tendency to break stuff. When working with toxic substances, this can be hazardous. As a result, I changed to the systems engineering field.

It turned out I had a knack for breaking stuff there too. For a long while, I avoided working on things I could break. Yet, I found I wasn’t growing as much as my peers. So I decided I needed to do some hands-on work.

As it turns out, I was breaking stuff, and people were getting mad. They had to fix deficiencies and errors I discovered. A time came people started expecting me to fix what I broke. That was a pivotal moment in my career.

I had to learn how to fix things, and I wasn’t trained as an engineer. My undergrad was in material science. I started learning how to fix things and became more knowledgeable about the technology. I eventually learned how to find defects, flaws, and deficiencies in designs.

I turned what I thought was a flaw into a skill. I could help build more reliable systems by “breaking them.” Penetration testers do this type of work. They look for vulnerabilities and ways to break “in.” I had a knack for finding ways to break “down” systems. If I can learn why a system could falter, I can help prevent it.

My recommendation for your systems: Plan to break your system before it breaks on you.

Want to Connect?

Miguel is a Principal Engineer and the author of the “Serverless Security” book. He has worked on multiple serverless projects as a developer and security engineer, contributed to open-source serverless projects, and worked on large military systems in various engineering roles.

Originally published on Medium

Photo by Marc Rafanell López on https://unsplash.com/@marcrafanell

Stop Installing Node.js and Global Npm Packages, Use Docker Instead and Protect Your System from Vulnerabilities

Miguel A. Calles — Wed, 23 Feb 2022 03:04:10 +0000

There is a way to keep our computers isolated from malicious npm packages and cybersecurity vulnerabilities. It’s almost like Node and npm will be on an island.

We can use a Docker container to run Node.js and install npm packages.

What are Docker containers and why should we use them?

Docker is a software technology that creates a container that runs on our computer. A container is like running a mini computer within ours and restricts access to our files.

The problem with running Node.js on our computer is the growth of malicious npm packages. There are some malicious actors that purposely put malware in npm packages. They create packages with similar names (called typosquatting) hoping we will install the incorrect version so they can deliver the malware.

These types of npm attacks have been growing significantly and will continue being an issue in 2022 and future years.

What if we installed a malicious npm package and we can limit the extent of the damage? That is where containers can help.

Suppose we installed an npm package that deployed ransomware. All our files would become a victim to the ransomware attack if we were running Node.js on our computer.

Suppose we installed it inside a container. A properly configured container will limit access to the files added to the container. In theory, only those specific files will be compromised and our personal files should be protected. We can stop the container, delete the container image, purge all files associated with the container, and run an antivirus scan just to be safe. Given we commit our code to a software repository, we probably only lost a little bit of our code.

Using a Docker container to run Node.js is like putting our code in quarantine so that an infection does not put a strain on the whole computer.

How do I set up Node.js and npm on my machine?

Start by installing Docker Desktop on our development machine. We will want to create a Docker account also to take advantage Docker scan feature that we will discuss later.

After we install it, go to the settings.

Enable “Docker Compose V2” in the “General” section.

General settings.

Set the desired resource load in the “Resources Advanced” section.

Advanced Resources settings.

Ensure software updates are enabled.

Software Updates settings.

Go to our code folder. Create a file called docker-compose.yml in the top-level directory (or in every folder that we want a customized container).

version: "3"
services:
  dev:
    image: "node:14.18.1-buster-slim"
    user: "node"
    working_dir: /home/node/dev
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./:/home/node/dev

The Docker Compose file creates and runs a Docker container without to build it.

The image: property defines the node container. The example uses an official Node.js container image. The version was selected based on the recommendation from the Docker scan.

The working_dir: property defines the home directory as /home/node/dev. (When we start the container it will show dev as the current folder.)

The volumes: property allows running Docker within container and puts all the files where the docker-compose.yml exists, and mounts them to the /home/node/dev directory within the container. (We can delete the first line if we do not need Docker running within the container.)

Using another image like one from Circle (e.g., circleci/node:14-bullseye) provides git and other common Linux utilities.

version: "3"
services:
  node:
    image: "circleci/node:14-bullseye"
    working_dir: /home/node/dev
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./:/home/node/dev

In our computer’s terminal, run the following command to start the container.

docker compose run --rm dev bash
# Or the following if Docker Compose v2 was not checked above
docker-compose run --rm dev bash

We will now see a prompt like node@502104098e72:~/code$ in the terminal. This means our terminal is now inside the Docker container running node.

Type the ls command to see our files. We should see our the files within the directory.

Open Docker Desktop and we will see our container running.

A container is running.

We can now run npm i -g some_package_name and npm ci within the container.

(If we have Node installed on our machine, we can try installing a different global npm package in our container. We open a new terminal window, try running the global npm package and we should get an error because it is only installed in our container.)

In the container’s terminal, type the exit command. Go to Docker Desktop and we should no longer see the container.

No container is running.

The --rm flag in the docker compose run commands tells Docker to delete the container after it terminates. This way we can keep our machine cleaner.

Keeping Docker up to date and clean

We should apply the Docker software updates when they become available.

After we apply the updates, we should scan our container for vulnerabilities with a Docker scan.

We need to accept the license to get started with Docker scan.

docker scan --accept-license --version

We can scan our Node.js container with the following command.

docker scan node

The scan output will recommend which container image to use. We will update the image: property within the docker-compose.yml file to have the recommend image.

Every so often we should clean up the images.

Cleaning up images.

And remove old volumes.

Removing volumes.

Conclusion

Using Docker containers is one way to protect our computers from malicious npm packages and Node.js vulnerabilities because code execution and runtimes are isolated to the container.

Want to Connect?

Originally published on Medium

Photo by Tom Winckels on Unsplash

DEV Community: Miguel A. Calles

Six Costly Findings in AWS Serverless Computing

An overview of serverless

Input injection

Over-privileged policies

Failures and service outages

Cost engineering

Encryption

Spaghetti

Watch the LayerOne presentation

Before you go

We migrated from Serverless Framework to AWS CDK, and this is what we learned: Lessons from the AWS CDK Day 2023 presentation

Introduction to serverless

What is serverless?

Types of serverless services

Why choose serverless?

An example of a serverless architecture on AWS

Using the Serverless Framework

History

The architecture

Serverless Framework services

Platform service

Accounts service

APIs service

Web app service

Findings

Moving to AWS CDK

Custom level 3 constructs

Serverless Framework services to CDK apps

Web app CDK app

Backend CDK app

What did we learn from migrating to AWS CDK from Serverless Framework?

More stuff comes "out of the box"

Better organization

Simpler deployments and better security

Watch the CDK Day 2023 presentation

Before you go

Setting Up our AWS Account and CDK Environment

Creating a New Directory

Using GitHub and GitHub Desktop

Setting up Docker Desktop

Creating Our Container

User Docker container to run Node.js and global npm packages | Better Programming

Setting Up the AWS Account

Setting Up Identity and Access Management (IAM)

Creating Our CDK app

Creating the CDK app

Bootstrapping the AWS account

Chapter Review

Describing the CDK Application | Chapter by Chapter

Miguel A. Calles ・ Jun 24, 2023 ・ Medium

Before you go

AWS CDK Serverless Cookbook: A Step-by-step Guide to Building a Serverless App in the Cloud

Miguel A. Calles ・ May 28

Endnotes

An Overview of AWS, Serverless Computing, and CDK

An Overview of Amazon Web Services

An Overview of Serverless Computing

Introduction to Cloud Computing Security | Serverless is Cool

Miguel A. Calles ・ Jun 6, 2023 ・ Medium

An Overview of Infrastructure as Code

Serverless vs. AWS CDK: Five Major Differences | JavaScript in Plain English

An Overview of the AWS Cloud Development Kit

Chapter Review

Setting Up our AWS Account and CDK Environment

Miguel A. Calles ・ Sep 23

Before you go

AWS CDK Serverless Cookbook: A Step-by-step Guide to Building a Serverless App in the Cloud

Miguel A. Calles ・ May 28

Endnotes

Using Docker for AWS CDK development

Introduction

Here is how I set up my Docker container

Conclusion

Before you go

Speed up AWS CDK deploys up to 80%

TL;DR

Introduction

The issue

This was the solution

Miguel A. Calles ・ Jun 24, 2023 ・
Medium

Miguel A. Calles ・ Jun 6, 2023 ・
Medium

Miguel A. Calles ・ May 12, 2023 ・
Medium

Miguel A. Calles ・ May 7, 2023 ・
Medium

Should we put secrets in the `.env` file?

How can we protect the `.env` file?

When to use `??` when checking a value?

When to use `||` when checking a value?