DEV Community: Adriano Sastre Vieira

Configuring Docker without Docker Desktop on Windows / WSL2

Adriano Sastre Vieira — Sun, 13 Feb 2022 14:24:08 +0000

Hello,

Since February 1st, 2022 Docker Desktop is not free anymore for companies that have more than 250 employees or more than $10 million in annual revenue: Docker FAQs.

But, as stated in the above link, it is still possible to use only Docker CLI and Engine for free:

Can I just install the Docker CLI instead of using Docker Desktop?

If you use a Mac, you can install Docker CLI and Engine inside a virtual machine, using VirtualBox or VMware Fusion for example, which may require purchasing a license for VirtualBox or VMware Fusion. On Windows you could install and run Docker CLI and Engine inside WSL2. If you use a Linux machine you can easily use the Docker CLI and Docker Engine, See the documentation on installing Docker Engine for instructions. Docker Desktop is not yet available for Linux.

This post purpose is provide a step-by-step to still run Docker commands on Windows via WSL2, after uninstalling Docker Desktop. For that, we're going to use the Ubuntu distribution.

So we're going to:

Uninstall Docker Desktop on Windows
Configure WSL2 on Windows
Install Ubuntu on Windows
Install Docker CLI + Compose on Ubuntu
Execute Docker commands.

About WSL and WSL2

WSL (Windows Sybsystem for Linux) allows you to run Linux command-line tools and apps alongside your Windows command-line, desktop and store apps, and to access your Windows files from within Linux. This enables you to use Windows apps and Linux command-line tools on the same set of files if you wish.

WSL2 is a new version of the WSL architecture that powers it to run ELF64 Linux binaries on Windows. Its primary goals are to increase file system performance, as well as adding full system call compatibility.

Uninstalling Docker Desktop on Windows

If installed, uninstall Docker Desktop on Windows via Control Panel - Programs and Features - Uninstall.

That will make docker commands stop working on Windows.

Enabling WSL on Windows

In order to enable WSL on Windows, execute the following commands in PowerShell running in admin mode:

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

Open PowerShell again and run the wsl command. If it does not work, restart your PC.

Configuring WSL2

Download the WSL2 kernel via: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi

Execute the downloaded wsl_update_x64.msi file.

WSL version 1 is the default, it's important to change to version 2, so all Linux installed distros will be v2 by default.

In order to do that, execute the following command in PowerShell running in admin mode:

wsl --set-default-version 2

Limiting WSL2 resources (optional)

If you want to limit WSL2 resources, create a file named .wslconfig in you user root folder on Windows (C:\Users<your_username>) and define the configurations, e.g.:

[wsl2]
memory=8GB
processors=4
swap=2GB

Installing Ubuntu on Windows

In the Windows Store app, choose the Ubuntu Linux distribution and download/install it.

After that, your WSL2 / Ubuntu is already working.

You can enter the Ubuntu terminal from the Windows start menu.

The first time Ubuntu is run, it's necessary to define an username. Uou can use the same Windows username and password. This will be the *root * user of you WSL2 instance.

Updating Ubuntu:

Important: All commands from this point on are performed in the Ubuntu terminal.

$ sudo apt-get update

$ sudo apt-get upgrade -y

Installing Docker Engine

Run the following commands in order to install the Docker Engine on the WSL2 Ubuntu instance:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
$ sudo apt install apt-transport-https ca-certificates curl net-tools software-properties-common -y
$ sudo apt-get update
$ apt-cache policy docker-ce
$ sudo apt install docker-ce -y

$ sudo usermod -aG docker <incluir aqui o seu usuário do Ubuntu>

$ sudo service docker start

$ service docker status

Installing Docker Compose

$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

$ docker-compose --version

Executing Docker on Ubuntu

From now on, you may execute docker and/or docker-compose commands at will via the Ubuntu terminal.

So suppose you were used to go to the Windows directory "C:\my-project" and run docker commands from there; now via the Ubuntu terminal, you still can go to any Windows directory, which are mounted under the /mnt directory, and run the docker commands.

E.g. in this case:

$ cd /mnt/c/my-project

$ docker ....

$ docker-compose ....

Installing other tools and dependencies

Depending on your use case, you may need to install other tools and dependencies on the WSL2 Ubuntu instance.

Installing Node.js and NPM (optional)

As it is very common, this section covers the Node.js / NPM installation on Ubuntu:

$ sudo apt install nodejs
$ curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -
$ sudo apt -y install nodejs

$ node -v

$ sudo apt install npm

$ sudo npm cache clean -f
$ sudo npm install -g n
$ sudo n stable
$ sudo npm install -g npm@latest

$ npm -v

---

That's it! Thanks for reading.

Adriano Sastre Vieira.

React basic CRUD app with Firebase Realtime Database - Hands-on

Adriano Sastre Vieira — Mon, 17 Jan 2022 10:31:42 +0000

Hi, I'm Adriano Sastre Vieira, software developer.

Today we're gonna build a basic React app, that connects to an API and perform the basic CRUD operations: POST, GET, PUT, DELETE.

I assume you have Node / npm / React installed and configured on your machine.

Google Firebase Realtime Database

As this is a CRUD POC, in order to run this project successfully, we need to have an API that provides the CRUD operations. Google Firebase is real handy to help us with that.

In order to configure it, please login to https://firebase.google.com (create the GCP account if you don't have one) and go to Console at https://console.firebase.google.com/. After that, follow these simple steps:

1 - Create a Project
2 - Create a Realtime Database (don't forget to select "Init on test mode" so you have access to the APIs)

Et voilà, you have a realtime database URL that looks like "https://--default-rtdb.firebaseio.com/" and it is super powerful, in summary realtime database is a serverless no-sql database that already provides and API with all CRUD operations (POST, GET, PUT, PATCH, DELETE) on the fly!

About React - A brief summary

React, a "Javascript library for User Interfaces", is a very popular free and open source library, and it allows developers to organize the way we use modern javascript to make the glue between HTML and CSS.

It allow us to build "Single Page Applications", aka SPA, giving the web app a more fluid, "reactive" look and feel, similar to mobile native apps. In other words, no more request, loading, response during the user experience on the web.

Once the team learn React, we are more productive and produce less error-prone code, by not using a javascript imperative approach, but declarative approach instead (e.g. instead of document.querySelector(‘button’)… we add extra attributes onClick to a button pointing to a function (to react to that element).

It is also good to follow some conventions (directory and files naming for example) when programming with react, as we'll see with this project.

The basics of React are the Components - the “Building blocks”, that allows us to make reusable and more easy to maintain code. Components should be focused on 1 thing only!

Components files usually contains a javascript function, exported as default, and the convention is that the files and function name are the same and starts with Capital letter, and are stored under a "components" folder (subfolders can be created to organize the project).

The components functions have the props attribute, that helps enabling dynamic content. props.children is useful on wrapping components.

The components functions returns JSX code - similar to html code, but its attributes are from plain javascript and not html, e.g.: className instead of class. On JSX code we can use curly braces ({}) in order to embed single-line-expressions javascript into ‘html’ code.

When it comes to Styling, we can create a Component.module.css file and import it on the component with "import classes from ‘./Component.module.css’;" so we can use this file's CSS classes only on the current Component.

React also provides some hooks, for example the useState hook, very useful to make our pages more dynamic, for example while we're loading asynchronous data from an API. There are others useful hooks, like useEffect and useHistory.

In order to change "pages" in our SPA React application without using the traditional request-response way, we can use the react-router-dom lib in order to implement a Router for our app.

With a Router, even on a single page, we have the illusion that the page changes, but the app stays fast and reactive, comparing to a new request/response. Router pages are react components, but loaded by the Router, and the convention is to save them on a ‘/pages’ folder.

That's basically the things we used on this project.

Open project on Visual Studio Code

This project was implemented with React 17.0.2 and javascript, and is public available on github:
https://github.com/adrianosastre/react-crud-app

Please clone and open it on VS Code, and look at these important chunks of code:

1 - API Endpoint configuration:

Rename the ".env.example" file to ".env" and configure the REACT_APP_API_URL key to your created realtime database URL.

2 - This Project structure:

This React CRUD project was created with the "npx create-react-app" command, and cleaned so only files that matters for this article was maintained.

Currently, it contains the basics for using React, and you can navigate the project noticing the following:

a. The project organization (folders, names convention)
b. The use of Routing (react-router-dom)
c. The components (how to pass values to children and parents components; wrapper components...)
d. The use of JSX on the components
e. Basic CSS styling, flex layout
f. React states and other main hooks
g. Interacting to the realtime database API data by using the fetch javascript API.

Run!

Run you project with the "npm run start" command and enjoy!

The project CRUD are "Things", so the first time the "Things List" is clicked you see an empty list (that makes sense because your realtime database is empty):

Click on the "Add Thing" link, fill in the form and click on the "Add Thing" button:

Once the "think" item is created, the app goes to the things list page:

Click on the card to view its details:

From the details page you can edit or delete the thing item.

That's it!

Thanks, I hope this was helpful, and I'm available for any comments or questions here or via adrianosastre@gmail.com

AWS Serverless and the DynamoDB Single Table Design - Hands-on with CDK v2

Adriano Sastre Vieira — Thu, 30 Dec 2021 00:25:12 +0000

Hi, I'm Adriano Sastre Vieira, software developer at Inatel Competence Center.

In this article I'll talk about AWS Serverless, also called managed services (because you don't have to manage a server in order to deploy these services), with more focus on DynamoDB and the Single Table Design concept.

Not recommended, but click here if you want to skip the theory and go strait to the hands on.

Serverless!

DynamoDB is a AWS managed database service. When we talk about AWS managed service, we're also talking about Serverless.

Serverless is the practice of using managed services with event driven compute functions to avoid or minimize infrastructure management, configuration, operations, and idle capacity.

But it's not just about computing, there is a wide range of things an application architecture may need, e.g. Compute, Storage, Data, Monitoring, Queue, Notification ...

So, it is correct to say that DynamoDB is a serverless database, or more specifically, an AWS managed database service.

Serverless is a big paradigm shift, potentially even more impactful than the move to the Cloud before it!

Check the articles below if you want to go deeper into Serverless:

Microservices

Yet on the related concepts, we have the Microservices.

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

These services are built around business capabilities and independently deployable by fully automated deployment machinery.

Many people are using Serverless applications to build a microservice architecture!

I believe this is the number one article for those who wants to go deeper on Microservices - from Martin Fowler.

Serverless and the AWS Cloud

Ok, some important things about AWS before we cut to the chase.

AWS - Fundamentals (The 5 Pillars, new "mental models")

When starting working with AWS, it's normal to feel quite lost, as it has about 200 services, many of them overlapping others on their responsibilities, doing similar stuff, and the AWS extensive documentation does not seem to help beginners, it's difficult to FOCUS and use what is best for each case.

In addition, when we change from a monolithic architecture to micro services, it's a big shift, it's like changing from classical physics to quantum physics!

So, a very good start point is to understand the AWS Fundamentals, which talks about the 5 pillars that requires new mental models, and summarizes the AWS services and concepts for each one:

Operational excellence: thinking about operations as automation (CloudFormation, CDK ...)
Security: zero trust, the principle of least privilege (IAM, data encryption ...)
Reliability: using fault isolation zones to limit blast radius (regions, quotas, throttling ...)
Performance Efficiency: think of your services as cattle, not pets. (horizontal vs vertical scaling, auto scaling ...)
Cost Optimization: OpEx (pay-as-you-go) instead of CapEx (one-time purchase)

So if you didn't yet, now it's a good opportunity to check the AWS Fundamentals!

The AWS Managed Services

When we talk about AWS managed services, these are the main ones:

There are many pros on using managed services:

Cost reduction: the initial cost tend to zero (free tier for most of them, after that pay as you use)
Security: e.g. no need to install security patches on servers
NoOps: e.g. no servers / storage to manage or to scale, no need for a infrastructure team
More scalable
High performance
Greener: if you don't need, you're not using the server resources
Productivity: developers focus on delivering business value

Some cool links to learn more about Serverless on AWS:

Serverless on AWS
AWS Deep Dive Serverless
Serverless Land
AWS Free Tier - in order to check if a AWS service is free or how much it costs

Minimum Stack: API Gateway + Lambda + DynamoDB

While this article focus is the DynamoDB, in order to talk about it in a more practical way, it's important to think in a minimum sample architecture where it fits.

The classical AWS example is:

An API (e.g. some CRUD) implemented via API Gateway;
This API is accessed by a client (e.g, the Postman, or a mobile/web app);
Lambda function(s) in order to manage this API and interface with the database
And the DynamoDB table in order to store data.

I'll briefly talk about API Gateway and Lambda before going to DynamoDB. Pay attention to the italic words, they are related to the Serverless concepts.

API Gateway

https://aws.amazon.com/api-gateway

Fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.

Handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management.

Lambda

https://aws.amazon.com/lambda

AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part of Amazon Web Services.

Runs code in response to events and automatically manages the computing resources required by that code.

Triggered by events (e.g. HTTP Calls via API Gateway, S3 new objects on a bucket, new SQS in a queue, new item in a DynamoDB table ...)

DynamoDB

https://aws.amazon.com/dynamodb/

Now we're talking. DynamoDB definitions and main characteristics from AWS:

Fast and flexible NoSQL database service for any scale. Key-value and document database that delivers single-digit millisecond performance at any scale.
Fully managed, multi-region, multi-active, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications.
Free-tier forever applied, up to 25GB storage, 25 read/write provisioned units (about 200M requests/month). After that: cost by storage and requests.

For those who are curious about its roots, read "The Dynamo Paper". Summary: it was created by Amazon (2004-2007), public released by AWS in 2012.

DynamoDB as part of a Serverless Architecture

When we first saw these propaganda, it seems all good news alright?

But wait, it is not that simple.

First, adopting DynamoDB in place of a relational database is just part of moving to a Serverless Architecture, although a very important one.

The idea of using NoSQL for any and all core business requirements is quite new, and this is because the whole system now works at a scale and event-driven nature before inexistent.

Bad designed DynamoDB tables has often the major impact on a serverless architectured system, both on performance and costs!

Don't get me wrong, DynamoDB is really great when the data is properly modeled! There are lots of benefits on using it, as easily integrating it with other managed services like lambdas, really high performance and scalability, and very low initial cost.

And yes, it is possible to use it correctly on the majority of the systems (remember: Amazon uses it on its shopping cart, so why can't we?); but in summary: DynamoDB is complex, doing it well even more so!

DynamoDB NoSQL design vs RDBMS

NoSQL design requires a different mindset than RDBMS design.

With RDBMS, you can go ahead and create a normalized data model without thinking about access patterns.

By contrast, you shouldn't start designing the schema for DynamoDB until you know the questions that need to be answered. Understanding the business problems and the application use cases up front is essential!.

To clarify, it follows some common access patterns examples:

Get a user's profile data
List the user's orders
Get an order and its items
List the user's orders by status

Of course, on a real system there are lots more.

DynamoDB design considerations

This AWS documentation "NoSQL Design" goes deeper on the concepts I've summarized in this image:

After you identify specific query requirements, you can organize data according to general principles that govern performance:

Keep related data together. Keeping related data in close proximity has a major impact on cost and performance. Instead of distributing related data items across multiple tables, you should keep related items in your NoSQL system as close together as possible.
Use sort order. Related items can be grouped together and queried efficiently if their key design causes them to sort together. This is an important NoSQL design strategy.
Distribute queries. It is also important that a high volume of queries not be focused on one part of the database, where they can exceed I/O capacity. Instead, you should design data keys to distribute traffic evenly across partitions as much as possible, avoiding "hot spots".
Using indexes. By creating specific global secondary indexes, you can enable different queries than your main table can support, and that are still fast and relatively inexpensive.

These general principles translate into some common design patterns that you can use to model data efficiently in DynamoDB.

This presentation Advanced Design Patterns for Amazon DynamoDB is great for better understanding it.

DynamoDB - table capacity

Per DynamoDB table, it is possible to configure the capacity as:

OnDemand: automatic and "infinite" scaling;
Provisioned: possible to define independent read and write unit capacities; also possible to configure auto-scaling rules, e.g. min/max scaling, how fast to scale.

It is very challenging to proper configure the DynamoDB table provisioned capacity, and it can imply on requests being throttled. Totally configurable, we can set different capacity values for reading and writing requests, and we can also configure read and write scaling: from how much percent of use the table starts reacting, what is the max and minimum capacity it can have, and how much time it waits till scaling up or down. All of these configuration can be done for the table and each of its indexes.

When correctly configured, a provisioned capacity leads to less costs than an on demand capacity. And certainly it is less complicated to configure and watch it for one single table than for many tables. In addition, when configured as on demand capacity, usually the DynamoDB takes more time to understand a high I/O volume of requests and scaling. During this time, we can have requests being throttled.

It is also possible to change between OnDemand and Provisioned once a day! This is very useful for scenarios where on a specific time range, it is not possible to predict the scaling at all, e.g. e-commerce system on Black Fridays.

DynamoDB PK, SK, Attributes

As key-value and document database, each DynamoDB table item may have different attributes!

But it is mandatory to define the primary key. In a nutshell:

Primary Key:
PK = Partition Key (aka hash key) (mandatory)
SK = Sort Key (aka range key) (optional)

Attributes: Binary, Number or String
Can be grouped in a JSON-like structure

This image exemplifies the DynamoDB structure:

And this shows an example of a populated DynamoDB table:

DynamoDB - How to query data

Querying data is normally the most important consideration when designing DynamoDB schema, as usually there are much more reading than writing operations.

There are basically 3 ways of querying DynamoDB data:

Get = one specific item, by PK or PK/SK
Query = several items, by PK/SK or indexed attributes
Scan = several items, by any table attribute

This is very important: the get and query operations are much faster comparing to the scan operation, which has a poor performance and a high cost.

So it is crucial to model a DynamoDB table in a way that it is possible to query all necessary data, for every access pattern, using get or query operations, and avoiding scan operations.

DynamoDB indexes

We may define 2 kinds of indexes on DynamoDB:

GSI = Global Secondary Index = more common, applied to all table items
LSI = Local Secondary Index = applied to a particular table partition (PK)

It follows the main pros and cons of using DynamoDB indexes:

Pro: performance: with an index, it is possible to query (instead of scan) on attributes other than the PK/SK
Con: behind the scenes, each GSI duplicates the table storage, along with its storage costs

There is also a limit of 20 GSI per table.

The following image provides more details on DynamoDB indexes:

DynamoDB TTL (time to live)

Optionally, a time to live attribute can be defined on DynamoDB tables, and it is very useful on scenarios where items have to be deleted after certain time is reached.

Another interesting behavior, each table item may have a different time to live value, or no TTL value at all!

DynamoDB - Single Table Design

Different from relational databases where we have to normalize the data, it is an AWS recommendation to maintain as few tables as possible when modeling with DynamoDB, as stated on NoSQL Design for DynamoDB documentation.

When we normalize data, we make the data access very flexible, but it reduces the scalability, due to the high cost of the joins operations.

But DynamoDB was built for enormous, high-velocity use cases, such as the Amazon.com shopping cart. Rather than working to make joins scale better, DynamoDB sidesteps the problem by removing the ability to use joins at all!

Again, DynamoDB was build with web scale in mind. It can grow almost infinitely without degrading performance. In order to achieve this DynamoDB removed joins completely.

You have to model the data in such a way that you can read the data, ideally, in a single request by denormalizing the data.

The main reason for using a DynamoDB single table is to retrieve multiple, heterogenous item types using a single request.

The following links are great to understand more about the single table design concepts:

Single Table Design - Pros and Cons

Summary of the advantages and disadvantages of applying the single table design:

DynamoDB - When and how (not) to use

I summarize in the following images when and how to use and not to use DynamoDB:

More about the true microservice here. In a nutshell, it favors using a DynamoDB single table per microservice, but not per the whole system.

Indeed the Faux SQL is a common mistake when starting using a NoSQL database as DynamoDB or MongoDB. More about the Faux SQL here.

DynamoDB - Hands on!

From Relational to NoSQL

In order to get the hands dirty on the DynamoDB single table design, let's imagine a simple system where we have users and orders.

RDBMS:

For this example, in the relational world we could have the following model:

DynamoDB

When we model it with DynamoDB, the first and more important question is: What are the access patterns for my system?
In other words, how the system will query the data in the database?

For this example we can think about the following access patterns:

List all users
Get user profile
Add user
Edit user
Delete User
List all users orders
List users orders by status
List users order items
Add user order
Edit user order
Delete user order

With that in mind, first we can model the user and order documents with the information we would need for all access patterns.

User example:

{
  "username": "adrianosastre",
  "fullName": "Adriano Sastre Vieira",
  "email": "adrianosastre@gmail.com",
  "addresses": [
    {
      "home": "My first address"
    },
    {
      "home": "My second address"
    }
  ]
}

Order example:

{
  "id": "1e499793-1084-4c34-9d4c-bbb9df88d2de",
  "username": "adrianosastre",
  "fullName": "Adriano Sastre Vieira",
  "address": 
  {
    "home": "My first address"
  },
  "items": [
    {
      "name": "iPhone",
      "value": 999.90,
      "quantity": 1
    }
  ]
}

Those "schemas" result in the following sample data for a DynamoDB single table:

Important things to note:

We have in the same table, different items for users and orders information. They even have different attributes. But the important is that they have different patterns in their PK and SK values, and these patterns are used in order to query data for all access patterns.
The "orderStatus" attribute, used only on order items, needs to be used on the "List users orders by status" access pattern. As it is not the PK or SK, we need to create an index in order to query data, not scan. More on that later.
We "denormalize" data, repeating user information inside order information. That's a common practice on DynamoDB and NoSQL modeling in general.

Again, there are more detailed articles explaining how to go from a relational to a NoSQL database, if you wanna go further:

IaC - the Infrastructure as Code

Before configuring the AWS services and coding the business logic, let's do it right!

Having the infrastructure as code is essential in order to code versioning (ex: git) all the AWS services configuration, instead of making it through the AWS Console (and totally losing control when something goes wrong!).

IaC is also critical to implement CI/CD pipelines.

CloudFormation

https://aws.amazon.com/cloudformation/

When it comes to the AWS IaC, the most basic level is the CloudFormation. Basically, it consists on yaml or json templates that describes your resources and its dependencies so you can launch and configure them together as a stack.

In other words, it is an automated process for resources creation. But it is still quite hard and error-prone to program IaC with CloudFormation.

CDK - The Cloud Development Kit

https://docs.aws.amazon.com/cdk/api/v2/

Released on 2019, we now have the AWS CDK: the official AWS open source software development framework to define your cloud application resources using familiar programming languages. And now in the end of 2021, we have CDK v2 released.

With CDK, you can model the infrastructure resources in high-level languages like Java, Typescript, Python or C#.

Behind the scenes, the CDK will generate the CloudFormation templates and deploy them as AWS CloudFormation Stacks.

It is much safer, easier and more fun to program AWS IaC code with CDK that with CloudFormation! Also, when compared to other IaC frameworks (e.g. serverless, terraform, etc), it has the following advantages:

Implemented and maintained by AWS
Easy integration with AWS services
More secure (e.g. roles automatically generated for services, from read/write permissions)

Step by step

Let's get the hands dirty!

Prerequisites

If not yet, create / install / configure the following:

1 - Create an AWS Account if you don't have yet.

2 - After logged into the AWS Console, add an IAM user with "Access type: Programatic access" and for the sake of this example, add the "Administrator Access" policy to this user. Copy its "Access key ID" and "Secret access key", we'll use them soon.

3 - Download and install VS Code: https://code.visualstudio.com/

4 - Install Node and NPM: https://nodejs.org/en/download/

After installing, check their version. At the moment of this writing I have node v16.13.1 and npm 8.3.0

node -v
npm -v

5 - Download and install AWS cli: https://aws.amazon.com/cli/

After installing, check its version and configure it to your AWS account IAM user created on step 2:

aws --version
aws configure

6 - Install AWS CDK:

Install CDK via NPM and check its version. At the moment I have 2.3.0 (build beaa5b2)

npm install -g aws-cdk
cdk --version

7 - Download and install Docker: https://www.docker.com/products/docker-desktop

Docker is only used in order to deploy the CDK project into AWS, we do not have containerized AWS services in this example.

8 - Download and install Postman: https://www.postman.com/ (used in order to test the APIs)

Architecture

As per the above architecture, this project consists on:

API Gateway, with the /users and /products RESTful resources
Lambdas: functions for users and orders, they handle the APIs and DynamoDB data
DynamoDB: one single table to store users and orders data

Open project on VS Code

This project was implemented with CDK v2 with Typescript, and is public available on github:
github.com/adrianosastre/DynamoDB-CDK-Hands-On

Please clone and open it on VS Code, and look at these important chunks of code:

1 - Resources application stack

The lib/resources-application-stack.ts file is IaC code, it creates the DynamoDB table and the users and orders Lambdas.

Important things to note:

The "singleTable" constant contains all DynamoDB table configuration: the PK, the SK, the GSI, the table capacity and scaling configuration on a few lines of code with CDK. In other words, modeling the data is complex, but implementing its infrastructure with CDK is simple.
The "grantReadWrite" method that manages the lambda permissions on the DynamoDB table. Again, CDK takes care of services permissions in a very easy and efficient way.
The lambdas were implemented as class readonly public attributes, so they can be accessed later on the API stack.

2 - API stack

The lib/api-gateway-stack.ts file, also an IaC code, creates the API Gateway with our users and orders resources.

Note that it uses the lambdas exposed on the lib/resources-application-stack.ts file in order to integrate its resources with the lambdas.

3 - The CDK main file

The file under the bin directory, in this case the bin/dynamoDB-CDK-Hands-On-Project.ts file, is the main file in the CDK structure.

It instantiates the stacks and care about its dependencies.

4 - Lambdas code

The users.js and orders.js files under lambda directory are not IaC code.

Instead, they are the "core" of the system, containing the business logic code behind the lambdas, and are executed every time they are triggered.

Deploying on AWS

Note: AWS cli must be correctly configured as explained in the Prerequisites session.

Only once, it is necessary to execute the following command so the CDK will deploy into your AWS account the required resources for it to deploy projects:

cdk bootstrap

After that, you can deploy the project to your AWS account via the following command:

cdk deploy --all

Every time you change the IaC or lambda code, you can use this command to redeploy, and it does not impact the services usage!

After the project is deployed, you can check in your AWS account the following resources:

CloudFormation: the stacks with the resources programmed via CDK were deployed here:

API Gateway: the DynamoDB-CDK-Hands-On-API API is deployed and public available:

And the API expanded showing its resources:

In addition, the Stages > prod > Invoke URL shows the base URL for this API, so you can use it, for example, with Postman:

Lambda: the functions to handle users and orders are deployed:

DynamoDB: The DynamoDB-CDK-Hands-On-Single-Table table was also created and deployed:

Also, at this moment you can see that the table have no items, and you can also double-check its capacity configuration:

Testing with Postman

After deployed, it is possible to test all the project URLs with Postman. Take the following images as references and use Postman with your API base URL.

Adding users:

In the lambda/users.js code, the put method from the DDB client was used:

Listing all users:

In the lambda/users.js code, the query method from the DDB client was used, and only querying the "USER#" value in the PK returned the results we want:

Getting a user profile data:

In the lambda/users.js code, the get method from the DDB client was used, in this case we need to provide values for the PK and SK to retrieve a specific item:

Editing a user:

In the lambda/users.js code, the update method from the DDB client was used, and we provided values for the PK and SK to update a specific item:

Deleting a user:

In the lambda/users.js code, the delete method from the DDB client was used, and again we provided values for the PK and SK to delete a specific item:

Adding an order for a user:

Listing all orders for each user:

In the lambda/orders.js code, the query method from the DDB client was used, and querying the "ORDER#${username}" value in the PK returned the results we want: (note that we have one order partition per username)

Listing users orders by status:

In the lambda/orders.js code, the query method from the DDB client was used, and here we used the index value so we can search for attribute values without using scan in order to get the results we want:

Listing a user specific order items:

Editing user order:

Deleting user order:

AWS troubleshooting

Things does not always go right from the first time.

In order to troubleshoot AWS services, e.g. the lambdas code, their logs results that can be double-checked on AWS CloudWatch Log Groups. Just click on the respective log group:

And after that, in the log stream messages:

Remember, logs are our best friends! As stated on this great old Optimal Logging article, "With optimal logging, you can even eliminate the necessity for debuggers!"

Tracking performance

Last but not least, in order to track performance, this project lambdas also have X-Ray enabled so you can verify X-Ray traces in ServiceLens.

It is so cool and useful to graphically see the services flow and how much time was spent in each service or function!

You can perform load tests with Postman, for example configuring it to send several user or order edit operations, and monitor the performance on AWS Console as per the above images.

That's all folks!

I hope this article was helpful, and I'm available for any comments or questions here or via adrianosastre@gmail.com

Thanks and have a nice 2022 :)