DEV Community: pedchenkoroman

Pattern Actor in AWS environment

pedchenkoroman — Tue, 15 Apr 2025 22:11:47 +0000

Hi everyone,
In this article, I’d like to show you how to use the Actor pattern in an AWS environment. It’s possible that you’re already using the Actor Model without realizing that this approach actually has a name—the Actor.

What is the Actor?

I’m absolutely sure I couldn’t explain it better than the person who invented and formalized the Actor Model, which is why I highly recommend watching this video.

Let me just add a quick recap and show you how to implement the Actor Model in AWS.

The Actor Model is a conceptual model used in computer science to handle concurrent computation. In this model, an actor is a computational entity that, in response to receiving a message, can:

Perform a task (e.g., process data or make a decision),
Send messages to other actors,
Create new actors, and
Update its internal state.

Each actor operates independently and communicates only by message passing, which helps avoid issues related to shared memory and makes systems easier to scale and reason about. Let me rephrase it use simply words.

Actor

Lightweight and it is easy to create thousand of them
Has own state
Has own mailbox (queue)
Can communicate with each other only through messages
Messages are processed in FIFO order
Process only one message at a time
Decoupled

In AWS, you can implement the Actor pattern using services like:

AWS Lambda (as individual actors)
Amazon SQS or SNS (for message passing)
Step Functions or EventBridge (for orchestration)
DynamoDb (as a storage/state)

It might seem like the Actor Model is simply a combination of a queue service, a Lambda function triggered by that queue, and a DynamoDB table — and you'd be mostly right. However, there are some subtle nuances, and I’d like to walk you through them.

Requirements

A typical use case well-suited for the Actor Model is a shopping cart in an e-commerce application. You'll easily come across this example in many articles, which is why I'll provide some slightly different ones instead.

Let's imagine a fintech application where a user can deposit money into their own account or transfer it to another account. In the fintech world, these operations are referred to as transactions, and they typically fall into two categories: deposit and transfer.

Another example is a store where products can arrive and depart at any time. To prevent the inventory from dropping below zero, we must ensure that all arrivals are processed before any departures.

🎯 I’ll use these two scenarios to demonstrate how to implement a proper, scalable and from ever going below zero solution using AWS resources and the Actor Model.

Now we do have a lot to cover so let's jump in and get started straight away

Solution

Initial Approach: Limited Scalability
We can split our transactions into two batches: the first containing only deposit transactions, and the second containing transfer transactions. These batches will then be sent to a FIFO queue for processing. Using the same messageGroupId for all messages, to help us to process the transaction in correct order and cover the scenario where a user deposits money before transferring it to another user.

With products the same idea first we will process all arrivals and only then departures.

Let’s take a visual look at how our flow works:

Imagine that Lambda 1 has already sorted and filtered the products/transactions and is sending them one by one to a queue. Lambda 2 is triggered by the queue and processes each message individually, storing useful information—such as the quantity of products or the cash balance of clients—in a DynamoDB table. The bottle neck of this approach is the process lambda because it is always only one instance.

Scalable Solution Using the Actor Model
To solve this issue, the Actor pattern comes in handy. Before I describe the solution, I’d like to stress out again that every Actor must have its own state, queue and perform a task. In the non-scalable solution, the processing Lambda function has all these characteristics and may look like an Actor—but what is the state of this Actor? You’ll probably say it's DynamoDB, and you’d be right—but that's a shared state for the entire application. What I’d like to do instead is break it down into the smallest possible pieces. In this case, it could be either the clientId or productId, couldn’t it?

So, this will help us solve our task—let’s take a look at it visually:

So, let's break down the actor implementation:

The first Lambda function already has the sorted list of products or transactions and sends them one by one to a FIFO queue, using messageGroupId as either the clientId or productId.
The queue triggers multiple instances of the processing Lambda function, creating them as needed. Each instance processes messages grouped by messageGroupId, effectively isolating the flow per client or product.

MessageGroupId in FIFO SQS:

Guarantees strict ordering within the same MessageGroupId.
Only one Lambda will process one message group at a time.
If two messages have the same MessageGroupId, they are processed one-by-one, in order.
If two messages have different MessageGroupIds (e.g., client-1, client-2), they can be processed concurrently by separate Lambda invocations.

As a result, we have a scalable solution where each processing Lambda function fits to the three core characteristics of the Actor Model.

💡 As a bonus if a message fails processing (Lambda error), SQS will not deliver the next message in that group until the failed one is successfully handled or moved to a DLQ.

Recap

🚀 The Actor Model is a powerful pattern for building scalable, concurrent systems—and AWS gives us all the tools to implement it effectively. By assigning state and queues per actor (e.g., per client or product), and combining Lambda with FIFO SQS queues, we can create robust, fault-tolerant systems that scale effortlessly.

Also, don’t forget that every AWS account has Lambda quotas—especially regarding concurrent executions. I recommend properly configuring these limits before going to production.

If you’d like to support my work, you can subscribe, give me kudos, or share your valuable feedback. Your support and insights mean a lot!

Send reports safely via email(mailbox).

pedchenkoroman — Mon, 07 Apr 2025 10:34:53 +0000

Hi everyone.
In this article, I will guide you on how to securely send reports via email using AWS. Before we dive in, I’ll provide more details about the AWS services involved and outline the problem we’re solving.

The problem

Let’s explore why attaching a report directly to an email is generally unsafe.

Email Interception & Lack of Encryption.

Standard email protocols (SMTP, POP3, IMAP) do not encrypt emails end-to-end.
If an attacker sniffs network traffic (e.g., on public Wi-Fi or an unsecured network), they can capture emails, including attachments.
Even TLS (used by Gmail, Outlook, etc.) encrypts emails only in transit; once delivered, they are stored unencrypted on mail servers.

Email Account Compromise

If a hacker gains access to your email account (phishing, credential stuffing, malware, etc.), they can:

Download all email attachments you’ve sent or received.
Forward emails to other accounts without you knowing.
Expose the report permanently—it stays in inboxes (yours and the recipient’s) indefinitely.

Forwarding & Unintended Access

A recipient might accidentally forward the email.
Emails can be archived and stored indefinitely, making them a long-term risk.
If their email account is hacked, your sensitive report is exposed to attackers.

Malware & Attachment Exploits

Hackers often embed malicious payloads in email attachments.
If your report is attached as a Word or Excel file, an attacker could inject macro-based malware and resend it.
Some email providers scan attachments and store copies, potentially exposing sensitive content.

Please be aware that this is not a perfect solution. The main drawback is that the report link is shared via email. If a malicious actor becomes aware of this approach, it could be relatively easy to trick users by phishing them with a fake link and a spoofed authorization page.

The most secure and recommended approach would be to notify users that a new report is ready, and make it accessible only through your application’s interface and a secure, custom mailbox.

AWS Services

To achieve the goal of this article, I will use the following AWS services:

AWS Cognito
AWS Identity and Access Management (IAM)
AWS S3
AWS Lambda
AWS Api Gateway

Before we begin I would like to mention that each of these services is self-sufficient and covering all of their features in a single article is nearly impossible. That's why I will go into some more depth about some of them. By the end of this article, I hope these concepts will be deeply engraved in your mind, like wisdom, empowering you to use them productively.
Let's break down every service with a feature that we will use and also the steps of our flow.

AWS Cognito
There are two services that we will use, and the first one is Identity Pools. An identity pool exchanges an external identity type for a set of temporary AWS credentials, allowing access to AWS resources. The external identity could be from Google, Facebook, Twitter, SAML 2.0, or even User Pools identities.
From an Identity Pools perspective, User Pools are just another form of identity. As you might have guessed, we will use User Pools for authentication. Additionally, I’d like to clarify that User Pools are used for sign-in and sign-up (with a customizable web UI), MFA, and other security features.
Identity and Access Management
We will use this service to create an IAM role, which will be attached to the Identity Pool. When the JWT token is passed to the configured Identity Pool, Cognito assumes the IAM role and returns temporary AWS credentials.
S3
Amazon S3 (Simple Storage Service) is a scalable, durable, and secure object storage service that allows you to store and retrieve any amount of data from anywhere. It’s used for storing files, images, videos, backups, logs, reports, and more.
We will use Amazon S3 to store the report in a private bucket..
Lambda
AWS Lambda is a serverless computing service that lets you run code without managing servers. You just upload your function, and AWS takes care of the rest—scaling, infrastructure, and execution.
We will use it to exchange a JWT token, generated by the user pool service, for temporary AWS credentials with an IAM role that grants access to the private S3 bucket.
API Gateway
AWS API Gateway is a fully managed service that helps you create, publish, and manage APIs at scale.
We will use it to create a simple GET endpoint and attach a Lambda function. This endpoint will serve as a callback URL for the User Pool.

The solution

Before we implement our solution, let's have a look visually at how our flow architecture looks at this point.

S3

First, we need to create a private S3 bucket. To create a bucket, you only need to provide a globally unique bucket name. All other settings, including Block Public Access settings for this bucket, can be kept as default. And by default, all public access is blocked.

Lambda

Then, we need to create a Lambda function. I'll keep all the logic within a single Lambda, but you can split it into two functions or even three to follow the Single Responsibility Principle.
The Lambda function will be attached to an API Gateway endpoint. The event will contain fileName and a Cognito code. For simplicity, I’m omitting event validation and error handling in the Lambda code, but you should include these for production use.

For all intents and purposes, we haven’t finished with the Lambda function yet. We still need to set all environment variables, but from a code perspective, we won’t be making any more changes. Since we haven’t created the necessary resources yet, we’re unable to do so.

Now is the time to do that.

Api Gateway

We need to create an API Gateway with a GET endpoint and attach our Lambda function to it. You should be able to handle this on your own. However, there is one important detail I’d like to highlight—please don’t forget to enable the Lambda Proxy Integration option. Without it, the event argument will be empty.

User pools

We need to create a User Pool. As I mentioned earlier, we use the User Pool as an identity store for Identity Pools. If your integrators prefer not to create another user account (with a separate login and password), they can use an alternative identity provider such as Google or Facebook.
To create a User Pool, simply fill out the required form. Additionally, don’t forget to provide the API Gateway endpoint link that was created in the second step.

Identity pools

We need to create an Identity Pool to enable swapping the JWT token issued by the User Pool for temporary AWS credentials. I want to emphasize once again that access to AWS resources is only possible through AWS credentials.
To create an Identity Pool, you’ll need to go through a few steps:

In the first step, choose Authenticated access and select Amazon Cognito User Pool as the authentication provider.

The second step is about configuring the default IAM role. You need to choose Create a new role and provide a meaningful name for it.

After creating the Identity Pool, we will assign the required permissions to this role.

The third step is about configuring your Cognito identity pool to accept users that sign in with a Cognito user pool. Simply choose the user pool created earlier.

You can easily skip the fourth step and just click the Next button.
In the final step, after reviewing, you can click the Create Identity Pool button.

IAM

In the previous step, when we created the Identity Pool, we only provided a name for the role that will be assumed with temporary credentials. We need to update its policy to grant read access to our private bucket.

Please open the permissions policies for this role and paste the following two actions. Don’t forget to replace BUCKET_NAME with the actual bucket name.

Set environment variables

Finally, navigate to the Lambda service, select the Lambda function, and open the Configuration tab. Here, we need to set all the required environment variables. If you're using the code from my gist, you’ll need to define six variables.

That’s it — we’re almost done. Due to the hardcoded file format in the Lambda function code, I would recommend uploading a txt file into the S3 bucket. Once you've done that, we’re ready to test.

To test, navigate to User Pools > App clients > Login pages. There, you can easily find the View login page link.

Click on this button, and it will navigate you to a new page where you'll find the sign-in form. Before you proceed with the sign-in process, you need to modify the URL in the browser’s address bar. Add &state=FILE_NAME_INTO_S3 at the end of the URL. The state parameter allows us to pass the file name into the Lambda function. More details about this parameter you can find here

🚀 Benefits of This Approach

✅ No Custom Frontend Needed – Everything is API-driven.
✅ Fully Secure – Users must authenticate before accessing S3.
✅ Short-Lived Credentials – Uses temporary AWS credentials via Cognito Identity Pool.
✅ Custom File Access – Users only get pre-signed URLs for their requested files.
✅ Serverless & Scalable – Uses Cognito, API Gateway, Lambda, and S3.

🎯 Next Steps

Would you like help with:

🔐 Adding RBAC (Role-Based Access Control) to limit file access?
🛠 Deploying everything in AWS using CloudFormation?

You’ll find answers to these questions in the extended version of this article on patreon.

10 Essential Questions to Improve One-on-One Meetings for Developers

pedchenkoroman — Mon, 10 Feb 2025 14:45:57 +0000

Hi folks,

As promised, this article is about soft skills. Many developers, myself included, have made the mistake of focusing only on hard skills, assuming that soft skills aren’t as important. But that’s not true—soft skills are just as essential as technical expertise.

In this article, I’ll share 10 powerful questions—five for managers and five for developers—to help improve team communication, motivation, and transparency. These questions can foster a strong and engaged team while preventing surprises like, "I want to switch teams" or "We’re not going to continue working with you."

5 Questions from a Manager/Lead's Perspective

First question

~~How is it going?~~ The question itself isn’t bad, but one-on-one meetings typically have limited time. Rephrasing it as, 'What has been your biggest success this week or since our last meeting?' helps start the conversation in a more meaningful way, and when someone highlights something, you can build follow-up questions based on that information.

Second question

~~How can I help you?~~ The question itself isn’t bad, but some developers might not answer honestly either it implies they are struggling and need help or they can not do something. Instead, you could ask something like, 'What is the most complicated part of your daily tasks?' The answer to this question can help address two issues at once.

For example, I once received an answer like, 'The most challenging part is doing the markup.' The solution was quite simple: we split the tasks into two parts, assigned the markup work to another developer, and had the original person act as a reviewer. Additionally, he attended some markup courses to improve his knowledge. As a result, his daily tasks became less challenging, reducing frustration and burnout. After four months, we no longer needed to split the tasks.

Third question

Is there anything in your tasks that seems unnecessary or meaningless? Asking this question might reveal areas of miscommunication or highlight where the team lacks sufficient information. As a manager or tech lead, your role is to align your goals with the team's goals. If someone doesn’t understand a task or sees it as meaningless, achieving it becomes nearly impossible.

Fourth question

"Is there any work that’s been undervalued?" From a lead or manager’s point of view, it might seem like just a ticket moving from one column to another on the board. But for a developer, it could represent hours of hard work. If the manager or lead doesn’t acknowledge this for any reason, the developer might feel upset or lead to demotivation. Asking this question can help you understand the developer's perspective and prevent a process known as "negativity bias." I personally rephrase it as "the empty space always fills with bad thoughts."

Fifth question

"Could you please rate the complexity/interest level of your tasks on a scale from 1 to 10?" These types of questions allow you to track a person's progress. I wouldn’t recommend asking them in every session, but it’s important to take notes to monitor changes over time. Tracking is important, but acting on the results is what truly matters.

5 Questions from a Developer's Perspective

First question

"Is there anything I should be doing that I'm not doing?" Asking this question can lead to new insights. You might discover unmet expectations or growth opportunities. For example, I once learned I should focus more on mentoring junior developers, which led me to improve my code reviews and task breakdowns.

Second question

"In what case should I ask for your help?" Leads and managers are often busy with meetings. This question, along with the next one, helps establish a clear guideline for when it's appropriate to ask for help. In many teams, there are no defined patterns for when someone should seek assistance versus when they should try to solve a problem on their own. Additionally, it's important to consider that a new team member might be hesitant to ask for help or, on the other hand, might seek guidance on every small issue.

Third question

"What situations do I need to notify you of?" Similar to the previous question, this one helps clarify communication boundaries. As a manager or tech lead, I don’t need to be informed about everything—such as the installation of an npm package or minor code refactoring. However, I do need to be notified about critical issues like production incidents, blockers affecting the team, unexpected delays, architectural decisions, or dependencies that could impact delivery. Defining these boundaries ensures smooth communication and helps avoid unnecessary disruptions while keeping everyone aligned on what truly matters.

Forth question

"What growth points do you see now?" It is a great question because it encourages proactive thinking about personal and professional development. It fosters a culture of continuous improvement and helps tailor development plans, such as training, mentorship, or new responsibilities, based on the person's aspirations. Sometimes, a developer feels stuck or unsure of their next steps. Discussing growth opportunities ensures you stay engaged, challenged, and motivated.

Fifth question

"Is there something I haven't asked or told you?" is a great question because it creates an open space for discussion and helps uncover important topics that might have been missed. Also it signals that you're genuinely interested in hearing anything that might be on the other person’s mind, even if it wasn’t covered in the conversation.

Final Thoughts

There are many great questions beyond the ones I’ve covered here. These aren’t my own inventions—they come from books, conferences, and videos that have shaped my perspective. Some excellent resources include Herding Cats by J. Hank Rainwater and Radical Respect by Kim Scott.

By incorporating these questions into your one-on-ones, you can build stronger relationships, improve communication, and create a more engaged team. Let me know what you think or if you have any other great questions to add!

Cheers!

Kaizen or how to start/stop ec2.

pedchenkoroman — Sun, 26 Jan 2025 10:59:57 +0000

Hi guys.

Today, I’d like to share with you a technique I’ve typically used in my work. This technique is called Kaizen.

So, what is Kaizen, and why should you care? Great question. Kaizen is a continuous improvement method that is used in manufacturing. For some, you might be familiar with similar methods like Lean in Project Management or the Toyota Production System (TPS). Kaizen was first introduced in the 1960’s when engineer Taiichi Ohno, who created TPS. It was designed to achieve objectives like improving quality, increasing profitability, and reducing costs.

Normally, each of my PRs consists of a dedicated task and a small improvement. This could be anything, such as adding logging, encapsulating code and covering it with unit tests or reduce unnecessary http requests. Some might refer to this as following the "Boy Scout Rule" and it will be absolutely right. In my humble opinion it is just adjacent topic.

So, I’d like to share a story about how I used this technique to reduce our AWS monthly bill.

Requirements

The application I’ve been working on includes some third-party integrations, and to test the entire workflow, I came up with the idea of spinning up an EC2 instance with primitive logic to simulate those integrations. Everything worked perfectly until I noticed that, most of the time, the instance was idle. So, I did a quick calculation and realized that I could save $1.46 per environment per month by optimizing this setup. Then I considered an effort to implement it and got a green light from a PM to implement it.

Calculations

Before

1 instances x 0.006 USD On Demand hourly cost x 730 hours in a month = 4.380000 USD

After

1 instances x 0.006 USD On Demand hourly cost x 486.66666666666663 hours in a month = 2.920000 USD

Solution

The solution is straightforward and consists of three steps:

Create a New AWS Role: Set up a new AWS role with permissions to start and stop EC2 instances.

Set Up CloudWatch Rules: Create two CloudWatch rules—one to start the EC2 instance and another to stop it at predefined times.

Implement a Lambda Function: Use AWS SDK within a Lambda function to send start and stop commands to the EC2 instance.

That’s pretty much it! The next step is to get rid of the EC2 instance entirely and use only AWS Lambda, with input and output stored in S3. Considering AWS Lambda’s generous free tier, I anticipate the cost will be $0.

It’s the little details that are vital. Little things make big things happen. ~ John Wooden

NOTE: The source code you can find here.

AWS Cognito + GraphQL Directive = ACL with minimal effort

pedchenkoroman — Mon, 13 Jan 2025 12:58:34 +0000

Hi, everyone,
I would like to share an idea/prototype for implementing an access control layer with minimal effort. As the title suggests, I will be using the AWS Cognito service and a GraphQL directive. AWS Cognito provides excellent functionality for controlling access to REST APIs. However, it doesn't fully meet our needs because, with GraphQL, there is typically only a single POST endpoint that handles numerous queries and mutations.

Before we begin, I want to clarify that this is not a step-by-step guide on how to use AWS Cognito or an introduction to GraphQL, including how to create a GraphQL server or a lambda function from scratch.

Requirements

Let’s imagine we have a front-end application and an Article GraphQL service. This service provides simple CRUD mutations, such as publish, delete, and saveDraft. Our task is to implement, with minimal effort, a way to manage access for our Cognito users to these endpoints. For example:

The first user can only save draft.
The second user can save draft and publish.
The third user has full access, including delete operations.

Implementation

Cognito
As mentioned earlier, I will not cover how to set up the AWS Cognito service (user pools and users) in this article. I assume you already have it configured. However, I want to highlight that Cognito users support custom attributes. For more details, you can refer to this link. Before creating our first custom attribute, we will develop our own language. This technique is known as a DSL.

A DSL (Domain-Specific Language) is a programming or scripting language designed to solve problems within a specific domain. Unlike general-purpose programming languages (such as Python, Java, or TypeScript), which are built to handle a wide range of applications, DSLs are tailored for specialized tasks.

First and foremost let's come up with a syntax.

* - means allow all actions
| - the separator between the namespaces
/ - the separator between a namespace and an action(s)

Let’s agree on the following rules:

article/* - allows all actions.
article/saveDraft - allows only the saveDraft action.
article/saveDraft,publish - allows both the publish and saveDraft actions.
article/*|user/* - allows everything for the article and user namespaces.

As you can see, creating your own DSL is straightforward, and this technique is widely used in various areas, such as SQL, configuration files, and Infrastructure as Code.

Now, let’s create a custom attribute that will include a list of actions we want to allow or deny.

Create custom attribute action.

The next step is to create the users with the action custom field.

Code
That’s pretty much it for the AWS Console setup; now we can dive into the code. You can easily find the complete code on GitHub. Here, I’ll focus on the most essential parts.

You can find a tutorial on creating and setting up an Apollo Lambda handler here. The key difference in my approach is that I parse the JWT token from the event and set the custom:action property in the context. As a result, the context includes the action property containing all the rules we added in AWS Cognito.

The next step is to parse the action property from the context and determine whether to allow or deny the action. For this task, we’ll use a GraphQL directive. You can find a guide on how to build it from scratch by following this link. Here, I’ll provide the code and an explanation of how to check access using it.

The logic of the directive is straightforward and involves three steps:

Locate the Namespace: Split the entire string using the namespace separator (we agreed earlier that it is |) and find the specific namespace by name.
Separate Namespace and Actions: Use the / separator to split the namespace from the actions.
Check Actions: Map all actions and verify whether the action is allowed or denied.

The final piece of our setup is the GraphQL schema.

As you can see, simply add the directive in front of the mutation and provide the namespace name as an argument.

That’s pretty much it! This approach seems not only useful and easy to implement but also flexible, cost-effective, and one of the fastest ways to add an Access Control Layer to your application.

To try it out:

Fork the repository and deploy it using your own AWS account. All resources are defined using AWS CDK. You can find instructions on how to bootstrap and deploy an AWS stack in this guide.
I’ve created a public Postman collection for your convenience. Please follow the guide provided within the collection.

If you’d like to support my work, you can subscribe, give me kudos, buy me a Ko-Fi, or share your valuable feedback. Your support and insights mean a lot!

Why you might want to build your own cli?

pedchenkoroman — Thu, 05 Dec 2024 09:00:00 +0000

Hi everyone,
In my last article, "Not only dynamoDb migration/seed scripting", I provided a solution how to manage migration/seed scripting and I proposed to run it by CLI command. If you haven't read it yet, you can find it here. In this article, I’d like to share why you might want to build your own CLI and what types of commands it could contain. However, I will not teach you how to build a CLI from scratch here, and I assume you are already familiar with NodeJS and TypeScript. Instead, this article focuses on the reasons for creating your own CLI and the types of commands it could include. The source code of CLI you will find here and to run it you can use this command

npx dev-to-proto-cli

Configure

The first command I’d like to share is configure. Imagine you are working on a project that follows a multi-services architecture, with at least three services plus a front-end. Each service has its own configuration file for four environments and might have the following structure:

service/environments
├── environment.development.yml/.env.dev
├── environment.tst.yml/.env.tst
├── environment.acc.yml/.env.acc
└── environment.yml/.env.prd

The file extension does not matter.

I see some disadvantages with this approach:

If you have repeatable configuration properties for multiple services, you need to copy and paste them into each service.
Maintenance becomes too complicated. Some properties may have the same value but different names across services.
The release tag depends not only on the codebase but also on the configuration.

This is where the configure command comes into play.

Instead of keeping the configuration in each service, you can store it elsewhere, such as in another repository, a secret manager, an S3 bucket, or any other source. I’ve created two examples of configuration: one stored in a GitHub Gist and another in a Bitbucket Snippet. Imagine that GitHub Gist is dev env and Bitbucket is tst
The configuration structure is a simple nested format. The top-level keys represent global settings for all services, while the nested sections contain specific properties for each service. The subsections can override or extend the global settings. All properties are in one file, and maintaining it is not difficult.
It depends on where you decide to keep it. In my team, we store it in a separate repository and use tags or branches for different scenarios.

log_level: INFO
region: us-east-1

foo:
  log_level: DEBUG
  region: us-west-1
  foo_prop_1: 1
  foo_prop_2: 2

bar:
  log_level: WARN
  region: us-west-2

baz:
  log_level: ERROR

How to use

I assume you have a similar line of code in your GitHub Actions or Bitbucket Pipeline to create an env files depending on env.

cp evnironment.${ENV}.yml environment.yml

Using your own CLI, you would have something like this:

npx dev-to-proto-cli configure --extension=env --service=foo --env=dev

It will generate either a config.yml or .env file with all root properties and the properties dependent on a service.

Other commands

In our team, we aim to create commands for a wide range of scenarios. Here is a brief overview of some of them:

Clean the Database: To clean a DynamoDB table or all tables, we use the clean-db command. The command looks like this:

npx dev-to-proto-cli --profile=dev --region=us-west-1 --provider=sso

If you run it and provide all credentials, it will clean the chosen tables. Please use it carefully.
NOTE: The clean-db command is useful only if your dynamoDb table is NOT attached to a custom VPC.

To get an overview of our tests, we use a Bitbucket pipeline to run daily tests. While you can view the results directly in the pipeline, I find this approach inconvenient for collecting statistics over a longer period, such as the last 30 days. For instance, comparing all failed tests or identifying flaky tests becomes impossible without manual effort or paid add-ons.

To address this, I created another CLI command that uses the public Bitbucket REST API to collect and analyze all test statistics. The command looks like this:

npx @own-cli statistic --start=2024-11-11 --end=2024-11-18

I’ve described only three of the commands we’ve implemented. Please feel free to share your ideas in the comments. Any feedback or kudos would be greatly appreciated!

In my next article, I'll show you how to build an ACL with minimal effort. Subscribe to stay tuned!

Not only dynamoDb migration or seed scripting

pedchenkoroman — Wed, 06 Nov 2024 07:54:50 +0000

Hi folks.

I would like to share with you the idea/prototype for managing and running migration or seed scripts for DynamoDB.
First, let's clarify what a migration and a seed script are.

Migration refers to the process of transferring data from one database to another, or making structural changes to a database, such as upgrading, transforming, or migrating to a new database management system (DBMS). Due to the fact that aws cdk helps us and handles for us DynamoDB structural changes. I would like to make an agriment that into this article migration refers only for transforming data.

Seed refers to the process of populating a database with initial data, often for development, testing, or setup purposes.

Second, let's outline the required functionality.
In the company where I've worked, the product follows a multi-service architecture. I identified five key requirements:

TypeScript support: The tool should support TypeScript.
Minimal migration/seed script logic*: The migration and seeding processes should require minimal scripting.
Ability to run specific scripts: The tool should allow developers to execute particular scripts.
Service independence: The tool should function independently of any specific services.
Leverage the latest AWS SDK version: The tool should utilize the most recent version of the AWS SDK.

Third, before diving into my own implementation, I explored existing packages. One option was the Dynamit CLI, created by my friend Victor Korzunin. While Dynamit CLI offers basic functionality and handles most common tasks, it doesn't fully meet all of the requirements outlined above. Therefore, I decided to implement my own solution.

To present my solution, I've created a separate repository dedicated to migration scripts. The source code you can find here. The repository structure is straightforward, consisting of three primary folders.

The first folder, named templates, contains files for the Plop package. Plop, a micro-generator framework, allows us to generate standardized templates for migration and seed scripts.
The second folder, scripts, this folder can contain subdirectories, each named after a specific service, to organize service-specific migration/seed scripts.
The final folder, framework, houses the core migration logic and interfaces.

First script

Let's run the npm run plop command to generate our first script. This will prompt you to provide a name for the script and the desired folder location.

Let's open the file and examine the code.

First, I'd like to draw your attention to the DynamoDBScriptTracker instance. The first argument in the constructor is a client, and the second is an object with two properties. The first property, scriptName, is automatically generated when I provide the name using the plop console command. The second property, scriptStore, refers to the name of the DynamoDB table where I store all previously executed script names.

NOTE: If you choose a different store for tracking, you only need to implement the ScriptTracker interface. This store could be anything—a relational database, a file, or another storage solution.

Then, as you can see, you need to implement three functions: read, write, and transform.

read: This required function retrieves items in chunks. The data source could be anything—an API, file, S3 bucket, or DynamoDB table.
transform: This optional function applies transformations to an array of items. Additionally, you can cover this function with unit tests.
write: This required function writes a batch of items to a target resource. You don't need to worry about the array's size, as it is automatically split into chunks under the hood.

Finally, you need to export an instance of the Migrator class to pass the arguments scriptTracker and operations. Additionally, there is one more optional argument, force, which allows you to skip execution checks and run the script directly.

If you open the migrator file, you will not find any complicated logic; there is only one public method, run, and that is pretty much it. The other private methods provide logging and recursively execute functions from the operations dependency. The implementation of Migrator class you can find below.

This implementation is straightforward and flexible, allowing it to be used not only for migrations but also as a seeding tool. Additionally, the data source and target can be entirely different from each other.

The only question left is: How do you run it?

Before we dive into running the solution, we first need to decide how to compile and store it. There are several options available, and I’ll outline a few:

We can build artifacts on a pre-push hook and push them to the repository (dist folder as an example).
We can build artifacts on a pre-push hook and store them in an S3 bucket.
We can set up a Bitbucket pipeline or GitHub Actions workflow to handle builds and storage. Note: that Bitbucket pipelines only store artifacts for 14 days, so you’d need to build them daily or on demand if you require longer retention.

There are plenty of other options as well, and you can choose the one that best fits your requirements.

To run it, I created my own CLI and published it in our private npm repository. There are several commands available, one of which is migration run. In my next article, I will discuss how to build your own CLI. For now, I will provide the code and the command.

The command looks like:

npx @company/cli migration --list="migration-foo, migration-bar"

and the code:

I hope the idea is clear: you simply provide a list of migration or seed script names, and the command looks up these files in a loop and then executes them. I’ve also omitted some details for clarity, but keep in mind that you’ll also need to provide additional arguments, such as the AWS token and environment.

Last but not least, if you appreciate my work, please subscribe.

A year-long journey

pedchenkoroman — Thu, 19 Sep 2024 12:17:28 +0000

Hi folks.

Long time no see! Frankly speaking, I've had good reasons for my absence. There have been a lot of changes in my life, but let's start at the beginning.

A new opportunity

First and foremost, I've taken on a new role at MyLette as a software engineer. The company recently acquired a new FinTech application and was in search of developers for a new team. Not only was I the first person to join the team, but I also brought my experience with AWS.

My initial significant task involved migrating the application from one customer to another, setting up all the necessary environments and connecting third parties. I'd be lying if I said the developers who worked on this application before didn't assist me. They were quite helpful, especially with the DEV environment, and to a lesser extent, the TST environment.

When all environments were set up and the team was complete, we had a couple of brainstorming sessions. Not only to evaluate the application but also to outline the next steps.

The first step was to create some integration tests and cover the core functionality. We chose the playwright framework for it, and I was responsible for setting it up. I'll share the results in my next articles.
The second one was to improve the deployment process. These findings were discovered during application deployment. I have a few words to say about the application. The application follows a multi-service architecture, and there were around 10 services. Most of the AWS resources were created by AWS CDK, but some resources were created manually. In addition to that, every service has its own configuration folder with files for each environment (dev, tst, acc, and prd). As part of this task, I created a config-gen CLI, and I'll share it in my next articles.
The third task was to address the findings from the AWS Well-Architected Framework. This framework consists of six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Each pillar includes several recommendations. One of the key recommendations was to eliminate direct lambda function invocations. Cross-service requests were previously implemented using the direct invocation approach, and I'll share it in my next articles.

There were many other interesting decisions and features implemented. I'll also share the most intriguing ones in my next articles.

Stay tuned.

The bot for BNG bank

pedchenkoroman — Wed, 17 Aug 2022 07:58:08 +0000

Introduction

Hi folks. My name is Roman Pedchenko and I am a full-stack developer. I always thought that my profession as a developer is just to write code but I had mistaken. I've realized that not only do I can create applications but also I can make some people's life easier. I bet everyone knows the situation in the world. In addition to that, there are lots of Ukrainian people who have come across some new challenges every day since February. Unfortunately, I could not solve all their problems but I've tried to solve one of them.

Problem

The people who have moved to Netherland and registered there have received some help and money from the government. But the problem is that they received a card from BNG bank and the plane operation such as checking their current balance becomes challenging.

Solution

I've created the telegram bot that after two steps of registration has just one button check your balance and when you click it it will send you the current balance. There are two options. The first one is just click to on the link and go through some steps of registration. The second is to set up the project on your own aws profile. If you choose this way please just open the repository and follow the steps.

Stack

It seems to me that if you are a developer you want to know what I use. First and foremost I use aws cdk to define resources. If you do not know what is it please open this one. Even though it is a really small bot and it can only parse the site and send the message with the current balance it uses at least four resources such as Lambda, DynamoDB, API Gateway and EventBridge. The first lambda is responsible for registering and handling the check balance event. The second lambda uses the puppeteer layer and receives the current balance and stores it. And last but not least lambda is invoked when the record with the new balance is changed and sent to the new balance. In addition to that, I have one more lambda and the main idea of this one is as a guard. It checks one specific header.

Architecture diagram

Conclusion

So, I published the bot to some telegram channels and I was wondered that more than 50 people are using it. Frankly speaking, I am happy that my knowledge and experience help someone. Thank you and take care.

Interface segregation principle (SOLID)

pedchenkoroman — Fri, 12 Aug 2022 22:00:00 +0000

Motivation

Hi guys. My name is Roman Pedchenko and I am a full-stack developer. Pleased to make your acquaintance. It is my first article and I ask you do not judge it too harshly. The idea to write the article appeared after my conversation with my friend Max Grom and I want to say thanks him.

Story

There are lot's of developers has an technical interview every day. Someone wants to receive a new job, someone the first one. But the problem is that you have to show your knowledge in a limited period of time which is why every answer is really important. In my humble opinion there are three types of answers. The first one is just academic knowledge. It means that you read about something but do not use it. The second one is you can describe or give an example from real world but you could not answer on the questioin what is it a principal or paradigm or pattern. And last but not least it is to combine the first and the second. Not only you know how to use it but also what you use. As you probably guess that the third
one amplifyes your position on an interview as a really good developer.

I bet everyone it does not metter you are a candidate or an interviever to prepare for the interview repeats SOLID principals. In addition to that I beleive that everyone tryes to use it every day but when someone asks could you explain them and gives some examples. It is always so dificult. In this article I will touch only one letter from abreviation but I hope it helps you to be more convincied.

Letter I

If you open wiki you will easily figure out that

The interface segregation principle (ISP) states no code should be forced to depend on methods it does not use.ISP splits interfaces that are very large into smaller and more specific ones so that clients will only have to know about the methods that are of interest to them.

I hope it sounds really easy to understand but as I wrote above not only teoretical knowled but also the examples where do we use it and here there are lots of people to get stuck. And here's a hint. It is easier than learning the definition itself. If you are a Angular developer that you are lucky person. Every time and every day when you create a component and add some hooks to component you use it.

export class AppComponent implements OnInit, OnDestroy {
  ngOnInit() {
  // some logic
  }

  ngOnDestroy() {
  // some logic
  }
}

As you can see we have to implement two interfaces in order to that hooks start to work and that's all. And oddly enough I believe that this answer will show you that at least you know the letter I from SOLID.

Thank you and break a leg at a job interview.