DEV Community: Farmin Farzin

Setting up Typescript and Webpack together

Farmin Farzin — Tue, 11 Oct 2022 20:17:51 +0000

Recently I became more interested and more engaged with Typescript and I'm moving some of my JS code to TS.

In one of my projects which was just a plugin, I had a huge JS file including the whole plugin code and I started to work on that.
So I started to split the code into multiple files and then I figured out that other people would appreciate it if I just move the code to TS so I started to add TS to the project.

First, I went through the code and made some functions based on the logic and as always I tried to make sure I apply some concepts of functional programming to make it easier for testing too.
Then I moved the functions into multiple files and export/import them. But I needed something to bundle them again back cause I needed to keep that format for my plugin.

Webpack

I added it to my project:

yarn add webpack webpack-cli

also added a new script to my package.json:

"build" : "webpack"

then I needed to configure it, so made a webpack.config.js file:

const path = require('path');
const webpack = require('webpack');

module.exports = (env, argv) => ({
  mode: argv.mode === 'production' ? 'production' : 'development',

  entry: {
    code: './src/start.ts', // The entry point for plugin code. bundle file name is 'code'
  },

  resolve: { extensions: ['.jsx', '.js'] },

  output: {
    filename: '[name].js',
    path: path.resolve(__dirname, 'dist'), // Compile into a folder called "dist"
  },

  plugins: [
    new webpack.DefinePlugin({
      global: {}, // Fix missing symbol error when running in developer VM
    }),
  ],
});

As I was testing it, I figured out some of the configurations to make sure my plugin is still working...
Awesome, after running yarn build I was getting my code.js file including all the codes from the functions that I exported from the other files.

Typescript

Alright, I wanted to have TS to make sure I'm type-checking. I just started to rename my .js files to .ts and then I installed the required dependencies.

yarn add typescript

and I added tsconfig.json file to my project

{
  "compilerOptions": {
    "module": "CommonJS",
    "target": "es5",
    "outDir": "./dist/",
    "sourceMap": true,
    "declaration": true,
    "lib": ["es2015", "dom"],
    "noImplicitAny": true,
    "allowJs": true,
    "moduleResolution": "node",
    "strict": true,
    "typeRoots": [
      "./node_modules/@types",
    ],
    "skipLibCheck": true,
  },
  "include": ["src/**/*.ts"]
}

now I was getting some weird errors, of course, cause I needed to hook my Typescript config with the webpack so I get a js file output after transpile-ing from TS and bundling the JS files

Webpack and Typescript (ts-loader)

Installed new dependency needed

yarn add ts-loader

made the changes to my webpack.config.js

const path = require('path');
const webpack = require('webpack');

module.exports = (env, argv) => ({
  mode: argv.mode === 'production' ? 'production' : 'development',

  entry: {
    code: './src/start.ts',
  },

  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },

  resolve: { extensions: ['.tsx', '.ts', '.jsx', '.js'] },

  output: {
    filename: '[name].js',
    path: path.resolve(__dirname, 'dist'),
  },

  plugins: [
    new webpack.DefinePlugin({
      global: {},
    }),
  ],
});

well, now when I ran yarn build, I had all my .ts files bundled into one JS file and had my type definitions beside it.
More, I added another script to watch the changes as I'm developing

"build:watch": "npm run build -- --watch"

Deploy Next.JS app on AWS using Amplify ("Invalid feature flag configuration" issue)

Farmin Farzin — Wed, 28 Sep 2022 02:45:10 +0000

In the Previous Post, I built my backend using Amplify and then I created my webapp with Next.JS and connected them.
I committed and pushed the code to my repository on GitHub and I want to configure my pipeline on Amplify to deploy my app in the full-stack way.

Let's jump to the Amplify Console, I chose my blog app that I have there. There are 2 sections:

Backend Environment: My API and data modelling that I did in the previous post
Host Environment: This is where I want to configure to deploy my webapp on AWS and also setup my pipeline so every time I push to my master branch, this automatically deploys the whole app. (FE and BE)

In the Host Environments tab, all the supported version control platforms are there.

I'm setting up with GitHub since my repository is there.
After choosing GitHub, it will ask for the repository and you might need to give access to Amplify on your GitHub to get your repository. Then I choose the branch I want to connect to and I DO NOT check monorepo option.
The next step is important since I need to connect this app to my backend. I can do it by finding my backend app in the list and pointing to the right environment.

Very Important Note

make sure you uncheck `Full-stack CI/CD allows you to continuously deploy frontend and backend changes on every code commit.
In the "Advanced Settings", I remove Amplify CLI from live package updates. Later I explain the issue that I had before the deployment with the different versions of Amplify CLI.

As you can see, Amplify automatically figures out that your app is Next.JS and make your build file. this is fantastic!
Click next and "save and deploy".

Identified Issue

The deployment fails in the build step with the error:
Invalid feature flag configuration These feature flags are defined in the "amplify/cli.json" configuration file and are unknown to the currently running Amplify CLI
This is because the version of the CLI on your local computer is not equal to the CLI version on the CI. We can fix that by making sure we have the same version on the CI.
I have CLI version 9.2.1 on my local and I make CI have the same version by modifying the build file.
On the left menu clicks on Build settings and Edit the build file by adding
`yaml
backend:
phases:
preBuild:
commands:
- npm i -g @aws-amplify/cli@9.2.1
build:
commands:
- '# Execute Amplify CLI with the helper script'
- amplifyPush --simple

Go back to the build history page and click re-deploy.

It should deploy successfully now! What I have done was, unchecking the full-stack CI/CD option on the Hosting Environment but then I deploy the backend in the build process, so in reality, I do deploy full-stack app every time we push to the Git.

That's it, from now on, every time I push a new code to my master branch, Amplify will deploy both the back and front end automatically.
If you look at your CloudFormation stacks on the AWS console, you'll find multiple stacks related to the app each one provision some resources either for API or Hosting.

Building my blog with AWS Amplify and Next.JS

Farmin Farzin — Mon, 26 Sep 2022 21:17:53 +0000

Is it me or everyone has the experience of going back and forth between having a custom personal blog or writing on a platform like Dev.to?!

I changed my mind multiple times in the last decade, I had my blogs done with WordPress, Jekyll, and Gatsby but then I moved to Dev.to and Medium and to be honest I might have put more effort into building blog sites rather than writing the blogs.

This time I want to have both, I'm gonna keep writing on Dev.to but also I want to re-build my new blog because I want to have a place to try new tech and enjoy building stuff. (instead of browsing on different social media at least I'd learn something eh?)

AWS Amplify

I'm a huge AWS and Serverless fan and because of that, I decided to have my blog built and deployed by AWS . Plus I'm an AWS Community Builder so it makes sense to have something to explore and build related to that.

So how exactly? I could just write my backend with services such as API Gateway, Lambda, and DynamoDB and host my frontend on S3, but I decided to make it a little bit more powerful (even though it's a blog and I don't need to) by using Amplify as my full-stack solution to build, test and deploy the app.
AWS Amplify is an amazing tool that I can easily provision my backend with IoC (infrastructure as code) and have my pipelines ready to work and connected to GitHub.
Amplify offers this great tool called Amplify Studio where you can graphically build your backend such as adding services or modeling your DB. I'm going to use it for:

build my data model
add file storage (S3 bucket)
add authentication (If needed)
add content (I can generate some mock data in case too) and more importantly, I'm gonna use it as my "headless CMS" since I can easily write my blogs there instead of implementing an editor on my frontend or writing directly on the DB!!

There is also Amplify Service in AWS Console that would help to configure my pipelines and environments and connect them to my GitHub branches.

It is worth mentioning that Amplify has a CLI tool that offers everything that we can do (and more) on the Studio and Console in CLI and it has a lot of options for scaffolding the code.
In general, I find two ways to implement the backend with Amplify:

Configuring and changing things on the Studio or Web Console (Getting deployed with CloudFormation) and then pull them on your local to have them as code and commit them to the Git
Implementing everything locally using Amplify CLI or just manually and push them to AWS (AWS CloudFormation Deployment) then commit to the Git

Next.js

On the Frontend side, I can easily scaffold a React app using AWS CLI but it uses create-react-app and will be deployed as SPA on S3.
Since I want to build a blog, SEO is a huge deal to me and I need SSR(server side) app to help me in that. because of this, my choice would be Next.js.(I'd go with SSG)
At this moment, Next.js is on version 12 but I will use version 11 since I know there are issues with Amplify and Next.js@12 (I will mention in another post)

Enough talking, let's code!

Backend

Let's head to Amplify Service in AWS Console here and create a new app (build a new app). Choose a name and click next. Now Amplify will prepare everything and creates the Amplify studio and some resources needed for the project.
After finishing setting up, you should see this page and you can launch your Studio.

Now we can do some data modeling easily. I just need a table for my post so I'll create one with this schema and deploy it.

After successful deployment, I have my backend fully ready to use this schema. what it means is that I have my APIs ready to do CRUD operations on the DB using this schema.

If you're wondering what has happened in the background, I have a plan to write a blog about What do we build on our AWS account with Amplify Data Model deployment?

The last thing to do in the Studio is to make some mock data to use on the frontend. To be honest, this is an amazing feature that will save a lot of time in the development process. Browse to content menu on the right and click on Actions => Auto-generate data and generate 10 rows of data.
Awesome, now I have some mocked data and actually, I will use this dashboard later as my "headless CMS" feature to write my blogs.

It's time to move to frontend and build my app and configure it to use my APIs.

Frontend

Yet, I don't have any code on our local and this is the time to build Full-Stack project locally.
Let's create a Next.JS app and then transform it into a full-stack app by pulling backend configuration.

Following Next.JS docs, I'll create our app running:

> yarn create next-app

? What is your project named? › my-blog

after setup is finished, if you run yarn dev, you should be able to see the app running on localhost:3000.

then make sure I have Amplify CLI installed on your computer.

npm install -g @aws-amplify/cli

Downgrade Next.JS and Webpack

As of today, Amplify doesn't support Next.JS@12 fully and because of that, it'd be better and safer to downgrade to version 11 (of course we'd miss some of the awesome features from v12 but that's fine). So I'm making it "next": "11.1.4" in the package.json. Since we're downgrading it to v11, we need to downgrade usage of webpack from 5 to 4 and for doing that we need to add webpack5: false to our next.config.js file. I also add SVG support to that file for the future and it looks like this:

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  swcMinify: true,
  webpack5: false,
  webpack: (config) => {
    config.module.rules.push({
      test: /\.svg$/,
      use: ["@svgr/webpack"],
    });

    return config;
  },
};

module.exports = nextConfig;

Connecting Frontend to our backend

As you could see on your Amplify Studio, there's a command for pulling the backend code and everything you need to connect your webapp to backend services.

amplify pull --appId {you-app-id} --envName staging

if this is the first time, it will authenticate you on the web and goes back to the terminal. You'll be prompted for your IDE, language, framework and source folder.

after the setup finishes, I'd have a lot of file changes on the repo. Let's commit the code to Git to make sure we keep track of the work that I'm doing.

Alright, let's do some coding and get the mocked blog posts that we have and show it on the blog.
let's install Amplify library: yarn add aws-amplify

open _app.js file and add amplify setup:

import Amplify from "aws-amplify";
import "../styles/globals.css";
import config from "../aws-exports";

Amplify.configure({
  ...config,
  ssr: true,
});

function MyApp({ Component, pageProps }) {
  return <Component {...pageProps} />;
}

export default MyApp;

Ok, now we can query some data in our index.js page to show it on the main page of our web app.

import React from "react";
import Head from "next/head";
import Link from "next/link";
import styles from "../styles/Home.module.css";
import { DataStore } from "aws-amplify";
import { Post } from "../models";

export default function Home() {
  const [posts, setPosts] = React.useState([]);

  React.useEffect(() => {
    async function fetchPosts() {
      const postData = await DataStore.query(Post);
      setPosts(postData);
    }
    const subscription = DataStore.observe(Post).subscribe(() => fetchPosts());
    fetchPosts();
    return () => subscription.unsubscribe();
  }, []);

  return (
    <div className={styles.container}>
      <Head>
        <title>My Blog</title>
      </Head>

      <main className={styles.main}>
        <h1>My Blog</h1>
        {posts.map((post) => (
          <Link key={post.id} href={`/posts/${post.id}`}>
            <a>
              <h2>{post.title}</h2>
            </a>
          </Link>
        ))}
      </main>
    </div>
  );
}

Amazing, now I see all my post titles.

I have everything I need to write/move my blogs now, I just need to do some coding on the frontend side to show them and style them as I want.
Let's commit the code to GitHub so I can build my pipeline for the deployment.

Continuous Deployment and Pipeline

In the next post, I'm gonna set up my CD.

Deploying different CloudFormation stacks based on different AWS accounts with AWS CDK

Farmin Farzin — Fri, 09 Sep 2022 03:01:57 +0000

AWS CDK is a great tool to write AWS infrastructure as code in your favorite language. Apart from all the awesome features that CDK offers, one of the things that I personally like is that it helps us to make sure we don't repeat ourselves or DRY principal

As most of the teams use multiple environments to build a product, we need to deploy resources to all those environments. Not that always the environments have the same resources or configurations, in fact, most of the time you'd have different resources or setups for your dev compared to prod or staging. In the AWS, you would have different accounts for different environments due to various reasons such as security, access distribution and etc.

In my case, I wanted to deploy to these environments as efficiently as possible. I have some common resources between these environments and some resources that are specific to each one. On the other hand, I want to have only one CDK app handling my deployments to different environments.

The solution that I came up with was using assume role I can figure out which account we want to deploy and based on that account number and a simple mapping between environments and accounts, I figure out which environment is the one we want to deploy so I just deploy the stacks that I want for that environment.

As you can see this is very easy since we're writing the infrastructure imperatively:

/* eslint-disable no-new */

const AWS = require("aws-sdk");
const cdk = require("@aws-cdk/core");
const { CommonStack } = require("./stack");
const { DevStack } = require("./dev-stack");
const { ProdStack } = require("./prod-stack");


const sts = new AWS.STS({});

const TEST_REGION = "eu-west-1";
const DEV_REGION = "eu-west-1";

const AWS_ACCOUNT_IDS = {
  development: "000000000",
  quality: "00000000",
  production: "000000000",
};

class App extends cdk.App {

  deployCommonStacks() {
    this.commonStack = new CommonStack(
      this,
      "commonStack",
      {}
    );
  }

  deployDevStacks() {
    this.devStack = new DevStack(
      this,
      "devStack",
      {}
    );    
  }

  deployProdStacks() {
    this.prodStack = new ProdStack(
      this,
      "prodStack",
      {}
    );    

  }

  async deployStacks() {
    const assumed = await sts.getCallerIdentity({}).promise();
    console.log("deployStacks() caller identity:", assumed);

    this.deployCommonStacks();

    if (assumed.Account === AWS_ACCOUNT_IDS.development) {
      console.log("Deploying to development account:");
      this.deployDevStacks();
    }

    if (assumed.Account === AWS_ACCOUNT_IDS.production) {
      console.log("Deploying to production account:");
      this.deployProdStacks();
    }

    return assumed.Account;
  }
}

const app = new App();
app
  .deployStacks()
  .then(acc => {
    console.log(`deployStacks() completed for account: ${acc}`);
  })
  .catch(e => {
    console.log("error in deployStacks():", e);
  });

Running Promises in Series with a delay between each (Throttling API calls)

Farmin Farzin — Thu, 08 Sep 2022 02:37:42 +0000

I wanted to call an endpoint updating an entity and I wanted to do it for 500 entities.
The API server had "rate limit" and I couldn't just use Promise.all since it will call the endpoint in parallel and the rate-limit would block my calls.

So I decided to write a script to bulk call the endpoint in series and have a 2 seconds wait before each call to make sure API server would not block them.

As I did that and it worked great, I thought it'd be advantageous to share!
Is there a better way for doing this?

How I'm doing this is starting from 0 and then adding up to that based on the array of data that I want to update. In the chain of promises that I have, I just added a delay before the API call, every time that I add 1 to the value that I'm passing to update the data till that number would be equal to my array length.

import fetch from "node-fetch";
import { data } from "./data.js";

const bearerToken = ""

const updateData = (id) => {
  return fetch(
    `https://www.url/${id}`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${bearerToken}`,
      },
    }
  );
};

const delay = () => {
  console.log(`Waiting: 2 seconds.`);
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve("2");
    }, 2000);
  });
};

const startTime = Date.now();

const doNextPromise = async (id) => {
  delay()
    .then((x) => {
      console.log(
        `Waited: ${x} seconds now calling the endpoint for updating data ${data[id]}`
      );
      return updateData(data[id])
        .then((res) => {
          if (res.status !== 200) {
            throw `Error updating data ${data[id]}: ${res.status}`;
          }
          return res.json();
        })
        .then((res) =>
          console.log(`Response: ${JSON.stringify(res)} for data ${data[id]}`)
        )
        .catch((e) => console.log(`Error: ${e}`));
    })
    .then((res) => {
      id++;
      if (id < data.length) doNextPromise(id);
      else console.log(`Total: ${(Date.now() - startTime) / 1000} seconds.`);
    });
};
doNextPromise(0);

Hacking Nespresso Expert Machine To Brew Coffee Using Custom Applications Via Bluetooth

Farmin Farzin — Wed, 07 Sep 2022 20:53:10 +0000

Nespresso Expert machine has Bluetooth ability which officially can be used only by Nespresso mobile app and it does not offer any API for 3rd party applications and services. Moreover, the Bluetooth services and characteristics are not documented and are easy to use by the other Bluetooth libraries plus there is no documentation for the Bluetooth packets payload that needs to be sent or received.

However, after searching a lot and sniffing the packets for a couple of days, I've been able to hack the machine and write the small Nodejs application using noble and express to control and monitor the machine with Rest API exposed by express through Bluetooth connection. As I did this application for my ex-company I cannot share the code but I'm going to explain how it works.

Thanks to this repo and also this nice medium post that basically helped me to understand how I need to sniff the packets.

Sniffing

First, I have installed the Nespresso app on my mobile and I connected to the machine and start using the application. Then, I sniffed the packets and opened them in the Wireshark to analyze them.

Authentication

I noticed every time on the app we start communicating with the machine (brew or monitor or just opening machine tab) it starts the conversation with a packet that has "8" at the start of its value and has 16 characters (mine was 85c55bc324a4170b) and it is writing on the 4th characteristic of the service 06aa1910f22a11e39daa0002a5d5c51b.
note: only one mobile can connect to the expert machine and every time that new mobile connects (which needs the device reset) the authentication packet will change.
I have used noble library for ble communication. I mention some hints ;) Authentication will be done by writing to 4th characteristic of the first service Buffer.from("85c55bc324a4170b", "hex"). after writing when you read the 5th characteristic, the data should be "2" which means authentication was successful.

Communication

I don't know why but before sending the brew command you need to write this Buffer.from("01100800000200c8000000", "hex") on the 4th character of the second service ("06aa1920f22a11e39daa0002a5d5c51b") and then inside the callback, you write this Buffer.from("03050704000000000101", "hex") which is low temp espresso. Now your machine should start brewing!!!

Brewing Types

Here are my observations:
0305070400000000 00 00 medium ristretto
0305070400000000 01 01 low espresso
0305070400000000 02 02 high lungo
0305070400000000 01 04 low hot water
0305070400000000 01 05 low americano
0305070400000000 01 07 low probably cleaning mode (all lights turning on together)
03060102 would stop the brewing (not always)

Machine Alarms (Errors)

On the second service, on 0th character read normally is like 0:64 1:9 2:13 3:64 4:128 5:0 6:255 7:255.
0: 64 is ok, 65: no water
1: 64 with 132 is brewing or busy, 64 with 2 ready, 64 with 66/67/7x :full disposal or no disposal on the second service, on 4th character read normally is like: 0:129 1:16 2:1 3:32 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0

For understanding slide error, read should be done after the command of brew
1: 195 slider error, 129 ok, 131 busy brewing

What you can do now?

What we have done was connecting it with an Alexa app and brewing coffee by asking Alexa to "make us a coffee". In fact, we added a box with different types of coffee capsules and made it cool for the demo but anyway you can just use the Alexa to brew you a coffee in the morning :)

Check out this gist at https://gist.github.com/farminf/94f681eaca2760212f457ac59da99f23

Have you ever accidentally delete all the table data with Prisma?

Farmin Farzin — Wed, 03 Aug 2022 19:30:02 +0000

This not a story about a bug but rather a feature!

Have you ever used table.updateMany or table.deleteMany in your code? we have and it was something like this:

const deleteChildrenOfThatParent = await prisma.children.deleteMany({
  where: {
    parentEmail: {
      equal: parent.email,
    },
  },
})

It was working great for a long time and you know what, we had TS checking the types too but somehow someday we lost all of our children data in the table.
We checked the database (Postgres) and we found out that at a certain time, all of the data has been wiped out! We blamed DevOps, hackers, AWS, ... but it was us not anyone else.

Finally, we figured out that we might had a situation that parent.email was undefined. But still we couldn't believe that deleteMany with a condition executed that. After some testing we were shocked but that was true and actually it made sense... our "where" clause made the code in a way that we deleted all the children only because of that undefined. we basically ran:

const deleteChildrenOfThatParent = await prisma.children.deleteMany({
  where: {},
  },
})

What could have saved us though?

Retro

So we had a retro around it and there were some angry stakeholders there! We promised them that this would never happen again! first thing first we just found every updateMany and deleteMany functions and we bookmarked them. We could just put a condition before each one to check for undefined but this wasn't a permanent solution neither scalable one.
After some investigation, we found a great solution, yeah Prisma itself came to help :p

Prisma Middleware

So what we needed to implement was something in the lower layer on the prisma which could check all the mutations to make sure we're not removing data that we don't want to delete. Prisma Middleware is that thing YES!

we just implemented the middleware in a way that we check every query (mutations) and filter out the updateMany and deleteMany ones and make sure that there is a condition on them!

async function main() {
  prisma.$use(async (params, next) => {
    // Check incoming query type
    if (params.model === 'Post') {
      if (params.action === 'deleteMany' || params.action === 'updateMany') {
        //Check the Where clause in params.arg.where, if it's empty just return
      }
    }
    return next(params)
  })

Sending DB migration Alerts To Slack Using AWS Serverless stack and AWS CDK

Farmin Farzin — Mon, 04 Oct 2021 19:51:35 +0000

As we were developing more features and scaling the product, our data guy was more confused with the changes on DB and wanted to catch up with the migrations that we were doing.

So I thought why not just let him know in Slack if there's a new DB migration every time that we release to production?

It should be easy right? We're using Prisma as our DB driver so any changes on the DB are tracked with schema.prisma file and we just deploy to production when we merge a PR to master branch on GitHub. So I just need to know when a PR gets merged if there's a change in the schema.prisma file, if so, then I can call Slack API and notify our data guy that we're having a new migration.

Let's do this!

I'm a Serverless and AWS-CDK fan and one of the main reasons is doing something like this, takes only a couple of hours!

So I need to:

Set up a Slack App to provide me a webhook URL to send messages to a specific channel
Build an endpoint that accepts Post and set it on webhook settings in the Github repo.
Write a Lambda function, which checks for DB migration change and calls the Slack API

Infrastructure as Code!

First thing first, let's start a CDK project:

mkdir pr-alert
cd pr-alert
cdk init app --language typescript

Ok, great! now I have everything ready to build a simple POST endpoint to use as a GitHub Webhook. I'll set the Webhook setting to Post to my endpoint every time we push. ("Just the push event.")
The reason is, in push events, there's a list of files that has been changed and then there's a property that indicated if the push is PR merged and there's a branch field too, so I can check if that's master or not. (more info on the hook)

Checking pr-alert.ts file in my bin folder, CDK initiate a new stack.

const app = new cdk.App();
new PrAlertStack(app, 'PrAlertStack', {});

and I see my stack in lib/pr-alert-stack.ts file where I should code my infrastructure.

Cool, cool! but before that, let me write my function which would have my whole logic to receive a webhook payload, finding if PR has been merged to master and then send a Slack message.
Let's create a file alert.js in a a new folder calling resources.

const main = async function (event, context) {
   console.log(JSON.stringify(event, null, 2));
}
module.exports = { main };

Awesome! so right now, every time I call this function, it should just print out the event in Cloudwatch for me. Later I'll write what I need...

Now, let's go back to my stack file and code my API endpoint, but before that, I need to install CDK packages for Lambda and API Gateway:

yarn add @aws-cdk/aws-apigateway @aws-cdk/aws-lambda

then let's jump into the stack

mport * as cdk from '@aws-cdk/core';

import * as lambda from '@aws-cdk/aws-lambda';
import * as apigateway from "@aws-cdk/aws-apigateway";


export class PRAlertStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // our function
    const handler = new lambda.Function(this, "alertHandler", {
      runtime: lambda.Runtime.NODEJS_12_X, 
      code: lambda.Code.fromAsset("resources"),
      handler: "alert.main",
      environment: {
        SLACK_CHANNEL_URL: "{GET_THIS_FROM_YOUR_SLACK_APP}",
        WEBHOOK_SECRET:"{GET_THIS_FROM_YOUR_GITHUB_REPO_WEBHOOKS_SETTING}"
      }
    });

    // our API
    const api = new apigateway.RestApi(this, "pr-alert-api", {
      restApiName: "PR Alert Service",
    });
    const postPRAlert = new apigateway.LambdaIntegration(handler, {
      requestTemplates: { "application/json": '{ "statusCode": "200" }' }
    });

    api.root.addMethod("POST", postPRAlert); 
  }
}

This is AWESOME! now we have our endpoint which is Post / in API Gateway, hooked with our lambda function, so every time we call this endpoint, we'll run the lambda function.
As soon as I deploy this, it will spit out the endpoint URL in my console.

cdk build
cdk deploy

note: for making cdk deploy work, you need to set up your AWS credential

Setting up a Github Webhook

Having the endpoint, I browse over to Github and make a new webhook: Repository Setting > webhooks > add webhook.
Paste the endpoint URL and choose a secret.

Send the alert to Slack from a Lambda function!

Let's go back to our function and write the logic. I break down the work into functions as I want to test them later and in general, I like it in this way:

Github's sending a signed secret over the call that needs to be validated to make sure it is coming from the right source.

const verifyGitHubSignature = (req = {}, secret = "") => {
  const sig = req.headers["X-Hub-Signature"];
  const hmac = crypto.createHmac("sha1", secret);
  const digest = Buffer.from(
    "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex"),
    "utf8"
  );
  const checksum = Buffer.from(sig, "utf8");
  console.log({ checksum, digest });
  console.log("timing", crypto.timingSafeEqual(digest, checksum));
  if (
    checksum.length !== digest.length
    // || !crypto.timingSafeEqual(digest, checksum)
  ) {
    return false;
  } else {
    return true;
  }
};

As part of the payload, there's commit prop which is an array of commits that a "push event" contains and inside each commit there's a list of files that have been changed in that commit.

const migrationCommit = (commits) => {
  const allModifiedFiles = commits.map((c) => c.modified);
  console.log({ allModifiedFiles: [].concat.apply([], allModifiedFiles) });
  if ([].concat.apply([], allModifiedFiles).includes("prisma/schema.prisma")) {
    return true;
  }
  return false;
};

Great! now I can just write the main body and use my functions:

const main = async function (event, context) {
  console.log(JSON.stringify(event, null, 2));
  const secret = event.headers["X-Hub-Signature"];
  if (!verifyGitHubSignature(event, githubSecret)) {
    return {
      statusCode: 403,
    };
  }
  try {
    var method = event.httpMethod;

    if (method === "POST") {
      if (event.path === "/") {
        const body = JSON.parse(event.body);
        const { ref, commits } = body;
        console.log({ ref, commits });
        if (ref.includes("master") && commits.length !== 0) {
          console.log("Pushed to Master");
          console.log("migrated?", migrationCommit(commits));
          if (migrationCommit(commits)) {
            // send message to the Slack
            await fetch(slackChannel, {
              method: "post",
              body: JSON.stringify({
                text: "<!here> DB Migration Alert: the commit that has been pushed to the master branch includes DB migration",
              }),
              headers: { "Content-Type": "application/json" },
            });
          }
        }
        return {
          statusCode: 200,
          headers: {},
          body: JSON.stringify("success"),
        };
      }
    }

    return {
      statusCode: 400,
    };
  } catch (error) {
    var body = error.stack || JSON.stringify(error, null, 2);
    return {
      statusCode: 400,
      headers: {},
      body: JSON.stringify({ error: body }),
    };
  }
};

By the way, I'm using 2 libraries that should be installed and packaged when I deploy to AWS. let's install them inside the resources package.

cd resources
npm init
yarn add crypto node-fetch

crypto helps me in decoding the Github signature and node-fetch enables me to call Slack API.

An exciting moment, let's deploy again:

yarn cdk deploy

Well, that's it, now every time we'd have a PR including DB migration, as soon as we merge it, we'll receive a message in Slack!

Find the whole project @

farminf / PR-Slack-Alert

Sending alert in slack if there was a DB migration in a merged PR

Frameworks War

Farmin Farzin — Tue, 14 Sep 2021 02:47:25 +0000

Last year we started building our portal from the scratch and we had the luxury of using any technologies, framework or library that we wanted to.

We didn't have a lot of time so we just wanted to speed up and write the business logic rather than wasting time on the configuration of tools together.
Our only restriction was the fact that we were all JS devs and we wanted to write JavaScript or TypeScript and also from the DevOps side, we knew we'll use K8s (As much as I love Serverless, unfortunately, it was out from the picture - Maybe another blog for another time to talk about this).

We had a list of suggestions and in there we had frameworks or multiple libs working together. We decided to pull out the libraries since we wanted an opinionated framework to give us the power to focus on writing the code for logic from the first day. Of course, there's a trade-off between those approaches as you would have the freedom to choose the best libraries (or the ones that you're comfortable with) or you can just limit yourself to a framework and its conventions/patterns.

This was very controversial as some people believe in the convention over configurations and some people believe in freedom of using whatever they want and configuring them as they want.

But How We Could Decide?

We prepared a list of features and considerations that we liked to have from the framework or library and gave them points.

Only Backend/Frontend or Full-Stack Some frameworks offer the Frontend in their architecture too, so there's no need to have a separated project but again, you'd be limited to their patterns.
Stable Version There are a lot of amazing frameworks out there, but not that all of them are production-ready!
Community Support You'd be surprised how much this matters. When you face a weird bug and want to scream and leave your job, you can just search it on the internet or open an issue on Github and people will help you.
Documentation No need to mention how much important it is.
DB Migration So this is one of the underrated features that will save your skin. The ability to manage your migrations, revert them, or apply them to new environments.
Debugging Easy debugging would save a lot of time from devs and make your delivery pipeline faster.
Learning Curve As always, the time constraint is the biggest issue in any product development and this will help for ramping up quickly as you code.
Testing There's always a way to test your code with the technology that you want. It depends on what type of testing. I would love to see the framework offering unit testing out of the box but for integration or end-to-end I might go with my solution.
Logging Any production application needs good logging. Get you out of trouble when your users are reporting bugs.
Configuration How much time we'll spend on the configuration of a project to have it set up? is it high or it's just a matter of running a command?
CLI tooling A lot of frameworks offer CLI commands which makes it easy to generate and scaffold a lot of code and saves a lot of time.
Backend API protocol We were looking for having Restful but some offer Graphql too!
Typescript support This was optional when we were making the decision but if I go back, I'd make it required.
Career One of my considerations when I want to learn technology X is "what is the future of X?". This time not for me but also the team, would devs be happy when they become experts of that framework? When we want to hire new devs, are they going to be excited working on it?

So Which Frameworks?

Our final list of frameworks were:

Decisions

The first decision that we made was around the first item on the list: Only Backend or Full-Stack.
We wanted to scale and we knew we're going to have multiple web apps consuming our API so we decided to have our Frontend separated.
It was easy, Next.js was the winner since it offers a ton on the frontend and satisfies most of our items on the list.

Then we took out the ones that didn't have production-ready version. Unfortunately Blitz and Redwood got eliminated and it was a pity honestly. I loved RedwoodJS so much that I started doing my side project just because I didn't want to miss learning it.

Later we got rid of Express + Prisma because for making it ready we needed to have a lot of configuration and adding a lot of other libraries for future requirements.

So we had the final list of Frameworks which had 'Stable Version', 'Community Support' and good 'Documentation'.
I'm not going to review those frameworks but as we researched and looked into them, not all of them had good tooling or CLI commands or DB migration management capability.

Between those we chose NestJS: it had almost all of the points that we wanted, the only issue at that time was Prisma which wasn't production-ready and its migration was experimental but right now after more than a year, we are super happy for the call that we made!
We have scaled 10x on the product side and 4x on the dev side and NestJs scaled very well, plus Prisma released a lot of features and made it easier for us to scale on the DB side.
Talking to devs on the team, they are super happy too. They have learned frameworks (both NextJs and NestJs) that are great for their career!

Leading An Agile Team

Farmin Farzin — Thu, 09 Sep 2021 20:11:09 +0000

"Agile Sucks" Era

Almost a decade ago, I was a developer in a team of 3, my manager, me and a designer/dev! My manager decided to apply some Agile concepts in a team of 3. We had stand-up every day, planning every two weeks and a lot of tickets on the Trello mostly for me!

Obviously, at that time I wasn't really interested in Agile since I really didn't know what that was and I actually thought the whole thing just slow us down which was true. Although I didn't know how I can give feedback about it to my manager, cause I thought I am not smart enough to understand the whole point of it.

Then I moved to other companies and in almost all of them, they had A version of Agile with different meetings and different tools to use. Interestingly, none of them were great, everyone was just following some rules. There was no mindset, no thinking behind those actions and a huge misalignment between different parties.

Fast forward this year: I'm leading an agile team and still, I'm not sure why we call ourselves Agile really. But then our company hired a new manager and I started reporting to him.
The first day he told me he is a big fan of Agile and he is very experienced in building agile teams and will support us to make some changes around it. I was like, well I know, everyone is saying that these days so... alright...
So he started sending us to some courses around Agile mindset. I wasn't really optimistic at first, but after having that course I started liking it and understanding it.
Then he slowly started gathering some metrics from the teams and started revising some of our processes and monitoring our interactions with the product team.

After only 5 months, we purposed our Jira to gather useful data, we slowly changed the way people were moving a story between states and started building the mindset instead of making some guidelines or best practices.

But Why Mindset?

I believe this is the most important part of Agile, you need to have the mindset, you can have no ticketing system, no meetings, no scrum master or product owner but just having that mindset helping you in decision making and improvement in your way of working every day.

I don't want to talk about beliefs, values and principles here but those are the things that make your mindset in general.

Business Value

For me, everything changed as soon as I started to focus on only "business value". The way that I approach different issues or requirements is different now cause first I look for that value and I try to measure that. A lot of time I pushed back on product because of this and even they were happy after with the results.
It is all about what your users gain, if you are solving a most difficult issue but your users can't get anything out of it or not much, then you're doing it wrong.

Right now, in the grooming meeting before thinking about how we can tackle new things to build or new features to add or even fixing what we have built, I just evaluate the value that we deliver to our users. If it is really what we need now and move the needle somewhere then alright, I'm on board. If it's just nice to have or the product wants to make something loveable only, I push back as hard as I can.

Experimentation

One of the most important elements for success, if you are in the early stages of your product, is experimentation. You or product have a lot of hypotheses but do you have enough resources to build all of those things? do you have time to wait and see if the new feature works for everyone? I'd say no and if this is yes, either you're super lucky or you're in dark.

Never build a big feature if you're not sure about the impact. You can easily understand its value by running an experiment with some small part of it on a portion of the users. Of course, you need to have a proper set of tools for experimentation and it is worth putting some effort to build those before doing any feature work.
This is how "feedback loop" works and how you can build a simple, ugly but great product for your users.

Team Metrics

As a team lead or Engineer manager, knowing how your team performs would be one of the best things to know, right? but let's call it system performance and not team performance, it's not about people, it is about the system. I believe people are doing the right thing if the system has been designed in the right way.

So, I didn't know that and didn't have any useful data to look even. We had some stats on our ticketing system but it was really hard to make any conclusion or even act based on those since they were incomplete and corrupted.

So the new manager started gathering some meaningful data and explained to everyone why we need those and what we should do for making sure we are providing the right data.

The most important part of this process was to make everyone sure that this data and the results won't be used in any performance review and it is not about people. There would be no name in the report and it'll be only about the team.
This is not even for comparing teams, the only purpose of gathering this data is finding bottlenecks and improve the flow of the system and that's it.

There is a great book called Accelerate, it explains what are the important metrics to focus on, and how to improve those. All of those are based on the researches that have been done on the different organizations and find out what are the important things to make a performant team.

Conclusion

I really like to write more about what we did to improve our delivery and how we are more agile today because I'd have appreciated it if someone was telling me sooner.

I think still there is a lot of room to improve but having the right mindset put me in the right direction.

PasswordLess Authentication with AWS CDK

Farmin Farzin — Tue, 31 Aug 2021 22:12:42 +0000

AWS CDK is a great framework to code the infrastructure on AWS, it is open-source, free, well documented and has a huge community.
For a lot of years, I was working with CloudFormation and YAML files, I didn't like it, I was spending a lot of time deploying my stacks and figuring out the errors, especially errors related to roles and permissions.

A couple of years ago, I joined a company whose whole architecture was Serverless, it was great because I was working with the same services too. We had our IaC with the Serverless framework and was working well. However, still wasn't great especially for the devs, writing in YAML files and reviewing it on the PR is not the best experience honestly.

I was lucky that my boss that time introduced AWS CDK to me and that time it was beta. I remember the first version of CDK that we tried and it was not mature, but, we liked it and we saw the future in it. So, we decided to write the new stacks with CDK and slowly catching up on the old YAML files.

We almost did after a year when CDK was released officially.

One of the great concepts about CDK is custom constructs. You can write a block of your infra and then use it anywhere multiple times.
There are great examples of these custom constructs in the documentation of CDK but also in Github you can find people building their own constructs and sharing them with others.
This is a repo presenting some of those awesome CDK constructs.

In this post, I want to write about the construct that I implemented a couple of years ago and used in multiple projects.

AWS CDK PasswordLess Construct

As you know there's no built-in passwordless authentication in AWS. (Some services like Firebase offer that out of the box).
So, I decided to build one and make it a custom construct so I can use it in any project that I want.
This Construct creates these resources:

Cognito User Pool
Cognito Pool Client
Cognito Lambda Trigger

Usage

You can add this construct to your project and have it as part of your IaC

yarn add aws-cdk-passwordless

import { CdkPasswordless } from "aws-cdk-passwordless";


new CdkPasswordless(this, "myPasswordLess", {
  mailSubject: "myStack - signIn", // subject of the mail arriving with code to confirm
  userPoolClientName: "myClientName",
  verifiedDomains: ["gmail.com"], // emails with the domains that are allow to signup
  postConfirmationLambda: lambda.Function(...) // passing a lambda which will be triggered after code confirmation
});

I should just note that postConfirmationLamda would be the function that you would call after the user passes the code confirmation and sign-in to your website. You can have any logic in there, I usually store a user object in the database in that function.

I'm not gonna go into the details about the code here as you can find the whole project in GitHub
but if you have any question just drop them down here :)