DEV Community: Genne23v

Writing Playwright UI Test

Genne23v — Mon, 17 Apr 2023 23:32:11 +0000

As Starchart is getting close to the launch, we have more UI changes than before. So I needed to learn how to write Playwright UI test for my recent PRs.

Playwright is a versatile UI testing frameworks which works on any browser and any platform. I used Selenium and Cypress before. In my opinion, Playwright's multi-browser support is easy to set up and the the way it creates a report that is easy to understand and debug. I really like that Playwright retries failed tests and sets a flaky flag even if it passes in another try. And you can run tests in parallel that reduces the deployment time on CI pipeline.

How to set up Playwright

You just need to run this command to install Playwright in your project.

npm init playwright@latest

Then you will have playwright.config.ts file and tests folder in your project root folder.

To run the test,

npx playwright test

To see the test reports,

npx playwright show-report

Writing Playwright test

import { test, expect } from '@playwright/test';

test.describe('Certificate Page', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/certificate');  //Pre-condition for each test
  });

  test('Request a Certificate', async ({ page }) => {
    const titleHeader = page.getByRole('heading', { name: 'Certificate' });
    const domainName = page.getByText('user1.starchart.com');

    await expect(domainName).toContainText('user1.starchart.com');
    await expect(titleHeader).toContainText('Certificate');
  });
});

You can set up a suite of tests by declaring test.describe('...'). Everything inside this function is a subset of this test suite.

And often you have to set up pre-condition or clear what certain test has changed. In this case, you can use test.beforeAll, test.beforeEach, test.afterAll, test.beforeEach.

Then the actual test goes with the tile of the test.

test('Request a Certificate', async ({ page }) => {

To test UI, you need to select UI element for action or assertion. There are multiple ways of selecting a certain UI element. When it is looking for fixed content in the UI component, Frame Locator is easy to write and read tests. And you can use filter locators when you have multiple elements from Frame Locator.
When the content in a UI element is dynamic, you can also use CSS locator.

Codegen Test Generator

You can simply let Playwright generate tests by using Codegen. Run this command in your terminal. You don't have to add an URL here.

npx playwright codegen [URL]

It will open up a browser. Every action you make to the browser will generate test code on the other window.

You can use Codegen in VS Code. Generating Tests

Playwright Configuration

In your playwright.config.ts file, import PlaywrightTestConfig type and devices. Add config setting in the object.

import type { PlaywrightTestConfig } from '@playwright/test';
import { devices } from '@playwright/test';

const config: PlaywrightTestConfig = {
  testDir: './test/e2e',  // Specify Playwright test folder
  timeout: 30 * 1000,  // Timeout for single test
  expect: {
    timeout: 5000,  // Timeout for expect()
  },
  fullyParallel: true,  // Run tests in parallel
  retries: 2,  // Retry set
  workers: 1,  // Number of workers running test
  reporter: 'html',  // Test report format
  use: {
    actionTimeout: 0,  // Time limit for action (0 is no limit)
    baseURL: 'http://localhost:8080',  //Base URL setup
    trace: 'on-first-retry',  // Trace condition set
    video: 'on-first-retry',  // Recording condition set
  },
  // Browser setting
  projects: [  
    {
      name: 'Google Chrome',
      use: {
        channel: 'chrome',
      },
      dependencies: ['setup'],
      testIgnore: /.*\.mobile\.spec\.ts/,
    },
    {
      name: 'Mobile Safari',
      use: {
        userAgent: devices['iPhone 12'].userAgent,
        viewport: devices['iPhone 12'].viewport,
        deviceScaleFactor: devices['iPhone 12'].deviceScaleFactor,
        isMobile: false,
        hasTouch: devices['iPhone 12'].hasTouch,
        defaultBrowserType: devices['iPhone 12'].defaultBrowserType,
      },
      dependencies: ['setup'],
      testIgnore: /.*\.desktop\.spec\.ts/,
    },
  ]

Conclusion

There are many other features to introduce. You can emulate browser setting such as locale, timezone, viewport, etc., as well as mock API, set a fixture. Official documentation is always the best place to start with and you can learn more details about features and ask questions to solve your problems. And there are always multiple framework choices for a certain purpose. I think it's good to try a different framework in your project to compare features and learn more technology from it. I think Playwright is a good choice for your UI testing.

What is HTTPS?

Genne23v — Sat, 08 Apr 2023 17:59:42 +0000

HTTPS stands for Hypertext Transfer Protocol Secure which is the encrypted version of HTTP. It is a protocol used for secure communication over the internet. When you visit an HTTPS website, data is encrypted to exchange between your web browser and the website. So your data is confidential from someone else and the data integrity can be ensured.

How does HTTPS work?

HTTPS needs a certificate from third-party to secure communication and verify that the website is compliant. This certificate is known as SSL (Secure Socket Layer) certificate. This certificate encrypts a connection and provides another layer of security for sensitive data that
must not be accessed by any attackers. This additional security can be extremely important to protect secret information such as user's identification information, address, credit card number, etc.

How is data secured in HTTPS?

HTTPS encryption uses TLS (Transport Layer Security) protocol. It requires private key and public key to encrypt data between two parties. The private key is controlled by the owner of a website. It's used to decrypt the data that is encrypted by a public key. The public key is available to everyone who interacts with the website securely.

How does SSL certificate look like?

Below picture is one example of SSL certificate.

Image Source: https://www.geocerts.com/support/generating-a-csr-for-a-wildcard-ssl-certificate

It contains following information.

The domain name that the certificate was issued for
Where the certificate was issued to
Which certificate authority issued it
Issue/Expiration date of the certificate
The certificate authority's digital signature
Associated subdomains
The public key (the private key must not be publicly available)

Once the certificate is issued, installation and activation are necessary on the website's origin server. When it's activated on the server, the website will load over HTTPS and all incoming/outcoming traffic from the website will be encrypted and secure.

Who can provide SSL certificate?

Domains need to obtain valid SSL certificate from a certificate authority (CA). A CA is a trusted third SSL certificate provider. The CA will also digitally sign the certificate with their own private key, allowing clients to verify it. Most CAs will charge a fee for issuing an SSL certificate. What Starchart is using is Let's Encrypt which provides SSL certificates for free as a non-profit SSL certificate provider. Both free and paid SSL certificate providers offers all necessary security. You need to choose a provider based on your need. You can read more in this article. A free or a paid SSL certificate - which one is better

Conclusion

To build a website that exchanges critical information such as transaction data, important personal information, etc., HTTPS is mandatory. Although I didn't write any code for certificate related part, Starchart gave me an opportunity to learn more about SSL certificate. I think this is one good aspect about contributing to open source project since I usually build a project on subjects that I'm most familiar with. I'm learning many tech stacks and more detailed knowledges about internet through Starchart.

What is DNS Record?

Genne23v — Sat, 01 Apr 2023 19:01:54 +0000

It's been about 10 weeks since I started working Starchar project which is the DNS service for student projects. At the beginning, many of us didn't have clear understanding about the terms like domain, subdomain, name, records, etc. Now I have a better understanding of these terms now. I would like to share the terms that I have learned in this project.

What is DNS Record?

A DNS record is a type of data stored in a domain name system (DNS) server that maps a domain name to an IP address or other information about the domain. So what Starchart will do is to create a DNS record that is connected to the user's IP address or domain. Also it can be linked with text type information. I will explain more details about these types.

Types of DNS Record

As I mentioned previously there are types of record that can be linked with domain name. Starchart supports A, AAAA, CNAME, TXT types.

A Record: Maps a domain name to an IPv4 address. So you can name your subdomain like my-project and enter your public IP address as an A record. Your local project can be accessed via my-project.user.mystudentproject.ca.
AAAA Record: Maps a domain name to an IPv6 address. It works same as above. You just need to select AAAA type and enter IPv6 format IP address.
CNAME (Canonical Name) Record: Maps a domain name to another domain name. If the user has an own domain already, it can be mapped with other domain name. So your project have two different domain names. Also CNAME is often used for subdomains.
TXT (Text) Record: Allows the domain owner to add arbitrary text to the DNS record. There are many different types of TXT records such as SPF, DKIM, DMARC, etc. This is one example of SPF record.

"v=spf1 ip4:192.168.0.1/16 -all"

This specific TXT record means that use SPF version 1 and email should be sent from a range between 192.168.0.1 and 192.168.255.255 and deny all sources. You can find more details of other TXT record types in this link.

There are also other types of record as below.

MX (Mail Exchanger) Record: Specifies the mail server responsible for handling email for a domain.
NS (Name Server) Record: Specifies the authoritative DNS server for a domain.

What will Starchart do?

Students can create their own subdomain under the root domain mystudentproject.ca using Starchart. Starchart will create a DNS record using student's subdomain name as well as HTTPS certificate. Then the project will be publicly accessed and secured on the internet.
Route53 allows up to 10,000 records in one hosted zone which means same as 10,000 subdomains under mystudentproject.ca. Another hosted zone will incur more cost. You can plan the budget using AWS Route53 pricing website.

More Terms in DNS

Route53 DNS records consist of Type, Name, Value, TTL and other optional values. Here Name means fully qualified domain name, aka FQDN. And Starchart only takes a subdomain from the user to complete the input for Route53 Name. TTL is Time to Live that decides how long it takes for record updates to reach the users. Longer TTL speeds up DNS lookups by increasing the chance of cached results, at the same time the updates to your records take longer to go into effect.

Conclusion

We didn't have the right naming for domain related variables. One example is that I completely forgot what my variable domain meant even though I wrote the DNS code by myself. So we changed the variable names to either fqdn or subdomain. I can imagine later a new contributor would have great confusion from variable names. If I had a better knowledge, I would have used the appropriate naming. I hope this post is helpful to understand DNS record system and Starchart.

Managing Work Scope in Software Project

Genne23v — Sat, 25 Mar 2023 13:59:59 +0000

Understanding the code base, writing good code, and flawless communication with maintainer are the fundamentals of the open source work. Luckily, I work with some of very experienced people in Starchart project. I learned that work can be faster when managing the work efficiently. Here's my lessons learned.

Minimize your work scope

If the PR has thousand line changes, it takes a lot of efforts not just for the code writer, but also the code reviewer. Also it means more chances of breaking existing code base. So it will take more than a week to get merged after rounds of reviews and updates. The longer the PR is under review, the more the PR gets conflicts especially if the project is very active. You can write sub issues of a large issue to minimize the work. If some parts of the issue is not the core of the work, you can separate for later follow-ups. Any work should be planned to finish within a week as project or organization usually is run on a weekly basis.

Split large scope of work into different phases

Planning big work into multiple phases is not easy task unless experienced. But this is something that I learned in Starchart project. The experienced code writer plans multiple steps for difficult tasks. I have seen the experienced developers build skeleton code without, write comments to show the intention, and structure the files without functionality yet. You can still have good code review feedbacks based on the function flow. Then slowly build up each module or functions. This way the work can be processed faster as you don't have to finish a large module at once. Instead you can finish your function, get it merged, and work on next function. I know it's not easy to plan things if the area of the code is new to the code writer. But I think it's a great skill and it's better to practice now.

Plan future improvement instead of fixing everything in one PR

Often times you get a feedback to change something during the code review. That change request can be essential but sometimes it can be improved later. Then you can suggest that you will write another issue to follow up. This way you can avoid another round of review and keep your change scope as originally intended.

Conclusion

When I work alone, I only focus on implementing features and fixing bugs. But larger project with a team requires more than implementing code. Design the system, continuously refactor code for better maintainability, manage issues and review someone else's code. And this planning the work will make a better look of the developer. The work progress is fast and easy to review. This technique will be more obvious in real world projects if you see PRs from the experienced.

Advanced Types in TypeScript

Genne23v — Sun, 19 Mar 2023 04:56:51 +0000

Strongly typed languages increase the safety of programs. It doesn't mean that strongly typed languages are always better than weakly typed languages. Weakly typed languages can reduce the development time. So programmers should choose a programming language based on the purpose of the program. Although I contributed some of Typescript projects, but I didn't have much knowledge about Typescript typing. I learned some of very useful typing techniques during Starchart code review.

When using the same type with different property names

You could face the situation that it doesn't make sense to have two different type names while only small portion of the properties varies. My DNS module in Starchart is a good example to explain this instance. At the beginning, we use the variable name as any type of domains. But in our app, this name could be subdomain or fully qualified domain name which can be shortened to fqdn. This is very confusing when writing a code to use the function. You don't have any idea whether you are going to get subdomain or fqdn which could result in a wrong implementation or frustrating the writer to look up all the code to figure what it actually is. So we can do something like this.

type Name = { subdomain: string } | { fqdn: string };
interface BaseDnsRequestData {
  type: string;
  value: string;
}
type DnsRequestData = BaseDnsRequestData & Name;

So DnsRequestData can have either subdomain or fqdn. When you extract the data, Typescript knows what type is passing only in runtime. So you should extract the data as below.

let name;
if ('subdomain' in data) {
  name = data.subdomain;
} else {
  name = data.fqdn;
}

When type naming is not necessary

Sometimes it's better not to name the type when passing data is obvious or varying similar names is more confusing. So you can instead of defining as below,

interface Record {
  type: string;
  name: string;
  value: string;
  description: string;
  ports: string;
}

type DnsRequestData = {
  type: Record['type'];
  name: Record['name'];
  value: Record['value'];
  description?: Record['description'];
  ports?: Record['ports'];
}

You can simplify the type without naming.

Pick<Record, 'type' | 'name' | 'value'> & Partial<Pick<Record, 'description' | 'ports'>>

Conclusion

It was really good for me not just to learn these techniques, but also to help me to write better code. It seems not important for someone who develops something quickly, but typing is very important when many contributors or developers are working together. It finds the potential errors quickly and gives an idea of what this code does. It also takes a lot of learning from the experienced. I never thought about this kind of problems when I was working alone. Having the experienced developers' feedback is really helpful to learn how to write better code. Thanks for reading my article!

JavaScript & TypeScript ORM Prisma

Genne23v — Sat, 11 Mar 2023 01:40:49 +0000

Our open-source project Starchart uses Prisma which is ORM for any JavaScript and TypeScript projects. As it states on the doc, it's the next generation ORM that provides a lot of convenient features. I'm really amazed how it makes things easier our project. Let me show how to use Prisma briefly in this blog.

Getting started

Once you have your project ready, you can install Prisma with below command.

npm install prisma --save-dev

And initiate Prisma in your project. You can choose your data source like SQLite, PostgreSQL, MySQL, MariaDB, MongoDB, MS SQL server, etc. In this case, I choose MySQL.

npx prisma init --datasource-provider mysql

Writing schema and init PrismaClient

Now it's ready to use Prisma. Let's set up some schemas.

model User {
  id         Int       @id @default(autoincrement())
  email      String    @unique
  createdAt  DateTime  @default(now())
  updatedAt  DateTime  @updatedAt
  name       String?
  public     Boolean   @default(false)
  record     Record[]
}

model Record {
  id           Int       @id @default(autoincrement()) 
  username     String   
  description  String    @db.VarChar(255)
  user         User      @relation(fields: [username], references: [username])
}

This is an example to show how to write Prisma schema. You can see how to set ID column, make a property unique in the table, set initial date property when created, as well as default value and length limitation. And the table relationship between User and Record. You can find more details on Prisma docs Defining attributes.

CRUD operation

Next step is to create a new SQL migration file and run the SQL migration file against database. Below command creates dev.db inside the prisma directory.

npx prisma migrate dev --name init

Now you need the PrismaClient initialization script.

import { PrismaClient } from '@prisma/client'

async function init() {
  const databaseUrl = new URL(process.env.DATABASE_URL);

  const prisma = new PrismClient({
    datasources: {
      db: {
        url: databaseUrl.toString(),
      }
    }
  });

  prisma.$connect();
}

You can write queries. Here's CRUD examples.

const user = await prisma.user.create({
  data: {
    name: 'John',
    email: 'john@starchart.com',
    records: {
        ....
    },
  },
});

await prisma.record.findMany({
  where: { username }
});

await prisma.record.update({
  where: { id },
  data: { description: 'Updated record' },
});

await prisma.record.delete({
  where: { id },
});

You can also use methods like findUnique, findFirst, updateMany, upsert, deleteMany, count, etc. If you have a bit of SQL experiences, you would understand what each of these command does. Let me leave a link to take a look at further details Prisma CRUD operations.

Seeding and Prisma Studio

What's great about Prisma is the various tools that makes your development easier. You can set up a seed script to create initial development data. In order to do so, you need to add a prisma script as below.

//In package.json for Javascript project
{
  "name": "starchart",
  "version": "0.0.1",
  "prisma": {
    "seed": "node prisma/seed.js"
  }
}

For Typescript, you need to install @types/node and add Typescript version script in package.jon.

{
  "name": "starchart",
  "version": "0.0.1",
  "prisma": {
    "seed": "ts-node prisma/seed.ts"
  },
  "devDependencies": {
    "@types/node": "^14.14.21",
    ....
  }
}

Then you can run npx prisma db seed to seed your database.

Prisma also provides the visual editor for easier data view/edit.

You can run npx prisma studio to see the visualized tables.

Conclusion

Developers need to work with database not just to write CRUD functions against database, but also database connection based on environment, create/change schema, migrate database, seed development database. Prisma can help you do these jobs easy in a sophisticated way. The fastest way to get familiar with Prisma is to go through the tutorial on the official website. I hope my article is helpful to know Prisma.

Create BullMQ Workflow

Genne23v — Sat, 25 Feb 2023 01:57:05 +0000

I think the one of the most challenging work for Starchart app is that we need to support long asynchronous job to complete. The app adds a record to DB and create a record in Route53. Then it needs to get a HTTPS certificate. It takes at least a few minutes to complete this process. There were multiple technical approaches to solve the problem. We chose to use BullMQ queue system. I hadn't tried any of queue systems before. It was a good opportunity to learn how to use BullMQ, especially Flows.

`BullMQ` setup

Before using BullMQ, a couple of dependencies are required.

npm i bullmq ioredis

Also you will need a redis docker container to test in your local. Here's the Redis Docker image link.

BullMQ Flows

I used BullMQ Flows as it suits our needs that the jobs should be run sequentially. What needs to happens is...

Add a record to MySQL database
Create a record in Route53
Continuously check Route53 whether it's deployed successfully or fails. It takes about 1 minutes according to AWS
Update the Route53 sync status in DB

Let's write a flow for these jobs.

const flowProducer = new FlowProducer({ connection: redis });

export const addDnsRequest = async ({ username, type, name, value }: JobRecord) => {
  // Step 1. Create a record in MySQL for a domain with pending status
  const addDbRecord: FlowJob = {
    name: `addDbRecord:${name}-${username}`,   //Set up unique name for each job
    queueName: 'add-db-record',      //Set a queue name. It will be used as a part of job name unless you configure it manually
    data: { username, type, name, value } as JobRecord,
    opts: {
      failParentOnFailure: true,    //If any one of the job fails, the whole flow will fail
      attempts: 5,    //Retry 5 times
      backoff: {      //Wait for 15 seconds when retrying. Each retry wait will be exponentially increased.
        type: 'exponential',
        delay: 15_000,
      },
    },
  };

  // Step 2. Request Route53 to create a record
  const createDnsRecord: FlowJob = {
    name: `createDnsRecord:${name}-${username}`,
    queueName: create-dns-record,
    data: { username, type, name, value } as JobRecord,
    opts: {
      failParentOnFailure: true,
      attempts: 5,
      backoff: {
        type: 'exponential',
        delay: 15_000,
      },
    },
  };

  // Step 3. Poll Route53 to check connection status of the domain until it's ready
  const checkDnsStatus: FlowJob = {
    name: `checkDnsStatus:${name}-${username}`,
    queueName: checkDnsStatusQueueName,
    children: [createDnsRecord],    //With this dependency, parent job will not move to queue until child job is processed
    opts: {
      failParentOnFailure: true,
      attempts: 5,
      backoff: {
        type: 'exponential',
        delay: 60_000,
      },
    },
  };

  // Step 4. Update the MySQL record with the active or error status
  const syncDbStatus: FlowJob = {
    name: `syncDbStatus:${name}-${username}`,
    queueName: syncDbStatusQueueName,
    children: [addDbRecord, checkDnsStatus],    //This job needs two children jobs to be completed
    opts: {
      failParentOnFailure: true,
      attempts: 5,
      backoff: {
        type: 'exponential',
        delay: 30_000,
      },
    },
  };

  return await flowProducer.add(syncDbStatus);    //All of jobs are added to the flow by adding the last parent process

Set up workers for each queue

Now we need to set up a Worker to execute these jobs. Since most of the workers are similar, I will just add the most useful example.

export const createDnsRecordWorker = new Worker<JobRecord>(
  'create-dns-record',    //Define a queue for the worker
  async (job) => {
    const { username, type, name, value } = job.data;    //Get data fed from the queue. 

    try {
      return createRecord(username, type, name, value);
    } catch (error) {
      logger.warn('Could not create a record in Route53', error);
      throw error;
    }
  },
  { connection: redis }
);

You can customize your worker based on your needs. When the whole flow should fail due to one of the process is failed, you need to throw UnrecoverableError.
Also you can capture values from multiple children jobs as below.

const values: { [jobKey: string]: string } = await job.getChildrenValues();
const key = Object.keys(values)[0];
const value = values[key];

Conclusion

I haven't had an experience to use queue system. By using BullMQ, we are able to solve many problems such as sending scheduled emails or notification email, creating HTTPS certificate, etc. I'm sure there are many problems that can be solved with queue system. I hope this article is useful to start writing your work flow.

Useful Regex For Web Developer

Genne23v — Fri, 17 Feb 2023 23:51:07 +0000

Regular expression is hard. But it's useful for web developers to limit the input from users. There are many common examples to validate user input but sometimes we need to understand it to add more logic to customize it. Let me introduce some regular expression that I often use for user input validation.

Limiting domain name input

For Starchart project, I needed to restrict user's subdomain name as following rules.

Domain name pattern should be [name].[studentId].rootDomain.com
Domain name can contain only alphanumerical characters, along with '-' and '_'
Domain name should not start or end with -
Domain name cannot contain multiple consecutive '-' or '_'
studentId only contains alphanumerical characters

After writing all possible pass/fail test cases, I came up with below regex. Let me explain each block of expression for better understanding.

/^(?!.*[-_]{2,})(?!^[-])[a-z0-9_-]+[^-_]\.[a-z0-9]+$/

^ means the beginning of the string.
(?!.*[-_]{2,}) -> ?! means the pattern inside the parentheses must not match for the overall regex pattern to be considered a match. .* means zero or more characters and the meaning of [-_]{2,} is two or more than two consecutive - - or _. So multiple - or _ must not be in the string.
(?!^[-]) -> ?! is the negative lookahead assertion that is explained above. ^[-] means the string must not start with -.
[a-z0-9_-]+ -> One or more of lowercase letters, numbers, -, and _ allowed.
[^-_]\. -> - or _ cannot be placed before .
[a-z0-9]+$ -> One or more of lowercase letters and numbers are allowed until the end of string

It's still difficult, but it's easier when you break down the expression.

Password requirement regex

Here's other example that I use for password validation.

^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[@$!%*?&])[A-Za-z\d@$!%*?&]{8,}$

?= is positive lookahead assertion that checks if the pattern inside the parentheses matches. In this first block of regex, it checks whether it has at least one lowercase letter. Next block checks if it has at least one uppercase letter.
\d represents numerical characters which are same as [0-9].
Next block checks if it has at least one of following special characters @$!%*?&.
Last block validates the string consists of alphanumerical characters and special characters (i.e. @$!%*?&) and its length must be at least 8 characters long.

Email validation

Here's another one for email validation.

^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$

[\w-\.] means it only allows one or more of word characters, -, and ..
([\w-]+\.) checks whether the pattern consists of word characters and - only followed by one ..
[\w-]{2,4} is the pattern that has only word characters and - and its length is between 2 and 4.

Conclusion

Regular expression is hard even for very experienced developers. And you don't want to use the one for production that is not proven by thousands of users. I had an experience that I couldn't add my website address on one of the biggest Canadian companies's website because my website address is treated invalid. It was frustrating. So it's always a better idea to use solutions that already exists and have been proven by many users. You can use a library like validator although you still have to add your own logic to customize it. Also there are better libraries for specific types of validation such as IP address, zip code, etc. But it's good to keep trying to read regex and practice so that you can add your own logic based on existing one.

Useful JavaScript / TypeScript Techniques

Genne23v — Sat, 11 Feb 2023 06:04:09 +0000

I had a lot of technical learning through code review this week. It's been over 2 years since I started JavaScript and I'm still very new to TypeScript. As a beginner, I've been trying to make something work, but often ignored to learn some of basics. I haven't felt the strong need of these techniques that I'm about to share. I think it's very useful to write better code.

Using import / export

JavaScript has many different forms of import / export. Although I can easily find the documentation how to use it, it's hard to remember at the beginning, then I started to understand through experiences. My very first question was when to use export const vs. export default. Here are differences.

You can export multiple element using export const and element name cannot be changed
You can export only one element using export default and the name can be changed

Here's matching export and import

export default Element = () => ...

import Element from './Element'
import MyName from './Element'

export const Element = () => ...

import { Element } from './Element'  //It works
import Element from './Element'      //Return empty object

Also you will face a situation that you want to use the same name for different modules. I had the exact case that I need to create a record in DNS module and database. Here's how you can use the same name while avoiding name conflict.

import * as dns from './dnsModule'
import * as db from './database'

//You can use same name methods as below
dns.createRecord()
db.createRecord()

I often use long descriptive function names. But I think the function names are better when they are concise and intuitive as possible. In above case, it does the same action create a record in different modules. Technically they are not the same thing, but it's easily understandable and intuitive for anyone with basic knowledge of programming.

Reducing parameters by passing object

I'm familiar with creating a class or an object in other languages like Java, C++, etc. I learned that TypeScript makes this much easier. You can declare a type by using keyword interface or type. I like the convenience that I can make some of properties optional and I can combine two different interfaces.

interface Engine {
  displacement: number
  fuelType: string
}

interface Wheel {
  size: number
  material?: string     //Optional property
}

type Car = Engine & Wheel;   //Car has all of Engine and Wheel properties

You can find more examples of using type and interface in this article. Official TypeScript website provides playground to practice how it works. type vs interface in TypeScript.

Conclusion

You can practice coding on your own, but I think learning is faster when I work in a company or participate in open source community. It's obvious that I can't have any advice if I work alone. Looking at other's code is very helpful too. I'm looking forward to next contribution to Starchart project.

Lesson Learned From Code Review

Genne23v — Fri, 03 Feb 2023 16:46:26 +0000

I had rounds of code reviews for my PR to add Route53 DNS module to Starchart. Here's what I learned from the code review.

Working code is not enough, it should be easily readable for other developers

I wrote a createRecord function that creates a record in Route53 hosted zone. At the beginning, it didn't have hostedZoneId as I was thinking we would use only one hosted zone and I could treat it as an environment variable. Then I needed ability to set a hosted zone for testing purpose and added hostedZoneId to the function as the last parameter. But this is not a good practice. Some people would think hostedZoneId is something that exists inside a record. It works same if I add hostedZoneId in any order in the function, but the code should look clearer for others to understand.

Don't let same logic duplicated on the code

Even though I know this rule and try not to write same logic in different functions, but it occurs when I don't look at my working code closely again. Duplicate code can be reduced by reviewing and refactoring multiple times before pushing it to repo. Again working code is not enough. Repeating code will result in bugs once someone fails to modify all of the same logic. Good refactoring skills come after a lot of practices. Once you confirm that the code is working, it's time to find places to improve.

Function should do only one thing whenever possible

This is a good principle to write good tests or write tests more easily. Also it's good to produce the right error message when something breaks. Function name should be as clear as possible. validate() does not give any clue what to validate. If it can be broken down to validateRecord() and validateDomain(), it will be much easier to read and probably people don't have to look into function implementation.

Don't use unnecessary syntax and use `linter` to set a rule to maintain consistency

Which code looks cleaner and easier to read?

function foo() {
    if (x) {
        return y;
    } else {
        return z;
    }
}

function foo() {
    if (x) {
        return y;
    }
    return z;
}

Both work same, but we don't want to write unnecessary statement. We can avoid unnecessary syntax or error-prone code by using linter. It will help you get used to certain principles.

Conclusion

You will look much more professional if you look through your code and improve before someone reviews. Also thoroughly take reviewer's comments and discuss if you have a different opinion. Opinions are always debatable. You will learn more from discussion.

Create Domains Using Route 53 SDK

Genne23v — Thu, 26 Jan 2023 01:00:20 +0000

The team for Starchart project is in planning phase. It's not as exciting as coding. But research is important and it's good time to explore technologies that I have not tried or learn more details about things that I tried. Starchart will create a domain for Seneca students. To do so, the team will use Route 53 and I spent some time on understanding AWS SDK for Route 53 since we need to automate the steps to create a domain in Starchart web app.

Setup Route 53 SDK

To learn Route 53 SDK, I started with official SDK page. To use Route 53 SDK, you need to install SDK by running npm install @aws-sdk/client-route-53.

Now I need to start coding. First step is import Route53Client and set up configuration.



const { Route53Client } = require('@aws-sdk/client-route-53');

const config = {
    credentials: {
        accessKeyId: '******',
        secretAccessKey: '******',
        sessionToken: '******',
    },
};

You need to get your own credential information from your AWS account. I use student account which might be different from normal users. I can get my accessKeyId, secretAccessKey, and sessionToken in AWS Details as shown in below picture.

Creating Hosted Zone

The first method that I need to try is CreateHostedZoneCommand. Hosted zone is a AWS term that represents a collection of records that can be managed together, belonging to single parent domain name. And here's code snippet to use CreateHostedZoneCommand.



const { Route53Client, CreateHostedZoneCommand } = require('@aws-sdk/client-route-53');

const input = {
        CallerReference: new Date().toString(),
        Name: 'test.starchart.com',
    };

const client = new Route53Client(config);
const command = new CreateHostedZoneCommand(input);
const response = await client.send(command);

To create a hosted zone, all you need is CallerReference and Name. You can also set up more input parameters such as HostedZoneConfig, VPC, etc. based on your need. CallerReference must a unique value for your request. If you use the same CallerReference, you will get an error. And Name is for your domain name. If response is successful, you will receive metadata of your request, location, change info, hosted zone info, nameserver info.

Now I have records inside the hosted zone that I created.

Add/Update/Delete Records in Hosted Zone

I can add a new record or update existing records with ChangeResourceRecordsSetCommand function. Below is the basic syntax how to create or update your record.



const { Route53Client, ChangeResourceRecordSetsCommand } = require('@aws-sdk/client-route-53');

const changeBatch = {
    Action: 'UPSERT',
    ResourceRecordSet: {
        Name: 'mydomain.starchart.com',
        Type: 'A', 
        TTL: 60,
        ResourceRecords: [
            {
                Value: '192.168.0.1',
            },
        ],
    },
};

const input = {
    ChangeBatch: changeBatch,
    HostedZoneId: '****************',
};

const client = new Route53Client(config);
const command = new ChangeResourceRecordSetsCommand(input);
const response = await client.send(command);

For input parameter, you need to provide changeBatch object and HostedZoneId. In this case, I use UPSERT action. This is a way to simplify the method that it can add a new record if it doesn't exist or update existing one. In the ResourceRecordSet, you should provide domain name, type, TTL (i.e. record cache time to live), and route traffic to. In this example, the new domain mydomain.starchart.com will be directed to A record, 192.168.0.1. You can find more options when you set routing policy other than this simple example. I found AWS CLI Command Reference page is very helpful to understand the parameters and refer to other ChangeBatch syntax.

List Records and View Change Status

When you fetch all your records or partial records, you can use ListResourceRecordSetsCommand.

Lastly GetChangeCommand method can check the status of specific ChangeBatch by providing ChangeInfo.Id from ChangeResourceRecordSetsCommand response as a parameter. GetChangeCommand returns ChangeInfo and ChangeInfo.Status will show either PENDING or INSYNC.

Conclusion

I used AWS S3, DynamoDB SDK before. So it didn't take much time to set up and configure. AWS SDK syntax is highly consistent and it's easy to use once you learn any one of these SDKs. As I mentioned above, AWS CLI reference page has more details explaining each parameter and response although you will rely on JavaScript SDK page more to write your code. Now I'm eager to write code for Starchart.

How to Write Dockerfile and Basic Docker Commands

Genne23v — Mon, 23 Jan 2023 20:54:31 +0000

Seneca OSD700 team project Starchart will use many Docker containers. I would like to summarize how to write Dockerfile and basic Docker commands.

First we need to write a Dockerfile which is a set of instruction to build a Docker image. A Docker image is a read-only template with instructions for creating a Docker container and the container is a runnable instance of an image. It's probably better to understand with below node project example.

FROM node:16.15.1
LABEL maintainer="Wonkeun No <wonkeun.no@gmail.com>"
ENV PORT=8080
WORKDIR /app
COPY package.json ./
RUN npm install
COPY ./src ./src
CMD npm start
EXPOSE 8080

Dockerfile must begin with FROM to specify what parent image the container uses. In this case, I use node version 16.15.1. So my app will be running on top of node image which someone built for everybody.

LABEL is not a mandatory command, but you can add some metadata information about the image as a key=value pair. You can add more like version, description, etc.

Environment variable can be set with ENV command. It can be overridden with docker command.

WORKDIR sets the working directory. Multiple WORKDIR is also possible. When a relative path is provided, it will be relative to previous WORKDIR.

Then, copy package.json and paste it to ./ directory with COPY instruction. And run npm install with RUN to download npm dependencies in this case.

Finally copy all ./src files and paste into container's ./src directory and run npm start with CMD with port 8080 open outside of container. You can run many RUN instructions, but CMD can be run once as a container launch command. You can still write multiple CMD, but all will be ignored by last CMD.

Once Dockerfile is written, it's highly recommended to ignore some of files that are not necessary in docker build by adding them to .dockerignore. This process helps to reduce the size of image and build faster. You secret information files should not be in the image either.

There is more information about Dockerfile in the official Docker site. Docker reference

Now it's ready to build the image with below command.

docker build -t <image>:latest .

-t can specify a tag for the image. This option can be combined with -i (i.e. shortened form of --interactive) to interact with container in a terminal. It's useful to see what is going on inside the container.

If the build is successful, you can find the image with below command.

docker image ls

Now it's time to actually run the container.

docker run --rm --name <image> --env-file .env -e LOG_LEVEL=debug -p 8080:8080 <image>:latest -d

Here's explanation for each option.

--rm removes the container when it exits automatically
--name assign a name to the container
--env-file specifies a file that defines environment variables
-e sets an environment variable for the container
-p binds the host machine's port with container's port (i.e. [HOST_MACHINE_PORT:CONTAINER_PORT]
-d runs the container in background and print container ID

Once you run above command, you will see the container's ID. Run below to see the logs from the container. -f option is to continuously print out the logs as it generates.

docker logs -f <container_id>

This article helps to create a Docker image and run a single container. I will write more about best practises of writing Dockerfile , DockerHub, and docker-compose.

DEV Community: Genne23v

Writing Playwright UI Test

How to set up Playwright

Writing Playwright test

Codegen Test Generator

Playwright Configuration

Conclusion

What is HTTPS?

How does HTTPS work?

How is data secured in HTTPS?

How does SSL certificate look like?

Who can provide SSL certificate?

Conclusion

What is DNS Record?

What is DNS Record?

Types of DNS Record

What will Starchart do?

More Terms in DNS

Conclusion

Managing Work Scope in Software Project

Minimize your work scope

Split large scope of work into different phases

Plan future improvement instead of fixing everything in one PR

Conclusion

Advanced Types in TypeScript

When using the same type with different property names

When type naming is not necessary

Conclusion

JavaScript & TypeScript ORM Prisma

Getting started

Writing schema and init PrismaClient

CRUD operation

Seeding and Prisma Studio

Conclusion

Create BullMQ Workflow

BullMQ setup

BullMQ Flows

Set up workers for each queue

Conclusion

Useful Regex For Web Developer

Limiting domain name input

Password requirement regex

Email validation

Conclusion

Useful JavaScript / TypeScript Techniques

Using import / export

Reducing parameters by passing object

Conclusion

Lesson Learned From Code Review

Working code is not enough, it should be easily readable for other developers

Don't let same logic duplicated on the code

Function should do only one thing whenever possible

Don't use unnecessary syntax and use linter to set a rule to maintain consistency

Conclusion

Create Domains Using Route 53 SDK

Setup Route 53 SDK

Creating Hosted Zone

Add/Update/Delete Records in Hosted Zone

List Records and View Change Status

Conclusion

How to Write Dockerfile and Basic Docker Commands

`BullMQ` setup

Don't use unnecessary syntax and use `linter` to set a rule to maintain consistency