DEV Community: Shinji NAKAMATSU

[Boost]

Shinji NAKAMATSU — Tue, 17 Dec 2024 04:43:00 +0000

Deploy Your Application to ECS via CI/CD with Ecspresso

Atsushi Miyamoto ・ May 1 '23

#ecs #deploy #cicd #githubactions

Implementing Secure Access Control using AWS WAF with IP Address and BASIC Authentication

Shinji NAKAMATSU — Sat, 21 Oct 2023 06:15:46 +0000

Photo by Damon Lam on Unsplash

Introduction

AWS WAF is a Firewall service that can be attached to CloudFront or ALB. By defining a Web ACL (Access Control List), you can block or allow access based on specific conditions.

It is common to want to restrict who or where can access your service while developing a new service to be exposed on the internet.

In this article, we share how to combine AWS WAF for allowing access based on specific IP addresses (CIDR) and BASIC authentication.

Requirements

Allow access if it matches the allowed IP addresses (CIDR)
Allow access if the request header contains correct BASIC authentication information
Block all other access

Overview of the Mechanism

Here's a summary of the mechanism:

Default to "allow access"
- Assuming that when opening the service, just removing the rule would remove the access restriction
Block access if the following (AND) conditions are met:
- Not an allowed IP address
- Lacks BASIC authentication information
When blocking access, request BASIC authentication with a custom response (WWW-Authenticate header is returned)

Terraform Code

Here's the code snippet:

// modules/waf/variables.tf
// Module parameters
variable "allowed_ip_list" {}
variable "basic_auth" {
  type = object({
    user     = string
    password = string
  })
}

// modules/waf/main.tf
terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      configuration_aliases = [aws.virginia]
    }
  }
}

locals {
  // Pre-calculate the expected value for the Authorization header
  credential = base64encode("${var.basic_auth.user}:${var.basic_auth.password}")
}

resource "aws_wafv2_web_acl" "main" {
  provider    = aws.virginia
  name        = "example-web-acl"
  description = "Web ACL example"
  scope       = "CLOUDFRONT"

  // Set the default action to allow
  default_action {
    allow {}
  }

  // BASIC Authentication
  rule {
    name     = "basic-auth"
    priority = 20

    action {
      block {
        // Set the action to block for the conditions mentioned later
        // Also, return a custom response requesting username and
        // password input for BASIC authentication
        custom_response {
          response_code = 401
          response_header {
            name  = "WWW-Authenticate"
            value = "Basic realm=\"Secure Area\""
          }
        }
      }
    }
    // Conditions for Block (AND) :
    statement {
      and_statement {
        statement {
          // Condition 1
          // The source IP address is not included in the allowed IP list
          not_statement {
            statement {
              ip_set_reference_statement {
                arn = aws_wafv2_ip_set.allowed_ips.arn
              }
            }
          }
        }
        statement {
          // Condition 2
          // The Authorization header does not contain
          not_statement {
            statement {
              byte_match_statement {
                positional_constraint = "EXACTLY"
                search_string         = "Basic ${local.credential}"
                field_to_match {
                  single_header {
                    name = "authorization"
                  }
                }
                text_transformation {
                  priority = 0
                  type     = "NONE"
                }
              }
            }
          }
        }
      }
    }
    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "example-basic-auth"
      sampled_requests_enabled   = true
    }
  }

  visibility_config {
    cloudwatch_metrics_enabled = true
    metric_name                = "example-web-acl"
    sampled_requests_enabled   = true
  }
}

// Set of allowed IP addresses for access
resource "aws_wafv2_ip_set" "allowed_ips" {
  provider           = aws.virginia
  name               = "example-allowed-ips"
  description        = "Authorized IP addresses"
  scope              = "CLOUDFRONT"
  ip_address_version = "IPV4"
  addresses          = var.allowed_ip_list
}

IP Address-based Access Control Settings

To perform access control based on IP addresses in AWS WAF, you first need to prepare an IP set.

// Set of allowed IP addresses
resource "aws_wafv2_ip_set" "allowed_ips" {
  provider           = aws.virginia
  name               = "example-allowed-ips"
  description        = "Authorized IP addresses"
  scope              = "CLOUDFRONT"
  ip_address_version = "IPV4"
  addresses          = var.allowed_ip_list
}

In the above, the list of IP addresses is stored in var.allowed_ip_list, and it's assumed that the module user will pass it as a parameter in the following manner:

module "waf" {
  src = "../modules/waf"
  allowed_ip_list = [
    "xxx.xxx.xxx.xxx/32",
    "yyy.yyy.yyy.yyy/32",
    "zzz.zzz.zzz.zzz/32"
  ]
  ... 
}

BASIC Authentication Settings

In the code below, we pre-calculate the string expected to be stored in the BASIC authentication header.

locals {
  // Pre-calculate the expected value for the Authorization header
  credential = base64encode("${var.basic_auth.user}:${var.basic_auth.password}")
}

In the following block, we reference that value to check whether the corresponding string is stored in the request's Authorization header.

              byte_match_statement {
                positional_constraint = "EXACTLY"
                search_string         = "Basic ${local.credential}"
                field_to_match {
                  single_header {
                    name = "authorization"
                  }
                }
                text_transformation {
                  priority = 0
                  type     = "NONE"
                }
              }

If the conditions do not match, the block below is set to return a custom response.

    action {
      block {
        // Set the action to block for the conditions mentioned later
        // Also, return a custom response requesting username and
        // password input for BASIC authentication
        custom_response {
          response_code = 401
          response_header {
            name  = "WWW-Authenticate"
            value = "Basic realm=\"Secure Area\""
          }
        }
      }
    }

In the custom response, the WWW-Authenticate response header requests BASIC authentication from the source. Typically, when a browser receives this response, it displays a dialog prompting for BASIC authentication information (User/Password).

Reference: WWW-Authenticate - HTTP | MDN

Combining IP Address and BASIC Authentication

Combining the conditions mentioned earlier results in the following rule. (Details are omitted for clarity in structure)

rule {
  // If the following statement matches, block the access and return a custom response
  action {
    block {
      // Request BASIC authentication in the custom response
    }
  }
  statement {
    and_statement {
      // The source IP address is not in the allowed IP list
      statement {
        // Invert the condition here
        not_statement {
      statement {
        // Does the IP address match the allowed IP list?
      }
    }
      }
      // BASIC authentication information is not included in the Authorization header
      statement {
        // Invert the condition here
        not_statement {
      statement {
        // Is BASIC authentication information included in the Authorization header?
      }
    }
      }
    }
  }
}

Note that, as the negation (not_statement) cannot be described directly under the rule block or and_statement, it needs to be re-wrapped with the statement block, making the structure somewhat difficult to understand.

In summary, this results in the code mentioned earlier.

Conclusion

We introduced a method to allow access to specific users using AWS WAF by combining IP address-based strict access control and looser access control through BASIC authentication, which can flexibly accommodate cases where temporary access is needed.

One point to note is that if you apply this to APIs that require authentication methods other than BASIC authentication (e.g., OAuth), some modification on the client-side using the API may be needed, bringing infrastructural concerns into the application layer. It's also viable to exclude such APIs from this rule if they already have authentication in place, depending on the risk assessment.

I hope this article helps someone out there.

References

Degraded Performance Caused by Next.js Webfont Optimization: A Case Study

Shinji NAKAMATSU — Thu, 05 Oct 2023 15:09:24 +0000

Photo by Charlie M on Unsplash

TL;DR

A website built with Next.js experienced performance issues under certain conditions.
The cause of the problem was a significant expansion of HTML size due to the Automatic Webfont Optimization feature.
As a solution, we utilized next/font to optimize the method of loading Webfonts and reduced the HTML size.

Specific Circumstances of the Project

Deploying and test-running the Next.js application on ECS
Providing multilingual support (Japanese, Korean, Chinese (Traditional, Simplified))
Originally started building with Next.js v12, and just last month, finally completed the update to v13

The Problem That Occurred

Our team had an opportunity to measure the performance of a particular website built with Next.js. Surprisingly, even though the number of requests was not so high, we noticed IPC (Inter-Process Communication) related errors occurring on the server side.

(Part of the occurring error)

TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11457:11)
    at async invokeRequest (/home/snaka/xxxxx/node_modules/next/dist/server/lib/server-ipc/invoke-request.js:17:12)
    at async invokeRender (/home/snaka/xxxxx/node_modules/next/dist/server/lib/router-server.js:254:29)
    at async handleRequest (/home/snaka/xxxxx/node_modules/next/dist/server/lib/router-server.js:447:24)
    at async requestHandler (/home/snaka/xxxxx/node_modules/next/dist/server/lib/router-server.js:464:13)
    at async Server.<anonymous> (/home/snaka/xxxxx/node_modules/next/dist/server/lib/start-server.js:117:13) {
  cause: Error: read ECONNRESET
      at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
    errno: -104,
    code: 'ECONNRESET',
    syscall: 'read'
  }
}

Another error also occurred:

cause: SocketError: other side closed
at Socket.onSocketEnd (/app/node_modules/next/dist/compiled/undici/index.js:1:63301)
at Socket.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1359:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'UND_ERR_SOCKET',
  socket: {
    localAddress: '127.0.0.1',
    localPort: 45850,
    remoteAddress: undefined,
    remotePort: undefined,
    remoteFamily: undefined,
    timeout: undefined,
    bytesWritten: 2937,
    bytesRead: 87753
  }
}

Upon further investigation, it became clear that the HTML built by Next.js was abnormally large. Specifically, its size was about 2MB, which was significantly larger compared to the size of a typical webpage's HTML.

The Cause of the Problem

To resolve the performance issue occurring on our website, we began a series of investigations to pinpoint the cause. During load testing, it was revealed that network bandwidth was a clear bottleneck, and we were particularly concerned about the significant bandwidth consumption when downloading a single HTML file.

Further investigation into the HTML revealed that it contained a large number of font definitions (@font-face rules). These definitions were not present in the original source code and were automatically inserted by some mechanism. Further research revealed that these font definitions originated from the Automatic Webfont Optimization feature introduced in Next.js v10.2.

Reference: Next.js 10.2 Release Notes

The problematic section was the following description in _document.tsx.

// _document.tsx
<Head>
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Noto+Sans+JP:wght@100;300;400;500;700;900&display=swap" />
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Noto+Sans+SC:wght@100;300;400;500;700;900&display=swap" />
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Noto+Sans+TC:wght@100;300;400;500;700;900&display=swap" />
  <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Noto+Sans+KR:wght@100;300;400;500;700;900&display=swap" />
</Head>

Sample code is available here:

https://github.com/snaka/code-pocket/tree/master/framework/nextjs/webfont/legacy-optimization

Size of the built HTML (about 2MB)

-rw-r--r--@ 1 snaka  staff  2359534 Sep 29 09:48 ja.html
-rw-r--r--@ 1 snaka  staff  2359533 Sep 29 09:48 ko.html
-rw-r--r--@ 1 snaka  staff  2359540 Sep 29 09:48 zh-CN.html
-rw-r--r--@ 1 snaka  staff  2359540 Sep 29 09:48 zh-TW.html

Solution

The solution was simple. We stopped loading Webfonts in _document.tsx and switched to using next/font, introduced in Next.js v13.

Optimizing: Fonts | Next.js

This change significantly reduced the size of the HTML and resolved the IPC error when loading the server. Additionally, by using next/font, it became possible to self-host Webfonts and control caching periods, etc., through the CDN.

The code for using Webfonts became as follows:

// index.tsx
import { Noto_Sans_JP, Noto_Sans_KR, Noto_Sans_SC, Noto_Sans_TC } from 'next/font/google'

const notoSansJP = Noto_Sans_JP({
  weight: ['100', '300', '400', '500', '700', '900'],
  preload: false
})
const notoSansSC = Noto_Sans_SC({
  weight: ['100', '300', '400', '500', '700', '900'],
  preload: false
})
const notoSansTC = Noto_Sans_TC({
  weight: ['100', '300', '400', '500', '700', '900'],
  preload: false
})
const notoSansKR = Noto_Sans_KR({
  weight: ['100', '300', '400', '500', '700', '900'],
  preload: false
})
// Use the loaded webfont as className={notoSans_JP.className}, etc.

Sample code is available here:

https://github.com/snaka/code-pocket/tree/master/framework/nextjs/webfont/next-font

Size of the built HTML (about 5KB)

-rw-r--r--@ 1 snaka  staff   5176 Sep 29 14:19 ja.html
-rw-r--r--@ 1 snaka  staff   5175 Sep 29 14:19 ko.html
-rw-r--r--@ 1 snaka  staff   5182 Sep 29 14:19 zh-CN.html
-rw-r--r--@ 1 snaka  staff   5182 Sep 29 14:19 zh-TW.html

In Choosing the Approach

In this case, we adopted a method of changing the Webfont loading method and using next/font without preload. This decision was made as a result of evaluating the following technical trade-offs.

Problems Before Improvement
- Increased communication caused instability in inter-process communication within the container
- The allocated CPU and memory performance for the container could not be fully utilized, necessitating the parallel operation of numerous containers
Negative Impact from Changing to next/font
- Using next/font without preload → A slight time lag occurred until the Webfont was applied
Consideration of Alternatives
- Change the deployment destination to a FaaS platform? → Judged as high-risk
- Personally, it was very interesting but...
Final Decision
- Utilizing next/font provided benefits of resolving communication bottlenecks and effectively utilizing container resources
- Compared cost-performance and a slight UX degradation, and prioritized cost-performance
- By loading divided files, we anticipate achieving service stability efficiently by distributing server load using CDNs.

Through this choice, while the client-side overhead increased due to the additional files to be loaded, the final performance evaluation (Lighthouse) showed a slight improvement. (Approximately 13% enhancement)

In Conclusion

In this article, we presented an example of a performance issue and its solution that occurred on a website built with Next.js. We confirmed that the Automatic Webfont Optimization feature could cause a significant expansion in HTML size in certain cases, which, in turn, could potentially trigger IPC errors on the server side.

As a solution, by utilizing next/font, we optimized the method of loading Webfonts, reduced the HTML size, and were able to alleviate the bottleneck on the server. This also enabled us to self-host Webfonts and control caching.

The speed of evolution in front-end technology is staggering, with new technologies and optimization methods being provided daily. While they can enhance the performance of a website when used appropriately, they are not omnipotent and can cause unexpected problems in some cases. Therefore, thorough testing is indispensable when introducing new features.

We hope this article proves helpful to someone.

This article was proofread by ChatGPT.

Pages Referred to

Avoiding SSR Pitfalls: Using `next/dynamic` in Your Next.js App

Shinji NAKAMATSU — Wed, 13 Sep 2023 13:46:26 +0000

tl;dr

Components wrapped in dynamic() as shown below are not rendered on the server-side but are exclusively rendered on the client-side.

export const PurchaseButton = dynamic(
  Promise.resolve((props: Props) => {
    // Component implementation
  }),
  { ssr: false }
)

The Problem

When components intended for server-side rendering (SSR) are implemented as demonstrated below, discrepancies can arise between Server-side and Client-side rendering, potentially leading to errors during React's hydration phase:

(error message)

Unhandled Runtime Error
Error: Text content does not match server-rendered HTML.

See more info here: https://nextjs.org/docs/messages/react-hydration-error

Followings are possible causes:

Adapting rendering based on the presence or absence of the window object.
Content changes based on rendering timing, such as the current date and time.

For more in-depth information, please refer to the following link:

Text content does not match server-rendered HTML | Next.js

How to Address this from the Component Consumer Side

To resolve such issues, one approach is to prevent the targeted component from being rendered on the Server-side.

Solution 2: Disabling SSR on specific components

To achieve this, you can leverage next/dynamic. However, in the official documentation, it is often introduced in the context of dynamic imports, possibly with file splitting in mind.

Reference: Optimizing: Lazy Loading | Next.js

const DynamicComponent = dynamic(() =>
  import('../components/hello').then((mod) => mod.Hello)
)

By following this pattern, the side that utilizes the component (component consumer side) needs to be aware of next/dynamic.
If you need the flexibility to toggle between enabling and disabling SSR on a case-by-case basis, that's perfectly fine. However, if the specific component is intended to be non-SSR compliant regardless of its usage context, it's best to handle this within the component itself.

Addressing the Issue Within the Targeted Component

By wrapping the targeted component entirely with dynamic(...), there's no need for the component consumer to be mindful of SSR considerations. It's worth noting that the first argument of dynamic() should be a Promise<...>, which typically involves passing the component wrapped in Promise.resolve().

export const PurchaseButton = dynamic(
  Promise.resolve((props: Props) => {
    // Component implementation
  }),
  { ssr: false }
)

Conclusion

Server-side rendering (SSR) offers advantages like faster page loads and SEO benefits. Nevertheless, certain components may introduce issues due to disparities between server-side and client-side behaviors.

Through the use of Next.js's next/dynamic feature, developers can selectively disable SSR for specific components, ensuring smoother hydration processes and overall enhanced application resilience.

This approach not only provides flexibility in managing components with client-specific dependencies but also streamlines the development process by reducing SSR-related bugs.

How to Reference Terraform Cloud State from ecspresso and ecschedule

Shinji NAKAMATSU — Tue, 11 Jul 2023 23:50:38 +0000

Photo by Ian Taylor on Unsplash

Introduction: ecspresso, ecschedule, and Terraform

ecspresso, ecschedule, and Terraform are powerful tools for achieving Infrastructure as Code (IaC) and managing resources in AWS. Let's take a closer look at each tool.

ecspresso

ecspresso is a tool designed for managing the definition of ECS services and tasks as code. It enables you to define and manage your ECS infrastructure using familiar coding practices.

You can find more information about ecspresso on the official GitHub repository: ecspresso GitHub.

kayac / ecspresso

ecspresso is a deployment tool for Amazon ECS

ecspresso

ecspresso is a deployment tool for Amazon ECS.

(pronounced the same as "espresso" ☕)

Documents

Differences between v1 and v2.
ecspresso Advent Calendar 2020 (Japanese)
ecspresso handbook (Japanese)
Command Reference (Japanese)

Install

Homebrew (macOS and Linux)

$ brew install kayac/tap/ecspresso

asdf (macOS and Linux)

$ asdf plugin add ecspresso
# or
$ asdf plugin add ecspresso https://github.com/kayac/asdf-ecspresso.git

$ asdf install ecspresso 2.3.0
$ asdf global ecspresso 2.3.0

aqua (macOS and Linux)

aqua is a CLI version manager.

$ aqua g -i kayac/ecspresso

Binary packages

Releases

CircleCI Orbs

https://circleci.com/orbs/registry/orb/fujiwara/ecspresso

version: 2.1
orbs:
  ecspresso: fujiwara/ecspresso@2.0.4
jobs:
  install:
    steps:
      - checkout
      - ecspresso/install:
          version: v2.3.0 # or latest
          # version-file: .ecspresso-version
          os: linux # or windows or darwin
          arch: amd64 # or arm64
      - run:
          command: |
            ecspresso version

version: latest installs different versions of ecspresso for each Orb…

View on GitHub

ecschedule

ecschedule is another valuable tool in the IaC ecosystem, specifically built for managing scheduled tasks on ECS. With ecschedule, you can define and manage tasks that need to be executed on a schedule, similar to cron jobs.

To learn more about ecschedule, visit the GitHub repository: ecschedule GitHub.

Songmu / ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

Synopsis

% ecschedule [dump|apply|run|diff] -conf ecschedule.yaml -rule $ruleName

Description

The ecschedule manages ECS Schedule tasks using a configuration file (YAML, JSON or Jsonnet format) like following.

region: us-east-1
cluster: clusterName
rules:
trackingId: trackingId1
- name: taskName1
  description: task 1
  scheduleExpression: cron(30 15 ? * * *)
  taskDefinition: taskDefName
  containerOverrides:
  - name: containerName
    command: [subcommand1, arg]
    environment:
      HOGE: foo
      FUGA: {{ must_env `APP_FUGA` }}
- name: taskName2
  description: task2
  scheduleExpression: cron(30 16 ? * * *)
  taskDefinition: taskDefName2
  containerOverrides:
  - name: containerName2
    command: [subcommand2, arg]

Installation

% brew install Songmu/tap/ecschedule
# or
% go install github.com/Songmu/ecschedule/cmd/ecschedule@latest

GitHub Actions

Action Songmu/ecschedule@main installs ecschedule binary for Linux into /usr/local/bin. This action runs install only.

jobs

…

View on GitHub

Terraform

Terraform is a widely adopted IaC tool that allows you to define and manage infrastructure resources in AWS, such as VPCs, RDS instances, and ElastiCache clusters. It provides a declarative approach to infrastructure management, enabling you to define your desired state and automatically provision and manage resources accordingly.

The Issue of Managing ECS Task Definitions with Terraform

ECS task definitions often include elements that are tightly coupled with the application's implementation, such as environment variable settings and command parameter configurations. When managing task definitions within Terraform, applying changes in Terraform and then reflecting them on the application side becomes a necessary step.

While this approach works well for occasional changes, frequent updates can introduce the potential for mistakes and inefficiencies.

To address this issue, it is recommended to decouple ECS services and task definitions from Terraform. By leveraging ecspresso and ecschedule, I can manage the application code and related ECS components together. This approach simplifies the management process and streamlines development workflows.

Leveraging Terraform Cloud State with ecspresso and ecschedule

One of the powerful features offered by ecspresso and ecschedule is the tfstate plugin. This plugin allows you to reference the configuration values of infrastructure resources managed by Terraform using the Mustache syntax.

Using the tfstate plugin, you can incorporate the following example of referencing values using the Mustache syntax:

{{ tfstate `path.to.resource` }}

This syntax enables you to access specific resource configurations managed by Terraform and utilize them within your ecspresso and ecschedule configurations seamlessly.

While the README documentation provides examples of referencing local paths and S3 bucket URLs, it does not explicitly cover referencing state managed by Terraform Cloud. However, by exploring the ecspresso source code, I discovered the usage of tfstate-lookup internally.

To reference Terraform Cloud state, I can utilize the following URL format: remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}. This format allows me to access the state of a specific workspace within my Terraform Cloud organization.

To make this work, I need to generate an API token for Terraform Cloud and set it in the TFE_TOKEN environment variable. The API token should have appropriate permissions based on my operational requirements.

Once the API token is set, I can configure ecspresso and ecschedule to reference the Terraform Cloud state. For example, in the config.yaml file, I can add the following configuration:

plugins:
  - name: tfstate
    config:
      url: remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}

With this configuration in place, I can now read and reference the configuration values of my Terraform-managed resources within ecspresso and ecschedule.

Before deploying changes, it is recommended to run commands such as ecspresso render or ecschedule diff to confirm that the configuration reflects the values from the Terraform Cloud state.

Conclusion

It is easier to manage ECS services, task definitions, and scheduled tasks as Infrastructure as Code (IaC) using ecspresso and ecschedule, rather than Terraform.
Set the Terraform Cloud API Token in the TFE_TOKEN environment variable.
The URL format to reference Terraform Cloud state is remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}.

The tfstate plugin provided by ecspresso and ecschedule allows me to reference and leverage the configuration values of my Terraform-managed resources. By utilizing this feature, along with proper configuration and the API token for Terraform Cloud, I can seamlessly integrate Terraform Cloud into my ecspresso and ecschedule workflows.

I hope this article provides valuable insights and guides you towards optimizing your infrastructure management practices.

Enhancing MUI Autocomplete and Formik Integration: Insights and Practical Tips

Shinji NAKAMATSU — Mon, 26 Jun 2023 03:58:22 +0000

Photo by Ilya Pavlov on Unsplash

I wanted to do something like this for work, but I couldn't find a good sample on the internet, so I had to experiment and figure it out. So, I'm publishing it as a blog post.

Demo

First, let me show you the demo.

Assuming a form where users enter a postal code in Tokyo, I have implemented an Autocomplete component that displays and allows the selection of address candidates based on keywords. (I have extracted only a few postal codes as including all of them would result in too many options)

When you press the Submit button, the contents of the form are outputted in JSON format.

Code

Here is the code:

import { SyntheticEvent, useMemo, useCallback, useState } from "react";
import "./styles.css";
import { useFormik } from "formik";
import { Autocomplete, TextField, debounce } from "@mui/material";
import { postalCodes, usePostalCode } from "./data";

type FormValues = {
  postalCode: string | null;
};

export default function App() {
  const [filterKeyword, setFilterKeyword] = useState<string>("");
  const [submitCode, setSubmitCode] = useState<string>("");

  // Custom Hook to extract the list of candidates
  const { data: filteredPostalCodes } = usePostalCode(filterKeyword);

  const formik = useFormik<FormValues>({
    initialValues: {
      postalCode: null
    },
    onSubmit: (values) => {
      console.log(JSON.stringify(values));
      setSubmitCode(JSON.stringify(values));
    }
  });

  const debouncedSetter = useMemo(
    () => debounce((keyword: string) => setFilterKeyword(keyword), 500),
    []
  );

  return (
    <div className="App">
      <h1>Formik + MUI Autocomplete Demo</h1>
      <form onSubmit={formik.handleSubmit}>
        <label htmlFor="postalCode">Postal Code: </label>
        <Autocomplete
          value={formik.values.postalCode}
          onChange={(_: SyntheticEvent, newValue: string | null) =>
            formik.setFieldValue("postalCode", newValue)
          }
          options={filteredPostalCodes?.map((p) => p.postalCode) ?? []}
          onInputChange={(_, newInputValue) => debouncedSetter(newInputValue)}
          // filterOptions={(x) => x} // Disable the default filtering as we handle it ourselves
          getOptionLabel={(option: string) => {
            return (
              postalCodes.find((p) => p.postalCode === option)?.address ?? ""
            );
          }}
          renderInput={(params) => (
            <TextField {...params} placeholder="Postal Code" id="" />
          )}
        />
        <button type="submit">Submit</button>
      </form>
      <div>{submitCode && <>Submit code: {submitCode}</>}</div>
    </div>
  );
}

Implementation Points

There are several implementation points that I want to explain briefly.

Match the type of Autocomplete's `value` with the field type in Formik

When using Autocomplete to input values into a Formik form, it is easier to handle if the types of each value match.

In the sample code, I created a form that allows users to input values into the postalCode field in Formik using Autocomplete. In this case, both the type of values.postalCode in Formik and the type of value in Autocomplete are implemented as string.

type FormValues = {
  postalCode: string | null;
};

// ... (snip) ...

<Autocomplete
  loading={isPostalCodeLoading}
  value={formik.values.postalCode} 
  onChange={(_: SyntheticEvent, newValue: string | null) =>
    formik.setFieldValue("postalCode", newValue)
  }

Pass an array of values to Autocomplete's `options`

Intuitively, we might assume that options should contain the strings displayed in the Autocomplete's dropdown list. However, it's a bit different.

In options, you should pass an array of values that will be used as the selected value assigned to value.

In the sample code, we assume that the postalCode value will be assigned to value, so we set an array consisting of postalCode as the elements.

options={filteredPostalCodes?.map((p) => p.postalCode) ?? []}

Use `getOptionLabel` to display values in a different format than what's in `options`

As mentioned earlier, the values set in options may not be user-friendly or easily understandable, so it's necessary to convert them into a more readable (and selectable) format.

In the sample code, we pass an array of postalCode to options, but we decided to display the corresponding address in the UI.

The implementation of the function takes an argument option, which is an element (i.e., postalCode) of options. We then find the corresponding element in the postalCodes array and return its address.

getOptionLabel={(option: string) => {
  return (
    postalCodes.find((p) => p.postalCode === option)?.address ?? ""
  );
}}

Use `null` to represent the "unselected" state in Autocomplete

The value that represents the "unselected" state in Autocomplete (i.e., value) is null. Therefore, in the initialValues of Formik, we set null as the initial value.

initialValues: {
  postalCode: null
},

If you mistakenly set "" (an empty string) instead, you will see a warning like this:

MUI: The value provided to Autocomplete is invalid.
None of the options match with `""`.
You can use the `isOptionEqualToValue` prop to customize the equality test.

MUI provides debounce functionality

While many articles suggest using lodash or implementing debounce manually, MUI actually provides debounce functionality, so you don't need to add a dependency on lodash or implement it yourself.

import { Autocomplete, TextField, debounce } from "@mui/material";

Memoize functions with debounce using `useMemo` or `useCallback`

It's easy to overlook, but functions wrapped with debounce need to be memoized to work correctly.

const debouncedSetter = useMemo(
  () => debounce((keyword: string) => setFilterKeyword(keyword), 500),
  []
);

Typically, you would use the useCallback hook to memoize a function. However, ESLint raises a warning saying "React Hook useCallback received a function whose dependencies are unknown. Pass an inline function instead. (react-hooks/exhaustive-deps)." Therefore, we use useMemo instead.

Use `inputValue` as the keyword for API search

In the sample code, the custom hook usePostalCode is designed to make an API call and retrieve a list of postal codes and their corresponding addresses based on the keyword provided.

const [filterKeyword, setFilterKeyword] = useState<string>("");

// Custom Hook to extract the list of candidates
const { data: filteredPostalCodes } = usePostalCode(filterKeyword);

// ... (snip) ...

onInputChange={(_, newInputValue) => debouncedSetter(newInputValue)}

Conclusion

With the combination of Formik and Autocomplete, we were able to create a form with autocomplete functionality.

To summarize the key points:

Match the type of Autocomplete's value with the field type in Formik.
Pass an array of values to Autocomplete's options.
Use getOptionLabel to display values in a different format than what's in options.
Use null to represent the "unselected" state in Autocomplete.
MUI provides debounce functionality, eliminating the need for lodash dependency.
Memoize functions with debounce using useMemo or useCallback.
Use the inputValue as the keyword for API search.

I hope this article will be helpful to someone.

Reference Links

https://formik.org/docs/api/formik

https://mui.com/material-ui/api/autocomplete/

Securing Your Next.js Application with Strict CSP

Shinji NAKAMATSU — Sun, 04 Jun 2023 07:43:49 +0000

( Photo by FLY:D on Unsplash )

Utilizing CSP (Content Security Policy) allows us to limit the execution of code on our website, thereby mitigating the scope of any potential damage caused by XSS (Cross-Site Scripting) attacks.

In this article, I'll share an example of how to handle CSP headers in Next.js.

tl;dr;

Set the policy according to Google's advocated Strict CSP
Use _document.tsx to embed the CSP header into the head as <meta httpEquiv="Content-Security-Policy" content={ ... Policy here ... } />

Prerequisites

Next.js 12.x
Both SSR (Server Side Rendering) and CSR (Client Side Rendering) are used as rendering methods
Loading external scripts using the Script component

What is Strict CSP?

Google's proposed Strict CSP is a setting for CSP aimed at realizing higher security.

Strict CSP depends on the application use-case, hence, careful consideration is required to decide how to use it depending on your application's specific use-cases.

Introduction - Content Security Policy

What is the strict-dynamic directive?

Strict CSP uses the strict-dynamic directive to give trust to scripts. strict-dynamic decides whether to trust a script based on a nonce value or hash value generated from the script. Then, trust is automatically propagated to scripts that are dynamically loaded from that trusted script.

By using the strict-dynamic directive, the tedious task of managing whitelist becomes unnecessary, as you only need to grant trust to the top-level script.

Implementing Strict CSP in Next.js

The implementation strategy was decided as follows:

Primarily adhere to Strict CSP
Trust scripts via nonce
- This is due to the complexity of calculating the hash values of multiple external scripts
Set the CSP header in the form of a meta tag
- Although it is possible to return response headers via next.config.js, the values cannot be dynamically changed
- I adopted the method of generating a meta tag in _document.tsx to dynamically generate nonce values

Implementation Example

Below is an example of the implementation.

import { randomBytes } from 'crypto'

import { Head, Html, Main, NextScript } from 'next/document'

const Document = () => {
  const nonce = randomBytes(128).toString('base64')
  const csp = `object-src 'none'; base-uri 'none'; script-src 'self' 'unsafe-eval' 'unsafe-inline' https: http: 'nonce-${nonce}' 'strict-dynamic'`

  return (
    <Html lang="ja">
      <Head nonce={nonce}>
        <meta httpEquiv="Content-Security-Policy" content={csp} />
      </Head>
      <body>
        <Main />
        <NextScript nonce={nonce} />
      </body>
    </Html>
  )
}

export default Document

By using _document.tsx, you can embed the CSP header in the form of <meta httpEquiv="Content-Security-Policy" content={ ... } />.
- Although you can set the response header in next.config.js, since the nonce needs to be generated for each page request, the CSP header is returned as a meta tag.
- Specify the following directives in accordance with Strict CSP
  - object-src 'none'
  - base-uri 'none'
  - script-src 'self' 'unsafe-eval' 'unsafe-inline' https: http: 'nonce-${nonce}' 'strict-dynamic'
    - The value of ${nonce} is a randomly generated dynamic value
Pass the generated nonce value to the nonce property of the Head, NextScript components imported from next/document.

This is a brief introduction to handling CSP headers in Next.js.

Let's check it out

Looking at the HTML generated by the running application, you'll find that the same nonce value set in the CSP header is set in each script tag.

By the way, if you forget to set the nonce in the Head component, you can confirm that the execution of the script stops and the following error message is displayed:

Refused to load the script '<URL>' because it violates the following Content Security Policy directive: "script-src 'self' 'unsafe-eval' 'unsafe-inline' https: http: 'nonce-JyWbm9+... (snip) ...wURE=' 'strict-dynamic'". Note that 'strict-dynamic' is present, so host-based allowlisting is disabled. Note that 'script-src-elem' was not explicitly set, so 'script-src' is used as a fallback.

In conclusion

That's all about how to handle CSP headers in Next.js.

When I was researching how to handle CSP headers in Next.js, I noticed there wasn't much compiled information on the topic, so I decided to compile it myself.

I hope this article is useful to someone.

References

Use a different git config for each directory using conditional include

Shinji NAKAMATSU — Wed, 31 Aug 2022 00:17:22 +0000

It is possible for git to use different config references depending on specific conditions.

This was made aware to me by a question in an earlier post. (Thanks to @francoislp asked this question).

For example, if you have a working directory cloned under the ~/work/ directory and want to use the following configuration:

Add a conditional Include to ~/.gitconfig like this:

[includeIf "gitdir:~/work/**"]
        path = ~/work/.gitconfig

Then put the following in ~/work/.gitconfig

[user]
        name = Foo
        email = foo@example.com

Then, for commits in the working directory under ~/work, the commit will be logged as Foo <foo@example.com>.

As follows:

10+ things I always setup in git when I prepare a new environment

Shinji NAKAMATSU — Fri, 29 Jul 2022 02:02:48 +0000

When you buy a car or a bicycle, you first adjust the seat position and saddle height to suit your body size.　It is the same with git configuration.

In this article, I will share the git setup I use all the time.

User name & Mail address

git config --global user.name "<name>" && \
git config --global user.email "<email>"

Replace <name> with my name and <email> with my mail address.

Alias for existing command

git config --global alias.co checkout
git config --global alias.st status
git config --global alias.br branch

Global ignore setting

git config --global core.excludesfile ~/.gitignore_global

This allows you to put your own specific ignore settings in ~/.gitignore_global in addition to the .gitignore for each project, which will be applied to all git operations across all projects.

Push only the branch you are now working on

git config --global push.default simple

Make `--rebase` the default behavior during git pull

git config --global pull.rebase true

This prevents unintentional creation of a merge commit if the branch being pulled has been modified locally.

Make `--prune` the default behavior during git fetch

git config --global fetch.prune true

This will remove local branches that were deleted remotely when git fetch or git pull was performed.

Set the width of indentation for tab characters

git config --global core.pager 'less -x4'

In this example, the pager (less command) option specifies tab indent width as 4

Use `nvim` as the editor to be used when committing

git config --global core.editor 'nvim'

I use several different text editors for different purposes, but I prefer to use nvim with git commit.

Do not fast-forward when merging

git config --global --add merge.ff false
git config --global --add pull.ff only

Fast-forward merging makes it difficult to follow the history of work on a branch. Therefore, avoid unintentional fast-forwards when merging.
However, fast-forwarding is not a problem for git pull cases in most cases ¹, so we enforce fast-forwarding in the case of pulls.
See also: gitのmerge --no-ff のススメ - Qiita

Output line numbers in the result of `git grep`

git config --global grep.lineNumber true

Visualize differences in whitespace (including newline codes)

git config diff.wsErrorHighlight all

EDIT: 2022-07-31

dotfile

As mentioned in the comments, these are stored in .gitconfig. And I have added these settings to the dotfiles repository.

https://github.com/snaka/my-dotfiles/blob/master/.gitconfig

Thanks to all who commented.

Set the Sidekiq log level to the Rails log level

Shinji NAKAMATSU — Tue, 26 Jul 2022 14:36:07 +0000

Sidekiq's default Log Level is INFO, and its settings are indepent from Rails.

Sidekiq uses the standard Ruby Logger class for logging.

To change the Log Level in the Logger class, give a constant value for Severity using the #level= method.

Set the Sidekiq Log Level for the Logger used by Sidekiq in config/initializers/sidekiq.rb as follows:

Sidekiq.configure_server do |config|
  config.logger.level = Rails.logger.level
end

Alias to remove deleted branches in a git remote at once

Shinji NAKAMATSU — Sat, 16 Jul 2022 01:17:22 +0000

When working with git, local branches that have already been used are accumulated. Then I want to delete them all at once, so I defined an alias.

git config --global alias.prune-no-longer-exist '!git branch -vv | grep '"'"': gone]'"'"' | grep -v '"'"'\*'"'"' | awk '"'"'{ print $1; }'"'"' | xargs -r git branch -d'

The alias prune-no-longer-exist will be available after executing the above command.

It can be used as follows

 $ git prune-no-longer-exist                                                                                                                                               
 Deleted branch xxxxx-account (was 9907023).
 Deleted branch xxxxx-account-accomplish (was 4b1bc41).
 Deleted branch xxxxx-to-gitignore (was 4154555).
 Deleted branch dependabot/bundler/puma-5.6.4 (was b8cf408).

Unnecessary branches are removed from the local machine to refresh it.

I referred to the following.

Know the difference between `rails new` with flags

Shinji NAKAMATSU — Sat, 28 May 2022 06:16:23 +0000

Whenever I do rails new, I always have to check the effect of --skip-xxx, so I created a repository.

What I made

https://github.com/snaka/rails_new_flags

In this repository, the code output from running rails new with the Rails version and flag combination can be referenced by git tags.

The name of the tag is in the form {Rails version}-{flag}.

For example, Rails 7.0.1 code output with --minimal can be referenced by the tag 7.0.1-minimal.

Currently, the supported Rails versions are as follows

6.0.x
6.1.x
7.0.x

Usage

It may be obvious when you see it, but just in case, here is how to use it.

For example, in Rails 7.0.1, there is a flag called --api and you want to find out how specifying it changes the generated code.

Table in README.markdown(Following is an excerpt) to the --api flag line, click on the 🔍 at the 7.0.1 position to see the diff on GitHub.

flag	6.0.4.4	6.1.4.4	7.0.1
--api	🔍	🔍	🔍

(Diff. display as of 2022-01-08)

From the above, we can see that specifying --api makes the following difference in the output results.

Gemfile に sprockets-rails, importmap-rails, turbo-rails, stimulus-rails, jbuilder, ... (Omitted) ... is removed
Parent class of ApplicationController changes from ActionController::Base to ActionController::API.
Middleware configuration changes to the minimum required by adding config.api_only = true setting.
And so on...

You can also compare arbitrary Tags by writing your own URLs. For more information on how to compare tags on GitHub, please refer to the following document.

Comparing commits - GitHub Docs

DEV Community: Shinji NAKAMATSU

[Boost]

Deploy Your Application to ECS via CI/CD with Ecspresso

Atsushi Miyamoto ・ May 1 '23

Implementing Secure Access Control using AWS WAF with IP Address and BASIC Authentication

Introduction

Requirements

Overview of the Mechanism

Terraform Code

IP Address-based Access Control Settings

BASIC Authentication Settings

Combining IP Address and BASIC Authentication

Conclusion

References

Degraded Performance Caused by Next.js Webfont Optimization: A Case Study

TL;DR

Specific Circumstances of the Project

The Problem That Occurred

The Cause of the Problem

Solution

In Choosing the Approach

In Conclusion

Pages Referred to

Avoiding SSR Pitfalls: Using `next/dynamic` in Your Next.js App

tl;dr

The Problem

How to Address this from the Component Consumer Side

Addressing the Issue Within the Targeted Component

Conclusion

How to Reference Terraform Cloud State from ecspresso and ecschedule

Introduction: ecspresso, ecschedule, and Terraform

ecspresso

kayac / ecspresso

ecspresso is a deployment tool for Amazon ECS

ecspresso

Documents

Install

Homebrew (macOS and Linux)

asdf (macOS and Linux)

aqua (macOS and Linux)

Binary packages

CircleCI Orbs

ecschedule

Songmu / ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

ecschedule

Synopsis

Description

Installation

GitHub Actions

Terraform

The Issue of Managing ECS Task Definitions with Terraform

Leveraging Terraform Cloud State with ecspresso and ecschedule

Conclusion

Enhancing MUI Autocomplete and Formik Integration: Insights and Practical Tips

Demo

Code

Implementation Points

Match the type of Autocomplete's value with the field type in Formik

Pass an array of values to Autocomplete's options

Use getOptionLabel to display values in a different format than what's in options

Use null to represent the "unselected" state in Autocomplete

MUI provides debounce functionality

Memoize functions with debounce using useMemo or useCallback

Use inputValue as the keyword for API search

Conclusion

Reference Links

Securing Your Next.js Application with Strict CSP

tl;dr;

Prerequisites

What is Strict CSP?

What is the strict-dynamic directive?

Implementing Strict CSP in Next.js

Implementation Example

Let's check it out

In conclusion

References

Use a different git config for each directory using conditional include

See also

10+ things I always setup in git when I prepare a new environment

Match the type of Autocomplete's `value` with the field type in Formik

Pass an array of values to Autocomplete's `options`

Use `getOptionLabel` to display values in a different format than what's in `options`

Use `null` to represent the "unselected" state in Autocomplete

Memoize functions with debounce using `useMemo` or `useCallback`

Use `inputValue` as the keyword for API search

Make `--rebase` the default behavior during git pull

Make `--prune` the default behavior during git fetch

Use `nvim` as the editor to be used when committing

Output line numbers in the result of `git grep`