DEV Community: Jam3

Modular Solution for Typeahead Feature

Jeff Ong — Thu, 10 Dec 2020 20:53:35 +0000

The feature that I am trying to build here is about providing a typeahead (or autocomplete) feature with a dynamic list of terms for a UI. Those terms could be words, phrases, or numbers. I am hoping to do the search operations without any HTTP requests. There are a few solutions available out there but most of them are too complex or too tightly coupled with the UI component.

The solution has to be:

Decoupled from the UI
Efficient at search operations
Flexible enough that a developer can implement it either on the front-end or back-end

Planning

Instead of breaking the search queries (prefixes) into arrays and compare it against every single word, phrase, and number in the form of an array, our solution will include a tree data structure and some recursive algorithms.

So let us begin with Ternary Search Tree (TST). It is an extension of Tries Tree where each node contains a character and is connected by edges forming a tree structure; TST is more space-efficient as it only contains up to 3 child nodes whereas Tries will contain up to 26 child nodes in an alphabetical vocabulary dataset.

Tries Tree - built with ['search', 'sections', 'seek', 'sear', 'seed', 'set', 'seo']

Ternary Search Tree - built with ['search', 'sections', 'seek', 'sear', 'seed', 'set', 'seo']

In a TST data structure, child nodes are stored as Binary Search Tree (BST) where greater alphabets are stored on the right and others are on the left. This structure allows us to reduce the size of the problem by half at every iteration in a search operation with the time complexity of O(log n) on average.

In a worse case scenario where the tree is not height-balanced meaning one side of the sub-tree has 2 or more edges than the other side, we have to hit every single node in a search operation until we reach the leaf node (the last node of the tree) or the matched suffix with the time complexity of O(n). We could prevent this by randomizing the data set and use the data item in the middle of the set as the root node. By doing this, we could guarantee that the tree will be height-balanced on average.

Average Case:
Search - O(log n)
Insert - O(log n)

Worst Case:
Search - O(n)
Insert - O(n)

Implementation

After some planning, I have put together a NPM package: typing-ahead implementing TST with the insert and search operations.

Those operations are written as recursive algorithms where each function call branches off to create a stack of calls and each call, in our case, reduces the size of the problem by half. We could trace back to each step of the recursion to debug our operations.

A set of tests (written with Jest) are also provided in the package to ensure that TST is height-balanced and the search operations return the expected results. One of the tests also validates the structure of the data model ensuring that it is a valid JSON schema.

Here is how the package works:

Import the module

const typeahead = require('typing-ahead');

Supply the dataset and record the model

const model = typeahead.generate([dataset]);

Use the model when running search queries

const results = typeahead.find('queries', model);

npm run test in typing-ahead package runs all available unit tests

Here is a sandbox that shows how it works.
https://codesandbox.io/embed/typing-ahead-pq64e?fontsize=14&hidenavigation=1&theme=dark

Conclusion

At the end of the day, we have a modular and fairly efficient solution for developers to implement either on the front-end layer of the applications or provide the search feature through remote services with caching.

This prototype is built with Jam3 NexJS Generator

Dynamic social image generation with CloudFront

Iran Reyes Fleitas — Sat, 08 Aug 2020 16:32:03 +0000

In this article, I will walk you through how to create dynamic images using Amazon CloudFront and AWS Lambda@Edge that will be shared on social networks.

This solution is a simplified version of one of the modules from a project we did at Jam3 some months ago. This project had strict performance requirements as it reached vast audiences from every continent. We had fun crafting the solution, and I hope you'll find it useful.

Stack

There are many ways to achieve this result. After analyzing the pros and cons of all of them, we decided to move forward with:

Goal

Our goal is to generate images that don't exist on the fly, and images that do exist will be cached and returned. Subsequence visits will get the CDN cached version until it expires.

Disclaimer:
We will use the dummy domains app.com and social.app.com as examples, but those are not real domains.

Use case - First time generation

Image https://social.app.com/12-34-10.jpg is requested for the very first time.
The cloud backend will generate that image on the fly and will return it as fast as possible.

Use case - Cached version

Image https://social.app.com/12-34-10.jpg is requested again in less than x number of days.
The image will be returned from the cache of Amazon CloudFront.

Use case - Cache expired version

Image https://social.app.com/12-34-10.jpg is requested again after x number of days.
The cached version of the image already expired, a new image will be cache and returned.

These images will be used in the og-image tag of a site, and when the site is shared on social networks with a dynamic URL query string, the right image will be rendered.

Example:
When the site https://app.com?social=12-34-10 is shared on Facebook, the image https://social.app.com/12-34-10.jpg will be fetched.

Solution

There are three (dummy) domains working here:

The main application domain: https://app.com
The dynamic image generation service: https://social.app.com
A secret domain, only accessible by social.app.com, with an excellent performance site that will be used to generate the dynamic image

When the main application (app.com) loads, we will dynamically build the URL in the og-image tag based on its query parameters. If the page is shared on social networks, like Facebook, the social network crawler will fetch the dynamic image. If the dynamic image is cached or was already generated, it will be returned immediately. In case it doesn't exist it will be generated, cached, and returned.

To generate the image, we created a high-performance site in which the markup was precisely the image we wanted to share. We loaded that site on AWS Lambda@Edge and, with Puppeteer, we took a screenshot that would be returned to the user.

We tried other approaches like compositing the image together using image libraries. But after many tests, the generation time on our approach was considerably less than the rest.

In-depth explanation

Updating og-image

When the user lands on https://app.com?social=12-34-10 we verify that the query string social has the expected format and, if it's valid, we update the site og-image metadata with the value https://social.app.com/12-34-10.jpg.

If the social query string doesn't exist or the value is incorrect, a default image will be used. This logic must be coded and tested carefully to avoid any attempt of reflected XSS.

Image generation service AWS infrastructure

To build the dynamic image on the site hosted at https://social.app.com, we will use some Amazon Web Services including Amazon Route53, Amazon S3, Amazon CloudFront, and Lambda@Edge. An extended production-ready application will also use AWS WAF, Amazon CloudWatch, Amazon SNS.

To briefly describe the AWS infrastructure, Route 53 will serve a CDN version of the site using CloudFront. Lambda@Edge will render the dynamic images and save them in an Image S3 bucket. The CloudFront distribution has a Web Firewall with some custom and out of the box rules, and everything is being logged to CloudWatch. If the Lambda@Edge fails the rendering or another suspicious activity/error is detected, CloudWatch will trigger alarms that will be delivered to the development team through Amazon Simple Notification Service (SNS).

We will use two lambdas at the edge of the CloudFront distribution. The first Lambda will validate that the URL path is what we are expecting; otherwise, it will return a default image avoiding any 404. The validation Lambda@Edge will be executed before the request hits the CloudFront origin, on the Origin Request event. If the URL path is what we are expecting, we will let the request through toward the CloudFront origin, aka Amazon S3.

We are executing this Lambda after the CloudFront cache to improve performance.

When the CloudFront origin responds, it can come with an actual object or with a 404. If the image wasn't found, the second Lambda@Edge will be executed on the Origin Response event and it will generate the image dynamically, save it in Amazon S3, and return it to the cache.

The Lambda@Edge will save the image in the S3 bucket for future requests. When the CloudFront cache expires, the cache will be updated on the object stored in S3 and will be distributed available again.

Image generation with puppeteer

Although there are many ways to generate dynamic images, there isn't a one-size-fits-all solution/approach for all cases. In our case, the images to generate had many variations and crafting them with libraries like sharp or jimp, or a similar image manipulation library was not performing well; we also evaluated using a third-party service like Cloudinary, but the desired compositions were not possible.

So our solution was to do all the magic in a mini React application and, with Puppeteer, take a screenshot. As you can imagine, our requirements for the React application were mainly hyper-performance. Using only hooks, brotli, webp, super lightweight dependencies, and other features, we were able to easily achieve the expected performance (loading times of 150-200ms).

We are not going to cover how to protect the third secret domain, but you can use basic-auth or tokens. Because the performance is outstanding, I would recommend avoiding more heavy solutions like OAuth v2 or JWT.
If you can afford a different domain for this third site, that would avoid any discovery using a DNS scanning tool for subdomains.

Implementation

Let's dive into some code.

Heads up:
I'll omit some advanced details that will be covered at the end of this article.

Validation Lambda

The validation Lambda@Edge looks similar to:

'use strict';

const isPathInvalid = require('./is-path-invalid');

const defaultPath = '/default.jpg';

exports.handler = async (event, context, callback) => {
  const request = event.Records[0].cf.request;

  if (isPathInvalid(request.uri)) {
    console.warn(`Invalid path, redirecting to ${defaultPath}`);
    request.uri = defaultPath;
  } else {
    console.log('Valid path, moving request to the origin');      
  }

  callback(null, request);
};

As you can see, there's some logic that validates the URL path. If the URL path has the expected format, the request will continue to the next Lambda. In case it's invalid, it will change the requested image for the default image.

The module isPathInvalid depends on your validation. If it's too simple, you might want to consider writing the validation directly instead of having it on a separate file. In our case, it's better to have it on another module because we can write unit tests over the isPathInvalid function.

Main Lambda

The main Lambda@Edge which generates and uploads assets to Amazon S3 looks similar to:

'use strict';

const path = require('path');

const generateImage = require('./generate-image');
const uploadImage = require('./upload-image');

exports.handler = async (event, context, callback) => {
  const response = event.Records[0].cf.response;
  const request = event.Records[0].cf.request;
  let globaErrorStatus = false;

  if (response.status == 404 || response.status == 403) {
    try {
      const urlPath = path.parse(request.uri);
      const s3key = urlPath.base;
      const imageId = urlPath.name;

      const imageBuffer = await generateImage(imageId);

      if (imageBuffer) {
        await uploadImage(imageBuffer, s3key);

        response.status = 200;
        response.statusDescription = 'OK';
        response.body = imageBuffer.toString('base64');
        response.bodyEncoding = 'base64';
        response.headers['content-type'] = [{ key: 'Content-Type', value: 'image/jpeg' }];
      } else {
        console.error('The image generation and the default is invalid, letting the request to fail');
        globaErrorStatus = true;
      }
    } catch (error) {
      console.error(`General exception, something failed on the way: ${error.message}`);
      globaErrorStatus = true;
    }
  }

  if (globaErrorStatus) {
    response.status = 500;
    response.statusDescription = "We couldn't retrieve the asset";
  }

  callback(null, response);
};

The following steps explain what this function is doing:

Step #1
It's executing only if the image was not found in origin.

if (response.status == 404 || response.status == 403)

Step #2
Assuming the URL Path is coming as we are expecting, because of the validation Lambda@Edge, we are parsing the URL Path we need.

const urlPath = path.parse(request.uri);
const s3key = urlPath.base;
const imageId = urlPath.name;

For a path like 12-23-12.jpg, s3key will be 12-23-12.jpg, and imageId will be 12-23-12.

Step #3
We generate a dynamic image and return a buffer with the raw image (we'll cover this functionality later on).

const imageBuffer = await generateImage(imageId);

Step #4
If the image was successfully generated we proceed to upload it to S3.

if (imageBuffer) {
  await uploadImage(imageBuffer, s3key);

Step #5
Once the image has been uploaded to S3, we prepare the CloudFront response for caching.

response.status = 200;
response.statusDescription = 'OK';
response.body = imageBuffer.toString('base64');
response.bodyEncoding = 'base64';
response.headers['content-type'] = [{ key: 'Content-Type', value: 'image/jpeg' }];

Step #6
Unexpected error handling in case something outside of our edge cases goes sideways. In that case, we will be returning a 500 error to the client.

if (globaErrorStatus) {
  response.status = 500;
  response.statusDescription = "We couldn't retrieve the asset";
}

callback(null, response);

Generation module

In the main Lambda, we used the function generateImage(imageId) to generate the dynamic image; we will go through our approach in this chapter.

A simplified version of the module looks like:

'use strict';

const fs = require('fs').promises;
const chromium = require('chrome-aws-lambda');

const domain = 'https://sharing-secret-site.com/';
const defaultImage = './assets/default.jpg';
const token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';

async function generateImage(imageId) {
  let browser = null;
  let imageBuffer = null;

  try {
    browser = await chromium.puppeteer.launch({
      args: chromium.args,
      defaultViewport: {
        width: 1200,
        height: 630,
        deviceScaleFactor: 2
      },
      executablePath: await chromium.executablePath,
      headless: chromium.headless
    });

    let page = await browser.newPage();

    const targetPage = `${domain}?social=${imageId}?token=${token}`;

    const pageResponse = await page.goto(targetPage, {
      waitUntil: ['networkidle0'],
      timeout: 3000
    });

    const pageStatus = pageResponse.status();

    if (pageStatus >= 400 && pageStatus < 600) {
      throw new Error(`${domain} is not loading`);
    }

    imageBuffer = await page.screenshot({ type: 'jpeg' });
  } catch (error) {
    try {
      imageBuffer = await fs.readFile(defaultImage);
    } catch (fsError) {
      console.error(`Exception while returning the default image: ${fsError.message}`);
    }
  } finally {
    if (browser !== null) {
      await browser.close();
    }
  }

  return imageBuffer;
}

module.exports = generateImage;

Highlights:

It's using chrome-aws-lambda because headless chrome by default is bigger than the maximum size supported by Lambda.
The sharing application will load only if the token is right. You can also add basic-auth to the headers.
We will wait until the network is idle, usually after the 500ms average.
We are taking the screenshot of the loaded site and returning the image buffer.
If there is any kind of error generating the screenshot, we will return the default image.

Note:
After some timing tests, it's better for us not to compress the screenshot before sending it, but it may be better to consider it for your case.

S3 Uploader module

Finally, this is what the module that sends the generated image to S3 looks like:

'use strict';

const AWS = require('aws-sdk');

const S3 = new AWS.S3({
  signatureVersion: 'v4'
});

const BUCKET = 'xxx-xxxxxxx-xxxx-xxxxxxx';

async function uploadImage(buffer, s3key) {
  return S3.putObject({
    Body: buffer,
    Bucket: BUCKET,
    ContentType: 'image/jpeg',
    CacheControl: 'max-age=31536000',
    Key: s3key,
    StorageClass: 'STANDARD'
  })
    .promise()
    .catch(error => {
      console.error('Exception while writing image to bucket', JSON.stringify(error));
    });
}

module.exports = uploadImage;

Highlights:

We are setting a high cache in origin (and CloudFront will respect it).
For the Lambda to be able to upload objects to S3, it will need permissions in the bucket.

Conclusion

The solution works as expected, and we recommend it if you have a similar use case.

Costs reductions:
If we want to reduce Lambda running costs and pre-populate the most expected images, we just need to upload them to S3.

Production-ready considerations

We've covered the main components of the solution to simplify this post, but to make the solution production-ready, there are a couple of extra things to have in mind. We will briefly cover some of them.

Main site query string validation & sanitization

The main site responsibility is to move the query string to the og-image and build the dynamic social image URL.

Requesting https://app.com?social=12-34-10 will render

<head>
 <meta property="og:image" content="https://social.app.com/12-34-10.jpg">
</head>

We strongly recommend avoiding passing any kind of domain in the query string. If you need to pass it, make sure you whitelist it to avoid "URL Redirection to Untrusted Site" vulnerabilities.
In our example, the social id has a specific format (like 12-34-10). We strongly recommend to sanitize it and verify that social id before update the og-image tag, and it will avoid "Reflected XSS" vulnerabilities.
And last but not least, if you use a regex to verify your social id, be aware of "ReDoS" vulnerabilities and make sure your regex is safe.

Cloudfront best practices

Use All Edge Locations for better performance.
Make sure you're using one SSL certificate per domain instead of wildcard SSL certificates.
Enable WAF (we will cover it later).
Use the latest version of TLS.
Enable Logging.
Enable IPv6.
Handle only HTTPS connections for the social and private domains, and Redirect HTTP to HTTPS for the main application.
Allow only GET and HEAD requests.
Cache as much as you can (there are multiple options).
Disable any type of compression (we are returning an image).

Route53

Setup query logging for your Hosted Zone.

Web Application Firewall

Everything outside of what we are expecting is an attempt to misuse the service. It's up to us how much we want other people to play with the service.

Enable WAF for CloudFront.
Enable logging with Kinesis Firehose.
Consider some managed rules like (IP reputation list or Known bad inputs).
Consider creating rules to block unexpected query strings.
Consider validating the URI path using a regex match.
Consider restricting the size of the URI path.
Consider blocking any request with a body (we are expecting only HTTP GET requests).

S3 best practices

The buckets should be private.
Enable versioning (you shouldn't expect different versions of the images).
Enable Cloudtrail for writing events.
Enable server access logging (on another private bucket).
Enable Server-Side encryption, KMS is recommended.
If you are not expecting to delete the stored images, you can enable "Object Lock" when creating the bucket to increase the security around the bucket.
Enabling "Transfer acceleration" will give you a better performance uploading the screenshots to S3 with an additional cost.
If Object Lock was not used, add an event when objects are deleted and receive a notification.
If you want to get isolated notifications per image created, you can create an event (if you enabled Cloudtrail, you will have this information).
Make sure to achieve the Least Privilege Permissions and be specific with your bucket policies.

CloudWatch

Consider adding CloudFront alarm when requests are more than the expected.
Consider adding CloudFront alarm when 4xx or 5xx happens.
Consider adding alarms for both Lambda@Edge based on Errors and Duration.
Consider adding alarms for both Lambda@Edge based on the content of the logs using Metric Filters.
Create a dashboard with a collection of metrics of your interests (CloudFront and Lambdas).

Testing

Make sure your code is unit test friendly, and your test coverage covers your critical flows.
Make sure to create integration tests that test all the functionality from end to end, with the AWS UI built-in test runner or better off AWS with lambda-local or serverless.

Infrastructure

High recommended to use a code repository (GitHub, Gitlab, Bitbucket, etc.) and keep the source code there.
High recommended having everything automated using a CI/CD platform.
Infrastructure as Code is essential to avoid differences between environments and keep track of the changes, tune your skills with CloudFormation, Terraform, or Serverless.

I hope you enjoyed reading this post and, most importantly, I hope it helps you.

Creating a localized experience for visitors from other countries using React Redux

Vadim Namniak — Tue, 15 Oct 2019 21:02:34 +0000

Getting Started

It is assumed you are already familiar with both React and Redux and looking to add internalization to your application. If you are not, there is a number of boilerplate options out there that can help you get started.
Feel free to check out our implementation of it that we use at Jam3.

Prerequisites

You are highly advised to read the i18next internationalization framework documentation to get an understanding of the main concepts and benefits of using it.

List of required extra dependencies:

i18next (37kB / 10.5kB)
react-i18next v.9 (12.4kB / 4.6kB)
i18next-browser-languagedetector (6kB / 2kB)
i18next-redux-languagedetector (2.2kB / 790B)
i18next-chained-backend (2.2kB / 933B)
i18next-fetch-backend (4.3kB / 1.7kB)

Take a sneak peek at these libraries before we proceed.

👉 Consider the overall additional cost of roughly 20kB (minified and gzipped) added to the production build.

Installation

Run this command in your terminal to install the above modules in one batch:
$ npm i --save i18next react-i18next@9.0.10 i18next-fetch-backend i18next-browser-languagedetector i18next-redux-languagedetector i18next-chained-backend

Configuration

The example we’ll be referring to is bootstrapped with Create React App with added Redux on top.
Here’s what our application structure will look like:

See the CodeSandbox example or check this GitHub repo.

Step 1: Creating translation files

We are going to use English and Russian translations as an example.
Let’s create two JSON files with identical structure and keep them in their respective folders:

/public/locales/en-US/common.json

/public/locales/ru/common.json

These files will serve as our translation resources that are automatically loaded based on the detected browser language.

Step 2: Creating the i18n config file

Make sure to check the complete list of available i18next config options.
This is our main localization config file:

/src/i18n/index.js

First off, we need to add the i18next-chained-backend plugin which allows chaining multiple backends. There are several backend types available for different purposes. We are using fetch to load our translation resources.
Then we are adding Browser Language Detector (connected with Redux store through Redux Language Detector) for automatic user language detection in the browser. Read more about the approach.
Next up, we use reactI18nextModule to pass i18n instance down to react-i18next.
Finally, we initialize i18next with basic config options.

Step 3: Adding i18next reducer to the store

Redux Language Detector provides i18nextReducer so you don’t need to implement your own reducers or actions for it — simply include it in your store:

/src/redux/index.js

👉 For your convenience, use Redux dev tools in dev environment and make sure you import composeWithDevTools from redux-devtools-extension/developmentOnly.

Step 4: Creating the main app file

There’s nothing specifically related to the internalization in this file.
We simply set the routes for our pages in a standard way.

/src/app/index.js

Step 5: Initializing the app and adding I18nextProvider

The provider is responsible for passing the i18next instance down to withNamespaces HOC or NamespacesConsumer render prop.

/src/index.js

We initialized our store and i18n config file with the same options to keep both in sync.

Step 6: Using translation keys

We‘ll use withNamespaces HOC that passes the t function as a prop down to the component. We need to specify the namespace(s), and the copy is now accessible via object properties using t function: t(‘homePage.title’).
Note, it is required to prepend the namespace when accessing the copy from multiple namespaces within one component e.g. t('shared:banner.title').

/src/pages/Home.js

Alternatively, we could use NamespacesConsumer component which would also give us access to the t function. We’ll cover it in the next step.

👉 You can test language detection by changing your default browser language. When using Chrome, go to chrome://settings/languages and move the languages up and down in the list.

Step 7 (Bonus part): Creating language switcher

Ok, we’ve implemented language auto-detection and dynamic translation resources loading. Now it’s time to take it up a notch and create a component that allows users switching the language through user interface.
Make sure to include this component in your app.

/src/components/LanguageSwitcher.js

NamespacesConsumer render prop provides access to the i18n instance. Its changeLanguage method can be used to change language globally. This will force the app to re-render and update the site with the translated content.

🎉That’s a wrap!

Code examples

Cryptography git commits

Donghyuk (Jacob) Jang — Sun, 18 Aug 2019 04:39:31 +0000

Git is an industry-standard version control tool used in IT fields, so it has a mandatory tool and a skill to learn for developers.

During the development stage, developers make hundreds / thousands of commits. However, it is not difficult to find unverified commits, although it would be related to security flaws and potentially impacts the vulnerability of projects.

This article will mainly focus on the importance of the signing(verified) commits. As well, there is one practice on how to pretend someone else in Git repositories.

Brief about SSH key
GPG key
An example of a security flaw without using GPG key
How to setup GPG key
How to create signing commits
Final thoughts

Brief about SSH

SSH-key added in Github

First of all, I would assume that you have already set up SSH-key on your machine to authenticate to Github and have it registered with your Github account. If not, please read Connecting to GitHub with SSH and follow the steps.

Setting up SSH-key means that you do not have to provide your Github username or password for all of your git activity. That is the main reason to set up an SSH-key. Does connecting to Github with an SSH-key mean all of your commits become secure? We will dig into this and talk more about the disadvantages of only using an SSH-key later in this article.

Then, what is GPG key?

GnuPG logo

Gnu Privacy Guard(GPG) is a tool that allows users to integrate an additional security layer easily with other applications. In this case, it will be Git.

GnuPG is a complete and free implementation of the OpenPGP standard as defined by RFC4880 (also known as PGP). GnuPG allows you to encrypt and sign your data and communications; it features a versatile key management system, along with access modules for all kinds of public key directories. GnuPG, also known as GPG, is a command line tool with features for easy integration with other applications. A wealth of frontend applications and libraries are available. GnuPG also provides support for S/MIME and Secure Shell (ssh).

Since its introduction in 1997, GnuPG is Free Software (meaning that it respects your freedom). It can be freely used, modified and distributed under the terms of the GNU General Public License .

The current version of GnuPG is 2.2.17. See the download page for other maintained versions.

Gpg4win is a Windows version of GnuPG featuring a context menu tool, a crypto manager, and an Outlook plugin to send and receive standard PGP/MIME mails. The current version of Gpg4win is 3.1.10.

The GNU Privacy Guard

What could happen if you do not use GPG key?

a costume under surveillance

One very common severe issue is that someone can set up other developers' Github username and email in their local machine, and create commits which will appear as "the someone else"' commits in the Git repository. Of course, this activity needs to meet a certain condition. The command below shows how to set up a different Github username and email within your terminal.

// setup username and email
$ git config --global user.name "No GPG"
$ git config --global user.email no.gpg@email.com

// verity username and email
$ git config --global user.name
> No GPG
$ git config --global user.email
> no.gpg@email.com

Here I want to share with you an example of two different commits that I pushed in a test Git repository(signing commit).

When I created this git repository, I made the initial push with a signed(verified) commit.

Screenshot of git commit history

And then, I asked a friend of mine (@alemesa) to follow the practice above and gave him my Github username and email. (These credentials for signing in can be found with very little effort.) He created a commit and pushed to the master branch. Check out the results below.

Screenshot of git commit history

The commit has appeared that it was created by me. However, there is no "verified" flag like the first initial commit.

If master branch is not protected like the testing Git repository, then anyone in the contributor list can push any code by pretending to be someone else. Please find details about how to protect branches here.

NOTE: If you want to test this activity, please send me a direct message that includes your Github username. I will add you into the collaborator list.

How to setup GPG key

The references will guide you on how to setup GPG key in your workstation, and how to add them into your Github account.

How to create signing commit(s)

Add file(s) before creating a commit:

// add a single file
$ git add [path file]

// add all file
$ git add .

The git command below is how to create a commit with or without your signature:

// create commit with a message without signing
$ git commit -m "commit message"

// add all file that you want to commit
$ git commit -S -m "commit message"

Once your signing commit(s) is created, the next step is to push to a git repo:

// push the commit
$ git push

The screenshot below is an example of git signing commit in the test git repository(signing commit)

detail of signing commit

Conclusion

To sum this article up, using GPG-key will increase security and authenticity level by encrypting its data, and adding a person's signature. If you or your organization have considered these, the steps above will bring its advantages and ensure trust with your commit workflow.

NOTE: this activity may not be suitable for all case. Please make sure using this for the right purpose.

References

Collaborators

Alejandro Mesa (@alemesa) - alejandro.suarez@jam3.com
Danny Paton - danny.paton@jam3.com

Managing .env variables for provisional builds with Create React App

Donghyuk (Jacob) Jang — Sun, 07 Apr 2019 21:41:41 +0000

When developing web applications by using Create React App, developers get NODE_ENV=development on their local environment and NODE_ENV=production on the production build by default. And, modifying NODE_ENV is forbidden. According to the Create React App, this is an intentional setting to protect the production environment from a mistake/accident deploying.

You will be able to see scripts like below in package.json after creating your web app.

// package.json
scripts: {
  "start": "react-scripts start", // `NODE_ENV` is equal to `development`.
  "build": "react-scripts build", // `NODE_ENV` is equal to `production`.
  ...
}

If you create or already have .env.development and .env.production in the root of your project, these files will be used for running each script. npm start will pick up .env.development, and npm build will use environment variables in .env.production.

-
What if you want to setup .env.staging?

This article will show you how to manage environment variables for provisional builds.

Let's dive into that! Oh, if you do not have any experiences of CRA, please Getting started

Story

Imagine that your project will have three separated provisional environments; development, staging, and production. Each environment is using different API endpoints. In addition to that, the project may require a REACT_APP_CUSTOM_NODE_ENV. This is because NODE_ENV will always be production for the build regardless.

Goal

Create .env.development, .env.staging, and .env.production.
Configure environment viriables for each build.
Modify scripts in package.json

Getting started

Step 1.

Thankfully, dotenv comes out of box. Let's create .env files under the root folder to manage environment variables. The files are .env, .env.development, .env.staging, and .env.production.

.env - Keep all common/shared environment variable
.env.development - Variables are used for the local development
.env.staging - Variables are used for the staging build
.env.production - Variables are used for the production build

For example;

#.env
REACT_APP_DOC_TITLE = "Document title"

//.env.developement
REACT_APP_API_ENDPOINT = "https://development-api.endpoint.com/"

#.env.staging

# NODE_ENV will always be set to "production" for a build
# more details at https://create-react-app.dev/docs/deployment/#customizing-environment-variables-for-arbitrary-build-environments
REACT_APP_CUSTOM_NODE_ENV = "staging"
REACT_APP_API_ENDPOINT = "https://staging-api.endpoint.com/"

#.env.production
REACT_APP_API_ENDPOINT = "https://api.endpoint.com/"

NOTE: The prefix REACT_APP_ is required when creating custom environment variables.

`.env.development` and `.env.production`

As a default behavior, those files will be served with no configuration. You do not even have to update scripts in package.json

`.env.staging`

Here is the main focus of this post. To target .env.staging file for the staging build, we need a library to achieve this.

1- Let's install env-cmd. This library will will help us on using/executing a selected environment file. See more detail

// execute command below at the root of project
$ npm install env-cmd --save
or
$ yarn add env-cmd

2- Add a script in package.json like below.

// package.json
scripts: {
  "start": "react-scripts start", // `NODE_ENV` is equal to `development`.
  "build": "react-scripts build", // `NODE_ENV` is equal to `production`.
  "build:staging": "env-cmd -f .env.staging react-scripts build", // `NODE_ENV` is equal to `production`.
  ...
}

3- Finally, test your build:staging script.

Conclusion

The intention of this technique is to use different custom environment variables for many provisional environments without ejecting the default CRA configs.

Resources

Special Thanks

@foxbit19 - found env-cmd version 8.x requries additional argument to link to the custom env file.
@j6000 - pointed out NODE_ENV for the build will always be set to "production" regardless.

How to prevent XSS attacks when using dangerouslySetInnerHTML in React

Donghyuk (Jacob) Jang — Mon, 04 Feb 2019 21:11:04 +0000

This article intends to show one of the techniques we use to mitigate cross-site scripting (XSS) attacks at Jam3. These vulnerabilities may appear when dangerouslySetInnerHTML is wrongly used, and our goal is to detect it ahead of time and to clean up untrusted values.

Dangerously Set innerHTML

This feature is designed to present and insert DOM formatted content data into the frontend. The use of the feature is a bad practice, especially when dealing with user inputs and dynamic data. You must consider its vulnerabilities in order to prevent XSS attack.

"Easy" to make thing safe is one of React philosophy. React is flexible and extendable which means that the bad practice can be turning into the best practice. Sanitizing props value is one obvious option and strongly recommended.

XSS attacks

Cross-site scripting (XSS) allows attackers(hackers) to inject malicious code into a website for other end-users. By doing this, attackers may have access to personal data, cookies, webcams, and even more. Read more about Cross-site scripting.

Copy https://placeimgxxx.com/320/320/any" onerror="alert('xss injection') and paste it in the input field in the xss injection example below:

Preventing XSS

This issue is not restricted to React; to learn how to prevent it in your web development OWASP has a good prevention cheat sheet. One approach to prevent XSS attacks is to sanitize data. It can be done either on the server-side or the client-side; in this article, we will focus on the client-side solution.

Preventing XSS with dangerouslyInnerSetHTML

Sanitizing content in the frontend when using dangerouslySetInnerHTML is always a good security practice, even with a trusted source of truth. For example, another development team in charge of maintaining the project changes the source of truth without realizing how it could impact the site. A change like that may cause a critical XSS vulnerability.

At Jam3 we avoid using dangerouslySetInnerHTML whenever possible. When it's required, we always apply sanitization security layers on both the back-end and front-end. In addition, we created an ESLint rule called no-sanitizer-with-danger inside eslint-plugin-jam3 to detect improper use of dangerouslySetInnerHTML.

ESLint rule

I assume that you are already familiar with ESLint. If not, Get Started.



$ npm i eslint eslint-plugin-jam3 -save-dev

Extend pluginsin the .eslintrc config file by adding jam3. You can omit the eslint-plugin- prefix. Then, configure the rules by adding jam3/no-sanitizer-with-danger to the rules. Note: error level 2 is recommended. With this option, exit code will be 1. error level 1 will give warning alert, but does not affect exit code. 0 means to turn the rule off. The plugin will check that the content passed to dangerouslySetInnerHTML is wrapped in this sanitizer function. The wrapper function name can be also be changed in the JSON file (sanitizer is the default wrapper name).

How to use it

Here is an unsafe way of using dangerouslySetInnerHTML.

Once the rule is enabled, your code editor will alert the lack of a sanitizer in dangerouslySetInnerHTML. For the purpose of this article we use dompurify, you can find an extended list of available sanitizers at the end of the article.

The sanitizer wrapper must have a name, for the purpose of this article we are creating const sanitizer = dompurify.sanitize;. However, it is recommended to create a sanitization utility to abstract your chosen sanitizer.

Sanitizer libraries

Our team has researched and tried many sanitizers and can recommend these 3 libraries.

dompurify

Strip out all dirty HTML and returns clean HTML data npm Weekly download 50k+
40 contributors
Earned 2800+ GitHub ⭐️
5.6kB MINIFIED + GZIPPED

xss

Escape HTML entity characters to prevent the attack which occurs to transform non-readable content for the end users
npm weekly download 30k+
18 contributors
Earned 2500+ github ⭐️
5.3kB MINIFIED + GZIPPED

xss-filters

Escape HTML entity characters to prevent the attack which occurs to transform non-readable content for the end users
npm weekly download 30k+
5 contributors
Earned 900+ github ⭐️
2.1kB MINIFIED + GZIPPED

Conclusion

To sum up, finding the most suitable sanitizer library for your project is very important for security. You might want to have a look at GitHub stars, npm download numbers, and maintenance routines. The use of no-sanitizer-with-danger in eslint-plugin-jam3 will be a great choice to help ensure all data is being properly purified and gain confidence that your project will be safe from XSS vulnerabilities.

NOTE: Please keep it in mind there is a performance disadvantage of sanitizing data in client-side. For example, sanitizing all data at once may slow down the initial load. To prevent this in large scale web applications, you can implement a "lazy-sanitizing" approach to sanitize on the fly.

Contributors

Donghyuk(Jacob) Jang / jacob.jang@jam3.com
Iran Reyes / iran.reyes@jam3.com
Peter Altamirano / peter.altamirano@jam3.com

Jam3 is a design and experience agency that partners with forward-thinking brands from around the world. To learn more, visit us at jam3.com.

Article by Donghyuk (Jacob) Jang

DEV Community: Jam3

Modular Solution for Typeahead Feature

Planning

Implementation

Conclusion

Dynamic social image generation with CloudFront

Stack

Goal

Solution

In-depth explanation

Updating og-image

Image generation service AWS infrastructure

Image generation with puppeteer

Implementation

Validation Lambda

Main Lambda

Generation module

S3 Uploader module

Conclusion

Production-ready considerations

Main site query string validation & sanitization

Cloudfront best practices

Route53

Web Application Firewall

S3 best practices

CloudWatch

Testing

Infrastructure

Creating a localized experience for visitors from other countries using React Redux

Getting Started

Prerequisites

Installation

Configuration

Step 1: Creating translation files

Step 2: Creating the i18n config file

Step 3: Adding i18next reducer to the store

Step 4: Creating the main app file

Step 5: Initializing the app and adding I18nextProvider

Step 6: Using translation keys

Step 7 (Bonus part): Creating language switcher

Code examples

Related documentation

Cryptography git commits

Table of Contents

Brief about SSH

Then, what is GPG key?

What could happen if you do not use GPG key?

How to setup GPG key

How to create signing commit(s)

Conclusion

References

Collaborators

Managing .env variables for provisional builds with Create React App

Story

Goal

Getting started

Step 1.

.env.development and .env.production

.env.staging

Conclusion

Resources

Special Thanks

How to prevent XSS attacks when using dangerouslySetInnerHTML in React

Dangerously Set innerHTML

XSS attacks

Preventing XSS

Preventing XSS with dangerouslyInnerSetHTML

ESLint rule

How to use it

Sanitizer libraries

Conclusion

Further reading and sources

Contributors

`.env.development` and `.env.production`

`.env.staging`