DEV Community: Vinicius Kiatkoski Neves

Use AWS through GitHub Actions without secret keys

Vinicius Kiatkoski Neves — Thu, 17 Mar 2022 12:47:38 +0000

Cover photo from Chris Barbalis on Unsplash.

It all started with the following question: How do we safely store AWS IAM User Keys (Access and Secret) created by IaC?

Imagine the following scenario: you have a Bucket that will host your Frontend assets. Your Frontend lives in another repository and you use, in my example, GitHub Actions to deploy (move) those files to the Bucket. You want to give permission to your GitHub Actions to perform that and only that.

You can, of course, add your credentials to the repository but you will give it too many permissions.

You can create a User, in your infrastructure, that has only the permission needed by GitHub Actions. This sounds way better already. The issue is: where do we store this User's keys?

We can have the keys as Output of our Stack. It is not safe, everyone that has access to the logs from your GitHub Action can see it (imagine an open source repository).
We can store these keys in SSM. It works but everyone that has access to SSM has access to it (not a big deal in my case).
We can store these keys in Secrets Manager. It works but everyone that has access to Secrets Manager has access to it (not a big deal in my case).
We can generate the keys manually. It works and only the person that generated the keys will know it (hopefully store them in GitHub Secrets and forget about it).

All the steps above work to some extent. The only one that doesn't require any manual effort is option 1. which is the least secure one.

The best approach, from my understanding, is another one. I would like to go through it step by step and how to setup it up using AWS CDK.

OpenID Connect (OIDC)

Since October of 2021 GitHub Actions allow it: https://github.blog/changelog/2021-10-27-github-actions-secure-cloud-deployments-with-openid-connect/

OpenID Connect allows you to connect an external provider (GitHub Actions) with your AWS account. You can read more about it here: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html

The external provider will assume a defined role that has the permissions (policies) we need. You can read more about IAM roles here: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html

Breaking it into steps:

We need a policy that allows us to move files into the Bucket (nothing more than that);
We need a role that will be assumed by GitHub Actions and this role needs to use the policy from step 1.
We need an OIDC identity provider to connect our GitHub Actions.

GitHub and AWS have an extensive documentation about it:

How does it look in the code?

Let's imagine we have a Bucket

const bucket = new Bucket(this, "Bucket");

Let's create the OIDC following the documentation

const provider = new OpenIdConnectProvider(this, "GitHubProvider", {
  url: "https://token.actions.githubusercontent.com",
  clientIds: ["sts.amazonaws.com"], // Referred as "Audience" in the documentation
});

Let's create our role (AWS CDK makes it easy to link the role with the identity provider)

const role = new Role(this, "Role", {
  assumedBy: new OpenIdConnectPrincipal(provider, {
    StringEquals: {
      "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", // "Audience" from above
    },
    StringLike: {
      "token.actions.githubusercontent.com:sub":
        "repo:<ORGANIZATION>/<REPOSITORY>:*", // Organization and repository that are allowed to assume this role
    },
  }),
});

The example above is a bit different from the one documented by GitHub. Here I'm making it more flexible (all branches, for example, can use it).

The assumedBy property can be customized:

You can play with the conditions: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html
You can use other properties from the payload: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#understanding-the-oidc-token

Let's grant the necessary permission to our role

bucket.grantPut(role);

Once deployed everything is ready on the AWS side, now we go to the GitHub side.

Let's update our GitHub Action to assume the role

# Before
...
- name: Configure AWS Credentials 🔧
  uses: aws-actions/configure-aws-credentials@v1.6.1 # Version at the time of this writing
  with: # We are loading keys stored in secrets
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: eu-central-1
- run: npm run deploy # Just an example

# After
...
permissions:
  id-token: write
  contents: read # This is required for actions/checkout (GitHub documentation mentions that)
...
- name: Configure AWS Credentials 🔧
  uses: aws-actions/configure-aws-credentials@v1.6.1 # Version at the time of this writing
  with: # We are loading keys stored in secrets
    role-to-assume: arn:aws:iam::<ACCOUNT>:role/<ROLE-NAME> # You can specify the role name when creating it or AWS will automatically generate one for you
    aws-region: eu-central-1
- run: npm run deploy # Just an example

We can now safely remove the secrets from GitHub and also don't need any "special" user to deploy through GitHub Actions.

This helps me a lot to "wrap" the knowledge and I believe it can help other people too.

References:

Importing existing AWS resources using AWS CDK

Vinicius Kiatkoski Neves — Mon, 21 Feb 2022 12:20:35 +0000

Cover photo from Gavin Allanwood on Unsplash.

I like infrastructure and I advocate AWS CDK for one reason: it allows us to use Typescript (or other languages) which is less of a blocker for new contributors in frontend. It is verbose but it does a lot of the weight lifting for you and is extremely flexible as well.

I've always been curious about the following situation: most companies start setting up their infrastructure through AWS Console. At some point Infrastructure as Code is introduced. How do we close the gap between them? Shall we code everything from scratch?

For a while (end of 2019) we can import existing AWS resources into a CloudFormation Stack. The blog post from AWS does a good job explaining how it works. There is another blog post that explains how to do it with AWS CDK.

My goal here is to make it visual and with code examples. If you follow the steps from both blog posts you are good to go but I found myself confused with some names in between and decided to summarize it for myself (as I'm pretty sure I won't remember how to do it next time) and maybe help you as well.

I'm assuming that you're familiar with AWS CDK and also with CloudFormation. If not, don't hesitate to ask questions and I can try to answer them or guide you to other resources.

AWS CDK output

AWS CDK "simply" translates some code we write into a CloudFormation template in this case, a .json file by the end. It is this file that is used by AWS CloudFormation to setup our infrastructure.

The output from AWS CDK can be found inside the cdk.out/ folder. The CloudFormation template from your stack is in the file MyStackName.template.json.

AWS CDK creates this file whenever we run synth or deploy (which runs synth beforehand). The main difference is that deploy uploads this file to AWS CloudFormation, while synth "only" creates it.

Importing existing AWS resources

I will manually create a bucket in my account and run a small example (the steps are similar for more resources but it takes more time). I called it viniciuskneves-aws-cdk-real-bucket and you can see it in the image below:

In order to import an existing resource to our stack, we first need to create the stack. We can create a stack with a dummy resource as follows:



import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const DummyBucket = new Bucket(this, "DummyBucket", {
      removalPolicy: RemovalPolicy.DESTROY, // Easier to remove later
    });
  }
}

The bucket removal policy was set to delete as this bucket will be deleted later and it is not our real bucket, which will be imported later on.

Now we can run deploy and, after deploying our Stack, we should be able to see it in AWS CloudFormation console:

Following the instructions from AWS, we need to create a template with the resource we would like to import in this case, a S3 bucket. We can rely on AWS CDK to do the job for us. To create a S3 bucket, we need the following piece of code:



import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const DummyBucket = new Bucket(this, "DummyBucket", {
      removalPolicy: RemovalPolicy.DESTROY,
    });
    const RealBucket = new Bucket(this, "RealBucket", {
      removalPolicy: RemovalPolicy.RETAIN, // Recommended in order to avoid data loss
    });
  }
}

Now we can run synth to obtain our CloudFormation template. This template contains a bucket but it is not yet deployed, we will use it to manually import the resource through the AWS console.

We can go to our stack in AWS CloudFormation console, click Stack Actions and Import resources into stack:

There we select the file created by synth in the step above (cdk.out/MyStackName.template.json):

This step tells CloudFormation which new template to use.
The next step is how to link the CloudFormation template with an existing resource. AWS S3 uses the bucket name for linking, so we need to input the bucket name (from the bucket created earlier) here:

In the last step you will see the applied changes and our imported bucket will be there:

After a while (in our example it happens almost immediately as we're talking about a single resource) you will find the resource imported in your stack:

Now we can try to run deploy again and we will get a message that no changes have been found in the stack, which means success to us:

From now on we can make changes in the stack and it will be applied as expected, as if the resource had been created by us, in AWS CDK, since the beginning:



const RealBucket = new Bucket(this, "RealBucket", {
  websiteIndexDocument: "index.html",
  removalPolicy: RemovalPolicy.RETAIN,
});

After deploying, we've:

We can finally remove the DummyBucket and as it had a destroy policy, the bucket will be removed as expected (no need to manually delete it):



import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const RealBucket = new Bucket(this, "RealBucket", {
      websiteIndexDocument: "index.html",
      removalPolicy: RemovalPolicy.RETAIN,
    });
  }
}

After deploying, we've:

Importing resources that don't follow defaults

Our example above followed a clean approach: the real bucket had no extra configuration, it was the default, the same setup from a bucket created from AWS CDK without any configuration.

What would happen if the bucket had, for example, static webhosting setup?

It will allow us to import the resource as we did in the example above. One curiosity: the defaults, from AWS CDK, won't override already set configuration.

If we deploy the stack, it won't disable static webhosting, for example. The only way to disable it (overwrite it), is to enable it through CDK, deploy, and then remove it from CDK and deploy again.

Even if you run cdk diff you won't get this as a diff as it will compare the output of synth with the template that has been uploaded to CloudFormation. Stack drift won't highlight it as well which is weird.

It is a good approach as it won't magically overwrite everything that has been working. It could be tricky though if you don't know that a resource had been imported and some defaults don't apply to it.

What would happen if the template had some setup, like static webhosting, when importing an existing resource?

It is the opposite of the situation above. The resource we create in AWS CDK to be imported has some setup and is not the default resource.

There are two cases:

Imported resource and template have the same setup (static webhosting enabled): no problem. The resource is imported and changes made in CDK will apply once you deploy.
Imported resource and template have a different setup (static webhosting enabled in template): the difference won't be applied but drift will warn you that there is a difference.

I hope it helps you, as it helped me, to visualize the process described in both blog posts mentioned in the introduction. I also went a bit further to explore drifts in the CloudFormation stack but it is a bit weird (check the examples above).

If you've further insights/questions, feel free to drop them in the comments!

Prettier, ESLint and Typescript

Vinicius Kiatkoski Neves — Mon, 20 Sep 2021 11:21:55 +0000

Cover image from peforce.com

I decided to write this article to sum up a struggle of mine. We've started a new project in the company, Prettier was set up, ESLint was set up and at some point, we added Typescript. By the end, Typescript was also set up. CI was linting, commit hooks were also linting, VSCode was fixing the code, and so on (that is what I thought).
At some point I was playing with the project and realized that some files were being warned by my editor but not when running the linter (npm run lint in my case). I got triggered. I have a hard time accepting that something works but I can't understand unless it is an external tool that I didn't have to set up myself but that was not the case here.

In this article, I will summarize some understandings that I've about integrating all the tools above. The main focus is how to set up Prettier, how to set up ESLint, how to integrate both, and by the end how to add Typescript to it.

Prettier

The first tool I want to explore is Prettier. I would leave it to you to read more about what it is but, in short, it is a code formatter. What does it mean? It means that it will keep your codebase consistent (in terms of coding style). Do you use ;? If yes, it will ensure that all your files have it, for example. I like it for two reasons: we barely have to discuss code formatting and it is easy to onboard new members to the team.

At the time of this writing, Prettier is in version 2.4.1, so keep in mind things might change (especially formatting) in future versions.

How to set up Prettier?

I will consider you have a project set up already so in short, you need to install it:

npm i prettier #--save-dev and --save-exact are recommended

Right now you can start using Prettier. You don't need any configuration (if you don't want it). You can run it against your codebase with:

npx prettier .

The . at the end means to run across your whole codebase. You can run for a specific file or pattern if you want.
This command will print the files formatted, nothing special. A more useful command happens when you add --write flag. Instead of printing the formatted code, it will write to the origin file.

Let's create a file called index.js with the following code:

// index.js
const a = 1

If we run npx prettier index.js, the output will be:

const a = 1;

It automatically adds the ; for us but it is not saved in the file. If we run npx prettier index.js --write though, the file will change and the ; will be added to it.

Cool, that is the simplest setup we can have with Prettier. The default rules are documented on their website and can be customized (a bit). We will take a look into it next but before I want to mention another flag: --check.

The --check flag, npx prettier index.js --check, is useful if you just want to check if a file (or the codebase with .) is Prettier compliant. It is useful for CIs and git hooks, for example, if you just want to warn the user (you can also enable --write in these scenarios).

If we consider the following code again:

// index.js
const a = 1

And run npx prettier index.js --check, we get the following output:

Checking formatting...
[warn] index.js
[warn] Code style issues found in the above file(s). Forgot to run Prettier?

Prettier configuration

You can configure Prettier to some extend. You can do it via the CLI or via a configuration file, which is more adequate. The configuration file could be in a variety of formats so you can choose the one that fits you best.

Add the configuration file to the root of your project (you can have configurations/folder but I would leave it up to you to explore this path) and start adding rules to it:

// .prettierrc
{
  "semi": false
}

With this configuration file and the following code, again, the --check run will succeed:

// index.js
const a = 1

npx prettier index.js --check:

Checking formatting...
All matched files use Prettier code style!

On top of that, you can also extend configuration and set up a few other things. Check their configuration documentation for more details.

ESLint

ESLint has been around for a while. In short, it does a bit more than Prettier as it analyzes your code to find problems (or patterns that you don't want, like variables that are not used should be removed). Again, I invite you to read ESLint documentation if you want to go deeper into the topic. I like ESLint for the simple reason it helps me to find problems and configure some patterns in the project (it might be useful when onboarding new people). It is extremely extensible as well in case you're interested.

At the time of this writing, ESLint is in version 7.32.0, so keep in mind things might change (especially formatting) in future versions. Version 8 is in beta at the moment.

How to set up ESLint?

In short, quite similar to Prettier but you need the configuration file. I will consider you have a project set up already so, in short, you need to install it:

npm i eslint #--save-dev is recommended

You need a configuration file. You can create one by yourself or you can run the command below that bootstraps one for you (it can add a lot of presets already):

npx eslint --init

Let's start with an empty configuration file though, it is enough to run ESLint:

// .eslintrc.js
module.exports = {
};

We can now run it, similar to Prettier:

npx eslint .

One thing to note here: ESLint runs only on .js files (by default).

Let's consider the same example as before:

// index.js
const a = 1

npx eslint index.js and we get:

1:1  error  Parsing error: The keyword 'const' is reserved
✖ 1 problem (1 error, 0 warnings)

This is simply the issue with a default ESLint configuration. It considers ES5 by default, so const is not allowed yet, and some older setup might make sense to your project but not in general.

We can spend hours configuring ESLint but in general, we get a default from a style guide (AirBnB for example) and apply it to our project. If you use the init command you can do so.

Let's install Airbnb ESLint configuration, it also requires eslint-plugin-import to be installed (following their documentation) so:

npm i eslint-config-airbnb-base eslint-plugin-import # --save-dev is recommended

Then we extend it in our configuration, so it will look like:

module.exports = {
  extends: [
    'eslint-config-airbnb-base', // or `airbnb-base`, you can omit `eslint-config-`
  ]
};

Running npx eslint index.js again we get:

1:7   error  'a' is assigned a value but never used  no-unused-vars
1:12  error  Missing semicolon                       semi

✖ 2 problems (2 errors, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

Cool! Now we get errors defined by the AirBnB guide. We can use the --fix option, which works similar to --write from Prettier, in case we want to fix the errors when possible.

ESLint allows you to extensively configure it if you want. It goes beyond the scope here and I will leave it up to you to explore and play with it: https://eslint.org/docs/user-guide/configuring/

Prettier + ESLint

There are a lot of tutorials online on how to connect both. I want to take a different approach and try to reason about each tool and how they connect.

I will assume that we have the following Prettier configuration:

// .prettierrc
{
  "semi": false
}

I will assume that we have the following ESLint configuration:

// .eslintrc.js
module.exports = {
  extends: [
    'eslint-config-airbnb-base',
  ]
};

I will assume the following script to run both tools:

// index.js
const a = 1

module.exports = { a }

If we run Prettier check, we get:

Checking formatting...
All matched files use Prettier code style!

Cool! If we run ESLint, we get:

1:12  error  Missing semicolon  semi
3:23  error  Missing semicolon  semi

✖ 2 problems (2 errors, 0 warnings)
  2 errors and 0 warnings potentially fixable with the `--fix` option.

Not so cool! Running ESLint with --fix will fix these issues. Now if we run Prettier again we get:

Checking formatting...
[warn] index.js
[warn] Code style issues found in the above file(s). Forgot to run Prettier?

If we run Prettier with --write it will fix but then ESLint will fail again. It will be like this forever. If the goal were just formatting, I would say pick one of the tools and ignore the other, but we want the power of both tools, especially as ESLint is more than just formatting your code.

Prettier provides two packages that integrate with ESLint.

eslint-config-prettier: turns off rules that might conflict with Prettier.
eslint-plugin-prettier: adds Prettier rules to ESLint.

Let's go step by step. First let's go and install eslint-config-prettier:

npm i eslint-config-prettier # --save-dev recommended

Our new .eslintrc.js will look like:

module.exports = {
  extends: [
    'eslint-config-airbnb-base',
    'eslint-config-prettier',
  ]
};

Considering the file below, again:

const a = 1

module.exports = { a }

It was a valid file for Prettier but invalid for ESLint. Using the new configuration, it becomes valid as the conflicting rule semi has been disabled.
It is fine if we want to ignore the rules from Prettier but in general, we want Prettier rules to override ESLint rules.
In case we delete the Prettier configuration file and use its defaults (which requires ;), running Prettier check will result in:

Checking formatting...
[warn] index.js
[warn] Code style issues found in the above file(s). Forgot to run Prettier?

The file is not valid anymore as it is missing the ; but the ESLint run won't fail, as the Prettier rules have been disabled when running ESLint.

One important thing to note here: the order used by extends, in the ESLint configuration, matters. If we use the following order we will get an error as AirBnB rules will override Prettier disabled rules when running ESLint:

module.exports = {
  extends: [
    'eslint-config-prettier',
    'eslint-config-airbnb-base',
  ]
};

Running npx eslint index.js:

1:12  error  Missing semicolon  semi
3:23  error  Missing semicolon  semi

✖ 2 problems (2 errors, 0 warnings)
  2 errors and 0 warnings potentially fixable with the `--fix` option.

To mitigate this issue let's install the plugin:

npm i eslint-plugin-prettier # --save-dev recommended

We can then update our .eslintrc.js file to:

module.exports = {
  extends: [
    'eslint-config-airbnb-base',
    'plugin:prettier/recommended',
  ]
};

We replaced eslint-config-prettier with plugin:prettier/recommended. Check ESLint docs about extending a plugin: https://eslint.org/docs/user-guide/configuring/configuration-files#using-a-configuration-from-a-plugin
I also recommend you to check what eslint-plugin-prettier is doing with our ESLint configuration: https://github.com/prettier/eslint-plugin-prettier/blob/a3d6a2259cbda7b2b4a843b6d641b298f03de5ad/eslint-plugin-prettier.js#L66-L75

Running ESLint again we will get:

1:12  error  Insert `;`  prettier/prettier
3:23  error  Insert `;`  prettier/prettier

✖ 2 problems (2 errors, 0 warnings)
  2 errors and 0 warnings potentially fixable with the `--fix` option.

Two things to note here:

We are getting ; errors again, that have been disabled earlier with eslint-config-prettier;
The error is coming from the rule prettier/prettier, which is added by the plugin. All prettier validations will be reported as prettier/prettier rules.

Typescript

Let's start from the very basic: running ESLint against TS files.
Right now, running ESLint against your codebase would be npx eslint .. That is fine until you want to run it against files that are not ending with .js.

Let's have these two files in our code base:

// index.js
const a = 1

// index.ts
const a = 1

Running npx eslint . we get:

1:7   error  'a' is assigned a value but never used  no-unused-vars
1:12  error  Insert `;`                              prettier/prettier

✖ 2 problems (2 errors, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

It runs against our JS file but not our TS file. To run against TS files you need to add --ext .js,.ts to the ESLint command. By default, ESLint will only check for .js files.

Running npx eslint . --ext .js,.ts

/index.js
1:7   error  'a' is assigned a value but never used  no-unused-vars
1:12  error  Insert `;`                              prettier/prettier

/index.ts
1:7   error  'a' is assigned a value but never used  no-unused-vars
1:12  error  Insert `;`                              prettier/prettier

✖ 4 problems (4 errors, 0 warnings)
  2 errors and 0 warnings potentially fixable with the `--fix` option.

Working like a charm so far. Let's add some real TS code and run it again. The TS file will look like this:

const a: number = 1

Running ESLint only against the .ts file:

1:8  error  Parsing error: Unexpected token :

✖ 1 problem (1 error, 0 warnings)

ESLint doesn't know, by default, how to parse Typescript files. It is a similar problem we faced when running ESLint for the first time with ES5 defaults.
ESLint has a configuration in which you can specify the parser you want to use. There is also a package, as you could imagine, that handles this parsing for us. It is called @typescript-eslint/parser.

Let's install it:

npm i @typescript-eslint/parser # --save-dev recommended

Now let's configure ESLint to use the new parser:

module.exports = {
  parser: "@typescript-eslint/parser",
  extends: [
    'eslint-config-airbnb-base',
    'plugin:prettier/recommended',
  ]
};

Running ESLint again (npx eslint index.ts):

1:7   error  'a' is assigned a value but never used  no-unused-vars
1:20  error  Insert `;`                              prettier/prettier

✖ 2 problems (2 errors, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

Cool! Now we can run ESLint on TS files. Nonetheless, we don't have any rules being used so we need to configure or use some styleguide, like the one we used by AirBnB before.
There is @typescript-eslint/eslint-plugin that offers us some defaults. Let's go with it for now:

npm i @typescript-eslint/eslint-plugin # --save-dev recommended

Adding it to our configuration:

module.exports = {
  parser: "@typescript-eslint/parser",
  extends: [
    'eslint-config-airbnb-base',
    'plugin:@typescript-eslint/recommended',
    'plugin:prettier/recommended',
  ]
};

Now running npx eslint index.ts:

1:7   error    Type number trivially inferred from a number literal, remove type annotation  @typescript-eslint/no-inferrable-types
1:7   warning  'a' is assigned a value but never used                                        @typescript-eslint/no-unused-vars
1:20  error    Insert `;`                                                                    prettier/prettier

✖ 3 problems (2 errors, 1 warning)
  2 errors and 0 warnings potentially fixable with the `--fix` option.

Cool! Now we have also proper linting in our Typescript file. We can also see that the Prettier rule still applies as expected.

Bear in mind that typescript-eslint is overriding eslint-config-airbnb-base in this case. It means that some rules won't work in TS files that are still valid on JS files. Let's have the files below to see it in action:

// index.js and index.ts
const a = 1;
a = 2;

Both files are identical. Running npx eslint . --ext .js,.ts we get:

/index.js
  2:1  error    'a' is constant                         no-const-assign
  2:1  warning  'a' is assigned a value but never used  @typescript-eslint/no-unused-vars

/index.ts
  2:1  warning  'a' is assigned a value but never used  @typescript-eslint/no-unused-vars

✖ 3 problems (1 error, 2 warnings)

The no-const-assign rule is overwritten by typescript-eslint for .ts files so we don't get the same error for both files.
To overcome it, we need to change the order of the extended configurations, typescript-eslint comes first and eslint-config-airbnb-base next. If we do so:

module.exports = {
  parser: "@typescript-eslint/parser",
  extends: [
    "plugin:@typescript-eslint/recommended",
    "eslint-config-airbnb-base",
    "plugin:prettier/recommended"
  ]
};

Running npx eslint . --ext .js,.ts:

/index.js
  2:1  error    'a' is constant                         no-const-assign
  2:1  error    'a' is assigned a value but never used  no-unused-vars
  2:1  warning  'a' is assigned a value but never used  @typescript-eslint/no-unused-vars

/index.ts
  2:1  error    'a' is constant                         no-const-assign
  2:1  error    'a' is assigned a value but never used  no-unused-vars
  2:1  warning  'a' is assigned a value but never used  @typescript-eslint/no-unused-vars

✖ 6 problems (4 errors, 2 warnings)

Cool! Now we get the same error for both files.

One side note: In this example, I have a codebase with JS/TS, it might not be your case and you might also use another style guide where conflicts won't happen.

That's all folks!

I hope this article helped you to learn or clarify some concepts behind ESLint, Prettier, and Typescript playing together.

In short, you've to understand which files ESLint will analyze and the order of the configurations you want. Image adding now this into a Vue project, for example, you need to add .vue to --ext .js,.ts,.vue and add (or configure) some style guide which will add some rules to your project.

Most boilerplates will already have some lint set up and you will mostly disable some rules but in case you want to customize it or update packages (especially major bumps), it is important to understand how to perform the changes and the impacts it might have in your project.

That's is all! Happy linting!

Creating a GitHub Action for your Slack release BOT

Vinicius Kiatkoski Neves — Tue, 29 Jun 2021 10:39:10 +0000

Almost all of us love Slack applications and BOTs that automate something. It could be from an alert, so you're notified as soon as a problem happens, or just a random good morning GIF. In this short tutorial, I would like to go through the process of creating a GitHub Action that posts a message as soon as a release happens (that is our use case) but the main topic is how you can automate things with GitHub Actions, how to reuse "things" across multiple actions and how to start "simple".

At Homeday we use such a process to announce new releases from our component library Homeday Blocks to our #frontend Slack channel and also to announce product releases to the relevant squad's channels. The release BOT looks like the one below:

Pre-requisite

You need to have a Slack application to perform the next steps. Slack has great documentation on how to setup a new application: https://api.slack.com/start

The main feature/functionality your application needs to have is: "Incoming Webhooks". It will allow you, later, to create a webhook that will post a message where you want. Each webhook will have an unique URL which you should keep private.

Creating a GitHub Action to trigger the webhook

GitHub Actions got quite popular in the past 1-2 years and are a good way to setup continuous integration to our applications on GitHub. It has way more features than just "run tests on every pull request" but I will leave it up to their docs to walk you through the multitude of options it offers: https://docs.github.com/en/actions

In short, a workflow has a name, a trigger, and job(s). The name helps us to easily identify what the workflow is supposed to do. The trigger defines when the workflow will run, for example, every push, every pull request, every comment, every release... The jobs are a sequence of steps that the workflow will perform, these can be GitHub Actions. Again, GitHub docs will do a better job than me explaining each of these blocks.

I would like to start creating a generic workflow that will trigger for every push:

# Action name
name: CD

# Action triggers
on:
  push:
    branches:
      - main

 # Action jobs
 jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2.3.4
      - run: npm ci
      - run: npm run build:app
      - run: npm run deploy

The workflow above is an example of how you can build and deploy your application for every push on main. Now let's enhance it to notify a channel, on Slack, every time the application is deployed.

# Action name
name: CD

# Action triggers
on:
  push:
    branches:
      - main

 # Action jobs
 jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2.3.4
      - run: npm ci
      - run: npm run build:app
      - run: npm run deploy
  slack:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2.3.4
      - name: Get latest commit info
        run: |
          echo "::set-output name=TITLE::$(git show -1 --format='%s' -s)"
        id: latest_commit
      - name: Announce on Slack 📢
        run: |
          curl ${{ secrets.SLACK_RELEASE_BOT_WEBHOOK_URL }} \
            --request POST \
            --header 'Content-type: application/json' \
            --data \
              '{
                "blocks": [
                  {
                    "type": "header",
                    "text": {
                      "type": "plain_text",
                      "text": "${{ steps.latest_commit.outputs.TITLE }}"
                    }
                  }
                ]
              }'

It adds a job by the end, called slack. This jobs needs the deploy job to run first so we only announce the release after the deployment. It gets the latest commit message to be used as release title (this is completely open to customization) and afterward, it sends a POST request to our webhook.

Two things are important to mention here:

SLACK_RELEASE_BOT_WEBHOOK_URL is defined as a secret in the repository
We reference the output of one step in another step with the help of set-output and id

Now, after deployment, we will have a message with the commit message in Slack. Quite similar to the intro image, the main difference is that there I've added some other blocks into the message.

## How to reuse it?

It works fine for the application we setup but now we need to add it to another application. The easiest way to do it is to copy-paste. It works fine until it doesn't 😅

If we want to keep consistency or help others to setup their workflows, copying a "random" curl might not be the best experience and can lead to a lot of errors, we can create our own GitHub Action. Clear advantages are easy to reuse and, most important, documentation.

Again, GitHub Actions docs do a great job here but it might lead you to unanswered questions, like: which kind of action should I create? Docker? Javascript? Composite?

The simplest, for this scenario, way to share the curl step through different workflows is creating a composite action. Composite actions allow you to group a sequence of steps without much extra setup. All you need is one file by the end.

There is, of course, one limitation. Composite actions won't allow you to reuse other actions inside of it. In short: you can't use the uses key in your steps.

You need to create a new repository and add a file called action.yml. This file will have the following template:

name: Slack Release BOT
description: Slack Release BOT used by us
inputs:
  webhook_url:
    required: true
    description: Slack webhook URL
  title:
    required: true
    description: Message title
runs:
  using: 'composite'
  steps:
    - shell: bash
      run: |
        curl ${{ inputs.webhook_url }} \
          --request POST \
          --header 'Content-type: application/json' \
          --data \
            '{
              "blocks": [
                {
                  "type": "header",
                  "text": {
                    "type": "plain_text",
                    "text": "${{ inputs.title }}"
                  }
                }
              ]
            }'

The inputs allow you to pass data from your main workflow to this action through with, the full example will show it. We encapsulated the curl into its own action, allowing all applications to use it once we push the code to our repository.

The example below shows how the main workflow will look like given that we have the above action under github-username/my-action:

# Action name
name: CD

# Action triggers
on:
  push:
    branches:
      - main

 # Action jobs
 jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2.3.4
      - run: npm ci
      - run: npm run build:app
      - run: npm run deploy
      slack:
        needs: deploy
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v2.3.4
          - name: Get latest commit info
            run: |
              echo "::set-output name=TITLE::$(git show -1 --format='%s' -s)"
            id: latest_commit
          - uses: github-username/my-action@main
            with:
              webhook_url: ${{ secrets.SLACK_RELEASE_BOT_WEBHOOK_URL }}
              title: ${{ steps.latest_commit.outputs.TITLE }}

Instead of performing the curl ourselves, we just use the new action, and using with we can pass the required variables.

Now we have a centralized repository where we can standardize the release, make its usage easier, and, on top of that, document what is going on, which would be hard if you just copy a curl from another workflow.

## How about the next steps?

There are other ways to create an action: docker and javascript. They are indeed more advanced and give you more flexibility, not something required by our example at the moment. In case we need to start adding conditions, fetch different, data and not so "step by step" approach, it makes sense to go for a Docker/Javascript action as it will become easier to manipulate data there.

The idea here is to start simple and develop further only if needed. You can group a sequence of steps using a composite action and reuse them across all your applications. It is also important to follow a release process and avoid that applications crash in case something gets updated.

If you would like to check the actual final code from our Slack release BOT Action, you can check it in this repository: https://github.com/homeday-de/slack-release-bot-action

Feel free to use it if it helps you. You can also check this pull request where we replace a curl step with the new action for every release of our component library: https://github.com/homeday-de/homeday-blocks/pull/744

Vue custom input

Vinicius Kiatkoski Neves — Tue, 09 Mar 2021 19:44:56 +0000

Most of us have faced it: build a custom input component. There are multiple reasons behind it but in general, it has custom styles and we should be able to reuse it.

Although it might sound simple it has some gotchas and from time to time we end up going through the documentation to check some details. It gets a bit more complicated if you're not that familiar with few Vue concepts.

Last month, February 2021, it happened again. When possible I try to help people in a Vue Slack group and this question popped up once again. Not exactly this question but the user had issues building a custom input component. The problem was related to some concepts.

To consolidate this knowledge for myself, and use it as some sort of documentation for others, I decided to wrap up the process of writing a custom input.

Table of contents

v-model and <input>
The wrong custom input component
The happy custom input component
- Adding validation (or operation on data change)
- Combining computed and v-model
Extra: the model property
So what?

v-model and `<input>`

Once we start building forms with Vue we learn the directive v-model. It does a lot of the hard work for us: it binds a value to an input. It means that whenever we change the input's value the variable will also be updated.

The official docs do a great job explaining how it works: https://vuejs.org/v2/guide/forms.html

In short we can have the following template and we're fine:

<!-- UsernameInput.vue -->
<template>
  <label>
    Username
    <input type="text" name="username" v-model="username">
  </label>
</template>

<script>
export default {
  name: 'UsernameInput',
  data() {
    return {
      username: 'Initial value',
    };
  },
}
</script>

We'll have an input that has Initial value as the initial value and the username data will be automatically updated once we change the input's value.

The problem with the above component is that we can't reuse it. Imagine we've a page where we need the username and the e-mail, the above component won't handle the e-mail case as the data is inside the component itself, not somewhere else (like the parent component, for example). That is where custom input components shine and also one of its challenges: keep the v-model behavior consistent.

The wrong custom input component

Well, why am I showing this example? The answer is: this is the first approach most of us will try.

Let's see how we are going to use our custom input component:

<!-- App.vue -->
<template>
  <custom-input :label="label" v-model="model" />
</template>

<script>
import CustomInput from './components/CustomInput.ue';

export default {
  name: 'App',
  components: { CustomInput },
  data() {
    return {
      label: 'Username',
      model: '',
    };
  },
}
</script>

The custom input expects a label and a v-model in this case and will look like the component below:

<!-- CustomInput.vue -->
<template>
  <label>
    {{ label }}
    <input type="text" :name="name" v-model="value" />
  </label>
</template>

<script>
export default {
  name: 'CustomInput',
  props: {
    label: {
      type: String,
      required: true,
    },
    value: {
      type: String,
      required: true,
    },
  },
  computed: {
    name() {
      return this.label.toLowerCase();
    },
  },
}
</script>

First, it expects the label as property and computes the name on top of that (it could also be a property). Second, it expects a value property and binds it to the <input> through v-model. The reason behind that can be found in the docs but in short, when we use v-model in a custom component it will get value as a property which is the value from the v-model variable used. In our example, it'll be the value from model defined in App.vue.

If we try the code above, it'll work as expected, but why is it wrong? If we open the console we will see something like this:

[Vue warn]: Avoid mutating a prop directly since the value will be overwritten whenever the parent component re-renders. Instead, use a data or computed property based on the prop's value. Prop being mutated: "value"

It complains that we're mutating a property. The way Vue works is: the child component has props that come from the parent component and the child component emits changes to the parent component. Using v-model with the value prop that we got from the parent component violates it.

Another way to see this issue is rewriting the App.vue like this:

<!-- App.vue -->
<template>
  <custom-input :label="label" :value="model" />
</template>

...

The main difference is using :value instead of v-model. In this case, we're just passing model to the value property. The example still works and we get the same message in the console.

The next step is to rework the example above and make sure it works as expected.

The happy custom input component

The happy custom input component doesn't mutate its prop but emits the changes to the parent component.

The docs have this exact example but we'll go a bit further here. If we follow the docs, our CustomInput should look like the one below:

<!-- CustomInput.vue -->
<template>
  <label>
    {{ label }}
    <input type="text" :name="name" :value="value" @input="$emit('input', $event.target.value)" />
  </label>
</template>

<script>
export default {
  name: 'CustomInput',
  props: {
    label: {
      type: String,
      required: true,
    },
    value: {
      type: String,
      required: true,
    },
  },
  computed: {
    name() {
      return this.label.toLowerCase();
    },
  },
}
</script>

This is enough to make it work. We can even test it against both App.vue, the one using v-model, where everything works as expected, and the one using :value only, where it doesn't work anymore as we stopped mutating the property.

Adding validation (or operation on data change)

In case we need to do something when the data changes, for example checking if it is empty and show an error message, we need to extract the emit. We'll have the following changes to our component:

<!-- CustomInput.vue -->
<template>
...
    <input type="text" :name="name" :value="value" @input="onInput" />
...
</template>

<script>
...
  methods: {
    onInput(event) {
      this.$emit('input', event.target.value);
    }
  }
...
</script>

Now we add the empty check:

<!-- CustomInput.vue -->
<template>
...
    <p v-if="error">{{ error }}</p>
...
</template>

<script>
...
  data() {
    return {
      error: '',
    };
  },
...
    onInput(event) {
      const value = event.target.value;

      if (!value) {
        this.error = 'Value should not be empty';
      }

      this.$emit('input', event.target.value)
    }
...
</script>

It kind of works, first it doesn't show any errors and if we type then delete it'll show the error message. The problem is that the error message never disappears. To fix that we need to add a watcher to the value property and clean the error message whenever it is updated.

<!-- CustomInput.vue -->
...
<script>
...
  watch: {
    value: {
      handler(value) {
        if (value) {
          this.error = '';
        }
      },
    },
  },
...
</script>

We could achieve a similar result adding an else inside onInput. Using the watcher enables us to validate before the user updates the input value, if desirable.

If we add more things we most probably will be expanding this component even more and things are spread all over the <script> block. To group things a bit we can try a different approach: use computed together with v-model.

Combining computed and `v-model`

Instead of listening to the input event and then emitting it again, we can leverage the power of v-model and computed. It is the closest we can get to the wrong approach but still make it right 😅
Let's rewrite our component like that:

<!-- CustomInput.vue -->
<template>
...
    <input type="text" :name="name" v-model="model" />
...
</template>

<script>
...
  computed: {
    ...
    model: {
      get() {
        return this.value;
      },
      set(value) {
        this.$emit('input', value);
      },
    },
  },
...
</script>

We can get rid of the onInput method and also from the watcher as we can handle everything within get/set functions from the computed property.

One cool thing we can achieve with that is the usage of modifiers, like .trim/number that would need to be manually written before.

This is a good approach for simple input components. Things can get a bit more complex and this approach doesn't fulfill all the use cases, if that is the case we need to go for binding value and listening to events. A good example is if you want to support the .lazy modifier in the parent component, you will need to manually listen to input and change.

Extra: the `model` property

The model property allows you to customize a bit the v-model behavior. You can specify which property will be mapped, the default is value, and which event will be emitted, the default is input or change when .lazy is used.

This is especially useful if you want to use the value prop for something else, as it might make more sense for a specific context, or just want to make things more explicit and rename value to model, for example. In most cases, we might use it to customize checkboxes/radios when getting objects as input.

So what?

My take comes from how complex your custom input needs to be:

It was created to centralized the styles in one component and its API is pretty much on top of Vue's API: computed + v-model. It falls pretty much on our example, it has simple props and no complex validation.

<!-- CustomInput.vue -->
<template>
  <label>
    {{ label }}
    <input type="text" :name="name" v-model="model" />
  </label>
</template>

<script>
export default {
  name: 'CustomInput',
  props: {
    label: {
      type: String,
      required: true,
    },
    value: {
      type: String,
      required: true,
    },
  },
  computed: {
    name() {
      return this.label.toLowerCase();
    },
    model: {
      get() {
        return this.value;
      },
      set(value) {
        this.$emit('input', value);
      },
    },
  },
}
</script>

Everything else (which means that you need to tweak a lot the previous setup to support what you need): listeners, watchers and whatelse you might need. It might have multiple states (think of async validation where a loading state might be useful) or you want to support .lazy modifier from the parent component, are good examples to avoid the first approach.

<!-- CustomInput.vue -->
<template>
  <label>
    {{ label }}
    <input type="text" :name="name" :value="value" @input="onInput" @change="onChange" />
  </label>
</template>

<script>
export default {
  name: 'CustomInput',
  props: {
    label: {
      type: String,
      required: true,
    },
    value: {
      type: String,
      required: true,
    },
  },
  /* Can add validation here
  watch: {
    value: {
      handler(newValue, oldValue) {

      },
    },
  }, */
  computed: {
    name() {
      return this.label.toLowerCase();
    },
  },
  methods: {
    onInput(event) {
      // Can add validation here
      this.$emit('input', event.target.value);
    },
    onChange(event) { // Supports .lazy
      // Can add validation here
      this.$emit('change', event.target.value);
    },
  },
}
</script>

Thanks @danilowoz for reviewing it

Multiple builds targeting different browsers

Vinicius Kiatkoski Neves — Mon, 24 Aug 2020 13:07:41 +0000

Every quarter at Homeday we have something called PEW. PEW stands for Product Engineering Week, which translates to a week where you cancel all your meetings and work on a topic you would like to explore. It can be done in groups or alone, it is up to you. Last PEWs I've done work regarding tests, assets compression and some Puppeteer as a service. This quarter I decided to go for build optimization and I would like to explore the topic in this article.

My idea

How about building different versions of our applications targeting different browsers so each browser downloads only the "fallbacks" it needs?

That is in short what I thought. Now I'm going to explore how is our current development lifecycle and where we can try to add this idea.

Development lifecycle

At Homeday we build mainly SPAs using Vue. So by the end of our development lifecycle we create a bunch of assets that are uploaded to S3 (in our case) and work as an application.

To "create a bunch of assets" we use Webpack which builds our code, creating one version of it at the end. This version is used by all our clients, which means that lots of different browsers will be using this very same version. Below you can visualize the current build process, from code to assets.

By "different browsers will be using this very same version" I mean that we have to be ready for some old browsers (some applications still need to support IE 11 which has a significant market share for us). So basically our version should support IE 11 and also the latest version on Chrome, for example. IE 11 doesn't have the same Javascript/CSS support as the latest Chrome, so at the end our code fallback to something that works on IE 11, adding polyfills and transpiling what is needed. This adds some extra Kb to our assets which the latest Chrome users don't need but they end up downloading.

The theory is exactly that one. The thing I needed to check now if how feasible it would be to ship different code for both browsers or how many browsers we would like to split it.

Targeting browsers when building

When we build our application using Webpack there are different loaders that ensure our code becomes a single (or multiple) JS/CSS file at the end. Well known loaders like babel-loader and postcss-loader ensure that our code works cross browser. The open question is: How do they know which browsers they have to fallback to? They can have their own defaults but there should be, somehow, a way to specify which browsers it should take into consideration.

There is one file called .browserslistrc (or an entry in package.json) that specify the browsers you expect your project to support. This file has a well defined structure and has its own project: browserslist. Loaders, like babel-loader and postcss-loader, use the browsers you specify in your .browserslistrc file to know which browsers they have to fallback to.

You can define not just one browser but a range of browsers with browserslist, I recommend you to check the project if you're not aware how to define those queries.

Now that we can specify the browsers we want to support, we need to check the browsers' distribution among our projects and check the savings we could have when targeting them in the build process. The browsers' distribution comes from Google Analytics in our case. I did this process for 3 of our projects and summarized it below:

Project 1:
- Current build (which supports IE 11 but doesn't need it): 273Kb
- Chrome 84: 241Kb (12% - 32Kb)
- Safari 13: 250Kb (9% - 23Kb)
Project 2:
- Current build (which suuports IE 11 and it is necessary): 302Kb
- Chrome 84: 269Kb (11% - 33Kb)
- Safari 13: 277Kb (8% - 25Kb)
Project 3:
- Current build (which supports IE 11 and it is necessary): 544Kb
- Chrome 83+: 504Kb (8% - 40Kb)
- Safari 13: 516Kb (5% - 28Kb)

All values are GZIP and counts for all JS + CSS files generated in the build

All in all modern browsers can save between ~20Kb - ~40Kb which is definitely a good number (it is not as good as the results I've got from Brotli compression during another PEW work but it is definitely something that we can work on).

Now that the idea is valid, it is time to implement it. The first step is to do multiple builds of our projects.

Multiple builds

Using browserslist we can specify different environments which allows us to set an environment variable (BROWSERSLIST_ENV) to select which environment we want to build to.

Now you can read the .browserslistrc file and select which environments are set and build based on them. We created a small script that does this job. You can setup it differently, what matters at the end is the possibility of running one command and building all the different versions.

const { readConfig } = require('browserslist/node');

const browserslistConfig = readConfig('.browserslistrc');
const browserslistConfigKeys = Object.keys(browserslistConfig).filter((_) => _ !== 'defaults'); // Browserslist default is removed and built separately

browserslistConfigKeys.forEach((key) => {
  // Here we build the app like: BROWSERSLIST_ENV=${key} npm run build:production
});

I've removed parts of the code that are not necessary for the example.

So right now what happens is the following:

We have a .browserslistrc file with environments set

since 2019

[chrome]
chrome 84

[safari]
safari 13

We build for each environment
1. Load first envionment, which is chrome in this case.
2. Call BROWSERSLIST_ENV=chrome npm run build:production
3. The entry point will be in /dist/chrome, so we will have /dist/chrome/index.html and /dist/js/...
4. Load second envionment, which is safari in this case.
5. Call BROWSERSLIST_ENV=safari npm run build:production
6. The entry point will be in /dist/safari, so we will have /dist/safari/index.html and /dist/js/...
7. Build default case
8. Call npm run build:production
9. The entry point will be in /dist, so we will have /dist/index.html and /dis/js/...

What we can note from here is that we still have the default /dist/index.html working as expected and all the assets are in the shared folders, /dist/js for example. The image below summarizes this process.

Let's check where we are going. We have multiple index.html files now. Each index.html points to a different entry point, a .js file in this case. This .js file is located in /dist/js. So what we need to do now is to route the browser to the specific index.html that uses the built version of our app for that browser.

Routing multiple builds

Once we're done with multiple builds of our application we can simply deploy it. Deploy means copying the files under /dist to somewhere, which is S3 in our case. What happens now is that our application works exactly as before. The reason behind it is that our default build creates /dist/index.html which is exactly how we were building our project.

What we need to do now is to route some of the requests to the new index.html files, under /chrome and /safari sub-directories. We need to route only index.html as all the assets are living in the same sub-directories (/js and /css) as before.

At Homeday we have CloudFront in front of S3 which means we can levarage the powers of Lambda@Edge. Lambda@Edge allows you to run a Lambda function (if you're not familiar, please check the official docs within CloudFront lifecycle events. You can also check the Lambda@Edge official docs if you want to go deeper in the topic.

We can place a Lambda function between CloudFront and S3, which allows us to route the request to S3 based on the User-Agent that we get from the request. We can compare the User-Agent with our queries in the browserslist definition and decide which route to take or just go to the default one (which would be the case without this Lambda function). This process should happen only for index.html and service-workers.js as we have a PWA here. The Lambda function can look like the following:

const { matchesUA } = require('browserslist-useragent');
const { readConfig } = require('browserslist/node');

const INDEX_HTML_REGEX = /\/index\.html/;
const SERVICE_WORKER_REGEX = /\/service-worker\.js/;
const BROWSERSLIST_CONFIG = readConfig('.browserslistrc');
const BROWSERSLIST_KEYS = Object.keys(BROWSERSLIST_CONFIG).filter((_) => _ !== 'defaults');

exports.handler = async (event) => {
  const { request } = event.Records[0].cf;
  const { uri, headers } = request;

  if (INDEX_HTML_REGEX.test(uri) || SERVICE_WORKER_REGEX.test(uri)) { // You can do it in the same Regex or leave it explicit as we do
    const userAgent = getUserAgentFromHeaders(headers);

    console.log('uri', uri);
    console.log('userAgent', userAgent);

    const path = BROWSERSLIST_KEYS.find((key) => {
      const browsers = BROWSERSLIST_CONFIG[key];
      const options = {
        browsers,
        allowHigherVersions: true,
      };

      return matchesUA(userAgent, options);
    });

    if (path) {
      console.log(`Redirect to ${path} version`);

      return { ...request, uri: `/${path}${uri}` };
    }

    console.log('Serving default version');
  }

  return request;
};

Once the user downloads the "right" index.html, it will fetch the required assets and serve the right version of the application for that user. There are 3 images below that represent the request scenarios. Consider that none of the files are cached in CloudFront/Browser.

Requesting index.html from a random browser that is not Chrome/Safari, which means we fallback to default (or what we had before). Lambda function doesn't do any routing job now and just forwards the request.

Requesting index.html from a Chrome browser, which means we should route to /chrome/index.html. Lambda function detects the User-Agent and routes the request to the right file, in this case /chrome/index.html.

Requesting app.1.js from a Chrome browser. As it is not index.html we shouldn't do anything. Lambda function doesn't do any routing job now and just forwards the request.

Things to consider

All in all the described approach works as expected. Nevertheless there are other things I would recommend doing as well:

Do not build for specific browsers. Build for a range of browsers. If you build for Chrome 83 and Chrome 84, for example, the changes of having the same output are quite high. Play with Browserslist queries and find the one that fits you best, also take a look into your analytics to understand the best approach to take.
Your build time is going to increase. You can build in parallel as well but at the end it is going to increase. So leverage the amount of builds you would like to have.
If you use CDN, as we use CloudFront, forwarding a header will imply on "loosening" your caching strategy, so have it in mind and do not forward all the headers. In this case we need just User-Agent.
Automate and make use of IaC (Infrastructure as Code). As we've everything in AWS, I end up using CloudFormation. Actually I used AWS SAM, as it makes it easier to define Lambda functions, but at the end I still need to use CloudFormation syntax for CloudFront distribution for example.
- This step can be done completely in a next iteration but I definitely recommend you to check it out. Imagine you update your .browserslistrc file. You need to deploy your Lambda again. Publish it. Update CloudFront distribution to use it. And whatever comes after. If everything is automated, at the end you run a command that does all those steps for you.
- If you use CloudFront as well and you're outside us-east-1 you will have few issues with Lambda@Edge as this function should be in us-east-1 and not in any other region to work.
Another way of achieving something similar is using bdsl-webpack-plugin. This approach has some drawbacks and it becomes cumbersome when using Service Workers. Nevertheless it is a way easier to be implemented.

Thank you for coming this far =]

I hope you enjoyed this article as much as I enjoyed exploring this topic. The benefits of such approach are quite clear but the implementation is a bit overwhelming.

We, Homeday, are currently not using it in production and I'm keen to give it a try and collect some metrics. I love to explore this kind of topic and work beyond the code, exploring architecture improvements and so on. I hope in the next PEWs I can explore similar topics and share our learnings as well!

Webpack concepts: Loaders and Plugins

Vinicius Kiatkoski Neves — Wed, 29 Jul 2020 17:56:20 +0000

This is the first time I write a series of articles. I decided to structure it like that as I don't have the full picture of what I'm going to be writing, I've just the first part of it written, and I also want to avoid a 10min reading article at the end.

I've been doing Vue, professionally, for a while now. As usual, whenever I create/setup a new project, I use Vue CLI which abstracts a lot of the heavy weight of creating a new project, which is great, especially if you've ever tried to setup a full Webpack config by yourself. The same applies if you use other CLIs to generate your project.
Nonetheless, I want to walk through the process of setuping a Webpack config by myself and highlight some benefits and drawbacks of using Vue CLI on the way. I will try to mimic as much as possible the behavior of Vue CLI but avoid the usage of its plugins for example.

Why I want to do that? Well, as sort of documentation/study for myself (I can't remember the last time I went through each Webpack loader and customized it, which is great, don't get me wrong) and also, maybe, to help people to understand the full process and challenges that comes with such setup (so you might get some feeling about why CLIs do a great job for us).

The content will be split between Webpack, Vue and enhancements. First I will go through Webpack and its concepts, focusing more on examples than in the theory (Webpack docs are great if we need to deep dive into some concepts). After that I want to be able to have a bare minimum Webpack config to build a Vue project. At the end I will explore some "enhancements", that is how I decided to call it, to my Webpack config. By enhancements I mean everything that "enhances" our developer experience or add some flavor to our build (file compressing, for example, or splitting...).

Webpack

Most probably you've heard about Webpack before, if not, check their official website which has tons of documentation about it. I think this image, from the offical website, summarizes quite well what Webpack does for us:

Once you start reading the documentation you will find out that the first two dependencies you've to install in order to use Webpack are webpack and webpack-cli. From here we can start customizing our Webpack config which tells Webpack how it should process our files.

The Webpack config starts with the entry property, which specifies the entry file to be processed with Webpack. Let's create the following webpack.config.js file, which tells Webpack that the entry point is ./src/main.js:



module.exports = {
  entry: './src/main.js',
};

If we add any Javascript code to main.js it will get processed by Webpack and the output will be placed inside dist/ folder (by default). To run Webpack we just need to npx webpack --config webpack.config.js.

It works (if we have a main.js file created) but it complains (warns) that the mode property is not set. From Webpack docs, the mode property:

Providing the mode configuration option tells webpack to use its built-in optimizations accordingly.

The mode property can be production, development or none. To set the mode property we can just pass --mode=MODE_WE_WANT to our npx webpack call. To make things more structured we can add two commands to our NPM scripts, one to build for production and another one for development:



"scripts": {
  "build:production": "webpack --config webpack.config.js --mode=production",
  "build:development": "webpack --config webpack.config.js --mode=development"
}

Ok, now we can add Javascript code to main.js and build it. Not that useful, but it is the basic setup and we made it work.

Loaders

Loaders are the magic behind Webpack. They allow us to process different file types in different ways and the outcome will be a single (or multiple) bundled file. From the docs, loaders are defined as:

webpack enables use of loaders to preprocess files. This allows you to bundle any static resource way beyond JavaScript.

Once we are developing for the web, one main concern that we usually have is: it should work cross browser. Which means, it should fallback features that are not present in old browsers. A project that does this kind of job is Babel and we can, via loader, use it with Webpack. We can tell Webpack to process Javascript files with Babel, which will do the job of compiling our code.

To use Babel with Webpack we need to setup its loader. From the Babel loader docs we endup with the following Webpack config:



module.exports = {
  entry: './src/main.js',
  module: {
    rules: [{
      test: /\.js$/,
      exclude: /node_modules/,
      use: {
        loader: 'babel-loader',
      },
    }],
  },
};

Let's take a step back first. If the main.js file contained something that doesn't need to be compiled, it won't make a huge difference here. For example, if it contains console.log('Hello World!'), the output will be the same with or without the loader. It gets more interesting once we started adding features that are not supported in all browsers.

First lets setup Babel configuration file, also known as babelrc, which allows us to use special syntax if we want or use the well known @babel/preset-env which:

is a smart preset that allows you to use the latest JavaScript without needing to micromanage which syntax transforms (and optionally, browser polyfills) are needed by your target environment(s). This both makes your life easier and JavaScript bundles smaller!

The setup needed is the following:



{
  "presets": ["@babel/preset-env"]
}

Now our project is ready to compile "new" Javascript syntax. Let's take the following code as example:



async function hey() {
  await Promise.resolve();

  return 'Hey!';
}

hey();

It uses a new syntax that won't work in old browsers. If we build our file without the Babel config, the output will be something like the following:



eval("async function hey() {\n  await Promise.resolve();\n  return 'Hey!';\n}\n\nhey();\n\n//# sourceURL=webpack:///./src/main.js?");

If we setup the Babel config file, the output will be like:



eval("function asyncGeneratorStep(gen, resolve, reject, _next, _throw, key, arg) { try { var info = gen[key](arg); var value = info.value; } catch (error) { reject(error); return; } if (info.done) { resolve(value); } else { Promise.resolve(value).then(_next, _throw); } }\n\nfunction _asyncToGenerator(fn) { return function () { var self = this, args = arguments; return new Promise(function (resolve, reject) { var gen = fn.apply(self, args); function _next(value) { asyncGeneratorStep(gen, resolve, reject, _next, _throw, \"next\", value); } function _throw(err) { asyncGeneratorStep(gen, resolve, reject, _next, _throw, \"throw\", err); } _next(undefined); }); }; }\n\nfunction hey() {\n  return _hey.apply(this, arguments);\n}\n\nfunction _hey() {\n  _hey = _asyncToGenerator( /*#__PURE__*/regeneratorRuntime.mark(function _callee() {\n    return regeneratorRuntime.wrap(function _callee$(_context) {\n      while (1) {\n        switch (_context.prev = _context.next) {\n          case 0:\n            _context.next = 2;\n            return Promise.resolve();\n\n          case 2:\n            return _context.abrupt(\"return\", 'Hey!');\n\n          case 3:\n          case \"end\":\n            return _context.stop();\n        }\n      }\n    }, _callee);\n  }));\n  return _hey.apply(this, arguments);\n}\n\nhey();\n\n//# sourceURL=webpack:///./src/main.js?");

Ok, let's break it into smaller pieces first. The first code is basically a copy and paste from the code we wrote. The second one has some magic so we don't see the usage of async/await for example. @babel/preset-env is doing its job here, we didn't need to tell it that we would be using a specific feature. Now the question that we might be asking is: what happens if I want to support only browsers that support async/await?

The answer is: Browserslist. You might have seen a file called .browserslistrc in the root of a project or inside package.json. This file is responsible for specifying which browsers you want to support and is used by multiple tools to bundle your code. Babel is one of them. If you run npx browserslist in your project you will see a list of browsers it should support. By default, at the time of this writing, the list is the following (which is the defaults setting from browserslist):



and_chr 81
and_ff 68
and_qq 10.4
and_uc 12.12
android 81
baidu 7.12
chrome 83
chrome 81
chrome 80
edge 83
edge 81
edge 18
firefox 78
firefox 77
firefox 76
firefox 68
ie 11
ios_saf 13.4-13.5
ios_saf 13.3
ios_saf 12.2-12.4
kaios 2.5
op_mini all
op_mob 46
opera 69
opera 68
safari 13.1
safari 13
samsung 12.0
samsung 11.1-11.2

What we can do here is to setup our own .browserslistrc file with modern browsers, for example last 2 Chrome versions, and run our build again. The output will be the following:



eval("async function hey() {\n  await Promise.resolve();\n  return 'Hey!';\n}\n\nhey();\n\n//# sourceURL=webpack:///./src/main.js?");

As the last 2 versions of Chrome support async/await, Babel won't transpile it so we will endup with "modern" code. As a side note, if you're not 100% sure of which browsers you need to support and so on, leave the default values and don't mess with your bundle.

Things are getting more interesting now. We are already using the Webpack magic to process our files. Now we are going to explore another Webpack concept, called plugins which will add more power to our build.

Plugins

Plugins do what loaders can't do. Ok, not that easy to understand but they hook into Webpack's lifecycle and do something. Webpack itself has a broad list of plugins that you can choose and each of them add a specific "power" to our build.

Let's take the example from Vue CLI, the environment variables. You can define some environment variables, they should begin with VUE_APP, and they will be available inside your application.

If we want to do something similar, let's say we want to define BASE_URL as an environment variable, we need to use a plugin for that. Webpack provides a plugin called DefinePlugin which allows you to create global constants which can be configured at compile time..

The Webpack config will look like:



plugins: [
  new webpack.DefinePlugin({
    'process.env': {
      BASE_URL: JSON.stringify('localhost'),
    },
  }),
],

Now we can use process.env.BASE_URL inside our code and it will be replaced by 'localhost'. Vue CLI does an extra job and allows you to define .env files which is out of scope for now.

Final config

Our final config, after all the setup above, will look like the following:



const webpack = require('webpack');

module.exports = {
  entry: './src/main.js',
  module: {
    rules: [{
      test: /\.js$/,
      exclude: /node_modules/,
      use: {
        loader: 'babel-loader',
      },
    }],
  },
  plugins: [
    new webpack.DefinePlugin({
      'process.env': {
        BASE_URL: JSON.stringify('localhost'),
      },
    }),
  ],
};

So far we will process the file src/main.js, it will use the DefinePlugin to replace process.env.BASE_URL by 'localhost' and it will be processed by Babel which will take into account our .browserslistrc definition to add (or not) missing features.

That is the basic of Webpack setup. You can imagine the heavy lift CLIs do and I hope we can understand a bit better how things work under the hood.

Go async in Vue 3 with Suspense

Vinicius Kiatkoski Neves — Mon, 22 Jun 2020 07:54:48 +0000

Vue 3 is coming with a some exciting new features. Composition API is the hottest one at the moment but there are others that excite me as much as it.

One of those new features is called Suspense and it really excites me about the benefits it brings. You might have heard about it already but I will try to show some examples of the usage of Suspense and where it can be beneficial.

What is Suspense?

Suspense is a state of mental uncertainty, anxiety, of being undecided, or of being doubtful. In a dramatic work, suspense is the anticipation of the outcome of a plot or of the solution to an uncertainty, puzzle, or mystery, particularly as it affects a character for whom one has sympathy. - Wikipedia

Back to Vue, Suspense is a component, that you don't need to import or do any kind of setup, with two <slot> that allows you to render a #fallback while the main component you want to load is not ready.

Ok, it seems vague. I will try to give you an example of how it is used. I also recommend you to take a look into its test cases, especially the first one to get familiar with it.

<Suspense>
  <template #default>
    <!-- Here the component I want to render -->
  </template>
  <template #fallback>
    <!-- Here a fallback component to be shown while my component is not ready -->
  </template>
</Suspense>

That is the basic blueprint of it and it tackles a really common use case: the v-if loading condition.

I consider it the first benefit of Suspense, as now we've some standard way of dealing with this scenario. Before Suspense each developer could choose the way they want to implement it, they still can, and it was kind of a nightmare in situations where multiple components were loaded, so you would have loadingHeader, loadingFooter, loadingMain, and so on.

At the beginning I wrote "while the main component you want to load is not ready", what it means is that the main component has some kind of async work, which plays nicely with an async setup() from the new Composition API.

Let's say we have the following component with some async work to be done in setup:

<template>
  <h1>I've some async work to do before I can render</h1>
</template>

<script>
export default {
  name: 'MyAsyncComponent',
  async setup() {
    await someAsyncWork();
  }
 }
 </script>

Now we want to use this component somewhere but we want to have a proper loading while it is not ready.

Suspense makes it more intuitive how it works and it really helps readability, check it:

 <template>
  <Suspense>
    <template #default>
      <MyAsyncComponent />
    </template>
    <template #fallback>
      <span>Loading... Please wait.</span>
    </template>
  </Suspense>
</template>

<script>
import MyAsyncComponent from '@/components/MyAsyncComponent.vue';

export default {
  name: 'App',
  components: { MyAsyncComponent }
}
</script>

Another cool thing about it is that you can have multiple Suspense components defined and have different fallbacks for each of them.

How do I handle errors?

Imagine the following: the call to someAsyncWork threw an exception. How do we handle it with Suspense?

We can use the errorCapture hook to listen to errors and conditionally render our Suspense. The component will look like the following:

<template>
  // Here we conditionally render based on error
  <h1 v-if="error">I failed to load</h1>
  <Suspense v-else>
    <template #default>
      <MyAsyncComponent />
    </template>
    <template #fallback>
      <span>Loading... Please wait.</span>
    </template>
  </Suspense>
</template>

<script>
import { ref, onErrorCaptured } from 'vue'
import MyAsyncComponent from '@/components/MyAsyncComponent.vue';

export default {
  name: 'App',
  components: { MyAsyncComponent },
  setup() {
    const error = ref(null);

    onErrorCaptured((e) => {
      error.value = e

      return true;
    });

    return { error };
  }
}
</script>

To be honest it is quite a boilerplate if you're doing it in multiple places and can be a bit cumbersome if you've multiple Suspenses.

I do encourage you to wrap this logic, and even improve it to your use case, in a new component. The following example shows a simple wrapper on top of it:

<template>
  <slot v-if="error" name="error"></slot>
  <Suspense v-else>
    <template #default>
      <slot name="default"></slot>
    </template>
    <template #fallback>
      <slot name="fallback"></slot>
    </template>
  </Suspense>
</template>

<script>
import { ref, onErrorCaptured } from 'vue'

export default {
  name: 'SuspenseWithError',
  setup() {
    const error = ref(null);

    onErrorCaptured((e) => {
      error.value = e

      return true;
    });

    return { error };
  }
}
</script>

So you can use it like this:

<template>
  <SuspenseWithError>
    <template #default>
      <MyAsyncComponent />
    </template>
    <template #fallback>
      <span>Loading... Please wait.</span>
    </template>
    <template #error>
      <h1>I failed to load</h1>
    </template>
  </SuspenseWithError>
</template>

<script>
import MyAsyncComponent from '@/components/MyAsyncComponent.vue';
import SuspenseWithError from '@/components/SuspenseWithError.vue';

export default {
  name: 'App',
  components: { MyAsyncComponent, SuspenseWithError },
}
</script>

Bear in mind that it is a simple and compact implementation that has not been tested in a real application. It also does not distinguished errors which might not be the ideal scenario for you.

Suspense with Vue Router

The main goal of this Dose is to show how to use the Suspense with Vue Router. All the other examples above were made to introduce Suspense and its power.

Suspense plays nicely with Vue Router. What I mean is that you can Suspense your <router-view> and in case the view has an async setup, you can show a fallback.

To be more specific: You can create your loading component that will be shown whenever your view is not ready due some async work that has to be performed.

You can achieve the same behavior with the navigation guards but for most of the cases, which do not involve a complex setup, the combination <router-view>, Suspense and async setup do a nice job!

The example below shows how it can be implemented:

<Suspense>
  <template #default>
    <router-view />
  </template>
  <template #fallback>
    <span>I'm a loading screen, I'm waiting the view to be ready!</span>
  </template>
</Suspense>

All in all

Suspense can be used to show a fallback element when an async work is needed in the main component
One component can have multiple suspended components inside
Error handling can be done with onErrorCaptured hook
A wrapper can be created to extract the error logic
Suspense plays nicely with Vue Router once we want to show a loading screen

The final result is shown below and you can also check the sample code here: vue-3-suspense.

Creating a Twitter BOT for Berlin English speakers

Vinicius Kiatkoski Neves — Wed, 01 Apr 2020 08:39:53 +0000

Creating a Twitter BOT for Berlin English speakers

I'm going to walk you through the process of creating @Berlinglish, a Twitter BOT that tweets Berlin's news in English for non-German speakers.
The project was developed using Javascript. It is an AWS Lambda function that has an AWS CloudWatch scheduler as trigger. The function crawls Berlin's latest news and tweets it =]

Berlin in English

@berlinglish

Hello Berlin! 🇩🇪

19:45 PM - 31 Mar 2020

Motivation

I'm working from home since mid-March due the Corona outbreak. On the first days I had been constantly reading the news about it but there is a problem: I live in Berlin and I don't speak proper German.
Berlin has its official English News channel which I think is super cool. It also has its official Twitter account @Berlin_de_News which tweets their news in German.
The issue here is that they don't offer an English option. The Twitter account tweets only the German's news so if you want to have the "latest" English news you would have to open their website.
That was my main motivation to create @Berlinglish, a bot that would tweet Berlin's News in English. The idea is that you can get notified every time that there is an update.

Enough of introduction and motivation. From now on I'm going to dive into how it was implemented and I would love to have your feedback. I hope the project evolves over time, I can see a lot of room for improvements, from tech to new ideas!

The project consists in 2 basic structures: Crawler and Twitter API =]
I'm going to also talk about the deployment, using AWS SAM in this case, and at the end I invite you to contribute (not just tech-wise) and share it =]

Crawler

First let me mention which webpage I'm crawling: https://www.berlin.de/en/news/

The idea is to go and grab the URL and title of every article in this page and tweet it. Thankfully this page is statically generated so I don't need to worry about any async requests made to extract the data that I need. This means that I need to download the source of the page and then parse it somehow.

Downloading page source

There are a lot of different ways of doing it. You can even do it from your terminal if you want: curl https://www.berlin.de/en/news/.
I chose axios as I do use it almost everyday at work. You don't need a library to do it and axios is indeed and overkill here.

Nevertheless, the code with axios looks like the following:

const axios = require('axios');

const BASE_URL = 'https://www.berlin.de';
const NEWS_PATH = '/en/news/';

async function fetchArticles() {
  const response = await axios(`${BASE_URL}${NEWS_PATH}`);

  console.log(response.data); //<!DOCTYPE html><html ...
}

The code is quite straightforward. I'm using BASE_URL and NEWS_PATH because I will need them later. The HTML that we want is under .data property from axios response.

That is all we need to do to grab the data that we need, now we need to parse it!

Parsing page source

The parsing step should be simple. Given an HTML document as input I want to extract some structured information out of it. My first idea is: take the article title and the article link. So every tweet will contain the title and the link to the original article. It is similar to what @Berlin_de_News does:

Berlin aktuell

@berlin_de_news

Neue Webseiten für digitale Kulturevents in Berlin dlvr.it/RSrPvC

16:29 PM - 30 Mar 2020

To parse the HTML, I've choosen cheerio which allows you to "jQuery" the input. This way I can navigate and select parts of the HTML document that I want to extract the data from.

The parsing code looks like the one below:

const cheerio = require('cheerio');

async function parseArticles(html) { // HTML is `response.data` from `fetchArticles`
  const $ = cheerio.load(html);
  // `.special` might include some "random" articles
  const articles = $('#hnews').parent().find('article').not('.special').map(function() {
    const heading = $(this).find('.heading');
    return {
      title: heading.text(),
      link: `${BASE_URL}${heading.find('a').attr('href')}`,
    };
  }).toArray();

  console.log('Fetched articles: ', articles);

  return articles;
}

I navigate through all the <article> from an specific part of the page and .map them. There are some specific things like #hnews, .parent() and .not() that are rules I followed to find the articles section. This is a sensitive part but it does the job for now. The same result could be achieved using other selectors as well.

The result is the following structure:

[
  {
    title: 'Article title',
    link: 'https://www.berlin.de/path/to/article/title'
  },
  {
    title: 'Article title 2',
    link: 'https://www.berlin.de/path/to/article/title-2'
  }
]

This concludes our crawler: it fetches the page and parses so we have a more structure data to work.

Next step is to tweet the extracted articles.

Tweeting

First step was to create a Twitter account/app.

~~Thankfully the handler @Berlin_en_News was not yet taken and it would be perfect for this case as the German version (official) is called @Berlin_de_News.~~

The paragraph above is outdated. Once I started writing the article I created @Berlin_en_News which was perfect but it got locked due a mistake of mine. After more than a week without getting unlocked I gave up and created a new account. That is when @Berlinglish was born. It is basically Berlin + English =]

Now I've all the necessary keys to use the Twitter API and I just need to start tweeting.

I ended up using a library called twitter to do so. It is not necessary to use a library as Twitter API seems really friendly but my goal was not to optimize or so in the beginning, I wanted to make it work first =]

This is the code needed to get the library ready to use (all Twitter keys are environment varibles):

const Twitter = require('twitter');
const client = new Twitter({
  consumer_key: process.env.TWITTER_API_KEY,
  consumer_secret: process.env.TWITTER_API_SECRET_KEY,
  access_token_key: process.env.TWITTER_ACCESS_TOKEN,
  access_token_secret: process.env.TWITTER_ACCESS_TOKEN_SECRET,
});

To tweet we need to use the following API: POST statuses/update. It has a lot of different parameters. At the beginning I'm ignoring most of them. I'm just using the place_id so it shows that the tweet is from Berlin.

The following code walks through the process of tweeting:

const placeId = '3078869807f9dd36'; // Berlin's place ID

async function postTweet(status) {
  const response = await client.post('statuses/update', { // `client` was instantiated above
    status, // Tweet content
    place_id: placeId,
  });

  return response;
}

for (const article of newArticles) { // `newArticles` come from the crawler
  const response = await postTweet([
    article.title,
    `Read more: ${article.link}`,
  ].join('\n'));

  console.log('Tweet response: ', response);
}

The BOT is almost ready. It misses on important aspect: it shouldn't tweet the same article again. So far it doesn't know which articles it already tweeted.

Filtering new articles

This process denifitely needs to be improved but it does the job for now (again) =]

I fetch the BOT's timeline and compare it with the articles' titles. The only tricky thing is that Twitter won't exactly use the article URL in the tweet itself, so some dirty "magic" had to be written for now. As I said, it does the job for now =]

async function homeTimeline() {
  const response = await client.get('statuses/user_timeline', {});
  const responseTitles = response.map((tweet) => tweet.text.split('\n')[0]); // Dirty "magic" 🙈

  console.log('Last tweets titles: ', responseTitles);

  return responseTitles;
}

const [articles, tweets] = await Promise.all([fetchArticles(), homeTimeline()]);
const newArticles = articles.filter(article => !tweets.includes(article.title));

With that in place I'm "sure" that it will tweet only the new articles.

Now the BOT itself is done. There is one major issue: I need to run it on my machine. The next step is to deploy it so it runs automatically =]

Deployment

I chose to deploy it to Lambda by convenience as I'm more familiar to it and this BOT won't run all day long. It will run every 30min (using a CloudWatch scheduler) at the moment which means that it would be a good use case for Lambda.

Everything was deployed using AWS SAM as I wanted to try the tool in a real project. It gives you a lot of flexibility but also some challenges if you compare it to Serverless Framework for example.

You can check the PR where I added the deployment here: https://github.com/viniciuskneves/berlinglish/pull/4

The configuration file template.yaml (which is used by SAM) is divided into 3 important blocks that I'm going to explore: Resources, Globals and Parameters.

Resources

In my case I'm using a Lambda Function and a CloudWatch scheduler as resources. The CloudWatch scheduler is automatically created for us once we define it as an event source for our function. The trickiest part here is to know how to define a schedule, which you would have to go through the docs if you want to understand it a bit better: https://docs.aws.amazon.com/eventbridge/latest/userguide/scheduled-events.html

Resources:
 TwitterBotFunction: # Defining an AWS Lambda Function
   Type: AWS::Serverless::Function
   Properties:
     Handler: index.handler
     Events:
       Scheduler: # CloudWatch Scheduler automatically created
         Type: Schedule
         Properties:
           Description: Schedule execution for every 30min
           Enabled: true
           Schedule: 'rate(30 minutes)' # Runs every 30min

Globals

Those are global settings applied to our resources. I could have defined them inside each resource for example but it doesn't make sense for the project so far.

I'm setting my runtime, which is Node.js for this project, a timeout for Lambda and also my environment variables which are used by my function (Twitter keys).

Globals:
 Function:
   Runtime: nodejs12.x
   Timeout: 5
   Environment:
     Variables:
       TWITTER_API_KEY: !Ref TwitterApiKey
       TWITTER_API_SECRET_KEY: !Ref TwitterApiSecretKey
       TWITTER_ACCESS_TOKEN: !Ref TwitterAccessToken
       TWITTER_ACCESS_TOKEN_SECRET: !Ref TwitterAccessTokenSecret

What is missing now is where do those keys come from, that is why I've added a Parameters block.

Parameters

Those are the parameters my build is expecting. I've decided to setup it like that in a way to avoid hardcoding the keys. There are different strategies here and I went for the quickest one for now.

Parameters:
 TwitterApiKey:
   Description: Twitter API Key
   NoEcho: true
   Type: String
 TwitterApiSecretKey:
   Description: Twitter API Secret Key
   NoEcho: true
   Type: String
 TwitterAccessToken:
   Description: Twitter Access Token
   NoEcho: true
   Type: String
 TwitterAccessTokenSecret:
   Description: Twitter Access Token Secret
   NoEcho: true
   Type: String

Now, once I call the deployment command I need to pass those parameters as arguments:

sam deploy --parameter-overrides TwitterApiKey=$TWITTER_API_KEY TwitterApiSecretKey=$TWITTER_API_SECRET_KEY TwitterAccessToken=$TWITTER_ACCESS_TOKEN TwitterAccessTokenSecret=$TWITTER_ACCESS_TOKEN_SECRET

Contribute and share

I hope I could briefly share the idea behind the BOT and I also hope you could understand it. Please, don't hesitate to ask, I will try my best to help you.

It has been a fun process, some learnings as Twitter account blocked by mistake, but at the end it was useful, at least to me. Now I don't need to open the news website every day and can just wait until I get notified about a new tweet =]

I would appreciate if you could share the project so it would help other people as well, especially in Berlin =]
I would also appreciate if you want to contribute to the project:

New ideas: add images to tweets, add comments... Anything that could be done on Twitter level to enhance the experience.
Project maintenance: I've setup some issues on GitHub and you're more than welcome to give it a try.
New sources: do you have any other sources that are worth adding? Let me know and we can work on it.
New city/topic: would you like to have it in your city as well? For a specific topic? Let's make it happen =]

Thank you and #StayHome =]

Securing your component library with visual testing

Vinicius Kiatkoski Neves — Wed, 11 Dec 2019 09:56:44 +0000

Find out how we added visual testing to our component library (Homeday-blocks) and why I think it is important to have visual testing in your test toolkit.

Background

As an introduction I want to share how I started with visual testing and why I decided to add it to our component library.

Currently we’re working on a new Homeday’s project and I decided to enhance our unit tests with some integration and visual tests. The idea is to have a secure and robust project, in which we feel comfortable adding new features, refactoring code and fixing future issues — which hopefully we will not have 😅

We’re using Cypress in this project, so I decided to integrate screenshot comparison tool with it and see how that works together. Unfortunately it didn’t work that well. I’ve tried two tools: cypress-plugin-snapshots and cypress-image-snapshot. Both are great, as they are free, however when dealing with visual testing you have to be ready to adjust some parameters and find magic numbers that work for you. As it is not my main focus to really dig deep into how visual testing really works (maybe in the future), I decided to find a better and mature tool to do so. I ended up using Percy. Just as a disclaimer: I’m not being sponsored by Percy.

Reading through Percy docs, I saw that they have an easy solution to integrate with Storybook. The good news is that our component library already uses Storybook. The bad news is that not all the components have stories (which means they don’t appear in Storybook). Since we introduced unit tests in our component library I’ve been thinking about ways to expand it and make our component library more robust then it currently is. That’s why I ended up trying out visual testing.

Every quarter we have a week to develop and try new stuff in the company. It is called PEW (Product Engineering Week). It works as following: Week before you select your topic; Monday morning you do a quick intro to your topic; Friday afternoon you present your results. During this week all ours meetings are canceled so we can really focus on our topics. My PEW project this quarter is to add visual testing to our component library =]

First steps

As I mentioned before, I’m going to use Percy. In order to do that you need an account and add your project there. It just asks you to authorize access to your repo account, that way you can see the status of the visual testing in your PR. This is not mandatory, but it helps you to keep track of your tests, specially if you’re opening a PR or so. I’m also relying on Storybook as our component library already uses it.

Most of what I'm going to show here can be found in Percy docs for Storybook as well.

You just need one dependency to start using Percy + Storybook: @percy/storybook 😅

npm i --save-dev --save-exact @percy/storybook # --save-exact is optional

After installing the dependency we need a PERCY_TOKEN environment variable to secure and match the local project and the Percy project. I also recommend to add it to your CI, as it is probably the tool that will fire your tests.

export PERCY_TOKEN=<my secret token here>

Before we run Percy, we need to first build Storybook, otherwise Percy doesn’t know where to find the stories. Now we can simply run npx percy-storybook command and it is going to screenshot every single story from Storybook.

With this simple setup, we can already benefit from the powers of visual testing. It is even better if you’re just starting a project, so you can carefully cover each case you want with visual testing. As we already have an in-use, project we can’t do that. We needed some further adoptions that I'll describe in the following section.

Knobs, repetitive stories and other corner cases

Our component library is open source, so I’ve also opened the Percy builds, so everyone that is interested can follow it as well.

The first issue I encountered was related to our Homepage. After running the first test I saw it took a screenshot from it.

We have a Welcome component which renders the first page in Storybook, so it becomes our Home. I don’t want to visual test it, so I had to find a way to ignore it. Percy docs are pretty clear about that, you simply add skip parameter to your story and it gets ignored.

storiesOf('Welcome', module)
  .addParameters({ percy: { skip: true } })
  .add('Welcome', () => ({
    ...

Second issue was similar to the first one, but the scenario is a bit different. We have some components with knobs, which allow the users from Storybook to play with them. It basically binds the component’s props with some inputs, so you can nicely interact with the component. I don’t want to test those. I need to create all possible variations in stories so I can take screenshots from all of them and compare over time. The knobs are great for users as they can play with the components, but not that useful when dealing with visual testing. In order to avoid that, I had to skip some Playground Stories (that is how we call the stories with knobs) and refactor some stories as well.

stories
  .add('Playground 🎛', () => ({
    components: { HdButton },
    props: {
      ...
    },
    template: `
      <HdButton
        :modifier=modifier
      >{{ text }}</HdButton>
    `,
  }), { percy: { skip: true } });

One of our stories makes sense only on mobile view, as it adapts the component on mobile. This is our third issue, which can be solved simply by adjusting the configuration. Instead of taking screenshots on mobile and desktop, I’ve added a parameter to specify the screen size. Just as a reminder: Every extra screenshot will add to your account limit and also will take extra time to process, so I decided to really avoid the unnecessary screenshots from the beginning.

}), { percy: { widths: [375] } });

The last issue: CI integration. As the project is open source, it means that people can fork it and open a PR. We (developers from the company) follow the same approach. We have our forks and open PRs from them to the main repo. We use Travis as CI, but it happens on other CIs as well, it doesn’t share environment variables to fork builds, due security issues. It means that PERCY_TOKEN won’t be available when CI builds. Without PERCY_TOKEN, the Percy command fails, which also fails the CI build. To avoid that, I had to add a check in the test command for the existence of the PERCY_TOKEN. If it does not exist, we skip the tests. If you want to run Percy in this scenario, you have to checkout the desired branch and run tests locally, with PERCY_TOKEN exported. Alternatively, you can create a new branch in the origin repo that will trigger the build.

"test:percy": "if [ $PERCY_TOKEN ]; then percy-storybook; else echo 'Skipping test: PERCY_TOKEN is not defined'; fi"

Final thoughts

You can find the final PR here. Fell free to add your comments and questions there.

The integration Percy + Storybook works smoothly and it is relatively simple to add it to a project. Percy also offers a pretty good free tier so you don’t have to worry about investing some money from the beginning.

There are still some challenges to be tackled, especially components that request some interaction or event to happen. Those need to be better explored and tested. It will probably lead to new stories being written.

Percy also allows you to customize your integration even deeper if you want, I do suggest you to read its documentation if you need to do such a thing.

I would recommend you to give it a try and also share your learnings and sorrows [=

Happy coding \o/

Thanks to Ilyas, Ante and Sinisa for reviewing the text.

"Ich bin eine Kartoffel" and your German skills during Hacktoberfest

Vinicius Kiatkoski Neves — Thu, 03 Oct 2019 13:52:20 +0000

I've been living in Germany since February. Coming from Brazil I can tell you that my German skills are no the best. I can also tell you that German is not easy to learn coming from Brazil (it might be easier for you depending on which languages you already know).

Nevertheless even before I came to Germany I knew one word: Kartoffel. Apparently it is the one of the most well known German words worldwide and it is heavily used among foreigners.

After some months here I realized that Kartoffel is a powerful word and also a German cliché. You can hear from almost all new students the following sentence: Ich bin eine Kartoffel. The translations is straightforward: I'm a potato.

"Ich bin eine Kartoffel" + purple + some inspiration from Sara Vieira and her talk "Build dumb shit" I decided to build ichbineinekartoffel.de.

I think the website talks by itself but you can contribute to make it even better and spread the word Kartoffel to the rest of the world. I've opened some issues where you can play a bit and help me to improve this masterpiece.

viniciuskneves / ichbineinekartoffel.de

Ich bin eine Kartoffel.

CSSConf EU 2019 in a short opinionated recap

Vinicius Kiatkoski Neves — Tue, 04 Jun 2019 12:20:00 +0000

This is a summary of what happened during CSSConf EU 2019 regarding its talks for those whom missed it or have been there and would like a recap. I haven't watched all the talks so I will give some opinion on the ones I've watched and ⭐ the ones I consider a "must watch".

I've also added the sketch notes from @lisi_linhart, @wilfredspringer, @clairikine and @SarahOnDaCloud which are a great summary of what have happened on each talk.

If you have some extra material that you would like to add here, let me know! It could be extra sketch notes, videos, comments, tweets... Everything you thing might be relevant to add to this list!

Hope you enjoy!

Talks

Why do we call HTML tags "tags"? A journey through the history of CSS, mathematics, and CS to find out! by Andrés Cuervo
⭐ Hello subgrid! by Rachel Andrew
Using DevTools to understand modern CSS layouts by Chen Hui Jing
⭐ Modern Typographic Systems with Variable Fonts by Jason Pamental
⭐ Don’t Believe the Rumors: Writing Tests for CSS is Possible by Gil Tayar
Let’s web dev like it’s 1999! by Ben Ilegbodu
Improving Website Performance with CSS Containment by Manuel Rego
⭐ Class 🥊Clash by Alex Tait
A Working Theory of Components by Elyse Holladay
⭐ Design System Magic with CSS Houdini by Samuel Richard
Closing Keynote by Sareh Heidari

Why do we call HTML tags "tags"? A journey through the history of CSS, mathematics, and CS to find out! by Andrés Cuervo

The name of the talk says by itself. Nothing technical but quite interesting. If you ever wanted to know why we call it tags this is the right talk. Andrés goes through the history and how the terminology tag was coined.
Slides are available here: https://slides.cwervo.com/mdx-slides/css-conf-eu-2019/?mode=#0

Liquid error: internal
Liquid error: internal

⭐ Hello subgrid! by Rachel Andrew

It was a masterpiece! She walked through some subgrid examples and what problems they do solve (make children elements align based on parent's grid). A really nice summary on how subgrids work and how we can use them. This includes a lot of the material used during the talk: https://www.smashingmagazine.com/2018/07/css-grid-2/ and https://www.smashingmagazine.com/2019/05/display-grid-subgrid/

Slides are available here: https://noti.st/rachelandrew/i6gUcF

Liquid error: internal
Liquid error: internal
Liquid error: internal
Liquid error: internal

Using DevTools to understand modern CSS layouts by Chen Hui Jing

Really nice demos on how to use Flexbox and DevTools to debug it. It would be nice to watch if you haven't played with Flexbox or if you would like to refresh your knowledge a bit. You can find all the demos and play with them here (There is also an awesome readme): https://github.com/huijing/slides/tree/gh-pages/58-cssconf-2019

Liquid error: internal
Liquid error: internal
Liquid error: internal
Liquid error: internal

⭐ Modern Typographic Systems with Variable Fonts by Jason Pamental

Another masterpiece in my opinion. He explains variable fonts and how we can benefit from it, some tips on typography and he also talks about the future with progressive font enrichment. You can check more about it here: https://rwt.io/typography-tips/progressive-font-enrichment-reinventing-web-font-performance and https://developers.google.com/web/fundamentals/design-and-ux/typography/variable-fonts/
Slides are available here: https://noti.st/jpamental/2gsS6v

Liquid error: internal
Liquid error: internal
Liquid error: internal
Liquid error: internal

⭐ Don’t Believe the Rumors: Writing Tests for CSS is Possible by Gil Tayar

Nice introduction to visual testing using tools like Cypress. There were some really simple and nice examples and why we should test our CSS and why it is not as complicated as we think. Gil even show some tools to automate part of the process and make it faster as this kind of testing might be quite expensive. Code examples: https://github.com/giltayar/css-testing-rumors
You can also find a tutorial about it here: https://css-tricks.com/an-intro-to-web-app-testing-with-cypress-io/
Official documentation: https://www.cypress.io

Liquid error: internal
Liquid error: internal
Liquid error: internal

Let’s web dev like it’s 1999! by Ben Ilegbodu

Ben shows his first website back in time. How it actually worked and how was it to develop websites in 1999. Really nice walk through the history of the web development. If you've never seen <frame> and <frameset> in action then you should have fun watching this talk.
Slides are available here: http://www.benmvp.com/slides/2019/cssconfeu/webdev.html#/

Liquid error: internal
Liquid error: internal
Liquid error: internal

Improving Website Performance with CSS Containment by Manuel Rego

The new CSS contain property (https://developer.mozilla.org/en-US/docs/Web/CSS/contain) in a nutshell. Visual examples of what happens to the DOM and the benefits of using it.
Slides are available here: https://people.igalia.com/mrego/talks/cssconf-eu-2019-css-containment/

Liquid error: internal
Liquid error: internal

CSS Logical Properties by Estefany Aguilar

Deeply technical (CSS Logical Properties) and with the cutest slides I've ever seen! Impossible to make a short summary of it, you need to read, play and watch the talk to try to understand it a bit better. Some examples: https://github.com/teffcode/logical-properties-examples
Slides are available here: https://docs.google.com/presentation/d/1-GOSaggySGIKmcNuDPNPE6ZZEM2H_oF2AOdcMvlG7kk/edit#slide=id.g562dbb611a_0_0

Liquid error: internal

⭐ Class 🥊Clash by Alex Tait

Have you ever heard of Functional CSS? If not this is a really nice introduction to it. The idea is to have classes that do a single job, which means add a specific CSS property to the element instead of a class that has a bunch of properties and it becomes hard to reuse it.A lot of examples and answers to the problems it solve. Alex mentions Tachyons as a good Functional CSS library. You can also read more about it here: https://rangle.io/blog/styling-with-functional-css/

Liquid error: internal
Liquid error: internal
Liquid error: internal

A Working Theory of Components by Elyse Holladay

Liquid error: internal

⭐ Design System Magic with CSS Houdini by Samuel Richard

Have you ever seen Houdini in action? If not this is the right place. Samuel explains what is Houdini and why it is so magical with some cool examples!
Some Houdini resources can be found in the tweet below:
Liquid error: internal
Slides are available here: https://talks.page.link/houdini-design-systems

Liquid error: internal

Closing Keynote by Sareh Heidari

Liquid error: internal

DEV Community: Vinicius Kiatkoski Neves

Use AWS through GitHub Actions without secret keys

OpenID Connect (OIDC)

Importing existing AWS resources using AWS CDK

AWS CDK output

Importing existing AWS resources

Importing resources that don't follow defaults

Prettier, ESLint and Typescript

Prettier

How to set up Prettier?

Prettier configuration

ESLint

How to set up ESLint?

Prettier + ESLint

Typescript

That's all folks!

Creating a GitHub Action for your Slack release BOT

Pre-requisite

Creating a GitHub Action to trigger the webhook

Vue custom input

v-model and <input>

The wrong custom input component

The happy custom input component

Adding validation (or operation on data change)

Combining computed and v-model

Extra: the model property

So what?

Multiple builds targeting different browsers

My idea

Development lifecycle

Targeting browsers when building

Multiple builds

Routing multiple builds

Things to consider

Thank you for coming this far =]

Webpack concepts: Loaders and Plugins

Webpack

Loaders

Plugins

Final config

Go async in Vue 3 with Suspense

What is Suspense?

How do I handle errors?

Suspense with Vue Router

All in all

Creating a Twitter BOT for Berlin English speakers

Creating a Twitter BOT for Berlin English speakers

Motivation

Crawler

Downloading page source

Parsing page source

Tweeting

Filtering new articles

Deployment

Resources

Globals

Parameters

Contribute and share

Securing your component library with visual testing

Background

First steps

Knobs, repetitive stories and other corner cases

Final thoughts

Thanks to Ilyas, Ante and Sinisa for reviewing the text.

"Ich bin eine Kartoffel" and your German skills during Hacktoberfest

viniciuskneves / ichbineinekartoffel.de

Ich bin eine Kartoffel.

CSSConf EU 2019 in a short opinionated recap

Talks

Why do we call HTML tags "tags"? A journey through the history of CSS, mathematics, and CS to find out! by Andrés Cuervo

⭐ Hello subgrid! by Rachel Andrew

Using DevTools to understand modern CSS layouts by Chen Hui Jing

⭐ Modern Typographic Systems with Variable Fonts by Jason Pamental

⭐ Don’t Believe the Rumors: Writing Tests for CSS is Possible by Gil Tayar

Let’s web dev like it’s 1999! by Ben Ilegbodu

Improving Website Performance with CSS Containment by Manuel Rego

CSS Logical Properties by Estefany Aguilar

⭐ Class 🥊Clash by Alex Tait

A Working Theory of Components by Elyse Holladay

⭐ Design System Magic with CSS Houdini by Samuel Richard

v-model and `<input>`

Combining computed and `v-model`

Extra: the `model` property