DEV Community: Gaute Meek Olsen

Best practice for images

Gaute Meek Olsen — Mon, 06 Feb 2023 19:39:14 +0000

Adding an image to your website is quite simple.

<img src="/apple.jpg" />

Let's pack our bags and go home.

But there are more things you should think about for a better user experience and performance improvement. Let's go through them one by one.

Alternative text

The alt attribute should be used for a textual description of the image. This is used by screen readers, so it's really important for accessibility. It is also displayed if the image can't be loaded.

<img src="/apple.jpg" alt="A green apple" />

If your image is only decoration and doesn't provide any value to the content you should set an empty string, alt="".

Figure

<figure> is an element that represents some self-contained content, optionally with a caption. It's often used with an image where we want a caption for the image.

<figure>
  <img src="/apple.jpg" alt="A green apple" />
  <figcaption>Largest green apple discovered</figcaption>
</figure>

Vector images

If your image is a Scalable Vector Graphics (SVG), you should keep it in that format instead of serving something like JPEG or PNG. That way your image can scale better to different sizes, also when the user zooms in, and there is less data transferred over the network most of the time.

Extra tip, you can use svgomg to clean up your .svg file.

Compress your image

This is the easiest measure you can do to improve your performance. You probably want your users to view the images in the best quality, but at what cost and is it necessary? If you serve an oversized image only to be viewed in a limited-sized box, you transfer unnecessary data over the network that take extra time before it can be displayed. I recommend you use a tool to resize and compress your image, e.g. Squoosh.

Avoid layout shift

Layout shift is when you think the content is loaded, but suddenly there is a jump because of a new element or an element changing height. Images typically do this, since they take up 0px until the source is loaded and the browser knows the actual height.

If you know the image size and the space it will take up on the page, you should set the width and height.

<img src="/apple.jpg" height="200" width="150" alt="A green apple" />

But often we do not know the exact space it will take up, because you have a dynamic layout that depends on the screen size. If you know the aspect ratio of the image, then that solves the problem. aspect-ratio: width / height.

img {
  aspect-ratio: 3 / 4;
}

But if you don't know that as well, you should set a sensible minimum height you know the image won't be smaller than.

img {
  min-height: 100px;
}

Different images for different screen sizes

Would you serve the same image to a mobile phone and a large external monitor? Probably not. The <picture> element can help with that, it will pick the first <source> that matches its criteria. <img> is used as a fallback and the place you put common attributes.

There are two approaches, either with media or sizes.

media

The media attribute accepts a media query and if true the source is valid to be shown.

<picture>
  <source media="(min-width: 1000px)" srcset="/large.png" />
  <source media="(min-width: 500px)" srcset="/medium.png" />
  <source srcset="/small.png" />
  <img src="/medium.png" alt="monster evolution demo" />
</picture>

sizes

The sizes attribute is a comma-separated list with a pair of a media query and the preferred minimum width. The first valid media query is chosen. With srcset you can let the browser know the size of the image before it's downloaded. It's also a comma-separated list with a pair of the source and width in pixels suffixed with w.

<picture>
  <source
    sizes="
      (min-width: 1000px) 300px,
      (min-width: 500px) 200px,
      100px,
    "
    srcset="/large.png 300w, /medium.png 200w, /small.png 100w"
  />
  <img src="/medium.png" alt="monster evolution demo" />
</picture>

The browser decides which image is best from the srcset, but we control it through sizes. (min-width: 500px) 200px means that if the screen is a minimum of 500 pixels, it should select an image that is 200px or wider. And if we look into our srcset that would be medium.png or large.png (the browser will be efficient and pick the smallest).

Note that when a larger image is loaded, the browser will keep using that even though the screen size decreases.

Modern image formats

There are some new image formats with better compression than JPEG and PNG, such as WebP, AVIF, and JPEG XL. You should check out the browser support on Can I use (WebP, AVIF, and JPEG XL) and choose accordingly. Again you can use Squoosh to convert your images and see how much you save.

<picture>
  <source type="image/jxl" srcset="/image.jxl" />
  <source type="image/avif" srcset="/image.avif" />
  <source type="image/webp" srcset="/image.webp" />
  <source type="image/jpeg" srcset="/image.jpg" />
  <img src="/image.jpg" alt="" />
</picture>

Modern image formats can be combined with how we served different sizes for each format.

Loading strategy

Should you load an image you might not need? Maybe, but you can use the loading attribute to not load all images eagerly. The default value is eager, so there is no need to specify that. But if you want the image to be hidden from the preload scanner and only load once it's about to enter the viewport, setting loading="lazy" is good.

The browser is smart about the fetch priority of images, but it can be improved if we gave it some hints. If we say nothing it can end up being low, high, or both. But if you want the image to skip a few places in the loading queue you can set fetchpriority to high. If the image is not that important you can set it to low so other requests can take priority.

We can also use decoding to change the prioritization of processing our content. You can leave it blank to let the browser know best. But if it's important to show the image early you can set it to sync. If it's okay to process other content before or at the same time as the image you can set it to async.

Putting it all together. If the image is your largest contentful paint (LCP) or the image is "above the fold", meaning it's visible without scrolling, you might want this code.

<img src="/apple.jpg" fetchpriority="high" decoding="sync" alt="A green apple" />

If the image is "below the fold" or not that important, you might want this.

<img src="/apple.jpg" loading="lazy" decoding="async" alt="A green apple" />

At last, what if your image is not found by the preload scanner because the image is client-side rendered by JavaScript or a background-image in CSS? You can fix this by preloading the image.

<link rel="preload" as="image" href="/apple.jpg" />

Caching

If a user is returning to a site, there is no need to re-download the same images. You can cache your images through a service worker. But probably the simplest way is to let the browser do it for you through Cache-Control in response headers. This can be configured by your hosting provider or your server. You set the max-age in seconds for how long it is okay to reuse it and immutable to ensure it's used. If your build step adds a hash to the filename you can set max-age to the highest supported value, a year, otherwise use a sensible value.

Cache-Control: max-age=31536000, immutable

Web safe fonts interactive directory

Gaute Meek Olsen — Sat, 07 Nov 2020 16:15:42 +0000

Check out my imitation of Google Fonts website, where you can browse and select a font for your project. The only difference, my website is just for web safe fonts.

You can find it at web-safe-fonts.vercel.app.

You can find the code at GitHub. It was created with Vite and Vue 3, which is a blast! I also tried out Vercel for the first time, which was just as easy as Netlify and Firebase hosting.

If you find any web safe fonts that are missing, give me a heads up in the comments, create an issue, or send a pull request. ✌

Quality Assess your website with Lighthouse

Gaute Meek Olsen — Fri, 06 Nov 2020 20:29:04 +0000

The best way to measure the quality of your website is by knowing how the users use it and think about it. But often we do not have the possibility to get that insight, you might not know who the users are or have the capacity to find out.

Here we can utilize tools to help us. One tool that we can use, which the article will cover, is Lighthouse. If you already know Lighthouse and are not bothering reading, just a heads up. We will cover how to utilize it for a non-technologist, during development, and automate it in the CI pipeline.

What is Lighthouse?

Lighthouse is an automated tool by Google for assessing and improving the quality of web pages. It has audits for multiple categories with specific details for each.

Performance: Audits for metrics like first paint and time to interactive to determine lag.
Best Practices: Looks for everything from HTTPS usage to correct image aspect ratios.
Search Engine Optimization (SEO): Checks for best practices to ensure your site is discoverable.
Accessibility: Checks for common issues that may prevent users from accessing your content.
Progressive Web App (PWA): These checks validate the aspects of a Progressive Web App.

Above is an example of how the results can look. After you have scanned your website it will provide you with a quality score from 0 to 100 for the categories Performance, Accessibility, Best Practices, and SEO. It also gives details for PWA.

You can even dig into the report further and find the results for each audit, such as "Time to interactive" and "Background and foreground colors have sufficient contrast ratio" to mention a few. Each audit has an explanation of why the audit is important and how to fix it.

Note that there exist other tools out there that might be better at assessing categories such as accessibility and performance. But Lighthouse is a great tool covering multiple areas, is free, and has good quality.

For the Non-technologist

If you own a website, are a project manager, or have any other reasons to validate the quality, you can do that really simple. Just visit web.dev/measure, enter your URL, and press RUN AUDIT. There you get the full report on the quality and maybe you can periodically check the report and tell your team what to focus on improving.

For the Developer

You can also run Lighthouse in Chrome DevTools. So make sure you have installed the Chrome browser, open a website, and hit F12. Navigate to the tab Lighthouse in the top navigation bar. There you can select which categories to audit and which device to simulate on. Then pressing the Generate report button will generate a report for you. This is an easy way to quality assess your website, even during development.

For the Machine

You can even run Lighthouse from the command line. Which means we can Automate our Lighthouse runs via shell scripts. It can be installed through NPM with npm install -g @lhci/cli and run with lhci autorun. I will show you how to implement this on every pull request on GitHub, giving the team full control over the quality of code being added to the repository.

If you don't use GitHub, you can still follow along as doing the same for other platforms are most likely almost the same.

First, let's create our workflow file in our repository. Create .github/workflows/lighthouse.yml. The file name can be what you want, but the .github and workflows folders should have those names. Your .yml file should look like this.

name: CI
on: pull_request

jobs:
  lighthouse-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: npm install & build
        run: |
          npm install
          npm run build
      - name: run Lighthouse CI
        run: |
          sudo npm i -g @lhci/cli
          lhci autorun

This says that on pull requests it will run our lighthouse-ci job with those steps. The steps will install and build our application, then run Lighthouse on the build. If you don't have a build step, you can omit the "npm install & build step". This will run Lighthouse with default settings. It will look for a folder named dist, build, out, or public for HTML files to scan.

We can also add a configuration file to control the Lighthouse run. In the root folder create lighthouserc.js. For a detailed specification of all the settings, check out the documentation. But here is an example of my final configuration.

module.exports = {
  ci: {
    collect: {
      staticDistDir: 'dist',
      isSinglePageApplication: true,
      url: ['', 'about', '404']
    },
    assert: {
      preset: "lighthouse:no-pwa",
      assertions: {
        "categories:performance": ["error", { "minScore": 0.9 }],
        "categories:accessibility": ["error", { "minScore": 1 }],
        "categories:seo": ["error", { "minScore": 0.9 }],
        "categories:best-practices": ["error", { "minScore": 0.9 }],
        "unsized-images": "off"
      }
    },
    upload: {
      target: "temporary-public-storage"
    }
  },
};

The collect option is for which pages to audit. If you have multiple .html files in the folder, you might only need the staticDistDir set. But if you have a Single Page Application (SPA) with multiple routes, you can add the staticDistDir, isSinglePageApplication: true, and url with an array of routes to assess.

assert allows you to control what it asserts. For preset you can set lighthouse:all, lighthouse:recommended, or lighthouse:no-pwa. Where all is very aggressive and can't have any failures. Recommended evaluates the assertions on what they recommend, which is a more realistic assessment. No PWA option is the same as recommended, except it doesn't assess the PWA category. You can also be more specific in assertions. Since recommended only gives error when the score is below 50, I'm using "categories:performance": ["error", { "minScore": 0.9 }] to give an error if it's below 90 to be more strict. You can also turn off specific rules, i.e. "unsized-images": "off".

Now if you want to keep the report, you can use upload with target to upload the report to a server. If you don't want to save them yourself, you can use the temporary-public-storage option provided by Lighthouse. This will upload the report on a temporary page you can get access to for free.

In the end, if you don't want to click into the workflow console in the pull request, you can use the Lighthouse CI GitHub app to set statuses. Here is an example of how Lighthouse has set a status for each URL in the pull request.

First, install the GitHub app Lighthouse CI and give access to the repositories you want. Then copy the token and store it under Settings > Secrets in your GitHub repository with the name LHCI_GITHUB_APP_TOKEN. Now you only need to update the workflow lighthouse.yml file with

# below - uses: actions/checkout@v2
with:
  ref: ${{ github.event.pull_request.head.sha }}
# below lhci autorun
env:
  LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

The complete file will be something like this.

name: CI
on: pull_request

jobs:
  lighthouse-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: npm install & build
        run: |
          npm install
          npm run build
      - name: run Lighthouse CI
        run: |
          sudo npm i -g @lhci/cli
          lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

Check out my GitHub repository for a demonstration with a pull request containing Lighthouse checks.

Conclusion

Lighthouse provides reports that help us create better websites. It's easy to use without any knowledge by passing in any URL in web.dev/measure, during development with Chrome DevTools, and even automate it into your pipeline to continually keep the quality on top.

Emojis Picker - Vite/Electron app

Gaute Meek Olsen — Sun, 27 Sep 2020 09:11:10 +0000

I was tired of googling emojis to copy/paste them. I wanted something similar to the keyboards on our mobile phone where all emojis are easily found. So I created Emojis Picker.

You can find it on Microsoft Store.

Try it out and let me know what you think 😄

The project is created using Vite, Vue 3, and Electron for a cross-platform desktop app.

You can check out the code on GitHub if you want a similar setup.

Global state in React with Vue!

Gaute Meek Olsen — Sat, 26 Sep 2020 18:56:42 +0000

There exist a million (or many) global state solutions in React. It seems like the community is struggling to find the best solution. So here I'm going to come with yet another one.

Recently Vue 3 was released. I know, Vue is another framework, but Vue solves the reactivity in a way that it's not tied to the framework. Which means we can use the reactivity everywhere, including React.

First, let's create a store file.
store.js

import { reactive } from 'vue'

const store = reactive({
  count: 0
})

const increase = () => store.count++

export { store, increase }

For an overview of what the Vue composition API can do except for reactive you can get an overview here.

Now we can import the reactive store object and the increase method from the store.js file anywhere we like to. The problem is that React functions don't know when to re-run the function to render the updated values. We will create a custom hook to deal with this.

useStore.js

import { useReducer, useEffect } from 'react'
import { watch } from 'vue'

function useStore(...stores) {
  const [ignored, forceUpdate] = useReducer(x => x + 1, 0)

  useEffect(() => {
    const stopWatch = watch(stores, forceUpdate)
    return stopWatch
  }, [])
}

export default useStore;

We can either use useState or useReducer to make the component update itself. We are watching the params stores with the Vue Composition API and calls forceUpdate on every change. Also, we stop watching on component unmount by returning stopWatch in useEffect. Any amount of stores could be passed into our useStore.

Bump.js

import React from "react";
import { increase } from './store'

export default function Bump() {
  return (
    <button onClick={increase}>+1</button>
  );
}

We could also do store.count++ directly here if we wanted.

Counter.js

import React from "react";
import { store } from './store'
import useStore from './useStore'

export default function Counter() {
  useStore(store)

  return (
    <p>{store.count}</p>
  );
}

Complete example on StackBlitz

Afterthoughts

I actually think this is a nice and simple way of handling a global state. No need for extra components, reduce, dispatch, and/or complete re-assigning of the whole state object. This way we can create exactly as many global stores as we want in a clean way.

Importing the whole Vue might create a bigger bundle size. But you can import only Vue's reactivity module @vue/reactivity and @vue-reactivity/watch or rely on tree shaking for a small bundle.

Now not every developer would want a different way of dealing with component state and global state, so the React way and Vue way might be confusing in the same project. But it's at least an interesting and fun idea.

Vue 3 is out!

Gaute Meek Olsen — Fri, 18 Sep 2020 19:13:57 +0000

Today Vue 3 was released!

// Detect dark theme var iframe = document.getElementById('tweet-1306992969728380930-438'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1306992969728380930&theme=dark" }

I'm really looking forward to using Vue 3 for all my projects. It's really a well-done framework and totally my favorite.

Some of the main benefits are

The Composition API
Performance Improvements
Improved TypeScript integration

See the keynote announcement here:

See my previous article, Vue 3 is Coming!, for all the benefits and changes to Vue 3, here on dev.to.

What's new in JavaScript - ES2020

Gaute Meek Olsen — Wed, 08 Jul 2020 14:00:20 +0000

In June 2020, a handful of new features arrived in the JavaScript language.

TLDR - To Long, Didn't Read

If you don't want to read the entire article, I have gathered the most noteworthy in this image.

Background

Ecma International is responsible for standardizing JavaScript. Therefore they make the ECMAScript specification. So when someone refers to ECMAScript you can think of this as a synonym to JavaScript. Since 2015 they create yearly editions. Therefore we refer to the edition with the year, ie ECMAScript 2015 is shortened to ES2015. But there are a lot who use the count of the number of editions when referring to an edition, therefore ES6 is the same as ES2015. Future features not yet released are referred to as ESNext.

In June ECMAScript 2020/ES2020/ES11 was released and is already implemented in modern browsers. Let's look at what advantages this gives us.

Nullish coalescing

If you want to assign a value, but want a default value in case it is null or undefined, you can use ??.

const name = person.fullName ?? 'anonymous';

Here the name will be "anonymous" if the object person doesn't have fullName set. If the person has a value for fullName, then that will be written to the variable name.

You might think that this is something you always have been able to do with ||. But this is only almost the same, if the value before || is falsy, the evaluation won't short-circuit and the next value will be used. But remember that an empty string '', 0, NaN, and false are falsy and will use the default value, something that might not be wanted if we want to assign those values. ?? uses instead nullish, which only checks for null or undefined.

const user = { preferredSound: 0 }
let sound = user.preferredSound ?? 50 // value is 0
let soundWrong = user.preferredSound || 50 // value is 50

50 is only used if preferredSound is not set or null, it should be possible to set the sound-level to zero.

Optional chaining

If you want to use properties that are nested in several levels in an object, you previously had to check if they are not null or undefined for the code not to crash. Now we can use ?. before accessing those properties so that the code after is only used if the value is not null or undefined.

Imagine that we have a house with an owner, that again owns a pet. Here we need to make sure that house, owner, or pet has a value or check them in advance to avoid getting the error "Cannot read property 'type' of null". Here you can see how we needed to deal with this before and after ES2020.

const house = { owner: { name: 'Jim', pet: null }};

// Old JavaScript
if(house && house.owner && house.owner.pet && house.owner.pet.type === 'dog'){
  console.log('owner has a dog');
}

// ES2020
if (house?.owner?.pet?.type === 'dog') {
  console.log('owner has a dog');
}

Promise.allSettled

If we have more asynchronous requests that run in parallel, you could gather them with Promise.all. But this will throw an exception if one of the requests fails. What if we want to let every request finish no matter if others fail or not. With Promise.allSettled it will return when all requests are settled, either resolved or rejected.

const promises = [Promise.resolve(1), Promise.reject(2)];
const [result1, result2] = await Promise.allSettled(promises);

Here we can still use the result1 value even though other promises were rejected.

matchAll

If you want to use regex to find all instances of a regular expression match, you can use match to get all the substrings. But what if you want both the substring and the index? Then you can use matchAll and iterate the matches.

Let's find all the numbers in a string.

const matches = 'Here are some numbers: 5 12 88'.matchAll(/\d+/g);
for (const match of matches) {
  console.log(match);
}

// Output:
// ["5", index: 22, input: "Here are some numbers: 5 12 88", groups: undefined]
// ["12", index: 24, input: "Here are some numbers: 5 12 88", groups: undefined]
// ["88", index: 27, input: "Here are some numbers: 5 12 88", groups: undefined]

BigInt

BigInt is a new primitive datatype in JavaScript, the same as Boolean, Number, String, Symbol, and undefined. BigInt can handle numbers above the safe integer limit of Number. That means if we want to deal with numbers larger than 9_007_199_254_740_991, it is wise to use BigInt. BigInt is represented with an n at the end of the number.

Let's add 2 to the number 9_007_199_254_740_991, the correct number should end with the digit 3.

9_007_199_254_740_991 + 2; // 9007199254740992
BigInt(9_007_199_254_740_991) + BigInt(2) // 9007199254740993n

Dynamic import

Before we only could import modules statically at the top of the file. Now with dynamic imports, we have the option to do this anywhere in the code on-demand and only when we need it. import() will return a promise with the module.

const module = await import('module');

Module namespace exports

With import and export of JavaScript modules, we have in most situations been able to rename the name of the module. Like this.

import * as values from 'module';
import { value as v } from 'module';

export { v as value };

But we have not been able to re-export something from another module with a name change directly. Now with ES2020 we can do it like this:

export * as someUtils from 'utils';

globalThis

If you write code that runs on multiple environments, for example, both the browser and a Node server, then they have different names for the global object. Browsers use window, Node uses global, and web workers use self. Now, globalThis will give you the correct global object no matter which environments the code runs in.

Here is an example where we want to check if we can prompt an alert to the user. If the code runs in a browser the globalThis will refer to window and alert will be available.

if (typeof globalThis.alert === 'function'){
  globalThis.alert('hi');
}

16 week streak of blogging - how to do it

Gaute Meek Olsen — Thu, 25 Jun 2020 07:10:59 +0000

It's time to stop and celebrate my own victory, I got the 16 week streak badge after blogging for every week on dev.to. I'm happy for myself right now 🥳

How you can do the same

I didn't know I was going to aim for the 16 week streak and write one article every week 4 months from now, but after getting some badges along the way just from wanting to write articles, I decided to go for the longest streak. So the first point is that you enjoy writing blogs.

I recommend writing articles because you will get a better understanding of the topic yourself, you can get feedback from the community, you promote yourself as a skilled person, you have documented something so you can easily look back and remember how it was, and you will contribute to the community.

Going for a streak writing every week, you might feel like it's going to be hard to keep up with the consistency. This helped me; writing about learnings from your own projects, both work and side projects. Mix it up with some short Today I Learned blogs that contain short useful information that is quick to write. After a while I created a to-do list with ideas for articles. Which helped me for weeks I wasn't sure what I wanted to write about. But don't plan everything, you will get ideas along the way. If you have a topic/project with a lot of different focus areas, you can split the article into a blog series. This will also help you write concise articles that are focused and easy to read. At the end I was going traveling without a computer, so writing some articles in advance and publishing from the mobile helped me.

Thank you for reading. And remember that the dev.to community is a kind one, so publishing content feels very safe and welcome 🤗

CI/CD to AWS with GitHub Actions

Gaute Meek Olsen — Sun, 21 Jun 2020 23:18:06 +0000

This article is part of the Twitter bot with Kotlin in AWS series showing how I created a Twitter bot for Vue 3 updates. But this article works as an independent article on how to automatically deploy to AWS from your GitHub repository.

Now let's wrap it up by setting up continuous build and deploy of our Kotlin Lambda function to AWS. This will make sure the Lambda function is always up to date with the code in the master branch, no need to manually deploy your changes.

AWS IAM User

First, we need an IAM user. If you already have one, you can use the existing Access key ID and Secret access secret key. Or you can just click the "Create access key" button and generate new tokens.

GitHub Actions

We need to add two secrets to the GitHub repository. Go to your repository, click the "Settings" tab, and then "Secrets" in the side menu. Add the secrets AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the IAM user.

Luckily the GitHub Actions run on machines with a lot of software pre-installed, which includes the AWS CLI and Gradle.

Now head to "Actions" in the tab menu. Click "Set up this workflow" from any template. Add this yml code, that will setup Gradle, configure the AWS CLI, build the project, and update the function.

name: CI/CD

on:
  push:
    branches: [ master ]

jobs:
  CI-CD:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Gradle setup
      run: gradle wrapper
    - name: Build & Deploy
      run: |
        aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws configure set default.region eu-west-1
        ./gradlew build
        aws lambda update-function-code --function-name twitter-bot-vue-3 --zip-file fileb://build/libs/twitter-bot-vue-3.jar

Wrap Up

That completes our series on how to create your own Twitter bot running in AWS. Check out the Twitter account @vue_3 for the completed product. Also you can find all the code on GitHub.

Secrets in AWS and reading from Kotlin

Gaute Meek Olsen — Sun, 21 Jun 2020 15:32:05 +0000

This article is part of the Twitter bot with Kotlin in AWS series showing how I created a Twitter bot for Vue 3 updates. But this article works as an independent article on how to manage your secret values and tokens in AWS and read them from Kotlin code.

We are using the AWS Secrets Manager to keep our keys and tokens safe. Make sure your role running the code has access to the Secrets Manager.

Adding secrets with AWS CLI

Run the create-secret command to create secret containing a JSON object with key/value's for all the secrets.

aws secretsmanager create-secret --name twitter-bot-vue-3/auth --description "Twitter app keys and tokens" --secret-string "{\"consumer-key\":\"API key\",\"consumer-secret\":\"API secret key\",\"access-token\":\"Access token\",\"access-token-secret\":\"Access token secret\"}"

Reading from Kotlin

First, we need to add a JSON serializer and AWS Secrets Manager to build.gradle.kts.
plugins:

kotlin("plugin.serialization") version "1.3.70"

dependencies:

implementation("software.amazon.awssdk:secretsmanager")
implementation("org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.20.0")

Now create a Secrets.kt file where we create a class that will hold the secrets.

import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonConfiguration
import kotlinx.serialization.json.content
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest

const val secretName = "twitter-bot-vue-3/auth"

class Secrets {
    val consumerKey: String
    val consumerSecret: String
    val accessToken: String
    val accessTokenSecret: String

    init {
        val client = SecretsManagerClient.builder().region(Region.EU_WEST_1).build()
        val secrets = client.getSecretValue(GetSecretValueRequest.builder().secretId(secretName).build())
        val json = Json(JsonConfiguration.Default)
        val jsonObject = json.parseJson(secrets.secretString()).jsonObject
        consumerKey = jsonObject["consumer-key"]!!.content
        consumerSecret = jsonObject["consumer-secret"]!!.content
        accessToken = jsonObject["access-token"]!!.content
        accessTokenSecret = jsonObject["access-token-secret"]!!.content
    }
}

We can now use the Secrets class in our Tweet.kt file.

fun sendTweet(message: String){
    val secrets = Secrets()

    val cb = ConfigurationBuilder()
    cb.setOAuthConsumerKey(secrets.consumerKey)
    cb.setOAuthConsumerSecret(secrets.consumerSecret)
    cb.setOAuthAccessToken(secrets.accessToken)
    cb.setOAuthAccessTokenSecret(secrets.accessTokenSecret)
    val twitter = TwitterFactory(cb.build()).instance
    twitter.updateStatus(message)
}

Posting a tweet from Kotlin

Gaute Meek Olsen — Tue, 16 Jun 2020 08:52:30 +0000

This article is part of the Twitter bot with Kotlin in AWS series showing how I created a Twitter bot for Vue 3 updates. But this article works as an independent article on how to use the Twitter API from Kotlin code.

Twitter Developer Console

First, we need to head over to the Twitter developer console. If you haven't applied for access you need to do that before creating an app. Now create an app and provide a name, description, and website URL(could be the GitHub repository URL). Leave the rest blank.

Go to the Keys and tokens tab and generate an Access token and access token secret. You will need the API key, API secret key, Access token, and Access token secret for later. These will be used to perform a tweet from the Twitter account that owns the Twitter developer app. If you want to tweet from another account's behalf, you can get tokens from the 3-legged OAuth flow, but I won't cover that here.

Kotlin Project

You can send a tweet from an HTTP request, but the difficulties lie in the authorization of the requests. Therefore we are using a library Twitter4J to do the heavy lifting for us. Other libraries can be found here.

In your build.gradle.kts file, add this line to your dependencies.

implementation("org.twitter4j", "twitter4j-core", "4.0.7")

Now add these functions to Tweet.kt

import twitter4j.TwitterFactory
import twitter4j.conf.ConfigurationBuilder

fun consumerKey() = "<API key>"
fun consumerSecret() = "<API secret key>"
fun accessToken() = "<Access token>"
fun accessTokenSecret() = "<Access token secret>"

fun sendTweet(message: String){
    val cb = ConfigurationBuilder()
    cb.setOAuthConsumerKey(consumerKey())
    cb.setOAuthConsumerSecret(consumerSecret())
    cb.setOAuthAccessToken(accessToken())
    cb.setOAuthAccessTokenSecret(accessTokenSecret())
    val twitter = TwitterFactory(cb.build()).instance
    twitter.updateStatus(message)
}

DynamoDB for our Kotlin Lambda function

Gaute Meek Olsen — Wed, 10 Jun 2020 11:09:16 +0000

This article is part of the Twitter bot with Kotlin in AWS series showing how I created a Twitter bot for Vue 3 updates. But this article works as an independent article on how to create a DynamoDB table and interact with it from Kotlin code.

If you haven't set up a Lambda function in Kotlin before, you can look here. Also remember to give your role running the code permission to access DynamoDB.

Now it's time to set up our DynamoDB table, add some items, and write code to read and update them.

Table setup

First, let's create our table through the AWS CLI. We are specifying our required partition key with the --attribute-definitions and --key-schema, saying that the attribute "Project" is a string and HASH for the partition key.

aws dynamodb create-table --table-name twitter-bot-vue-3 --attribute-definitions AttributeName=Project,AttributeType=S -
-key-schema AttributeName=Project,KeyType=HASH --billing-mode PAY_PER_REQUEST

For my use case, I want an item for each changelog file I'm looking up. They will be defined beforehand, so you can add them through the AWS console, AWS CLI or code if you want. This is how to add an item with the CLI. (for Linux based systems, you can change to single quotes ' on the start and end and skip the escaping of quotes).

aws dynamodb put-item --table-name twitter-bot-vue-3 --item "{\"Project\": {\"S\": \"Vue 3\"}, \"Changelog\": {\"S\": \"
https://github.com/vuejs/vue-next/blob/master/CHANGELOG.md\"}, \"LastRecordedChange\": {\"S\": \"\"}}"

Dependencies

Now go to the build.gradle.kts file and add these two dependencies (add them to the existing dependencies in your file). You can find the latest version here.

dependencies {
    implementation(platform("software.amazon.awssdk:bom:2.13.18"))
    implementation("software.amazon.awssdk:dynamodb")
}

Read and update items

I have the file ProjectsDB.kt which has two methods that interact with the DynamoDB table.

import software.amazon.awssdk.services.dynamodb.DynamoDbClient
import software.amazon.awssdk.services.dynamodb.model.*

data class Project(val name: String, val changelog: String, var lastRecordedChangelog: String)

const val tableName = "twitter-bot-vue-3"

fun readProjects(): List<Project>{
    val client = DynamoDbClient.create()

    val result = client.scan {
        it.tableName(tableName)
    }
    return result.items().map {
        Project(it["Project"]!!.s(), it["Changelog"]!!.s(), it["LastRecordedChange"]!!.s())
    }
}

fun updateProjectLastRecordedChange(project: Project){
    val client = DynamoDbClient.create()

    val key = mutableMapOf(Pair("Project", createAttributeValueString(project.name)))
    val updates = mutableMapOf(Pair("LastRecordedChange", AttributeValueUpdate
        .builder()
        .value(createAttributeValueString(project.lastRecordedChangelog))
        .action(AttributeAction.PUT)
        .build()
    ))

    client.updateItem {
        it.tableName(tableName)
        it.key(key)
        it.attributeUpdates(updates)
    }
}

fun createAttributeValueString(value: String): AttributeValue = AttributeValue.builder().s(value).build()

Updating the Lambda function

First, we need a function that checks the latest update to the changelog files in the Vue repositories.

import java.net.URL

fun checkChangelog(url: String): String{
    val rawGithubUrl = url.replace("https://github.com/", "https://raw.githubusercontent.com/").replace("blob/", "")
    val text = URL(rawGithubUrl).readText()
    return text.lines().first()
}

Now let's update the Lambda function with some logic.

fun runLambda(){
    val projects = readProjects()
    for(project in projects){
        val latestChange = checkChangelog(project.changelog)
        if(latestChange != project.lastRecordedChangelog){
            project.lastRecordedChangelog = latestChange
            // tweet(project)
            updateProjectLastRecordedChange(project)
        }
    }
}

That's it. If you want a temporary tweet function that prints the tweet message you can use this.

fun tweet(project: Project){
    val regex = """\[.+]""".toRegex()
    val version = regex.find(project.lastRecordedChangelog)?.value?.trim('[', ']')
    val message = "${project.name}, ${if(version.isNullOrEmpty()) "new version" else version} is out! (${project.changelog}) #VueJS"
    println(message)
}