DEV Community: Florian Kapfenberger

Password protect your (Vercel) site with Cloudflare Workers

Florian Kapfenberger — Sat, 03 Apr 2021 00:00:00 +0000

I am working on a new project where I already setup the deployment pipeline. The app has a backend and a frontend. The frontend is built with Next.js and therefore I deployed it to Vercel. Because the state of the current project is not publishable I want to protect the access with a password. Vercel offers this option, but you need to have the Pro plan that costs 20$ per month.

Alright that would not be too bad, but wait why is there a small info icon next to password protection 🤔

This is way out of my budget and I don't understand why Vercel is selling this feature with such a high price tag. Netlify for example offers password protection too, but they have it included in the Pro plan for 19$ / month and no additional costs.

So I thought maybe I can add password protection with something else. And yes there is a solution: Cloudflare Workers!

To learn about Cloudflare Workers (CW) I would suggest you look at the docs. Basically, they are similar to serverless functions, but without the downsides.

Let's protect our site with Cloudflare Workers

First of all your domain needs to be managed by Cloudflare to be used together with CW. If you have this working it is just a matter of minutes to have your site protected.

To create a CW you need to go to your Dashboard and select Workers on the right side of the navigation.

Next, you click Create a Worker and you should be shown the editor where we can write our code.

Of course, I was not the first one that had the idea to use CW to protect their site with it. After a quick Google search, I found this Repository on GitHub by dommmel.

Thanks to his efforts in creating this worker it was as easy as copying the contents of the index.js file and past it into the editor.

The only things you need to change now are the NAME, and PASS at the top. Between lines 76 and 77 you paste the following code:

  const [protocol,,domain] = request.url.split('/');

  if(request.url.startsWith(`${protocol}//${domain}/.well-known/acme-challenge`)) {
    return fetch(request)
  }

This will allow Vercel to still be able to generate Let's Encrypt certificates.

After that your code should look like this with your custom username and password:

const NAME = "CUSTOM_USER"
const PASS = "SUPER_SECRET_PASSWORD"

/**
 * RegExp for basic auth credentials
 *
 * credentials = auth-scheme 1*SP token68
 * auth-scheme = "Basic" ; case insensitive
 * token68     = 1*( ALPHA / DIGIT / "-" / "." / "_" / "~" / "+" / "/" ) *"="
 */

const CREDENTIALS_REGEXP = /^ *(?:[Bb][Aa][Ss][Ii][Cc]) +([A-Za-z0-9._~+/-]+=*) *$/

/**
 * RegExp for basic auth user/pass
 *
 * user-pass   = userid ":" password
 * userid      = *<TEXT excluding ":">
 * password    = *TEXT
 */

const USER_PASS_REGEXP = /^([^:]*):(.*)$/

/**
 * Object to represent user credentials.
 */

const Credentials = function(name, pass) {
  this.name = name
  this.pass = pass
}

/**
 * Parse basic auth to object.
 */

const parseAuthHeader = function(string) {
  if (typeof string !== 'string') {
    return undefined
  }

  // parse header
  const match = CREDENTIALS_REGEXP.exec(string)

  if (!match) {
    return undefined
  }

  // decode user pass
  const userPass = USER_PASS_REGEXP.exec(atob(match[1]))

  if (!userPass) {
    return undefined
  }

  // return credentials object
  return new Credentials(userPass[1], userPass[2])
}

const unauthorizedResponse = function(body) {
  return new Response(
    body, {
      status: 401,
      headers: {
        "WWW-Authenticate": 'Basic realm="User Visible Realm"'
      }
    }
  )
}

/**
 * Handle request
 */

async function handle(request) {
  const [protocol,,domain] = request.url.split('/');

  if(request.url.startsWith(`${protocol}//${domain}/.well-known/acme-challenge`)) {
    return fetch(request)
  }

  const credentials = parseAuthHeader(request.headers.get("Authorization"))
  if ( !credentials || credentials.name !== NAME ||  credentials.pass !== PASS) {
    return unauthorizedResponse("Unauthorized")
  } else {
    return fetch(request)
  }
}

addEventListener('fetch', event => {
  event.respondWith(handle(event.request))
})

Now that we have our worker finished we need can change the name at the top left and with Save and Deploy we can (duh..) deploy it!

The worker is now ready to be used and we can assign it to a custom domain. On Cloudflare open up your domain and go to the Workers tab.

Click Add Route and type the text from the placeholder into the input field. If your domain is example.com add *example.com/*. In the select box, select the newly created CF. This instructs Cloudflare to use this worker whenever the URL matches the route we defined.

On the DNS tab make sure the domain is proxied through Cloudflare (orange cloud), otherwise, the CW would not work.

Make it work with Vercel

Now that the Cloudflare Worker is set up and running, we still need to change a few settings in Cloudflare to make it work nicely with Vercel.

The first thing we check is on the SSL/TLS tab in Cloudflare. Make sure you have it set to Full (strict) otherwise you will get a Too many redirects error and can't access your site.

Finally, we need to tell Cloudflare that the route /.well-known/* can be accessed with HTTP instead of HTTPS. This is used by Vercel to create a Let's Encrypt certificate.

To do this we need to go to the Page Rules tab on Cloudflare and create a new Page Rule. Enter in the first input field /.well-known/* and select in the select field SSL and Off. With Save and Deploy your rule will be applied.

We are finally done 🥳

Now your page should be protected using Basic HTTP authentication with the username and password you provided in the Cloudflare Worker.

I have set up a simple demo site where I protected an HTML site hosted on Vercel using the Cloudflare Worker we just created.

The site is accessible at https://protected-site-example.phiilu.com/

Username: phiilu
Password: shiba-inus-are-awesome!

Conclusion

Thanks to Cloudflare Workers we can simply add password protection to our sites!

In my opinion, such a feature should not cost 150$ a month and I hope that Vercel will rethink this choice and adds this as an included feature in the Pro plan.

Allow your mobile visitors to easier share your articles with the Web Share API

Florian Kapfenberger — Sun, 14 Feb 2021 00:00:00 +0000

Sharing websites is a fundamental process of the web and providing a way to make it as easy as possible can allow users to easily share the work you created.

The Web Share API allows the user to use the native share mechanism of the operating system. You probably already have seen this when you want to share an image or video using a native app.

Because this is using the native sharing mechanism of the OS the Web Share API is not available in all browsers and OS. At the time of writing it is only available in Chrome (only Windows and Chrome OS), Edge, Safari, iOS Safari, Chrome for Android, Firefox for Android, and Samsung Internet. You can find an up-to-date list on caniuse.com.

Use the Web Share API if it is available

Browsers that support the Web Share API will expose the share method on the navigator object. There is also a canShare method, but this method is not available in all browsers. So the best way to check if it is available would be to check if the share method exists.

if (window.navigator.share) {
  // Web Share API available
} else {
  // Use a fallback method
}

Share URLs with the Web Share API

Using the Web Share API is very easy. In its basic version, it is just one function that you call and that's it.

const data = {
  title:
    "Allow your mobile visitors to easier share your articles with the Web Share API",
  url:
    "https://phiilu.com/share-articles-on-social-media-with-the-web-share-api",
  text:
    "Allow your mobile visitors to easier share your articles with the Web Share API by Florian Kapfenberger (@phiilu)",
};
const sharePromise = window.navigator.share(data);

The share method takes an object as its first argument. Here you can define what should be prefilled by the share method that will be used later on.

title: The title that should be prefilled (if supported)
url: The website/URL you want to share
text: A custom text which should be prefilled before sharing

Share files with the Web Share API

By adding a files array to the share function you can allow the user to share images too.

Unfortunately, I can't get it working with the devices I have. I tested it with an iPhone 12 (iOS 14) and on macOS.

Here is a CodeSandbox if you want to test it on your device. If the code is not quite right, maybe fork it and write me on Twitter and I will update it!

How I added the Web Share API to my blog

If you are reading this on my blog and you are using a supported device you can test it out right now!

If supported, you should see a button Share Anywhere. Try it out :)

In React I created a new Share component. If the share method is available I will show the Share Anywhere button additional to the "Fallback" share buttons.

You can find the up-to-date file on my GitHub.

import * as React from 'react';
import toast from 'react-hot-toast';

import Button from '@components/Button/Button';

function Share({ title, url, onClick }) {
  const [isSareApiAvailable, setIsSareApiAvailable] = React.useState(false);

  React.useEffect(() => {
    setIsSareApiAvailable(!!window.navigator.share);
  }, []);

  function handleSocialShare() {
    try {
      const res = window.navigator.share({
        title,
        text: `${title} by Florian Kapfenberger (@phiilu)`,
        url
      });

      toast.promise(res, {
        loading: {
          title: 'Share the post with the world',
          text: 'Select how you want to share the post',
          dismiss: false
        },
        success: {
          title: 'Shared successfully',
          text: 'Thank you for sharing my post!',
          dismiss: true
        },
        error: { title: 'So close', text: 'Oh okay.. Maybe next time :)', dismiss: true }
      });
    } catch (error) {
      // do nothing
    }
  }

  return (
    <ul className="space-y-2 sm:items-start sm:space-x-2 sm:space-y-0 xl:space-y-2 sm:flex xl:space-x-0 xl:block">
      {isSareApiAvailable && (
        <li className="w-full">
          <Button
            tracking={{
              event: 'click',
              value: 'Share Anywhere clicked',
              name: 'Share Anywhere clicked'
            }}
            variant="secondary"
            onClick={handleSocialShare}>
            Share Anywhere
          </Button>
        </li>
      )}
      <li className="w-full">
        <Button
          tracking={{
            event: 'click',
            value: 'Share on Twitter clicked',
            name: 'Share on Twitter clicked'
          }}
          variant="twitter"
          onClick={onClick(
            `https://twitter.com/share?text=${title} via @phiilu&url=${url}`,
            'twitter-share',
            'width=550,height=235'
          )}>
          Share on Twitter
        </Button>
      </li>
      <li className="w-full">
        <Button
          tracking={{
            event: 'click',
            value: 'Share on Hacker News clicked',
            name: 'Share on Hacker News clicked'
          }}
          variant="hackernews"
          onClick={onClick(
            `https://news.ycombinator.com/submitlink?u=${url}&t=${title}`,
            'hn-share',
            'width=550,height=350'
          )}>
          Share on Hacker News
        </Button>
      </li>
    </ul>
  );
}

export default Share;

Conclusion

With just a few lines of code, you can improve the sharing process for your mobile users, and for unsupported browsers, you can provide a fallback method. Hopefully, this will not be needed anymore in the future.

Sources:

Accessing the Clipboard in Javascript using the Clipboard API

Florian Kapfenberger — Sun, 07 Feb 2021 00:00:00 +0000

Welcome to my new series where I will write about built-in Browser-APIs and demonstrate how you can use them today. Most of the APIs I will write about will be highly experimental and still will have the "Draft" status. So don't expect a wide browser adoption and only try this with modern browsers. You can always check caniuse.com or MDN for up-to-date information about the compatibility in the browsers.

The first API I want to write about is the Clipboard API. With the Clipboard API, you can read from and write to the Clipboard of the user. It is not something new as you already can do this with the document.execCommand().

The problem is that execCommand() is a synchronous operation and will block the page while calling it. Also, a browser might implement the document.execCommand('copy') function differently than another one. This could lead to inconsistent results. You can read more about the old way here.

The Clipboard API tries to solve this by being asynchronous and offer a more consistent API across browsers.

Permissions & Security for accessing the Clipboard

With the new Clipboard API, you will need the clipboard-read permission to read the user clipboard content. When you want to write to the clipboard you will need to have the clipboard-write permission. Although you will need this permission, the browser grants you the permission automatically when the page is in the current tab.

The Permissions API is not supported in all browsers yet. Safari for example did not implement it, however, you can access the Clipboard API in Safari without requesting permissions. Firefox on the other hand supports the Permissions API but did not implement the clipboard-read and clipboard-write permissions yet. You already see that there might be issues if you start using it today.

Also, you will need to have a secure context while accessing the Clipboard API. This usually means you can only access this API when you are on localhost or a webpage with a valid SSL/TLS certificate.

You can access the Clipboard API on the navigator object. If navigator.clipboard is undefined, then the Clipboard API won't be available on your browser.

Listening for Clipboard Events

If you want to act on if the user is copying, pasting, or cutting you can very easily get those events by attaching an event listener to the document. The names for these events are copy, paste, and cut events.

function handlePasteEvent(event) {
   handleClipboard();
  event.preventDefault();
}

document.addEventListener("paste", handlePasteEvent);

The event that gets passed to the handler function will implement the ClipboardEvent interface. This interface will allow you to override the content of what is being passed to or from the Clipboard. Very handy if you want to format or remove the formatting of text that is being copied.

Reading from the Clipboard
For browsers that implement the Permissions API, you should first check if you have permission to read from the Clipboard.

I tried it out without querying for permission and it was working fine. This could obviously change in the future.

navigator.permissions.query({ name: "clipboard-read" }).then(function (result) {
  if (result.state === "granted") {
    // you can read from the clipboard
  }
});

Text

If you only want to read text you can use the convenience method readText(). This method will return a Promise with the contents of the clipboard.

const text = await navigator.clipboard.readText()

Images

If you want to read images from the clipboard you will have to use the read() method. This method will return a Promise containing an array of ClipboardItem objects.

Here is an example of how you can read an image from the clipboard:

navigator.clipboard
  .read()
  .then(async (clipboardItems) => {
    for (const clipboardItem of clipboardItems) {
      for (const type of clipboardItem.types) {
        const blob = await clipboardItem.getType(type);
        if (type === "image/png") {
          const imageUrl = URL.createObjectURL(blob);
          console.log(imageUrl); // blob:http://localhost:3000/e53d5d55-c674-454b-9587-2501a4396eed
        }
      }
    }
  })
  .catch((error) => {
    console.log(error);
  });

Basically what we are doing here is to iterate over each item and check the MIME type and if it is a PNG image we will create an object URL that can be passed to the src attribute of an image to display it.

The URL.createObjectURL() static method creates a DOMString containing a URL representing the object given in the parameter. The URL lifetime is tied to the document in the window on which it was created. The new object URL represents the specified File object or Blob object. - MDN

Writing to the Clipboard

As I already said you won't need to ask for the clipboard-write permissions because the browser will grant it automatically if the page is in the active tab.

Text

If you just want to write text to the clipboard you can use the convenience writeText() method. Just pass the text you want to put on the clipboard as the first argument and you are done!.

await navigator.clipboard.writeText("Hello World");

Images

If you want to write images to the clipboard you will have to use the write() method. This method takes an array of ClipboardItem objects as the first argument.

Here is an example of how you can read an image from the clipboard:

const result = await fetch(
  "https://images.dog.ceo/breeds/pembroke/n02113023_15926.jpg"
);
const blob = await result.blob();
await navigator.clipboard.write([
  new ClipboardItem({
    "image/png": blob,
  }),
]);

Basically will have to create a Blob out of that image. The easiest way I know how to achieve this is to call the URL of the image with fetch.

Using the API with React

I am mostly developing with React nowadays, therefore I want to show how I used it with it. There is not much different than using it without React, because we won't need access to any DOM elements and can do everything inside an event handler or useEffect hook.

I created an MVP of a Clipboard Manager without persistence, but it demonstrates most of the Clipboard API features very well.

It is using Next.js and Tailwind CSS. I kept everything inside the index.js file. If you have any questions don't hesitate to ask me on Twitter.

Conclusion

At the time of writing the support for the Clipboard API is still limited and I think I won't be using it in production right now.

I think it is important that the web will get more consistent APIs that are supported in most browsers and do the hard work for us. Thinking about how to copy a text to the clipboard should not be complicated.

The Clipboard API will finally deliver a consistent API without using hacky methods. If you want to get more examples of how this API will help you, check out the W3C Use Cases section.

Sources:

MDN: https://developer.mozilla.org/en-US/docs/Web/API/Clipboard
web.dev: https://web.dev/async-clipboard/
W3C: https://w3c.github.io/clipboard-apis/

Use your Raspberry Pi as a Docker server with docker-machine

Florian Kapfenberger — Thu, 07 Jan 2021 00:00:00 +0000

If you have a slow computer or you somehow can't install Docker using the official Docker Desktop installer, you might want to use your Raspberry Pi (or any other server you have) to be your dedicated Docker server.

Recently I had the problem that I could not install Docker Desktop in my VM that was running on my Unraid server. The problem is that on Ryzen CPUs nested virtualization is not (yet) supported on the OS that I used. Nested virtualization means that you can run a VM inside a VM.

So I could not use the official Docker for Mac installer to install Docker, I had to find another way.

The answer to this was docker-machine.

What is docker-machine?

docker-machine is a CLI program from Docker that allows you to use docker as you would normally, but execute the commands on a server instead. This is useful if you have multiple servers running docker and you can control them all by just changing the docker environment with one command docker-machine env YOUR_ENV.

If you don't have docker-machine installed, try to install it with your OSs package manager or download it from GitHub.

On macOS you can install it like this from brew:

brew install docker-machine

Even if you can't install Docker Desktop, you should still be able to download the docker CLI, you will need it.

Preparing our server

Before we use docker-machine we need to make some configurations on the server first.

I would recommend creating a new user that will be used to login with docker-machine.

adduser docker-machine

Next, we need to allow the new user to execute commands with sudo, but with the addition that the user can use it without entering its password. This might be a security risk, so make sure the password of the docker-machine user is strong and your server is properly hardened.

Edit the sudoers file by writing visudo into your terminal and append a new line after the entry for root.

docker-machine ALL=(ALL) NOPASSWD: ALL

Save and quit the editor.

The docker-machine generic driver does not recognize the raspbian OS, so we need to trick it, by temporarily changing the ID field in the /etc/os-release to debian.

#/etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Finally, you will need to set up public-key authentication on your server with the docker-machine user. I have documented the process on how you can do this in another article. Scroll down to the "Skip passwords with public key authentication" section and follow the steps to set it up.

Now your server should be ready to install docker using docker-machine.

Install docker on your server with docker-machine

Now that you have the docker-machine CLI installed on your machine, we can use it to install docker on the server using just one command.

docker-machine create \
  --driver generic \
  --generic-ip-address=192.168.1.13 \
  --generic-ssh-user=docker-machine \
  --generic-ssh-key ~/.ssh/id_rsa \
  rpi

Running pre-create checks...
Creating machine...
(rpi) Importing SSH key...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with debian...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env rpi

That's it you should now have docker installed on your server!

Change your docker environment with docker-machine

Now if you want to execute docker commands on the server you just need to change your docker environment with the following command:

eval $(docker-machine env rpi)

If you always want to use the server environment, put the line into your .bashrc or .zshrc file.

Start using Docker

After changing the environment with docker-machine you can use docker the same way that you would do normally!

docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Useful docker-machine commands

docker-machine ls - list all environments
docker-machine rm ENV - deletes an existing environment
docker-machine ip ENV - get the IP address of the server
docker-machine ssh ENV - open an SSH connection to the server

Conclusion

docker-machine is a cool tool and it is very easy to set up and use. I find this approach very interesting. For now, the setup is working quite well and I did not see any drawbacks to it.

The only different thing is that I will have to use the IP of my server instead of localhost if I want to access services running on docker.

My thoughts on completing the React Hooks workshop from Epic React by Kent C. Dodds

Florian Kapfenberger — Tue, 22 Dec 2020 00:00:00 +0000

I finally got some time and finished the second workshop of the Epic React by Kent C. Dodds course. It's been a while since I wrote my last article about the first workshop, but here I am and trying to summarize my experience and learnings from the second workshop React Hooks.

Some people are already finishing the whole Epic React course and it took someone over 3 weeks by spending 4 hours each day. That's 84 hours or 3 1/2 days spent on the whole course. I think I need to increase my speed on completing more workshops 😅 . This just shows how much content you get with this course.

Now to my thoughts on the React Hooks workshop!

What will I learn?

After completing the React Hooks workshop, you will be familiar with the most used hooks in React and how they work. useState, useRef, and useEffect will be explained and how you use them by programming a Tic-Tac-Toe game.

Additionally, you will learn in which order hooks will run. Which useEffect will run first? What about my lazy initializer in my useState? When does my component get rendered? All these questions will get answered in this workshop.

If you are still creating React components using classes and find hooks confusing, you will also get some exercises on how you can convert class components to function components with hooks.

You will learn the concept of lifting state, so you will know when you need to move a certain state up to the parent.

Finally, you will learn how you can fetch data from an external API with React hooks and learn about ErrorBoundary.

What did I learn?

I am already quite familiar with React hooks, but there are still somethings I learned in this workshop.

I was not aware of the lazy initializer function that you can pass to the useState function.

Instead of doing this:

const [data, setDate] = useState(getSomeComplexDefaultState());

You can write it like this:

const [data, setDate] = useState(() => getSomeComplexDefaultState());

This is very useful for complex initial state. Using lazy initializers can in certain situations increase your web performance.

The next thing I never looked up before, was the hook flow. Now I am more aware in which order my hooks in nested components run!

Finally, in the last module about data fetching, I learned a lot about the React ErrorBoundary component and using the react-error-boundary package. Kent C. Dodds has a blog post on his website, if you want to learn about it.

Conclusion

So after my second workshop, I have to say that I already know a lot about React, but at the same time, I don't. I am very glad that I bought this course and how much I gain from it. Knowing those little details is very valuable and can help you a lot when you encounter strange bugs in your code.

The next workshop is Advanced React Hooks. After finishing it you will hear from me again 😄

Generate a sitemap for your Next.js site and submit it to Google

Florian Kapfenberger — Mon, 07 Dec 2020 00:00:00 +0000

One way to improve your SEO is by submitting a sitemap to Google. Sitemaps are an XML representation of your website and its pages. It allows search engines to know what pages your website has, without crawling each link on your page first.

By submitting a sitemap to the Google Search Console you can help Google to better identify your pages and you can see problems if some of the URLs are not being indexed.

Most websites have sitemaps and you can easily find them by looking at the robots.txt and searching for the Sitemap entry.

Here are a few sitemaps by popular websites:

Google: https://www.google.com/sitemap.xml
The Verge: https://www.theverge.com/sitemaps
NY Times: https://www.nytimes.com/sitemaps/new/news.xml.gz

Generating the sitemap.xml

With a CMS like WordPress, you usually get a sitemap generated by default, but if you created your site using Next.js you need to take care of this yourself.

Gladly this is not a too hard problem to solve and you probably can just copy my code with some adjustments and you should be fine.

As in my post where I described how to generate an RSS feed with Next.js, I will create a new file inside my lib folder and name it sitemap.js.

Here is the content of my sitemap.js:

const globby = require('globby');
const contentful = require('./contentful').default;
const { SitemapStream, streamToPromise } = require('sitemap');
const { Readable } = require('stream');
const fs = require('fs');

// pages that should not be in the sitemap
const blocklist = ['/newsletter-success', '/404'];

async function generateSitemap() {
  if (process.env.NODE_ENV === 'development') {
    return;
  }

  const baseUrl = process.env.BASE_URL;
  const pages = await globby([
    'src/pages/**/*{.js,.mdx}',
    '!src/pages/**/[*',
    '!src/pages/_*.js',
    '!src/pages/api'
  ]);

  // normal page routes
  const pageLinks = pages
    .map((page) => {
      const path = page
        .replace('pages', '')
        .replace('.js', '')
        .replace('.mdx', '')
        .replace('src/', '');
      return path === '/index'
        ? { url: '/', changefreq: 'daily', priority: 0.7 }
        : { url: path, changefreq: 'daily', priority: 0.7 };
    })
    .filter((page) => !blocklist.includes(page.url));

  // post routes
  const posts = await contentful.getEntries('post', { order: '-fields.publishedDate' });
  const postLinks = posts.map((post) => ({
    url: `/${post.slug}`,
    changefreq: 'daily',
    priority: 0.7
  }));

  // tag routes
  const tags = await contentful.getEntries('tag');
  const tagLinks = tags.map((tag) => ({
    url: `/tag/${tag.slug}`,
    changefreq: 'daily',
    priority: 0.7
  }));

  const links = [...pageLinks, ...postLinks, ...tagLinks];
  const stream = new SitemapStream({ hostname: baseUrl });

  const xml = await streamToPromise(Readable.from(links).pipe(stream)).then((data) =>
    data.toString()
  );

  fs.writeFileSync('./public/sitemap.xml', xml);
}

export default generateSitemap;

I am using two external dependencies in this code, globby and sitemap, so you have to add them to your package.json.

yarn add -D globby sitemap

Let's breakdown the code into smaller pieces and let me explain what we are doing here.

// if (process.env.NODE_ENV === 'development') {
//   return;
// }

const baseUrl = process.env.BASE_URL;
const pages = await globby([
  "src/pages/**/*{.js,.mdx}",
  "!src/pages/**/[*",
  "!src/pages/_*.js",
  "!src/pages/api",
]);

First of all, we don't want to run this script when we are in development mode. For now, it is commented, because we need to test our function first, but after it is working you can uncomment it and it will only run in production.

Next, I defined some variables. baseUrl will be your domain of the website like in my case the value will be https://phiilu.com.

I am using a package named globby to get all the pages from the pages folder using a glob. It will return all the pages that are either .js or .mdx and not starting with [ or _ like you would use for dynamic pages or the _app.js. It won't return the files that are in the pages/api folder too.

const blocklist = ["/newsletter-success", "/404"];

// normal page routes
const pageLinks = pages
  .map((page) => {
    const path = page
      .replace("pages", "")
      .replace(".js", "")
      .replace(".mdx", "")
      .replace("src/", "");
    return path === "/index"
      ? { url: "/", changefreq: "daily", priority: 0.7 }
      : { url: path, changefreq: "daily", priority: 0.7 };
  })
  .filter((page) => !blocklist.includes(page.url));

Here I am creating an array of pageLinks that will be used to generate the sitemap. We only want to have the name of the file, so we are replacing the strings we won't need with an empty string. You can adjust the values for changefreq or priority for your needs. Those values will tell the search engine how often the page is changing and how important it is.

I also have some pages that I don't want to have inside the sitemap.xml, so I created a blocklist array and filter the pages that are in that array.

// post routes
const posts = await contentful.getEntries("post", {
  order: "-fields.publishedDate",
});
const postLinks = posts.map((post) => ({
  url: `/${post.slug}`,
  changefreq: "daily",
  priority: 0.7,
}));

// tag routes
const tags = await contentful.getEntries("tag");
const tagLinks = tags.map((tag) => ({
  url: `/tag/${tag.slug}`,
  changefreq: "daily",
  priority: 0.7,
}));

This part may not be necessary for you, depending on if you are using a headless CMS or not. I am using Contentful to store my posts and tags so I will have to fetch the data from there and create the postLinks and tagLinks array which will have the same structure as the pageLinks array.

const links = [...pageLinks, ...postLinks, ...tagLinks];
const stream = new SitemapStream({ hostname: baseUrl });
const xml = await streamToPromise(
  Readable.from(links).pipe(stream)
).then((data) => data.toString());
fs.writeFileSync("./public/sitemap.xml", xml);

Last but not least, I created the final links array which will contain all links that I want to have inside my sitemap.xml. To generate the sitemap.xml I am using a package named sitemap that will be doing the hard work of generating the correct XML content. Finally, the XML data will be saved to a file inside the public folder, so Next.js can serve it.

Generate the sitemap during the build

I will be using the same approach as with generating the RSS feeds. Importing the generateSitemap function in the pages/index.js file and adding it to the getStaticProps will make sure Next.js will call this function during the build of the pages/index.js page and generate the sitemap.

export async function getStaticProps() {
  // ...
  await generateRssFeed();
  await generateSitemap();
  return {
    props: { posts, ogImage, baseUrl },
  };
}

Submit the sitemap to Google

Finally, now that you have your sitemap.xml file, you can submit this file to the Google Search Console. After adding your domain to the Search Console you just have to go to the Sitemap page and submit the URL where your sitemap.xml file is available. After a few days, you will see how many pages of your sitemap are being indexed by Google and which ones will have some problems.

Conclusion

Generating a sitemap can be very useful for search engines and having one can only be a benefit. If you are using Next.js you need to take care of this yourself, but as I have shown you in this tutorial it is not very difficult as the hard part will be done by existing packages anyway.

Generate RSS feeds for your static Next.js blog

Florian Kapfenberger — Fri, 20 Nov 2020 00:00:00 +0000

On the internet, there are many ways to allow visitors to get updated for new content on your blog. A pretty common option is to offer a Really Simple Syndication or more commonly known as RSS feed.

An RSS feed is a standardized XML file that contains information about the website and about all articles. Many people like to use RSS readers like Feedly or Feeder to read blog posts, so it's a good idea to offer it!

Next to RSS there also exists the Atom and JSON feed. Atom should be the successor of RSS and has some improvements over RSS. JSON feed is basically RSS, but instead of using XML, it uses JSON.

In this tutorial, I will show you how to generate all 3 feeds in Javascript with the help of the feed package and add them to your static Next.js site. You can see the final results of the different feeds by clicking the link:

Generating RSS feeds

Inside the Next.js project create a new file where you want to put all the logic for generating RSS feeds. I created my file inside a lib folder and named it rss.js.

Next, add the feed package from npm to your project.

yarn add feed

Before writing any code, let's think about what we want to do. There are 3 steps we need to do:

create a new Feed instance and initialize it with common data about the blog
loop over all articles and add them to the feed
generate the web feeds that you want and store them in files

Sounds not too complicated so let's get started.

Creating a new Feed instance

Inside the rss.js file create a new function and name it something like generateRssFeed.

Next, we want to initialize the Feed instance.

const baseUrl = "https://phiilu.com";
const date = new Date();

const feed = new Feed({
  title: `Phiilu's Blog`,
  description: "Welcome to my blog!",
  id: baseUrl,
  link: baseUrl,
  language: "en",
  image: `${baseUrl}/images/logo.svg`,
  favicon: `${baseUrl}/favicon.ico`,
  copyright: `All rights reserved ${date.getFullYear()}, Florian Kapfenberger`,
  updated: date,
  generator: "Next.js using Feed for Node.js",
  feedLinks: {
    rss2: `${baseUrl}/rss/feed.xml`,
    json: `${baseUrl}/rss/feed.json`,
    atom: `${baseUrl}/rss/atom.xml`,
  },
  author,
});

This is meta-information about your blog. You can define the title, description, image, favicon, and other data that will be used inside RSS readers.

Add articles to the feed

Now you need to gather all articles from your blog and add them to the feed.

I am using Contentful as my headless CMS, so I am fetching the data using the Contentful API.

If you are using MDX you probably need to read the files from your disk and extract the frontmatter data.

const author = {
  name: "Florian Kapfenberger",
  email: "hey@phiilu.com",
  link: "https://twitter.com/phiilu",
};

const posts = await contentful.getEntries("post", {
  order: "-fields.publishedDate",
});

posts.forEach((post) => {
  const url = `${baseUrl}/${post.slug}`;
  feed.addItem({
    title: post.title,
    id: url,
    link: url,
    description: post.description,
    content: markdown.toHTML(post.content),
    author: [author],
    contributor: [author],
    date: new Date(post.rawDate),
  });
});

Here I am fetching my articles using the Contentful API and adding them to the Feed instance. My articles are written in Markdown, but RSS feeds expect them to be in HTML. This is why I am using the markdown package to transform my Markdown into HTML for the content property.

Generate the different feed files

Next.js will serve files from the public folder, so this is the location where we want to store the different feeds.

fs.mkdirSync("./public/rss", { recursive: true });
fs.writeFileSync("./public/rss/feed.xml", feed.rss2());
fs.writeFileSync("./public/rss/atom.xml", feed.atom1());
fs.writeFileSync("./public/rss/feed.json", feed.json1());

Thanks to feed we can generate all three formats by just calling the correct function!

Steps summary

After these 3 steps, your rss.js file should look similar to this one.

const Feed = require('feed').Feed;
const contentful = require('./contentful').default;
const markdown = require('markdown').markdown;
const fs = require('fs');

async function generateRssFeed() {
  if (process.env.NODE_ENV === 'development') {
    return;
  }

  const baseUrl = process.env.BASE_URL;
  const date = new Date();
  const author = {
    name: 'Florian Kapfenberger',
    email: 'hey@phiilu.com',
    link: 'https://twitter.com/phiilu'
  };

  const feed = new Feed({
    title: `Phiilu's Blog`,
    description: 'Welcome to my blog!',
    id: baseUrl,
    link: baseUrl,
    language: 'en',
    image: `${baseUrl}/images/logo.svg`,
    favicon: `${baseUrl}/favicon.ico`,
    copyright: `All rights reserved ${date.getFullYear()}, Florian Kapfenberger`,
    updated: date,
    generator: 'Next.js using Feed for Node.js',
    feedLinks: {
      rss2: `${baseUrl}/rss/feed.xml`,
      json: `${baseUrl}/rss/feed.json`,
      atom: `${baseUrl}/rss/atom.xml`
    },
    author
  });

  const posts = await contentful.getEntries('post', { order: '-fields.publishedDate' });

  posts.forEach((post) => {
    const url = `${baseUrl}/${post.slug}`;
    feed.addItem({
      title: post.title,
      id: url,
      link: url,
      description: post.description,
      content: markdown.toHTML(post.content),
      author: [author],
      contributor: [author],
      date: new Date(post.rawDate)
    });
  });

  fs.mkdirSync('./public/rss', { recursive: true });
  fs.writeFileSync('./public/rss/feed.xml', feed.rss2());
  fs.writeFileSync('./public/rss/atom.xml', feed.atom1());
  fs.writeFileSync('./public/rss/feed.json', feed.json1());
}

export default generateRssFeed;

Generate RSS feeds during the build

Now that we have our little utility function to generate the RSS feeds, we need to define when the RSS feeds should get generated.

To keep it simple I am importing this function in the pages/index.js file and adding it to the getStaticProps. Doing it this way will make sure Next.js will call this function during the build of the pages/index.js page and generate the feeds.

export async function getStaticProps() {
  // ...

  await generateRssFeed();

  return {
    props: { posts, ogImage, baseUrl },
  };
}

The cleaner way probably is to call this function somewhere in the webpack config by using a plugin or something like this.

Conclusion

There is a whole market out there for RSS readers and offering RSS for your readers can be very valuable to grow your audience.

Thanks to the feed package it is very easy to generate RSS feeds with Javascript too.

What do you think about RSS? Do you offer RSS on your blog? Are you using RSS readers and if so which one do you use? I am very interested in what you think, lets chat in the comments below!

Here are 6 Google Analytics alternatives that care about your privacy

Florian Kapfenberger — Mon, 09 Nov 2020 21:44:41 +0000

Photo by Edho Pratama on Unsplash

I tried out many analytics SaaS on my blog to find a great alternative for my tracking needs. I was using Google Analytics before, but now I want to use something that cares about my and other user's privacy.

My feature requests are very simple:

tracks page views without using cookies
privacy-focused
can track events
offers a way to use a custom domain to deny ad blockers
(optional) possibility to share the dashboard via a link

Most alternatives I found offer (most of) those features and even more. Before going over each alternative, I want to talk about why you might want to get rid of Google Analytics.

Why you should not use Google Analytics

Privacy. Google Analytics is a free service from Google. When something is free, you probably will be paying with something else. In Google's case, you will be paying with the data that you collect for them. Google is a big ad company that generates money by using your data. So this is one big reason not to use Google Analytics. I try to reduce my services that depend on Google and I always look for alternatives. Pieter Levels created a website where you can find alternatives for Google services called No More Google.

Cookies. Another reason why you might not want to use Google Analytics is that you will have to show those ugly cookie banners. If you are using services that put cookies on your site you will have to show those cookie banners to stay GDPR compliant. You will also have to put in place the option to opt-out of it and have a policy page. It's not that difficult to implement, but you could skip it.

Complex Dashboard. Google Analytics is way more than just tracking pageviews, therefore it has lots of features and the dashboard is not that easy to use. If you only want to track page views and maybe some events then I think Google Analytics is overkill for your use case.

Ad-Blockers. If you use Google Analytics you will not track all page views, because of the well-known domain name that it is blocked by most ad blockers. With other services, you can use a custom domain that is not blocked by ad blockers and you will get way more analytics data.

In the next sections, I want to give you an overview of 6 different analytics services that I can recommend without blinking an eye. Some of them are free to use and open-source, others will require a subscription.

Plausible Analytics

https://plausible.io/

Simple and privacy-friendly alternative to Google Analytics

Plausible Analytics is one of my favorite Analytics services I have used.

The UI is very modern and clean, it is privacy-friendly, can track multiple websites in one account, you can self-host it because it is open source, it allows the use of custom domains, sends you weekly or monthly reports and much more.

It ticks almost every checkbox of the features I need and if it is not currently implemented it will be soon. Plausible Analytics is working on more features and they have a public roadmap where you can see what will be added next!

I like to host services myself therefore I love that Plausible is open source and that I can host it myself. Of course, with self-hosting, I need to take care that the server is always online and the database is backed up regularly.

If I don't want to do that, I can let Plausible take care of it for a very fair price depending on the amount of traffic you have. They even offer a free trial of 30 days without requiring a credit card.

Fathom

https://usefathom.com/ (affiliate link)

Your website analytics should be simple, fast and privacy-focused

Fathom is one of the first analytics SaaS that focused on privacy.

It offers a clean and easy to use dashboard with only the information that you will need. You can use your Fathom account for multiple websites, the pricing is fair, you can use custom domains and you can track events by creating goals.

They are currently working on new branding and as it looks like a new dashboard with additional features like API access.

Additional to analytics Fathom also provides website monitoring. You can get notified if your website ever goes offline and that is included with your account for FREE!

Recently Fathom published the Phantom Analyzer. A simple tool where you can check if a site is using some shady trackers or if your site is trackers free.

If you care about privacy then I would suggest that you check out Fathom. They offer 7 days free trial and if you decide to delete your account, they make sure that your data is gone for good!

Umami

https://umami.is/

own your website analytics

Umami is an open-source alternative to Google Analytics developed by Mike Cao that you can host yourself.

Umami offers simple page and event tracking with a beautiful minimalistic UI. You can add as many websites as you like. Create a shareable link for the analytics and see real-time visitors.

I like how simple and clean the dashboard from Umami is. New features and improvements are added regularly. Hosting is very easy too and documented well!

Hosting the analytics service yourself will allow you full control of the data that is tracked of your visitors. As I already said if tracking data is critical for you, you should probably invest some time into setting up some backup strategies. Therefore Umami is "free", but you will have to pay with your own time and stress I guess 😅

Simple Analytics

https://simpleanalytics.com/ (affiliate link)

Simple, clean, and friendly analytics

I found Simple Analytics via Twitter and thought I might check it out too.

Simple Analytics offers privacy-friendly analytics that won't sell your data. It has a strong focus on being privacy-friendly and transparent with what it will track and store.

You will see all the important information you need at a glance with a beautiful dashboard. Event tracking is possible but stated as being highly experimental.

Simple Analytics have a public roadmap too, where you can see what will be implemented next and even request missing features!

Splitbee

https://splitbee.io/

Your friendly all-in-one analytics & conversion tool.

Splitbee is a bit different than the other analytics SaaS, as it offers additional features too that are not only focusing on analytics.

Splitbee offers a modern UI where you can see your top pages and top sources in addition to how many unique users you got in that period.

Additional to analytics you can do automation where you can send emails or trigger a webhook when a certain event or pageview happens.

You can do "experiments" like A/B testing or Redirect testing.

If you are using Splitbee with your app where users sign in you can identify users so you can track them and see how they interact with your app.

I haven't used many of the offered features besides page and event tracking. I could add automation where I send myself a message when some successfully sign up for my newsletter, which is neat.

Matomo

https://matomo.org/

Google Analytics alternative that protects your data and your customers' privacy

Matomo is probably the most known alternative to Google Analytics. I have used it for some time, but I am not a big fan of it because it offers way too much functionality for my use case.

It is probably the most mature alternative to Google Analytics as it offers a great selection of plugins and settings. If you just need to track page views with some events then I think Matomo might not be the best choice.

Like Plausible Analytics and Umami, you can host Matomo yourself which I think is great!

Conclusion

You see that there are many great alternatives for Google Analytics out there that focus on privacy and won't sell your data.

I think there is no need to continue using Google Analytics for simple projects that won't use any other Google Services.

If you care about privacy you should probably switch to one of the mentioned alternatives.

Every single product I mentioned offer great features and I can recommend them all. I wish I could use all of them, but that would be counterproductive to bloat a website with all possible trackers 😁

I think Plausible Analytics is the one that speaks to me the most. I like the UI and the reason that it is open source. They have a great product and offer a good service with their hosted solution which allowed them to reach an MRR of over 6K$.

Do you still use Google Analytics and if so are you thinking about using an alternative? Let me know what you think in the comments I am very interested to hear what you have to share!

Generate Open Graph images for your static Next.js site

Florian Kapfenberger — Fri, 30 Oct 2020 21:31:21 +0000

You probably know those fancy images that you see on Twitter or Slack when someone shares a link to a website. The information that you see is meta tags stored inside HTML. They even have a fancy name and belong to the Open Graph protocol.

Here is an example from my blog's homepage:

<meta property="og:title" content="Home"/>
<meta property="og:description" content="Welcome to my blog!"/>
<meta property="og:type" content="website"/>
<meta property="og:url" content="https://phiilu.com/"/>
<meta property="og:image" content="https://phiilu.com/images/og/977261ad2dded809bf3f4bdcf453f416.png"/>

Those meta tags are very easy to put onto your site, but the image one can be a bit tricky because we need real images that are different for each page we have!

In this post, I want to show you how you can design your own images and generate them on build time for your static Next.js site!

The principle is very simple. We have a webpage where we generate an HTML site that looks like the OG image we want to display when somebody shares our site. Next, we use Next.js build process to crawl this site, generate an image of the webpage, and save it somewhere, where Next.js can access it.

So let's start with building our OG Image using CSS!

Create your OG image page

In my case, I generated a separate app with the only purpose of generating the images and displaying them. You could also add the functionality we are going to write into your existing app, but make sure to deploy the code we write in this section first!

What we are going to write is basically a webpage that takes some query parameters and generates the image we want. For example, if we access our app with this URL (go check it out)

https://og-image.phiilu.com/phiilu.com?title=Hello%20World&url=https://phiilu.com/hello-world

we get a webpage which generates this image:

So all you need is a simple route and a component that looks similar to this one.

import Image from "next/image";
import { useRouter } from "next/router"
import GoogleFonts from 'next-google-fonts';

import Heading from "@components/Heading/Heading"

// The function `getFontSize` will increase or decrease the // font size of the title depending on its length.
function getFontSize(length) {
  if (length > 55) {
    return `text-6xl`
  }

  if (length > 32) {
    return `text-7xl`
  }

  return `text-8xl`
}

// Example URL: http://localhost:3000/phiilu.com?title=Hello%20mein%20Name%20ist%20Florian!&url=https://phiilu.com/hello-world
const PhiiluCom = () => {
  const router = useRouter();

  const searchParams = new URLSearchParams(router.asPath.split(/\?/)[1]);
  const link = searchParams.get("url")

  if(!link) return null;

  const linkURL = new URL(link)
  const title = searchParams.get("title")

  return (
    <>
      <GoogleFonts href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@300;400;600;700&family=Source+Sans+Pro:wght@300;400;600;700&display=swap" />
      <div
        className="relative flex flex-col justify-between px-8 pt-24 pb-16 space-y-8 bg-indigo-100 border-indigo-500 shadow-md"
        style={{ width: 1200, height: 630, borderWidth: 16 }}
      >
        <div className="absolute top-0 right-0 mt-6 mr-6">
          <Image
            src='/images/phiilu.com-logo.svg'
            alt="logo"
            width="96"
            height="96"
            className="w-24 h-24"
          />
        </div>

        <div className="max-w-screen-lg">
          <Heading
            noMargin
            className={`${getFontSize(title.length)} text-indigo-800`}
          >
            {title}
          </Heading>
        </div>
        <div className="flex justify-between">
          <div className="flex items-center space-x-6">
            <img
              src="https://pbs.twimg.com/profile_images/1220392920538386432/NuYyL5b5_400x400.jpg"
              alt="Florian Kapfenberger"
              className="flex-none w-32 h-32 rounded-full shadow-md handsome"
            />
            <div className="flex flex-col text-indigo-900">
              <p className="text-4xl font-semibold font-open-sans">Phiilu.com</p>
              <p className="text-2xl font-open-sans">
                <span className="path">{linkURL.pathname}</span>
              </p>
            </div>
          </div>
        </div>
      </div>
    </>
  )
}

export default PhiiluCom

Of course, I am using Next.js here as well. I am using Tailwind, but you can use plain CSS or any other framework to style the page.

You can style your image like you want to, the only important part is that your OG image has the right dimensions. I am using 1200 for the width and 630 as the height as this is what most sites recommend.

To be fair those are not the perfect values for Twitter cards, but there are many different recommendations out there. Maybe in the future, I might generate different formats for different services.

To make the image dynamic I am using URLSearchParams to get the values from the URL query. If you don't know what URLSearchParams is you can check out my blog post where I explain how it works.

Now that your design is done and you are happy with it, you need to deploy your app somewhere. If you are using Next.js I would recommend Vercel or Netlify.

You can check out the full source code on GitHub if you want.

phiilu / og-image-app

This simple React app builds pages which will be used as OG images.

The OG image generator function

Now that we have our little OG image app ready, we can finally continue and generate our OG images!

First, create a new file where you want to put your code. I chose to put mine into a lib folder and named the file getOgImage.js.

We basically need to do 3 things:

check if we already have the OG image and if yes just return the location
if not, we need to open the browser with the correct query parameters and take a screenshot of the page
store the screenshot where Next.js can use it

All these steps translate into this code:

const playwright = require('playwright-aws-lambda');
const { createHash } = require('crypto');
const fs = require('fs');

async function getOgImage(path, baseUrl = 'https://og-image.phiilu.com') {

  if (process.env.NODE_ENV === 'development') {
    return 'og image will be generated in production';
  }

  const url = `${baseUrl}${path}`;
  const hash = createHash('md5').update(url).digest('hex');
  const browser = await playwright.launchChromium({ headless: true });
  const ogImageDir = `./public/images/og`;
  const imagePath = `${ogImageDir}/${hash}.png`;
  const publicPath = `${process.env.BASE_URL}/images/og/${hash}.png`;

  try {
    fs.statSync(imagePath);
    return publicPath;
  } catch (error) {
    // file does not exists, so we create it
  }

  const page = await browser.newPage();
  await page.setViewportSize({ width: 1200, height: 630 });
  await page.goto(url, { waitUntil: 'networkidle' });
  const buffer = await page.screenshot({ type: 'png' });
  await browser.close();

  fs.mkdirSync(ogImageDir, { recursive: true });
  fs.writeFileSync(imagePath, buffer);

  return publicPath;
}

export default getOgImage;

Let's take a deeper look into what some of these lines mean.

async function getOgImage(path, baseUrl = 'https://og-image.phiilu.com') {
...
}

Our function will take 2 parameters. The first one is the path with the query parameters that will generate the OG image like /phiilu.com?title=Hello%20World&url=https://phiilu.com/hello-world. The second one is optional, which is the baseUrl of our OG image app.

if (process.env.NODE_ENV === 'development') {
  return 'og image will be generated in production';
}

During the development of this feature, you might want to comment this block otherwise, the images are only generated in production. This is usually what you want because it can slow down your development workflow.

  const url = `${baseUrl}${path}`;
  const hash = createHash('md5').update(url).digest('hex');
  const ogImageDir = `./public/images/og`;
  const imagePath = `${ogImageDir}/${hash}.png`;
  const publicPath = `${process.env.BASE_URL}/images/og/${hash}.png`;

Here we define our OG image url with the two parameters from the function.

Then we create a hash of this url. The hash will be the filename of the image and lets us decide if we already generated an image for this request or not.

ogImageDir is the directory where we will store the OG images for Next.js.

imagePath is the file path where we will save the image.

publicPath is the absolute URL where our image will be available and the content value for the <meta name="og:image" /> tag.

try {
  fs.statSync(imagePath);
  return publicPath;
} catch (error) {
  // file does not exists, so we create it
}

Here we use the fs.statSync method to check if we already have an image for the requested URL. If the file already exists we will return the publicPath otherwise the method will throw an error and we can continue with our logic.

const browser = await playwright.launchChromium({ headless: true });
const page = await browser.newPage();
await page.setViewportSize({ width: 1200, height: 630 });
await page.goto(url, { waitUntil: 'networkidle' });
const buffer = await page.screenshot({ type: 'png' });
await browser.close();

This is probably the most interesting part. Here we are using
Playwright to control a headless browser.

Playwright is a great library to scrap webpages and automate your browser!

So we use Playwright to create a new empty page in the browser and set the viewport to exactly the dimensions we have specified in our OG image app. Next, we will browse to the url and wait until the network has become idle. Now we use the very cool method screenshot to generate a screenshot of the page and save the buffer inside a variable. When we are finished we can close the browser.

fs.mkdirSync(ogImageDir, { recursive: true });
fs.writeFileSync(imagePath, buffer);

recursive: true means if the directory already exists it will not be created again. This works exactly like the command mkdir -p DIR in the terminal

Last but not least we create the ogImageDir and create a new file with the contents of the saved buffer.

DONE! We have successfully generated the OG image and saved it where Next.js can serve it!

Generate the OG images during the build

The last thing that is missing is that we call the function inside our pages.

Go to a page where you want to generate the OG image and call the function inside the getStaticProps function.

This is what my getStaticProps in the [slug].js file inside the pages folder looks like:

export async function getStaticProps({ params: { slug } }) {
  const post = await contentful.getEntry('post', slug);
  const ogImage = await getOgImage(
    `/phiilu.com?title=${post.title}&url=${process.env.BASE_URL}/${slug}`
  );

  return {
    props: { post, ogImage }
  };
}

Now you have the ogImage prop available inside the page component and we can render the <meta /> tag inside the Head component.

import Head from 'next/head';

const PostDetails = ({ post, ogImage }) => {
  return (
    <>
      <Head>
        <meta name="og:image" content={ogImage} />
      </Head>
      <Layout>{/* ... */}</Layout>
    </>
  );
};

some parts of the component have been skipped, to make the example easier

My blog at phiilu.com is open-source, if you want you can have a look at the whole repo here.

phiilu / phiilu.com

Personal Website

Conclusion

That's all you need to generate dynamic OG images and store them locally.

This was not my first approach to serve dynamic OG images. In my first approach, I had a serverless function that would take a screenshot of the page and returns the image in a Base64 string. I basically copied the idea from Wes Bos. You can watch the video here.

It worked well for some time, but I was not happy with the performance. Sometimes the function took way too long to return the image and I also had some problems where the function was removed after deployment via a webhook on Netlify.

This is why I searched for another solution. My next thought was to do the same logic but store the images at Cloudinary. As soon as I had finished it, I was thinking if I could skip the Cloudinary altogether and store the image locally. After some research, I discover I can just store the image inside the ./public folder and Next.js can find it without any problem. I love the idea of serving the OG images from the same Vercel CDN that the Next.js app is on.

After writing this post I discovered that Vercel has built https://og-image.vercel.app/, which will solve the same problem, but differently. For now, I will continue using my custom brewed solution but I might check it out for a future project.

If you liked this post, you might like some of my tweets too. Go follow me @phiilu on Twitter where I share things related to web development!

My thoughts on completing the React Fundamentals workshop from Epic React by Kent C. Dodds

Florian Kapfenberger — Mon, 12 Oct 2020 21:15:34 +0000

I just finished my first workshop of the Epic React by Kent C. Dodds course. After each workshop, I thought I might write down my thoughts and summarize my learnings.

Epic React is a BIG online course with a total of 8 different workshops and I don't know how many hours of videos, I just know that there are a lot!

If you want to learn or — like in my case — get better using React than this is probably the best course money can buy. Kent is an amazing instructor and he explains things very understandable. This epic course is so different than other online courses. It's more like an actual workshop where you attend in person. It will make sure that you will get your hands dirty and not just watch the videos!

The first module was React Fundamentals. It will teach you: How React works, what JSX is, and how to do simple things like creating components, add styling, and implement forms.

How I approached this course

Usually, when I buy an online course, I just watch the videos and maybe try out a few things on my own. Kent's workshops, however, are not intended to just watch them, YOU will be doing most of the exercises and watch the solution afterward.

With each workshop, you will get access to a repository hosted on GitHub. It has all the files that you will need to follow along.

These are the steps I did for each section of the workshop:

Watch the intro video where Kent introduces the topic/problem
Open the corresponding file in the editor and browser
In the browser read the whole description about the topic and what should be done
Solving the exercise (+ extra credit) and if there is a test run it to verify it is working
Go back to the video and watch how Kent solves it and listen to the tips he gives
Complete the section and go to the next one
Repeat

I like this approach and it helps me to try things out and finally understand them.

What did I learn?

More than I expected! I think I am already pretty good at React, but as it turns out I have skipped some fundamentals. 😅

At the beginning of this workshop, Kent will explain how React works under the hood without JSX. I knew JSX is just syntactical sugar and React uses plain JavaScript methods under the hood, but I never had to use them. Learning how the function React.createElement work will let you understand JSX much better.

Another thing was writing custom PropTypes. I have used PropTypes before but never wrote a custom one myself. It's good to know that this exists, but to be fair I think I won't be writing PropTypes a lot in the future. I think switching to TypeScript would be better than defining PropTypes on top of JavaScript.

Finally, I now completely understand why React needs the key prop when looping over an array and rendering it. The demo with the input focus made me realize how important it is to use something unique to the item and not generated dynamically like the index!

Conclusion

So far I like this course and I am very happy that I bought it and I can't wait to go through the next workshop which will be about hooks!

If you are learning React or already use it at work don't hesitate to buy it. I know it is expensive, but compared to the value you will get for that money, it's not! In some countries, parity purchasing power is available too.

Now back to learning and going through the React Hook workshop! Of course, I will give you an update on how the workshop was later on.

Happy coding!

This post has 8 reactions! - Using the dev.to API to update the post title with the reactions count!

Florian Kapfenberger — Sat, 10 Oct 2020 10:38:27 +0000

This is just a silly experiment inspired by this video from Ben Awad where he reverse-engineered the TikTok API to update his profile with live stats of one of his TikTok videos.

I wanted to try this out too and thought it might work on the dev.to platform too. I looked up the API documentation and found the endpoints that I need to use to make this work!

I don't have any bad intentions and don't want to hit the API too often. Some of the API endpoints have rate limits too, therefore I am making sure that I only make requests when it is necessary and every 30 seconds.

Here is the code that I used to make this work:

phiilu / dev.to-reactions-in-article-title

Using the dev.to API to update the article with the count of reactions.

import axios from 'axios';
import dotenv from 'dotenv';

dotenv.config();

const POST_ID = process.env.POST_ID;
const BASE_URL = `https://dev.to/api`;
const client = axios.create({
    baseURL: BASE_URL, headers: {
        'api-key': process.env.API_KEY
    }
})
let currentReactionsCount = 0;

const API = {
    article(id = POST_ID) {
        return client.get(`/articles/${id}`);
    },
    updateArticle(id = POST_ID, body) {
        return client.put(`/articles/${id}`, body);
    }
};

const sleep = async (amount = 30000) => new Promise(resolve => setTimeout(resolve, amount))

while (true) {
    try {
        const { data } = await API.article();
        const { public_reactions_count } = data;
        const newTitle = `This post has ${public_reactions_count} reactions! - Using the dev.to API to update the post title with the reactions count!`

        if (currentReactionsCount !== public_reactions_count) {
            await API.updateArticle(POST_ID, { title: newTitle });
            currentReactionsCount = public_reactions_count;
            console.log(`Article updated - Reactions: ${currentReactionsCount}`)
        } else {
            console.log(`Reaction count was the same`)
        }
    } catch (error) {
        console.log(error.message)
    } finally {
        await sleep();
    }
}

You can create a new API key inside your account settings and the API documentation can be found here.

Now if this is working this post should get updated every 30 seconds with the current reaction count!

Edit:

I noticed that the API might not return up-to-date data and therefore takes maybe longer than 30 seconds.

Add Apple Maps to your website using a serverless function and host it on Netlify

Florian Kapfenberger — Tue, 06 Oct 2020 16:37:59 +0000

Cover Photo by oxana v on Unsplash

I bet at one point in your developer career you were asked or wanted to implement an interactive map on your website. Interactive maps are great to visualize locations and routes for users and have replaced the need of using an old-school map helping us to navigate in the real world!

So you are now on your way to implement such a feature and the first thing you will do is search on google "How do implement an interactive map on a website" and probably the first few results will point you to use Google Maps. But what if I tell you that Google is not the only player in the game and there are different maps provider too!

In this blog post, we will be focusing on Apple Maps! Yeah, you heard correctly Apple Maps. "But doesn't Apple Maps suck? Apple Maps on a website? I thought that only exists on iOS?" you might ask and the answers are as usual "it depends" and "yeah it exists!".

You know Apple has released Apple Mapkit for JS in 2015 and since that year you can implement Apple Maps on websites! The most popular service I can find using Apple Maps is DuckDuckGo. If you search "where is vienna" you will find a small map on the right of the search results using Apple Maps. If you click it, you will get redirect to a site which has a bigger map.

Okay now you know you could use Apple Maps on your website, but how do you make it work? Well, it is not as easy as it seems, because Apple handles the API key differently than Google Maps.

Usually when you want to use a service you will sign up on their site and generate an API key that you than can use. Apple instaed will give you a private key which you can use to generate Json Web Token (JWT) tokens that will be used to validate your access to Apple Maps. This is why we need backend code to hide this private key and therefore we will implement this with a serverless function.

In this post, I will show you how to get the credentials from Apple, create a small example using Apple Maps + serverless function, and finally deploy it using Netlify.

Prerequisites

You will need to have a paid Apple Developer membership, otherwise, you will not be able to download the credentials for using Apple Maps
A free Netlify account to be able to deploy the final website

Sadly you will need to be in Apple's Developer program if you want to follow along, if you can't follow along you may still check out the final version here and take a look at the code on GitHub.

Obtaining the credentials from Apple

I did not figure out these steps on my own and they can be found on Apple's website too.

First head over to the account page of your Apple Developer membership. We need to create two things: Maps ID and a MapKit JS private key.

Maps ID

In the sidebar select Certificates, Identifiers & Profiles and than Identifiers and click the small plus icon.

Next, give it a description and an identifier using a reverse domain like maps.com.phiilu.example and click Continue and Register.

Now you have created a Maps ID which we will need for our Private Key.

MapKit JS private key

Back on the Certificates, Identifiers & Profiles overview page select Keys and create a new key by selecting the small plus icon.

Give the Key a name and you should see MapKit JS as an option in the services. If not then you did not create a Maps ID.

Next select Configure next to the MapKit JS, select the previously created Maps ID, and click Save. Review your settings and click Continue.

You will see a summary of the services you enabled for this key and can register your key with Register.

Finally, your key is registered and you can download it! As the warning says this is a ONE TIME DOWNLOAD and you won't be able to download the key again.

You will download a file named AuthKey_Z11AA36DZ4.p8 or similar and if you open this you will find 6 lines of text, which will be your private key.

This key should ALWAYS be kept private and never be leaked to the public, hence why we need to create a serverless function to safely store the key in backend code.

Take note of the generated Key ID too, because you will need it later.

Team ID

The Team ID is already created for you, but you need to copy it from the top right next to your name.

Now that we have everything we need and we can start integrating Apple Maps on our website.

Setting up the development environment

This project is using the Netlify CLI, so you will need to install it globally using npm.

npm install netlify-cli -g

Next, you will need to clone the git repository and checkout the branch tutorial-start.

git clone git@github.com:phiilu/apple-maps-example.git
cd apple-maps-example && git checkout tutorial-start

To start the project you need to install the dependencies and run the development server.

npm install
npm start

When you open the browser at http://localhost:18080 you will see a simple styled website where we will be adding Apple Maps.

To keep it simple we will be using basic HTML, vanilla Javascript, and styles using Tailwind. It will be compiled using Parcel and the serverless functions can be tested locally using the Netlify CLI.

Project structure explained

Inside the project directory run the following command to list all folders and files.

tree -I 'dist|node_modules'

.
├── README.md
├── netlify.toml
├── package-lock.json
├── package.json
├── postcss.config.js
├── src
│   ├── functions
│   ├── index.html
│   ├── index.js
│   └── index.pcss
└── tailwind.config.js

2 directories, 9 files

Most of these files should not need explaining, but I will explain the most interesting ones.

netlify.toml

netlify.toml contains the configuration for Netlify and the Netlify CLI.

[build]
  command = "npm run build"
  functions = "src/functions"
  NODE_ENV = "12"
  publish = "/dist"

[dev]
  port = 18080
  publish = "/dist"
  command = "npm run dev"

[production]
  publish = "/dist"
  command = "npm run build"

[[redirects]]
  from = "/api/*"
  to = "/.netlify/functions/:splat"
  status = 200

It will tell Netlify where our serverless functions are, which Node.js version we want to use, which port we want the development server should run, how it can start our project in development mode, and redirects all requests from /api/* to the correct Netlify function.

src directory

src will contain all of the code we write. It contains the functions directory where we will be putting our serverless functions and our HTML, Javascript, and CSS files.

Tailwind

A utility-first CSS framework for rapidly building custom designs. - tailwindcss.com

Lots of the other files you see are to configure Tailwind. I just like to use Tailwind to add styling to my projects, but we won't be adding more styles, so you will not need to know how it works.

Adding Apple Maps to the website

Let's start adding Apple Maps to the website! When you open the cloned repo and open up the index.html file you will see that there is already Apple MapKit pulled in from Apple.

<!-- index.html -->
<script src="https://cdn.apple-mapkit.com/mk/5.x.x/mapkit.js"></script>

This line will load the latest MapKit from Apple. You can find how Apple versions MapKit here.

Further down the file, you will find the div where we want to mount Apple Maps.

<div
  id="apple-maps"
  style="height: 800px"
  class="w-full rounded-md shadow-lg"
/>

This div has set a height and some classes that will make it look a bit nicer. If you open up http://localhost:18080/ you should see a simple website with navigation, a search box, and a blank space with a shadow around.

Now we want to add Apple Maps to this div, so open up the index.js file and paste this code:

mapkit.init({
    authorizationCallback: function (done) {
        fetch("/api/token")
            .then((res) => res.text())
            .then(done);
    },
    language: navigator.language || navigator.userLanguage,
})

const center = new mapkit.Coordinate(48.210033, 16.363449) // Vienna
const map = new mapkit.Map("apple-maps", {
    center,
    cameraDistance: 15000,
})

Now you should see a Map showing up in your browser... well not really. If you have a look at the console you will see an error and a warning.

Let's debug our code and find out why it is not working.

You can see, we are mounting the map correctly using the id of apple-maps that we specified in the index.html file.

But before mounting we are calling mapkit.init(...) and fetching a new token from /api/token. This token is how Apple Maps verifies that this site has access to Apple Maps, but we did not write any code that would generate such a token yet.

In the next section, we will write the serverless function that will generate our token for Apple Maps.

Creating a serverless function to create a valid JWT token for Apple Maps

It's time to take our generated private key and use it to generate JWT tokens for our frontend!

Configure .env with our obtained secrets

Before working on the serverless function we will need to create a .env file in the root of your project and put in our obtained credentials.

MAPS_KEY_ID=<MAP_KEY_ID>
APPLE_TEAM_ID=<TEAM_ID>
APPLE_MAPS_KEY=<PRIVATE_KEY>
SITE_ORIGIN=http://localhost:18080

Put in your information that you saved from the previous sections.

APPLE_MAPS_KEY is a little special, because we downloaded it as a file. You will need to copy the contents of the file and replace all line breaks with \n. Your APPLE_MAPS_KEY entry should look like this:

APPLE_MAPS_KEY="-----BEGIN PRIVATE KEY-----\nMIGTAgEAMBMAyyqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQg698QaJcP/pbDrXHE\nSzt8Igy6KgeUm2ky1LZZoBDcVzygCgYI2oZIzj0DAQehRANCAAST56qF4NUi1v0Y\nrWfvSNVbSdQjT7M6syC7bJYiB5zTlwzzGeIU2kilRNY7p3KlUnC5QGISGHjN3FL+\n7wjrPBNa\n-----END PRIVATE KEY-----"

Don't worry, this is not my real private key

Your .env file should look something like this:

MAPS_KEY_ID=AAAAAAB994
APPLE_TEAM_ID=1234A8E9Z5
APPLE_MAPS_KEY="-----BEGIN PRIVATE KEY-----\nMIGTAgEAMBMAyyqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQg698QaJcP/pbDrXHE\nSzt8Igy6KgeUm2ky1LZZoBDcVzagCgYI2oZIzj0DAQehRANCAAST56qF4NUi1v0Y\nrWfvSNVbSdQjT7M6syC7bJYiB5zTlwzzGeIU2kilRNY7p3KlUnC5QGISGHjN3FL+\n7wjrPBNa\n-----END PRIVATE KEY-----"
SITE_ORIGIN=http://localhost:18080

Writing the serverless function

Create a token folder inside the functions folder and open it in your editor. Next scaffold a package.json using npm inside the token folder.

npm init -y

As I already mention we need to generate JWT tokens, so we will be using the jsonwebtoken package.

npm i jsonwebtoken

Now create a file named token.js and paste this code:

const jwt = require('jsonwebtoken');

const origin = process.env.SITE_ORIGIN;
const privatekey = process.env.APPLE_MAPS_KEY.replace(/\\n/gm, '\n');
const keyid = process.env.MAPS_KEY_ID;
const issuer = process.env.APPLE_TEAM_ID;

exports.handler = async function (event, context) {
  const token = jwt.sign({
    origin
  }, privatekey, {
    algorithm: 'ES256',
    expiresIn: "1d",
    keyid,
    issuer,
  });

  return {
    statusCode: 200,
    body: token,
  }
}

This simple function will generate a JWT token that is signed using your private key. The private key needs to replace the \\n with real \n, because Netlify somehow adds this automatically. I found the solution through this comment on GitHub.

The token will be valid for 1 day and will only work on the website configured in SITE_ORIGIN. You can change these options if you want and adjust them to your needs.

Apple has documented what each field is and what is used for at their documentation.

Restart your development server and open up http://localhost:18080/ and you should see a map with the center being Vienna 🥳

Deploy the website to Netlify

Now that our Map is working we want to publish our little project on Netlify and we will do this using the Netlify CLI that you already know.

If you did not login to Netlify using the CLI you will have to do this first.

netlify login

Next, go to the root of your project directory and run netlify deploy. This will walk you through and create the site on Netlify.

netlify deploy
This folder isn't linked to a site yet
? What would you like to do? +  Create & configure a new site
? Team: Florian Kapfenberger's team
Choose a unique site name (e.g. isnt-phiilu-awesome.netlify.app) or leave it blank for a random name. You can update the site name later.
? Site name (optional): apple-maps-example

Site Created

Admin URL: https://app.netlify.com/sites/apple-maps-example
URL:       https://apple-maps-example.netlify.app
Site ID:   44aa1f3e-f887-47e0-bf55-4c80f2ec01ab
Deploy path:        /Users/florian/Code/privat/projects/apple-maps-example/dist
Functions path:     /Users/florian/Code/privat/projects/apple-maps-example/src/functions
Configuration path: /Users/florian/Code/privat/projects/apple-maps-example/netlify.toml
Deploying to draft URL...
✔ Finished hashing 10 files and 1 functions
✔ CDN requesting 9 files and 1 functions
 ›   Warning:
 ›   {}
 ›
    TypeError: Cannot read property '0' of undefined

As you can see I got an error the first time, so I reran the script and it worked!

netlify deploy
Deploy path:        /Users/florian/Code/privat/projects/apple-maps-example/dist
Functions path:     /Users/florian/Code/privat/projects/apple-maps-example/src/functions
Configuration path: /Users/florian/Code/privat/projects/apple-maps-example/netlify.toml
Deploying to draft URL...
✔ Finished hashing 10 files and 1 functions
✔ CDN requesting 9 files and 1 functions
✔ Finished uploading 10 assets
✔ Deploy is live!

Logs:              https://app.netlify.com/sites/apple-maps-example/deploys/5f7b795ed6d83e3ab0a3f4a2
Website Draft URL: https://5f7b795ed6d83e3ab0a3f4a2--apple-maps-example.netlify.app

Our site may be deployed, but it is still not working, because we need to configure the environment variables in Netlify first.

Open up netlify.com, log in and go to the Build & deploy settings of the created project and add the environment variables from your .env file.

The last step is to deploy your project again, but this time with the --prod flag.

netlify deploy --prod

Your project is now live and you can visit it at the URL the Netlify CLI displays you or you can check out mine at https://apple-maps-example.netlify.app/

Adding a Searchbox using Apple Maps (optional)

For now, we just displayed a map but did not interact with it. In this section, I want to show you how you can easily search for cities and transition to them using just a few lines of code.

Open up index.js and append the following lines to it:

// index.js

const search = new mapkit.Search({
    language: navigator.language || navigator.userLanguage,
    getsUserLocation: true,
    region: map.region,
});

const searchFormElement = document.getElementById("search-form");
const searchBoxElement = document.getElementById("search-box");

searchFormElement.addEventListener("submit", (e) => {
    e.preventDefault();
    const query = searchBoxElement.value;
    if (query) {
        search.search(query, (error, data) => {
            console.log(data);
            if (error) {
                console.log("Error", error);
                return;
            }
            map.setRegionAnimated(data.boundingRegion);
        });
    }
});

Using new mapkit.Search(...) we can create an instance of Search and use it to search for locations on the map.

The document.getElementById(...) lines will be used to get references to the form and input elements of the page.

After we have the reference we can add an event listener for the submit event on the form. We prevent the default even, which would reload the page and get the text from the searchBoxElement.

Next, we perform the search using search.search(...). The first argument is the query and the second one will be a callback when the request has finished. If we don't have an error we will use map.setRegionAnimated(data.boundingRegion) to transition to the found location on the map.

Try it out! Open up http://localhost:18080/ or https://apple-maps-example.netlify.app/ and search for a city like Berlin and press enter. You will see the map transition to the center of Berlin.

If it works, you can deploy your changes with netlify deploy --prod.