DEV Community: Felix

Locate Website Visitors in Next.js with IP and Supabase

Felix — Sun, 02 Jun 2024 11:30:34 +0000

Using API routes with Next.js as well as an IP address geolocation service is an easy way to display the last visitor of a website. I wanted to use this in a widget on my website homepage. I got this idea after visiting Rauno Freiberg's website – he's a designer at Vercel and I admire his work.

First of all, I mapped out what needs to be done:

Locating the user. There are a few different ways to locate a user on a website, I chose to go with the IP address. This isn't always extremely precise, but it's good enough for getting a rough location to display.
Storing user locations in a database. There obviously needs to be something on the server side remembering locations of users to display to the next user.
APIs for updating and retrieving the last user's location. We also need to think about the order of requests, to make sure that when a user visits the website and the last location is updated, he doesn't get his location back from the database but the location of the previous user.

I also had a few priorities for this project: speed and minimal server load. I don't want the last visitor feature to cause a significant increase in page loading times for the client, or to increase server actions beyond what is provided in the Vercel free tier. We'll have a look at how to do this in the best way possible.

Fetching the country from IP

As IP addresses change frequently, and the ranges get reallocated, it's difficult to know which range of addresses maps to which country. Luckily, MaxMind has a free IP lookup tool called GeoLite2, which they update frequently. It's more or less precise, but good enough for what we're trying to do.

After creating an account and getting a license key, getting the city and country from an IP address is quite easy using the @maxmind/geoip2-node NPM package. Here's some example code:

import { WebServiceClient } from '@maxmind/geoip2-node';

const MAXMIND_USER = '1234567';
const MAXMIND_LICENSE = 'Your license here';

const ip = '43.247.156.26'; 
const client = new WebServiceClient(MAXMIND_USER, MAXMIND_LICENSE, {host: 'geolite.info'});
const response = await client.city(ip);
const data = {country: response.country ? response.country.names.en : null, city: response.city ? response.city.names.en : null}
console.log(data)
`}</code2>

I added some extra logic after noticing that not all IP addresses map to specific cities in the lookup tool.

## Implementing the database

To simplify things in the database, I decided to create a table with `ip`, `city` and `country` columns. Setting the new country is equivalent to adding a new row, instead of updating the last row. This to make it easier to handle the location of the previous user/current user. There's a primary key, `ID`, which will be automatically set.

I decided to go with Supabase, as I've already used their services and am impressed with the ease of use and the excellent Next.js integration.

I created a country table, and I disabled Row-Level-Security as this would require some form of authentication, and we're going to do all the database queries on the server-side as you'll see next, so the client will not have access to the database.

![](https://felixrunquist.com/static/5e00a3c789e4bcf773b65f0b802d7b7f6c06be84.png)
_The columns of our views table_

## Adding the middleware and APIs

In order to fully separate the user from the database, I decided to perform all database queries on the server, and handle the country updating logic using Next.js middleware. This has the added benefit of not sending any additional client requests to log the users' country, which lightens the client-side stack.

To retrieve the country, instead of using `getServerSideProps`, which would make each page dynamic and block it from loading until the database has responded, we do this on the client side through an API route. Need a refresher on Next.js page generation? [Have a look at this post](https://felixrunquist.com/posts/generating-sitemap-rss-feed-with-next-js).

Let's map this out:

![](https://felixrunquist.com/static/3d59e45fdc8d8f4672541d8819d5205f2b85f12f.png%20)
_A diagram of the different requests to the server and database_

### Middleware

Let's first look at the middleware: It makes a request to an API endpoint. For added security, we can prevent the client from interfering with the `set-last-country` API by adding an API key: a secret that will be shared between the middleware and the API to ensure that only the middleware is able to update the country.

To prevent the same user from constantly setting their country as the last, for example if they navigate across multiple pages, we can set a session cookie from the middleware to prevent subsequent requests from triggering the country update.

import { NextRequest, NextResponse } from 'next/server'

import { SITE_PATH, COUNTRY_SET_KEY, VIEW_SET_KEY } from '@/lib/constants'

export const config = {
matcher: ['/:path', '/posts/:path*', '/pages/:path*', '/creations/:path*']
}

export async function middleware(req) {
const res = NextResponse.next()

//check if there is no cookie set
if(!req.cookies.has('country')){
res.cookies.set('country', 'true')

const forwarded = req.headers.get("x-forwarded-for")
var ip = forwarded ? forwarded.split(/, /)[0] : req.headers.get("x-real-ip")
try {
  fetch('https://' + req.headers.get('host') + '/api/set-last-country', {
    method: 'POST',
    body: JSON.stringify({ip, key: COUNTRY_SET_KEY})
  })
} catch(error){
  console.log(error)
}

}
return res
}


We're using the matcher to only trigger the middleware on certain routes, and the country cookie is used to check if the user's country has already been uploaded to the database during the current session. We then send the IP to the API, as well as the API key mentioned earlier.

You'll also notice that there's no `await` key preceding the fetch, this speeds up loading times since it isn't necessary to wait for the database to update the last country in order to serve the page to the user.

### API setter

In the `set-last-country` API, we'll need to do the following:

- Use geolite2 to get a country and city from the IP address
- Store the country and city in the Supabase database. 

We also need to make sure that the API only accepts POST requests, as it's the method the middleware uses for sending the data.

import { WebServiceClient } from '@maxmind/geoip2-node';
import { createClient } from '@supabase/supabase-js'

import { COUNTRY_SET_KEY, MAXMIND_USER, MAXMIND_LICENSE } from '@/lib/constants';

export default async function handler(req, res) {

if(req.method != 'POST'){
console.log("Unauthorized")
return res.status(403).json({error: 'Unauthorized'})
}

const body = JSON.parse(req.body)

if(body.key != COUNTRY_SET_KEY){
console.log("Unauthorized")
return res.status(403).json({error: 'Unauthorized'})
}

//Get country from body.ip
const client = new WebServiceClient(MAXMIND_USER, MAXMIND_LICENSE, {host: 'geolite.info'});
const response = await client.city(body.ip);
const data = {country: response.country ? response.country.names.en : null, city: response.city ? response.city.names.en : null, ip: body.ip}

//Send to SB
const supabase = createClient(process.env.NEXT_PUBLIC_SUPABASE_URL, process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY)

var { error } = await supabase
.from('visitors')
.insert(data)

res.status(200).json(data)
}

### API getter

The `get-last-country` is the only API route we want to make available to the client. For this reason, we're not going to be using any API keys. To reduce overhead, caching can be used: since it doesn't really matter if we get the last user, or the 10th last user, we can cache the result for 10 minutes which will reduce server requests as well as database access. Caching on the client and server side can be done using special headers.

We also need to make sure that when the user visits the website and the last location is updated, he doesn't get his location back from the database but the location of the previous user. To do this, we can add an additional column in our `country` table to check if the second to last row has been read or not. If it hasn't been read, we'll return that row, otherwise we'll return the last row. This ensures that the user never gets their location on the first visit, but they might on the next.

import { createClient } from '@supabase/supabase-js'

export default async function handler(req, res) {
res.setHeader('Vercel-CDN-Cache-Control', 'max-age=1200');//Cache for 20 minutes
res.setHeader('CDN-Cache-Control', 'max-age=600');//Cache for 10 minutes
res.setHeader('Cache-Control', 'max-age=600');//Cache for 10 minutes

const supabase = createClient(process.env.NEXT_PUBLIC_SUPABASE_URL, process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY)
var { data, error } = await supabase
.from('visitors')
.select('id, country, city, read')
.order('id', { ascending: false })
.limit(2)

if(!data[1].read){
var { error } = await supabase
.from('visitors')
.update({read: true})
.eq('id', data[1].id)
data = data[1]
}else{
data = data[0]
}

delete data.read
delete data.id
res.status(200).json({cached: false, value: data})

}

## Client-side logic

Now that we've implemented everything on the server side, all that's left to do is to create a component which will display the location of the last visitor:

[View the article](https://felixrunquist.com/posts/locating-last-visitors-in-next-js) for the live preview below:

`/App.js`:

import { useEffect, useState } from 'react'
import styles from './lastvisitor.module.scss';

export default function LastVisitor(){
const [location, setLocation] = useState("")
useEffect(() => {
getLastVisitor()
}, [])
async function getLastVisitor(){
const res = await fetch('http://felixrunquist.com/api/get-last-country')
if(res.status == 200){
const json = await res.json()
setLocation((json.value.city ? json.value.city + ', ' : "") + json.value.country)
}
}
return (

Last visitor: {location}

)
}
/lastvisitor.module.scss:

.container {transition: opacity .3s ease-in-out; }
.container.hidden {opacity: 0; }
.container:not(.hidden) {opacity: 1; }

The last visitor element stays visually hidden until the data has been fetched.

Closing notes

That's it, we now have a working last visitor system! Since the priority here is to reduce server load, caching and other features are used to prevent too many requests from being made. This comes with a tradeoff: the last visitor might not always be accurate.

For additional safeguards against server usage, we could also implement rate limiting using @upstash/ratelimit, as mentioned in Next.js documentation.

If you want to dig further into the different methods of generating content in Next.js, I've written an article which explains how getServerSideProps, getStaticProps and revalidate work.

Have any questions? Feel free to send me a message on Twitter!

References

Maxmind documentation, helpful for translating an IP address to a rough geographical location
Rauno Freiberg's website with the last visitor counter is what got me inspired to create one!
Josh W. Comeau's article on hit counters explains the integration of serverless functions in static sites

Credits to Big Loong's template which was used as a starting point for the article illustration. I made it with Spline, a tool I discuss in this article.

This article was retrieved from felixrunquist.com.

Making the Internet More Human

Felix — Mon, 15 Jan 2024 16:28:30 +0000

In the early days of the internet, the World Wide Web was a collection of static webpages. Updating these pages required a meticulous process of manually uploading each change to the server as a separate file. Given this long procedure and the sluggish download speeds associated with dial-up modems at the time, it was in everyone’s best interest to deliver information as plainly and efficiently as possible. One couldn’t run the risk of making the user wait a lot of time just to load a webpage full of garbage.

Then came the technological improvements that led to the internet as we know it today. ADSL and fiber allowed for a thousand-fold increase in speed. Scripting technologies like Javascript and AJAX allowed webpages to seamlessly fetch items from the server helped build complex sites like Google Maps. The problem with the internet of today is that these tools that were built to make navigating the internet faster and easier, have had the precise opposite effect. Driven by counterintuitive design trends and legislation, Javascript has been used extensively to display intrusive pop-ups and to override native scroll behavior. The overhead in data rates has been used to deliver annoying autoplay videos. The agressive expansion big corporations has led to a high proportion of ads silencing what’s important.

This has impacted people differently depending on revenue and location. On average, bandwidths in urban areas are twice as high as in rural areas. In developing regions such as Africa and the CIS, average internet bandwidth is at less than a third of that of developed regions such as Europe. This means that internet-heavy technologies such as autoplaying web video have an even worse impact on users in rural and developing regions.

The aim of this article is to explore issues in information delivery and to figure out how what we can do to make the internet a better place for everyone.

Accessibility

A common design trend on websites has been to use new CSS specifications to drastically change how the website is displayed on the web. The more drastic ones go as far as implementing custom scrollbars (or worse, not displaying any scrollbars at all!) and scrolling interactions, also known as scroll hijacking.

Most users are familiar with the way their operating system handles common interactions such as scrolling, and they may have customised the settings to their liking. Another problem is that OS implementations are much likelier to accommodate for accessibility, whereas custom implementations often overlook accessibility.

A customised scrollbar on css-tricks.com

These recent design trends are part of a bigger problem: the trade for design over function. A good-looking website isn’t necessarily the best website: Regardless of it’s content, a website shouldn’t make users feel disoriented, or even worse, unable to navigate the website.

I personally looked into ten websites featured on Awwwards, a company that makes “Website Awards that recognize and promote the talent and effort of the best developers, designers and web agencies in the world”. Of the ten websites I visited on desktop,

7 implement scroll jacking i.e. the scrolling speed and inertia is different from my OS settings
2 use a custom cursor i.e. the pointer is not the default one provided by my OS
1 website uses a custom scrollbar
1 website displays no scrollbar at all.

Say what you want, but if ~70% of the “talent and effort of the best developers” results in unfamiliar scroll behavior, this is telling of a deeper issue in the web design world.

Best practices

Here are a few best practices that can be taken into consideration to reduce potential accessibility issues.

Use native implementations when possible. For many website interactions, browsers expose OS elements and functionality. Because they are directly provided by the OS, they handle accessibility … such as tabular navigation better. They can often be accessed through a CSS class. For example, scrollbars, cursors, hyperlinks.

Respect contrast recommendations. Make sure the foreground and background colors respect WCAG guidelines on color contrast (here’s a tool to check the contrast between colors). Bad contrast will disproportionately impact users with color blindness.

Don’t display irrelevant or unexpected content. Many users will click to your site as a result of a search. If they’ve clicked on that link, it’s because they expect the webpage and its content to match what they were looking for.

For example, if a page is a programming tutorial in the form of an article, they will likely visit it if they’re looking for a written tutorial. In this case, displaying a large, autoplaying video would be irrelevant, and most likely be more annoying than useful. They won’t be likely to be expecting pages and pages of backstory on the history of the tutorial or of the language.

The a11y project has developed a web accessibility checklist with best practices regarding content, layout and visual design. While it may be difficult to tick everything off the list, and I certainly don’t claim to have done so, it’s a good indicator to know where you are at regarding accessibility.

Search engines

Much of the online content we come across today is with the help of search engines. To be able to display the content efficiently, search engines need to index websites beforehand. This is done with the help of robots, called crawlers, that navigate webpages and search for keywords that would be relevant to the page. With the development of crawlers, techniques to optimise webpages for them were developed, appropriately dubbed Search Engine Optimisation.

The issue is that crawlers have trouble replicating the ways humans navigate, and often fall for techniques aimed to improve search rank, i.e. the position of a webpage in a search result. Crawlers tend to appreciate verbose pages with a diverse range of keywords, while humans often prefer straight to the point content from which they can glean information as efficiently as possible. Think of someone searching for a recipe: they have the exact meal they want in mind, and their goal is to know the ingredients and preparation steps. More often than not, results will contain a lengthy backstory on the history of the recipe, the author’s grandma etc. before getting to the actual recipe. This isn’t relevant to the user’s search, but the crawlers latch onto the numerous recipe-related keywords in the backstory which is the reason the recipe ranked so high up in the users search.

Over time, websites have become less and less targeted towards humans, and more towards search engines. This paradoxical, since the end consumer remains the same whether the website has been found directly or through a search engine: the human reader. The situation is that since search engines control people’s access to a website, websites optimised for humans are less likely to be visited than websites optimised for search engines. Websites less visited earn less revenue, so website creators have no choice but to favor crawlers over humans, and to have an inefficient delivery of information.

A robot feeding websites to a human. [AI-generated]

The need for human input

The main factor that has led to this dystopian trend is the complete removal of human input from the process of crawling websites. As long as robots are unable to replicate human behavior, we need to add humans to the feedback loop so that content relevant to humans is favored. This could be done in the form of a upvote/downvote button at the bottom of search results.

Human input on search results

A large amount of the content we come across on the internet is sponsored. It is estimated that a whopping 21.2% of posts shown to users on Facebook and 20.6% on Instagram are advertisements. Revenue from advertisements make up over 80% of Google’s annual revenue in 2022. This is a conflict of interest as the company which controls the way 91% of people search earns more by manually overriding its algorithm.

Let’s look at the some of the main types of ads:

Pop-up and prestitial ads. These ads open a modal in front of content, often as soon as the page loads. On video content, they appear before the video. They provide a frustrating experience to the user as they provide an obstacle before the user is able to access content.

Sticky ads. These ads appear when scrolling down and stick to the viewport for some time as the user scrolls, lengthening the amount of time the ad is displayed.

Inserted ads. These ads appear in content such as social media posts or emails, and are crafted to look visually similar to the rest of the page. This is frustrating for the user as they have trouble distinguishing ads from what’s important.

Inserted ads in Gmail and Twitter respectively

Most users are frustrated by countdown ads which they can’t click or scroll away from, as well as ads that take up a high proportion of the screen (over 30%). It is clear that ads impact mobile and desktop devices differently: the same ad size takes up a higher proportion of the screen on mobile than on desktop.

Over the years, as online companies seek to increase year-on-year revenue, they have traded usability for profitability by making their ad strategy increasingly agressive in an attempt to squeeze the most money out of their services. Ad density has increased, and they are harder to distinguish from non-sponsored content. Youtube tried showing as many as 10 unskippable ads in 2022, Gmail experimented embedding ads in the middle of content in 2023, and I long for the sweet old times when Instagram didn’t show ads.

This has resulted in a game of cat and mouse – as platforms increase the proportion of ads, users become frustrated. This increases their chances of using an ad blocker, which in turn causes websites to use ad block detectors to as an incentive to turn them off. The situation has gotten so bad that I’ve found found that disabling Javascript on some websites makes the whole navigation experience more friendly: there’s much less ads and this even removes paywalls on some websites.

Cat and mouse: Websites detecting ad blockers and displaying pop-ups

Youtube ad blocker detection, which rolled out in Q4 2023

Now don’t get me wrong, I’m not saying we should remove all ads, as they help offset operating costs of platforms and keep them free, however the agressive growth tactics of large corporations has led to an excessive amount of ads on the web. I believe we need to rethink the way we present ads in order to put the end user back into the frame.

The coalition for better ads is an organisation formed by a group of large corporations in order to improve the standards of ads online. They provide guidelines and best practices for different types of ads. While this is on the right track, it isn’t enough: for example, despite Google being a board member of the organisation, and having pledged to block ads that are deemed below certain standards regarding user experience on Chrome, still extensively uses mid-roll ads on its video platform without blocking them.

Google Chrome’s pre-installed ad limiter

A note on cookies

With the introduction of GDPR in the EU in 2018, almost all websites display cookie pop-ups when you first load a page. While it may not be that annoying on websites you frequently visit, such as Google, since it’s only displayed once, it becomes quite frustrating on websites one rarely visits, especially on mobile, when clicking on an article link on Twitter for example.

While it was a good idea to introduce legislation around the usage of cookies, the current implementation is catastrophic. Cookie pop-ups provide a frustrating experience, an obstacle to the website which is precisely why Javascript was developed in the first place. Worst of all, the process of giving one’s cookie consent hasn’t been standardized: there is a variety of Javascript-based extensions that have varying ways of displaying information to the user, meaning that one can’t rely on muscle memory, as well as causing accessibility issues.

The legislation also lacks clarity – some websites use the cookie pop-up as an ultimatum: Accept cookies or pay a subscription to access the site. It is uncertain whether this is allowed, as the Article 5(3) of the ePrivacy Directive, also known as the ‘Cookie Law’, “the storing of information, or the gaining of access to information already stored, in the terminal equipment of a subscriber or user is only allowed on condition that the subscriber or user concerned has given his or her consent, having been provided with clear and comprehensive information.“

It is hard for average users to discern whether websites enforce cookie choices or not, as most websites store at least one cookie whether or not the user has consented.

In this catch-22 situation, a variety of browser extensions have been developed to accept cookies automatically.

A solution would be to put the burden of cookie consent on browses, the same way that website notification requests are handled: deciding to refuse cookies on a particular website would effectively prevent it from storing any cookies, removing the question of enforcement on the websites’ side. It would also allow for setting a global policy: one could choose a default behavior so that one doesn’t have to choose every time.

A request to display notifications in Google Chrome

One could even define a standard for sending cookie choices in HTTP headers, which would allow the browser to notify the website whether the user has chosen to allow only essential, or all cookies.

Designing spring animations for the web

Felix — Mon, 24 Jul 2023 07:57:33 +0000

One of the things that has constantly evolved since the beginning of the digital age is the way we present content. From printed tape to blinking lights to ascii terminals, right until the development of the graphical user interface with the Xerox Alto and the Macintosh, there was one thing in common: severely limited computational resources. In fact, resources were so limited that one could see content animating before one’s eyes.

Scrolling through a list of files on the Apple Lisa. Credits: Zhizu.com

With the technological improvements in computing power, this got to the point where the delay in content animation became imperceptible to the human eye. What was once animated now became a jarring flash to the user. This raises an important question: How does one provide contextual information to the user?

The answer: Voluntarily slowing down the speed at which content presents itself. This is done by allocating some computer resources to tweening content, i.e. creating intermediate frames between the first and last state, and there you have it: an animation. It might seem paradoxical to use more computer resources towards doing what one used to do with fewer resources, but it actually allows for more control.

If you’ve been on any form of digital device in the past fifteen years, you’ve most experienced many different types of animations. Animation, when done right, can make content feel more natural and less stark to the user. However, all these options have led to a widely diverse set of implementations: We’ve all come across the one site which dispenses with any animation (Jarring cookie banners for example), or even worse, a site that implements animation, but in a way that feels counterintuitive. It’s the wild west out there. This post aims to provide good practices for animations, as well as ways to implement them in Javascript and CSS.

A cookie banner that is disruptive and annoying

Reasons for using animations

There are many cases for using animations can improve user experience:

User-triggered: These are animations that run after a user action like pressing a button or opening a new window.
Loading states: These trigger when content is being fetched or a lengthy process is running – it provides reassuring feedback to the user and lets them know that the page isn’t hanging.
Presenting content: This helps content that might not be expected by the user seem less jarring, such as a notification for example.

How can we make animations feel more intuitive? By imitating the physical world. In the words of the first Apple Human Interface guidelines in 1987, use metaphors from the real world. Users are already familiar with the movements of objects in the real world, and by replicating these, we can create motion that feels intuitive.

How does a physical interaction work?

Let’s go through a bit of physics to understand how interactions work (If you don’t like physics, you can totally skip this section!). In the human-scale physical world, we use Newtonian dynamics to study the evolution of an object in space. An object is subject to multiple forces, and these contribute to the change in velocity through a simple equation:

\sum_i F_i \hspace{0.5cm} (1)

The product of mass and acceleration of an object is equal to the sum of the forces. This is known as Newton’s second law.

When it comes to a spring, a specific force is present, which tends to pull it back to its resting position – this is known as Hooke’s law:

x_0) \hspace{0.5cm} (2)

Where k is the spring constant, a measure of its stiffness, l0 is the resting position and l is the position of the spring.

Another thing to take note of is derivatives. Speed is the derivative of the position with respect to time, acceleration is the derivative of velocity with respect to time:

\frac {dx} {dt}\ a(t) = \frac {dv} {dt} = \frac {d^2x} {dt^2}

If we combine the two previous equations (1 and 2), we get the following:

m\frac {d^2x} {dt^2} = -k(x(t) – x_0)

For any of you that have studied maths on a higher level, you’ll recognize that what we have is a differential equation, i.e an expression of a function based on its derivatives. The mathematical resolution is beyond the scope of this article, but the essential thing to retain is that we can get an expression of the position of an objet at a given time based on the forces that act upon it. The solution of the above differential equation is the following:

cos(\omega t), \hspace{0.5cm} \omega = \sqrt {\frac k m}

View the blog post to see a simulation of a spring controlled by such a force.

You’ll instantly notice something that feels wrong with the animation: It continues infinitely! That’s because there’s no loss of momentum. In reality, there is energy loss which leads the spring to a stable state. In a spring, this is called damping. Think of it as an object attached to a spring scraping against the ground. The friction slows the object, bringing it to an eventual stop.

View the blog post to see the simulation.

The damping force is represented as a function of velocity:

F_d = -\alpha v \hspace{0.5cm} (3)

By combining all the forces above (1, 2 and 3), we get the following differential equation:

\frac {d^2x} {dt^2} = -k (x – x_0) – \alpha \frac {dx} {dt}

With the following solution:

e^ {-\beta t } cos(\omega t), \hspace{0.5cm} \beta = \frac \alpha {2m}, \hspace{0.5cm} \omega = \sqrt {\frac k m – (\frac \alpha {2m})^2}

As the damping force represents energy loss through friction, which is almost always present in the real world, it leads to a much more realistic simulation.

Phew! That was a lot of physics. Let’s see how we can implement this in the context of a web-based animation.

Spring animations

Let’s look at the basics of a spring animation. There are three main things we can control:

Mass (m)
Spring stiffness (k)
Damping (d)

You’ll notice that, contrary to usual CSS animations, we don’t control the duration of an animation. This is because controlling timing makes you lose control of speed. For example, a 0.2s-long animation could be rather slow for an element which doesn’t move much, but it could be extremely fast for an element moving across the screen. We can get a precise idea of an easing function by studying a graph of it: On the horizontal axes time is represented, and the progress on the vertical axis:

View the blog post to see the simulations.

Easing or timing function?

The terms “easing function” and “timing function” are used interchangeably to define the function that specifies the speed curve of an animation (W3 schools). However, I prefer and will use the term “easing function” as timing function seems to imply that you can control the duration of an animation, and I just mentioned that we are not seeking to do that.

When we think of a spring, we tend to think of it as being bouncy. However, springs don’t necessarily have to bounce; they actually have two characteristics:

Overdamped : The spring doesn’t bounce.
Underdamped : The spring will bounce.

The condition for an overdamped spring is the following:

\alpha > \sqrt {4mk}

Try adjusting the sliders in the following simulation to see how each parameter adjusts speed and bounce.

View the blog post to see the simulation.

CSS implementation

There are a few ways to implement spring animations. As we have to consider characteristics such as mass, and damping, Bézier curves are ineffective for this. I first wanted to try a CSS-only implementation, to make spring animations as easy as other easing animations, however there are a few difficulties to consider:

At its native level, CSS does not allow the implementation of user-defined functions
There is no way to use transition easing functions other than the ones provided (ex: ease-in or quad), or using Bézier curves

This means that we need to use a CSS pre-processor, such as Sass, to calculate the easing function. This is how it works: We’ll first define functions in Sass to calculate our spring values, then implement these as an animation. Next, we’ll use CSS animations as substitutes for transitions. Here’s the function defined in SASS:


@use ‘sass:math’;

@function get-animation($time, $stiffness, $mass, $damping, $from, $to){
  $alpha: calc($damping / (2 * $mass));
  $omega: math.sqrt(calc($stiffness / $mass) – math.pow($alpha, 2));

  @return calc($from + ($to – $from) * (1 – math.pow(math.$e, calc(-1 * $alpha * $time / 50)) * math.cos($omega * calc($time/ 50))));
}

@mixin spring-animation($name, $attribute, $prefix, $suffix, $from, $to, $mass, $stiffness, $damping) {
  @keyframes test {
    @for $i from 0 through 200 {
      #{$i * .5%} {
        #{$attribute}: #{$prefix} + get-animation($i, $stiffness, $mass, $damping, $from, $to) + #{$suffix};
      }
    }
  }
}

@include spring-animation(‘test’, transform, ‘translateX(‘, ‘rem)’, 0, 5, .5, 10, 1);

This code uses Sass math functions, which are only available in Dart Sass. Check which flavor of Sass you have installed before proceeding!>

With the parameters I supplied, it compiles to the following code:

View the blog post to see the code.

As you can see, implementing spring functions in CSS is complex, and involves hacky workarounds. While the SASS function is simple enough, it still lacks functionality such as timing and compiles to CSS which is over 600 lines long. It’s much easier to do this in Javascript. One could implement an equivalent spring function manually, or use libraries which already support spring animation, such as framer-motion.

I found an npm package called sass-spring which allows for more control, however the documentation is very sparse and I couldn’t get it to work.

Javascript implementation

Framer motion, as its name suggests, is a tool created by Framer to make it easier to implement animations in React. It provides motion-enabled equivalents for many HTML elements, such as divs. The great thing about this package is that it allows for much more flexibility than CSS transitions, for example animating to an auto width.

Here’s an example of animating with framer-motion in React. We’re creating a draggable div that animates back to its resting position when released.

View the blog post to see the simulation.

It’s this easy! The great thing is that you can save transitions in a file so that they’re easy to reuse in your project and so that they stay consistent:

View the blog post to see the simulation.

If you want to implement spring animations manually in Javascript, Maxime Heckel wrote an excellent article on it!

Design considerations

There are a couple of things to keep in mind when animating: accessibility and performance.

Privilege GPU-based animations

In a web browser, animations can be processed by the GPU or the CPU. It’s best practice to run animations on the GPU: it’s specifically designed to handle graphics, whereas the CPU is less optimal for the task as it has to run other processes as well. Such animations that run on the GPU are called composited animations. I won’t go into details, but for the most part, animations leveraging the transform or the opacity properties are composited.

Let’s see how one might implement this for an animation that expands content into full-screen. We don’t want the content itself to change size, we just want the container to clip it. One might think of leveraging the clip-path property, however animations on this property are non-composited so they aren’t optimized. Instead, one could scale the container, and apply an inverse scaling on the content. This ensures that we use composited animations, and setting the overflow property to hidden will prevent children from appearing beyond the container limits.

The container with overflow set to hidden, clipping its content when one scales it. The content has an inverse scaling to make sure it stays the same size.

Here’s how one could program it:

import { useState } from ‘react’;
import { motion } from ‘framer-motion’;

const transition = {type: “spring”, stiffness: 100, damping: 15};
const scale = 0.8;

export default function App(){
  const [expanded, setExpanded] = useState(false)

  return (
    <motion.div onclick="{()"> setExpanded(!expanded)} className=’container’ style={{overflow: ‘hidden’, width: ’10rem’, height: ’10rem’}} transition={transition} initial={false} animate={{borderRadius: expanded ? 0 : 2 / scale + ‘rem’, transform: ‘scale(‘ + (expanded ? 1 : scale) + ‘)’}}&gt;
      <motion.div style="{{width:" height: padding: background: classname="inner" transition="{transition}" initial="{false}" animate="{{transform:" : scale>
          <p>Click to {expanded ? ‘minimize’ : ‘expand’}</p>
      </motion.div>
    </motion.div>
  )

}

Lastly, avoid animating properties which are heavy on resources, such as blur.

Make sure spring animations are necessary

As spring animations do have a bit of overhead compared to classic CSS animations, make sure the additional step of implementing a spring animation will be noticed. Animations are most distinguishable when using properties that influence the scale and position of an element. Other properties, such as opacity or color, are clipped outside of their bounds, meaning that you won’t see a bounce effect. These properties can be animated using classic CSS easing functions as it’s difficult to perceive the difference.

View the blog post to see a comparison chart.

Put native interactions first

For many device interactions, an OS implementation already exists. Scrolling, for example, is specific to each device, and the user is already comfortable with the way their device implements these through continued interaction. Before overriding the default interactions, consider whether it’s really necessary: It could leave the user confused that things aren’t working the way they’re used to. Plus, you run the risk of breaking accessibility features implemented by the operating system.

For example, if you want to implement a smooth scroll animation when navigating to an element on the page, instead of implementing your own version of it, use the native API: window.scrollTo. This will make sure that the OS handles scrolling, and that it will animate the way the user is used to. Here’s an example of smoothly scrolling to an element in React:

export default function App(){
  function scroll(e){
    e.preventDefault(); //Prevent default “jump” scrolling behavior
    const element = document.querySelector(e.target.getAttribute(‘href’));
    const rect = element.getBoundingClientRect();
    window.scrollTo({top: window.scrollX + rect.top, behavior: ‘smooth’});
  }

  return (
    &lt;&gt;
      <a onclick="{smoothScroll}" href="#element">Click me to scroll to #element</a>
      <div id="element">
        <p>I’m #element!</p>
      </div>
    &gt;
  )

}

Complement, not distract

Animations should complement the user, not distract them. Like the annoying cookie banner example above, use of animations shouldn’t grab attention but merely improve the user experience by providing context. If an object expands from a closed state, it shows that it is the same and provides more context.

Animation frequency

Consider the amount of times the user will be triggering the animation: If it’s just a few times, an animation could surprise the user and give originality to your website. If the user experiences the animation quite often, however, it could start to feel sluggish.

For example, launching an application on Mac never results in the window animating as it opens. This is because during normal use, a user could be opening dozens of applications a day, making the animation redundant. Even though it could be perceived as janky to see a window pop up without notice, this is expected by the user as they triggered it. Plus, it makes launching applications feel faster.

In contrast, even though a user could be receiving many notifications a day, these are unexpected, meaning that they often aren’t user-initiated. A sliding animation helps the user contextually.

Not every design follows this. For instance, Microsoft in its 2016 Office update decided to animate all cursor movements in Word, Powerpoint and Excel. This is a great example of what not to do: users generally type thousands of characters, meaning the animation runs thousands of times, and instead of feeling snappy, like the press of a key, it feels sluggish. The typing of a character is user initiated, so it’s expected, making it a great candidate for a non-animated transition (I should also note that Microsoft has been known for its questionable design practices, such as putting the Shut Down and Restart buttons in the Start menu, something I didn’t think much of until an older friend pointed out the oxymoron).

Animated caret in Word 2016

The caret isn’t animated in the current version of Word for Mac

A flowchart to help decide whether you should animate an object of not

Lastly, beware of animations that can become associated with annoyances, such as an animation appearing consistently during frustrating moments.

The spinning circle of doom, on Windows and MacOS respectively.

Conclusion

We’ve seen the evolution of animations in the past fifty years, as well as some examples of good and bad animation design: Not every animation improves the user experience. We should keep in mind not to use animations as distractions, and to be careful of animations that can quickly become frustrating. After time, one develops an intuition for which animation is right. I discuss examples of good design animation in the following post.

We’ve seen CSS and Javascript implementations of spring animations, which are a realistic representation of everyday physics. Look out for the everyday animations you see, such as the swaying of a branch in the wind, or opening an app on your smartphone, and see if you can distinguish which type of animation it is: damped or undamped, bouncy or not.

Let me know what you think and don’t hesitate to share your thoughts over on Twitter!

References

https://phys.libretexts.org/Bookshelves/University_Physics/Book%3A_University_Physics_(OpenStax)/Book%3A_University_Physics_I_-_Mechanics_Sound_Oscillations_and_Waves_(OpenStax)/15%3A_Oscillations/15.06%3A_Damped_Oscillations

Framer Motion documentation

Sass documentation

This article was retrieved from felixrunquist.com.

Creating 3D models in Spline for Three.js

Felix — Mon, 10 Jul 2023 10:42:35 +0000

If you’ve been online lately, you’ve probably heard of Three.js, a web framework for 3D graphics. What you can do with it is absolutely stunning. However, if you want to display any complicated models which go beyond what a cube or a sphere can do, it’s best to create the model in a graphics application and then import it in Three.js. That’s where Spline comes in.

People have already built incredible things with Three.js. Here are a few examples: My room in 3D by Bruno Simons
Atmos by Leeroy
Joshua’s world

What is Spline?

Spline is a new online-based 3D software that allows just about any designer to create with intuitive tools. It doesn’t overwhelm you with menus and tools, it has just the basics. It has both a free and a paid version, but for most, the free tier does enough.

Creating a 3D model in Spline

To show what can be done, I decided to make a simple model: a “Roundel” – a circular disk with my logo which I was planning on using as part of my website identity.

To start off, create an account in Spline and open a new project. It will show you an window with a rectangle and a light, an empty slate if you will.

Opening a new project in Spline

If you’ve already used a 3D editor before, this should feel like child’s play. Otherwise, it’s a bit like using a 2D graphics editor like Figma or Sketch, except that everything is now in 3D.

There are four main sections to the editor:

On the left, the items list shows the objects you have in your 3D scene.
On the top center, a toolbar has options for adding shapes and lights, exporting, and previewing.
On the right, the edit panel allows you to change attributes of the selected object.

Most of the commands are intuitive and explained by the onboarding process, but here are the most important ones:

Alt + Drag – pan around the 3D scene
Scroll – Shift around the 3D scene
Alt + Scroll – Shift around on one axis only

In order to make my roundel, I started off by adding a circle. This is done by selecting “+” on the toolbar and selecting ellipse (O). Click and drag in the scene to place the circle. In order to make the ellipse circular, hold shift while dragging to constrain proportions.

We now have a 2D circle. To make it have volume and convert it to a cylinder, we can “extrude” it: Select the circle and add a value to the “extrusion” slider, under the shape menu. It might not look like we’ve changed much at first, but if you Alt+Drag, you’ll see that the circle now has a volume.

My extruded cylinder

I found the corners a little too sharp for my liking, so I decided to add a bevel to the cylinder’s vertices. This can be done by dragging the “Bevel” slider right under the extrusion slider.

Next, it’s time to add some text. This is delightfully easy in Spline, and I wish other 3D editors had the same ease-of-use. Select the “T” icon in the toolbar, and click on one of the surfaces of the cylinder. Each surface should highlight in red as you hover over it, indicating that the text will snap to it.

Each surface highlights when you hover it with the text tool.

Type your text and don’t worry if you don’t see anything – Every object you add to the scene has a default dull gray color – we’ll tweak the colors next!

Select the ellipse and add the color you want in the “Material” section of the object editor. I decided to go with my site’s pastel orange. I colored the text in black in the same way.

My colored cylinder and text

I wanted to change the font to reflect my site’s wordmark. This was extremely easy to do: select the text, go to the “font” menu. You can even upload a custom font!

My cylinder with cusomized font, rescaled

Next, to make everything 3D, I also extruded the text.

The last part is lighting. Since we’ll be exporting the design in a file format that lacks support for lighting, you don’t have to worry about this part, but I still believe this deserves mentioning. In the physical world, multiple cues exist to give us information about an object’s physical location: The stereoscopic information from our eyes, the texture of the object and the shadows it casts. Since we don’t get stereoscopic information on screens, we need to rely more heavily on other cues such as the reflectivity of the object and the shadows it casts. That’s where different lights come in play.

The three different light types in Spline

I won’t dive into the specifics of each light as it’s beyond the scope of this article, but you can play around with the different lights and see which one looks best. Most of the time I use a point light which is a single source of omnidirectional light. I like to position it so that it casts nice shadows – this is why I extruded the “Felix” lettering. When you’re in the editor, lights are represented by wireframes. Below you can see my model with a light in the top left corner casting shadows from my lettering.

My point light

Exporting

There are multiple formats you can export your 3D file in, some are free and others aren’t (for instance, you can export in formats suited for 3D printing or augmented reality). The easiest way for displaying the 3D model on the web is to use the embedded viewer or the code export, which provides you with code to import the model and display it in your web application.

However, I try to be careful about the technology I use, so I don’t get locked up in a proprietary “ecosystem”, and as Spline is closed-source, I feel more comfortable exporting the model to an open-source file format, even though it does complicate things a bit. To combat this, I’ll show you how you can do this in a React/Node.js project for free. After much trial and error, here’s the workflow I came up with:

Export the model as GLTF or GLB (It’s best to use GLB in production as it’s a compressed version of GLTF, GLTF being just JSON). It’s one of the free exports and is supported by Three.js. Save that to the public directory of your web project. The issue with spline exports to GLTF/GLB is that it doesn’t retain lighting or materials, only geometries are exported, however there is a workaround for that.

I use Next.js for my projects, this is what I found for that but a lot of it applies to other frameworks!

In order to see the different export options, click the “Exports” button on the middle toolbar.

The exports panel

If you go back to the “Exports” panel of your spline file and select the “Code export” tab, Spline automatically generates the positioning as well as the lighting in Three.js format. You can select which flavor of Three.js you use (I use react-three-fiber), and copy the code. For instance, here’s the code for my model:

/*
  Auto-generated by Spline
*/

import useSpline from ‘@splinetool/r3f-spline’
import { PerspectiveCamera } from ‘@react-three/drei’

export default function Scene({ …props }) {
  const { nodes, materials } = useSpline(‘https://[external spline URL]’)
  return (
    &lt;&gt;
      <color attach="background" args="{['#74757a']}"></color>
      <group dispose="{null}">
        <pointlight name="Point Light" castshadow intensity="{0.82}" distance="{1379}" shadow-mapsize-width="{1024}" shadow-mapsize-height="{1024}" shadow-camera-near="{100}" shadow-camera-far="{100000}" position="{[-254.65,"></pointlight>
        <group name="Group">
          <mesh name="Text" geometry="{nodes.Text.geometry}" material="{materials['Text" castshadow receiveshadow position="{[-46.56," rotation="{[0,"></mesh>
          <mesh name="Ellipse" geometry="{nodes.Ellipse.geometry}" material="{materials['Ellipse" castshadow receiveshadow position="{[-8.73," rotation="{[0," scale="{1}"></mesh>
        </group>
        <perspectivecamera name="1" makedefault="{true}" far="{100000}" near="{70}" fov="{45}" position="{[-949.88," rotation="{[-0.46," scale="{1}"></perspectivecamera>
        <hemispherelight name="Default Ambient Light" intensity="{0.75}" color="#eaeaea"></hemispherelight>
      </group>
    &gt;
  )
}

Add the code provided from your Spline file as a new component (make sure to add the code provided by your project and not the one above as it’s project-specific!). You’ll notice that in the code there’s an external link to your project hosted on Spline servers, but don’t worry about that, we’ll get to it.

Next, you’ll want to install the following repositories to your project using npm or yarn:

@react-three/fiber
three
@react-three/drei

We’re going to alter the code provided by Spline to import the self-hosted GLTF file. Replace the line with useSpline with the following code:

//Replace ‘/felix_roundel.glb’ with the path to your gltf/glb file
const gltf = useLoader(GLTFLoader, ‘/felix_roundel.glb’, loader =&gt; {
const dracoLoader = new DRACOLoader();
  dracoLoader.setDecoderPath(‘/draco/gltf/’);
  loader.setDRACOLoader(dracoLoader);
})
const { nodes, materials } = gltf;

This uses a “loader” which is a function that loads your 3D file. We use DRACOLoader as well, as it’s an optimization library for importing point clouds and meshes.

Next, you’ll want to head to the following path of your project: /node_modules/three/examples, and copy the draco file to the public folder of your project. This is one of the quirks of the dracoLoader: It not only needs a library import, but it needs to import a decoder using the setDecoderPath method (see https://stackoverflow.com/questions/56071764/how-to-use-dracoloader-with-gltfloader-in-reactjs). You can supply it with a local decoder hosted in your project’s public folder as I describe above, or alternatively you can link it to an externally-hosted library.

Remove the default export in the Scene function, and wrap it around a <Canvas> element. This is needed to prevent errors with paths (see https://stackoverflow.com/questions/75950437/nextjs-react-three-drei-typeerror-failed-to-parse-url). I know, there’s a lot of quirks to navigate around, but it’ll be worth it in the end! This is what your code should look like for now:


import { Canvas } from “@react-three/fiber”;
import { GLTFLoader } from “three/examples/jsm/loaders/GLTFLoader”;
import { DRACOLoader } from ‘three/examples/jsm/loaders/DRACOLoader’
import { PerspectiveCamera } from ‘@react-three/drei’
import { useLoader } from “@react-three/fiber”;

export function Object(){
  const gltf = useLoader(GLTFLoader, ‘/felix_roundel.glb’, loader =&gt; {
  const dracoLoader = new DRACOLoader();
    dracoLoader.setDecoderPath(‘/draco/gltf/’);
    loader.setDRACOLoader(dracoLoader);
   })
   const { nodes, materials } = gltf;
  return (
    &lt;&gt;
      <color attach="background" args="{['#74757a']}"></color>
      <group dispose="{null}">
        <pointlight name="Point Light" castshadow intensity="{0.82}" distance="{1379}" shadow-mapsize-width="{1024}" shadow-mapsize-height="{1024}" shadow-camera-near="{100}" shadow-camera-far="{100000}" position="{[-254.65,"></pointlight>
        <group name="Group">
          <mesh name="Text" geometry="{nodes.Text.geometry}" material="{materials['Text" castshadow receiveshadow position="{[-46.56," rotation="{[0,"></mesh>
          <mesh name="Ellipse" geometry="{nodes.Ellipse.geometry}" material="{materials['Ellipse" castshadow receiveshadow position="{[-8.73," rotation="{[0," scale="{1}"></mesh>
        </group>
        <perspectivecamera name="1" makedefault="{true}" far="{100000}" near="{70}" fov="{45}" position="{[-949.88," rotation="{[-0.46," scale="{1}"></perspectivecamera>
        <hemispherelight name="Default Ambient Light" intensity="{0.75}" color="#eaeaea"></hemispherelight>
      </group>
    &gt;
  )
}

export default function Scene(){
  return (
    <canvas>
      <object></object>
    </canvas>
  )
}

If you run the code, you’ll notice that everything looks a bit sad and gray. This is because react-three-fiber can’t find the materials in the GLTF/GLB file (exporting textures is a premium feature). We’ll add our own materials for the meshes. There are different materials available in Three.js, the most useful one I find is MeshPhongMaterial. It’s realistic and allows for shininess and shadows, and responds well to different types of lights. This is how you can define a material:

const material = new THREE.MeshPhongMaterial({
    color: 0x000000,
    transparent: false, opacity: 0.5,
    specular: 0x050505,
    shininess: 100
});

The color is provided in hexadecimal format, for instance if one wanted a bright red one would type 0xff0000. I had a few issues with color fidelity in my model (my oranges looked yellow, many colors appeared washed out), to combat this I used THREE.Color:

const color = new THREE.Color(‘#FF9A03’).convertSRGBToLinear();
const material = new THREE.MeshPhongMaterial({
    color: color.getHex(),
    transparent: false, opacity: 0.5,
    specular: 0x050505,
    shininess: 100
});

Create as many materials as you need and replace add them to the materials attribute of each mesh. As I wanted my model to have a transparent background, I also decided to remove the <color> tag.

You should be done! This is the final code for my model. You can view the output it generates here.


import { Canvas } from “@react-three/fiber”;
import { GLTFLoader } from “three/examples/jsm/loaders/GLTFLoader”;
import { DRACOLoader } from ‘three/examples/jsm/loaders/DRACOLoader’
import { OrbitControls, PerspectiveCamera } from ‘@react-three/drei’
import { useLoader, } from “@react-three/fiber”;
import * as THREE from ‘three’

export function Object(){
  const gltf = useLoader(GLTFLoader, ‘https://felixrunquist.com/felix_roundel_2.glb’, loader =&gt; {
  const dracoLoader = new DRACOLoader();
    dracoLoader.setDecoderPath(‘https://raw.githubusercontent.com/mrdoob/three.js/dev/examples/js/libs/draco/’);
    loader.setDRACOLoader(dracoLoader);
   })
   const { nodes, materials } = gltf;
   const color = new THREE.Color(‘#FF9A03’).convertSRGBToLinear();
   const ellipseMaterial = new THREE.MeshPhongMaterial({
        color: color.getHex(),
        transparent: false, opacity: 0.5,
        specular: 0x050505,
        shininess: 100
   });
   const textMaterial = new THREE.MeshPhongMaterial({
        color: 0x000000,
        transparent: false, opacity: 0.5,
        specular: 0x050505,
        shininess: 100
   });
  return (
    &lt;&gt;
      <group dispose="{null}">
        <pointlight name="Point Light" castshadow shadow-mapsize-width="{1024}" shadow-mapsize-height="{1024}" shadow-camera-near="{100}" shadow-camera-far="{100000}" position="{[-254.65,"></pointlight>
        <group name="Group">
          <mesh name="Text" geometry="{nodes.Text.geometry}" material="{textMaterial}" castshadow receiveshadow position="{[-46.56," rotation="{[0,"></mesh>
          <mesh name="Ellipse" geometry="{nodes.Ellipse.geometry}" material="{ellipseMaterial}" castshadow receiveshadow position="{[-8.73," rotation="{[0," scale="{1}"></mesh>
        </group>
        <perspectivecamera name="1" makedefault="{true}" far="{100000}" near="{70}" fov="{45}" position="{[-949.88," rotation="{[-0.46," scale="{1}"></perspectivecamera>
        <hemispherelight name="Default Ambient Light" intensity="{0.75}" color="#eaeaea"></hemispherelight>
      </group>
    &gt;
  )
}

export default function Scene(){
  return (
    <div style="{{width:" height:>
      <canvas shadows>
        <orbitcontrols></orbitcontrols>
        <object></object>
      </canvas>
    </div>
  )
}

If you aren’t seeing any shadows, add the shadows attribute to your \<\Canvas\> tag.

All done! I hope this helps you get started with Spline and Three.js. Let me know if you have any issues and what 3D projects you make!

This article was retrieved from felixrunquist.com.

Adding search to a React/Next.js blog

Felix — Sun, 02 Jul 2023 11:15:09 +0000

When I built my Next.js-based website, one of the things I really wanted to implement was the ability to search through posts. I recently came across Kent C. Dodd’s match-sorter package, a Node.js package which allows for speedy filtering through text arrays. It immediately made me think of its usage as a search algorithm.

Here’s the idea: On the client, we can provide the page with an array of post metadata: titles, excerpts and slugs. We add an event listener to the search input which runs matchSorter on the array with the content of the search input, and we display the resulting posts.

I had a look at Kent C. Dodd’s website and it turns out that he uses match-sorter as well in his search algorithm!

The basic usage of match-sorter is really simple: You provide it an array and a string to match with, and it returns an ordered array based on what ranks the highest. This makes for a really simple implementation below, with react functions to manage state:

import { matchSorter, rankings } from ‘match-sorter’
import {useState, useEffect} from ‘react’
const search = [‘Hi’, ‘Hello’, ‘Hi there’]
export default function App(){
  const [input, setInput] = useState(‘\’)
  const [results, setResults] = useState([])
  useEffect(() =&gt; {
    var res = matchSorter(search, input)
    setResults(res)
  }, [input])
  return (
    &lt;&gt;
      <input type="text" value="{input}" oninput="{e"> setInput(e.target.value)}/&gt;
      {results.map(i =&gt; <p key="{i}">{i}</p>)}
    &gt;
  )
}

However, in the case of an article search system, we want to do more: we’d like to be able to search not only through an array of titles, but through other information, such as a short excerpt for example. This data will be fetched from the CMS and provided to the browser, resulting in an object like so:

const posts = [
  {title: ‘How I built my website’, excerpt: ‘Trying to figure out how to build my website was a challenge for me. As a web designer, I wanted it to be fully customisable, but I also wanted ease of use to reduce friction and make it easy to maintain.’, url: ‘https://felixrunquist.com/posts/how-i-built-my-website’},
  {title: ‘Implementing online payments with Stripe’, excerpt: ‘A tutorial on setting up a credit card payment form on your NextJS site. We will cover signing up and the Stripe SDK’, url: ‘https://felixrunquist.com/posts/online-payments-stripe’},
  {title: ‘Next.js HTTP Authentication with JWT and cookies’, excerpt: ‘I was recently trying to figure out a way to implement simple HTTP authentication for a personal Next.js project. I augmented the basic HTTP auth with JWT and cookies’, url: ‘https://felixrunquist.com/posts/next-js-http-authentication-jwt-cookies’},
  {title: ‘Creating and ordering a custom PCB’, excerpt: ‘As part of a home automation project, I wanted to house a thermometer, motion sensor and display in a small form factor. I decided to design and order my own PCB.’, url: ‘https://felixrunquist.com/posts/creating-and-ordering-custom-pcb’}
]

The smart thing about the matchSorter function is that it can sort through arrays of objects as well: You can tell which key you’d like it to sort through, and you can even provide specific criteria for each key: for example, you can choose to match exact case, or complete equality.

Let’s see how one could implement this:

const posts = [
  {title: ‘How I built my website’, excerpt: ‘Trying to figure out how to build my website was a challenge for me. As a web designer, I wanted it to be fully customisable, but I also wanted ease of use to reduce friction and make it easy to maintain.’, url: ‘https://felixrunquist.com/posts/how-i-built-my-website’},
  {title: ‘Implementing online payments with Stripe’, excerpt: ‘A tutorial on setting up a credit card payment form on your NextJS site. We will cover signing up and the Stripe SDK’, url: ‘https://felixrunquist.com/posts/online-payments-stripe’},
  {title: ‘Next.js HTTP Authentication with JWT and cookies’, excerpt: ‘I was recently trying to figure out a way to implement simple HTTP authentication for a personal Next.js project. I augmented the basic HTTP auth with JWT and cookies’, url: ‘https://felixrunquist.com/posts/next-js-http-authentication-jwt-cookies’},
  {title: ‘Creating and ordering a custom PCB’, excerpt: ‘As part of a home automation project, I wanted to house a thermometer, motion sensor and display in a small form factor. I decided to design and order my own PCB.’, url: ‘https://felixrunquist.com/posts/creating-and-ordering-custom-pcb’}
]//Retrieved by the CMS

import { matchSorter, rankings } from ‘match-sorter’
import {useState, useEffect} from ‘react’
export default function App(){
  const [input, setInput] = useState(‘\’)
  const [results, setResults] = useState([])
  useEffect(() =&gt; {
    var res = matchSorter(posts, input, {keys: [{key: ‘title’}, {key: ‘excerpt’}]})
    setResults(res)
  }, [input])
  return (
    &lt;&gt;
      <input type="text" value="{input}" oninput="{e"> setInput(e.target.value)}/&gt;
      {results.map(i =&gt; <a href="{i.url}" key="{i.title}"><h2>{i.title}</h2>
<p>{i.excerpt}</p></a>)}
    &gt;
  )
}

As the array to search through is being passed to the client, you probably don’t want to provide too large objects which could add to loading times, and cause performance issues. I’d limit myself to one thousand items!

We’ve now implemented a basic search function, that has an input, a function to display results, and a function that updates results when the input is written to. Let’s say the search page is at www.example.com/search. What if one was to reload the page, or if one wanted to share the url to the exact results? One could store the current search state in the URL, with the query parameter. For instance, searching “Recipes” would dynamically update the url so that it points at www.example.com/search?q=Recipes. Like that, refreshing or sharing the search results would bring you back to the exact same state.

This can be done in React using react-router-dom (it can also be done in Next.js using useRouter), a library that manages URLs and paths. First you define a retrieval function, checking on page load whether a query string has been provided or not. When the text input is written to, you can dynamically update the query parameter.

const posts = [
{title: ‘How I built my website’, excerpt: ‘Trying to figure out how to build my website was a challenge for me. As a web designer, I wanted it to be fully customisable, but I also wanted ease of use to reduce friction and make it easy to maintain.’, url: ‘https://felixrunquist.com/posts/how-i-built-my-website’},
{title: ‘Implementing online payments with Stripe’, excerpt: ‘A tutorial on setting up a credit card payment form on your NextJS site. We will cover signing up and the Stripe SDK’, url: ‘https://felixrunquist.com/posts/online-payments-stripe’},
{title: ‘Next.js HTTP Authentication with JWT and cookies’, excerpt: ‘I was recently trying to figure out a way to implement simple HTTP authentication for a personal Next.js project. I augmented the basic HTTP auth with JWT and cookies’, url: ‘https://felixrunquist.com/posts/next-js-http-authentication-jwt-cookies’},
{title: ‘Creating and ordering a custom PCB’, excerpt: ‘As part of a home automation project, I wanted to house a thermometer, motion sensor and display in a small form factor. I decided to design and order my own PCB.’, url: ‘https://felixrunquist.com/posts/creating-and-ordering-custom-pcb’}
]//Retrieved by the CMS

import { matchSorter, rankings } from ‘match-sorter’
import {useState, useEffect} from ‘react’
import { useSearchParams, BrowserRouter as Router} from “react-router-dom”;
export default function App(){
  return (
    <router>
      <search></search>
    </router>
  )
}

function Search(){
  const [searchParams, setSearchParams] = useSearchParams();
  const [input, setInput] = useState(‘\’)
  const [results, setResults] = useState([])
  useEffect(() =&gt; {
    setInput(searchParams.get(‘search’))//Get search parameters on load
  },[])

  useEffect(() =&gt; {
    var res = matchSorter(posts, input, {keys: [{key: ‘title’}, {key: ‘excerpt’}]})
    setResults(res)
    if(input){//Update query parameters
      setSearchParams({ ‘search’: input });
    }
  }, [input])
  return (
    &lt;&gt;
      <input type="text" value="{input}" oninput="{e"> setInput(e.target.value)}/&gt;
      {results.map(i =&gt; <a href="{i.url}" key="{i.title}"><h2>{i.title}</h2>
<p>{i.excerpt}</p></a>)}
    &gt;
  )
}

That’s all there is to it! Luckily, the react-sorter package does most of the heavy lifting, we just did the content fetching and state implementation. Let me know if this was helpful by reaching out to me on Twitter!

External links:

Match-sorter documentation: https://github.com/kentcdodds/match-sorter
URL-based state: https://blog.logrocket.com/use-state-url-persist-state-usesearchparams/

This article was retrieved from felixrunquist.com

Web design inspiration: Ten perfect examples

Felix — Sun, 25 Jun 2023 18:09:46 +0000

In no particular order, here’s a collection of websites I find inspiring, both by their design and their functionality. For each website, I’ll dive into the specific details which made me want to mention it.

While I do feature personal as well as corporate projects alike, I do my best to credit the original creator when possible.

1. Carl Hauser’s blog

theprocess

This is the process. Every day sketches, brainfarts and ideas flying around in my backpack and apartment. I hope they can inspire or help someone out there.

Carl Hauser’s website is a showcase of personal design products he has made. I love the minimalist aesthetic of the website: each element is separated into a rounded box, and the color combination of beige, black and yellow works really well. On the homescreen, you’re greeted by a page of rough sketches, the date and ID on top gives them an industrial feeling.

The projects page lists more finished projects, with detailed design, color scheme and typography for each product. The intricate renders make you feel like you’re looking at a finished product, which could be the next product unveiled by Apple or Nothing.

Carl Hauser also has an Instagram page where he features his designs.

2. Sketch Blog

Beyond the Canvas · Sketch Blog

Welcome to Beyond the Canvas from Sketch — get the latest design news, inspiration, resources and tutorials from a global team of design experts.

I chose to feature the Sketch Blog not for the content, but for the design of the site – from its soft pastels to the well-chosen monospace heading font, the look and feel is stunning. The devil being in the details, I particularly like the way margins work out: the content margin is aligned with the header so they both start at the same position.

3. JP Silva’s portfolio

JP – Webflow Expert and Brand Designer

I’m a skilled Webflow expert and brand designer specializing in creating premium templates for various industries. Please view my portfolio for examples of my work, and contact me to discuss your project.

JP Silva’s website breaks the boundaries of web design and introduces interesting ways to navigate, from the rotating cubes that allude to Rubik’s cubes, to the scroll-based animations. The use of an uppercase font is interesting.

4. Vita Architecture

Vita Architecture | High-End Residential Architects in London

Vita Architecture is a boutique architectural practice in London & Surrey. Our projects are bespokely designed to deliver innovative and crafted architecture.

Vita Architecture is a great example of motion-first design. From the moment you hit “enter” in the search bar, you’re transported to a site with flowing visuals and extensive imagery. Anything from scrolling to clicking a link provides a contextual animation which makes the entire site feel fluid. Plus, this website has been previously featured by Apple in a product announcement!

The only critique I have is the customized scrolling behavior which is jarring – I don’t recommend changing the default scrolling behavior (the speed/direction) as it feels counter-intuitive for the user who is used to the way native scrolling works on their device. It can feel like wading through sludge!

5. Outreach.space

Outreach. A Mystery from Beyond.

Inspired by the true events surrounding 1I/’Oumuamua, the first interstellar object ever observed by humankind passing through the solar system. At the time of the sighting, the object has already passed the closest point in its orbit to Earth; soon it will be out of telescopic range. Its explorati…

Outreach tells the story of a space voyage, and the discovery of a weird asteroid. It makes you feel like being teleported through space, and the delightful visuals interspersed with

I love the use of Garamond, reminiscent of the typeface Apple used in the 80s. Plus, this site is built on Next.js and hosted on Vercel, just like mine, so it’s nice to see what can be done with these frameworks – there really is no limit!

The only disturbing thing is the loading modal – it isn’t interactive and scrolling on isn’t disabled when the modal displays. Since it’s a content-heavy site and therefore takes a while to load, the loading modal was displayed for a while – I thought the site was broken – providing more intent as to the loading state, for example an animation, could help with this.

6. Tinkersynth by Josh W. Comeau

Generative Art Machine

Tinkersynth is an experimental art project. Create unique designs by manipulating whimsical machines and making serendipitous discoveries.

Tinkersynth is a website that lets you create visual art by using a waveform generator, much like you would use a synthesizer to generate sounds. It’s a fun tool, with many whimsical details built into it, like each button having a little animation on click.

The skeuomorphic control panel is a fun way visualize the effects that can be applied – the little animations as you click and press on the sliders make you want to tinker with it even more. I can’t imagine how complicated everything must’ve been to code – handling all the states, transitions and animations!

Tinkersynth was built by Josh W. Comeau, a React developer and course writer.

7. Atmos

Atmos

Get on board and discover the most surreal facts about the aviation industry!

Atmos is an interactive experience built on Three.js. From the grainy yet simple visuals to the relaxing sound effects, It was developed by Leeroy, a Canadian web design agency. I particularly like the choice of colors – soft blues and warm orange tones which make you feel like you’re up in the sky. The airy music is well chosen to add to the effect, it almost makes you feel like you’re in a meditation session.

This post on Awwwards delves into the design of the site and some of the choices that were made, from the path the airplane follows to the bubbly clouds. It’s truly amazing that such an immersive experience can be created using only web technologies.

P.S: try to spot the easter egg!

8. Rauno Freiburg’s portfolio

Rauno Freiberg

Creating software that makes people feel something.

I love the shader effects, how it fades away to reveal the content when you scroll. The navigation is unconventional (it resembles the MacOS dock), but certainly functional! Rauno has rethought how users should interact with elements. The crafts page features a feed on what he is working on, and I love the interactive demos! He’s also built separate projects, uiw.tf and ui.gallery, which explore unconventional yet intuitive ways to interact with visual content.

The more time you spend on the site, the more details you see – like sound effects or the “last visit” location indicator: it fills you with a moment of unbridled glee to see your location as last visited when you refresh the page (unless you use a VPN, of course).

9. Cobbler

Cobbler

For reps, Cobbler is your producers’ best friend. For studios, Cobbler is your rep.

Something very striking is the bold color palette the designers chose to go with this site. Surprisingly enough, the saturated blue and pale green work extremely well together! The bold, outlined shapes and grainy backgrounds add to the look, and it makes for a very polished site. I’m obsessed with the font they use, Helveesti, which has mini serifs and a slight tapers on vertical stems.

If anything, the animated visuals and colorful elements could be drawing the user away from the actual content (case in point: I’ve scrolled through the website, visited a few pages, yet I can’t remember what services Cobbler provides…).

10. Cosmos.so

Manifesto — Cosmos

You are now entering the Cosmos. Where ideas are sacred, exploration is infinite and connection is cosmic. A future where free expression is possible for every curator in the universe.

Cosmos is positioning itself as the new Pinterest alternative. To showcase this, they’ve built an interactive website. In particular: the manifesto page. It’s such a calming experience, from the music to the scroll-controlled timeline, which makes a flower bloom in the background.

The dark/light theme transition is very well thought-out, the entire experience feels delightfully intuitive. An issue to point-out accessibility-wise: the scroll bar is customized as well as the scrolling behavior – this can make the user feel unsafe from the native implementations.

11. A bonus one: Bento layouts

My Bento Box

Here’s a bonus one. This is less of a particular site, but more of a general design trend that has blew up lately on Twitter. After Apple has begun using groups of little cards in a grid to summarize what has been presented, people followed suit and the form has been termed “bento box”: A group of rectangular elements in a grid. The site I’ve linked is an excellent example of Bento Boxes, and I’ll share some other ones below.

There’s even a site to make your link in bio as a bento: bento.me

A classic: The bento layout from Apple

Another layout by Bolt

A layout I came up with for my site redesign

This article was retrieved from felixrunquist.com

Website update: code blocks, search, lightbox and more!

Felix — Sun, 18 Jun 2023 17:13:45 +0000

Since I got my website to a relative production-ready stage last year, I wanted to release an update to my website to work on a few much-needed improvements.

Revamped post experience

I’ve always felt like my posts layout has felt pretty bland, which doesn’t integrate with the rest of my website design. I started off building my website from Next.js’ WordPress example, which, although modern in its design, is very minimal. I decided to add a few features to build on my existing content:

Code blocks

As I constantly write posts about programming, code blocks are an elemental part of my posts. The code blocks I started off with use Prism.js, however I knew from the beginning that this was a temporary solution. While it does support a lot of languages for syntax highlighting, it lacks support for things such as file trees, and interactive code playgrounds. A package that has been all the rave in the dev environment as of late is Sandpack by CodeSandbox. It’s a Javascript-oriented playground that features the same in-browser bundler used on CodeSandbox for running client-side node.js. It’s very performant and has excellent code formatting and file tree support. The only thing it lacks is support for languages other than Javascript, say if you want to use it not for the previews but just for code blocks.

Full-bleed images and lightbox

Most online reading experiences feature full-bleed images, meaning that images in a post expand to the size of the browser content while the text remains within its container (it’s not a good practice to have text span across wide screens as it makes it hard to read). I implemented a CSS class for these kind of images.

Another feature on sites such as Medium is that one can click on images in a post to expand them. This is called a Lightbox. You can test out the Lightbox functionality on my website at the original article.

Post headers

Something I didn’t like with Next.js’ wordpress example is the look of posts. Everything is huge, and while that might be the look they are going for, it’s tiring having to scroll everywhere. For instance, the cover image is positioned under the fold, which means that a visitor needs to scroll before deciding if they want to read the post based on the title and cover image. I decided on a responsive design to make sure the cover image is above the fold on all devices.

Next.js’ blog example with cover image under the fold

My updated design

I also decided to include small flourishes such as drop caps, which felt authentic to my design.

Post categories and search

Something I’ve wanted to do from day one is to implement a post search feature. I initially thought it would be quite time consuming, with hacky fetches and asynchronous calls to my CMS, but I realized that I could do all the searches client-side with Kent C. Dodd’s match-sorter package. I also wanted to make post categories visible to users. Since they are already implemented in the CMS I use (WordPress) I only had to do half the work and implement it client-side. Since I have a lot of whitespace, I found a space for my search bar and categories elements.

The search bar being used

Categories being displayed

Header and footer

I wasn’t very happy with the header transition I have – as you scroll and it collapses, it’s always felt a little jittery. I replaced my trigger-based animation (i.e. when you scroll past a certain point, the size transitions) with a scroll-based animation (i.e. the scroll controls the animation progress). Here’s the difference:

/App.js:

import ‘./styles.css’
import { useState, useEffect } from ‘react’
export default function App(){
const [expanded, setExpanded] = useState(true);
useEffect(() =&gt; {
document.addEventListener(‘scroll’, scroll)
return ()=&gt;{
  document.removeEventListener(‘scroll’, scroll)
}
},[])
function scroll(){
setExpanded(window.scrollY &lt;= 50)
}
return (
<div classname="container">
  <div classname="{'header'" expanded :>
    <h1>Header </h1>
    <p>{expanded ? ‘Expanded’ : ‘Collapsed’}</p>
  </div>
</div>
)
}

/styles.css:

body {margin: 0; }
* {font-family: sans-serif; }
.container {width: 100%; height: 200%; height: 200vh; }
.header {width: 100%; height: 4rem; background: #8CC0DE; transition: height .3s ease-out; position: fixed; }
.header p, .header h1 {margin: .5rem 0; display: inline-block; }
.header.expanded {height: 8rem; }

Trigger-based animation

/App.js:

import ‘./styles.css’
import { useState, useEffect } from ‘react’
export default function App(){
const [progress, setProgress] = useState(0);
useEffect(() =&gt; {
document.addEventListener(‘scroll’, scroll)
return ()=&gt;{
  document.removeEventListener(‘scroll’, scroll)
}
},[])
function scroll(){
setProgress(Math.max(Math.min(window.scrollY / 50, 1), 0))
}
return (
<div classname="container">
  <div classname="{'header'}" style="{{'--progress':" progress>
    <h1>Header </h1>
    <p>Progress: {progress}</p>
  </div>
</div>
)
}

styles.css:

body {margin: 0; }
* {font-family: sans-serif; }
.container {width: 100%; height: 200%; height: 200vh; }
.header {width: 100%; height: 4rem; background: #8CC0DE; position: fixed; }
.header p, .header h1 {margin: .5rem 0; display: inline-block; }
.header {height: calc((1 – var(—progress)) * 8rem + var(—progress) * 4rem); }

Scroll-based animation

Notice how in the first example, the header stays expanded, right until you scroll past a certain point? The scroll-based animation feels much more responsive as it adapts to your scrolling speed.

When it comes to the footer, I never got to finishing it. I spent time positioning the links to social media and other items I wanted to feature.

Design

To make the colors and fonts I use on this website easily retrievable, I made a design page which groups common elements. You can find the page here.

A few elements featured on my design page

A never-ending journey

I believe a website is never fully finished, and there are always tweaks that can be made. I’ve compiled a list of things I’d like to change, and you can track my progress here. As a bit of fun, I wanted to explore the current trend that are bento grids and create one featuring everything I’ve changed as part of the redesign.

This article was retrieved from felixrunquist.com

Automatically cross-publishing posts to dev.to with RSS

Felix — Thu, 15 Jun 2023 17:02:32 +0000

While one might think that cross-publishing from your own site to other platform might diminish the reach of your own website, there’s actually a way to do this while boosting your site’s credibility. Plus, this can also be done (semi-)automatically! Dev.to is a developer-centric social network. It’s a place where people can publish and read articles about software development.

Cross-publishing, or syndication, is the act of publishing a post on your website onto third-party sites, such as dev.to. This generally improves the visibility of your website, as long as it’s done right.

Canonical links

Crawlers rely on links to index websites. Used as a way to indicate duplicate content, canonical links also provide an indicator to crawlers to instead direct users to the provided url when they search for the page:

<link rel="canonical" href="https://felixrunquist.com" />

Dev.to provides a setting to enable canonical links on articles you publish on the platforms. This means that crawlers will see that article as if it was published on your website, and you will benefit from the enhanced visibility that comes from publishing on a developer platform!

Beware! If you don’t provide a canonical link, the cross-published content could rank higher in search results than your website, which would drive traffic away!

Auto-publishing from your RSS feed

In order for dev.to to automatically detect when you’ve posted on your website, you can provide it an RSS feed for it to fetch content from (if you don’t have an RSS feed, fret not, I explain what it is and how to set one up with Next.js in this article). It’s a matter of heading to Settings > Extensions, in dev.to, and scrolling to the “Publishing to DEV Community from RSS” section. Here, you can provide the link to your RSS feed.

The RSS section in dev.to settings

You can click on the “Fetch Now” button for the site to retrieve the latest content. Content isn’t published automatically, but it is added into your drafts. I selected the option to mark the RSS source as canonical, which will let dev.to automatically handle the canonical links I mentioned earlier.

Publishing posts

In order for the site to accurately fetch content from your website, you need to have a few fields configured in your RSS feed for each post:

<title> – this will be used for your post’s title
<link> will be used for cross-linking back to the post
<description> contains a brief description of the post
<content> has the actual post content. You can provide it as raw text, but you can also put html content in a <cotnent:encoded> tag.

Once the post lands in your dev.to drafts (visible by going to your dashboard), all that’s left to do is to preview the article to make sure that everything is formatted correctly, as there could be a few issues especially if you used highly-customized elements. Next, change the published attribute in the Markdown metadata to true, and it will be visible to others once you hit save!

If you’re unfamiliar with Markdown and are unsure of the syntax, the dropdown under “Editor Basics” in the editor is extremely helpful!

The markdown metadata of a published post

Generating a sitemap and RSS feed with Next.js

Felix — Sun, 04 Jun 2023 14:33:25 +0000

There are many options for generating content with Next.js. These different methods can be used for automatically generating sitemaps and RSS feeds based on content fetched from a CMS. I’ll explore the different options available, their pros and cons.

The difference between Statically and Dynamically rendered content

Next.js terminology can be a little befuddling. To most people, SSR, SSG and ISR are nothing but meaningless acronyms. Fundamentally, Next.js projects can serve content in three different ways:

Static content : These are files put in the /public directory and are served by next.js as they are. For example, an image stored at /public/image.png will be served at www.example.com/image.png.
Build-time content : This content is created at built time. On a page, this is characterized by exporting a getStaticProps function. In Next.js jargon, they call it Static Site Generation (SSG), a name I don’t find very appropriate as the content isn’t completely static (it’s built), and the entire site doesn’t have to use this. It would be more appropriate to call this Build Generation.
Dynamic content : This content is created every time someone visits a page. For example, this is what a PHP-based site does for every page written in PHP. On a Next.js page, this is characterized by exporting a getServerSideProps function. It’s commonly called Server Side Rendering (SSR), which isn’t very appropriate as it implies that other forms of generating content don’t involve any action from the server. I’ll call this Dynamic Generation.

As for Incremental Static Regeneration (ISR), it’s a balance between build-time and dynamic content: The content is generated at build time, but is regenerated in the background on every request (meaning that the visitor doesn’t get disrupted), in a specified window.

This is done by specifying a revalidate time:

export async function getStaticProps() {
  const res = await fetch(‘https://…/posts’);
  const posts = await res.json();

  // If the request was successful, return the posts
  // and revalidate every 10 seconds.
  return {
    props: {
      posts,
    },
    revalidate: 10, //This is the revalidate time in seconds
  };
}

Phew, that was a lot of information! Now we can finally get to the bottom of sitemaps and rss feeds. For each content serving method, one can serve files such as sitemaps or RSS feeds in an equal way.

Methods of serving built XML files

What is a sitemap?

A sitemap is an XML-formatted file that is basically an index of your site: It provides a way for crawlers (search engine robots) to see the pages available on your site, and index them properly. This improves visibility to search engines.

Static sitemaps

The first, most obvious option is to hand-code a sitemap and have it live in your /public folder. Here’s an example:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://felixrunquist.com</loc>
</url>
<url>
<loc>https://felixrunquist.com/posts</loc>
</url>
<url>
<loc>https://felixrunquist.com/contact</loc>
</url>
<url>
</urlset>

However, the obvious issue here is that it every time you add a new page to your site, or if you fetch content via CMS (Content Management System), you need to remember to hand-code the changes yourself. This can get quite tricky and time-consuming.

Building dynamic sitemaps

The next example is to add a sitemap.xml.js file in pages and use Dynamic Generation (or SSR if you prefer) using getServerSideProps. Since that function is run on every request, a req and res object is passed to the function, like an API, the res object is used to specify that the file type is XML and it is used to end the request before the main function is run. This is a method mentioned in Next.JS documentation.

/pages/sitemap.xml.js:

const URL = 'https://felixrunquist.com'
function generateSiteMap(posts) {
  return \`<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
       <loc>https://felixrunquist.com</loc>
     </url>
     <url>
       <loc>https://felixrunquist.com/posts</loc>
     </url>
     \${posts
       .map(({ id }) => {
         return \`
       <url>
           <loc>\${\`\${URL}/\${id}\`}</loc>
       </url>
     \`;
       })
       .join('')}
   </urlset>
 \`;
}

export default function SiteMap() {//This function is there just to avoid compile errors
}

export async function getServerSideProps({ res }) {
  const posts = await fetchPosts();//External function to fetch posts

  const sitemap = generateSiteMap(posts);// generate the XML sitemap with the posts data

  res.setHeader('Content-Type', 'text/xml');// send the XML to the browser
  res.write(sitemap);
  res.end();

  return {
    props: {},
  };
}

However, if you’re fetching data from a CMS, such as custom posts, this means that for every request to your Next.js site, there will also be one to the CMS which greatly increases overhead. This also affects loading times, as the sitemap won’t be served until all the content is fetched and built.

The best of both worlds

For my sitemap, I didn’t like the idea of having to manually update it every time, nor did I like the added overhead with Dynamic Generating. I found a way (advocated by Next.js) to generate a sitemap.xml file in the /public folder at build time. However, their method modifies the next.config.js file to run the function as a script.

Instead, I decided to call the function in the getStaticProps function of my index.js page. This means that the sitemap will be generated at build time, but I can also add a revalidate time in order to use incremental regeneration if I wish.

This is the idea: At build-time (or on regeneration), fetch the Next.js page content as well as any CMS page content, then compile XML sitemap code. Using the fs module, this will be written to pages/sitemap.xml.

/public/robots.txt:

User-agent: *
Allow: /

Sitemap: https://felixrunquist.com/sitemap.xml

/pages/index.js:

import {generateSitemap} from 'lib/generate.js'
export function getStaticProps(){
  generateSitemap();
  return {
    props: {}
  }
}

/lib/generate.js:

import fs from 'fs'
const SITE_PATH = 'https://felixrunquist.com';

function getSitemapXML(posts) {
  return \`<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
       <loc>\${SITE_PATH}</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/posts</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/contact</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/showcases</loc>
     </url>
     \${posts
       .map(({ node }) => {
         return \`
       <url>
           <loc>\${SITE_PATH + '/posts/' + node.slug}</loc>
       </url>
     \`;
       })
       .join('')}
   </urlset>
 \`;
}

export default async function generateSitemap(){
  const posts = await getPosts()//Fetch posts via API


  const sitemap = getSitemapXML(allPosts.edges);// Generate the XML sitemap with post data
  fs.writeFileSync('public/sitemap.xml', sitemap);//Write file as a statically-served file
}

To indicate to search engines where your sitemap file is, you also need to add a robots.txt file. Since this will be changed rarely, I added this as a static asset.

What about RSS feeds?

An RSS (Really Simple Syndication) feed is a file that lists the posts and pages of your website in a computer-readable XML format, which makes a standardized format for notifying other sites and services that you’ve published new content. This is most useful if you publish blog posts or other dynamic content regularly.

To serve this file, I’m going to use the example using Build Generation, which is the best option, in my opinion.

I used the feed package which makes exporting RSS data as XML really easy.

import fs from 'fs'
import { Feed } from 'feed'; //Uses the RSS feed generator library
import {SITE_PATH, SITE_DESC} from './constants';

export default async function generateRSS() {
   const feedOptions = {
    title: 'Posts | RSS Feed',
    description: SITE_DESC,
    id: SITE_PATH,
    link: SITE_PATH,
    image: \`\${SITE_PATH}/og-image.png\`,
    favicon: \`\${SITE_PATH}/favicon/favicon-32x32.png\`,
    copyright: \`All rights reserved \${new Date().getFullYear()}\`,
    generator: 'Feed for Node.js'
   };

   const feed = new Feed(feedOptions);
   const posts = await getPosts();//Fetch posts from API
   posts.edges.forEach(({node}) => {
     feed.addItem({
       title: node.title,
       id: \`\${SITE_PATH}/posts/${node.slug}\`,
       link: \`\${SITE_PATH}/posts/${node.slug}\`,
       description: node.excerpt,
       content: node.content,
       date: new Date(node.date),
      });
   });
   fs.writeFileSync('public/feed.xml', feed.rss2());//Write as static asset

}

You can also add a link tag to your header to indicate the presence of an RSS feed to crawlers and RSS readers:

<link rel="alternate" href="feed.xml" type="application/rss+xml" title="RSS feed for posts" />

The RSS can also used to cross-publish to different platforms such as dev.to, which I will explain in a future post.

Customizing WordPress GraphQL with custom fields

Felix — Wed, 31 May 2023 15:47:08 +0000

If you’ve used WordPress for a while, and have ever wanted to add custom metadata to a post, such as a caption/subtitle for example, this can be done with something called “custom fields”. However, enabling them in WordPress doesn’t mean they’re visible to a static site using GraphQL to fetch data from WordPress.

What is a static website?

A “static” site is a site made of code that doesn’t change once it’s built. Think of it as a book: after printing, the content doesn’t change. This has many advantages over dynamic sites such as WordPress:

Speed: Since the content doesn’t change, multiple copies of the site can be stored on multiple servers, so that the user gets served from the closest server, reducing latency.
Better SEO: As the content can be served served quicker without having to use javascript, your website can be higher up in search results
Security: The site doesn’t contain databases visible to users which they potentially gain access to, and other vulnerabilities.

For a while, having a static site meant that whenever you want to change anything, you would have to manually go through each HTML file and edit each change manually. This has changed with the arrival of services such as Next.js and Gatsby, which can retrieve external data and create HTML files from them at build time.

I personally have a Next.js website with WordPress as the CMS. They interface with WPGraphQL, a plugin for WordPress creates a GraphQL API. GraphQL is an open source data query language originally developed by Facebook.

How to add custom fields in WordPress

Enabling custom fields is fairly simple. From the post editor, click on the three dots on the top right and head to Preferences > Panels.

The post editor preference pane

Once you do this, you need to reload the page to apply the changes, and the custom fields appear underneath the post content (I spent quite a while trying to find them in the sidebar, to no avail). There may already be a custom field or two, such as mine which has a classic-editor-remember field, to remember if I last used the WordPress block editor or the custom editor on my post, I believe.

Custom fields on the bottom of the page

Making custom fields visible to GraphQL

Once you’ve added your custom fields, it’s time to make them visible to the GraphQL API. In a nutshell, GraphQL serves data through a type: each type has a set number of fields (these fields aren’t the same as WordPress custom fields!) that can be populated so that you know what to expect in the data. For example, a post type could have the following fields:

Title
Content
Description
Author
Published date
Last modified date

In order to make the custom fields appear, the post type has to be changed to reflect the new field.

To do this, there’s a bit of PHP involved to add this as a field type in GraphQL.

I found that the best way to do this was via the WordPress admin panel, by editing the theme PHP files with the Theme File Editor. To do this, go to Themes>Theme File Editor. I had the Hello Elementor theme preinstalled with my version of WordPress, which is of no importance since I only use the WordPress website as a CMS, the website part visible to users is implemented with Next.js.

The theme file editor

Next, go to the relevant functions.php file (in my theme it’s called elementor-functions.php). This is where we’re going to add the code relevant to adding a GraphQL field. I wanted to modify the post type to add a caption field.

This is the code I added to do so:


/**
 * 
 * Custom fields for WP Graph QL
 * 
 **/
add_action( ‘graphql_register_types’, function() {
  register_graphql_field( ‘Post’, ‘caption’, [
     ‘type’ => ‘String’,
     ‘description’ => __( ‘The caption of the post’, ‘wp-graphql’ ),
     ‘resolve’ => function( $post ) {
       $caption = get_post_meta( $post->ID, ‘caption’, true );
       return ! empty( $caption ) ? $caption : ”;
     }
  ] );
} );

Be careful modifying WordPress PHP files! If you do something wrong, it could muck up the whole site and prevent even the admin panel from showing, even something as small as a syntax error, such as a forgotten semicolon. Make sure you have a backup of your WordPress site before making any changes!

The function runs when GraphQL is compiling its types, and we’re registering a new caption field to the post type. To fetch the data, we’re querying the caption field on a WordPress post with a known ID, using the built-in function get-post-meta. Finally, if there is no caption on a given post, we return an empty string.

In order to test that the new field works on the post type, WPGraphQL comes with a built-in IDE to test queries. In the WordPress admin panel, head to GraphQL > GraphiQL IDE, and input the following query in the left pane:

query AllPosts {
  posts(first: 20, where: {orderby: {field: DATE, order: DESC}}) {
    edges {
      node {
        title
        excerpt
        date
        caption
      }
    }
  }
}

The above code fetches the last 20 post ordered by date, and it fetches the title, excerpt, date and caption custom field. If everything works, it should retrieve the data and display any captions you’ve added to posts on the left pane when you click the “Play” button. If there is an error mentioning an unknown field type, it means the PHP code you entered for the custom field is not being recognized. This might mean that the functions.php file is the wrong file. I had two -functions.php files in my theme files, and the first file I added the code into resulted in an error, so I added it to the second file.

Sources:

PHP Code – https://master–wpgraphql-docs.netlify.app/getting-started/custom-fields-and-meta/

Creating and ordering a custom PCB

Felix — Mon, 29 May 2023 16:39:46 +0000

I’ve always found something thrilling about circuitboards. I remember the excitement of finding a discarded motherboard on the side of the road as a child, imagining the tiny little electrons flowing in their ordered, angled city.

As part of a home automation project, I wanted to house a thermometer, motion sensor and display in a small form factor. A problem I came across was fitting all the modules, board and cables in such a small box!

The box I made out of MDF, along with an ESP32 DevKit V1 for comparison

The box I made using boxes.py and an online laser cutting service was only 6x6x2cm on the inside, barely enough to fit in my sensors and screen, let alone free-form dupont cables. It was absolute mayhem.

This image came from an Arduino forum titled Breadboard problem – am I going mad? We’ve all been there…

I decided it was time to try my hand at creating a custom PCB. I’ve seen people online create their own through online PCB services, and researched the best way to design one.

Software

There is EDA (Electronics Design Automation) software such as Autodesk EAGLE, but the free version being quite limited and the paid version requiring a Fusion 360 subscription, which is upwards of 560 € / year or $630 USD (I know!), that option was quickly thrown out.

I then came across KiCad, which has checked all the boxes for me. Honestly, it’s a godsend. It’s an open source PCB design suite that has a wide range of tools from designing circuits to drawing the actual PCB to viewing the model in 3D, to even converting images to PCB-ready footprints! If it even is possible to find something where it lacks, it definitely makes up for it with the vast amount of add-on packages available to install.

The wide range of tools offered by KiCad

As I haven’t ever done any digital electronic design, there definitely was a learning curve, but it wasn’t very steep.

The first thing I did was to create a new project in KiCad, which initializes files for circuitry and PCB layout. This step is important if you want to create schematics and then design them onto a PCB, which you can’t do if you create a separate schematic or PCB file. I then opened up schematic editor. For a newbie, the amount buttons and information may be a little overwhelming at first but it quickly starts to feel comfortable.

After a bit of googling, I found the “insert component” button on the top of the right sidebar, which seemed like a good place to start. The software then prompted me to select the component I wanted among its huge library (I never once had to fetch a component from an external library). For example, you can insert a 1×4 connector by typing “01×04” in the search bar.

After adding all the components I needed for my project (mostly connectors to join all my modules up), it was time for me to specify the wiring between the components. There are two ways of doing this: The first way is to select the “add wire” icon (“W”) on the right, and join two or more pins, but this is tedious and the shapes you define won’t make it to the final PCB as this is just a schematic. The other way is to label pins, and KiCad automatically joins up pins with the same label. There is a scary amount of label types that can be added, but I just selected the first option and hoped for the best, which worked in my favor.

4 (four!) different labels one can add!

Something I had to look out for was the fact that the labels have to properly join up with the pins for them to be recognized.

An improperly connected pin vs a properly connected one – the square and the circle vanish

I wanted to build a circuit to fit an ESP32 D1 mini, an I2C RTC/Temp sensor, an I2C oled screen, PIR and IR receiver/emitters so I could use a remote with my device. I added components with the right number of pins while making sure that the pin order was correct.

Once I had my circuit done, there were a few things left to do: KiCad gets upset if you don’t define power symbols, so I selected “Add Power Symbol” (“P”) and added the power and ground pins for my circuit (3.3V and GND in my case). Once you do that, you get a symbol to add to a pin just like any label. This defines that pin, and all the ones with the same label, as belonging to that power symbol.

My finished schematic

Another thing to do is to define footprints for all the components – this gives the software an idea of how the component looks like in the real world. To do this, click on the “Run footprint assignment tool” button on the top bar, and it opens a list of the components in the schematic in the middle and a list of available footprints to assign on the left. Unassigned components are highlighted in yellow, and then it’s a matter of selecting the right footprints. For connectors, you can choose a “connector” or a “header” configuration. The default header/connector size is 2.54mm (This works with dupont cables and Arduino/Espressif boards). For example, if I wanted a 1×4 pin connector to map out to a 1×4 header, I’d select the Connector_PinHeader_2.54mm family on the left, and search “1×04” on the top bar (Note: don’t type “01×04” as connector footprints don’t have a leading “0” like components do). Once you find the right configuration, double click on it.

Searching for 1×4 headers

Building the PCB

After assigning all the footprints on my circuit, I went to the “Tools” tab bar and selected “Update PCB from Schematic”. This sends the circuit to the PCB builder, ready to start building a physical circuit. At first the components are haphazardly placed out onto the screen, and the circuits jumbled about (don’t worry – the PCB isn’t going to come out like that and I will get to the tracing later) but you can move them about to place them. This is where the measuring tools and grid comes in handy: I went to Settings>Display Options>Full window crosshair so that I could make sure components were aligned when I moved them. I would select each component, align it, and measure by pressing the space bar to reset the “dx” and “dy” properties on the bottom of the screen: these let you see how much your cursor moves to properly space out components. Another tool is the Dimension tool: after selecting the User.Comments layer on the right, you can click between two points to show the distance between them, this is useful for later reference.

Something that seems trivial but that is important to make sure your PCB comes out the way you envisioned it is drawing diagrams. This helps make sure you get your spacing, number of components and orientation right.

My messy but functional diagram

Once everything is positioned to taste, it’s time to create a shape for the PCB. This is known as edge tracing. I clicked on the “Edge.Cuts” layer (in the layer list on the right) and then selected the line tool on the right toolbar to create a square around my PCB. There are other tools such as the arc tool to create rounded edges.

A note on pads:

There are two main types of pads (the plated surfaces on which components are mounted) in KiCad:
Through-hole which is a hole that penetrates the entire thickness of the board and SMD (Surface Mount Device) which is just a plated surface that doesn’t come with a drilled hole.

It’s time for the last building stage: creating traces, or wires, between components: The easiest way to do this, especially for a simple PCB with few components, is to use an auto tracer. This algorithmically determines the traces to join up pins the same way the schematic is laid out. I downloaded a plugin called “Freerouting” that does precisely this. You may need to install/update Java to make this work like I did. In the PCB editor, select Tools>External Plugins>Freerouting from the menu bar. This launches the tool and automatically traces on the front and back PCB layer.

My traced PCB – it’s finally coming to shape!

There were a few traces I wasn’t happy with, so I modified them by selecting the layer on which the trace is on (either F.Cu – front or B.Cu – back), deleting the traces I want to change, and selecting the “Route Tracks” tool (“X”). This handy little tool avoids traces which aren’t part of the circuit and does things like 45º angles (This is important for high-frequency signals which bounce back off the corner of 90º angles). There are a few things to consider when tracing: The front and back are obviously electrically isolated so front and back traces can intersect but not traces on the same side. Pins traverse the entire board so this is a useful place to join traces from different sides to the same circuit, and if you want to do this without a pin you can make a hole, called a “via” which does this (This adds holes to drill so avoid if you want to keep costs down).

One can also add labels to components that will be printed out onto the PCB, this is known as a “Silkscreen”. Select the silkscreen layer (front or back) in the layers list and you can place text or shapes onto it. Make sure that the silkscreen doesn’t overlap with any pads as they will get clipped. The silkscreen is printed on top of the traces so having them overlap isn’t an issue.

Another thing I found tricky was to switch the side of components: You can mount components on either side, and although for through-holes it doesn’t change much (As the pads which go through the thickness of the board are symmetric on both sides), for surface-mounted components it makes a huge difference: By double clicking a component to make the footprint properties dialog come up, you can select the side it’s mounted on in the “position” section.

A component positioned on the front side

The next step is to make sure there aren’t any issues with the design: PCBs must follow a DRC (Design Rule Check) to make sure board edges are properly defined, different traces/components don’t intersect… This is done by clicking on the “DRC window” icon on the top bar, and selecting “Run DRC”. If it finds any issue, it will put arrows in the PCB diagram and selecting the error in the window highlights the component and lists the error. I had a few issues with overlapping labels and unconnected pads, but they were quickly resolved.

My DRC issues

After everything is in order, select View>3D viewer (Option-3) to get a rendered view of the PCB. I find that this helps with mentally representing all the components in space, and making sure everything is how you envisioned it. Seeing the front and back traces separately also helps to make sure the pads are wired optimally.

My 3D model

Ordering

It’s time to order the PCB! To get the file ready for export, choose File>Plot…. There are 4 things to export: Traces (front and back), Masks (font and back), Silkscreen (Front and back), and the drill holes. To export drill holes, go to “Generate Drill Files…”. I found it best to select the “PTH and NPTH in single file” so that you get one simple file for all the drill holes. Select “Generate drill File” and “Plot” on the dialogs, and it should export the gerber files in the same folder as the KiCad project.

I researched a few online PCB ordering services: PCBWay, JLCPCB and AllPCB are based in China, while OSHPark is USA-based. After looking at reviews and comparing prices, I found that the best way to go was PCB way, especially for a hobby project. To upload the KiCad model, I zipped a folder containing all the gerber files and uploaded it on the website. It automatically populated the fields for me, and it was just a matter of selecting the PCB color I wanted (I went with purple) as well as the delivery service. As the minimum order for most PCB manufacturing services is 5 boards, I got the idea of modifying my board so that it could fit an ESP32 DevKit as well, and so that I could use the I2C and various pin support as well for other projects, unfortunately I got that idea after I had ordered my PCB and it was too late to change.

The ordering process was quick, and I could see on my order history the stage at which my board was, from drilling to plating to cutting.

The board came within 10 days from ordering, I was very impressed with the speed! I got a pack of 5 boards in purple, and I’m quite pleased with the quality of the board, from the thickness to the screen printing resolution. I was warned by KiCad that the font used for my name wasn’t of uniform thickness, but JLCPCB handled it very well.

My finished creation!

The verdict

Overall, I’m quite happy with the design process with KiCad, as well as the quality of service offered by JLCPCB. Although I do feel bad about not supporting local businesses and ordering PCBs from across the globe, the price, especially for a hobby project, made it a no-brainer for the quality of the finished product. Receiving your own creation in the mail is such a rewarding feeling!

If I was to do anything differently next time, I would definitely add drill holes which would have made mounting it easier, I would also take the time to think about what other project the board could be used for given the minimum order amount of 5.

KiCad, FTW!

DEV Community: Felix

Locate Website Visitors in Next.js with IP and Supabase

Fetching the country from IP

Closing notes

References

Making the Internet More Human

Accessibility

Best practices

Search engines

The need for human input

Ads

A note on cookies

Further reading

Designing spring animations for the web

Reasons for using animations

How does a physical interaction work?

Spring animations

CSS implementation

Javascript implementation

Design considerations

Privilege GPU-based animations

Make sure spring animations are necessary

Put native interactions first

Complement, not distract

Animation frequency

Further reading

Conclusion

References

Creating 3D models in Spline for Three.js

What is Spline?

Creating a 3D model in Spline

Exporting

Adding search to a React/Next.js blog

Web design inspiration: Ten perfect examples

1. Carl Hauser’s blog

2. Sketch Blog

3. JP Silva’s portfolio

4. Vita Architecture

5. Outreach.space

6. Tinkersynth by Josh W. Comeau

7. Atmos

8. Rauno Freiburg’s portfolio

9. Cobbler

10. Cosmos.so

11. A bonus one: Bento layouts

Website update: code blocks, search, lightbox and more!

Revamped post experience

Code blocks

Full-bleed images and lightbox

Post headers

Post categories and search

Header and footer

Design

A never-ending journey

Automatically cross-publishing posts to dev.to with RSS

Canonical links

Auto-publishing from your RSS feed

Publishing posts

Generating a sitemap and RSS feed with Next.js

The difference between Statically and Dynamically rendered content

What is a sitemap?

Static sitemaps

Building dynamic sitemaps

The best of both worlds

What about RSS feeds?

Customizing WordPress GraphQL with custom fields

What is a static website?

How to add custom fields in WordPress

Making custom fields visible to GraphQL

Creating and ordering a custom PCB

Software

Building the PCB

Ordering

The verdict