DEV Community: Vincent

How to fix Spotlight using hundreds of GBs on your mac

Vincent — Wed, 14 May 2025 08:26:15 +0000

tl;dr; To solve Spotlight using hundreds of Gbs, boot in recovery mode and delete its folder: rm -rf /System/Volumes/Data/.Spotlight-V100.

Then configure Spotlight to ignore folders containing millions of non-search-relevant files.

A few days ago my computer started to act VERY weirdly. I have an Apple M3 Pro running macOS Sequoia. Everything started to slow down and I could not figure out why.

Looking at the Activity Monitor I saw of course mentions of mds and mds_stores. I immediately blamed Spotlight which has been a big CPU consumer since it was ever created.

But the issue was not the CPU usage, my computer was actually having a hard time writing to disk.

Looking at tools like DaisyDisk I discovered that:

My disk was almost full (1TB!)
Most of the used space (500GB) was in the "hidden space"

DaisyDisk help page says "Large amounts of hidden disk space may also indicate to file system errors. Follow our troubleshooting guide for details."

And it turns out, my Storage settings in macOS was also saying I had huge amounts of "System data" being used:

After multiple hours of searching around how to solve this, including commands like sudo mdutil -E, sudo mdutil -X, and disk repair attempts ... I finally found the exact issue.

Before trying out any other solution, if you have huge amounts of system data showing up as used in your storage system settings then follow these steps:

Boot in recovery mode
Open the terminal
Remove the Spotlight folder rm -rf /System/Volumes/Data/.Spotlight-V100
Boot in normal mode
See that you recovered your space!
Now make sure to ignore any folder containing a lot of non-relevant files. For example avoid Dev/ style folders with node_modules in them, or any temp/ logs/ folders that have millions of files in them
Activate Spotlight indexing: sudo mdutil -E / (it should already be on but whatever)

Bonus tip: how to show folder sizes in finder

Before I understood this was a Spotlight issue (always bet on Spotlight for cpu or space issues first), I was trying to find what are the biggest folders on my computer.

To solve this, there's a very nice way to always display the total sizes of folders on screen in finder.

In any finder window, use ⌘ + J then check "Calculate all sizes" and click on "Use as Defaults". That's it, now you can easily navigate and spot the biggest space consumers.

If this has been useful, drop me a comment. See ya!

How to add Firebase service account json files to Vercel

Vincent — Tue, 24 Aug 2021 12:19:47 +0000

If you're using Firebase, Next.js, Vercel, firebase-admin, and if you've been Googling for the past hour on how to load your Google service account JSON file the easiest way possible, then you've come to the right place.

This blog post explains how to store the service account json file in a single environment variable and then pass it down to the Firebase admin SDK.

Here's how to do it:

1. Download the service account JSON file

Follow the steps at https://firebase.google.com/docs/admin/setup and download the JSON file.

2. Remove line breaks on the JSON file

Go to https://www.textfixer.com/tools/remove-line-breaks.php and paste the content of the JSON file.

Do not remove the \n characters from the private_key fields or the key won't be valid afterwards.

3. Create a new Vercel environment variable with the JSON content

Either via a command line:

vercel env add FIREBASE_SERVICE_ACCOUNT_KEY
? What’s the value of FIREBASE_SERVICE_ACCOUNT_KEY?

(paste the JSON content and submit)

Or via the Vercel web ui: https://vercel.com/docs/environment-variables

In both cases, the value will be encrypted.

4. Use it locally in .env.local

This is the tricky part, because you have to use single quotes instead of double quotes when surrounding the environment variable value in your file.

I am using the gitignored .env.local as explained in the Next.js documentation: https://nextjs.org/docs/basic-features/environment-variables#loading-environment-variables.

# .env.local
FIREBASE_SERVICE_ACCOUNT_KEY='JSON file content without line breaks'

5. Use the environment variable

Here's an example Next.js api route that will receive a Firebase id token and verify it. See https://firebase.google.com/docs/auth/admin/verify-id-tokens#web

// pages/api/user.ts

import * as admin from "firebase-admin";
import type { NextApiRequest, NextApiResponse } from "next";

type Data = {
  uid: string;
};

const serviceAccount = JSON.parse(
  process.env.FIREBASE_SERVICE_ACCOUNT_KEY as string
);

if (admin.apps.length === 0) {
  admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    databaseURL: process.env.NEXT_PUBLIC_FIREBASE_DATABASE_URL,
  });
}

// example usage 
export default async function revenueCatApi(
  req: NextApiRequest,
  res: NextApiResponse<Data>
): Promise<void> {
  const firebaseToken = req.headers?.authorization?.split(" ")[1];

  if (!firebaseToken) {
    res.status(404).end();
    return;
  }

  const decodedToken = await admin.auth().verifyIdToken(firebaseToken);

  res.status(200).json({ uid: decodedToken.uid });
}

That's it! Hope this helps.

I made an app: Discover how people react to you on GitHub

Vincent — Mon, 08 Mar 2021 23:13:45 +0000

Hey there dev community, I always wanted to build this: an app that lists and summarizes all the reactions I received on my GitHub comments.

It took me some time (one week) but I made it! You can access it here: https://sourcekarma.vercel.app.

You can share your score on Twitter and also get a nice badge for your GitHub readme:

vvo / vvo

Hi there 👋

Maker

TurnShift: Automate Your Team Schedule
Tweets @vvoyer: JavaScript, React, Next.js, Building In Public
Next.js News: The Next.js Monthly Newsletter (sold)
Source Karma: Discover how people react to you on GitHub 👍 (archived)

View on GitHub

The technical and design part were super fun, I am using:

Vercel
Next.js
Puppeteer (to generate social images and GitHub badges)
AWS RDS (PostgreSQL)
Craftwork (for the awesome illustrations)
Fathom (for stats)

Give it a try and please ask any questions you have here!

Best,
Vincent

PS: Here's the launch tweet

// Detect dark theme var iframe = document.getElementById('tweet-1368953460947828746-517'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1368953460947828746&theme=dark" }

How to verify signatures from Slack incoming requests with Next.js

Vincent — Thu, 08 Oct 2020 13:54:00 +0000

Hi there, this is a short blog post for anyone looking to handle Slack incoming requests with Next.js.

When creating a Slack application that will interact with your web application, Slack will call you back on some actions like:

events API
any interactivity action like a button click in a message

Here's an example Slack postMessage call:

import { WebClient } from "@slack/web-api";
const web = new WebClient(token);
web.postMessage({
  channel: "#general",
  // fallback for mobile and email notifications:
  text: "Please visit <https://google.com>",
  blocks: [{
    type: "section",
    text: {
      type: "mrkdwn",
      text: "Please visit https://google.com"
    }
  }, {
    type: "actions",
    elements: [{
      type: "button",
      text: {
        type: "plain_text",
        text: "google",
      },
      url: `https://google.com`,
      action_id: "go_to_google_button",
    }],
  }]
});

If you do this, you'll get this result:

And if you click on the "google" button, it will open Google in your browser. Pretty cool way to link from Slack to your web app for example.

But, that's not the end of the story. If you only do this then Slack will complain because a button, even if it only open a new url, is still an interactive element from Slack point of view.

So Slack will try to call the URL defined in your application "Interactivity & Shortcuts" Slack settings.

And it will fail because you've not defined a route for that.

Now to solve that, if you're using Next.js, you have to define a route as follow:

// pages/api/slackInteractiveEndpoint.js

import { createMessageAdapter } from "@slack/interactive-messages";

// you can find this secret in your app "Basic information" tab
const slackInteractions = createMessageAdapter(slackSigningSecret); 

slackInteractions.action(
  { type: "button", actionId: "go_to_google_button" },
  // no need to reply, it's just a button to a url
  () => {},
);

export default slackInteractions.requestListener();

export const config = {
  api: {
    bodyParser: false,
    externalResolver: true,
  },
};

Then add this route in your Slack application "Interactivity & Shortcuts" tab. Test one message, click on the button and .. voilà!

Because the Slack SDK is responsible for parsing the body and replying to the request, we tell Next.js to let it go via:

bodyParser: false
externalResolver: true

Hope this helped you! Post a comment otherwise :)

Self-hosted analytics with umami on Vercel

Vincent — Thu, 20 Aug 2020 10:44:36 +0000

EDIT Sep 4, 2020 After having stability issues with self-hosted Umami. I decided to use Fathom Analytics and have been happy about it since then. I especially like the automated weekly email report on all my websites. Get $10 off by using my link: https://usefathom.com/ref/Y8XVBV.

Umami is a self-hosted open-source alternative to Google Analytics. It provides simple and fast website analytics for your projects. It is built with Next.js and can run on MySQL or PostgreSQL. I believe it's a recent project and you can read more on the associated Hacker News discussion: https://news.ycombinator.com/item?id=24198329

Here's how it looks:

You can even share public pages of your analytics!

I just started releasing products like Next.js News and wanted to have some information on the people visiting my website.

So far the products I knew about were:

Fathom: paid or self-hosted, in go
Plausible Analytics: paid or self-hosted, in Elixir
Simple Analytics: paid
Google Analytics: you need a PHD to use it
Ackee: self-hosted
66Analytics: paid

I am ready to pay for services, but the idea of having a self-hosted analytics service that was good enough was interesting. So let's see how to install umami on Vercel.

1. Create a database

For Umami to work, you need a database. I went for the latest version of PostgreSQL on an AWS t2.micro which fits their free tier. But use whatever you want.

2. Create the necessary database tables

Use the provided SQL dumps:

schema.postgresql.sql
or schema.mysql.sql

You can find them here: https://github.com/mikecao/umami/tree/master/sql. It will also create the first user.

I use https://tableplus.com/ to connect to my local or production databases and execute queries on them.

3. Create a new project on Vercel

Directly import the project from https://github.com/mikecao/umami, Vercel will fork it on your GitHub account.

4. Configure the Vercel project

Change the build command for: npm run build-postgresql-client && npm run build if you're using PostgreSQL. Or npm run build-mysql-client && npm run build if you're using MySQL.

Add two environment variables:

HASH_SALT, used to encrypt passwords in umami database. It should contain a random string. You can generate good ones with 1password's strong password generator
DATABASE_URL, It's the full url to access your database. It should look like this: postgresql://dbuser:dbPassword@dbHost[:dbPort]/dbName

This is all you need, for details you can have a look at umami's install documentation.

Now you can click on deploy!

5. Login to the umami UI

Go to your newly created Vercel application and login with:

user: admin
password: umami

Change your password immediately.

6. Add websites

Now you can add some websites, get the tracking code, add it to your code annnnd DONE. You now have a fully functional self-hosted analytics that looks good and is sufficient for a start.

PS: Since what is deployed is a fork, you will have to update your fork to benefit from umami new features. See GitHub documentation on how to sync a fork.

Thanks for reading!

If you enjoyed this post, follow me on twitter at @vvoyer and share the article for your followers.

Coding the Jamstack missing parts: databases, crons & background jobs

Vincent — Wed, 19 Aug 2020 13:24:30 +0000

Summary:

For my current product, I use Next.js deployed on Vercel (If you're using Netlify, this article is also for you). And I use AWS for the other services I need: databases, crons, and background jobs.

The AWS parts are deployed with AWS Cloud Development Kit (CDK). An Infrastructure as Code tool allowing you (for example) to create PostgreSQL databases on AWS, using JavaScript files.

I talk about AWS in this article, and it might be disturbing for readers: Because of its apparent complexity, you may not think of AWS as a solution for your Jamstack services needs. I thought the same. Then I discovered how easy it was to deploy AWS services using CDK and jumped right in.

From reading issues and forums, I think we share a common struggle: how do you couple a Jamstack application with the essential infrastructure you need (databases, crons & background jobs) to create great products. This article is here to answer that.

Bonus: At the end of this article I provide you a takeaway example you can deploy on AWS and $5,000 credits.

Discovering the Jamstack

jamstack.wtf

As soon as I started to use platforms like Vercel and Netlify, I thought: what about databases, crons, background jobs...? Jamstack platforms are providers allowing you to easily deploy static websites and API calls (through "lambda functions"), but that's it. If you need more than that, you need to use other services.

Coming from Heroku, I was disappointed because it meant I had to spend time researching other solutions, comparing them, and deciding if I should use it or not.

And on top of that, it would possibly mean having lots of different moving parts.

Why do we even need crons and background jobs?

At some point you need a database, that's obvious. As for crons and background jobs:

Crons

Let's say you're building a product that monitors Twitter for specific words. For this to work, static generators and API routes won't be enough: you need something that can frequently call your API that in turn will check Twitter for such words. This is a cron job. You cannot rely on user actions to achieve such work.

Sure you could use EasyCron. But this would be another service to configure and pay for.

Background jobs

Now in your UI, you also have a button that will generate a PDF report for specific words and send it to an email address. This is potentially a "long" running job, something like 10 seconds. The right way to handle this is to use background jobs that will make sure this job is processed without your user having to wait in your UI for it to be done.

One could say that this could be handled by just having an API route that replies immediately and continue to execute some code. Unfortunately, 1. This won't work on AWS (Vercel and Netlify are using AWS and they document this limitation) and 2. If the job fails, you'd have to record it and inform the user they need to click again.

Jamstack platforms like Vercel and Netlify won't help you (for now) whenever you need some code to be executed on behalf of users.

Once I knew that and what I needed, I went solutions-hunting.

Requirements

The solution I am looking for, should, for the most parts:

Be a single solution, I don't want to maintain a job queue hosted on x.com and cron jobs hosted on y.com. This would be a mess. This is the case on Heroku where they have in-house solutions (databases) but also third-party solution providers (queues) with different UIs.
Be cheap, I am cheap
Be on AWS, because I got credits. Plus, most of Vercel's infrastructure is hosted on AWS (Same for Netlify). So I thought the network between my Vercel API routes and my database would be good.
Be easy to administrate, modify, and deploy. Ideally with code, not YAML files. And without requiring me to create different projects.

Google to the rescue

Solution

After multiple searches, I settled on using AWS this way:

database: RDS with latest PostgreSQL engine.
cron jobs: EventBridge coupled with lambda
background jobs: SNS coupled with lambda. One could also use SQS, which is a real queue system, or a more complex pattern. But I yet have to become more AWS experienced before diving into SQS and its configuration.

For anyone experienced with AWS, this may be the usual stack. It took me days to find and understand those solutions.

Deploying the infrastructure

Great! Now, how do I easily deploy and maintain all of that? As a new user, I found the AWS console pretty good. You can create your infrastructure in just a few clicks and even code your lambda functions inside your browser: amazing.

But, I said I needed the solution to "Be easy to administrate, modify and deploy. Ideally with code.".

After more research, I understood that what I was looking for was Infrastructure as Code tools (IaC) to control AWS services. IaC tools can range from python files to configure servers, to gigantic YAML files to deploy AWS services.

The right tool for the job: AWS Cloud Development Kit

I found the AWS Cloud Development Kit (CDK) randomly browsing IaC tools. It is mentioned on the Pulumi vs. Other Solutions page which is another IaC tool.

Here's an example CDK stack creating a lambda that will get called every two minutes:

import * as cdk from "@aws-cdk/core";
import * as events from "@aws-cdk/aws-events";
import * as targets from "@aws-cdk/aws-events-targets";

import { NodejsFunction } from "aws-lambda-nodejs-webpack";

export default class ExampleStack extends cdk.Stack {
  constructor(scope, id, props) {
    super(scope, id, props);

    // we define the lambda, it's a local file that exports a `handler` function
    const checkTwitterLambda = new NodejsFunction(this, "checkTwitterLambda", {
      entry: "jobs/checkTwitter.js",
      timeout: cdk.Duration.seconds(30),
    });

    // run jobs/checkTwitter.js every 2 minutes
    const rule = new events.Rule(this, "ScheduleRule", {
      schedule: events.Schedule.cron({ minute: "*/2" }),
    });
    rule.addTarget(new targets.LambdaFunction(checkTwitterLambda));
  }
}

Reading this code, as a JavaScript developer, I immediately thought it was the right tool for me. After being able to use JavaScript on the backend (Node.js), on the frontend (Next.js), now I can even define my infrastructure needs in JavaScript. CDK is also available for TypeScript, Python, Java and C#/.Net (And they do this through code generation with https://github.com/aws/jsii. Mind, blown.).

Note: It's amazing how good CDK is as a tool, but how bad they are, for now, SEO wise. It's extremly hard to stumble on CDK if you're searching for infrastructure tools.

If you're interested in: how CDK works, the only issue I had with it or the other IaC tools I tried. You can read the folded sections.

Otherwise, feel free to jump into the next part about the full application example!

The only issue I had with CDK
I hit a single issue with CDK: They run Parcel inside Docker, by previously mounting your whole JavaScript project as a Docker volume. Unfortunately, on MacOS, mounting big volumes with DOcker is notoriously slow. And even with the latest updates to Docker volumes (Mutagen-based file synchronization), it will still be slow. Mounting a small Next.js project would take between 20s and 2 minutes.

I spent a LOT of time on this issue, and I was lucky to be able to discuss at length with CDK maintainers like Jonathan Goldwasser. At some point they will provide a Node.js construct that will not use Docker (this is almost released, see: https://github.com/aws/aws-cdk/pull/9632).

In the meantime, I created my own construct with webpack which works pretty well: https://github.com/vvo/aws-lambda-nodejs-webpack.

Other IaC tools I tried: Architect and Pulumi

Architect

https://arc.codes/

Architect looked like what I needed: control AWS services via YAML files while also getting a local environment replicating your AWS infrastructure.

I played with it a little, asked some questions on their slack, only to discover that once you were using Architect, your whole codebase should be controlled and deployed by it. You can't easily mix a Next.js codebase deployed on Vercel and cron jobs hosted on AWS crontrolled by Architect.

Next!

Pulumi

The first real Infrastructure as Code (Code as in JavaScript code, not YAML files) tool I tried was Pulumi. Pulumi is multi-cloud. I was happy to find out you could easily create lambda functions on AWS that would get regularly called, just like cron jobs. While also using SQS, the queue from AWS and link it to lambdas as well.

I had dumb issues with Pulumi: not being able to easily use import statements if you're not using TypeScript, all dependencies from my package.json would be installed on AWS, instead of only the dependencies my jobs are using.

Pulumi was the first IaC tool I really tried and most probably suffered from my lack of investment in it. I am sure it's a wonderful tool.

But I needed to try another one. Next!

A full application example

Here's the first takeaway. I spent some time creating an example CDK stack you can reuse: https://github.com/vvo/nextjs-vercel-aws-cdk-example.

If you need a database, crons, or background jobs for your Jamstack application and you do not know where to start: start here and use this example. It will save you time and money. I guarantee you that in just an afternoon work you'll get it working.

The example provides:

An AWS stack ready to be deployed with CDK, including a PostgreSQL database, sample crons and background jobs as lambdas
A Next.js application you can deploy on Vercel, with a button to trigger the sample background job on AWS
Local equivalents to the AWS services for you developer environment

You can pick whatever interest you in this example, so that if you're using a different framework (Nuxt), a different Jamstack platform (Netlify), you can still totally benefit from the CDK example stack.

The example also contains more information on how to run it and links to the right documentation you may need.

AWS Cost

The AWS calculator

I am still new to AWS and there are free tiers, and credits, but I still tried to estimate the cost of the application example using the AWS pricing calculator.

Hypothesis:

we use a t3.micro, 5GB SSD on-demand instance for the database
we have 10 cron jobs that runs every 5 minutes, for 5 seconds and uses 300MB of memory each (that's ~90,000 lambda requests)
we have one background job that runs 10,000 times a month, for 5 seconds, uses 300MB of memory (that's ~10,000 lambda requests)

The AWS cost would be $16.85/month without the free tier. You can find the computation here: https://calculator.aws/#/estimate?id=8c37f67bc09a5142d821c25ae3478959ea8ce6f3

And, if we switch the t3.micro for a t2.micro and use the free tier, then the monthly AWS cost is $0. I've used the t2.micro and frankly it is performant-enough for a product launch.

Next steps

This setup is good enough but far from perfect for sure. I expect AWS experts to roast me when they see. I see multiple next steps and will probably write more blog post as I make progress in my AWS usage. Here's a list of things that could be better or just things I don't know yet how to solve:

Secrets handling, as soon as you use Vercel and AWS then you will have "secrets" like database credentials that will be both in AWS secrets manager and Vercel environment variables. Ideally they should be in a single place (AWS secrets manager) and then read from Vercel. I am not sure yet what's the right path to follow.

Dual deployment, manual deployments, right now whenever I push my code to GitHub, Vercel deploys my application to their servers, fine! But as for AWS I need to manually run yarn cdk deploy to have my lambdas and infrastructure up to date. Ideally everything (infrastructure, lambdas, migrations, frontend) should be deployed automatically when I push to GitHub.

Pull requests handling, if I open a pull request on GitHub, the whole infrastructure, database, frontend, should be deployed in an isolated way. This is not done yet too.

Error handling, retries, right now if a cron fails or a background job fails, it will be retried, up to a certain point though (lambda retries). I know this could bite me in the future, so we'll see.

PS: AWS credits 💰💰💰

I promised you you could get up to $5,000 in AWS credits, here's how:

👉 Secret has over 150+ offers for credits on Algolia, Twilio, Airtable, Notion... You can save thousands of dollars. Immediately. Use my promo link: https://www.joinsecret.com/?via=vincentvoyer to get a 20% discount on the platinum or any other Secret plan.

You can also get $1,000 AWS credits with AWS founders activate, directly on aws.com/activate. And if you've already done that you can still get $4,000 from Secret!

You don't need to be inside a startup incubator to benefit from those offers.

Thanks for reading!

If you enjoyed this post, follow me on Twitter @vvoyer, share the article, and discuss on Hacker News.

Enable network throttling on iPhone in 5 steps

Vincent — Thu, 11 Jun 2020 12:41:22 +0000

It's useful to be able to test your fancy new single page application or progressive web app in different network conditions. On desktop you can do so by using network throttling on Chrome dev tools for example.

But on Safari on your iPhone, things are messier. The top result on Google for "ios safari network throttling" gives a Stack Overflow answer that goes like this:

😤 Installing a proxy!

Well turns out there's another way. 😌

1. Install XCode on your mac

Open the Mac App Store on your mac, search for Xcode and install it.

2. Connect your iPhone

Connect your iPhone to your mac via a USB cable, say yes and confirm any request from your mac or iPhone.

3. Launch XCode

Just launch it.

4. Access developer settings on iPhone

Now that XCode was launched, you should be able to access a new "developer" settings on your iPhone.

I am not even sure you still need XCode running now, you can try to close it.

5. Use the Network Link Conditioner

Inside developer settings, there's a Network Link Conditioner submenu that you can use to throttle your connection on iPhone.

Here's a full demo:

Enjoy!

How to solve "window is not defined" errors in React and Next.js

Vincent — Wed, 10 Jun 2020 11:16:03 +0000

Next.js is a React framework with pre-rendering abilities. This means that for every page, Next.js will try to generate the HTML of the page for better SEO and performance.

This is why, if you're trying to do this:

// components/Scroll.js
window.addEventListener("scroll", function() {
  console.log("scroll!")
});

Then it will fail with "ReferenceError: window is not defined":

Because in the Node.js world, window is not defined, window is only available in browsers.

There are three ways to solve that:

1. First solution: typeof

While you can't use:

if (window !== undefined) {
  // browser code
}

Because this would try to compare a non-existent variable (window) to undefined, resulting in the mighty "ReferenceError: window is not defined". You can still use:

if (typeof window !== "undefined") {
  // browser code
}

Because typeof won't try to evaluate "window", it will only try to get its type, in our case in Node.js: "undefined".

PS: Thanks to
Rogier Nitschelm for reminding me about this. I initially tried to do if (typeof window !== undefined) and this failed hard because of the reasons mentioned earlier.

The other solutions below are more exotic but still worth it.

2. Second solution: the useEffect hook

The "React" way to solve this issue would be to use the useEffect React hook. Which only runs at the rendering phase, so it won't run on the server.

Let's update our scroll.js component:

// components/Scroll.js

import React, { useEffect } from "react";

export default function Scroll() {
  useEffect(function mount() {
    function onScroll() {
      console.log("scroll!");
    }

    window.addEventListener("scroll", onScroll);

    return function unMount() {
      window.removeEventListener("scroll", onScroll);
    };
  });

  return null;
}

What we've done here is to turn our initial JavaScript file into a true React component that then needs to be added to your React tree via:

// pages/index.js

import Scroll from "../components/Scroll";

export default function Home() {
  return (
    <div style={{ minHeight: "1000px" }}>
      <h1>Home</h1>
      <Scroll />
    </div>
  );
}

Tip: The way we use useEffect in the example is to register and unregister the listeners on mount/unmount. But you could also just register on mount and ignore any other rendering event, to do so you would do this:

// components/Scroll.js

import React, { useEffect } from "react";

export default function Scroll() {
  useEffect(function onFirstMount() {
    function onScroll() {
      console.log("scroll!");
    }

    window.addEventListener("scroll", onScroll);
  }, []); // empty dependencies array means "run this once on first mount"

  return null;
}

3. Third solution: dynamic loading

A different solution is to load your Scroll component using dynamic imports and the srr: false option. This way your component won't even be rendered on the server-side at all.

This solution works particularly well when you're importing external modules depending on window. (Thanks Justin!)

// components/Scroll.js

function onScroll() {
  console.log("scroll!");
}

window.addEventListener("scroll", onScroll);

export default function Scroll() {
  return null;
}

// pages/index.js

import dynamic from "next/dynamic";

const Scroll = dynamic(
  () => {
    return import("../components/Scroll");
  },
  { ssr: false }
);

export default function Home() {
  return (
    <div style={{ minHeight: "1000px" }}>
      <h1>Home</h1>
      <Scroll />
    </div>
  );
}

If you do not need the features of useEffect, you can even remove its usage completely as shown here.

Finally, you could also load your Scroll component only in _app.js if what you're trying to achieve is to globally load a component and forget about it (no more mount/unmount on page change).

I have used this technique to display a top level progress bar with NProgress in this article:

Use NProgress with Next.js (Router and fetch events)

Vincent ・ Jun 9 '20

#javascript #nprogress #nextjs #fetch

Use NProgress with Next.js (Router and fetch events)

Vincent — Tue, 09 Jun 2020 14:19:11 +0000

Today I was trying to add NProgress https://github.com/rstacruz/nprogress to my Next.js project.

I wanted the progress bar to:

show when switching routes / pages
show when any fetch call is made
display only after a delay, I don't want to show a loader at EVERY interaction, only when the requests are "slow"

Here's a demo of how NProgress looks like:

Since I hit met some challenges while implementing all of that, I felt like it would be good to share how I did it. So here it is:

First, install the nprogress package:

npm install nprogress

Then edit or create your _app.js and add:

// global styles are required to be added to `_app.js` per Next.js requirements.
import "nprogress/nprogress.css";

const TopProgressBar = dynamic(
  () => {
    return import("components/TopProgressBar");
  },
  { ssr: false },
);

export default function MyApp({ Component, pageProps }) {
  return <>
    <TopProgressBar />
    <Component {...pageProps} />
  </>
}

Here we use dynamic imports and the ssr option to make sure our TopProgressBar is loaded only in browser environements.

If you're wondering how relatively loading components/TopProgressBar works, just configure you jsconfig.json as shown in the Next.js documentation.

Finally create components/TopProgressBar.js:

import Router from "next/router";
import NProgress from "nprogress";

let timer;
let state;
let activeRequests = 0;
const delay = 250;

function load() {
  if (state === "loading") {
    return;
  }

  state = "loading";

  timer = setTimeout(function () {
    NProgress.start();
  }, delay); // only show progress bar if it takes longer than the delay
}

function stop() {
  if (activeRequests > 0) {
    return;
  }

  state = "stop";

  clearTimeout(timer);
  NProgress.done();
}

Router.events.on("routeChangeStart", load);
Router.events.on("routeChangeComplete", stop);
Router.events.on("routeChangeError", stop);

const originalFetch = window.fetch;
window.fetch = async function (...args) {
  if (activeRequests === 0) {
    load();
  }

  activeRequests++;

  try {
    const response = await originalFetch(...args);
    return response;
  } catch (error) {
    return Promise.reject(error);
  } finally {
    activeRequests -= 1;
    if (activeRequests === 0) {
      stop();
    }
  }
};

export default function () {
  return null;
}

Here we register to Next.js router events and monkey patch the global fetch. I was worried monkey patching fetch would fail to register early on but turns out it works 🤷‍♂️!

As you can see, TopProgressBar renders nothing, I guess there might be issues of doing things this way so if you encounter some, just let me know!

That's it!

If you're wondering if NProgress is still maintained because of the low number of commits and "high" number of issues, the good news is that they are working on a new version for 2020: https://github.com/rstacruz/nprogress/pull/218

Even if you're not using Next.js, you should be able to adapt this code to your favorite platform or framework.

Check for extraneous props in React (and save hours of debugging)

Vincent — Mon, 11 May 2020 12:19:28 +0000

Photo by Nicole De Khors from Burst

2022 update: TypeScript provides this feature by default when buidling components in .tsx files. So if you're reading this, and not yet using TypeScript, not may be a good time to give it a try (no more proptypes, no more weird setups).

I have been doing more and more React recently and been bit by a nasty habit of me: passing down unknown props to my React components and wondering why it's not working for 10 minutes (OR MORE 😤). If you've already done some React I am sure you'll understand what I mean.

A debugging story
eslint-plugin-react
Unknown prop warning (DOM components only)
The solution: prop-types-exact
- ESLint configuration
Summary
Bonus: generating Prop Types with VSCode
Going further: Static Type Checking

A debugging story

Let's start with an example.

On Monday, I create a new Input component:

// Input.js
import React from "react";
import PropTypes from "prop-types";

export default function Input({
  label,
  type
}) {
  return (<label>{label} <input type={type} /></label>)
}

Input.propTypes = {
  label: PropTypes.string,
  type: PropTypes.string
};

I use it this way: <Input type="text" label="name" />. All good here's my little input with a label.

On Tuesday, after some heavy code afternoon, I decide that it should be labelText instead of label (naming is hard). Fine, let's change the calling code: <Input type="text" labelText="name" /> and now let's change the implement... "HEY WANNA GRAB A COFFEE?" says a friend? Sure sure ok...

⏳ 30 minutes later, I go back to coding. "Ok let's see this page I was working on ... Hmm, why is this not working? Why is my label blank?"

There's no error in console:

And:

"AH! Yes, that prop, it's not the right name, let's fix it... 😖"

It's easy to spot the mistake if you're reading this right now, but in real life, this happens to me all the time: passing down unknown props to a React component will not trigger a warning or error by default. And this is annoying to the point that I needed a solution.

Let's see how we can solve that.

eslint-plugin-react

I already use eslint-plugin-react and highly recommend it. Its recommended setting will activate the prop-types rule that will warn you if some of your props are missing in your prop types definition, like this:

This is good and you should use it if you like it. But that's not sufficient, it won't warn you about extraneous props when you use your component. It will only warn on missing props validation.

Unknown prop warning (DOM components only)

React has a built-in unknown prop warning activated by default. But this only works on DOM components. So if you try to use <div someProp="yep"> it will warn you:

(only if your prop has a capital letter in it, I just discovered this 🤣)

But it won't warn you of extraneous properties on your own reusable components.

The solution: prop-types-exact

Let's change our previous Input.js component to use airbnb/prop-types-exact:

// Input.js

import PropTypes from "prop-types";
import React from "react";

import exactPropTypes from "prop-types-exact";

export default function Input({
  label,
  type
}) {
  return (<label>{label} <input type={type} /></label>)
}

Input.propTypes = exactPropTypes({
  label: PropTypes.string,
  type: PropTypes.string
});

Now let's refresh our browser and page using <Input type="text" labelText="name" />:

🙌 NICE! Now I can directly see that I passed down some prop that is not recognized by my component: fast and easy debugging instead of a blank console.

ESLint configuration

If you start wrapping your proptypes with exactPropTypes then eslint-plugin-react will no more recognize where are your prop types declared. To fix this, add propWrapperFunctions (an eslint-plugin-react configuration) to your ESLint settings:

{
  "settings": {
     "propWrapperFunctions": [
        "exactPropTypes"
      ]
   }
}

Summary

Using React, Prop Types, ESLint and eslint-plugin-react you can:

know when a proptype is missing
know if you're using a prop on a component that is not defined as a proptype
know when a proptypes is not used in your component
know if you're passing down a wrong proptype (like a number instead of a string)

Bonus: generating Prop Types with VSCode

There's an extension that you can use to generate proptypes easily with VSCode: React PropTypes Generate and it looks like this:

Going further: Static Type Checking

As the React documentation says, on bigger codebases static type checking can help you detect those cases too. As I am not using static type checking I can't tell you if all the cases we saw in this article are covered. If you're using TypeScript or Flow, let me know in the comments how's the experience on extraneous proptypes usage.

PS: As you guess a friend would not interrupt you in the middle of a coding session BUT that's just for the story 😘

Thanks!

Google sheets: get currency rate exchange for a specific date

Vincent — Tue, 28 Apr 2020 19:42:49 +0000

Hi there, here's a quick post on how to get and display a currency rate exchange for a specific date in Google Spreadsheets. This is useful in some situations.

The formula to use is GOOGLEFINANCE and combine it with the INDEX formula to get only the relevant data.

Here's an example: =index(GoogleFinance("CURRENCY:USDEUR", "price", DATE(2020, 04, 10)), 2, 2) would get the USD to EUR rate as for 2020/04/10.

Enjoy!

How to do upserts in Knex.js (PostgreSQL)

Vincent — Sun, 26 Apr 2020 12:11:11 +0000

tl;dr; Since November 2020, you can use Knex.js's onConflict/merge/ignore API features to do upserts in Knex.js: https://knexjs.org/#Builder-onConflict

Hey there, here's a quick post on something that took me way too long to figure out how to do.

For my Node.js database needs, I am using Knex.js together with Objection.js. My database of choice is PostgreSQL. At some point, I needed to do UPSERTs.

What's an upsert? It's a way to express "I have this list of records, identified by this key or combination of keys, please insert them (if they are new) or update them (if they are already existing). Update or insert, that's it.

Upsert is not part of the SQL standard but luckily this has been implemented in PostgreSQL (and other engines) behind the keywords ON CONFLICT DO UPDATE/NOTHING.

~~This has yet to be implemented in Knex.js and there's a promising pull request that would implement it for all engines.~~

November 2020 update: The pull request got merged on Knex and you can now natively do upserts and find or create actions using Knex.

Here's an example:

Find a user, update the name if they exist, otherwise create the user 👇

const [user] = await knex("users")
  .insert({
    name: "Vincent Voyer",
    email: "vincent@codeagain.com"
  })
  .onConflict("email")
  .merge()
  .returning("*");

Create a user if they do not exist, otherwise do nothing 👇

await knex("users")
  .insert({
    name: "Vincent Voyer",
    email: "vincent@codeagain.com"
  })
  .onConflict("email")
  .ignore();

⚠️ You cannot use returning("*") with ignore() though. But you can still request your user afterward.

The new onConflict/ignore/merge features are available in Knex.js >= 0.21.10 and well documented here: https://knexjs.org/#Builder-onConflict.

I am leaving the previous article content, which discusses how to do upserts before it was available in knex.js here:

For now, here's what you can do:

import Knex from "knex";

const knex = Knex({
  client: 'pg',
  connection: process.env.DATABASE_URL,
});

const records = [
  { user_id: 2, team_id: 20, role: "admin" },
  { user_id: 3, team_id: 10, role: "member" },
  { user_id: 4, team_id: 10, role: "admin" },
];

async function upsert() {
  const result = await knex.raw(
      `? ON CONFLICT (user_id, team_id)
              DO UPDATE SET
              role = EXCLUDED.role,
              updated_at = CURRENT_TIMESTAMP
            RETURNING *;`,
      [knex("roles").insert(records)],
    );
}

This would insert or update the records, identified by the combination of user_id, team_id. This means you would need the table roles to have a unique constraint of user_id, team_id.

You could also do this:

import Knex from "knex";

const knex = Knex({
  client: 'pg',
  connection: process.env.DATABASE_URL,
});

const records = [
  { user_id: 2, team_id: 20, role: "admin" },
  { user_id: 3, team_id: 10, role: "member" },
  { user_id: 4, team_id: 10, role: "admin" },
];

async function upsert() {
  const result = await knex.raw(
      `? ON CONFLICT user_id
            DO NOTHING
            RETURNING *;`,
      [knex("roles").insert(records)],
    );
}

If you wanted to insert or do nothing (only insert the new records).

This is the most straightforward solution I have seen which does not try to be smart and generic. You could always wrap all of that and extend the knex query builder if you wanted something more generic and reusable. If you happen to do so or if you have more questions, drop me a comment here!