DEV Community: James Perkins

How to ratelimit tRPC routes with Unkey

James Perkins — Fri, 17 May 2024 00:00:00 +0000

Ratelimiting is not just a feature; it's a lifeline for production applications. Without it, you could face a skyrocketing bill. Your server could be pushed to its limits, leaving real users stranded and your application's reputation at stake.

Unkey provides ratelimiting that is distributed globally and can be easily added to any server, ensuring your protection. We will discuss the features of our service, including synchronous and asynchronous protection, identifier overrides, and how our analytical data can help you identify spikes in usage and how it can be used in a tRPC application.

Prerequisites

To get up and running with the app and follow along, you need:

A fundamental understanding of Next.js, primarily regarding routes and server-side data loading.
Basic familiarity with tRPC.
An application with user authentication implemented. This example uses Auth.js, but you can use any provider you like.
Access to your UNKEY_ROOT_KEY, which you can get by signing up for a free account

In this post, we will use create-t3-app for the demo, so feel free to use that if you want an easy way to use tRPC + Auth.js in a Next.js application.

Installing the `@unkey/ratelimit` package

Before we start coding, we need to install the @unkey/ratelimit package. This package gives you access to Unkey's API with type safety.

npm install @unkey/ratelimit

Updating our env

We need to use the UNKEY_ROOT_KEY to run our ratelimiting package, so we must first update the env.js file in the src directory. Add UNKEY_ROOT_KEY: z.string() to the server object and UNKEY_ROOT_KEY: process.env.UNKEY_ROOT_KEY to the runtimeEnv object.

Now that it is updated add your Unkey root key to your .env as UNKEY_ROOT_KEY which can be found in the Unkey dashboard under settings Root Keys.

Adding ratelimiting to a procedure

Now that the package is installed and our .env has been updated, we can configure our ratelimiter. Inside the server/api/routers/post file, we have a create procedure. This procedure allows users to create posts; currently, users can create as many as they like and as quickly as they like.

Configure our ratelimiter

In this example, we will configure our ratelimiter in the procedure itself. Of course, you can abstract this into a utility file if you prefer. First, we must import Ratelimit from the @unkey/ratelimit package and TRPCError and env.

import { z } from "zod";import { createTRPCRouter, protectedProcedure, publicProcedure,} from "~/server/api/trpc";import { posts } from "~/server/db/schema";import { env } from "~/env";import { TRPCError } from "@trpc/server";import { Ratelimit } from "@unkey/ratelimit";

To configure the Ratelimiter, we need to pass four things along, the root key, the namespace, the limit, and the duration of our ratelimiting. Inside the mutation, add the following:

const unkey = new Ratelimit({ rootKey: env.UNKEY_ROOT_KEY, namespace: "posts.create", limit: 3, duration: "5s",});

The namespace can be anything, but we are using the tRPC route and procedure to make it easier to track in Unkey's analytics. We now have the ability to rate-limit this procedure, allowing only three requests per five seconds.

Using our ratelimiting

To use the ratelimit, we need an identifier. This can be anything you like, such as a user ID or an IP address. We will be using our user's ID as they are required to be logged in to create a new post. Then, we can call unkey.limit with the identifier, and unkey will return a boolean of true or false, which we can use to make a decision.

const { success } = await unkey.limit(ctx.session.user.id);

So now we have the boolean we can check if it's false and then throw a TRPCError telling the user they have been ratelimited and stop any more logic running.

const { success } = await unkey.limit(ctx.session.user.id);if (!success) { throw new TRPCError({ code: "TOO_MANY_REQUESTS" });}

At this point, our code is ready to test. Give it a whirl, and try posting multiple times. You will see that the posts won't update anymore after you are rate-limited.

What about more expensive requests?

Unkey allows you to tell us how expensive a request should be. For example, maybe you have an AI route that costs you a lot more than any other route, so you want to reduce the number of requests that can be used.

const { success } = await unkey.limit(ctx.session.user.id, { cost: 3,});

This request costs three instead of one, giving you extra flexibility around expensive routes.

Faster response

Although Unkey response times are fast, there are some cases where you are willing to give up some accuracy in favor of quicker response times. You can use our async option, which has 98% accuracy, but we don't need to confirm the limit with the origin before returning a decision. You can set this either on the limit request or on the configuration itself.

const unkey = new Ratelimit({ rootKey: env.UNKEY_ROOT_KEY, namespace: "posts.create", limit: 3, duration: "5s", async: true,});// orconst { success } = await unkey.limit(ctx.session.user.id, { async: true,});

While this is a small overview of using Unkey's ratelimiting with tRPC, we also offer other features that aren't covered here, including:

Overrides for specific identifiers
Metadata
Resources flagging

You can read more about those features in our documentation on Ratelimiting.

How to build authentic communication in your team

James Perkins — Fri, 23 Feb 2024 00:00:00 +0000

Before starting Unkey, I worked at several companies ranging in size from 5 to 5,000 employees, and one of the biggest hurdles a company can face is how to encourage authentic communication.

What is authentic communication?

Authentic communication has been covered many times over the years, but the idea has a few core concepts:

Being able to communicate in the style that fits you
Being able to be your genuine self regardless of the interaction you are having
Being able to have a difficult conversation and not feel you are being dismissed
Listening to other parties to understand their point of view

How do you build this into a team?

Building authentic communication into a team requires a lot of work and understanding, but it pays off in the long run for the health of the team and the company. Here are some steps that can help:

Model the behavior: Leaders in the organization or team should lead by example when communicating with others. They should practice active listening, be open to feedback, and show empathy.
Create a safe environment: The team should feel safe to express their thoughts and feelings without fear of judgment or retribution.
Encourage feedback: Regular feedback sessions help team members feel heard and valued. It can also provide opportunities to address any issues or misunderstandings. It also is a great place to come up with new ideas.
Celebrate diversity: Each team member brings a unique perspective and communication style. Celebrating these differences can encourage more authentic communication.

While all of these may seem obvious when the team is smaller, ensuring everyone has an equal opportunity to share their thoughts becomes harder when the team grows.

How are we attempting this at Unkey?

Unkey lives on being open from the top; Andreas and I ensure everyone understands how Unkey runs, what we expect, and the company's direction. So, for our team to feel empowered to be their authentic self when communicating, we do the following:

Updates: Every update we send out to investors and customers is public and can be accessed by the team; I intentionally send a Slack message every month in the general channel. These updates include our current status for KPIs, the money we have spent, the money we have left, usage, and paying customers.
Open communication: We communicate in the general channel using threads to keep things tidy, whether it is an idea, direction, or potentially something we are concerned about. This allows the team to provide their feedback, concerns, or excitement.
Meetings are for everyone: Unkey only runs two meetings as a team; one is planning every Monday, and the other is demo every other Friday. These meetings are open forums and flexible; the team can provide their thoughts about what is up next for us and things they want to work on and provide input on ideas they might have. After anything is demoed, we open the floor for discussions surrounding the feature; we can talk about the code behind it and changes that might improve it, knowing that feedback is welcomed. We also ensure everyone gets time on the floor so no one is left out.
Open door policies: Andreas and I have an open door policy, albeit more of an open Slack policy, which could lead to a meeting. Our team can come to us with concerns, problems they are facing, feedback, and ideas at any point and communicate them to us however they want.
Dedicated space: Andreas meets with the developers twice a month, and I meet with them once a month. This time is for them and not for us. They can come with technical questions, business questions, vision questions, or to chat about how they are feeling.

I understand that our current implementation wont scale past 20 or 30 team members; we will tweak this to allow for authenticated communication as we keep this a core value at Unkey.

The UX of UUIDs

James Perkins — Thu, 07 Dec 2023 00:00:00 +0000

TLDR: Please don't do this:

https://company.com/resource/c6b10dd3-1dcf-416c-8ed8-ae561807fcaf

The baseline: Ensuring global uniqueness

Unique identifiers are essential for distinguishing individual entities within a system. They provide a reliable way to ensure that each item, user, or piece of data has a unique identity. By maintaining uniqueness, applications can effectively manage and organize information, enabling efficient operations and facilitating data integrity.

Lets not pretend like we are Google or AWS who have special needs around this. Any securely generated UUID with 128 bits is more than enough for us. There are lots of libraries that generate one, or you could fall back to the standard library of your language of choice. In this blog, I'll be using Typescript examples, but the underlying ideas apply to any language.

const id = crypto.randomUUID();// '5727a4a4-9bba-41ae-b7fe-e69cf60bb0ab'

Stopping here is an option, but let's take the opportunity to enhance the user experience with small yet effective iterative changes:

Make them easy to copy
Prefixing
More efficient encoding
Changing the length

Copying UUIDs is annoying

Try copying this UUID by double-clicking on it:

c6b10dd3-1dcf-416c-8ed8-ae561807fcaf

If you're lucky, you got the entire UUID but for most people, they got a single section. One way to enhance the usability of unique identifiers is by making them easily copyable. This can be achieved by removing the hyphens from the UUIDs, allowing users to simply double-click on the identifier to copy it. By eliminating the need for manual selection and copy-pasting, this small change can greatly improve the user experience when working with identifiers.

Removing the hyphens is probably trivial in all languages, heres how you can do it in js/ts:

const id = crypto.randomUUID().replace(/-/g, "");// fe4723eab07f408384a2c0f051696083

Try copying it now, its much nicer!

Prefixing

Have you ever accidentally used a production API key in a development environment? I have, and its not fun. We can help the user differentiate between different environments or resources within the system by adding a meaningful prefix. For example, Stripe uses prefixes like sk_live_ for production environment secret keys or cus_ for customer identifiers. By incorporating such prefixes, we can ensure clarity and reduce the chances of confusion, especially in complex systems where multiple environments coexist.

const id = `hello_${crypto.randomUUID().replace(/-/g, "")}`;// hello_1559debea64142f3b2d29f8b0f126041

Naming prefixes is an art just like naming variables. You want to be descriptive but be as short as possible. I'll share ours further down.

Encoding in base58

Instead of using a hexadecimal representation for identifiers, we can also consider encoding them more efficiently, such as base58. Base58 encoding uses a larger character set and avoids ambiguous characters, such as upper case I and lower case l resulting in shorter identifier strings without compromising readability.

As an example, an 8-character long base58 string, can store roughly 30.000 times as many states as an 8-char hex string. And at 16 chars, the base58 string can store 889.054.070 as many combinations.

You can probably still do this with the standard library of your language but you could also use a library like nanoid which is available for most languages.

import { customAlphabet } from "nanoid";export const nanoid = customAlphabet( "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz",);const id = `prefix_${nanoid(22)}`;// prefix_KSPKGySWPqJWWWa37RqGaX

We generated a 22 character long ID here, which can encode ~100x as many states as a UUID while being 10 characters shorter.

| | Characters | Length | Total States |
| UUID | 16 | 32 | 2^122 = 5.3e+36 |
| Base58 | 58 | 22 | 58^22 = 6.2e+38 |

The more states, the higher your collision resistance is because it takes more generations to generate the same ID twice (on average and if your algorithm is truly random)

Changing the entropy

Not all identifiers need to have a high level of collision resistance. In some cases, shorter identifiers can be sufficient, depending on the specific requirements of the application. By reducing the entropy of the identifiers, we can generate shorter IDs while still maintaining an acceptable level of uniqueness.

Reducing the length of your IDs can be nice, but you need to be careful and ensure your system is protected against ID collissions. Fortunately, this is pretty easy to do in your database layer. In our MySQL database we use IDs mostly as primary key and the database protects us from collisions. In case an ID exists already, we just generate a new one and try again. If our collision rate would go up significantly, we could simply increase the length of all future IDs and wed be fine.

Conclusion

By implementing these improvements, we can enhance the usability and efficiency of unique identifiers in our applications. This will provide a better experience for both users and developers, as they interact with and manage various entities within the system. Whether it's copying identifiers with ease, differentiating between different environments, or achieving shorter and more readable identifier strings, these strategies can contribute to a more user-friendly and robust identification system.

IDs and keys at Unkey

Lastly, I'd like to share our implementation here and how we use it in our codebase. We use a simple function that takes a typed prefix and then generates the ID for us. This way we can ensure that we always use the same prefix for the same type of ID. This is especially useful when you have multiple types of IDs in your system.

import { customAlphabet } from "nanoid";export const nanoid = customAlphabet( "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz",);const prefixes = { key: "key", api: "api", policy: "pol", request: "req", workspace: "ws", keyAuth: "key_auth", // <-- this is internal and does not need to be short or pretty vercelBinding: "vb", test: "test", // <-- for tests only} as const;export function newId(prefix: keyof typeof prefixes): string { return [prefixes[prefix], nanoid(16)].join("_");}

And when we use it in our codebase, we can ensure that we always use the correct prefix for the correct type of id.

import { newId } from "@unkey/id";const id = newId("workspace");// ws_dYuyGV3qMKvebjMLconst id = newId("keyy");// invalid because `keyy` is not a valid prefix name

I've been mostly talking about identifiers here, but an api key really is just an identifier too. It's just a special kind of identifier that is used to authenticate requests. We use the same strategies for our api keys as we do for our identifiers. You can add a prefix to let your users know what kind of key they are looking at and you can specify the length of the key within reason. Colissions for API keys are much more serious than ids, so we enforce secure limits.

It's quite common to prefix your API keys with something that identifies your company. For example Resend are using re_ and OpenStatus are using os_ prefixes. This allows your users to quickly identify the key and know what it's used for.

const key = await unkey.key.create({ apiId: "api_dzeBEZDwJ18WyD7b", prefix: "blog", byteLength: 16, // ... omitted for brevity});// Created key:// blog_cLsvCvmY35kCfchi

]]>

Unkey raises 1.5 million pre-seed

James Perkins — Wed, 15 Nov 2023 00:00:00 +0000

The Unkey pre-seed round was led by Essence VC and joined by Sunflower Capital, The Normal Fund, and some fantastic friends.

Essence VC leads our round.

When we met Tim Chen for the first time, Andreas and I were blown away by his experience and his immediate understanding of our vision. We only needed ten minutes before we were bouncing ideas and thinking about where to go with Unkey.

Tim also introduced us to our other partners (Sunflower Capital and The New Normal Fund), who were just as excited to see how API authentication and authorization could be simplified.

Angels

Our angels are a fantastic group of people in no particular order:

Andrew Miklas (Functional Capital / Ex PageDuty Co Founder)
Ant Wilson & Rory Wilding (Supabase Co Founder / Head of Growth at Supabase)
Paul Copplestone (Supabase Co Founder)
Theo Browne (Ping / YouTube / Twitch / Twitter)
Ian Livingstone (Startup Advisor / Investor)
Preston-Werner Ventures (Led by Tom Preston-Werner Co-Founder of GitHub)
George & Chris Perkins (James mum and dad)
Zain Allarakhia (ex Pipe)

How it all started

Andreas and I started Unkey in the middle of June and launched on June 21st, the community loved Unkey and we got some awesome feedback. Once we understood that this side project, which started as a way to solve an annoying problem we both had experienced, was needed by more and more people, we set our sights on building an entire business for it.

We worked nights and weekends for months to ensure Unkey made scaling user-facing APIs more accessible than ever. Shout out to all the friends and family who supported us getting through the 16+ hour days.

How's it going

Since the launch in June, we have had success with the product:

1.2k stars on GitHub
We have done over 13 million API key verifications.
1.6k users have signed up

What's next?

Now that Unkey is funded, we can focus on building the best developer experience in the API space. You may be wondering what is next.

Hiring 1-2 more engineers.
Create an easy-to-use permission system that allows you to control access through RBAC, ABAC, or REBAC.
Create a new gateway system that allows any developer to understand and configure.

Talking about hiring, if you are interested in being part of the team. Check out the job posting and apply.

Thank you for all the support, and we look forward to bringing API scalability to developers like never before.

Why we built Unkey

James Perkins — Tue, 04 Jul 2023 20:59:35 +0000

What is Unkey?

Unkey provides a way for you to create, manage and revoke API Keys that belong to your API. It's a simple concept, but one that we think is important. We built Unkey to layer it into your API so that you can issue keys, manage them and revoke them as needed. This means we need a simple and great DX and fast response times.

Why did we build it?

We built Unkey because we were tired of copying and pasting the same API management into a new codebase. Similar to authentication, you can do it yourself, but having a dedicated company that is working on improvements, and worrying about security and latency means you can focus on the business and core features.

What are the current features?

Unkey provides the following features out of the box:

An admin dashboard to manage your APIs, issue keys, revoke keys and see usage stats.
A simple API with SDKs for typescript
Short-lived keys
Rate limited keys
Metadata attached to keys

Deep dive into the features

Admin dashboard

Our admin dashboard gives you access to several features in a simple-to-use interface. You can create new APIs, issue keys, revoke keys and see usage stats. You can also invite other users to your account so that they can manage your APIs.

Simple API with SDKs

We wanted the DX for Unkey to be as simple as possible. We've built a simple API that you can use to create, manage and revoke keys. We wanted to make sure the API was easy to integrate and didn't take engineering time away.

No API is great without SDKs so we built a typescript one and our community also built an Elixir and Python SDK.

Short-lived keys

When we started Unkey we wanted the ability to issue a short-lived key but we didn't want to restrict the period. We understand that each use case is different so you can pass us an expiration or select it in the UI.

Once the key expires, we revoke the key and the user can no longer access your content. This is great for audit teams, trials, or payment systems that give you access for some time.

Rate limited keys

Rate limiting is essential to all businesses that have an API. Unkey gives you the ability to set each key to a different rate limit. We handle the rate limiting for you, it's as simple as telling us the total amount of burstable requests, the refill rate, and how quickly they should be refilled.

Once you have the key with the limits, with each request we return the amount the request limit, how many remaining, and the reset time. Once a user hits the rate limit we will return json "valid": false alongside a code RATELIMITED if the rate limit has been hit, so you know that this user shouldn't be able to access your endpoints.

Metadata attached to keys

We wanted to give you the ability to attach metadata to your keys so you can easily make business decisions based on the data instead of having to look up the key in your database. You can include anything that you want to read on the server, for example:

{
  "billingTier":"PRO",
  "trialEnds": "2023-06-16T17:16:37.161Z"
}

This allows you to pull less data from other systems in your infrastructure to make business decisions faster.

Built with speed in mind

Unkey was built to have minimal impact on your API, we have our database in the United States but replicate this into other regions so that we can serve your requests as fast as possible. We have our API distributed across the world so that requests will go to the closest region to your user, process at the closet database, and then return the response. Unkey also automatically caches the key on creation, so that the first request is as fast as the second or third request.

On average our requests add less than 40ms to your requests, below are some of the latest requests we are monitoring through a product called Planetfall

Ready to get started? Head over to Unkey and get started for free today.

Rate limiting in Next.js in under 10 minutes

James Perkins — Thu, 23 Mar 2023 08:00:00 +0000

Rate limiting is an important feature that you need in production applications. It may sound scary, but it's actually fairly easy to implement API rate limiting in today's world. In this blog post, we'll walk through the process of implementing rate limiting using Upstash and Redis.

In this example we are using my t3-clerk-minimal app that you can find here. Make sure you have your Clerk keys in your application before starting.

What is Upstash?

Upstash is a serverless data stack that can be served to you on the edge using Redis or Qstash. Most people prefer Redis, and it has a really good free tier. We're going to use Upstash to implement rate limiting because they have a rate limit package that makes it dead simple to implement anywhere that you're using APIs.

First, you need to sign up for an account. Once you've done that, you'll need to click on your console and create a new database. Give it a name, and set the region to the correct region for your area (e.g. US East One or US West).

You can enable TLS if you want. Go ahead and create your Redis for Upstash.

Setting up the ENV

Now that your Redis instance has been created, you'll need to set up the ENV with your Redis REST URL and token. You can copy these information from the Upstash console.

From here, go to your IDE (e.g. Visual Studio Code) and add a .env file. You should already have your Upstash URLs and token in there. Close the .env file once you've pasted your Redis instance details.

Using Clerk for User Authentication

If you haven't used Clerk before, it's a user authentication and management system for the modern web, supporting frameworks like Next.js, Remix, and Gatsby. In this example, we'll be using Clerk for user authentication and protected routes.

To implement rate limiting, we need an identifier for the user (e.g. user ID, IP address) to determine if the same user is making multiple requests. We'll use the user ID from Clerk for this purpose.

Creating the Rate Limiter

First, we need to install two packages, upstash-rate-limit and upstash-redis. You can do so by running:

npm install @upstash/ratelimit @upstash/redis

Next, import both packages in your example router and initialize the rate limiter:

import RateLimit from '@upstash/ratelimit';
import Redis from '@upstash/redis';

const rateLimiter = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(2, "3 s")
});

This example sets a rate limit of 2 requests per 3 seconds. You can adjust the values as needed.

Implementing the Rate Limiter in a Protected Route

Now, we will implement the rate limiter in a protected route. First, create your route (in this example, we'll call it expensive). Inside the route, use the rate limiter with the user ID from Clerk:

import { TrpcError } from '@trpc/server';

// other routes
 expensive: protectedProcedure.query(async ({ ctx }) => {
    const { success } = await rateLimiter.limit(ctx.auth.userId);
    if (!success) {
      throw new TRPCError({ code: "TOO_MANY_REQUESTS" })
    }
    return "expensive"
  })

This code checks whether the rate limit has been exceeded; if it has, a "Too Many Requests" error is thrown.

Testing the Rate Limiter

Now that the rate limiter has been implemented in your application, you can test your protected route to ensure the rate limiting is working as intended. In this example, we will test the rate limiter using a "click here to protected procedure" button on a web page.

Run your application with npm run dev, and navigate to your localhost URL. Click the button, sign in using your preferred authentication method, and you should be redirected to the protected page. Open your browser console and refresh the page - you should encounter a "Too Many Requests" error if you refresh too often.

If you wait a few seconds and refresh again, the error should be gone, indicating that the rate limiting is working as intended.

Conclusion

In this blog post, we've demonstrated how to implement rate limiting using Upstash and Redis, as well as integrating the rate limiter with Clerk for user authentication. Implementing rate limiting is an important aspect of maintaining a secure and stable production application, and this guide should help you get started in adding it to your own projects.

If you found this guide helpful, please consider subscribing to my Youtube channel or the newsletter for more tips and tutorials for web development. Happy coding!

Zod Typesafe User Input

James Perkins — Sat, 26 Nov 2022 17:24:50 +0000

What is Zod?

Zod is a TypeScript-first schema declaration and validation library. It ensures that your data is valid before it is submitted to the server, which means less invalid data and fewer errors on your end. Additionally, Zod allows you to create custom input types to tailor the user experience to your specific needs. Overall, using Zod can help streamline your development process by making sure that your forms, inputs, and API requests are error-free from the start. If you're looking for a reliable way to create typesafe forms and inputs, Zod is worth checking out!

How can Zod help with type safety in forms and input from users?

Zod can help ensure that your forms and input are type-safe, so you won't accidentally allow incorrect data types to be entered into your system. This can help prevent errors from happening in the first place and means that you don't have to write custom validation code for your forms and input fields.

Zod can also help enforce rules around what data can be entered into your forms and input fields, ensuring that only valid data is accepted. By using Zod, you can avoid having to write any extra code or manually check user input against these rules - it will all be handled automatically by Zod. This makes your forms and inputs more reliable and robust against errors.

Overall, using Zod can help improve the quality of your forms and input, making them easier to use, more accurate, and less prone to error.

Why should developers consider using Zod?

Zod offers type-checking for JavaScript, which can be helpful for developers who want to ensure their code is error-free. Zod's validation methods can also be used to enforce input from users, ensuring that only valid data is submitted. Using Zod can help make your development process more efficient and reduce the risk of introducing bugs into your codebase.

How to get started with Zod

First, you need to make sure you have strict mode enabled in your Typescript project.

Then install the Zod package.

npm install zod

Simple schemas

Now you can create a schema to handle your needs, lets's say we want all inputs to be a string we can do that:

import { z } from "zod";

// my schema should be a string

const mySchema = z.string();

mySchema.parse("Zod is awesome"); 
mySchema.parse(69) // throws an error because it is not a string

Now in some cases, we don't want to throw an error we want Zod to tell us we have had an invalid input so we can handle it somewhere else in the stack

import { z } from "zod";


const mySchema = z.string();

mySchema.safeParse("Zod is still really cool");
// This will return { success: true: data: "Zod is still really cool"}


mySchema.safeParse(69)
// This will return { success: false: error: ZodError }

Object schemas

Object schemas allow you to handle complex inputs and validate them and give you an inferred type.

import { z } from "zod";

const User = z.object({
  name: z.string(),
  username: z.string(),
  age: z.number(),
  email: z.string()
});

User.parse({ name: "James Perkins", username: "jamesperkins", age: 33, email: "contactme@jamesperkins.dev" });

// extract the inferred type
type User = z.infer<typeof User>;
// {name: string, username: string, age: number, email: string }

Built-in string validation

One part no one wants to write is the regex to validate strings, I wrote a regex package a few years ago because who wants to remember how to validate the user input of an email? Zod provides a handful of validators, lets's improve our User object:

import { z } from "zod";

const User = z.object({
  name: z.string(),
  username: z.string().min(5),
  age: z.number(),
  email: z.string().email()
});

User.parse({ name: "James Perkins", username: "jamesperkins", age: 33, email: "contactme@jamesperkins.dev" });

// extract the inferred type
type User = z.infer<typeof User>;
// {name: string, username: string, age: number, email: string }

Now a user has to input an email in our email field, which will be validated, and our username must be at least five characters.

Custom error messages

Zod allows you to provide a custom error to present to the user so they understand why they need to change input.

import { z } from "zod";

const User = z.object({
  name: z.string(),
  username: z.string().min(5, {message: "The username must be at least 5 characters"),
  age: z.number(),
  email: z.string().email({message: "The inputted email is invalid, please enter a valid email")
});

User.parse({ name: "James Perkins", username: "jamesperkins", age: 33, email: "contactme@jamesperkins.dev" });

// extract the inferred type
type User = z.infer<typeof User>;
// {name: string, username: string, age: number, email: string }

This is just the beginning of Zod, if you are interested, you can check out their documentation.

My top three tools (Mac edition)

James Perkins — Sun, 31 Jul 2022 22:16:00 +0000

I spend hours on my computer each day, whether it be writing, coding, or creating videos on YouTube. I have my work machine and personal MacBooks set up the same way, so I never have to worry about context switching. Below are the top three tools I use:

AltTab

AltTab is probably a dev tool you didn’t know you needed and once you have it, you wondered why you didn’t know about this earlier.

AltTabbrings the power of Windows’s “alt-tab” window to Macs, if you have ever used a Mac one thing you will notice is there are zero previews when tabbing through your windows. This can be annoying when you have multiple Chrome windows or maybe two visual studio codes open as you can’t see which one to tab to.

With AltTab you get to preview the window before you select it, and it can replace the default alt-tab experience. This is probably number one on my install list.

Xnapper

Xnapper is a screenshot software developed by Tony Dinh it allows you to create awesome screenshots with minimal effort. It’s currently in beta but the developer rate is unreal. It gives you preset options for Twitter, YouTube thumbnails, Reddit, square, 16:9, and more.

On top of this, you can have emails auto redacted, and padding and insets added. Add backgrounds, border-radius. Add a background and some shadow. A screenshot has never looked better!

Below is an example!

Warp

Warp is a terminal built on Rust. It’s super fast and has awesome features. Warpis a terminal that allows you to do a whole lot right from your keyboard. It gives you the ability to look through your history, and create workflows, and has IntelliSense. This is currently for Mac only but they are building one for Windows and one for Linux as well! So all the other users out there can have a great experience.

Command Palette

The command palette allows you to search through all the different commands you can use within the Warp terminal. For example, create a new tab is command + T or command + ] is a new pane. The command pane is completely searchable so you can type in what you are looking for and hit Enter.

Command History

Command History is one of the best features of Warp, not because it’s a powerful or revolutionary feature, but because I and every developer have pressed the up arrow dozens of times to find that one command.

You get two options when you press control + r you can scroll through all of the history items or you can just type at the beginning of the command. Here is an example of me typing the word yarn into my history search:

Custom workflows

Custom workflowsare your alias on steroids, they are created using**.yml**files and can be either user-specific or project specific.

For user-specific ones you add them to**~/.warp/workflows**and for project ones**{{path\_to\_project}}/.warp/workflows**. The format is the same regardless, here is my**code\_profile**one:

name: Change code profiles
description: Change code profile for visual studio code
author: James Perkins
author_url: https://github.com/perkinsjr
tags: ['macos', 'shell', 'vscode']
shells:
    - zsh
    - bash
command: code --user-data-dir {{user_data_dir}} --extensions-dir {{extension_dir}}
arguments:
    - name: user_data_dir
      description: Directory of user-data
    - name: extension_dir
      description: Directory for extensions

My custom workflow takes two arguments which and when used will open my code_profiles stored for visual studio code.

So that is it, my top three tools that I use every single day of the week, and absolutely love them!

A/B testing with Next.js Middleware and TinaCMS

James Perkins — Fri, 03 Jun 2022 12:56:16 +0000

A/B testing can be an incredibly useful tool on any site. It allows you to increase user engagement, reduce bounce rates, increase conversion rate and effectively create content.

Tina opens up the ability to A/B test, allowing marketing teams to test content without the need for the development team once it has been implemented.

Introduction

We are going to break this tutorial into two sections:

Setting up A/B Tests with NextJS's middleware.
Configuring our A/B Tests with Tina, so that our editors can spin up dynamic A/B Tests.

Creating our Tina application

This blog post is going to use the Tailwind Starter. Using the create-tina-app command, choose "Tailwind Starter" as the starter template:

# create our Tina application

npx create-tina-app@latest a-b-testing

✔ Which package manager would you like to use? › Yarn
✔ What starter code would you like to use? › Tailwind Starter
Downloading files from repo tinacms/tina-cloud-starter. This might take a moment.
Installing packages. This might take a couple of minutes.

## Move into the directory and make sure everything is updated.

cd a-b-testing
yarn upgrade
yarn dev

With your site running, you should be able to access it at [http://localhost:3000](http://localhost:3000)

Planning our tests

The home page of the starter should look like this:

This page is already setup nicely for an A/B test, its page layout ([slug].tsx) renders dynamic pages by accepting a variable slug.

Let's start by creating an alternate version of the homepage called home-b. You can do so in Tina at http://localhost:3000/admin#/collections/page/new

Once that's done, go to: http://localhost:3000/home-b to confirm that your new /home-b page has been created.

Setting up our A/B tests

Utlimately, we want our site to dynamically swap out certain pages for alternate page-variants, but we will first need a place to reference these active A/b tests.

Let's create the following file at content/ab-test/index.json:

{
  "tests": [
    {
      "testId": "home",
      "href": "/",
      "variants": [
        {
          "testId": "b",
          "href": "/home-b"
        }
      ]
    }
  ]
}

We'll use this file later to tell our site that we have an A/B test on our homepage, using /home-b as a variant.

In the next step, we'll setup some NextJS middleware to dynamically use this page variant.

Delivering dynamic pages with NextJS middleware

NextJS's middleware allows you to run code before the request is completed. We will leverage NextJS's middleware to conditionally swap out a page for its page-variant.

You can learn more about NextJS's middleware here.

Start by creating the pages/_middleware.ts file, with the following code

//pages/_middleware.ts

import { NextRequest, NextResponse } from 'next/server'
import abTestDB from '../content/ab-test/index.json'
import { getBucket } from '../utils/getBucket'

// Check for AB tests on a given page
export function middleware(req: NextRequest) {
  // find the experiment that matches the request's url
  const matchingExperiment = abTestDB.tests.find(
    test => test.href == req.nextUrl.pathname
  )

  if (!matchingExperiment) {
    // no matching A/B experiment found, so use the original page slug
    return NextResponse.next()
  }

  const COOKIE_NAME = `bucket-${matchingExperiment.testId}`
  const bucket = getBucket(matchingExperiment, req.cookies[COOKIE_NAME])

  const updatedUrl = req.nextUrl.clone()
  updatedUrl.pathname = bucket.url

  // Update the request URL to our bucket URL (if its changed)
  const res =
    req.nextUrl.pathname == bucket.url
      ? NextResponse.next()
      : NextResponse.rewrite(updatedUrl)

  // Add the bucket to cookies if it's not already there
  if (!req.cookies[COOKIE_NAME]) {
    res.cookie(COOKIE_NAME, bucket.id)
  }

  return res
}

There's a little bit going on in the above snippet, but basically:

We check if there's an active experiment for our request's URL
If there is, we call getBucket to see which version of the page we should deliver
We update the user's cookies so that they consistently get delivered the same page for their given bucket.

You'll notice that the above code references a getBucket function. We will need to create that, which will conditionally put us in a bucket for each page's A/B test.

// /utils/getBucket.ts

export const getBucket = (matchingABTest: any, bucketCookie?: string) => {
  // if we already have been assigned a bucket, use that
  // otherwise, call getAssignedBucketId to put us in a bucket
  const bucketId =
    bucketCookie ||
    getAssignedBucketId([
      matchingABTest.testId,
      ...matchingABTest.variants.map(t => t.testId),
    ])

  // check if our bucket matches a variant
  const matchingVariant = matchingABTest.variants.find(
    t => t.testId == bucketId
  )

  if (matchingVariant) {
    // we matched a page variant
    return {
      url: matchingVariant.href,
      id: bucketId,
    }
  } else {
    //invalid bucket, or we're matched with the default AB test
    return {
      url: matchingABTest.href,
      id: matchingABTest.testId,
    }
  }
}

function getAssignedBucketId(buckets: readonly string[]) {
  // Get a random number between 0 and 1
  let n = cryptoRandom() * 100
  // Get the percentage of each bucket
  const percentage = 100 / buckets.length
  // Loop through the buckets and see if the random number falls
  // within the range of the bucket
  return (
    buckets.find(() => {
      n -= percentage
      return n <= 0
    }) ?? buckets[0]
  )
}

function cryptoRandom() {
  return crypto.getRandomValues(new Uint32Array(1))[0] / (0xffffffff + 1)
}

The above code snippet does the following:

Checks if we're already in a bucket, and if not, it calls getAssignedBucketId to randomly put us in a bucket.
Returns the matching A/B test info for our given bucket.

That should be all for our middleware! Now, you should be able to visit the homepage at http://localhost:3000, and you will have a 50-50 chance of being served the contents of home.md, or home-b.md. You can reset your bucket by clearing your browser's cookies.

Using Tina to create A/B Tests

At this point, our editors can edit the contents of both home.md & home-b.md, however we'd like our editors to be empowered to setup new A/B tests.

Let's make our content/ab-test/index.json file from earlier editable, by creating a Tina collection for it.

Open up your schema.ts and underneath the Pages collection, create a new collection like this:

/// ...,
    {
      label: "AB Test",
      name: "abtest",
      path: "content/ab-test",
      format: "json",
    }

We now need to add the fields we want our content team to be able to edit, so that would be the ID, the page to run the test against, and the variants we want to run. We also want to be able to run tests on any number of pages so we will be using a list of objects for the tests field.

If you want to learn more about all the different field types and how to use them, check them out in our Content Modeling documentation.

    {
      label: "AB Test",
      name: "abtest",
      path: "content/ab-test",
      format: "json",
      fields: [
        {
          type: "object",
          label: "tests",
          name: "tests",
          list: true,
          ui: {
            itemProps: (item) => {
              return { label: item.testId || "New A/B Test" };
            },
          },
          fields: [
            { type: "string", label: "Id", name: "testId" },
            {
              type: "string",
              label: "Page",
              name: "href",
              description:
                "This is the root page that will be conditionally swapped out",
            },
            {
              type: "object",
              name: "variants",
              label: "Variants",
              list: true,
              ui: {
                itemProps: (item) => {
                  return { label: item.testId || "New variant" };
                },
              },
              fields: [
                { type: "string", label: "Id", name: "testId" },
                {
                  type: "string",
                  label: "Page",
                  name: "href",
                  description:
                    "This is the variant page that will be conditionally used instead of the original",
                },
              ],
            },
          ],
        },
      ],
    }

You may notice the ui prop. We are using this to give a more descriptive label to the list items. You can read about this in our extending Tina documentation.

We also need to update our RouteMappingPlugin in .tina/schema.ts to ensure that our collection is only editable with the basic editor.

      const RouteMapping = new RouteMappingPlugin((collection, document) => {
        if (collection.name == 'abtest') {
          return undefined
        }
        // ...

Now, restart your dev server, and go to: [http://localhost:3000/admin#/collections/abtest/index](http://localhost:3000/admin#/collections/abtest/index). Your editors should be able to wire up their own A/B tests!

The process for editors to create new A/B tests would be as follows:

Editor creates a new page in the CMS
Editor wires up the page as a page-variant in the "A/B Tests" collection

And that's it! We hope this empowers your team to start testing out different page variants to start optimizing your content!

How to keep up to date with Tina?

The best way to keep up with Tina is to subscribe to our newsletter. We send out updates every two weeks. Updates include new features, what we have been working on, blog posts you may have missed, and more!

You can subscribe by following this link and entering your email: https://tina.io/community/

Tina Community Discord

Tina has a community Discord full of Jamstack lovers and Tina enthusiasts. When you join, you will find a place:

To get help with issues
Find the latest Tina news and sneak previews
Share your project with the Tina community, and talk about your experience
Chat about the Jamstack

Tina Twitter

Our Twitter account (@tina_cms) announces the latest features, improvements, and sneak peeks to Tina. We would also be psyched if you tagged us in projects you have built.

Using Next Middleware to access and use geolocation in a non dynamic route

James Perkins — Thu, 05 May 2022 23:22:07 +0000

Using Next Middleware to access geolocation

I don’t use Middleware from Next.js often, but I have dabbled a few times. Recently someone in another Developer’s Discord, I also have a Discord if you want to chat Jamstack, asked about how to add the Geolocation to a non-dynamic page.

What is `_middleware`?

Middleware allows you to use code over configuration. This gives you so much more flexibility over how your applications act when a user visits. The Middleware runs before the incoming request is completed, this allows you to modify the response by rewriting, redirecting, adding headers, or even streaming HTML.

Adding Geolocation

Depending on your situation, depends on where you need your _middleware.(js.tsx) for this example it will run on every page. However, you can use it in nested routes to run for specific pages or dynamic routes.

Create your file where you need it, inside that file you need to import {NextResponse} from “next/server”

Then create export async function middleware(req){} this is where we are going to be doing the work to add our user's location.

The Middleware from Next.js has access to cookies, nextUrl , i18n , ua, geo, ip in the request which means we can use this to add geolocation to our application.

To rewrite our request we are going to need to access the nextUrl and geo from our request.

For example const { nextUrl: url, geo } = req now we can modify the nextUrl to add searchParams (query parameters) to our request.

Then finally return our new URL using NextResponse below is the full request.

import { NextResponse } from 'next/server'

export async function middleware(req) {
  const { nextUrl: url, geo } = req
  const country = geo.country || 'US'

  url.searchParams.set('country', country)

  return NextResponse.rewrite(url)
}

Updating our page

Now we need to add and use our geolocation to the page, to do that we need to first pass the middleware through getServerSideProps to pass it to the page. Currently, the _middleware has all the info on the server.

// pass the _middleware to the index.js 

export const getServerSideProps = ({ query }) => ({
  props: query,
})

Finally, we can use it on the page, however, we want by passing the props to the page:

import Head from 'next/head'
import Image from 'next/image'
import styles from '../styles/Home.module.css'
import { useRouter } from 'next/router'

// pass the _middleware to the index.js 

export const getServerSideProps = ({ query }) => ({
  props: query,
})

export default function Home(props) {
    // use it right from the props
  console.log(props.country)
// or you can grab it from the router.
  const router = useRouter();

  console.log(router.query);
  return (
    <div className={styles.container}>
      <Head>
        <title>Create Next App</title>
        <meta name="description" content="Generated by create next app" />
        <link rel="icon" href="/favicon.ico" />
      </Head>

      <main className={styles.main}>
        <h1 className={styles.title}>
          Welcome to {props.country}
        </h1>
      </main>
    </div>
  )
}

Now you can handle whatever you need knowing the geolocation.

One thing to note, geolocation made not be available so just be cautious, that is why we have a back up plan of US

Next.js - How to optimize fonts

James Perkins — Mon, 02 May 2022 23:53:17 +0000

Next.js Font Optimized

When using 3rd party fonts we have to make sure they are optimized, in the times before Next.js 10.2 you had to manually optimize them for example:

<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Merriweather+Sans&display=swap" rel="stylesheet" />
<link
   href="https://fonts.googleapis.com/css2?family=Squada+One:wght@400&display=swap"
   rel="stylesheet"
   />

After 10.2 Next.js can now optimize Google fonts and Typekit with a variety of font optimization, inside your _document.js you can now provide the font:

import Document, { Html, Head, Main, NextScript } from 'next/document'

class MyDocument extends Document {
  render() {
    return (
      <Html>
        <Head>
          <link
            href="https://fonts.googleapis.com/css2?family=Inter&display=optional"
            rel="stylesheet"
          />
        </Head>
        <body>
          <Main />
          <NextScript />
        </body>
      </Html>
    )
  }
}

export default MyDocument

You can even provide the preferred display options as a query parameter :

block : The text blocks (is invisible) for a short period
swap: There is no block period (so no invisible text), and the text is shown immediately in the fallback font until the custom font loads
fallback: This is somewhere in between block and swap. The text is invisible for a short period of time (100ms). Then if the custom font hasn't been downloaded, the text is shown in a fallback font (for about 3s), then swapped after the custom font loads.
optional: This behaves just like fallback, only the browser can decide to not use the custom font at all, based on the user's connection speed (if you're on a slow 3G or less, it will take forever to download the custom font and then swapping to it will be too late and extremely annoying)
auto: This basically ends up being the same as fallback.

Warp is the future of terminals

James Perkins — Sat, 16 Apr 2022 23:56:18 +0000

I spend a lot of time working with tooling, whether it's for my job, content creation, or just development tools in general. Warp is a brand new terminal experience, so let's talk about it.

What is Warp?

Warp is a terminal that allows you to do a whole lot right from your keyboard. It gives you the ability to look through your history, create workflows and has intellisense. This is currently for Mac only but they are building one for Windows and one for Linux as well! So all the other users out there can have a great experience.

Command Palette

The command palette allows you to search through all the different commands you can use within the Warp terminal. For example create new tab is command + T or command + ] is a new pane. The command pane is completely searchable so you can type in what you are looking for hit Enter.

Command History

Command History is one of the best features of Warp, not because it's powerful or revolutionary feature, but because I and every developer has press the up arrow dozens of times to find that one command.

You get two options when you press control + r you can scroll through all of the history items or you can just type in the beginning of the command. Here is an example of me typing the word yarn into my history search:

As you can see I can now just select what I need from the list, speeding up the time spent remembering what did I type?

Workflows

Workflows are the powerhouse to this terminal application, workflows allow you to run commands similar to aliases. Each workflow gets a searchable title and description which make easy to find what you are looking for. The workflow pane can be opened by typing shift + control + r.

When you choose a workflow, you will prompted to fill in any arguments you might need to make the command work which you can navigate using shift + tab :

Custom workflows

Custom workflows are your alias on steroids, they are created using .yml files and can be either user specific or project specific.

For user specific ones you add them to ~/.warp/workflows and for project ones {{path_to_project}}/.warp/workflows. The format is the same regardless, here is my code_profile one:

name: Change code profiles
description: Change code profile for visual studio code
author: James Perkins
author_url: https://github.com/perkinsjr
tags: ['macos', 'shell', 'vscode']
shells:
    - zsh
    - bash
command: code --user-data-dir {{user_data_dir}} --extensions-dir {{extension_dir}}
arguments:
    - name: user_data_dir
      description: Directory of user-data
    - name: extension_dir
      description: Directory for extensions

My custom workflow takes two arguments which and when used will open my code_profiles have stored for visual studio code and it looks like this in the application.

This is just scratching the surface of the feature set that Warp offers and how it can be used. I recommend giving it a shot and seeing what you think.

DEV Community: James Perkins

How to ratelimit tRPC routes with Unkey

Prerequisites

Installing the @unkey/ratelimit package

Updating our env

Adding ratelimiting to a procedure

Configure our ratelimiter

Using our ratelimiting

What about more expensive requests?

Faster response

How to build authentic communication in your team

What is authentic communication?

How do you build this into a team?

How are we attempting this at Unkey?

The UX of UUIDs

The baseline: Ensuring global uniqueness

Copying UUIDs is annoying

Prefixing

Encoding in base58

Changing the entropy

Conclusion

IDs and keys at Unkey

Unkey raises 1.5 million pre-seed

Essence VC leads our round.

Angels

How it all started

How's it going

What's next?

Why we built Unkey

What is Unkey?

Why did we build it?

What are the current features?

Deep dive into the features

Admin dashboard

Simple API with SDKs

Short-lived keys

Rate limited keys

Metadata attached to keys

Built with speed in mind

Rate limiting in Next.js in under 10 minutes

What is Upstash?

Setting up the ENV

Using Clerk for User Authentication

Creating the Rate Limiter

Implementing the Rate Limiter in a Protected Route

Testing the Rate Limiter

Conclusion

Zod Typesafe User Input

What is Zod?

How can Zod help with type safety in forms and input from users?

Why should developers consider using Zod?

How to get started with Zod

Simple schemas

Object schemas

Built-in string validation

Custom error messages

My top three tools (Mac edition)

AltTab

Xnapper

Warp

Command Palette

Command History

Custom workflows

A/B testing with Next.js Middleware and TinaCMS

Introduction

Creating our Tina application

Planning our tests

Setting up our A/B tests

Delivering dynamic pages with NextJS middleware

Using Tina to create A/B Tests

How to keep up to date with Tina?

Tina Community Discord

Tina Twitter

Using Next Middleware to access and use geolocation in a non dynamic route

Using Next Middleware to access geolocation

What is _middleware?

Adding Geolocation

Updating our page

Next.js - How to optimize fonts

Next.js Font Optimized

Installing the `@unkey/ratelimit` package

What is `_middleware`?