DEV Community: Richard Kovacs

A/B Testing Next.js Applications

Richard Kovacs — Fri, 07 Jun 2024 09:34:00 +0000

When you build a website, your goal is potentially to attract visitors. It is especially true if your website is a software as a service (SaaS) that you want to show to as many potential customers as possible. An obvious choice is to run some marketing campaigns and see how the website (and the software) works with real users. Some folks even promote their landing pages well before the software is built to know the market's reaction before jumping into a new project that could last several months. If there are no registrations or downloads during the marketing phase, they go to the next project and don't waste time on a useless idea.

But how can you determine whether a landing page is good without having something to compare it to? What if you only have to change the motto or the main hero image, and those small changes would result in 5 times as many registrations? How can you test these alternatives with a single website using a single domain? This concept is called A/B Testing.

What is A/B Testing?

A/B Testing (or split testing) is when you have two versions of the same website, version A and version B (hence the name), and you want to test which one converts visitors into users better. The change between the versions can be anything:

The background color of the primary call to action button,
The hero image,
The motto,
The alignment of the text,
Or anything you can think of.

When two versions of the same website are available to the public, you gain invaluable information. You can not only calculate how many visitors out of 1000 buy or sign up for your product but also test whether an extremely simple change results in a significantly better (or worse) performance. You can experiment with any combination you can think of.

Large companies often do this and have an easier time than us indie makers since they have regular site visitors. It is very easy for them to test a new feature or website motto on 1% of the users because even that one percent is probably tens of thousands or even millions of people. When an individual launches a new product, they have fewer visitors, meaning less information. Still, running an A/B test on two user groups with 50-50% can have surprising results.

How to A/B Test a Website?

No need to worry because A/B testing is well-established, and multiple tools are made exactly for this purpose. Here is a non-exhaustive list of them:

These tools are A/B Testing beasts with in-depth analysis, extensive options, and much more to offer than most of you probably need. For this reason, most of them are rather pricy. Some are entirely out of your budget range, at least initially.

But running just a slight A/B test on your site still makes sense. Even if it is as simple as changing the main call-to-action button from "I'm interested" to "I need this". You don't know which will work, but if you never test alternatives, you never will.

How to A/B Test for Free?

– Mom, can we have A/B testing?
– We have A/B testing at home.
– A/B testing at home:

It's hard to find a tool with a small feature set suitable for individuals who want to test simple websites with a limited number of regular visitors. As a developer with multiple online tools, I also faced this problem. And just like a stereotypical developer would do, I developed my solution because I didn't want to pay for these expensive tools. I will probably need them later, but not at the start. Here is how mine works and how you can also use it for free on your website. Note that I am using Next.js, so this guide will work best if you have the same setup.

Install the nexperiment Node module with your favorite package manager as a first step. Feel free to check out the source code on GitHub and the module on NPM.

The first version of my A/B testing tool was less than 100 lines of code, and it served an extremely simple feature: to change one phrase somewhere on the site to a different one. As I mentioned, this can be a call to action button, a motto, or anything else you need. My goal was to test different button texts and mottos. The tool is as simple as a React Hook and a Context.

Once you successfully installed the module, wrap your application in the ABTestProvider context in one of your Layout files. Then, pass the items you want to test in the items parameter. The prefix parameter is optional, but ab-test- is its default value. It is the prefix used for the generated IDs to differentiate them from the rest of the application in a later step. It should look something like this:

import { TestItems, ABTestProvider } from "nexperiment";

export default function Layout({ children }: { children: React.ReactNode }) {
  return (
    <ABTestProvider items={items} prefix="ab-test-">
      {children}
    </ABTestProvider>
  );
}

The items parameter has the following interface:

const items: TestItems = {
  motto: {
    A: "Your words, now available in audio. Simple as that.",
    B: "Converting your blog into audio has never been easier.",
  },
  cta: {
    distribution: 0.7,
    A: "Let's hear it",
    B: "I need this",
  },
};

If you don't set a distribution, the default value will be 0.5, meaning that each version has a 50% probability of becoming visible to any user. In the example above, the motto will be A or B 50% of the time, while in the case of the call-to-action button text, A will be rendered 70% of the time and B in the rest.

You can now use the useABTest hook and its getItem method to retrieve a value in your code. Here is an example based on the above:

export default function HeroSection() {
  const abStore = useABTest();
  const cta = abStore.getItem("cta");

  ...

  return (
    <div>
      <button id={cta.id}>{cta.text}</button>
    </div>
  );
}

This code will select one of the call-to-action texts and the correspondent ID. When a user opens the website, the hook will choose one of the texts from the options for the button and store it in the local storage. It will also return a unique ID, which will be ab-test-cta-A or ab-test-cta-B, in case you didn't change the prefix to something else earlier. You will need these IDs in the next step because, so far, you only have a random website but no analytics about user behavior. The point of saving the chosen configuration to local storage is that once users see a version, they are stuck with it and won't experience the site changing on each refresh. They will permanently become user A or user B.

Tracking User Behavior

As I mentioned, using the module is completely free. There are no servers behind, no blocking JavaScript, nothing. You must rely on a separate analytics tool to collect user information. I chose Google Analytics (GA) and Google Tag Manager (GTM) to keep things simple, but you can use any other tool you are familiar with. From now on, the post will be about how to set up tracking with GA and GTM, so feel free to skip to the last part if you are going with an alternative.

The point of generating unique IDs in the previous step was to have the ability to easily track what consequences a simple change on the landing page has. If you don't use the generated ID, you will have no information about user behavior because you cannot decide which version a user saw before clicking the button.

With these IDs, tracking button clicks is easy. You only have two steps left:

You have to configure a Trigger in GTM that collects every click on elements with IDs that start with ab-test- (or your custom prefix),
You must configure a Tag that uses this Trigger to send events to Google Analytics.

If you are unfamiliar with how to configure the above, you can find plenty of useful information about setting up Tag Manager on a site using the Next.js App Router in one of my previous posts. I only cover the basics there, but it is more than enough for this simple A/B testing use case.

Once you have everything set up, you should slowly see the events appearing in Google Analytics as the users visit both versions of your website.

Tips and Caveats

If you need an extensive A/B testing solution with in-depth analysis and dozens of features, this module is not for you. The potential users are indie makers on a small budget testing a simple and new landing page with different setups.
This module will only work with Next.js and React applications. I don't intend to extend it to other frameworks because this is the stack I am currently using. But I figured it might also be helpful for others using the same stack.
In its current version, you cannot configure boolean, image, or composite items where multiple texts belong to the same configuration. It only supports single strings. However, these use cases look simple and common enough that I plan to add them in later versions.
You can configure different interactions using the unique ID, not just a single button text and its click. For example, it is possible to configure the hero text as a variable and then use the corresponding ID for the login button in the navigation bar. This way, you can track which motto attracted more users. Use the same name when calling getItem in both locations. The hook will return the same information for a given item name regardless of how many times you call it.

If you are wondering about this now, you are completely fine without A/B testing. It's not a requirement for a successful business. You probably don't need it if you write amazing copy on each website on the first try. But if you like experimenting like me, you now have a simple tool in your arsenal to get started.

Getting Started with GTM in Next.js App Router

Richard Kovacs — Thu, 06 Jun 2024 08:00:29 +0000

Have you ever thought about tracking user behavior on your website? If yes, you are probably familiar with Google Analytics, the state-of-the-art user tracker companion. How about Google Tag Manager? Also yes? Of course, since that's how you've found this post. If you are at the first step of setting it up, you are in the right place. But in case you have never heard about it, I still recommend reading it to the end because who knows? You might learn something you didn't even know you needed. So let's get started.

Motivation

Why did I write this post? There are a couple of reasons. First, I have always felt that there is much more to Google Analytics than simply seeing how many active users you have currently and had in the last few days. But I had no idea how to configure this. I also knew it was possible to see which buttons your users press the most, but again, I didn't know how to do it.

Second, I found a convenience library made by the Next.js team that makes it extremely easy to integrate Google Tag Manager and Google Analytics into a website. Still, sadly, as of writing this post, it contains an unresolved bug many face when installing it. I will uncover how to overcome this and what my workaround is.

And finally, my ultimate reason was that I wanted to configure all of the above on ReadSonic, my audioblog website. So, if you have the same ambitions as me and work with the same tech stack, namely Next.js, and Google, then look no further because this post was written for you.

A quick disclaimer before I jump into it: this post isn't an in-depth guide about Google Tag Manager. It is a simple Next.js and GTM tutorial to get you started as fast as possible.

What is Google Tag Manager?

Google Tag Manager (GTM) is a marketing tool that helps you track user behavior with extremely granular configuration for each kind of interaction. It can track scroll behavior, clicks, video plays, back and forward navigation inside the page, and many more, just to mention a few. But the primary goal of this post is to show how to set up GTM with the Next.js App Router, so I will simply stay at tracking clicks on the website.

GTM is free to use, and you can also easily connect it to Google Analytics to see the tracked clicks in one location. GTM defines numerous user interaction types by default, but of course, you can define custom ones, although they recommend using the built-in interactions for better results.

GTM can look extremely overwhelming on the first visit. You find Tags, Triggers, and Variables everywhere. You feel they are somehow connected and need them all, but you don't know how and when. On top of that, many guides on the internet use the same terminology as GTM, so you aren't helped with those either. Here is my attempt.

The first step you must make if you just opened Google Tag Manager is to create a new Account and a Container. In my case, it looks like the image below.

You have to choose the name and location of the new account. Then, you have to configure the name and type of the container. Next, you must agree to the Google Tag Manager Terms of Service Agreement and, depending on your location, potentially GDPR. Once your account and container are created, you will see a code snippet that you can use to embed GTM on your website.

You may use this option, but I will show you a more convenient method designed explicitly for Next.js. Don't worry if you accidentally close this popup since you can reopen it later by pressing your GTM tag in the toolbar. It has a format like this: GTM-XXXXXXXX.

You are done with the first step, so let's explore a more exciting territory: Variables.

What are Variables?

You definitely need them, but chances are you don't have to define custom ones, contrary to what other guides might suggest. Variables have two major types: built-in and user-defined ones. There are many more types inside these two categories, but explaining them is out of the scope of this post. For now, you have to understand three things:

there are built-in variables you can choose from,
you can define custom ones if they are not enough, and finally,
you most probably won't need custom ones for simple use cases.

Simply put, Variables are metadata in your website that you can refer to. The id of a button is a Variable. Its class is also a Variable. And so is its target, text, and URL. The same is true for any other element. You can refer to any div, a tag, button, span, etc., by variables in GTM. But the hostname, URL, and the path the user is currently on are also Variables. There are specific Variables for forms, videos, and navigation history. I think you get the idea now.

The most basic use case is this: you have some buttons on your landing page, each having a different id, and you want to track clicks on them. For this, first, you must enable clicks in the Variables menu by clicking the Configure button in the upper right corner.

Enable each Click Variable like the image above, and that's it. Understanding what a Variable is is important because, in the next step, you have to refer to them when you define Triggers.

What are Triggers?

While it's relatively hard to choose the most important component of GTM, if I had to choose one, I would say Triggers are these.

Triggers are user interactions GTM can track. They have multiple types. Like in Variables, you can also define custom Triggers or events your app can listen to. Click-type Triggers can be the click on a link or any element in the DOM, like a button.

But the depth of scrolling a user does on your site is also a Trigger. And so is the loading of the DOM.

So, how are Triggers and Variables connected? For the sake of simplicity, I will stay at the simplest kind of Trigger: a click.

To simplify things even more, I must mention that you can define a Trigger that listens for every click. In this case, you don't need a Variable at all. However, usually this is not the case. Collecting every single click, even those on a div, would probably bloat your logs, and you don't want that.

What you want instead is to listen to specific clicks. For example, you would collect every click on the main call-to-action button in your hero section with id="cta-button". Or you could also collect every navigation to the login page with a link or button having id="login-button".

Remember what Variables were? Metadata in your website, like an id or a class. To filter clicks based on these, you must use them when setting up a Trigger. Here is what it looks like on the UI.

NOTE: If you want to track clicks on a button with other elements in itself, child elements can hijack that click if they are below the cursor. A common example is a button with an icon in it. A click of the button will usually be a click on the image. To overcome this, you can set pointer-events: none; on the image. This way, every click will happen on the button and show up correctly in your analytics.

Okay, so you already know that you can listen to user events on your site (Triggers), and you can filter them based on some conditions (Variables). But you still have no idea how to collect and view these. You probably also want to provide this information to other services like Google Analytics or some other external user behavior monitoring tool. For this, you need Tags.

What are Tags?

Since GTM is only for configuration, you need a place to collect and view events that happened on your page. This is what Tags are for. Tags are the connection to the world outside GTM.

When you define a Tag, you must choose a Trigger that the Tag should listen to, and then you must configure where the event should be forwarded. As you probably already guessed, Tags also have multiple types, and you can define custom Tags, but I will keep things at the beginner level for this post. If you are a Google user, forwarding events to Google Analytics 4 is an obvious choice. In the image below, you can see how the GA4 Event type is chosen for the Tag Type, and below it, there is the Trigger from the previous image.

In this case, you must also configure a Measurement ID, which is the ID of your Google Analytics 4 web data stream. It has the format G-XXXXXXXXXX. This ID is crucial to finding the correct GA4 configuration where GTM should forward the button clicks. Finally, you must provide an Event Name, which will be displayed in Google Analytics. The event name can be static, or it can be a Variable. The same Variable I spoke about earlier. If you are using a variable like Click Text, the event's name in Google Analytics will be the innerText of the button clicked. It is especially useful when you define Triggers based on class, and the same Tag collects multiple button clicks. Click Text makes it easy to differentiate between them. You can, of course, use other available Variables if you want.

When you define your Measurement ID, there is one thing you must be aware of. You will probably see the warning below.

The Google Tag is a specific kind that helps establish the data flow from the website to Google Analytics or other destinations. You can find more information on the Tag Manager Help page if interested.

Simply create it with the Create tag button on the right. Again, use your Google Analytics ID as the Tag ID when creating it. You only need to do this once. Other Tags created later will find your existing Google Tag and won't throw the warning anymore.

With this setup, you are done. You should see the events in Google Analytics soon if everything goes well. Disable any ad blockers if you experience nothing being shown on the dashboard. Using them is a common cause of missing events since they usually prohibit website trackers from working correctly.

When you created the container in the first step, you received a guide (a code snippet) that you must include in your website to make GTM work. But as I mentioned at that point, you don't need it at all because Next.js already did the hard work for you.

Integration with Next.js

Since you are here, you are probably familiar with Next.js, so I won't make this post longer by introducing it. Next.js makes it ridiculously easy to integrate some third-party apps. Luckily for us, one of them is Google Tag Manager.

NOTE: I am using the App Router in this guide. Ensure you are also using it when following the examples below. There's a good chance it would also work with the Pages Router, but I only tested and used the App Router in the following snippets.

Execute the following command to install the third-party extension module:

npm i @next/third-parties

When it is finished, you can integrate GTM with a simple line:

import { GoogleTagManager } from '@next/third-parties/google';

export default function RootLayout({
  children,
}: {
  children: React.ReactNode
}) {
  return (
    <html lang="en">
      <body>{children}</body>
      <GoogleTagManager gtmId="GTM-XXXXXXXX" />    <-- The line you need
    </html>
  )
}

Put the above code in your root layout.tsx file to include GTM on every page of your web application. This module also exposes helper functions to send custom events to GTM. However, if you are satisfied with the built-in Triggers and Variables I explained earlier, the code above is enough to send every button click to GTM. No additional code is required.

Integrating Google Analytics

The third-party module also helps with Google Analytics integration and lots of other web analytics tools, but as of writing this post, there is an unresolved bug with it, which results in the following error:

To fix this, I only used the third-party extension for GTM and solved GA configuration with a custom snippet. Here is my workaround in case you need it.

import Script from "next/script";
import { GoogleTagManager } from "@next/third-parties/google";
import conf from "../conf/config";

export default function Analytics() {
  return process.env.NEXT_PUBLIC_ENVIRONMENT === "production" ? (
    <>
      <Script
        src={`https://www.googletagmanager.com/gtag/js?id=${conf.googleAnalyticsId}`}
        strategy="afterInteractive"
      />
      <Script id="google-analytics" strategy="afterInteractive">
        {`
          window.dataLayer = window.dataLayer || [];
          function gtag(){dataLayer.push(arguments);}
          gtag('js', new Date());

          gtag('config', '${conf.googleAnalyticsId}');
        `}
      </Script>
      <GoogleTagManager gtmId={conf.googleTagManagerId} />
    </>
  ) : null;
}

To avoid errors locally and in the test environment, I have a guard that only includes Google Analytics and Tag Manager when the app runs on production. The <Script> tags are what GA gives you for manual setup, and down below is the GoogleTagManager, which is reduced to a single line if you are using the third-party extension. This <Analytics /> component is included in the root layout instead of GoogleTagManager.

The final result

Hopefully, you did everything by my description, and the UIs haven't updated much since I released this post. 😁

If everything is set up, click some buttons you connected with GTM to Google Analytics, then open the dashboard and check the events arriving in a minute or two. Here is the result in my case.

The landing page of ReadSonic has some call-to-action buttons at the top named Let's hear it and Read more , and it also has some sample voices with names the user can try before registering. You can see in the image above that I pressed these buttons several times when I tested that GTM was working and forwarding events to GA correctly. You don't see here primary-cta and secondary-cta anywhere because I am using the Click Text Variable as the event's name. Remember this part? You can configure the forwarded click events to appear as the innerText of the pressed button in Analytics. You can see it in action above.

Summary

This post won't turn you into The Master of Tag Manager, but that wasn't the goal either. Since the App Router in Next.js is relatively new, and so is GA4, I haven't found any introductory posts about integrating them with GTM. Now we have one.

By following the steps above, you should have a working setup that can track any particular click on your site, and they will appear in your Analytics dashboard.

There's much more to GTM than simple Variables, Triggers, and Tags, but if your requirements are as simple as mine, you don't need a complicated solution. Go with the built-in options and make your life easier.

See you next time.

Three Key Concepts to Build Future-Proof AI Applications

Richard Kovacs — Tue, 04 Jun 2024 07:18:41 +0000

Current AI models won't reach AGI. Not even larger ones. GPT-5, GPT-6, Llama 4, Llama 5, Gemini Ultra Pro Max, it doesn't matter. We are currently living in the age of mixed computing. Deterministic computing is still the dominant type, as the majority of humanity isn't even aware of the capabilities of probabilistic computing, aka Artificial Intelligence.

As the age of commercial chatbots has just started, many feel that current state-of-the-art language models aren't capable enough to remove significant weight from our shoulders. Hallucinations are frequent, calculations are incorrect, and running inference on problems that don't require AI just because it is the buzzword nowadays is expensive compared to running deterministic algorithms.

Current chatbots are simply not powerful enough. However, future models will also be insufficient, as they will just combine and rephrase information from their training set faster and better.

But what if they don't even have to be more powerful? What if I told you that a model today is just enough if you know how to give it some steroids? And these steroids are RAG, fine-tuning, and function calling.

This post is not about the how but the why. I won't go into how to fine-tune a model, embed documents, or add tools to the model's hands because each is a large enough topic to cover in a separate post later. What I want to answer below is the why.

"That's Just an AI Wrapper"

Many indie developers have heard at least one of the following phrases:

"Your product is just a frontend for ChatGPT."
"That's just an AI wrapper."
"Why would I use your product if Llama is free and open source?"

And so on. This post is not about deciding whether an AI wrapper is a valid product. Although there are indeed apps that are really just a better frontend before the OpenAI API, I want to point out a different kind. I am speaking about the apps that offer much more than a better UI and some folders for your prompts. These are the apps that will survive the next OpenAI release or the emergence of a better model.

In fact, these apps will be among the first to utilize it. When an app offers more than a better design that OpenAI will probably ship sometime in the future, it is protected against future releases. When a language model is not the product itself but just a tool to augment something happening in the background, it becomes harder replaceable.

The following points also answer comments from larger companies, such as "Why should I start using ChatGPT? It just steals my company data." or "Llama is our preference since you can self-host it, but it isn't powerful enough for our use cases yet."

Well, it is already powerful enough. You just lack some information about the best ways to augment it. Lucky for you, this post contains exactly what you need.

Fine-Tuning

Depending on how long you have used ChatGPT or any other chatbot, you probably have developed a perfect sense to detect whether it wrote a piece of text or was human-written. By the way, the same is true for image models, but let's stay at language models for now. Another useful test to conduct on raw models is using them in languages other than English. They probably won't be as capable since less information was found on the internet written in that language when the providers trained the model.

But this must not necessarily be the case. With fine-tuning, you can change the default style of the model to suit your needs better. Since I am Hungarian, I have plenty of use cases requiring a fine-tuned model for the Hungarian language. Since it is an extremely rare language (only official in Hungary), the sources on the internet that can be used for training are minimal compared to English. Large providers probably won't care about improving their global models in Hungarian, but they usually offer the ability to fine-tune them for ourselves.

A fine-tuned Hungarian GPT-4 model would probably handle Hungarian questions much better than the base model. The same is true for programming languages. Let's say you want to build a coding assistant for Python programmers. Meta did exactly this with Code Llama. This model will perform much better in answering Python-related questions than the Llama foundation model.

But languages are not the only thing you can fine-tune for. The standard guideline is that if you require a specific style from your model, you should fine-tune it for that style. Some examples are:

Sarcasm
Formal style
Short answers
Viral tweets
Texts without the word "delve"

This is a non-exhaustive list. Of course, you could write these in the system prompt as well, but you would waste countless precious tokens (not to mention the cost) in every message. When you fine-tune a model, it will inherently know the style you want to achieve without further prompting. You achieve better results for less money.

A fine-tuned model is also less susceptible to new model releases. If your application's added value lies in a well-constructed training set, you can easily fine-tune a new model on it and switch to that one when released.

Fine-tuning will help you tailor the style of any model, but it won't extend its knowledge. You need something else for that.

Retrieval Augmented Generation (RAG)

Public chatbots have plenty of knowledge about the world. However, one thing they don't have access to is private documents. Whether they are your private files or the internal files of the company you work for, these files could not have been part of any commercial model's training set because they are inaccessible on the open internet. And unless you don't know about Retrieval Augmented Generation (RAG), you might think that the time of personal and private company assistants is still far away.

With RAG, this hypothesis quickly becomes false. Imagine that you have a bunch of internal software documentation, financial statements, legal documents, design guidelines, and much more in your company that employees frequently use. Since these are internal documents, any commercial chatbot without RAG would be unusable for any question regarding these files. However, you can give the model access to these documents with RAG.

First, you have to embed every document into a vector database. Then, when a user asks something, related sentences from the embedded documents can be retrieved with the help of the same embedding model that was used to embed them. In the next step, these sentences must be injected into the model's context, and voilà, you just extended a foundation model's knowledge with thousands of documents without requiring a larger model or fine-tuning.

Of course, you can combine these if you want. It makes perfect sense to do so. When you have a dataset ready for fine-tuning and a knowledge base embedded in a vector database, the model you use in the background matters less. Let's say you use a fine-tuned GPT-3.5 and have 1000 documents embedded. Then, OpenAI releases GPT-4. Two things can happen in this case:

Either your use case is so unique that the fine-tuned model performs much better than GPT-4 or
You fine-tune the new model on your dataset and switch to it immediately.

In neither case did you have to change your embedding logic since a different model handles that (an embedding model). Also, in neither of the cases did the new model's release pose any danger to your application, regardless of whether it is an internal or a SaaS application.

At this point, hopefully, I could convince you that smaller models with some extensions can be more than enough for a variety of use cases. Also, these use cases can be completely outside the normal capabilities of state-of-the-art foundation models.

However, you might still think that today's models lack a crucial feature: the ability to interact with the outside world. This is what the next section is about.

Function Calling

I first encountered function calling in the OpenAI API, but today, they aren't the only ones offering this capability. In fact, you could also try it yourself with a small model. Just write a prompt that tells the model to return a JSON object that you will use to call a function in the next step.

Yes, OpenAI doesn't really call any function with their models. The way function calling works is that they fine-tuned some of their models to recognize when they face a problem for which a better tool is available. The available tools are functions that you, the developer, wrote and provided the documentation for. When the model decides it is time to call a function for a given task, it will return a specific message containing the function's name to call and its parameters. What you do with that information is up to you, but your implementation will probably pass these parameters to the chosen function. Then, you have to pass the function's response back to the model, based on which it will create the answer.

Here is a simplified example of a message history with function calls.

You ask the model for the price of Bitcoin.
The model has access to a function named get_price(symbol). The model will return a message telling you to call the get_price function with "BTC" as the symbol. It will also give you a unique ID that represents this function call.
You call the function in your application. It returns 64,352. You must add a message to the message history with the unique ID from the previous step and the returned price, 64,352. The model will know from the unique ID that this number answers its previous question.
Based on the previous three messages, the model answers: "The current price of Bitcoin is $64,352."

This is how a typical function calling scenario looks like with a simple tool or function. When the model has access to more tools, it may return multiple tool calls, and your job is to call each function and provide the answers. Note that the model never calls any function. It is your job to do so. What the model is capable of depends on your implementation. You can write your functions with the least possible privileges (as you should), and the model won't cause any trouble.

Putting It All Together

Let's stop for a moment and consider the implications of the above examples. Let's say you are using a simpler model like GPT-3.5. (Crazy how GPT-3.5 is now considered a simpler model, right?) Many more capable models are out there, but you are still using GPT 3.5. It has no access to the internet, has a knowledge cutoff a few months back in the past, speaks too vaguely, and cannot do anything else than provide textual answers. Sounds pretty limited compared to the newest alternatives.

Unless...

We give it some superpowers in the form of the above concepts. I currently have an idea in my pipeline that is ideal for demonstration purposes. It's time to build a financial analyst chatbot.

GPT 3.5 out of the box is pretty far from this dream. My first step was to add some tools in its hand to fetch real-time market information such as the actual price of stocks, dividends, well-known ratios, financial statements, analyst recommendations, etc. I could implement this for free since the yfinance Python module is more than enough for a simple purpose like mine.

At this point, the model could tell from the numbers the actual state of each company. The amount of information available for the model was only dependent on me since the API can handle 128 functions, more than enough for most use cases.

However, one key input of financial analysis was still missing: news. For that, I needed something up-to-date and free for experimentation. I found Tavily to be the perfect candidate because it has access to real-time information from the web, such as news and blog articles. By the way, Tavily uses RAG under the hood.

One last flaw in my application is that the answers are too vague. It doesn't provide specific insights; it just summarizes what it retrieves with the configured functions and RAG. Also, it usually never answers direct questions like "Which stock should I buy out of these two"? This behavior is the result of OpenAI's training. It looks like one of the key points of alignment is that it won't provide financial advice no matter how you ask. Of course, I could write a long system prompt to convince it to answer, but sending it at the start of every conversation would probably cost more in the long run than creating a fine-tuned model that behaves exactly as I desire.

Fine-tuning is still ahead of me, but that is the missing piece in my chatbot. When I have my dataset ready, it won't matter if OpenAI releases a stronger model next week. I can still use Tavily for news, and the new model will still be able to call my functions. I can still fine-tune it with my dataset to fit my needs.

This way, the app will offer much more than just a chat frontend, meaning it will be future-proof for longer than simple API wrappers. Any application utilizing these techniques will quadruple the underlying model's capabilities.

Also, Tavily is just one specific example that is ideal for my use case. It's not the only one and definitely won't suit everyone's needs. There are other providers out there. You could also build a vector database and integrate RAG into your application using your custom records.

Case Studies

The concepts I explained above aren't new. Two excellent examples of apps that utilize at least some of them are Perplexity and Consensus. In my honest opinion, Perplexity nailed the combination of language model answers and web browsing. It uses RAG under the hood and probably also utilizes function calls and fine-tuning.

Consensus is well-known for providing scientific paper references to user questions. It also provides a consensus based on multiple retrieved papers. As you probably have guessed, they also use RAG in the background.

Neither of these apps is in danger of future model releases because their use case is very specific, and both of them offer much more than the underlying model's raw answer.

There is literally no limit to what you can already do with current models if you are creative enough.

Let's build!

Bring Your Own Next.js

Richard Kovacs — Wed, 22 May 2024 20:14:00 +0000

I am a huge fan of Next.js, but I follow a strict personal policy of only using it for the front end — no server actions, no edge functions, nothing else outside TSX. Of course, I had a very good reason to do so: vendor lock-in.

Vendor lock-in is when you are using a specific product that only works on the provider's platform. In my case, the product is Next.js, and the platform is Vercel. When you are using a product like this, your fate is totally in the hands of the provider. They can increase your costs, deprecate features, have regular outages, etc. And if you are dependent on them, taking your stuff somewhere else is extremely painful, if not impossible.

While the frontend part is easy to move elsewhere, I cannot say the same about lambda functions. They work the best on Vercel as Vercel has the architecture ready for it. I have only found one viable alternative so far in the form of AWS Amplify, which takes a quarter of a lifetime to set up correctly compared to deploying to Vercel with a single click.

But even if AWS caught up to Vercel in simplicity, they would be one step behind at each new Next.js release.

What Happened?

My view on the above totally changed recently when I stumbled upon a specific part in the docs about self-hosting Next.js. It isn't brand new; they probably have had it there for ages, but it only caught my attention last week.

Apparently, you have the option to host Next.js yourself. You just need a Node environment or Docker. Of course, the docs mention that you can achieve the best performance if you host your site on Vercel, but I am totally okay with that. Honestly, the best thing Vercel could have done to Next.js is to enable anyone to take their app wherever they prefer, thus eliminating vendor lock-in. An awesome move! Now, let's see how it works in practice.

Hosting Next.js Locally with Node.js

I know the development mode is also running locally on Node, but I am speaking about the production version. Hosting the app locally this way is pretty straightforward. You are actually only two commands away from it: next build and next start.

The first command will build your entire application and put it in the .next folder at the root of your project. If you are interested, the app will be located in the .next/server folder. You will find everything here: frontend, lambda functions, edge functions, and everything your app contains.

The second command will start and continuously serve the app from here.

And that's it. You now have a full-stack Next.js application running on your machine in production mode. You can take this to any provider you prefer. Just build the app and make sure to start it. Only Node.js is required.

However, this is not the only option the docs provide. You have another alternative.

Containerized Next.js

If you are a fan of microservices or a regular user of a provider that supports Docker containers, a containerized Next.js application might better suit your needs. The docs contain steps for this setup as well, although it only works out of the box if you clone their example on GitHub. If you want to dockerize an existing application, follow the guide below.

I chose the Docker Compose example from the official Next.js repository. I prefer Compose because settings are much more readable than in a single-line docker command.

This blog was written in Next.js, and I currently host it on Vercel. The blog posts are written in Markdown, and when you open one of the posts like this one, it is server-side rendered into the HTML you are looking at right now, so this blog was a good candidate for containerization as it contains multiple client-side rendered pages but also an SSR page.

Standalone Output

One of the first steps is to change your output to standalone in your next.config.js.

module.exports = {
  output: 'standalone',
}

This setting ensures that only the necessary files and dependencies are copied to the output folder, making the result much smaller.

Dockerfile

The next step is to obtain a Dockerfile. I used the Dockerfile from the Docker Compose example. You can find plenty of other examples in the repository, such as plain Docker option, Docker with multi-stage and multi-environment setups, etc.

The Dockerfile was almost perfect, but it needed some tweaking to work with this blog's source. The builder step copies the source and some config files into the image, but since I am using Tailwind, I had to add two additional lines below line 22:

COPY postcss.config.js .
COPY tailwind.config.js .

I also had to add another line to this list to copy my markdown pages.

COPY _posts ./_posts

If your application contains other config files and/or folders, make sure that they are copied as well.

I also use shadcn/ui components, but the components.json was not required since that file is only needed when generating a new component from the CLI. The production version does not use it, as the file is not needed during build time.

I also have some environment variables in the repository. I had to add two more lines below line 29 for each required at build time. I had to do the same for each run time variable below line 66. For example, if your app is using EXTERNAL_URL, you have to add the following two lines in the corresponding places:

ARG EXTERNAL_URL
ENV EXTERNAL_URL=${EXTERNAL_URL}

With these modifications, the Dockerfile was ready.

Docker Compose

I started with the Docker Compose present in the example, but it also needed some minor changes. Since I put both the Dockerfile and the docker-compose.yml in the root of the repo, I had to change the context of the build setting to ./. I also had to change the dockerfile parameter to simply Dockerfile.

In the next step I had to define the environment variables. Going with the previous example, if you rely on EXTERNAL_URL, your build step needs the following args:

EXTERNAL_URL: ${EXTERNAL_URL}

Here is how the final compose file looks like:

version: "3"

services:
  next-app:
    container_name: next-app
    build:
      context: ./
      dockerfile: Dockerfile
      args:
        EXTERNAL_URL: ${EXTERNAL_URL}
    restart: always
    ports:
      - 3000:3000
    networks:
      - my_network

networks:
  my_network:
    external: true

How to Start a Containerized Next.js?

If you have everything set up correctly, you can start your containerized application by running docker compose build followed by docker compose up.

⚠️ Remember that this is an optimized production build. Auto restart on file changes won't work, as the goal of this build is to be deployed to a provider.

Caveats

Here are some pitfalls I faced when porting my app to Docker. Pay extra attention to these as fixing them required the most time in my case.

Environment Variables

Make sure that you define your environment variables in every location. Build-time variables should be present in the builder step, and run-time variables should be added to the runner step. Additionally, add them to the compose file. Since the production build is optimized, it won't tell you that this was the problem; it will just throw an error.

Sharp

You will probably see an error saying the sharp package is required in production builds. Next.js uses this package under the hood when you define an Image from the next/image package. However, simply installing it in some cases won't solve the problem as there is an ongoing version mismatch problem.

One option is to define the following variable:

ENV NEXT_SHARP_PATH=/app/node_modules/sharp

If the variable above does not help, ensure your next package is updated to the latest version. Updating next to version 14.2.3 fixed the error, without needing the environment variable. The sharp version that worked was 0.33.3.

Missing Folders, Configs

Double-check that every required folder and config file is copied to the images. As I am using Tailwind, I needed both postcss.config.js and tailwind.config.js. I also had to add my _posts folder. Check which folders and configs your app relies on and copy each.

Standalone Build

Finally, if you are using Docker, ensure the standalone output mode is correctly set in your next.config.js.

Wrapping It Up

Vendor lock-in is a sneaky move, and I'm glad I was wrong about Vercel. Offering the option to take your app to another provider builds trust, which is essential to keeping your users long-term. At least, that's my view on the topic.

With that being said, I am still hosting this blog with them, and I don't plan to change this anytime soon. However, there are other private projects where I need the self-hosted option, which is now fairly easy based on the steps above.

If you still have questions left or want to add something to this post, let me know on any of the social channels I am active on. You can find them down below.