DEV Community: Rajeev R. Sharma

Building Brew Haven: A/B Testing My Coffee Shop Dreams with DevCycle

Rajeev R. Sharma — Tue, 17 Dec 2024 15:54:37 +0000

This is a submission for the DevCycle Feature Flag Challenge: Feature Flag Funhouse

Do you love coffee? As developers, many of us jokingly claim to be "powered by coffee", and the thought of opening a quaint coffee shop someday often lingers in the back of our minds — perhaps as a post-dev career dream.

When brainstorming ideas for a fun project to showcase feature flags, this coffee shop fantasy kept nudging me: "Pick me! Build me!". So, I decided to indulge that thought and create Brew Haven, a dummy coffee shop app that explores the power of feature flags.

What I Built

As you might have guessed, I built a playground to explore the power of feature flags, packaged as a coffee shop app. The app features customizable menus, seasonal items, and dynamic A/B testing for promotions using the DevCycle SDK. It also leverages the DevCycle Management APIs to power an admin panel, giving shop managers full control over these features in real time.

Demo

You can try out the live app here: Brew Haven

The admin page uses a dummy password auth. Use admin@123 passowrd to view the admin panel and play around with the available feature flags.

Video Demo

The following video gives a short walkthrough of the app.

Showcasing Feature Flags in Brew Haven

1. The Coffee Menu

The shop menu is customizable and evolves with the seasons—thanks to feature flags. The Coffee Menu feature flag consists of 3 variables that control:

Showing nutritional info for health-conscious customers.
Enabling the seasonal menu for limited-time offerings.
Allowing order customization for that perfect latte.

You can choose from 3 different variations of the flag:

Basic: Disables all menu variables.
Standard: Allows order customization and shows nutrition info.
Seasonal: Enables all three variables.

2. Payment and Ordering

This feature flag makes the ordering process adaptable. Depending on the selected Payment and Ordering flag variation, admins can:

Allow online payments.
Enable loyalty points redemption.
Enable live order tracking.

This flag also consists of 3 variables to control the above aspects, and the following three variations:

Basic: Disables all the variables.
Standard: Allows online payments and live order tracking.
Premium: Allows loyalty points apart from other standard features.

3. A/B Testing with Promotions

The app also demonstrates A/B testing through a promotions feature flag, which allows admins to:

Offer discounts to a random subset of customers.
Customize promotion details, like discount percentages/amount and minimum cart values to avail the promotions.

This feature flag consists of four variables that allows setting:

The promotion text (string).
Discount value (number).
Discount type (enum with values none, amount, percentage).
Min cart value (number, 0 for no min cart value).

These feature flags made it easy to bring my coffee shop dreams closer to reality, blending fun experimentation with real-world use cases like A/B testing and customizations.

App Screenshots

Home Page

Menu Page

Checkout Page

Admin Page

My Code

The source code of the app is open source, and is available on GitHub.

ra-jeev / brew-haven

Dummy Coffee Shop App with DevCycle Feature Flags

Brew Haven ☕

Project Overview

Brew Haven is a dynamic coffee shop web application that demonstrates the use of feature flags using DevCycle. The app provides a simple customizable menu, checkout experience, and an innovative admin interface for managing feature variations and A/B testing.

Key Features

Home page showcasing the coffee shop
Interactive menu display
Dummy checkout process
Feature flag management through DevCycle
Admin page for feature flag control

Technologies Used

React
Vite
Shadcn UI
DevCycle SDK
Netlify Functions

Prerequisites

Node.js (recommended version 18+)
pnpm package manager
DevCycle account
Netlify account and CLI (for local development and deployment)

Environment Setup

Clone the repository
Install dependencies:

pnpm install

Rename .env.example to .env and update the following environment variables:

VITE_DEVCYCLE_CLIENT_SDK_KEY=your_client_sdk_key
DEVCYCLE_API_CLIENT_ID=your_client_id
DEVCYCLE_API_CLIENT_SECRET=your_client_secret
DEVCYCLE_PROJECT_ID=your_project_id

Local Development

To run the application in development mode:

# For standard development
pnpm dev

# For testing admin page with Netlify Functions
netlify dev

Deployment

The project is…

View on GitHub

Technologies Used

To bring Brew Haven to life, I used the following:

React with Vite for a fast and modular frontend.
Shadcn UI for styling and components.
Netlify Functions for secure serverless DevCycle Management API calls.
DevCycle SDK for feature flag integration on the client side.

Client Side Usage of DevCycle SDK

After adding the DevCycle React SDK dependency, we first initialize the DevCycleProvider in the App.tsx file as shown below:

// src/App.tsx

import { withDevCycleProvider } from "@devcycle/react-client-sdk";
// ...

function App() {
  return (
    <ThemeProvider defaultTheme="system" storageKey="coffee-shop-ui-theme">
      <Router>
        <Layout>
          <Routes>
            <Route path="/" element={<Home />} />
            <Route path="/menu" element={<Menu />} />
            <Route path="/checkout" element={<Checkout />} />
            <Route path="/orders" element={<Orders />} />
            <Route
              path="/admin"
              element={
                <ProtectedRoute>
                  <Admin />
                </ProtectedRoute>
              }
            />
          </Routes>
        </Layout>
      </Router>
      <Toaster />
    </ThemeProvider>
  );
}

export default withDevCycleProvider({
  sdkKey: import.meta.env.VITE_DEVCYCLE_CLIENT_SDK_KEY,
  options: {
    logLevel: "debug",
  },
})(App);

And then we create a useFeatureFlags hook to get the various feature flags variables associated with the app as shown below:

// src/hooks/use-feature-flags.ts

import { useVariableValue } from "@devcycle/react-client-sdk";
import { featureKeys } from "@/lib/consts";

export function useFeatureFlags() {
  const showNutritionInfo = useVariableValue(
    featureKeys.SHOW_NUTRITION_INFO,
    false,
  );
  const enableOnlinePayment = useVariableValue(
    featureKeys.ENABLE_ONLINE_PAYMENT,
    false,
  );
  const showPromotionalBanner = useVariableValue(
    featureKeys.SHOW_PROMOTIONAL_BANNER,
    "",
  );

  // etc.

  return {
    showNutritionInfo,
    enableOnlinePayment,
    showPromotionalBanner,
    // ...
  };
}

Interacting with DevCycle Management APIs

For securely calling the Management APIs, we use Netlify serverless functions so that the DevCycle API's client_id and client_secret are not exposed to the client.

Before we can interact with the DevCycle APIs, we need to generate an auth token using the DevCycle APIs client id and secret. The below code shows how to generate the auth token:

// netlify/functions/feature-flags.mts

interface AuthToken {
  access_token: string;
  expires_in: number;
  token_type: string;
}

let tokenCache: { token: string; expiresAt: number } | null = null;

async function getAuthToken() {
  if (tokenCache && tokenCache.expiresAt > Date.now()) {
    return tokenCache.token;
  }

  try {
    const response = await fetch("https://auth.devcycle.com/oauth/token", {
      method: "POST",
      headers: {
        "Content-Type": "application/x-www-form-urlencoded",
      },
      body: new URLSearchParams({
        grant_type: "client_credentials",
        audience: "https://api.devcycle.com/",
        client_id: process.env.DEVCYCLE_API_CLIENT_ID!,
        client_secret: process.env.DEVCYCLE_API_CLIENT_SECRET!,
      }),
    });

    if (!response.ok) {
      throw new Error(`Auth failed: ${response.status}`);
    }

    const data: AuthToken = await response.json();

    tokenCache = {
      token: data.access_token,
      expiresAt: Date.now() + data.expires_in * 1000 - 5 * 60 * 1000,
    };

    return data.access_token;
  } catch (error) {
    console.error("Failed to get auth token:", error);
    throw error;
  }
}

Now we can fetch the available feature flags, and their config (to get the currently active variation) using the below code:

// netlify/functions/feature-flags.mts

interface Distribution {
  _variation: string;
  percentage: number;
}

interface FeatureConfig {
  _feature: string;
  _environment: string;
  status: string;
  targets: {
    _id: string;
    name: string;
    distribution: Distribution[];
    audience: {
      name: string;
      filters: {
        operator: "and" | "or";
        filters: { type: string }[];
      };
    };
  }[];
}

async function getFeatures(
  featuresUrl: string,
  headers: Record<string, string>,
) {
  const featuresResponse = await fetch(featuresUrl, { headers });

  if (!featuresResponse.ok) {
    throw new Error(`Failed to fetch flags: ${featuresResponse.status}`);
  }

  const featuresData = await featuresResponse.json();

  // Fetch the config for each of the feature flags
  // We're only fetching the configs for the development env
  const configPromises = featuresData.map(async (feature: any) => {
    const configResponse = await fetch(
      `${featuresUrl}/${feature._id}/configurations?environment=development`,
      { headers },
    );

    if (!configResponse.ok) {
      console.error(`Failed to fetch config for feature ${feature._id}`);
      return null;
    }

    const configs: FeatureConfig[] = await configResponse.json();

    return {
      ...feature,
      targets: configs[0].targets,
      status: configs[0].status,
    };
  });

  const featuresWithConfigs = await Promise.all(configPromises);
  return featuresWithConfigs.filter((f) => f !== null);
}

To update the feature flags config (changing the variation, or toggle the flag altogether), we can use the below function:

async function updateFeature(
  featuresUrl: string,
  featureId: string,
  headers: Record<string, string>,
  update: any,
) {
  const response = await fetch(
    `${featuresUrl}/${featureId}/configurations?environment=development`,
    {
      method: "PATCH",
      headers,
      body: JSON.stringify(update),
    },
  );

  if (!response.ok) {
    throw new Error(`Failed to update feature: ${response.status}`);
  }

  return await response.json();
}

Finally, here is the Netlify function that uses the above functions to serve the client requests:

export default async (req: Request) => {
  try {
    const { method } = req;
    const token = await getAuthToken();
    const featuresBaseUrl = `https://api.devcycle.com/v1/projects/${process.env.DEVCYCLE_PROJECT_ID}/features`;
    const headers = {
      Authorization: `Bearer ${token}`,
    };

    if (method === "GET") {
      const features = await getFeatures(featuresBaseUrl, headers);

      return Response.json({ features });
    }

    if (method === "PATCH") {
      const body = await req.json();
      const { featureId, update } = body;

      const data = await updateFeature(
        featuresBaseUrl,
        featureId,
        {
          ...headers,
          "Content-Type": "application/json",
        },
        update,
      );

      return Response.json({ data });
    }

    return new Response(JSON.stringify({ error: "Method not allowed" }), {
      status: 405,
      headers: {
        "Content-Type": "application/json",
      },
    });
  } catch (error) {
    console.error("Error:", error);
    return new Response(
      JSON.stringify({
        error: error instanceof Error ? error.message : "Internal server error",
      }),
      {
        status: 500,
        headers: {
          "Content-Type": "application/json",
        },
      },
    );
  }
};

This Netlify function is called by the client in the following way:

// src/lib/api.ts

export async function getFeatureFlags() {
  try {
    const response = await fetch("/.netlify/functions/feature-flags");
    if (!response.ok) {
      throw new Error("Failed to fetch flags");
    }

    const data = await response.json();
    return { success: true, data: data.features as Feature[] };
  } catch (error) {
    return {
      success: false,
      error: error instanceof Error ? error.message : "Unknown error",
    };
  }
}

// and, so on...

The above code snippets capture how the DevCycle SDK and its Management APIs are used within the app. You can go through the shared source code to view the complete implementation in more detail.

My DevCycle Experience

This was a fun project to build, thanks to the challenge prompt that said something about "fun". After the initial small learning curve of using the DevCycle dashboard — to create feature flags, adding variables and understanding how variations work — things were quite smooth.

The only issue I faced was with the onboarding tutorial app. The DevCycle web console got stuck in some state like "Waiting for the application launch" with the next button disabled. I had to ultimately login from an incognito browser window to get to the dashboard. But, doing this skipped the tutorial, and I had to figure out the tutorial code working (the feature flags creation etc) on my own as the project readme doesn't provide much info on this.

Additional Prize Categories

API All-Star

Wrapping Up

Brew Haven isn’t just a coffee shop app — it’s a showcase of how feature flags can make your projects more dynamic, responsive, and fun.

Feature flags allow you to:

Do gradual rollouts.
Reduce deployment risks.
Enable rapid experimentation.
Personalize user experiences.

Next time you’re sipping coffee and dreaming big, think about how feature flags can brew innovation into your projects. ☕

Until next time.

Keep adding the bits, and soon you'll have a lot of bytes to share with the world.

This is a simple app to capture your voice notes that auto transcribes the recordings, and optionally refines the transcriptions, using AI. This is a detailed guide on how to use Cloudflare Workers AI, along with other services to build this app.

Rajeev R. Sharma — Tue, 03 Dec 2024 05:25:20 +0000

Building Vhisper: Voice Notes App with AI Transcription and Post-Processing

Rajeev R. Sharma ・ Dec 2 '24

#nuxt #cloudflare #ai #webdev

Building Vhisper: Voice Notes App with AI Transcription and Post-Processing

Rajeev R. Sharma — Mon, 02 Dec 2024 14:00:00 +0000

After wrapping up my last project—a chat interface to search GitHub—I found myself searching for the next idea to tackle. As a developer, inspiration often comes unexpectedly, and this time, it struck while scrolling through my GitHub feed. A repo, starred by Daniel Roe (Nuxt Core Team Lead), caught my eye. It was an Electron-based voice notes app designed for macOS.

Something about the simplicity of voice notes combined with the technical challenge intrigued me. Could I take this concept further? Could I build a modern, AI-powered voice notes app using web technologies? That urge to build led me here, to this blog post, where I’ll walk you through building Vhisper, a voice notes app with AI transcription and post-processing, built with the Nuxt ecosystem and powered by Cloudflare.

And before you say it, I must make a confession: “Hi! My name is Rajeev, and I am addicted to talking/chatting.”.

Project Overview

Now that the formalities are done, let’s focus on what we’ll be building in this project. The goal is to create Vhisper, a web-based voice notes application with the following core features:

Recording Voice Notes: Users can record voice notes directly in the browser.
AI-Powered Transcription: Each recording is processed via Cloudflare Workers AI, converting speech to text.
Post-Processing with Custom Prompts: Users can customize how transcriptions are refined using an AI-driven post-processing step.
Seamless Data Management (CRUD): Notes and audio files are efficiently stored using Cloudflare’s D1 database and R2 storage.

To give you a better sense of what we’re aiming for, here’s a quick demo showcasing Vhisper’s main features:

You can experience it live here: https://vhisper.nuxt.dev

By the end of this guide, you’ll know exactly how to build and deploy this voice notes app using Nuxt, NuxtHub and Cloudflare services—a stack that combines innovation with developer-first simplicity. Ready to build it? Let’s get started!

Project Setup

Before setting up the project let’s review the technologies used to build this app:

Nuxt: Vue.js framework for the application foundation
Nuxt UI (v3): For creating a polished and professional frontend
Drizzle: Database ORM
Zod: For client/server side data validation
NuxtHub: Backend (database, storage, AI etc.), deployment and administration platform for Nuxt
Cloudflare: Powers NuxtHub to provide various services

Prerequisites

To follow along, apart from basic necessities like Node.js, npm, and some Nuxt knowledge, you’ll need:

A Cloudflare account to use Workers AI and deploy your project. If you don’t have one, you can set it up here.
A NuxtHub Admin Account for managing apps via the NuxtHub dashboard. Sign up here.

ℹ Note: Workers AI models will run in your Cloudflare account even during local development. Check out their pricing and free quota.

Project Init

We’ll start with the NuxtHub starter template. Run the following command to create and navigate to your new project directory:

# Create project and change into the project dir
npx nuxthub init voice-notes && cd $_

If you plan to use pnpm as your package manager, add a .npmrc file at the root of your project with this line to hoist dependencies:

# .npmrc
shamefully-hoist=true

Now, install the dependencies:

Nuxt modules:
```
pnpm add @nuxt/ui@next
```

Drizzle and related tools:

pnpm add drizzle-orm drizzle-zod @vueuse/core

Icon packs:

pnpm add @iconify-json/lucide @iconify-json/simple-icons

Dev dependencies:
```
pnpm add -D drizzle-kit
```

Update your nuxt.config.ts file as follows:

export default defineNuxtConfig({
  modules: ["@nuxthub/core", "@nuxt/eslint", "nuxt-auth-utils", "@nuxt/ui"],

  devtools: { enabled: true },

  runtimeConfig: {
    public: {
      helloText: "Hello from the Edge 👋",
    },
  },

  future: { compatibilityVersion: 4 },
  compatibilityDate: "2024-07-30",

  hub: {
    ai: true,
    blob: true,
    database: true,
  },

  css: ["~/assets/css/main.css"],

  eslint: {
    config: {
      stylistic: false,
    },
  },
});

We’ve made the following changes to the Nuxt config file:

Updated the Nuxt modules used in the app
Enabled required NuxtHub features
And, added the main.css file path.

Create the main.css file in the app/assets/css folder with this content:

@import "tailwindcss";
@import "@nuxt/ui";

Testing the Setup

Run the development server:

pnpm dev

Visit http://localhost:3000 in your browser. If everything is set up correctly, you’ll see the message: “Hello from the Edge 👋” with a refresh button.

💡 Troubleshooting Tip: If you encounter issues with TailwindCSS, try deleting node_modules and pnpm-lock.yaml, and then run pnpm install to re-install the dependecies.

Building the Basic Backend

With the project setup complete, let’s dive into building the backend. We’ll begin by creating API endpoints to handle core functionalities, followed by configuring the database and integrating validation.

But before jumping to code, let’s understand how you’ll interact with various Cloudflare offerings. If you’ve been attentive, you should know the answer, NuxrHub, but what is NuxtHub?

What is NuxtHub?

NuxtHub is a developer-friendly interface built on top of Cloudflare’s robust services. It simplifies the process of creating, binding, and managing services for your project, offering a seamless development experience (DX).

You started with a NuxtHub template, so the project comes preconfigured with the @nuxthub/core module. During the setup, you also enabled the required Cloudflare services: AI, Database, and Blob. The NuxtHub core module exposes these services through interfaces prefixed with hub. For example, hubAI is used for AI features, hubBlob for object storage, and so on.

Time is ripe now to work on the first API endpoint.

`/api/transcribe` Endpoint

Create a new file named transcribe.post.ts inside the server/api directory, and add the following code to it:

// server/api/transcribe.post.ts 
export default defineEventHandler(async (event) => {
  const form = await readFormData(event);
  const blob = form.get("audio") as Blob;
  if (!blob) {
    throw createError({
      statusCode: 400,
      message: "Missing audio blob to transcribe",
    });
  }

  ensureBlob(blob, { maxSize: "8MB", types: ["audio"] });

  try {
    const response = await hubAI().run("@cf/openai/whisper", {
      audio: [...new Uint8Array(await blob.arrayBuffer())],
    });

    return response.text;
  } catch (err) {
    console.error("Error transcribing audio:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to transcribe audio. Please try again.",
    });
  }
});

The above code does the following:

Parses incoming form data to extract the audio as a Blob
Verifies that it’s an audio blob and is less than 8MB in size using a @nuxthub/core utility function ensureBlob
Passes on the array buffer to the Whisper model through hubAI for transcription
Returns the transcribed text to the client

Before you can use Workers AI in development, you’ll need to link it to your Cloudflare project. As we’re using NuxtHub as the interface, running the following command will create/link a new or existing NuxtHub project with this project.

npx nuxthub link

`/api/upload` Endpoint

Next, create an endpoint to upload the audio recordings to the R2 storage. Create a new file upload.put.ts in your /server/api folder and add the following code to it:

// server/api/upload.put.ts
export default defineEventHandler(async (event) => {
  return hubBlob().handleUpload(event, {
    formKey: "files",
    multiple: true,
    ensure: {
      maxSize: "8MB",
      types: ["audio"],
    },
    put: {
      addRandomSuffix: true,
      prefix: "recordings",
    },
  });
});

The above code uses another utility method from the NuxtHub core module to upload the incoming audio files to R2. handleUpload does the following:

Looks for the files key in the incoming form data to extract blob data
Supports multiple files per event
Ensures that the files are audio and under 8MB in size
And, finally uploads them to your R2 bucket inside recordings folder while also adding a random suffix to the final names
Returns a promise to the client that resolves once all the files are uploaded

Now we just need /notes endpoints to create & fetch notes entries before the basic backend is done. But to do that we need to create the needed tables. Let’s tackle this in next section.

Defining the `notes` Table Schema

As we will use drizzle to manage and interact with the database, we need to configure it first. Create a new file drizzle.config.ts in the project root, and add the following to it:

// drizzle.config.ts
import { defineConfig } from 'drizzle-kit';

export default defineConfig({
  dialect: 'sqlite',
  schema: './server/database/schema.ts',
  out: './server/database/migrations',
});

The config above mentions where the database schema is located, and where should the database migrations be generated. The database dialect is set to sqlite as that is what Cloudflare’s D1 database supports.

Next, create a new file schema.ts in the server/database folder, and add the following to it:

// server/database/schema.ts
import crypto from "node:crypto";
import { sql } from "drizzle-orm";
import { sqliteTable, text } from "drizzle-orm/sqlite-core";

export const notes = sqliteTable("notes", {
  id: text("id")
    .primaryKey()
    .$defaultFn(() => "nt_" + crypto.randomBytes(12).toString("hex")),
  text: text("text").notNull(),
  createdAt: text("created_at")
    .notNull()
    .default(sql`(CURRENT_TIMESTAMP)`),
  updatedAt: text("updated_at")
    .notNull()
    .default(sql`(CURRENT_TIMESTAMP)`)
    .$onUpdate(() => sql`(CURRENT_TIMESTAMP)`),
  audioUrls: text("audio_urls", { mode: "json" }).$type<string[]>(),
});

The notes table schema is straightforward. It includes the note text and optional audio recording URLs stored as a JSON string array.

Finally, create a new file drizzle.ts in the server/utils folder, and add the following to it:

// server/utils/drizzle.ts
import { drizzle } from "drizzle-orm/d1";
import * as schema from "../database/schema";

export { sql, eq, and, or, desc } from "drizzle-orm";

export const tables = schema;

export function useDrizzle() {
  return drizzle(hubDatabase(), { schema });
}

Here we hook up hubDatabase with the tables schema through drizzle and export the server composable useDrizzle along with the needed operators.

Now we are ready to create the /api/notes endpoints which we will be doing in the next section.

`/api/notes` Endpoints

Create two new files index.post.ts and index.get.ts in the server/api/notes folder and add the respective codes to them as shown below.

index.post.ts

// server/api/notes/index.post.ts
import { noteSchema } from "#shared/schemas/note.schema";

export default defineEventHandler(async (event) => {
  const { user } = await requireUserSession(event);

  const { text, audioUrls } = await readValidatedBody(event, noteSchema.parse);

  try {
    await useDrizzle()
      .insert(tables.notes)
      .values({
        text,
        audioUrls: audioUrls ? audioUrls.map((url) => `/audio/${url}`) : null,
      });

    return setResponseStatus(event, 201);
  } catch (err) {
    console.error("Error creating note:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to create note. Please try again.",
    });
  }
});

The above code reads the validated event body, and creates a new note entry in the database using the drizzle composable we created earlier. We will get to the validation part in a bit.

index.get.ts

// server/api/notes/index.get.ts
export default defineEventHandler(async (event) => {
  try {
    const notes = await useDrizzle()
      .select()
      .from(tables.notes)
      .orderBy(desc(tables.notes.updatedAt));

    return notes;
  } catch (err) {
    console.error("Error retrieving note:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to get notes. Please try again.",
    });
  }
});

Here we fetch the notes entries from the table in descending order of updatedAt field.

Incoming data validation

As mentioned in the beginning, we’ll use Zod for data validation. Here is the relevant code from index.post.ts that validates the incoming client data.

const { text, audioUrls } = await readValidatedBody(event, noteSchema.parse);

Create a new file note.schema.ts in the shared/schemas folder in the project root directory with the following content:

// shared/schemas/note.schema.ts
import { createInsertSchema, createSelectSchema } from "drizzle-zod";
import { z } from "zod";
import { notes } from "~~/server/database/schema";

export const noteSchema = createInsertSchema(notes, {
  text: (schema) =>
    schema.text
      .min(3, "Note must be at least 3 characters long")
      .max(5000, "Note cannot exceed 5000 characters"),
  audioUrls: z.string().array().optional(),
}).pick({
  text: true,
  audioUrls: true,
});

export const noteSelectSchema = createSelectSchema(notes, {
  audioUrls: z.string().array().optional(),
});

The above code uses the drizzle-zod plugin to create the zod schema needed for validation (The above validation error messages are more suitable for the client side. Feel free to adapt these validation rules to suit your specific project requirements.).

Creating DB Migrations

With the table schema and API endpoints defined, the final step is to create and apply database migrations to bring everything together. Add the following command to your package.json's scripts:

// ..
"scripts": {
  // ..
  "db:generate": "drizzle-kit generate"
}
// ..

Next, run pnpm run db:generate to create the database migrations. These migrations are auto applied by NuxtHub when you run or deploy your project. You can test it by running pnpm dev and checking the Nuxt Dev Tools as shown below (this is a local sqlite database that is used in the dev mode).

We are done with the basic backend of the project. In the next section, we will code the frontend components and pages to complete the whole thing,

Creating the Basic Frontend

We’ll start with the most important feature first: recording the user voice, and then we’ll move on to creating the needed components and pages.

`useMediaRecorder` Composable

Let’s create a composable to handle the media recording functionality. Create a new file useMediaRecorder.ts in your app/composables folder and add the following code to it:

// app/composables/useMediaRecorder.ts
interface MediaRecorderState {
  isRecording: boolean;
  recordingDuration: number;
  audioData: Uint8Array | null;
  updateTrigger: number;
}

const getSupportedMimeType = () => {
  const types = [
    "audio/mp4",
    "audio/mp4;codecs=mp4a",
    "audio/mpeg",
    "audio/webm;codecs=opus",
    "audio/webm",
  ];

  return (
    types.find((type) => MediaRecorder.isTypeSupported(type)) || "audio/webm"
  );
};

export function useMediaRecorder() {
  const state = ref<MediaRecorderState>({
    isRecording: false,
    recordingDuration: 0,
    audioData: null,
    updateTrigger: 0,
  });

  let mediaRecorder: MediaRecorder | null = null;
  let audioContext: AudioContext | null = null;
  let analyser: AnalyserNode | null = null;
  let animationFrame: number | null = null;
  let audioChunks: Blob[] | undefined = undefined;

  const updateAudioData = () => {
    if (!analyser || !state.value.isRecording || !state.value.audioData) {
      if (animationFrame) {
        cancelAnimationFrame(animationFrame);
        animationFrame = null;
      }

      return;
    }

    analyser.getByteTimeDomainData(state.value.audioData);
    state.value.updateTrigger += 1;
    animationFrame = requestAnimationFrame(updateAudioData);
  };

  const startRecording = async () => {
    try {
      const stream = await navigator.mediaDevices.getUserMedia({ audio: true });

      audioContext = new AudioContext();
      analyser = audioContext.createAnalyser();

      const source = audioContext.createMediaStreamSource(stream);
      source.connect(analyser);

      const options = {
        mimeType: getSupportedMimeType(),
        audioBitsPerSecond: 64000,
      };

      mediaRecorder = new MediaRecorder(stream, options);
      audioChunks = [];

      mediaRecorder.ondataavailable = (e: BlobEvent) => {
        audioChunks?.push(e.data);
        state.value.recordingDuration += 1;
      };

      state.value.audioData = new Uint8Array(analyser.frequencyBinCount);
      state.value.isRecording = true;
      state.value.recordingDuration = 0;
      state.value.updateTrigger = 0;
      mediaRecorder.start(1000);

      updateAudioData();
    } catch (err) {
      console.error("Error accessing microphone:", err);
      throw err;
    }
  };

  const stopRecording = async () => {
    return await new Promise<Blob>((resolve) => {
      if (mediaRecorder && state.value.isRecording) {
        const mimeType = mediaRecorder.mimeType;
        mediaRecorder.onstop = () => {
          const blob = new Blob(audioChunks, { type: mimeType });
          audioChunks = undefined;

          state.value.recordingDuration = 0;
          state.value.updateTrigger = 0;
          state.value.audioData = null;

          resolve(blob);
        };

        state.value.isRecording = false;
        mediaRecorder.stop();
        mediaRecorder.stream.getTracks().forEach((track) => track.stop());

        if (animationFrame) {
          cancelAnimationFrame(animationFrame);
          animationFrame = null;
        }

        audioContext?.close();
        audioContext = null;
      }
    });
  };

  onUnmounted(() => {
    stopRecording();
  });

  return {
    state: readonly(state),
    startRecording,
    stopRecording,
  };
}

The above code does the following:

Exposes recording start/stop functionality along with the current recording readonly state
Captures user’s voice using the MediaRecorder API when startRecording function is invoked. The MediaRecorder API is a simple and efficient way to handle media capture in modern browsers, making it ideal for our use case.
Captures audio visualization data using AudioContext and AnalyserNode and updates it in real-time using animation frames
Cleans up resources and returns the captured audio as a Blob when stopRecording is called or if the component unmounts

`NoteEditorModal` Component

Next, create a new file NoteEditorModal.vue in the app/components folder and add the following code to it:

<!-- app/components/NoteEditorModal.vue -->
<template>
  <UModal
    fullscreen
    :close="{
      disabled: isSaving || noteRecorder?.isBusy,
    }"
    :prevent-close="isSaving || noteRecorder?.isBusy"
    title="Create Note"
    :ui="{
      body: 'flex-1 w-full max-w-7xl mx-auto flex flex-col md:flex-row gap-4 sm:gap-6 overflow-hidden',
    }"
  >
    <template #body>
      <UCard class="flex-1 flex flex-col" :ui="{ body: 'flex-1' }">
        <template #header>
          <h3 class="h-8 font-medium text-gray-600 dark:text-gray-300">
            Note transcript
          </h3>
        </template>

        <UTextarea
          v-model="noteText"
          placeholder="Type your note here, or use voice recording..."
          size="lg"
          :disabled="isSaving || noteRecorder?.isBusy"
          :ui="{ root: 'w-full h-full', base: ['h-full resize-none'] }"
        />
      </UCard>

      <NoteRecorder
        ref="recorder"
        class="md:h-full md:flex md:flex-col md:w-96 shrink-0 order-first md:order-none"
        @transcription="handleTranscription"
      />
    </template>

    <template #footer>
      <UButton
        icon="i-lucide-undo-2"
        color="neutral"
        variant="outline"
        :disabled="isSaving"
        @click="resetNote"
      >
        Reset
      </UButton>

      <UButton
        icon="i-lucide-cloud-upload"
        :disabled="!noteText.trim() || noteRecorder?.isBusy || isSaving"
        :loading="isSaving"
        @click="saveNote"
      >
        Save Note
      </UButton>
    </template>
  </UModal>
</template>

<script setup lang="ts">
import { NoteRecorder } from "#components";

const props = defineProps<{ onNewNote: () => void }>();

type NoteRecorderType = InstanceType<typeof NoteRecorder>;
const noteRecorder = useTemplateRef<NoteRecorderType>("recorder");
const resetNote = () => {
  noteText.value = "";
  noteRecorder.value?.resetRecordings();
};

const noteText = ref("");
const handleTranscription = (text: string) => {
  noteText.value += noteText.value ? "\n\n" : "";
  noteText.value += text ?? "";
};

const modal = useModal();
const isSaving = ref(false);
const saveNote = async () => {
  const text = noteText.value.trim();
  if (!text) return;

  isSaving.value = true;

  const audioUrls = await noteRecorder.value?.uploadRecordings();

  try {
    await $fetch("/api/notes", {
      method: "POST",
      body: { text, audioUrls },
    });

    useToast().add({
      title: "Note Saved",
      description: "Your note was saved successfully.",
      color: "success",
    });

    if (props.onNewNote) {
      props.onNewNote();
    }

    modal.close();
  } catch (err) {
    console.error("Error saving note:", err);
    useToast().add({
      title: "Save Failed",
      description: "Failed to save the note.",
      color: "error",
    });
  }

  isSaving.value = false;
};
</script>

The above modal component does the following:

Displays a textarea for allowing a manual note entry
The modal integrates the NoteRecorder component for voice recordings and manages the data flow between the recordings and the textarea for user notes.
Whenever a new recording is created, it captures the emitted event from the note recorder component, and appends the transcription text to the textarea content
When the user clicks the save note button, its first uploads all recordings (if any) by calling the note recorder’s uploadRecordings method, and then save the note by calling the notes API endpoint created earlier.
The save note button first uploads all recordings (if any) asynchronously by calling the uploadRecordings method, then sends the note data to the /api/notes endpoint. Upon success, it notifies the parent by executing the callback passed by it, and then closes the modal.

`NoteRecorder` Component

Create a new file NoteRecorder.vue in the app/components folder and add the following content to it:

<!-- app/components/NoteRecorder.vue --> 
<template>
  <UCard
    :ui="{
      body: 'max-h-36 md:max-h-none md:flex-1 overflow-y-auto',
    }"
  >
    <template #header>
      <h3 class="font-medium text-gray-600 dark:text-gray-300">Recordings</h3>

      <div class="flex items-center gap-x-2">
        <template v-if="state.isRecording">
          <div class="w-2 h-2 rounded-full bg-red-500 animate-pulse" />
          <span class="mr-2 text-sm">
            {{ formatDuration(state.recordingDuration) }}
          </span>
        </template>

        <UButton
          :icon="state.isRecording ? 'i-lucide-circle-stop' : 'i-lucide-mic'"
          :color="state.isRecording ? 'error' : 'primary'"
          :loading="isTranscribing"
          @click="toggleRecording"
        />
      </div>
    </template>

    <AudioVisualizer
      v-if="state.isRecording"
      class="w-full h-14 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg mb-2"
      :audio-data="state.audioData"
      :data-update-trigger="state.updateTrigger"
    />

    <div
      v-else-if="isTranscribing"
      class="flex items-center justify-center h-14 gap-x-3 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg mb-2 text-gray-500 dark:text-gray-400"
    >
      <UIcon name="i-lucide-refresh-cw" size="size-6" class="animate-spin" />
      Transcribing...
    </div>

    <div class="space-y-2">
      <div
        v-for="recording in recordings"
        :key="recording.id"
        class="flex items-center gap-x-3 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg"
      >
        <audio :src="recording.url" controls class="w-full h-10" />

        <UButton
          icon="i-lucide-trash-2"
          color="error"
          variant="ghost"
          size="sm"
          @click="removeRecording(recording)"
        />
      </div>
    </div>

    <div
      v-if="!recordings.length && !state.isRecording && !isTranscribing"
      class="h-full flex flex-col items-center justify-center text-gray-500 dark:text-gray-400"
    >
      <p>No recordings...!</p>
      <p class="text-sm mt-1">Tap the mic icon to create one.</p>
    </div>
  </UCard>
</template>

<script setup lang="ts">
const emit = defineEmits<{ transcription: [text: string] }>();

const { state, startRecording, stopRecording } = useMediaRecorder();
const toggleRecording = () => {
  if (state.value.isRecording) {
    handleRecordingStop();
  } else {
    handleRecordingStart();
  }
};

const handleRecordingStart = async () => {
  try {
    await startRecording();
  } catch (err) {
    console.error("Error accessing microphone:", err);
    useToast().add({
      title: "Error",
      description: "Could not access microphone. Please check permissions.",
      color: "error",
    });
  }
};

const { recordings, addRecording, removeRecording, resetRecordings } =
  useRecordings();

const handleRecordingStop = async () => {
  let blob: Blob | undefined;

  try {
    blob = await stopRecording();
  } catch (err) {
    console.error("Error stopping recording:", err);
    useToast().add({
      title: "Error",
      description: "Failed to record audio. Please try again.",
      color: "error",
    });
  }

  if (blob) {
    try {
      const transcription = await transcribeAudio(blob);

      if (transcription) {
        emit("transcription", transcription);

        addRecording({
          url: URL.createObjectURL(blob),
          blob,
          id: `${Date.now()}`,
        });
      }
    } catch (err) {
      console.error("Error transcribing audio:", err);
      useToast().add({
        title: "Error",
        description: "Failed to transcribe audio. Please try again.",
        color: "error",
      });
    }
  }
};

const isTranscribing = ref(false);
const transcribeAudio = async (blob: Blob) => {
  try {
    isTranscribing.value = true;
    const formData = new FormData();
    formData.append("audio", blob);

    return await $fetch("/api/transcribe", {
      method: "POST",
      body: formData,
    });
  } finally {
    isTranscribing.value = false;
  }
};

const uploadRecordings = async () => {
  if (!recordings.value.length) return;

  const formData = new FormData();
  recordings.value.forEach((recording) => {
    if (recording.blob) {
      formData.append(
        "files",
        recording.blob,
        `${recording.id}.${recording.blob.type.split("/")[1]}`,
      );
    }
  });

  try {
    const result = await $fetch("/api/upload", {
      method: "PUT",
      body: formData,
    });

    return result.map((obj) => obj.pathname);
  } catch (error) {
    console.error("Failed to upload audio recordings", error);
  }
};

const isBusy = computed(() => state.value.isRecording || isTranscribing.value);

defineExpose({ uploadRecordings, resetRecordings, isBusy });

const formatDuration = (seconds: number) => {
  const mins = Math.floor(seconds / 60);
  const secs = seconds % 60;
  return `${mins}:${secs.toString().padStart(2, "0")}`;
};
</script>

This component does the following:

Allows recording the user’s voice with the help of useMediaRecorder composable created earlier. It also integrates the AudioVisualizer component to enhance the user experience by providing real-time audio feedback during recordings.
On a new recording, sends the recorded blob for transcription to the transcribe API endpoint, and emits the transcription text on success
Displays all recordings as audio elements for users perusal (using URL.createObjectURL(blob)). It utilizes the useRecordings composable to manage the recordings
Uploads the final recordings to R2 (the local disk in dev mode) using the /api/upload endpoint, and returns the pathnames of these recordings to the caller (the NoteEditorModal component)

`AudioVisualizer` Component

This component uses an HTML canvas element to represent the audio waveform along a horizontal line. The canvas element is used for its flexibility and efficiency in rendering real-time visualizations, making it suitable for audio waveforms.

The visualization dynamically adjusts based on the amplitude of the captured audio, providing a real-time feedback loop for the user during recording. To do that, it watches the updateTrigger state variable exposed by useMediaRecorder to redraw the canvas on audio data changes.

Create a new file AudioVisualizer.vue in the app/components folder and add the following code to it:

<!-- app/components/AudioVisualizer.vue -->
<template>
  <canvas ref="canvas" width="640" height="100" />
</template>

<script setup lang="ts">
const props = defineProps<{
  audioData: Uint8Array | null;
  dataUpdateTrigger: number;
}>();

let width = 0;
let height = 0;
const audioCanvas = useTemplateRef<HTMLCanvasElement>("canvas");
const canvasCtx = ref<CanvasRenderingContext2D | null>(null);

onMounted(() => {
  if (audioCanvas.value) {
    canvasCtx.value = audioCanvas.value.getContext("2d");
    width = audioCanvas.value.width;
    height = audioCanvas.value.height;
  }
});

const drawCanvas = () => {
  if (!canvasCtx.value || !props.audioData) {
    return;
  }

  const data = props.audioData;
  const ctx = canvasCtx.value;
  const sliceWidth = width / data.length;

  ctx.clearRect(0, 0, width, height);
  ctx.lineWidth = 2;
  ctx.strokeStyle = "rgb(221, 72, 49)";
  ctx.beginPath();

  let x = 0;
  for (let i = 0; i < data.length; i++) {
    const v = (data[i] ?? 0) / 128.0;
    const y = (v * height) / 2;

    if (i === 0) {
      ctx.moveTo(x, y);
    } else {
      ctx.lineTo(x, y);
    }

    x += sliceWidth;
  }

  ctx.lineTo(width, height / 2);
  ctx.stroke();
};

watch(
  () => props.dataUpdateTrigger,
  () => {
    drawCanvas();
  },
  { immediate: true },
);
</script>

`useRecordings` Composable

The NoteRecorder component uses the useRecordings composable to manage the list of recordings, and to clear any used resources. Create a new file useRecordings.ts in the app/composables folder and add the following code to it:

// app/composables/useRecordings.ts
export const useRecordings = () => {
  const recordings = ref<Recording[]>([]);

  const cleanupResource = (recording: Recording) => {
    if (recording.blob) {
      URL.revokeObjectURL(recording.url);
    }
  };

  const cleanupResources = () => {
    recordings.value.forEach((recording) => {
      cleanupResource(recording);
    });
  };

  const addRecording = (recording: Recording) => {
    recordings.value.unshift(recording);
  };

  const removeRecording = (recording: Recording) => {
    recordings.value = recordings.value.filter((r) => r.id !== recording.id);
    cleanupResource(recording);
  };

  const resetRecordings = () => {
    cleanupResources();

    recordings.value = [];
  };

  onUnmounted(cleanupResources);

  return {
    recordings,
    addRecording,
    removeRecording,
    resetRecordings,
  };
};

You can define the Recording type definition in the shared/types/index.ts file. This allows for auto import of type definitions in both client & server sides (The intended purpose of the shared folder is for sharing common types & utils between the app & server). Also, while you’re at it, you can also define the Note type.

// shared/types/index.ts
import type { z } from "zod";
import type { noteSelectSchema } from "#shared/schemas/note.schema";

export type Recording = {
  url: string;
  blob?: Blob;
  id: string;
};

export type Note = z.output<typeof noteSelectSchema>;

Creating the Home Page

Now that we have all the pieces ready for the basic app, it is time to put everything together in a page. Delete the content of the home page (app/pages/index.vue), and put the following content to it:

<!-- app/pages/index.vue -->
<template>
  <UContainer class="h-screen flex justify-center items-center">
    <UCard
      class="w-full max-h-full overflow-hidden max-w-4xl mx-auto"
      :ui="{ body: 'h-[calc(100vh-4rem)] overflow-y-auto' }"
    >
      <template #header>
        <span class="font-bold text-xl md:text-2xl">Voice Notes</span>
        <UButton icon="i-lucide-plus" @click="showNoteModal">
          New Note
        </UButton>
      </template>

      <div v-if="notes?.length" class="space-y-4">
        <NoteCard v-for="note in notes" :key="note.id" :note="note" />
      </div>
      <div
        v-else
        class="my-12 text-center text-gray-500 dark:text-gray-400 space-y-2"
      >
        <h2 class="text-2xl md:text-3xl">No notes created</h2>
        <p>Get started by creating your first note</p>
      </div>
    </UCard>
  </UContainer>
</template>

<script setup lang="ts">
import { LazyNoteEditorModal } from "#components";

const { data: notes, refresh } = await useFetch("/api/notes");

const modal = useModal();
const showNoteModal = () => {
  modal.open(LazyNoteEditorModal, {
    onNewNote: refresh,
  });
};

watch(modal.isOpen, (newState) => {
  if (!newState) {
    modal.reset();
  }
});
</script>

On this page we’re doing the following:

Fetch the list of existing notes from the database and display them using the NoteCard component
Shows a new note button which when clicked opens the NoteEditorModal. On successful note creation the refresh function is called to refetch the notes
The modal state is reset on closure to ensure a clean slate for the next note creation

The cards and modals headers/footers used in the app follow a global style that is defined in the app config file. Centralizing styles in the app configuration ensures consistent theming and reduces redundancy across components.

Create a new file app.config.ts inside the app folder, and add the following to it:

// app/app.config.ts
export default defineAppConfig({
  ui: {
    card: {
      slots: {
        header: "flex items-center justify-between gap-3 flex-wrap",
      },
    },
    modal: {
      slots: {
        footer: "justify-end gap-x-3",
      },
    },
  },
});

You’ll also need to wrap your NuxtPage component with the UApp component for the modals and toast notifications to work as shown below:

<!-- app/app.vue -->
<template>
  <NuxtRouteAnnouncer />
  <NuxtLoadingIndicator />
  <UApp>
    <NuxtPage />
  </UApp>
</template>

`NoteCard` component

This component displays the note text and the attached audio recordings of a note. The note text is clamped to 3 lines with a show more/less button to show/hide rest of the text. Text clamping ensures that the UI remains clean and uncluttered, while the show more/less button gives users full control over note visibility.

Create a new file NoteCard.vue in the app/components folder, and add the following code to it:

<template>
  <UCard class="hover:shadow-lg transition-shadow">
    <div class="flex-1">
      <p
        ref="text"
        :class="['whitespace-pre-wrap', !showFullText && 'line-clamp-3']"
      >
        {{ note.text }}
      </p>
      <UButton
        v-if="shouldShowExpandBtn"
        variant="link"
        :padded="false"
        @click="showFullText = !showFullText"
      >
        {{ showFullText ? "Show less" : "Show more" }}
      </UButton>
    </div>

    <div
      v-if="note.audioUrls && note.audioUrls.length > 0"
      class="mt-4 flex gap-x-2 overflow-x-auto"
    >
      <audio
        v-for="url in note.audioUrls"
        :key="url"
        :src="url"
        controls
        class="w-60 shrink-0 h-10"
      />
    </div>

    <p
      class="flex items-center text-sm text-gray-500 dark:text-gray-400 gap-x-2 mt-6"
    >
      <UIcon name="i-lucide-clock" size="size-4" />
      <span>
        {{
          note.updatedAt && note.updatedAt !== note.createdAt
            ? `Updated ${updated}`
            : `Created ${created}`
        }}
      </span>
    </p>
  </UCard>
</template>

<script setup lang="ts">
import { useTimeAgo } from "@vueuse/core";

const props = defineProps<{ note: Note }>();

const createdAt = computed(() => props.note.createdAt + "Z");
const updatedAt = computed(() => props.note.updatedAt + "Z");

const created = useTimeAgo(createdAt);
const updated = useTimeAgo(updatedAt);

const showFullText = ref(false);

const shouldShowExpandBtn = ref(false);
const noteText = useTemplateRef<HTMLParagraphElement>("text");
const checkTextExpansion = () => {
  nextTick(() => {
    if (noteText.value) {
      shouldShowExpandBtn.value =
        noteText.value.scrollHeight > noteText.value.clientHeight;
    }
  });
};

onMounted(checkTextExpansion);

watch(() => props.note.text, checkTextExpansion);
</script>

And we are done here. Try running the application and create some notes. You should be able to create notes, add multiple recordings to the same note etc. Everything should be working now, or is it?

Try playing the audio recordings of the saved notes, are these playable?

Serving the Audio Recordings

We can’t play the audio recordings because these are saved in R2 (local disk in dev mode), and nowhere we are serving these files. It is time to fix that.

If you look at the /api/notes code, we save the audio urls/pathnames with an audio prefix

await useDrizzle()
  .insert(tables.notes)
  .values({
    text,
    audioUrls: audioUrls ? audioUrls.map((url) => `/audio/${url}`) : null,
  });

The reason to do so was to serve all audio recordings through an /audio path. Create a new file […pathname].get.ts in the server/routes/audio folder and add the following to it:

export default defineEventHandler(async (event) => {
  const { pathname } = getRouterParams(event);

  return hubBlob().serve(event, pathname);
});

What we’ve done above is to catch all requests to the /audio path (by using the wildcard […pathname] in the filename), and serve the requested recording from the storage using hubBlob.

With this, the frontend is complete, and all functionalities should now work seamlessly.

Further Enhancements

What you’ve created here is a basic version of the application—with all must-have features—that you saw in the beginning of the article. You can further refine the app and take it closer to the demo by:

What you’ve created here is a solid foundation for the application, complete with the core features introduced earlier. To further enhance the app and bring it closer to the full demo version, consider implementing the following features:

Adding a settings page to save post processing settings.
Handle post processing in the /transcribe api route.
Allowing edit/delete of saved notes.
Experimenting with additional features that fit your use case or user needs.

If you get stuck while implementing these features, do not hesitate to look at the application source code. The complete source code of the final application is shared at the end of the article.

Deploying the Application

You can deploy the application using either the NuxtHub admin dashboard or through the NuxtHub CLI.

Deploy via NuxtHub Admin

Push your code to a GitHub repository.
Link the repository with NuxtHub.
Deploy from the Admin console.

Learn more about NuxtHub Git integration

Deploy via NuxtHub CLI

npx nuxthub deploy

Learn more about CLI deployment

Source Code

You can find the source code of Vhisper application on GitHub. The source code includes all the features discussed in this article, along with additional configurations and optimizations shown in the demo.

ra-jeev / vhisper

Voice Notes with AI transcriptions and post processing

Vhisper - In-browser Voice Notes

Vhisper is a serverless voice notes application built with Nuxt 3 that leverages various Cloudflare services through NuxtHub for it to work. It allows users to record voice notes, transcribe and post process them using AI, and manage them through a simple, intuitive interface.

Try it Out

Live demo: https://vhisper.nuxt.dev

Preview

vhisper-demo.mov

Key Features

User Authentication: Secure access with username/password.
Record Voice Notes: Record multiple audio clips per note. Real-time audio visualization during recording.
Speech-to-Text Transcription: Automatically transcribe recordings into text using Whisper AI model.
Post-Processing: Optionally correct and refine transcriptions for better accuracy and clarity (Llama 3.1 model). Local settings persistence.
Notes Management: View and manage saved notes with playback for each audio recording.

Read the associated blog post to learn how to create this application from scratch.

Technologies Used

Nuxt: Vue.js framework for the application foundation
Nuxt UI (v3): For…

View on GitHub

Conclusion

Congratulations! You've built a powerful application that records and transcribes audio, stores recordings, and manages notes with an intuitive interface. Along the way you’ve touched upon various aspects of Nuxt, NuxtHub and Cloudflare services. As you continue to refine and expand Vhisper, consider exploring additional features and optimizations to further enhance its functionality and user experience. Keep experimenting and innovating, and let this project be a stepping stone to even more ambitious endeavors.

Thank you for sticking with me until the end! I hope you’ve picked up some new concepts along the way. I’d love to hear what you learned or any thoughts you have in the comments section. Your feedback is not only valuable to me, but to the entire developer community exploring this exciting field.

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Rethinking GitHub Search: How I built a Chat Interface to search GitHub

Rajeev R. Sharma — Fri, 04 Oct 2024 17:54:43 +0000

Here I am again, exploring the world of chat interfaces. If you've been following my journey (I mean my last post), you might think, 'Rajeev, you're becoming obsessed with chatting!' And you wouldn't be entirely wrong. While I'm not always the most talkative person in a room, give me the right topic—like simplifying how we interact with technology—and my excitement becomes difficult to contain.

This time, my enthusiasm has led me to tackle a challenge many developers face often: searching GitHub efficiently. Let me introduce Chat GitHub, a tool where the complexity of code repositories meets the simplicity of conversation.

Why Chat GitHub?

So, why a chat interface for GitHub search? Well, GitHub search, as powerful as it is with all its filters, can’t beat the simplicity of natural language queries. Let’s be real—it won’t answer simple questions like, “When did I make my first commit?” (What filters would you even use to find this information with the current GitHub Search?). GitHub certainly knows (reminds me of “I Know What You Did Last Summer”), but it just won’t answer you directly.

Natural language makes the experience frictionless. Instead of crafting complex queries, you can just ask GitHub, like you would a colleague.

When @atinux mentioned the idea to me, I was onboard immediately (also because I was itching to build something…).

How it Works?

At its core, Chat GitHub leverages the power of OpenAI's language models to interpret your natural language queries and translate them into GitHub's search syntax. Here's a simplified breakdown of the process:

You enter a query in plain English
The AI model interprets your request
It then selects the appropriate search endpoint, and generates the necessary GitHub API query parameters
We send a request to GitHub's API with these details
The results are fetched and presented to you in a clean, easy-to-digest chat interface

Here's a sneak peek of what the chat interface looks like:

You can try it out live here: https://chat-github.nuxt.dev

We’ll explore the key aspects of this process in the sections to follow. Ready? Let’s get started.

Project Setup

This project follows a similar setup to my last one Hub Chat (GitHub link), and I’ve reused several components with some slight modifications. I won't bore you by repeating the same details, but if you’re new here, feel free to follow along with both posts (previous post) for a more complete picture.

Tech Stack

Here’s a quick breakdown of the tech stack:

Nuxt 3: For the overall framework and routing.
OpenAI APIs: To handle the natural language processing.
GitHub API: To fetch the data you’re looking for—remember? Only it knows what you did last summer.
NuxtUI: To make sure the UI is smooth and responsive.
Nuxt-Auth-Utils: For user authentication and handling GitHub login (Only authenticated users can start a chat).
Nuxt MDC: For parsing and displaying the markdown responses
NuxtHub: For deployment, database, and caching (all powered by Cloudflare).

Prerequisites

GitHub account: You'll need this to generate a GITHUB_TOKEN for API queries, and to create an OAuth App for authentication.
OpenAI account: For creating an OpenAI API key, so the app can process your queries.
Optional: If you're looking to deploy the project yourself, you'll need Cloudflare and a NuxtHub account.

Setting up the project

Follow the setup process from my previous article. Just remember to add the nuxt-auth-utils, @octokit/rest and OpenAI dependencies.

# Add the nuxt-auth-utils module
npx nuxi module add auth-utils

# Add the Octokit Rest & OpenAI libraries
pnpm add @octokit/rest openai

Once the dependencies are set up, you should be able to run the project with pnpm dev and see the default app UI at localhost:3000.

Configs and Environment Variables

At this point, you can enable the hub database and cache in the nuxt.config.ts file for later use, as well as create the necessary API tokens and keys to place in the .env file.

Enabling database and cache in nuxt.config.ts:

hub: {
  cache: true,
  database: true,
},

Next, generate the following tokens and keys, and store them in your .env file (located in the root of your project):

GitHub Token: Create a GitHub token (no special scope required) for making API calls.
OpenAI API Key: Generate a key from your OpenAI dashboard.
GitHub OAuth App: Create a GitHub OAuth app and get its CLIENT_ID and CLIENT_SECRET. This will be used to authenticate users (only authenticated users can start a chat). For more information on how to create a GitHub OAuth app, refer to the GitHub documentation.

Your .env should contain the following entries:

NUXT_SESSION_PASSWORD=at_least_32_chars_string
NUXT_OAUTH_GITHUB_CLIENT_ID=github_oauth_client_id
NUXT_OAUTH_GITHUB_CLIENT_SECRET=github_oauth_client_secret
NUXT_GITHUB_TOKEN=your_personal_access_token
OPENAI_API_KEY=your_openai_api_key

Note: NUXT_SESSION_PASSWORD will automatically be created by nuxt-auth-utils in development if you haven’t set it manually.

And that’s it for the setup—phew!

In the next section, we’ll explore the core of the chat process: making sense of the user query, converting it to a GitHub API call amd generating a final response.

From User Query to GitHub API Call

Now, let’s break down how Chat GitHub processes your query, identifies the necessary actions, and makes the appropriate GitHub API call. This involves three main steps: interpreting the user query, calling the necessary tools (the GitHub API), and generating the final response.

1. Interpreting the User Query

The first challenge is understanding what the user is asking for. To do this, the system relies on OpenAI’s language models to parse natural language inputs. It takes into account several factors:

Intent Recognition: What is the user trying to achieve? Are they searching for commits, issues, repositories, or user profiles?
Parameter Extraction: Once the intent is clear, the model extracts necessary parameters like repo name, user, dates, and other filters. These details are critical for constructing a meaningful API call.
Tool Selection: Based on the query, the AI determines if a GitHub API tool needs to be invoked (for example, searching commits or issues).

To guide the AI, I created a detailed system prompt to provide context. Here’s a portion of that prompt:

const systemPropmt = `You are a concise assistant who helps \
users find information on GitHub. Use the supplied tools to \
find information when asked.

Available endpoints and key parameters:

// ...

2. issues (Also searches PRs):
  - Sort options: comments, reactions, reactions-+1, ...
  - Query qualifiers: type, is, state, author, assignee, ...
  - use "type" or "is" qualifier to search issues or PRs (type:issue/type:pr)

// ...

When using searchGithub function:
1. Choose the appropriate search endpoint.
2. Formulate a concise query (q) as per the user's request.
3. Add any relevant sort or order parameters if needed.
4. Always use appropriate per_page value to limit the number of results.

// ...

Examples: 
// ...

2. Find the total number of repositories of a user
arguments: 
{
  "endpoint": "repositories",
  "q": "user:<user_login>",
  "per_page": 1
}

Summarize final response concisely using markdown when appropriate \
(for all links add {target="_blank"} at the end). Do not include \
images, commit SHA or hashes etc. in your summary.`

Note: I restricted the AI to only the most important GitHub search endpoints /search/commits, /search/issues, /search/repositories and /search/users.

In addition to the system prompt, we create tools definitions that lists the types of tools, their names, and their specific parameters (in this case I only create one function tool, searchGithub).

const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'searchGithub',
      description:
        'Searches GitHub for information using the GitHub API. Call this if you need to find information on GitHub.',
      parameters: {
        type: 'object',
        properties: {
          endpoint: {
            type: 'string',
            description: `The specific search endpoint to use. One of ['commits', 'issues', 'repositories', 'users']`,
          },
          q: {
            type: 'string',
            description: 'the search query using applicable qualifiers',
          },
          sort: {
            type: 'string',
            description: 'The sort field (optional, depends on the endpoint)',
          },
          order: {
            type: 'string',
            description: 'The sort order (optional, asc or desc)',
          },
          per_page: {
            type: 'string',
            description:
              'Number of results to fetch per page (max 25)',
          },
        },
        required: ['endpoint', 'q', 'per_page'],
        additionalProperties: false,
      },
    },
  },
];

A couple of key design decisions we made:

Complimentary System Prompt & Tool Definition: The system prompt gives context, while the tool definition ensures the API queries are correctly structured.
Result Limit: While GitHub allows more items per response, we restrict results to 25 to keep things manageable.
No Pagination: We decided to skip pagination, keeping things simple and avoiding unnecessary complexity.

Now, the AI is ready to handle the user query and transform it into a structured format that the system can use. Here is the relevant code:

let _openai: OpenAI;
function useOpenAI() {
  if (!_openai) {
    _openai = new OpenAI({
      apiKey: process.env.OPENAI_API_KEY,
    });
  }

  return _openai;
}

export const handleMessageWithOpenAI = async (
  event: H3Event,
  messages: OpenAI.Chat.ChatCompletionMessageParam[]
) => {
  const openai = useOpenAI();
  const response = await openai.chat.completions.create({
    model: MODEL, // used gpt-4o
    messages, // contains system prompt and the complete chat history
    tools, // defined above
  });

  const responseMessage = response.choices[0].message;
  const toolCalls = responseMessage.tool_calls;

  if (toolCalls) {
    messages.push(responseMessage);

    for (const toolCall of toolCalls) {
      const functionName = toolCall.function.name;
      if (functionName === 'searchGithub') {
        const functionArgs = JSON.parse(toolCall.function.arguments);
        const functionResponse = await searchGithub(
          functionArgs.endpoint,
          {
            q: functionArgs.q,
            sort: functionArgs.sort,
            order: functionArgs.order,
            per_page: functionArgs.per_page,
          }
        );

        // ..
      }
    }

    // ...
  }

  return responseMessage.content;
}

2. Using Tool Calls for GitHub Search

Once the AI determines the need for a GitHub API call, it generates a tool_call with the necessary parameters. Here’s how we handle this with the searchGitHub function:

export type SearchParams = {
  endpoint: string;
  q: string;
  order?: string;
  sort?: string;
  per_page: string;
};

const allowedEndpoints = {
  commits: 'GET /search/commits',
  issues: 'GET /search/issues',
  repositories: 'GET /search/repositories',
  users: 'GET /search/users',
} as const;

type EndpointType = keyof typeof allowedEndpoints;

let _octokit: Octokit;

function useOctokit() {
  if (!_octokit) {
    _octokit = new Octokit({
      auth: process.env.NUXT_GITHUB_TOKEN,
    });
  }

  return _octokit;
}

export const searchGithub = async (
  endpoint: string,
  params: Omit<SearchParams, 'endpoint'>
) => {
  if (!endpoint || !allowedEndpoints[endpoint as EndpointType]) {
    throw createError({
      statusCode: 404,
      message: 'Endpoint not supported',
    });
  }

  const octokit = useOctokit();
  const endpointToUse = allowedEndpoints[endpoint as EndpointType];

  try {
    const response = await octokit.request(endpointToUse as string, params);

    return response.data;
  } catch (error) {
    console.error(error);
    throw createError({
      statusCode: 500,
      message: 'Error searching GitHub',
    });
  }
};

The function is a straightforward GitHub API search using the @octokit/rest library. The endpoint and parameters are passed from the tool_call, and we make the request to GitHub’s API, fetching the appropriate data. You can learn more about the GitHub search APIs from the official GitHub Docs.

3. Generating the Final Response

Once the GitHub API returns its results, the final step is to package them into a user-friendly response. The AI processes the raw data—whether commits, issues, repos, or users—and generates readable summaries. Here’s the relevant code snippet:

if (tool_calls) {
  // ...

  for (const toolCall of toolCalls) {
    // ...

    messages.push({
      tool_call_id: toolCall.id,
      role: 'tool',
      content: JSON.stringify(functionResponse),
    });
  }

  const finalResponse = await openai.chat.completions.create({
    model: MODEL,
    messages: messages,
  });

  return finalResponse.choices[0].message.content;
}

// ..

In the code above, you can see how we take the API response and push it to the messages array, preparing it for the AI to format into a concise response that’s easy for the user to understand.

Managing GitHub API Rate Limits

If you’ve used the GitHub API (or any third-party API), you’ll know that most of them come with rate limits. So, how can we minimize GitHub API calls? The answer is simple: we avoid making duplicate calls by caching every GitHub response. If a user requests the same information that another user (or even themselves) asked for earlier, we pull the data from the cache instead of making another API call.

Now, the question is: how do we implement this in Nuxt? It’s actually quite easy, thanks to Nitro’s Cached Functions (Nitro is an open source framework to build web servers which Nuxt uses internally). Let’s take a look at the revised searchGithub function below:

export const searchGithub = defineCachedFunction(
  async (
    event: H3Event,
    endpoint: string,
    params: Omit<SearchParams, 'endpoint'>
  ) => {
    if (!endpoint || !allowedEndpoints[endpoint as EndpointType]) {
      throw createError({
        statusCode: 404,
        message: 'Endpoint not supported',
      });
    }

    const octokit = useOctokit();
    const endpointToUse = allowedEndpoints[endpoint as EndpointType];

    try {
      const response = await octokit.request(endpointToUse as string, params);

      return response.data;
    } catch (error) {
      console.error(error);
      throw createError({
        statusCode: 500,
        message: 'Error searching GitHub',
      });
    }
  },
  {
    maxAge: 60 * 60, // 1 hour
    group: 'github',
    name: 'search',
    getKey: (
      event: H3Event,
      endpoint: string,
      params: Omit<SearchParams, 'endpoint'>
    ) => {
      const q = params.q.trim().toLowerCase().split(' ');

      const mainQuery = q.filter((term) => !term.includes(':')).join(' ');
      const qualifiers = q.filter((term) => term.includes(':')).sort();

      const finalQuery = [...(mainQuery ? [mainQuery] : []), ...qualifiers]
        .join(' ')
        .replace(/[\s:]/g, '_');

      let key = endpoint + '_q_' + finalQuery;

      if (params.per_page) {
        key += '_per_page_' + params.per_page;
      }

      if (params.order) {
        key += '_order_' + params.order;
      }

      if (params.sort) {
        key += '_sort_' + params.sort;
      }

      return key;
    },
  }
);

We’ve modified our earlier function to use cachedFunction, and added H3Event (from the /chat API endpoint call) as the first parameter—this is needed because the app is deployed on the edge with Cloudflare (more details here). The most important part here is how we create a unique cache key. Here’s a breakdown:

Query Qualifiers: First, we break down the q parameter into its qualifier pairs (e.g., for finding the first PR of a GitHub user like ra-jeev—yes, that’s me—the q value would be author:ra-jeev type:pr). Sometimes, it may contain the direct query string too, and the code accounts for this.
Sorting Qualifiers: Next, we sort the qualifiers alphabetically. This is because the AI could create the same query as type:pr author:ra-jeev. We could handle this in the system prompt, but why over-complicate things for the AI?
Creating the Cache Key: Finally, we combine all the qualifiers and other parameters into a single string, separated by underscores.

We set the cache duration to 1 hour, as seen in the maxAge setting, which means all searchGitHub responses are stored for that time. To use cache in NuxtHub production we’d already enabled cache: true in our nuxt.config.ts.

Now that we’ve tackled rate limiting, it’s time to shift our focus to response streaming. In the next section, we’ll explore how to implement streaming for a more seamless and efficient user experience.

Adding AI Response Streaming

Enabling AI response streaming is usually straightforward: you pass a parameter when making the API call, and the AI returns the response as a stream. In our Hub Chat project, for example, we handled the stream chunks directly client-side, ensuring that responses trickled in smoothly for the user.

But in the case of Chat GitHub, there’s an additional complexity—tool_call responses. So, we have two approaches to manage:

Enable streaming only for the final response generation, but this limits us when there’s no tool_call, wasting an opportunity to stream from the start.
Enable stream response for both the OpenAI calls, but this requires us to manage a stream more carefully on the server side (as we need to handle the tool_call, if any, there itself).

I took the seemingly complex path and went ahead with the second approach—”Two roads diverged in a wood, and I—I took the one less traveled by, And that has made all the difference.”

Enable Streaming in OpenAI Calls

Here’s how the OpenAI API call was modified to enable streaming:

export const handleMessageWithOpenAI = async function* (
  event: H3Event,
  messages: OpenAI.ChatCompletionMessageParam[],
) {
  const openai = useOpenAI();
  const responseStream = await openai.chat.completions.create({
    model: MODEL,
    messages,
    tools,
    stream: true, // enable streaming
  });

  const currentToolCalls: OpenAI.ChatCompletionMessageToolCall[] = [];

  for await (const chunk of responseStream) {
    const choice = chunk.choices[0];

    // if it is normal text chunk just yield it
    if (choice.delta.content) {
      yield choice.delta.content;
    }

    // if the delta contains tool_calls, then collect all chunks
    if (choice.delta.tool_calls) {
      for (const toolCall of choice.delta.tool_calls) {
        if (toolCall.index !== undefined) {
          if (!currentToolCalls[toolCall.index]) {
            currentToolCalls[toolCall.index] = {
              id: toolCall.id || '',
              type: 'function' as const,
              function: {
                name: toolCall.function?.name || '',
                arguments: '',
              },
            };
          }

          if (toolCall.function?.arguments) {
            currentToolCalls[toolCall.index].function.arguments +=
              toolCall.function.arguments;
          }
        }
      }
    }

    // once it has returned all chunks with tool_calls, we will
    // get the final finish_reason as 'tool_calls'. We can call
    // the mentioned tool now
    if (choice.finish_reason === 'tool_calls') {
      messages.push({
        role: 'assistant',
        tool_calls: currentToolCalls,
      });

      for (const toolCall of currentToolCalls) {
        if (toolCall.function.name === 'searchGithub') {
          try {
            const functionArgs = JSON.parse(toolCall.function.arguments);
            const toolResult = await searchGithub(
              event,
              functionArgs.endpoint,
              {
                q: functionArgs.q,
                sort: functionArgs.sort,
                order: functionArgs.order,
                per_page: functionArgs.per_page,
              }
            );

            messages.push({
              role: 'tool',
              tool_call_id: toolCall.id,
              content: JSON.stringify(toolResult),
            });
          } catch (error) {
            console.error('Error parsing tool call arguments:', error);
            throw error;
          }
        }
      }

      try {
        const finalResponse = await openai.chat.completions.create({
          model: MODEL,
          messages,
          stream: true,
        });

        for await (const chunk of finalResponse) {
          if (chunk.choices[0].delta.content) {
            yield chunk.choices[0].delta.content;
          }
        }
      } catch (error) {
        console.error(
          'Error generating final response or saving user query :',
          error
        );

        throw error;
      }
    }
  }
};

The revised handleMessageWithOpenAI function works like this:

Converted it to an AsyncGenerator: This allows the function to yield data chunks progressively as they are received.
Added stream: true to both OpenAI API calls: This tells OpenAI to stream the response back to us.
Yielding Response Chunks: For each chunk of text that we get from the stream, we simply yield it to the caller.
Handling tool_calls: Since streaming is enabled, the tool_call information also arrives in chunks. We collect these chunks until the OpenAI API signals the completion of this part (finish_reason === 'tool_calls'), and then invoke the searchGitHub function using the parameters provided by the tool_call.
Final Response: After the GitHub search is done, we yield the response in chunks in the same way.

Converting to a ReadableStream

To complete the process, the chunks from handleMessageWithOpenAI are converted into a ReadableStream format, which is then returned to the client (not shown here). Here’s how that works:

export const asyncGeneratorToStream = (
  asyncGenerator: AsyncGenerator<string, void, unknown>
) => {
  let cancelled = false;
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const value of asyncGenerator) {
          if (cancelled) {
            break;
          }

          controller.enqueue(
            encoder.encode(`data: ${JSON.stringify({ response: value })}\n\n`)
          );
        }

        // Send done to signal end of stream
        controller.enqueue(encoder.encode(`data: [DONE]\n\n`));

        controller.close();
      } catch (err) {
        console.log('Error in stream:', err);

        /* eslint-disable @stylistic/operator-linebreak */
        const errorMessage =
          err instanceof Error
            ? err.message
            : 'An error occurred in the stream';
        /* eslint-enable @stylistic/operator-linebreak */

        controller.enqueue(
          encoder.encode(
            `event: error\ndata: ${JSON.stringify({
              message: errorMessage,
            })}\n\n`
          )
        );

        controller.close();
      }
    },
    cancel(reason) {
      console.log('Client closed connection. Reason:', reason);
      cancelled = true;
    },
  });

  return stream;
};

This code transforms the AsyncGenerator we created earlier into a ReadableStream. Here’s a breakdown of what’s happening:

AsyncGenerator: We pass the AsyncGenerator from handleMessageWithOpenAI as an argument to this function.
Creating a ReadableStream: Inside the start method of the ReadableStream, we wait for chunks from the AsyncGenerator.
Formatting Chunks: For each text chunk received, we format it according to the Server-Sent Events (SSE) convention (You can read more about SSE in my previous post). Each chunk is wrapped in this format: data: ${JSON.stringify({ response: value })}\n\n.
Encoding the Stream: Using TextEncoder, we encode the chunks before sending them to the client. This encoding step is crucial, especially in production environments, as it ensures that the client correctly receives the stream in real-time.
Error Handling: If an error occurs (as we throw errors from the AsyncGenerator), we send an event: error message to the client, signaling that something went wrong, and then close the stream to terminate the connection cleanly.

And then this stream is returned to the client from the API endpoint

// inside /api/chat event handler
return asyncGeneratorToStream(
  handleMessageWithOpenAI(event, llmMessages)
);

Handling the Stream on the Client-Side

To handle the streamed response, we’ll use a refactored version of the useChat composable, initially created for the Hub Chat project. This time, we’ll extend it to handle error events. Here's the updated code:

export function useChat(apiBase: string, body: Record<string, unknown>) {
  async function* chat(): AsyncGenerator<string, void, unknown> {
    try {
      const response = await $fetch(apiBase, {
        method: 'POST',
        body,
        responseType: 'stream',
      });

      let buffer = '';
      const reader = (response as ReadableStream)
        .pipeThrough(new TextDecoderStream())
        .getReader();

      while (true) {
        const { value, done } = await reader.read();

        if (done) {
          if (buffer.trim()) {
            console.warn('Stream ended with unparsed data:', buffer);
          }

          return;
        }

        buffer += value;
        const messages = buffer.split('\n\n');
        buffer = messages.pop() || '';

        for (const message of messages) {
          const lines = message.split('\n');
          let event = '';
          let data = '';

          for (const line of lines) {
            if (line.startsWith('event:')) {
              event = line.slice('event:'.length).trim();
            } else if (line.startsWith('data:')) {
              data = line.slice('data:'.length).trim();
            }
          }

          if (event === 'error') {
            const parsedError = JSON.parse(data);
            console.error('Stream error:', parsedError);

            throw new Error(
              parsedError.message ?? 'Failed to generate response'
            );
          } else if (data) {
            if (data === '[DONE]') return;

            try {
              const jsonData = JSON.parse(data);
              if (jsonData.response) {
                yield jsonData.response;
              }
            } catch (parseError) {
              console.warn('Error parsing JSON:', parseError);
            }
          }
        }
      }
    } catch (error) {
      console.error('Error sending message:', error);
      throw error;
    }
  }

  return chat;
}

Here’s a breakdown of what the code does:

We use Nuxt’s $fetch to make a POST request to the API endpoint, passing the responseType: 'stream'. This tells the client to expect a streaming response.
Once we receive the ReadableStream, we create a streamReader for it. This allows us to process the chunks one at a time as they arrive. We also pass the chunks through a TextDecoder to convert the raw bytes into readable text.
The stream is in Server-Sent Events (SSE) format, so we parse and handle the event and data parts appropriately. Each data chunk is parsed and returned to the client component for rendering.
The code also listens for and handles any error events that may occur, ensuring a smoother user experience by gracefully handling stream interruptions or API errors.

And this concludes the road less traveled that we took earlier. In the next section, we’ll cover how we can authenticate our users.

User Authentication with GitHub OAuth

Now that our backend is ready to handle client requests, how do we restrict access to authenticated users? And how do we provide context to the AI, like answering a query such as, “When did I make my first ever commit?” To control who can access the backend, we use authentication. And to give the AI context about the user, we rely on GitHub OAuth for authentication.

We’re leveraging nuxt-auth-utils, which simplifies integrating GitHub OAuth into our Nuxt app. This allows us to authenticate users with their GitHub accounts and manage sessions effortlessly.

Server-Side Implementation

On the server side, we need to create a route that handles the GitHub access token when the user logs in. Make sure to create a GitHub OAuth app and add the CLIENT_ID and CLIENT_SECRET to the .env as mentioned in the setup section.

To implement this, create a github.get.ts file in the route/auth folder of the server directory with the following content:

export default oauthGitHubEventHandler({
  async onSuccess(event, { user }) {
    await setUserSession(event, {
      user: {
        id: user.id,
        login: user.login,
        name: user.name,
        avatarUrl: user.avatar_url,
        htmlUrl: user.html_url,
        publicRepos: user.public_repos,
      },
    });

    return sendRedirect(event, '/chat');
  },
  // Optional, will return a json error and 401 status code by default
  onError(event, error) {
    console.error('GitHub OAuth error:', error);
    return sendRedirect(event, '/');
  },
});

The event handler name is important and should be oauthGitHubEventHandler (more details can be found here). On successful login, we call the setUserSession utility function to store the user details in an HTTP cookie and redirect them to the chat page.

For our API routes, we can then call the requireUserSession utility to ensure only authenticated users can make requests. Below is the full /api/chat endpoint handler:

export default defineEventHandler(async (event) => {
  const userSession = await requireUserSession(event);

  const { messages } = await readBody(event);
  if (!messages) {
    throw createError({
      statusCode: 400,
      message: 'User messages are required',
    });
  }

  const llmMessages = [
    {
      role: 'system',
      content: getSystemPrompt(userSession.user.login),
    },
    ...messages,
  ];

  return asyncGeneratorToStream(
    handleMessageWithOpenAI(event, llmMessages, userSession.user.login)
  );
});

As you can see, we retrieve the currently logged-in GitHub user’s details and pass the login info into the system prompt. This gives OpenAI the context it needs to answer queries like, “When did I make my first commit?” The getSystemPrompt utility helps with that:

export const getSystemPrompt = (loggedInUserName: string) =>
  systemPrompt +
  `\n\nNote: The currently logged in github user is "${loggedInUserName}".`;

Client-Side Handling

On the client side, we use the built-in AuthState component from nuxt-auth-utils to manage authentication flows, like logging in and checking if a user is signed in.

Here’s how to handle sign-in:

<AuthState v-slot="{ loggedIn }">
  <UButton
    v-if="loggedIn"
    size="lg"
    trailing-icon="i-heroicons-arrow-right-16-solid"
    to="/chat"
  >
    Go to Chat
  </UButton>

  <div v-else class="flex flex-col items-center justify-center gap-y-2">
    <UButton
      size="lg"
      icon="i-simple-icons-github"
      to="/auth/github"
      external
    >
      Sign in with GitHub
    </UButton>
    <p class="text-sm text-gray-600 dark:text-gray-300 text-center">
      Start Chatting Now!
    </p>
  </div>
</AuthState>

Notice the external attribute on the Sign-In button. This attribute is essential—it tells the framework to treat the GitHub authentication as an external process. Without it, the framework will try to redirect you to the /auth/github route on the client side, causing errors (It did get me for sure).

This completes the server-side and client-side setup for user authentication. You can go through the shared GitHub repo to see how we also restrict navigation on the client side by using an auth middleware, ensuring that only authenticated users can access the chat page.

Bonus: Enhancing Engagement

As the project neared completion, I kept thinking about how to make it more engaging. I wanted to show what users were querying about and who they were querying for. However, I didn’t want to save every type of query—especially those like “When did I make my first commit?”—as they were both pointless and numerous. Instead, I decided to save only the queries made about other users, repositories, and other entities, excluding self-referential queries.

Database Setup

To achieve this, we needed a database, which is why we enabled database: true in our nuxt.config.ts file. This setting binds Cloudflare’s D1 database in production and uses its platform proxy during development.

First, we create the necessary database tables using the hubDatabase server composable:

await hubDatabase().exec(
  `CREATE TABLE IF NOT EXISTS queries (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    text TEXT NOT NULL,
    response TEXT NOT NULL,
    github_request TEXT NOT NULL,
    github_response TEXT NOT NULL,
    queried_at DATETIME DEFAULT CURRENT_TIMESTAMP
  );
`.replace(/\n/g, '')
);

// ... other tables and indexes

Next, we utilize utility functions to save the queries. The saveUserQuery function is invoked only if a tool call was made from the handleMessageWithOpenAI function we discussed earlier.

export type ToolCallDetails = {
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  response: any;
  request: SearchParams;
};

export type UserQuery = {
  userMessage: string;
  toolCalls: ToolCallDetails[];
  assistantReply: string;
};

const getAvatarUrl = (toolCall: ToolCallDetails) => {
  let avatarUrl;
  const responseItem = toolCall.response.items[0];

  if (responseItem) {
    if (responseItem.author) {
      avatarUrl = responseItem.author.avatar_url;
    } else if (responseItem.user) {
      avatarUrl = responseItem.user.avatar_url;
    } else if (responseItem.owner) {
      avatarUrl = responseItem.owner.avatar_url;
    } else if (responseItem.avatar_url) {
      avatarUrl = responseItem.avatar_url;
    }
  }

  return avatarUrl;
};

const shouldSaveUserQuery = (
  toolCall: ToolCallDetails,
  loggedInUser: string
) => {
  const responseItem = toolCall.response.items[0];
  if (
    responseItem &&
    ((responseItem.author && responseItem.author.login === loggedInUser) ||
      (responseItem.user && responseItem.user.login === loggedInUser) ||
      (responseItem.owner && responseItem.owner.login === loggedInUser) ||
      responseItem.login === loggedInUser)
  ) {
    return false;
  }

  return true;
};

export const saveUserQuery = async (
  loggedInUser: string,
  userQuery: UserQuery
) => {
  const toolCall = userQuery.toolCalls[0];
  const matchedUser = toolCall.request.q.match(/(?:author:|user:)(\S+)/);
  if (matchedUser) {
    const queriedUser = matchedUser[1].toLowerCase();
    if (queriedUser !== loggedInUser) {
      const avatarUrl = getAvatarUrl(toolCall);

      await storeQuery(
        userQuery.userMessage,
        userQuery.assistantReply,
        toolCall,
        { login: queriedUser, avatarUrl }
      );
    }
  } else if (shouldSaveUserQuery(toolCall, loggedInUser)) {
    await storeQuery(userQuery.userMessage, userQuery.assistantReply, toolCall);
  }
};

It checks if the query involves the currently logged-in user. If it does, we simply return without logging anything; otherwise, we store the query details in the database using:

const storeQuery = async (
  queryText: string,
  assistantReply: string,
  toolCall: ToolCallDetails,
  queriedUser?: { login: string; avatarUrl?: string }
) => {
  try {
    const db = hubDatabase();

    const queryStmt = db
      .prepare(
        'INSERT INTO queries (text, response, github_request, github_response) VALUES (?1, ?2, ?3, ?4)'
      )
      .bind(
        queryText,
        assistantReply,
        JSON.stringify(toolCall.request),
        JSON.stringify(toolCall.response)
      );
    if (queriedUser) {
      const [batchRes1, batchRes2] = await db.batch([
        queryStmt,
        db
          .prepare(
            `INSERT INTO trending_users (username, search_count, last_searched, avatar_url)
              VALUES (?1, 1, CURRENT_TIMESTAMP, ?2)
              ON CONFLICT(username)
              DO UPDATE SET search_count = search_count + 1, last_searched = CURRENT_TIMESTAMP, avatar_url = COALESCE(?2, avatar_url)`
          )
          .bind(queriedUser.login, queriedUser.avatarUrl),
      ]);

      console.log('storeQuery: ', batchRes1, batchRes2);
    } else {
      const res = await queryStmt.run();

      console.log('storeQuery: ', res);
    }
  } catch (error) {
    console.error('Failed to store query: ', error);
  }
};

API Endpoints

We then create API endpoints to retrieve the trending users and recent queries, as shown below (note the use of endpoint caching):

export const getRecentQueries = async () => {
  const db = hubDatabase();
  console.log('getRecentQueries');
  const result = await db
    .prepare(
      'SELECT id, text, response, queried_at FROM queries ORDER BY queried_at DESC LIMIT ?'
    )
    .bind(10)
    .all<RecentQuery>();

  console.log('getRecentQueries: ', result);
  return result.results;
};

export default defineCachedEventHandler(
  async () => {
    const results = await getRecentQueries();

    return results;
  },
  {
    maxAge: 10 * 60, // 10 minutes
  }
);

Frontend Integration

This setup allows us to display the data in the frontend, providing users with insights into trending queries and recently searched users, as illustrated in the screenshot below.

You can go through the shared GitHub repo to see the whole implementation in detail.

And with that, now you can also know what you did last summer—phew!

Source Code

The project’s code is open source and can be checked here

ra-jeev / chat-github

GitHub Search, powered by OpenAI, through an intuitive chat interface.

Chat GitHub: Search GitHub in Plain English

Try It Out

Chat GitHub Demo: https://chat-github.nuxt.dev

Overview

Chat GitHub is an innovative tool that allows you to search and explore GitHub using natural language queries, powered by AI. Say goodbye to complex search parameters and hello to intuitive, conversation-style GitHub exploration.

Read the blog post on how I created Chat GitHub.

Features

Natural Language Search: Query GitHub using plain English, no need for complex search parameters.
Comprehensive GitHub Data: Search across multiple GitHub endpoints:
- Users (/search/users)
- Repositories (/search/repositories)
- Issues (/search/issues)
- Commits (/search/commits)
AI-Powered Responses: Leverages OpenAI to interpret queries and provide concise, relevant information.
User Authentication: Secure access via GitHub OAuth.
Trending Users: Discover popular GitHub users based on search trends.
Anonymized Query Logging: Privacy-focused logging of search queries for analytics and trending features.

Built with

Nuxt: Vue.js framework for the application foundation
Nuxt UI: Module for creating a…

View on GitHub

Deployment

You can deploy the app using your NuxtHub admin panel or manually through the NuxtHub CLI. For more details on deploying an app through NuxtHub, refer to the official documentation.

The best part is that this project is now listed as a template on the NuxtHub Templates page. So, if you already have a NuxtHub account, you can deploy this project in one click using the button below (Just remember to add the necessary environment variables in the panel).

Further Enhancements

Currently, we rely on the AI's ability to generate GitHub API queries from natural language input. While we’ve provided it with details about various qualifiers, it can still struggle with more complex or intricate queries. I also experimented with tool-calling models from Cloudflare’s Workers AI and Groq API, and found that gpt-4o performed better for these tasks. Claude 3.5 Sonnet should definitely offer even better results, but it tends to be more expensive.

Here are a few alternative approaches we could explore to improve query accuracy and results:

Fine-tune/train a tool-calling model specifically on the GitHub Search API documentation.
Create embeddings from the GitHub Search documentation and store them in a vector database. Cloudflare offers a free tier for its vector database now, which we could leverage. When a user query is made, we could retrieve relevant information from the embeddings and include it in the system prompt.

If you have any other ideas for improving this project—or any feedback—please feel free to share them in the comments!

Conclusion

And there you have it—a GitHub search tool, powered by OpenAI, wrapped in a chat interface, and ready for action. We’ve gone through setting up the project, streaming responses, authenticating users, and even added a sprinkle of engagement to make things interesting.

While the project has plenty of room for improvements, it’s a solid foundation to build upon. And who knows—maybe with a few tweaks, it’ll be predicting your next GitHub repo before you even think of it :-).

May all your commits be bug-free!

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Create Your Own Cloudflare Workers AI LLM Playground Using NuxtHub and NuxtUI

Rajeev R. Sharma — Thu, 29 Aug 2024 16:05:17 +0000

You might be wondering, 'Another LLM (Large Language Model) playground? Aren't there plenty of these already?' Fair question. But here's the thing: the world of AI is constantly evolving, and every day one new tool or another pops up. One such AI offering from NuxtHub was launched recently, and I couldn’t resist taking it for a spin. And let’s be honest—some of us just can't resist adding a 'dark mode', something the Cloudflare Workers AI models playground hasn’t embraced yet.

But beyond these practical reasons, there’s something even more satisfying: the joy of creation and learning. So, if you're ready to dive in, maybe you'll pick up some new concepts like Server-Sent Events, response streaming, markdown parsing, and, of course, deploying your own chat playground without any of the usual fuss. So, get comfy—however you like—and let's get started!

Project Overview

Alright, so what exactly are we going to create here? As you might have guessed already, we will be creating a chat interface to talk to different text generation models supported by Cloudflare Workers AI. Below is a brief list of capabilities we will build along the way:

Ability to set different LLM params, like temperature, max tokens, system prompt, top_p, top_k etc while keeping some of these optional
Ability to turn LLM response streaming on/off
Handle streaming/non-streaming LLM responses on both, the server and the client side
Parsing LLM responses for markdown and display it appropriately
Auto-scrolling the chat container as the response is streamed from the LLM endpoint
Adding the dark mode (this one is trivial but let's add it here for completeness)

This is how the interface will look when we are through this article:

You can try it out live here: https://hub-chat.nuxt.dev

We will cover each of the tasks in detail in the following sections.

Project Setup

Now that we've set the stage for our LLM playground project, let's dive into the technologies we'll be using and get our development environment ready.

Technologies we'll use

Nuxt 3: Nuxt 3 is a powerful Vue.js framework that will serve as the foundation of our application.
Nuxt UI: A Nuxt module that will help us create a sleek and responsive interface.
NuxtHub: NuxtHub is a deployment and administration platform for Nuxt, powered by Cloudflare. We can use NuxtHub to access different Cloudflare offerings like D1 database, Workers KV, R2 Storage, Workers AI etc. In this project, we'll use it to access the LLMs as well as to deploy our project
Nuxt MDC: For parsing and displaying the chat messages

Prerequisites

Apart from the basic prerequisites like Node/Npm, Code Editors, and some VueJs/Nuxt knowledge, you'll need the following to follow along:

A Cloudflare account: To use the Workers AI models, as well as to deploy your project on Cloudflare Pages for free. If you don't have it already, you can set it up here.
A NuxtHub Admin Account: NuxtHub admin is a web based dashboard to manage NuxtHub apps. You can create your account here.

Setting up the project

We can either start with a Nuxt (or Nuxt UI) template and add NuxtHub on top of it, or we can use the NuxtHub starter template and add Nuxt UI to it. We'll take the second approach:

Create a new NuxtHub project

# Init the project and install dependencies
npx nuxthub init cf-playground

# Change into the created dir
cd cf-playground

Add the Nuxt UI module. The below command will install the @nuxt/ui dependency as well as add it as a module in your nuxt config file
```
npx nuxi module add ui
```
Similarly, add the Nuxt MDC module
```
npx nuxi module add mdc
```

Now we have setup everything that we need for this project. You can try running the project with pnpm dev (or an equivalent command if you're using a different package manager) and visit http://localhost:3000 in your browser. If you've dark mode enabled on your system then you'll notice that dark mode is already up and running in the project (We just need a way to toggle it).

With our development environment set up, we're ready to start building our LLM playground. In the next section, we'll begin implementing our user interface using NuxtUI components.

Creating the UI Components

First, we will create a component that will let us set numerical values for LLM settings using either a range slider or an input box. Let's call it RangeInput.

The RangeInput Component

We utilize a FormGroup component from NuxtUI to create this component. We put a Range slider in the default slot and an input in the hint slot to achieve the desired outcome. Here is the complete component code

<template>
  <UFormGroup :label="label" :ui="{ container: 'mt-2' }">
    <template #hint>
      <UInput
        v-model="model"
        class="w-[72px]"
        type="number"
        :min="min"
        :max="max"
        :step="step"
      />
    </template>
    <URange v-model="model" :min="min" :max="max" :step="step" size="sm" />
  </UFormGroup>
</template>

<script setup lang="ts">
const model = defineModel({ type: Number, default: undefined });

defineProps({
  label: {
    type: String,
    required: true,
  },
  min: {
    type: Number,
    default: undefined,
  },
  max: {
    type: Number,
    default: undefined,
  },
  step: {
    type: Number,
    default: undefined,
  },
});
</script>

Instead of taking the model value as a prop and then emitting the changes manually, we use the defineModel macro to achieve two way binding. If the initial model value is undefined the input box will be empty. This will help in making some of the LLM params optional.

The LLMSettings Component

LLMSettings component utilizes many RangeInputs that we created above as most of the settings are numerical values. Apart from RangeInputs we use a textarea for the system prompt, a toggle for enabling/disabling response streaming, a select menu for choosing the LLM model, and an accordion for hiding the optional params.

Here are the relevant parts of the component

<template>
  <div class="h-full flex flex-col overflow-hidden">
    <!-- Settings Header Code -->
    <UDivider />
    <div class="p-4 flex-1 space-y-6 overflow-y-auto">
      <UFormGroup label="Model">
        <USelectMenu
          v-model="llmParams.model"
          size="md"
          :options="models"
          value-attribute="id"
          option-attribute="name"
        />
      </UFormGroup>

      <RangeInput
        v-model="llmParams.temperature"
        label="Temperature"
        :min="0"
        :max="5"
        :step="0.1"
      />

      <RangeInput
        v-model="llmParams.maxTokens"
        label="Max Tokens"
        :min="1"
        :max="4096"
      />

      <UFormGroup label="System Prompt">
        <UTextarea
          v-model="llmParams.systemPrompt"
          :rows="3"
          :maxrows="8"
          autoresize
        />
      </UFormGroup>

      <div class="flex items-center justify-between">
        <span>Stream Response</span>
        <UToggle v-model="llmParams.stream" />
      </div>

      <UAccordion
        :items="accordionItems"
        color="white"
        variant="solid"
        size="md"
      >
        <template #item>
          <UCard :ui="{ body: { base: 'space-y-6', padding: 'p-4 sm:p-4' } }">
            <RangeInput
              v-model="llmParams.topP"
              label="Top P"
              :min="0"
              :max="2"
              :step="0.1"
            />

            <!-- Other optional params -->
          </UCard>
        </template>
      </UAccordion>
    </div>
  </div>
</template>

<script setup lang="ts">
type LlmParams = {
  model: string;
  temperature: number;
  maxTokens: number;
  topP?: number;
  topK?: number;
  frequencyPenalty?: number;
  presencePenalty?: number;
  systemPrompt: string;
  stream: boolean;
};

const llmParams = defineModel('llmParams', {
  type: Object as () => LlmParams,
  required: true,
});

defineEmits(['hideDrawer', 'reset']);

const accordionItems = [
  {
    label: 'Advanced Settings',
    defaultOpen: false,
  },
];

const models = [
  {
    name: 'deepseek-coder-6.7b-base-awq',
    id: '@hf/thebloke/deepseek-coder-6.7b-base-awq',
  },
  { 
    name: 'llama-3-8b-instruct', 
    id: '@cf/meta/llama-3-8b-instruct', 
  },
  // ...other models
]
</script>

The ChatPanel Component

This is the most important component of all as it handles the core chat functionality of the app. It consists of three parts:

Chat Header

It displays the app name/label apart from some global buttons for clearing chats, dark mode toggling and for showing the settings drawer on mobile devices

<template>
  <div class="flex items-center justify-between p-4">
    <div class="flex items-center gap-x-4">
      <h2 class="text-xl md:text-2xl text-primary font-bold">Hub Chat</h2>
      <UTooltip text="Clear chat">
        <UButton
          color="gray"
          icon="i-heroicons-trash"
          size="xs"
          :disabled="clearDisabled"
          @click="$emit('clear')"
        />
      </UTooltip>
    </div>
    <div class="flex items-center gap-x-4">
      <ColorMode />
      <UButton
        icon="i-heroicons-cog-6-tooth"
        color="gray"
        variant="ghost"
        class="md:hidden"
        @click="$emit('showDrawer')"
      />
    </div>
  </div>
</template>

<script setup lang="ts">
defineEmits(['clear', 'showDrawer']);

defineProps({
  clearDisabled: {
    type: Boolean,
    default: true,
  },
});
</script>

Chats Container

For parsing and displaying the chat messages. It also shows a message loading skeleton when streaming is off, as well as a NoChats placeholder when needed.

<div ref="chatContainer" class="flex-1 overflow-y-auto p-4 space-y-5">
  <div
    v-for="(message, index) in chatHistory"
    :key="`message-${index}`"
    class="flex items-start gap-x-4"
  >
    <div
      class="w-12 h-12 p-2 rounded-full"
      :class="`${
        message.role === 'user' ? 'bg-primary/20' : 'bg-blue-500/20'
      }`"
    >
      <UIcon
        :name="`${
          message.role === 'user'
            ? 'i-mdi-user'
            : 'i-heroicons-sparkles-solid'
        }`"
        class="w-8 h-8"
        :class="`${
          message.role === 'user' ? 'text-primary-400' : 'text-blue-400'
        }`"
      />
    </div>
    <div v-if="message.role === 'user'">
      {{ message.content }}
    </div>
    <AssistantMessage v-else :content="message.content" />
  </div>
  <ChatLoadingSkeleton v-if="loading === 'message'" />
  <NoChats v-if="chatHistory.length === 0" class="h-full" />
</div>

To keep the user informed of the progress, we show a message loading skeleton when streaming is off. To do this we utilize a loading variable which can have three states: idle, message & stream. So when a non-streaming request is made we set the ref to message and the loading skeleton is shown.

User Message Textbox

For entering user message. It shows up as a single line textarea that resizes automatically when needed.

<div class="flex items-start p-3.5 relative">
  <UTextarea
    v-model="userMessage"
    placeholder="How can I help you today?"
    class="w-full"
    :ui="{ padding: { xl: 'pr-11' } }"
    :rows="1"
    :maxrows="5"
    :disabled="loading !== 'idle'"
    autoresize
    size="xl"
    @keydown.enter.exact.prevent="sendMessage"
    @keydown.enter.shift.exact.prevent="userMessage += '\n'"
  />

  <UButton
    icon="i-heroicons-arrow-up-20-solid"
    class="absolute top-5 right-5"
    :disabled="loading !== 'idle'"
    @click="sendMessage"
  />
</div>

For allowing the user to send message on hitting enter, we use the keydown event listener with the needed modifiers (@keydown.enter.exact.prevent). Similarly, for adding a newline we use enter + shift keys.

Now that we have our UI components in place, let's shift our focus to the backend. We'll set up the AI integration and create an API endpoint to handle our chat requests. This will bridge the gap between our frontend interface and the AI model.

Setting up AI and API Endpoint

Integrating AI into our project is very simple thanks to NuxtHub. Let's look at our nuxt.config.ts file in the root dir of the project.

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  compatibilityDate: '2024-07-30',
  // https://nuxt.com/docs/getting-started/upgrade#testing-nuxt-4
  future: { compatibilityVersion: 4 },

  // https://nuxt.com/modules
  modules: ['@nuxthub/core', '@nuxt/eslint', '@nuxtjs/mdc', "@nuxt/ui"],

  // https://hub.nuxt.com/docs/getting-started/installation#options
  hub: {},

  // Env variables - https://nuxt.com/docs/getting-started/configuration#environment-variables-and-private-tokens
  runtimeConfig: {
    public: {
      // Can be overridden by NUXT_PUBLIC_HELLO_TEXT environment variable
      helloText: 'Hello from the Edge 👋',
    },
  },

  // https://eslint.nuxt.com
  eslint: {
    config: {
      stylistic: {
        quotes: 'single',
      },
    },
  },

  // https://devtools.nuxt.com
  devtools: { enabled: true },
});

To enable AI, we just need to add the ai: true flag under hub config options above. If needed, we can enable other NuxtHub Cloudflare integrations like D1 database (database: true), Workers KV (kv: true) etc. While we are here, we can remove runtimeConfig as we don't need it. (You can also remove/modify the eslint config as per your code editor settings).

If we want to use AI in dev mode (which we definitely do), we need to link this project to a Cloudflare project. This is needed as we will be talking directly to the Cloudflare APIs as no workers are running yet (and also because the AI usage will be linked to your Cloudflare account). This is a straight forward task, just run the following command

npx nuxthub link

Running the link command will make sure that we are logged into to our NuxtHub account, and will allow us to create a new NuxtHub project (backed by Cloudflare) or choose an existing one.

Creating Chat API Endpoint

Let's create a chat api endpoint. Create a new file chat.post.ts ("post" in the file name signifies that this endpoint will only accept HTTP POST requests) in the server/api directory and add the following code to it

export default defineEventHandler(async (event) => {
  const { messages, params } = await readBody(event);
  if (!messages || messages.length === 0 || !params) {
    throw createError({
      statusCode: 400,
      statusMessage: 'Missing messages or LLM params',
    });
  }

  const config = {
    max_tokens: params.maxTokens,
    temperature: params.temperature,
    top_p: params.topP,
    top_k: params.topK,
    frequency_penalty: params.frequencyPenalty,
    presence_penalty: params.presencePenalty,
    stream: params.stream,
  };

  const ai = hubAI();

  try {
    const result = await ai.run(params.model, {
      messages: params.systemPrompt
        ? [{ role: 'system', content: params.systemPrompt }, ...messages]
        : messages,
      ...config,
    });

    return params.stream
      ? sendStream(event, result as ReadableStream)
      : (
          result as {
            response: string;
          }
        ).response;
  } catch (error) {
    console.error(error);
    throw createError({
      statusCode: 500,
      statusMessage: 'Error processing request',
    });
  }
});

hubAI is a server composable that returns a Workers AI client for interacting with the LLMs. We pass the messages as well as the LLM params that we received from the frontend and run the requested model.

If you look at the above code carefully you'll notice that this endpoint supports both: streaming and non-streaming responses. To return a stream from the endpoint we just need to call the sendStream utility function.

We are through with our server side code. Let's look at how to handle the stream response on the client side in the next section.

Consuming Server Sent Events

Before diving into how to handle stream responses, let's understand why streaming is crucial for LLM interactions and how Server Sent Events (SSE) facilitate this process.

Why Stream LLM Responses with Server Sent Events?

Large Language Models (LLMs) can take considerable time to generate complete responses, especially for complex queries. Traditionally, web applications wait for the entire response before displaying anything to the user. However, this approach can lead to long waiting times and a less engaging user experience.

Server Sent Events (SSE) offer a solution to this problem.

💡 SSE is a technology that allows a server to push data to a web page at any time, enabling real-time updates without the need for the client to constantly request new information.

Here's how SSE benefits LLM responses:

Immediate Feedback: As soon as the LLM starts generating content, it can be sent to the client and displayed, providing immediate feedback to the user.
Improved Perceived Performance: Users see content appearing progressively, giving the impression of a faster, more responsive system.
Real-time Interaction: The gradual appearance of text mimics human typing, creating a more natural and engaging conversational experience.

💡 Think of it like ordering at a restaurant: Instead of waiting for all dishes to be prepared before serving, SSE allows the "kitchen" (server) to send out each "dish" (piece of the response) as soon as it's ready. This way, you start "eating" (reading and processing) sooner, and the overall experience feels faster and more satisfying.

Technically, SSE works by establishing a unidirectional channel from the server to the client. The server sends text data encoded in UTF-8, separated by newline characters. This simple yet effective mechanism allows for real-time updates in web applications, making it ideal for streaming LLM responses.

Now that we understand the importance of streaming for LLM responses and how SSE enables this, let's look at how to implement this in our Nuxt 3 application.

Handling Server Sent Events with Nuxt 3 POST Requests

Since ours is a POST request, we need handle it differently. Nuxt 3 docs gives you an excellent starting point to do this. Reproducing the code from the docs

// Make a POST request to the SSE endpoint
const response = await $fetch<ReadableStream>('/chats/ask-ai', {
  method: 'POST',
  body: {
    query: "Hello AI, how are you?",
  },
  responseType: 'stream',
})

// Create a new ReadableStream from the response with TextDecoderStream to get the data as text
const reader = response.pipeThrough(new TextDecoderStream()).getReader()

// Read the chunk of data as we get it
while (true) {
  const { value, done } = await reader.read()

  if (done)
    break

  console.log('Received:', value)
}

We need to set the request responseType flag as stream, and set the type of $fetch response as ReadableStream. Then we create a stream reader while decoding the received chunks by piping the events through a TextDecoder.

Below is a glimpse of the events data received from our chat endpoint (sent by the LLM)

data: {"response":"Hello","p":"abcdefghijklmnopqrstuvwxyz0123456789abcdefghij"}

data: {"response":"!","p":"abcdefgh"}

data: [DONE]

These events may contain more than one data line per event, and some events may not contain a complete JSON object (rest of the object will arrive with the next event). To handle this stream we can create a composable with a streamResponse generator function as shown below

export function useChat() {
  async function* streamResponse(
    url: string,
    messages: ChatMessage[],
    llmParams: LlmParams
  ) {
    let buffer = '';

    try {
      const response = await $fetch<ReadableStream>(url, {
        method: 'POST',
        body: {
          messages,
          params: llmParams,
        },
        responseType: 'stream',
      });

      const reader = response.pipeThrough(new TextDecoderStream()).getReader();

      while (true) {
        const { value, done } = await reader.read();

        if (done) {
          if (buffer.trim()) {
            console.warn('Stream ended with unparsed data:', buffer);
          }

          return;
        }

        buffer += value;
        const lines = buffer.split('\n');
        buffer = lines.pop() || '';

        for (const line of lines) {
          if (line.startsWith('data: ')) {
            const data = line.slice('data: '.length).trim();
            if (data === '[DONE]') {
              return;
            }

            try {
              const jsonData = JSON.parse(data);
              if (jsonData.response) {
                yield jsonData.response;
              }
            } catch (parseError) {
              console.warn('Error parsing JSON:', parseError);
            }
          }
        }
      }
    } catch (error) {
      console.error('Error sending message:', error);

      throw error;
    }
  }

  // For handling non-streaming responses
  async function getResponse() {}

  return {
    getResponse,
    streamResponse,
  };
}

To handle non streaming responses we can also create a simple $fetch call wrapper function in the same composable (check out the code in the linked Github Repo).

Now that we understand how to consume Server Sent Events and handle streaming responses, let's put all the pieces together in our main chat interface. We'll integrate the components we've built, set up the chat API call, and use our useChat composable to manage the LLM responses.

Final Chat Interface Page

The below is the final code of the index page (our app has only one page). Here we use all the components that we created before, and call the chat api endpoint. Then we use the useChat composable to handle the LLM responses.

<template>
  <div class="h-screen flex flex-col md:flex-row">
    <USlideover
      v-model="isDrawerOpen"
      class="md:hidden"
      :ui="{ width: 'max-w-xs' }"
    >
      <LlmSettings
        v-model:llmParams="llmParams"
        @hide-drawer="isDrawerOpen = false"
        @reset="resetSettings"
      />
    </USlideover>

    <div class="hidden md:block md:w-1/3 lg:w-1/4">
      <LlmSettings v-model:llmParams="llmParams" @reset="resetSettings" />
    </div>

    <UDivider orientation="vertical" class="hidden md:block" />

    <div class="flex-grow md:w-2/3 lg:w-3/4">
      <ChatPanel
        :chat-history="chatHistory"
        :loading="loading"
        @clear="chatHistory = []"
        @message="sendMessage"
        @show-drawer="isDrawerOpen = true"
      />
    </div>
  </div>
</template>

<script setup lang="ts">
import type { ChatMessage, LlmParams, LoadingType } from '~~/types';

const isDrawerOpen = ref(false);

const defaultSettings: LlmParams = {
  model: '@cf/meta/llama-3.1-8b-instruct',
  temperature: 0.6,
  maxTokens: 512,
  systemPrompt: 'You are a helpful assistant.',
  stream: true,
};

const llmParams = reactive<LlmParams>({ ...defaultSettings });
const resetSettings = () => {
  Object.assign(llmParams, defaultSettings);
};

const { getResponse, streamResponse } = useChat();
const chatHistory = ref<ChatMessage[]>([]);
const loading = ref<LoadingType>('idle');
async function sendMessage(message: string) {
  chatHistory.value.push({ role: 'user', content: message });

  try {
    if (llmParams.stream) {
      loading.value = 'stream';
      const messageGenerator = streamResponse(
        '/api/chat',
        chatHistory.value,
        llmParams
      );

      let responseAdded = false;
      for await (const chunk of messageGenerator) {
        if (responseAdded) {
          // add the response to the current message
          chatHistory.value[chatHistory.value.length - 1]!.content += chunk;
        } else {
          // add a new message to the chat history
          chatHistory.value.push({
            role: 'assistant',
            content: chunk,
          });

          responseAdded = true;
        }
      }
    } else {
      loading.value = 'message';
      const response = await getResponse(
        '/api/chat',
        chatHistory.value,
        llmParams
      );

      chatHistory.value.push({ role: 'assistant', content: response });
    }
  } catch (error) {
    console.error('Error sending message:', error);
  } finally {
    loading.value = 'idle';
  }
}
</script>

This completes bulk of the coding. The only things remaining are:

Response parsing for markdown and display
Auto scrolling the chat container

Let's tackle these in the next section.

Polishing the Chat Interface

We will finish the remaining items in our task list now and call it a day. Stay with me, it won't take long now.

Using Nuxt MDC to Parse & Display Messages

If you look at the Chats Container code in one of the previous sections you'll notice a component AssistantMessage for displaying the response. Reproducing the relevant code here

<div v-if="message.role === 'user'">
  {{ message.content }}
</div>
<AssistantMessage v-else :content="message.content" />

We use the parseMarkdown utility function from the MDC module that we included earlier to parse the content, and then use the MDCRenderer component from the same module for displaying it. To take care of streaming response we add a watch for the message content and redo the parsing with useAsyncData composable.

<template>
  <MDCRenderer class="flex-1 prose dark:prose-invert" :body="ast?.body" />
</template>

<script setup lang="ts">
import { parseMarkdown } from '#imports';

const props = defineProps<{
  content: string;
}>();

const { data: ast, refresh } = await useAsyncData(useId(), () =>
  parseMarkdown(props.content)
);

watch(
  () => props.content,
  () => {
    refresh();
  }
);
</script>

💡 We could have used the <MDC> component instead of using parseMarkdown + <MDCRenderer> combination. <MDC> handles the parsing internally using the same parseMarkdown function, then why we didn't use it?

The <MDC> component was causing overwriting of previous messages that were similar (the start of the message) to the latest message from the API endpoint. This happens because we don't have a unique key in our messages. Here is the relevant code from the <MDC> component.

const key = computed(() => hash(props.value))

const { data, refresh, error } = await useAsyncData(key.value, async () => {
  if (typeof props.value !== 'string') {
    return props.value
  }
  return await parseMarkdown(props.value, props.parserOptions)
})

As you can see, the key for useAsyncData is generated based on the content props value. So if the server streaming responses start with the same letters (e.g., Here's a joke..., Here's another...), the key will be same, and all similar previous responses from the server gets overwritten in the UI.

In the AssistantMessage component we are using useId composable to generate a unique id for each useAsyncData call, so the issue doesn't occur.

Using MutationObserver to Auto Scroll the Chats Container

In the ChatPanel component, the only scrolling part is the chats container. There can be other ways to observe the changes in the container but the simplest one I found is a MutationObserver.

💡 The MutationObserver interface provides the ability to watch for changes being made to the DOM tree.

In the case of streaming, we keep appending new content to the latest message that is being displayed. So to handle this effectively, we create a MutationObserver, give it a target to watch (the ChatsContainer), and some criteria to watch for (childList, subtree & characterData).

Here is the relevant code to do this

const chatContainer = ref<HTMLElement | null>(null);
let observer: MutationObserver | null = null;

onMounted(() => {
  if (chatContainer.value) {
    observer = new MutationObserver(() => {
      if (chatContainer.value) {
        chatContainer.value.scrollTop = chatContainer.value.scrollHeight;
      }
    });

    observer.observe(chatContainer.value, {
      childList: true,
      subtree: true,
      characterData: true,
    });
  }
});

onUnmounted(() => {
  if (observer) {
    observer.disconnect();
  }
});

When the ChatsPanel gets mounted we create a new MutationObserver which when based on the given criteria, we make the chats container scroll to its fullest.

Bonus: Handling the Dark Mode

As previously mentioned, the dark mode is already working; we just need a way to toggle it. We can also change the flavor of gray we want in the app. This can be done by adding an app.config.ts file in the app directory.

// app.config.ts
export default defineAppConfig({
  ui: {
    primary: 'orange',
    gray: 'slate',
  },
});

Add the following in your app.vue file for setting the required background colors.

<script setup lang="ts">
useHead({
  bodyAttrs: {
    class: 'bg-white dark:bg-gray-900',
  },
});
</script>

And add a new ColorMode component in your app/components directory.

<template>
  <ClientOnly>
    <UButton
      :icon="isDark ? 'i-heroicons-moon-20-solid' : 'i-heroicons-sun-20-solid'"
      color="gray"
      variant="ghost"
      aria-label="Theme"
      @click="isDark = !isDark"
    />

    <template #fallback>
      <div class="w-8 h-8" />
    </template>
  </ClientOnly>
</template>

<script setup lang="ts">
const colorMode = useColorMode();

const isDark = computed({
  get() {
    return colorMode.value === 'dark';
  },
  set() {
    colorMode.preference = colorMode.value === 'dark' ? 'light' : 'dark';
  },
});
</script>

Now you can use this color mode toggle button anywhere you like. In our app we've added it in the ChatHeader component.

Phew! And we have completed all the tasks we had set out to complete at the beginning of this article.

Deploying the App

You can deploy the project in multiple ways. I would recommend to do it through the NuxtHub Admin console. Push your code to a Github repository, link this repository with NuxtHub and then deploy from the Admin console.

But if you want to see it live right away then you can use the below command

npx nuxthub deploy

💡 If you do your first deployment with the NuxtHub CLI, you won't be able to attach your GitHub/GitLab repository later on due to a Cloudflare limitation.

For more details on deployment you can visit the NuxtHub Docs.

Source Code

I only covered the most important parts of the source code here. You can visit the linked GIthub Repo for the complete source code. It should be self explanatory, but if you have question then please feel free to drop a comment below.

ra-jeev / hub-chat

Cloudflare Workers AI LLM Playground with Nuxt

Cloudflare Workers AI LLM Playground with Nuxt 💬

Demo: https://hub-chat.rajeevs.workers.dev

Udpate (04/23/2025)

The project is now deployable on Cloudflare Workers using NuxtHub. The main branch supports workers deployment (along with Nuxt UI v3). To deploy using Cloudflare Pages + Workers, please see the pages branch.

Overview

This project is a chat interface to interact with various text generation models supported by Cloudflare Workers AI. It allows users to set different LLM parameters, toggle response streaming, handle streaming/non-streaming responses, parse markdown in responses, and includes a dark mode.

Read the blog post on how I created this LLM playground.

Features

Auto updating AI models list (uses Cloudflare API)
Select the text generation model to interact with
Set different LLM parameters (temperature, max tokens, system prompt, top_p, top_k, etc.)
Toggle LLM response streaming on/off
Handle streaming and non-streaming LLM responses on both server and client sides
Parse and display markdown…

View on GitHub

Conclusion

Congratulations! You've successfully built a feature-rich LLM playground using Nuxt 3, NuxtUI, and Cloudflare Workers AI through NuxtHub. We've covered a wide range of topics, including:

Handling streaming responses using Server Sent Events
Parsing and displaying markdown content in chat messages
Implementing auto-scrolling for a better user experience etc.

You can take this project as the starting point and improve it further by:

Adding the ability to talks to other types of LLMs, e.g. text-to-image, image-to-text, speech recognition etc.
Implementing user authentication and session management
Adding support for multiple conversation threads etc.

Thank you for staying till the end. I hope you learned some new concepts from this article. Do let me know your learnings in the comments section. Your feedback and experiences are valuable not just to me, but to the entire community of developers exploring this fascinating field.

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Beyond Gut Feeling: Leveraging AI for Smarter Domain Name Decisions

Rajeev R. Sharma — Wed, 31 Jul 2024 10:45:00 +0000

If you work in tech-related fields, sooner or later there will come a time when you have to pick a domain name for the idea you've been working on. Whether it's an app or a service, it still needs a name. And if you're anything like me, this is one of the trickiest parts of building that idea. The perfect domain name can make your project more memorable, boost SEO, and even influence user trust. But how do you know if you're making the right choice?

Traditionally, selecting a domain name has been a mix of creativity, gut feeling, and perhaps a quick poll among people in your circle of influence. The post-GPT era might have added another ingredient to the mix: consulting your trusted AI/LLM. But if you have talked to this advisor, it generally throws some names at you without revealing why it chose these names. And this is where "Name Insights" steps in, offering not just suggestions, but insights into the 'why' behind those domain name choices.

Introduction

I have been thinking about domain names (hoarding to be frank) for quite some time now. Maybe because I have bought more domains in the recent past than my usual appetite allows me to. And the first step to this whole process is: knowing the name that you want to buy.

After the usual availability checks, you wonder whether that weird-looking name is right for the purpose you're buying it for. Let's be honest, all the good ones are already taken (why does it feel like I'm not writing a tech article?) or the price is simply beyond your reach. Sometimes you zero down on more than one name, so which one should you go ahead with? These were the guiding questions behind building Name Insights.

Name Insights aims to address these challenges by providing detailed, contextual answers to help you make informed decisions about domain names.

App Features

Name Insights offers three powerful services to assist in your domain name decision-making process. Each service utilizes an AI-driven scoring system that evaluates domains based on six key aspects of what makes a good domain name. Let's explore each feature:

Name Insights & Scoring

This feature provides an unbiased view of what kind of app or service is best suited for a given domain name. It:

Evaluates the domain on 6 different parameters, namely:
1. Brand Impact: Memorability, brandability, uniqueness, emotional appeal.
2. Usability: Length, spelling simplicity, pronunciation clarity, absence of numbers/hyphens.
3. Relevance and SEO: Relevance to purpose, keyword inclusion, extension potential.
4. Technical Considerations: TLD appropriateness, potential social media availability.
5. Legal and Cultural Factors: Potential trademark risks, cultural/linguistic considerations.
6. Market Potential: Ability to target desired audience, scalability for business growth.
Provides an overall score based on weighted parameters.
Offers a brief explanation for each score.
Highlights pros and cons of the domain name.

Domain Names Comparison

This feature helps you choose between different options you might have. It:

Compares two different domain names
Uses the same scoring strategy as the insights feature
Determines a "winning" name
Provides a brief summary explaining the choice

Domain Name Ideas

This feature is most useful when you're starting from scratch. It:

Builds upon the same domain name scoring strategy, and offers five distinct domain name suggestions based on your app/service idea
Provides an overall score for each suggestion
Breaks down different category scores
Offers a brief summary of strengths and weaknesses for each name

App Results Screenshots

Name Insights Page (for dev.to domain)

Name Comparison results for similar sounding domains

Name ideas results for a generic code snippets vault app

Technical Details

Here is a brief overview of the app stack and approach

Nuxt3: Name Insights uses the full-stack capabilities of Nuxt3 as its backbone for a faster turnaround time. You can achieve the same results using another framework of your choice, but for me it made more sense because of where it is hosted (see point 2).
NuxtHub: The app is hosted on Cloudflare Pages using NuxtHub. NuxtHub offers a great DX for hosting serverless Nuxt apps, and for talking to various Cloudflare services. At the time of writing this article it supports Cloudflare features such as KV, D1, and R2.
NuxtUI: The app UI is built using NuxtUI, a collection of prebuilt components based on HeadlessUI & TailwindCSS.
Claude 3.5 Sonnet / Anthropic AI: The core features of the app are thanks to the Anthropic AI SDK, using the Claude 3.5 Sonnet model. The best in class reasoning capabilities of the model allows for a good overall experience of the app features.

At the heart of the app features is the domain scoring strategy. At present, the app relies on the advanced reasoning capabilities of the Claude 3.5 Sonnet model to grade a domain name. The key to the implementation lies in carefully crafted system prompts (which you call Prompt Engineering). These prompts instruct the AI on how to analyze domain names, what factors to consider, and how to present the results. E.g. the below is the core part of the system prompt for scoring a domain and generating insights:

const nameScorePrompt = `You are an AI assistant specialized in 
evaluating and scoring domain names. Analyze the given domain name 
as if you're seeing it for the very first time, without any prior 
knowledge of its actual use or purpose.

Evaluate the domain based on these categories and weights:
1. Brand Impact (30%)
2. Usability (20%)
3. Relevance and SEO (20%)
4. Technical Considerations (15%)
5. Legal and Cultural Factors (10%)
6. Market Potential (5%)

Provide a score out of 100 for each category, along with a brief, 
unbiased explanation. Calculate the weighted overall score out of 100 
(rounded to the nearest integer). Include concise lists of strengths 
and weaknesses of the domain name based solely on its characteristics, 
not its known use. 
`

const response = await this.anthropic.messages.create({
  model: "claude-3-5-sonnet-20240620",
  max_tokens: 1000,
  temperature: 0.4,
  system: systemPrompt,
  messages: [
    {
      role: "user",
      content: `Analyze the domain name: ${domainName}`,
    },
  ],
});

The prompt is intentionally detailed and specific to get the desired results from the LLM. We incorporate the required JSON structure into the prompt to receive the output in JSON format. This output is then parsed using the following function:

export function extractAndParseJson<T>(text: string): T {
  try {
    return JSON.parse(text) as T;
  } catch (error) {
    console.error("Error parsing JSON:", error);
  }

  // Remove the starting "\", only used for displaying the back-ticks
  const backtickPattern = \/```

(?:json)?\s*([\s\S]*?)\s*

```/g;
  const matches = text.match(backtickPattern);

  if (matches) {
    for (const match of matches) {
      const content = match.replace(backtickPattern, "").trim();
      try {
        return JSON.parse(content) as T;
      } catch (error) {
        continue;
      }
    }
  }

  throw new Error("No valid JSON found in the text");
}

To try out different LLMs from other AI services, the backend implementation is kept generic using interfaces which every AI service needs to implement and extend.

Here is the AIService interface:

export interface AIService {
  getDomainScore(domainName: string): Promise<string>;
  compareDomains(firstDomain: string, secondDomain: string): Promise<string>;
  getDomainSuggestions(purpose: string): Promise<string>;
}

And the BaseAIService

export abstract class BaseAIService implements AIService {
  protected getSystemPrompt(promptType: SystemPromptType): string {
    return getSystemPrompt(promptType);
  }

  abstract getDomainScore(domainName: string): Promise<string>;

  abstract compareDomains(
    firstDomain: string,
    secondDomain: string
  ): Promise<string>;

  abstract getDomainSuggestions(purpose: string): Promise<string>;
}

This approach provides a common system prompt for all AI services while allowing individual services the flexibility to override and define their own custom system prompts if needed.

App & Repo Links

You can try out the app live at https://name-insights.nuxt.dev/

The complete app code can be found here:

ra-jeev / domain-name-insights

AI powered tool to make your domain names decisions easier

NuxtHub Starter Template

This starter lets you get started with NuxtHub in seconds.

Documentation

Features

Image upload with hubBlob()
Save chat messages with hubDatabase()
Save server redirects with hubKV()
Cache an API response with cachedEventHandler()
Generate the API documentation with Scalar within the NuxtHub Admin

Setup

Make sure to install the dependencies with pnpm.

pnpm install

Development Server

Start the development server on http://localhost:3000:

pnpm dev

Production

Build the application for production:

pnpm build

Check out the deployment documentation for more information.

Deploy

Deploy the application on the Edge with NuxtHub on your Cloudflare account:

npx nuxthub deploy

Then checkout your server logs, analaytics and more in the NuxtHub Admin.

You can also deploy using Cloudflare Pages CI.

View on GitHub

Limitations

As the app relies entirely on an LLM for its core functionality, the output is not deterministic. This means that if you analyze the same domain name multiple times, you may receive slightly different scores. This variability is inherent to large language models, which can produce different outputs based on subtle differences in how they process the input each time.

However, in my testing, I've found that this limitation doesn't significantly impact the app's utility. The variance in scores tends to be low, and the overall insights remain consistent. Name Insights is an excellent sounding board for giving you a head start in your domain name search.

Further Enhancements

This is just the beginning. Below are some of the actions items that can further enhance the app outcome and usefulness:

Integrating other LLMs and utilize multiple models simultaneously for arriving at a score and generating insights
Adding real world data to the mix. This will make the scoring more trustworthy and accurate.
Adding real time domain name availability checks, and possibly the social media handles.

Conclusion

Selecting the perfect domain name is a critical step in establishing your online presence. With its three distinct features Name Insights aims to make the process easier at various stages of the domain name search journey. It helps you to:

Gain objective insights into the strengths and weaknesses of potential domain names
Make informed comparisons between different options
Generate fresh ideas tailored to your specific needs

Irrespective of the slight variability in results, the overall consistency and depth of analysis provided by Name Insights make it an invaluable resource for smarter domain name decisions.

I hope you liked reading the article. Do try out the app and it would mean the world to me if you can share your feedback with me.

Until next time...!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Note: This article used an AI service for rephrasing some of its text.

StoryBlobs: An adventure with Netlify Blobs

Rajeev R. Sharma — Sun, 12 May 2024 20:25:21 +0000

This is a submission for the Netlify Dynamic Site Challenge: Build with Blobs.

What I Built

StoryBlobs is collaborative story writing app. The way it works is:

A user starts a story by adding a title, the story premise and the opening paragraph, and an optional cover image
On publishing the story it becomes visible in the home feed
Now any one can read the story till now, and continue writing it after logging in

Demo

Here is the live app: StoryBlobs

Home Feed

Story Page

Create Story Page

Login Page

Mobile Display

Platform Primitives

To remain true to the challenge's objective of utilizing Netlify Platform primitives, StoryBlobs doesn't use any external database or storage.

It uses Netlify Blobs not only to store the app's users details and story blobs, but also the story cover images. Of course, the skeptic in me says this might not be ideal for a production load, but we're in it for the fun, right?

So how is everything tied up together? Let's see.

The app uses three different blob namespaces (stores, in Netlify Blobs lingo)

enum BlobStores {
  User = 'users',
  Stories = 'stories',
  Images = 'images',
}

User Data Handling

To handle email/password login, for every new user create, 2 entries are added to the users store, one using id as the key, while the other one using email. The latter is used to query against email to check for an existing user. Here is the createUser function.

export const createUser = async (user: User & { password: string }) => {
  const store = getStore(BlobStores.User);

  const existingUser = await store.getMetadata(user.email, {
    consistency: 'strong',
  });
  if (existingUser) {
    throw new Error('User already exists');
  }

  // save with email key
  await store.setJSON(user.email, user);

  // also save with id key
  await store.setJSON(user.id, user);
};

Write now I'm not doing query against id and have put it for future usage. Old habits die hard you say?

Now you can pretty much guess the login functionality. We fetch the user, verify against the hashed password and done.

export const getUserByEmail = async (email: string) => {
  const store = getStore(BlobStores.User);

  return await store.get(email, { type: 'json' });
};

Story Text Data Handling

Any story consists of one story head (the meta part), and multiple story blobs written by same or multiple people. To organize the head, and other blobs the pattern used is as shown below

// for story head
const key = '<story-slug>'

// for story blobs
const key = '<story-slug>/<blob-id>'

Having the above pattern allows me to list all parts of a story using a store list method call with prefix <story-slug>.

Here are the relevant functions for this part

Create Story

export const createStory = async (story: Story) => {
  const store = getStore(BlobStores.Stories);

  await store.setJSON(story.head.slug, story.head);

  const blobKey = `${story.head.slug}${SEPARATOR}${story.blobs[0].id}`;

  await store.setJSON(blobKey, story.blobs[0]);
};

Add to Story

export const addToStory = async (slug: string, blob: StoryBlob) => {
  const store = getStore(BlobStores.Stories);

  const blobKey = `${slug}${SEPARATOR}${blob.id}`;

  await store.setJSON(blobKey, blob);
};

Get Story by Slug

export const getStoryBySlug = async (slug: string) => {
  const store = getStore(BlobStores.Stories);

  let head;
  const parts = [];
  const { blobs } = await store.list({ prefix: slug });

  for (const blob of blobs) {
    const part = await store.get(blob.key, { type: 'json' });
    if (part.slug) {
      head = part;
    } else {
      parts.push(part);
    }
  }

  return {
    head,
    blobs: parts,
  };
};

Get All Stories

While populating the home page feed, we just want the story heads of different stories. To achieve this we can turn on the directories flag while making the store list call.

export const getAllStories = async () => {
  const store = getStore(BlobStores.Stories);

  const { blobs } = await store.list({ directories: true });
  const promises = [];
  for (const blob of blobs) {
    promises.push(store.get(blob.key, { type: 'json' }));
  }

  const stories = await Promise.all(promises);
  stories.sort((a, b) => (a.createdAt > b.createdAt ? -1 : 1));
  return stories;
};

Story Cover Image Handling

While publishing a story we first save the image as a blob with a new key of the form of ${cuid}.${image file extension}. Here are the relevant code parts for image handling

Image Upload from Nuxt Frontend

const uploadCover = async () => {
  // selectedFile.value is a File object
  if (selectedFile.value) {
    const formData = new FormData();
    formData.append('file', selectedFile.value);

    const res = await $fetch('/api/images/upload', {
      method: 'POST',
      body: formData,
    });

    return res.fileName;
  }
};

Image Upload Server Side

// ...
const formData = await readFormData(event);
const file = formData.get('file') as File;
const extension = file.name.split('.').pop();
const fileName = `${createId()}.${extension}`;

await saveImage(fileName, file);
// ...

The fileName created above is used as the key for saving the blob

export const saveImage = async (key: string, file: File) => {
  const store = getStore(BlobStores.Images);

  return await store.set(key, file);
};

While reading back the image file we read it as a blob so that everything works as expected

export const getImage = (key: string) => {
  const store = getStore(BlobStores.Images);

  return store.get(key, { type: 'blob' });
};

Netlify Image CDN

Apart from Netlify Blobs, the app uses Netlify Image CDN to optimize image display. To do this, it uses the @nuxt/image package which is integrated with Netlify Image CDN. This makes it easier to do image manipulation on the fly.

The Complete Source Code

ra-jeev / story-blobs

Nuxt UI Minimal Starter

Look at Nuxt docs and Nuxt UI docs to learn more.

Setup

Make sure to install the dependencies:

# npm
npm install

# pnpm
pnpm install

# yarn
yarn install

# bun
bun install

Development Server

Start the development server on http://localhost:3000:

# npm
npm run dev

# pnpm
pnpm run dev

# yarn
yarn dev

# bun
bun run dev

Production

Build the application for production:

# npm
npm run build

# pnpm
pnpm run build

# yarn
yarn build

# bun
bun run build

Locally preview production build:

# npm
npm run preview

# pnpm
pnpm run preview

# yarn
yarn preview

# bun
bun run preview

Check out the deployment documentation for more information.

View on GitHub

Problems Faced

Some of the problems I faced during the development of StoryBlobs.

Nuxt Blobs Configuration Context

StoryBlobs is a Nuxt app deployed on Netlify (the whole app is deployed as a Netlify function) yet Netlify Blobs doesn't work out of the box. I got some configuration errors.

As per the Netlify Blobs repo, configuration context can be read automatically from the execution environment. Since that didn't happen, I assume that the server function created for the app is in Lambda Compatibility mode. So I need to call connectLambda(event) method. But the event types are a mismatch.

Instead of putting more effort into finding out a solution, I simply set the NETLIFY_BLOBS_CONTEXT environment variable which is a base64 encoded string created using the below code

const obj = {
  siteId: "<your_site_id>",
  token: "<your_personal_access_token>" //created from Netlify Dashboard
}

const val = btoa(JSON.stringify(obj))

Things started working as expected after setting the environment variable.

Do not forget to redeploy if no code was changed.

Lack of Directories flag support for local development

We can use almost every functionality of Netlify Blobs locally by using the Netlify CLI and running the netlify dev command from the app root dir. This sets the needed configuration context for Netlify Blobs.

But the pattern I used above (directories) doesn't work locally. On using the '/' separator locally, I got 500 Status Code from Netlify Blobs. As an alternative, I tried using the '#' as separator and then querying with prefix ${slug}#, but that also didn't work and returned all entries starting with ${slug}. So had to put some workarounds for local testing.

Before we part

Despite the above issues, working with Netlify Blobs was quite smooth. If they can support sorting the listing results natively, and maybe add a bulk get method, one can go quite far using Netlify Blobs :-).

I hope you enjoyed reading the post. Please do say "hi" in the comments section, to keep the conversation going.

Until next time...

Web Share Target API: One Byte Explainer ✍️

Rajeev R. Sharma — Sat, 30 Mar 2024 17:22:25 +0000

This is a submission for DEV Challenge v24.03.20, One Byte Explainer: Browser API or Feature.

Explainer

The experimental Web Share Target API allows installed PWAs (Progressive Web Apps) to act as a content sharing target.

You use share_target entry in your web app manifest to register as a share target and appear in the system's share dialog.

Additional Context

As this is an experimental API, the adoption is low.

For more information visit the official specification draft, and the MDN docs

Other Resources

Learn how to share contents from your web apps: Web Share APIs

Web Share API: One Byte Explainer ✍️

Rajeev R. Sharma — Fri, 29 Mar 2024 16:24:02 +0000

This is a submission for DEV Challenge v24.03.20, One Byte Explainer: Browser API or Feature.

Explainer

Web Share APIs are used to share text, links, files and other contents in web apps by showing the native share dialogs.

navigator.canShare() validates content shareability, and navigator.share() opens the native share dialog.

Additional Context

Web Share APIs use the web-share permission policy, and your app must have the permission granted for the API to work. Also, the sharing must be triggered by a UI event (e.g. button click).

For more information you can visit the MDN docs

Other Resources

A related real life issue I faced while using the Web Share API & the clipboard API
Learn how to make your app receive the shared content and appear in the native share dialogs: Web Share Target API

What I Want from 2024

Rajeev R. Sharma — Tue, 16 Jan 2024 13:10:52 +0000

Photo by Patrick Fore on Unsplash

Do you ever experience déjà vu? For me, it's a regular phenomenon as soon as the calendar flips to the second half of December each year. Like clockwork, a voice in my head begins popping up—now and again—to ask, "What did you achieve this year?" The question shifts to its future tense form after the 1st of January, and typically, after a couple of weeks into the new year, the voice dies down, only to resurface again after 11 months.

This time around, it was different. The voice was present from early November itself, and I think it will remain there for quite some time. The question is also different: "What have you achieved till now, and what do you want to do?" Maybe my recent complications with my eye have a role to play in this, or the lack of satisfactory answers for the year-end voices, or maybe both—who knows?

Well, I have no answer for this changed question either, but I am willing to take a stab at a question with a reduced scope—after all, we love reduced scope, don't we?—"What do I want from 2024?"

If 2024 is listening—don't come at me for using "listening" when "reading" would have been more appropriate—here are the top 3 things I want from you:

The light: To see a few steps—for starting something in the physical world—if the complete path is asking too much.
The strength: To continue in the tech space by creating and launching new stuff—both free and paid—with at least one generating some revenue.
The ability: To write and produce some quality content—mostly tech—and contribute to the community

No want is ever complete without giving something in return. So, what I'm willing to give in return for this?

My 100% commitment and willpower
My self—doubts
My frustrations and anger

I know this may not seem like a fair bargain to you, but in return, you'll also receive a strong backlink from the future me: "2024 is the year that changed everything for me".

Of course, you may not always get what you want, and it's difficult to give or let go of the things that you have, but there's no harm in trying, is there?

Creating a Quiz Generator using the ChatGPT Firebase Extension

Rajeev R. Sharma — Tue, 28 Nov 2023 15:00:00 +0000

You often want to test your knowledge whenever you're learning a new topic. You can get ready-made quizzes on the subject, but generally, they tend to be too broad. What if you only want to test yourself on the content you've just read? This is the problem statement we will tackle in this article: given some content, how can you create a quiz based on it?

Introduction

While helping my son with his studies, I often struggle with creating questions to ask him to test his knowledge. It has more to do with the idea of reading the content myself first, which I find tedious. I can ask the questions at the chapter's end, but that doesn't feel comprehensive enough. In these situations, I often think, "If only someone could read the chapter for me, comprehend the content, and generate relevant questions along with the answer keys".

With the advancements in AI and the recent interest in its generative capabilities (ChatGPT, Bard, etc.), I wondered if such a solution is possible using AI. To keep the experiment short, I looked for a low-code solution to achieve this, and Firebase extensions seemed like the right choice.

What is generative AI?

As the name suggests, generative AI is a type of artificial intelligence that can create new content or data based on its learning from existing data. This new content or data can be similar to the existing data or be entirely original. It usually works by giving some content to the AI along with your instructions (generally called a prompt) on how to use that content, and the AI gives back an appropriate response.

Now that you know about generative AI, you should have a basic idea about the approach you'll need to take to get the desired results. Let's break it down into steps

Get the text content from the user for which they want to generate questions
Pass the content along with a suitable instruction set to the AI
Parse the response from the AI and present it to the user in a suitable format

Implementation

The implementation of the quiz generator is quite simple: use a textarea to get the text content from the user and pass it to the configured ChatGPT Firebase extension. You can configure the said extension as shown below.

Configuring the ChatGPT extension

Go to the Firebase extensions hub, search for chat in the search box, and click "Chatbot with ChatGPT" from the search results.

You can install this extension from the extension page in one of your Firebase projects. Click on the "Install in Firebase console" button, select your project from the new tab that opens up, and complete the steps for installing the extension into the project.

Click the Next button. This extension creates a cloud function called to generate a response, and we need to enable the Secrets Manager API to store our OpenAI API Key securely. Based on the current state of your project, you may need to enable other APIs (Cloud Functions, Artifact Registry, etc.). Click Enable and then click Next

Next, the extension needs to use Cloud Firestore (If you haven't created a Cloud Firestore yet, it will create one for you in datastore mode. For using it with Firebase, you'll need to change it back to the native mode from Google Cloud Console) and Secrets Manager to process the input text, the OpenAI API response, and the stored OpenAI API key. Click Next

Finally, we need to configure the extension. You need to configure the following

Add your OpenAI API Key (you can create it here). Make sure to click on the Create Secret button to create a secret in the Secrets Manager
Select the desired model (GPT 3.5/4)
Set the temperature to a low value (0.2-0.4). The temperature value (allowed range 0-2) dictates the randomness of the AI response; higher values make the output more random. I used a temperature of 0.3.
Set the cloud function location (ideally set it to the exact location where your Cloud Firestore is located)
You'll also need to configure the collection path and the field names where the input text and the API response will be stored. For this test, I used requests as the collection name and left the field names as it is.

We can leave the rest of the settings as it is and click on the Install extension button. Wait for the installation to complete; now we're ready to test it.

Testing the extension

To test the extension you need to create the collection that you configured in the last section (requests in my case) and add a prompt field to it.

The way this extension works is:

You add the input text and your instructions to the prompt field
The cloud function (generateAIResponse) created by the extension gets invoked automatically
The function sends your prompt to the OpenAI API and creates a status field (of type map) with its state key set to PROCESSING
On a successful response from the API call, the state is changed to COMPLETED and the API call output is stored in a new field named response

Prompt Structure

How do you structure your prompt? A good prompt should start by setting a context so that ChatGPT behaves accordingly. Then, you state your requirement, followed by the input text, and finally, you give a response cue to the AI.

To summarize, you can use the below prompt structure

Context + Instruction + Input Text + Response Cue

For the quiz generator you can use the following initial prompt

“You're a teacher who needs to create quizzes to test your student's
knowledge. Using the given text content create several distinct MCQs for the
purpose.

The content: {{text_content}}

Your response:”

The above prompt generates a response in a format similar to the one below. Please note that the format may vary slightly for you.

1. question text
    a) Option 1
    b) Option 2
    ...
...

This is a very good start, but the API is not returning the answers to these questions. Also, it is difficult to use it as a quiz as the response lacks structure.

Structuring the response

For better control over the whole thing, you need a structured response, such as a JSON formatted output, from the ChatGPT API call. Tweak the instruction part of your prompt to the following:

Using the given text content creates several distinct MCQs for the purpose. 
Return your response as an RFC8259-compliant JSON array using the structure
[{"question": "the question", "choices": ["choice 1", "choice 2", ...],
"answer": "the correct choice"}, {...}]

Here I’ve intentionally removed any spaces from the JSON to save some tokens from the response. Rerunning the same test with the changed instruction gives us the desired structured result. Now, you can display the questions & the choices in any way you want using your favorite technology. As you have the answer key, you can also convert these questions into a quiz.

Note: Once in a while, you might notice some discrepancy in the API responses. You can further play around with your instructions to eliminate or minimize such cases.

Using images as content

You can extend it further and use images that contain text (say, a snapshot of a book's pages) as the starting point of the quiz generator. The underlying approach of the solution remains the same; you're just adding a step at the beginning. Using another Firebase extension, you can extract the text from an image and use that as input like before.

Configuring the Cloud Vision extension

Go back to the Firebase extensions hub, search for vision, and click on "Extract Image Text with Cloud Vision AI" from the search results. Install the extension by following the steps you completed for the previous extension.

This extension enables the Google's Cloud Vision API and creates a new cloud function named extractText. Apart from the Cloud Firestore User permission, it also needs the Storage Admin permission (the extension reads the images from Cloud Storage).

Finally, configure the extension as follows (you should keep your cloud function's location near your Cloud Storage/Firestore's location). You can leave the rest of the settings as it is and install the extension

Once the extension has finished installing, you can test it out by uploading an image to your cloud storage bucket under the snapshots folder. A few seconds after adding the image, you should see a new collection, extractedText with a single document containing the text that was extracted from the image.

For my test, I used the following screenshot

And this is the result of the Cloud Vision extension

You can use this text as input to the ChatGPT extension and generate questions as before.

Conclusion

Firebase extensions are powerful low-code options for many different use cases. As we saw in this article, leveraging the power of ChatGPT and Cloud Vision extensions could make the basic quiz generator functionality work without writing a single line of code.

Now, you can integrate Cloud Storage and Firestore into your front end to create a personalized and engaging quiz generator that generates relevant questions based on the given content.

I hope you liked reading the article. If you've questions or just want to say hi, just drop a message in the comments section.

-- Keep adding the bits, soon you'll have more bytes than you may need. :-)

Creating an OpenAI powered Writing Assistant for VS Code

Rajeev R. Sharma — Tue, 20 Jun 2023 13:12:36 +0000

Rabbit holes are many, and as a developer, you keep falling into one or the other during your journey. This is not necessarily a bad thing—granted that it may frustrate you—that is how you learn something extra (the 0.01 of 1.01³⁶⁵). This is one such story where my urge to create something new pushed me into writing a brand new VS Code extension.

Introduction

I've always wondered how a VS Code extension works and what it takes to implement one. Recently this urge got the best of me and I decided to finally create a VS Code extension. But what to build? I like using Hashnode's AI editor for rephrasing texts and other writing assistance, so I thought why not implement a similar functionality for VS Code? Since writing assistance is most useful for a markdown file (which can also work as a blog post), I created a basic writing assistant for markdown (and text) files.

Here is a simple GIF showcasing how the extension works.

Ready to dive into how I implemented it? Let's get started.

Getting Started

Before we can start building the extension, we need to gather and prepare the necessary tools. In this case, the needed tools are node, git, yeoman and generator-code. For a newcomer like myself, this basic tutorial is perfect. I recommend going through it to learn the fundamentals.

Run yo code command and pick New Extension (js/ts) from the choices as shown below

I haven't selected the webpack option yet. We can always do so later on. This is my current directory structure.

Implementing the extension

We're mostly interested in the package.json and extension.ts files while building the extension. The important fields in the package.json file are

activationEvents: When should our extension be activated
main: The entry point of the extension code (the entry is in the out folder which gets generated when you debug or run the extension)
contributes: What does this extension contributes to the VS Code commands and settings

For my extension I wanted an experience similar to the Hashnode AI Editor, so adding commands to the VS Code command palette was not what I was after. What helped me here was the sample extensions directory on GitHub. Their code-actions sample was exactly what I had in mind (and it targets only the markdown files).

Target Markdown and Text files

To target specific types of files you need to change the activationEvents. Since we want to work only with Markdown and Text files at the moment, this is what my package.json says

// ...
"activationEvents": [
  "onLanguage:markdown",
  "onLanguage:plaintext"
],
"main": "./out/extension.js",
"contributes": {},
// ...

This extension will only get activated for the languages mentioned above.

Also, since we don't want any command palette commands, we can remove everything under the contributes key.

Showing the actions in the light bulb menu

There is one function called activate inside the extension.ts file which is of interest. This is where you need to write your code. Notice that I've removed all the unnecessary boilerplate code

// This method is called when your extension is activated
// Your extension is activated the very first time the 
// command is executed
export function activate(context: vscode.ExtensionContext) {}

There is another complementary function called deactivate which can be used if you need to do any kind of resource cleaning before the extension is deactivated.

If you run/debug the extension now, and create a new markdown file in the new VS Code window which pops up, you won't notice any difference as there is no palette command, and also there is no code in the activate function. Let's change that and add the following in the extension.ts file

class MyCodeActionProvider implements vscode.CodeActionProvider {

  provideCodeActions(
    document: vscode.TextDocument,
    range: vscode.Range | vscode.Selection,
    context: vscode.CodeActionContext,
    token: vscode.CancellationToken
  ): vscode.ProviderResult<(vscode.CodeAction | vscode.Command)[]> {
    console.log('inside the provideCodeActions method');
    throw new Error('Method not implemented.');
  }
}

export function activate(context: vscode.ExtensionContext) {
  const actionProvider = vscode.languages.registerCodeActionsProvider(
    ['markdown', 'plaintext'],
    new MyCodeActionProvider(),
    {
      providedCodeActionKinds: [
        vscode.CodeActionKind.RefactorRewrite,
        vscode.CodeActionKind.QuickFix,
      ],
    }
  );

  context.subscriptions.push(actionProvider);
}

What we're doing here is informing VS Code about the kind of code actions we're providing which include RefactorRewrite & QuickFix. The actual "commands/code actions" need to be provided by the provideCodeActions method of the MyCodeActionProvider class. If you debug the extension now you should see the below console log in your original project window's debug console.

inside the provideCodeActions method

We're making progress. Replace the provideCodeActions method's code with the following

provideCodeActions(
  document: vscode.TextDocument,
  range: vscode.Range | vscode.Selection,
  context: vscode.CodeActionContext,
  token: vscode.CancellationToken
): vscode.ProviderResult<(vscode.CodeAction | vscode.Command)[]> {
  // If there is nothing selected, we won't provide any action
  if (range.isEmpty) {
    return;
  }

  // supported actions and their kinds
  const actions = [
    {
      id: 'rephrase',
      title: 'Rephrase selected text',
      kind: vscode.CodeActionKind.QuickFix,
    },
    {
      id: 'headlines',
      title: 'Suggest headlines',
      kind: vscode.CodeActionKind.QuickFix,
    },
    {
      id: 'professional',
      title: 'Rewrite in professional tone',
      kind: vscode.CodeActionKind.RefactorRewrite,
    },
    {
      id: 'casual',
      title: 'Rewrite in casual tone',
      kind: vscode.CodeActionKind.RefactorRewrite,
    },
  ];

  const cActions = [];
  // prepare the code actions for the above actions
  for (const action of actions) {
    const cAction = new vscode.CodeAction(action.title, action.kind);
    cAction.command = {
      command: `my-shiny-extension.${action.id}`,
      title: action.title,
      arguments: [action.id],
    };

    cActions.push(cAction);
  }

  return cActions;
}

Debug/run the extension now and you should see the above actions in the bulb tooltip when you select some text in a markdown/text file.

Of course, nothing will happen if you click on any of these actions. This is because we've only created the actions, but haven't written any code to handle them.

Handling the Code Actions

To handle the actions we need to register these commands with the VS Code extension context. This can be done inside the activate function. Let's do a little bit of refactoring.

Move the actions out of the provideCodeActions method and make it a class property of MyCodeActionProvider

public static readonly actions = [
  {
    id: 'rephrase',
    title: 'Rephrase selected text',
    kind: vscode.CodeActionKind.QuickFix,
  },
  {
    id: 'headlines',
    title: 'Suggest headlines',
    kind: vscode.CodeActionKind.QuickFix,
  },
  {
    id: 'professional',
    title: 'Rewrite in professional tone',
    kind: vscode.CodeActionKind.RefactorRewrite,
  },
  {
    id: 'casual',
    title: 'Rewrite in casual tone',
    kind: vscode.CodeActionKind.RefactorRewrite,
  },
];

Change its reference inside the provideCodeActions method to

for (const action of MyCodeActionProvider.actions) {
  // ...
}

Add a new method handleAction which will handle the actions when a user clicks on them. The actionId argument will be passed by the caller. Remember we had passed arguments: [action.id] while returning the code actions from the providecodeActions method?

handleAction(actionId: string) {
  console.log(`handleAction for ${actionId}`);
}

Now change the activate function as shown below

export function activate(context: vscode.ExtensionContext) {
  const myActionProvider = new MyCodeActionProvider();
  const actionProvider = vscode.languages.registerCodeActionsProvider(
    ['markdown', 'plaintext'],
    myActionProvider,
    {
      providedCodeActionKinds: [
        vscode.CodeActionKind.RefactorRewrite,
        vscode.CodeActionKind.QuickFix,
      ],
    }
  );

  context.subscriptions.push(actionProvider);
  for (const action of MyCodeActionProvider.actions) {
    context.subscriptions.push(
      // use the same id which we used in the command field 
      // of the code actions
      vscode.commands.registerCommand(
        `my-shiny-extension.${action.id}`,
        (args) => myActionProvider.handleAction(args)
      )
    );
  }
}

For all the actions which we support, we're registering a corresponding command with the VS Code extension context. The command id that we use here must match the command id which we returned from the provideCodeActions method.

If we run/debug the extension now and click any of our actions from the light bulb menu we should see the corresponding console log in the debug console.

Integrating with the OpenAI APIs

Now the only thing remaining is: using the OpenAI APIs to make changes to any written text. Let's get it over with.

Add the OpenAI library to the codebase

yarn add openai

Import it into the extension.ts file

import { OpenAIApi, Configuration } from 'openai';

And replace the handleAction method's code with the following

async handleAction(actionId: string) {
  const editor = vscode.window.activeTextEditor;
  if (
    !editor ||
    editor.selection.isEmpty ||
    !['rephrase', 'headlines', 'professional', 'casual'].includes(actionId)
  ) {
    // return if no active editor, or no active selection 
    // or if unsupported actionId passed
    return;
  }

  // Create the OpenAI Service
  const openAiSvc = new OpenAIApi(
    new Configuration({
      apiKey: '<your_open_ai_api_key>',
    })
  );

  // Get the currently selected text
  const text = editor.document.getText(editor.selection);
  // current selection range
  let currRange = editor.selection;

  try {
    // Adding a filler/loading text before making the API call
    const fillerText = '\n\nThinking...';
    editor
      .edit((editBuilder) => {
        // insert the filler text after the current selection end
        editBuilder.insert(currRange.end, fillerText);
      })
      .then((success) => {
        if (success) {
          // Select the filler text now
          editor.selection = new vscode.Selection(
            editor.selection.end.line,
            0,
            editor.selection.end.line,
            editor.selection.end.character
          );

          // store this new selection range
          currRange = editor.selection;
        }
      });

    // Create the prompt prefix based on the action id
    let promptPrefix = '';
    switch (actionId) {
      case 'rephrase':
        promptPrefix =
          'Rephrase the following text and make the sentences more clear and readable';
        break;
      case 'headlines':
        promptPrefix = 'Suggest some short headlines for the following text';
        break;
      case 'professional':
        promptPrefix =
          'Make the following text better and rewrite it in a professional tone';
        break;
      case 'casual':
        promptPrefix =
          'Make the following text better and rewrite it in a casual tone';
        break;
    }

    // Make the OpenAI API Call using the desired model and configs
    /* eslint-disable @typescript-eslint/naming-convention */
    const response = await openAiSvc.createCompletion({
      model: 'text-davinci-003',
      prompt: `${promptPrefix}:\n\n${text}\n\n`,
      temperature: 0.3,
      max_tokens: 500,
      frequency_penalty: 0.0,
      presence_penalty: 0.0,
      n: 1,
    });
    /* eslint-enable @typescript-eslint/naming-convention */

    // We'd reuqested for only one result, use that
    let result = response.data.choices[0].text;
    editor
      .edit((editBuilder) => {
        if (result) {
          // replace the filler text with the actual result
          editBuilder.replace(
            new vscode.Range(currRange.start, currRange.end),
            result.trim()
          );
        }
      })
      .then((success) => {
        if (success) {
          // Select the resulting text (the text can be longer
          // and span over multiple lines, so we treat it 
          // appropriately to make a complete selection)
          editor.selection = new vscode.Selection(
            currRange.start.line,
            currRange.start.character,
            currRange.end.line,
            editor.document.lineAt(currRange.end.line).text.length
          );

          return;
        }
      });
  } catch (error) {
    console.error(error);
  }

  // In case of API error, show an error text instead
  editor.edit((editBuilder) => {
    editor.selection = new vscode.Selection(currRange.start, currRange.end);
    editBuilder.replace(editor.selection, 'Failed to process...');
  }):
}

And we're done with the code. If you run/debug the extension you should see appropriate text replacements for your text. Running it on a couple of my sentences gives me the following results

As always, you can play around with the prompts and get better consistent output from the OpenAI API.

Adding Extension Settings

You may have noticed that we've added the OpenAI API key directly in the code. We should move it to VS Code settings under our extension name. To do that we need to change the contributes key in the package.json file. While we're doing that we can also move the maxTokens property instead of hardcoding it to 500 in the code.

// ..
"contributes": {
  "configuration": {
    "title": "My Shiny Extension",
    "properties": {
      "myShinyExtension.openAiApiKey": {
        "type": "string",
        "default": "",
        "description": "Enter you OpenAI API Key here"
      },
      "myShinyExtension.maxTokens": {
        "type": "number",
        "default": 1200,
        "description": "Enter the maximum number of tokens to use for each OpenAI API call"
      }
    }
  }
},
// ..

Now these two entries will appear under "My Shiny Extension" in the VS Code settings. To read the values of these entries we can use the below code (put it just above the OpenAI service creation).

const configs = vscode.workspace.getConfiguration('myShinyExtension');
const openAIApiKey = configs.get<string>('openAiApiKey');
const maxTokens = configs.get<number>('maxTokens');
if (!openAIApiKey) {
  vscode.window.showErrorMessage(
    'Missing OpenAI API Key. Please add your key in VSCode settings to use this extension.'
  );

  return;
}

Do remember that we're creating the OpenAI service instance on each API call. This is not optimal and you should move it to the constructor and handle the error scenarios appropriately.

Conclusion

As is evident from the article, developing a VS Code extension can be easy and fun. If you're observant you can recreate an existing thing (Hashnode AI editor in this case) in your way someplace else, and learn a ton along the way.

To publish an extension we need to complete a few more steps and optionally bundle it using webpack or a suitable bundler. To learn more about the process you can visit these links: 1. publishing an extension, 2. Bundling an extension

This was just a sneak peek of the extension I created. If interested, you can look at the complete source code of the extension here.

If you want to try out the extension (Write Assist AI), you can get it from here.

I hope you liked reading the article. If you found any mistakes in the article, please let me know in the comments. Your suggestions and feedback are most welcome.

-- Keep adding the bits, soon you'll have more bytes than you may need. :-)

DEV Community: Rajeev R. Sharma

Building Brew Haven: A/B Testing My Coffee Shop Dreams with DevCycle

What I Built

Demo

Video Demo

Showcasing Feature Flags in Brew Haven

1. The Coffee Menu

2. Payment and Ordering

3. A/B Testing with Promotions

App Screenshots

Home Page

Menu Page

Checkout Page

Admin Page

My Code

ra-jeev / brew-haven

Dummy Coffee Shop App with DevCycle Feature Flags

Brew Haven ☕

Project Overview

Key Features

Technologies Used

Prerequisites

Environment Setup

Local Development

Deployment

Technologies Used

Client Side Usage of DevCycle SDK

Interacting with DevCycle Management APIs

My DevCycle Experience

Additional Prize Categories

Wrapping Up

This is a simple app to capture your voice notes that auto transcribes the recordings, and optionally refines the transcriptions, using AI. This is a detailed guide on how to use Cloudflare Workers AI, along with other services to build this app.

Building Vhisper: Voice Notes App with AI Transcription and Post-Processing

Rajeev R. Sharma ・ Dec 2 '24

Building Vhisper: Voice Notes App with AI Transcription and Post-Processing

Project Overview

Project Setup

Prerequisites

Project Init

Testing the Setup

Building the Basic Backend

What is NuxtHub?

/api/transcribe Endpoint

/api/upload Endpoint

Defining the notes Table Schema

/api/notes Endpoints

Creating DB Migrations

Creating the Basic Frontend

useMediaRecorder Composable

NoteEditorModal Component

NoteRecorder Component

AudioVisualizer Component

useRecordings Composable

Creating the Home Page

NoteCard component

Serving the Audio Recordings

Further Enhancements

Deploying the Application

Deploy via NuxtHub Admin

Deploy via NuxtHub CLI

Source Code

ra-jeev / vhisper

Voice Notes with AI transcriptions and post processing

Vhisper - In-browser Voice Notes

Try it Out

Preview

Key Features

Technologies Used

Conclusion

Rethinking GitHub Search: How I built a Chat Interface to search GitHub

Why Chat GitHub?

How it Works?

Project Setup

Tech Stack

Prerequisites

Setting up the project

Configs and Environment Variables

From User Query to GitHub API Call

1. Interpreting the User Query

2. Using Tool Calls for GitHub Search

`/api/transcribe` Endpoint

`/api/upload` Endpoint

Defining the `notes` Table Schema

`/api/notes` Endpoints

`useMediaRecorder` Composable

`NoteEditorModal` Component

`NoteRecorder` Component

`AudioVisualizer` Component

`useRecordings` Composable

`NoteCard` component