DEV Community: Saeed

Real-Time Updates with Convex DB and Authentication using Clerk in Next.js 🔥🔥

Saeed — Tue, 08 Apr 2025 07:08:10 +0000

In modern web applications, real-time functionality and secure authentication are crucial features. In this post, I'll show you how to combine Convex DB's powerful real-time capabilities with Clerk's authentication system to build a responsive Next.js application.

What We'll Build
We'll create a simple collaborative task list where:

Users can sign in with Clerk
All connected users see updates in real-time
Changes persist in Convex DB
UI updates instantly without page refreshes

Prerequisites

Node.js installed
Basic knowledge of Next.js
Convex and Clerk accounts (both have generous free tiers)

**Setting Up the Project
**First, create a new Next.js app:

npx create-next-app@latest convex-clerk-demo
cd convex-clerk-demo

Installing Dependencies

npm install convex @clerk/nextjs @clerk/themes
npm install convex-dev @clerk/types --save-dev

Configuring Clerk
Go to Clerk Dashboard and create a new application
Copy your publishable key and add it to your .env.local file:

NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_publishable_key
CLERK_SECRET_KEY=your_secret_key
NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in
NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up
NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL=/
NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL=/

Update your next.config.js:

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    serverActions: true,
  },
};

module.exports = nextConfig;

Create a middleware.js file at your project root:

import { authMiddleware } from "@clerk/nextjs";

export default authMiddleware({
  publicRoutes: ["/", "/sign-in(.*)", "/sign-up(.*)"],
});

export const config = {
  matcher: ["/((?!.*\\..*|_next).*)", "/", "/(api|trpc)(.*)"],
};

Wrap your app with the ClerkProvider in app/layout.tsx:

import { ClerkProvider } from '@clerk/nextjs'
import { dark } from '@clerk/themes'

export default function RootLayout({
  children,
}: {
  children: React.ReactNode
}) {
  return (
    <ClerkProvider appearance={{ baseTheme: dark }}>
      <html lang="en">
        <body>{children}</body>
      </html>
    </ClerkProvider>
  )
}

Setting Up Convex

Install the Convex CLI globally:

npm install -g convex-dev

Initialize Convex in your project:

npx convex init

Follow the prompts to create a new project or link to an existing one
Add your Convex deployment URL to .env.local:

NEXT_PUBLIC_CONVEX_URL="https://your-app-name.convex.cloud"

Start the Convex dev server:

npx convex dev

Defining Our Data Schema

Create convex/schema.ts:

import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  tasks: defineTable({
    text: v.string(),
    completed: v.boolean(),
    userId: v.string(),
  }).index("by_user", ["userId"]),
});

Creating Convex Mutations and Queries

Create convex/tasks.ts for our task operations:

import { mutation, query } from "./_generated/server";
import { v } from "convex/values";

export const getTasks = query({
  args: {},
  handler: async (ctx) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) return [];

    const userId = identity.subject;
    return await ctx.db
      .query("tasks")
      .withIndex("by_user", q => q.eq("userId", userId))
      .collect();
  },
});

export const addTask = mutation({
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) throw new Error("Not authenticated");

    const userId = identity.subject;
    await ctx.db.insert("tasks", { 
      text, 
      completed: false, 
      userId 
    });
  },
});

export const toggleTask = mutation({
  args: { id: v.id("tasks") },
  handler: async (ctx, { id }) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) throw new Error("Not authenticated");

    const task = await ctx.db.get(id);
    if (!task) throw new Error("Task not found");
    if (task.userId !== identity.subject) {
      throw new Error("Not authorized");
    }

    await ctx.db.patch(id, { completed: !task.completed });
  },
});

Building the UI
Create app/page.tsx:

"use client";

import { useQuery, useMutation, useConvexAuth } from "convex/react";
import { api } from "@/convex/_generated/api";
import { SignInButton, UserButton, useUser } from "@clerk/nextjs";
import { useState } from "react";

export default function Home() {
  const { isAuthenticated } = useUser();
  const { isLoading } = useConvexAuth();

  const tasks = useQuery(api.tasks.getTasks) || [];
  const addTaskMutation = useMutation(api.tasks.addTask);
  const toggleTaskMutation = useMutation(api.tasks.toggleTask);

  const [newTaskText, setNewTaskText] = useState("");

  if (isLoading) return <div>Loading...</div>;

  const handleAddTask = async (e: React.FormEvent) => {
    e.preventDefault();
    if (!newTaskText.trim()) return;
    try {
      await addTaskMutation({ text: newTaskText });
      setNewTaskText("");
    } catch (err) {
      console.error("Error adding task:", err);
    }
  };

  const handleToggleTask = async (id: string) => {
    try {
      await toggleTaskMutation({ id });
    } catch (err) {
      console.error("Error toggling task:", err);
    }
  };

  return (
    <main className="max-w-2xl mx-auto p-4">
      <div className="flex justify-between items-center mb-8">
        <h1 className="text-2xl font-bold">Collaborative Task List</h1>
        {isAuthenticated ? <UserButton afterSignOutUrl="/" /> : <SignInButton />}
      </div>

      {isAuthenticated ? (
        <>
          <form onSubmit={handleAddTask} className="mb-6 flex gap-2">
            <input
              type="text"
              value={newTaskText}
              onChange={(e) => setNewTaskText(e.target.value)}
              placeholder="Enter a new task"
              className="flex-1 p-2 border rounded"
            />
            <button 
              type="submit" 
              className="bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600"
            >
              Add Task
            </button>
          </form>

          <ul className="space-y-2">
            {tasks.map((task) => (
              <li 
                key={task._id} 
                className="flex items-center gap-4 p-2 border rounded"
              >
                <input
                  type="checkbox"
                  checked={task.completed}
                  onChange={() => handleToggleTask(task._id)}
                  className="h-5 w-5"
                />
                <span className={task.completed ? "line-through text-gray-500" : ""}>
                  {task.text}
                </span>
              </li>
            ))}
          </ul>
        </>
      ) : (
        <div className="text-center py-8">
          <p className="mb-4">Sign in to view and manage your tasks</p>
          <SignInButton mode="modal">
            <button className="bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600">
              Sign In
            </button>
          </SignInButton>
        </div>
      )}
    </main>
  );
}

Enabling Real-Time Updates
The magic of Convex is that our UI already updates in real-time! The useQuery hook automatically subscribes to changes in the underlying data. When any client makes a change through a mutation, all subscribed clients will receive the update instantly.

Deploying Your Application

Deploy your Convex backend:

npx convex deploy

Deploy your Next.js application to Vercel or your preferred hosting provider.

Security Considerations
Notice how we:

Validate authentication in all Convex functions
Check ownership before modifying tasks
Only return tasks belonging to the current user

This ensures data privacy and security

Advanced Features to Explore

Presence: Show which users are currently online
Optimistic Updates: Improve perceived performance
Realtime Notifications: Alert users of changes
Collaborative Editing: Multiple users editing the same item

References
Convex Documentation
Clerk Documentation
Next.js Documentation

Conclusion
By combining Convex's real-time database with Clerk's authentication, we've built a secure, collaborative application with minimal boilerplate. The integration is seamless and provides a great developer experience while offering powerful features to end users.

Give it a try and let me know in the comments what you're building with this stack!

Optimizing Line Chart Performance with LTTB Algorithm

Saeed — Sat, 01 Feb 2025 16:43:10 +0000

When working with large datasets, rendering all points in a line chart can cause significant performance issues. For example, plotting 50,000 data points directly can overwhelm the browser and make the chart unresponsive. Tools like amCharts and ApexCharts struggle with such datasets, while ECharts performs better but still isn't optimized for extremely large datasets.

To address this, I combined ECharts with the Largest Triangle Three Buckets (LTTB) algorithm. LTTB is a downsampling algorithm that reduces data points while preserving the visual structure of the dataset.

In this post, I’ll explain how the LTTB algorithm works step by step with an example of reducing 10 points to 4. For each step, I'll include the relevant code, explanations, and resulting data transformations.

LTTB Algorithm Code
Here’s the full code for the LTTB algorithm:

export function lttb(data: Point[], threshold: number): Point[] {
  const dataLength = data.length;
  if (threshold >= dataLength || threshold === 0) {
    return data; // No downsampling needed
  }

  const sampled: Point[] = [];
  let sampledIndex = 0;
  const bucketSize = (dataLength - 2) / (threshold - 2);
  let a = 0; // Start point
  let nextA = 0;

  sampled[sampledIndex++] = data[a]; // Add the first point

  for (let i = 0; i < threshold - 2; i++) {
    let avgX = 0;
    let avgY = 0;
    let avgRangeStart = Math.floor((i + 1) * bucketSize) + 1;
    let avgRangeEnd = Math.floor((i + 2) * bucketSize) + 1;
    avgRangeEnd = avgRangeEnd < dataLength ? avgRangeEnd : dataLength;
    const avgRangeLength = avgRangeEnd - avgRangeStart;

    for (; avgRangeStart < avgRangeEnd; avgRangeStart++) {
      avgX += data[avgRangeStart].x;
      avgY += data[avgRangeStart].y;
    }

    avgX /= avgRangeLength;
    avgY /= avgRangeLength;

    let rangeOffs = Math.floor((i + 0) * bucketSize) + 1;
    const rangeTo = Math.floor((i + 1) * bucketSize) + 1;
    const pointAX = data[a].x;
    const pointAY = data[a].y;

    let maxArea = -1;

    for (; rangeOffs < rangeTo; rangeOffs++) {
      const area = Math.abs(
        (pointAX - avgX) * (data[rangeOffs].y - pointAY) -
        (pointAX - data[rangeOffs].x) * (avgY - pointAY)
      );
      if (area > maxArea) {
        maxArea = area;
        nextA = rangeOffs;
      }
    }

    sampled[sampledIndex++] = data[nextA]; // Add the most important point
    a = nextA;
  }

  sampled[sampledIndex++] = data[dataLength - 1]; // Add the last point

  return sampled;
}

Example: Reducing 10 Points to 4 Points
We’ll reduce the following dataset of 10 points to 4 points using the LTTB algorithm:

Original Dataset

const data = [
  { x: 1, y: 10 },
  { x: 2, y: 20 },
  { x: 3, y: 15 },
  { x: 4, y: 25 },
  { x: 5, y: 30 },
  { x: 6, y: 20 },
  { x: 7, y: 40 },
  { x: 8, y: 35 },
  { x: 9, y: 45 },
  { x: 10, y: 50 }
];
const threshold = 4; // Reduce to 4 points

Step 1: Add the First Point

sampled[sampledIndex++] = data[a]; // Add the first point

✅ The algorithm always starts by adding the first data point. This ensures that the downsampled dataset begins correctly.

📊 Resulting Data:

sampled = [
  { x: 1, y: 10 }
];

Step 2: Calculate the Bucket Size

const bucketSize = (dataLength - 2) / (threshold - 2);

✅ The bucket size determines how many data points fall into each "bucket." In this case:

$bucketSize=dataLength−2threshold−2=10−24−2=4\text{bucketSize} = \frac{\text{dataLength} - 2}{\text{threshold} - 2} = \frac{10 - 2}{4 - 2} = 4$

So, each bucket contains 4 data points.

Step 3: First Bucket (Points 2–5)

let avgX = 0;
let avgY = 0;
let avgRangeStart = Math.floor((i + 1) * bucketSize) + 1;
let avgRangeEnd = Math.floor((i + 2) * bucketSize) + 1;
avgRangeEnd = avgRangeEnd < dataLength ? avgRangeEnd : dataLength;
const avgRangeLength = avgRangeEnd - avgRangeStart;

for (; avgRangeStart < avgRangeEnd; avgRangeStart++) {
  avgX += data[avgRangeStart].x;
  avgY += data[avgRangeStart].y;
}

avgX /= avgRangeLength;
avgY /= avgRangeLength;

✅ The algorithm calculates the average point of the first bucket (points 2–5). This is done by summing their x and y values and dividing by the number of points.

$avgX=2+3+4+54=3.5\text{avgX} = \frac{2 + 3 + 4 + 5}{4} = 3.5$

$avgY=20+15+25+304=22.5\text{avgY} = \frac{20 + 15 + 25 + 30}{4} = 22.5$

Step 4: Find the Most Significant Point

let rangeOffs = Math.floor((i + 0) * bucketSize) + 1;
const rangeTo = Math.floor((i + 1) * bucketSize) + 1;
const pointAX = data[a].x;
const pointAY = data[a].y;

let maxArea = -1;

for (; rangeOffs < rangeTo; rangeOffs++) {
  const area = Math.abs(
    (pointAX - avgX) * (data[rangeOffs].y - pointAY) -
    (pointAX - data[rangeOffs].x) * (avgY - pointAY)
  );
  if (area > maxArea) {
    maxArea = area;
    nextA = rangeOffs;
  }
}

✅ The algorithm identifies the point in the bucket that forms the largest triangle with the average point and the previously selected point.

For this bucket, (2,20) has the largest area

sampled = [
  { x: 1, y: 10 },
  { x: 2, y: 20 }
];

Step 5: Second Bucket (Points 6–9)

The algorithm repeats the process for the second bucket (points 6–9), computing:

The most significant point in this bucket is (6, 20).

$avgX=6+7+8+94=7.5\text{avgX} = \frac{6 + 7 + 8 + 9}{4} = 7.5$

$avgY=20+40+35+454=35\text{avgY} = \frac{20 + 40 + 35 + 45}{4} = 35$

📊 Updated Data:

sampled = [
  { x: 1, y: 10 },
  { x: 2, y: 20 },
  { x: 6, y: 20 }
];

Step 6: Add the Last Point

sampled[sampledIndex++] = data[dataLength - 1]; // Add the last point

✅ The algorithm always ends by adding the last point.

📊 Final Data:

sampled = [
  { x: 1, y: 10 },
  { x: 2, y: 20 },
  { x: 6, y: 20 },
  { x: 10, y: 50 }
];

Conclusion
By combining ECharts with the LTTB algorithm, I was able to downsample my dataset efficiently and improve chart performance significantly. If you work with large datasets, this method is a great way to balance performance and visual accuracy. Let me know if you’ve tried this approach or if you have other techniques to share!

Introducing Framer Motion

Saeed — Thu, 18 Aug 2022 11:07:02 +0000

Animations are powerful when done right.However, creating eye-catching animations with CSS can be tricky.That's where Framer Motion comes in.With Framer Motion, you don't have to be a CSS expert to create beautiful animations.
**

What is Framer Motion? ** Framer Motion is an animation library that simplifies the creation of animations.Its simplified API helps us abstract the complexity behind animations and allows us to create animations from scratch with ease.One difference is that Framer Motion only supports React, while Pose supports React-Native and Vue. If you are currently using Pose, I would recommend upgrading to Framer Motion as Pose is deprecated.

2. How Framer Motion works

Motion components are the building blocks of Framer motion. Motion components are created by prepending motion to your regular HTML and SVG element (for example, motion.h1). Motion components can accept multiple props, with the basic prop being the animated prop. This prop takes an object in which we define the properties of that component that we want to animate. The properties we define are animated when the component is deployed in the DOM.

Let's animate an h1 text using Framer Motion. First we install the Framer Motion library and import Motion.

npm install framer-motion
import { motion } from 'framer-motion';

Then we convert the h1 into a motion component.

<motion.h1 
  animate={{x: 20, y: -20}}>
  This is a motion component
</motion.h1>

This moves h1 20 pixels to the right and 20 pixels up when loading. If no units are added, calculations are performed using pixels. However, you can explicitly specify the units on which to base the calculations, animate={x: "20rem", y:"-20rem"}}>.

By default, a motion component is animated from the state defined by its styles to those in the animated prop. However, if we wanted, we could hijack and define the initial animation state of the component using the initial prop. While the animated prop is used to define the behavior of components when mounted, the initial prop defines their behavior before mounting.

If we want our h1 to come in from the left, we control that with the initial prop.

<motion.h1
    initial={{x: -1000}}
    animate={{x: 20}}>
   This is a motion component
</motion.h1>

Now, when the h1 mounts, it slides in from the left.

The transition _prop allows us to define how the animations occur. With it, we define how values animate from one state to another. Among other things, we can define the _duration, delay, and _type _of animation using this prop.

<motion.h1
    initial={{ x: -1000 }}
    animate={{ x: 0 }}
    transition={{
        type: "tween",
        duration: "2",
        delay: "1"
    }}>
    This is a motion component
</motion.h1>

Variants

As you can see, the code is getting bigger and bigger and soon, these new props will have even more relevance than those related to React logic. We can use variants to isolate code related to animations and much more.
The variant prop in Framer Motion allows us to extract our animation definitions into a variant object.
Variants not only make our code cleaner, but allow us to create even more powerful and complex animations.
When we extract our animation definitions into variant objects, we have the following:

const Variants = {
  initial: { x: -1000 },
  animate: { x: 0 },
  transition: {
    type: "tween",
    duration: 2,
    delay: 1
  }
}

Instead of passing the animation definitions into a component’s initial and animate props directly, we extract these definitions into standalone variant objects. In the variant objects, we define variant names that describe each animation’s name as variants.

<motion.h1
      variants={Variants}
      initial='initial'
      animate='animate'
      >

In the variants prop, we pass in the name of the variant objects for each motion component and then pass in the animations to the initial and animate props.

Conclusion

This is just an introductory guide to framer-motion, there is a lot more inside the library, especially a lot of utility hooks to make even crazier animations effortlessly.
Hopefully, this Blog gives you a solid foundation to add some great animations to your library and build upon them over time to suit all your needs.

A short introduction to MongoDB and Mongoose

Saeed — Mon, 15 Aug 2022 10:55:00 +0000

1. Introduction to MongoDB

MongoDB was first born by MongoDB Inc. in October 2007. It was part of the PaaS (Platform as a Service) product like Windows Azure and Google App Engine. It was later made open source in 2009.
MongoDB uses JSON like files schemas, meaning data are stored as documents or collections which can be strings, numbers, floats and even arrays and objects.
MongoDB is also known as NoSQL database; it allows the insertion of data without interrupting the service and provides the real-time application changes.
Due to the vast complexity in web development and usage, the relational database cannot cope with the scale and agility
challenges of modern applications. When comparing relational database to NoSQL, the NoSQL database has better performance and is more scalable.

2. Pros and Cons of MongoDB

The outstanding advantages of MongoDB convincing the choice of current users are:

Since MongoDB uses data in the JSON form, each collection has different sizes and documents. Nevertheless, they are very flexible when it comes to archiving.
MongoDB data is usually not bound to each other; it does not support join query like in RDBMS, so when users insert, delete or update, it will not spend too much time to check if it satisfies the constraints like in RDBMS or not.
MongoDB is easy to scale. In MongoDB, the concept “cluster” refers to clusters of nodes containing data to communicate with each other. Adding a new node to the cluster helps users expand the system quickly.
The unique identifier _id will always be indexed automatically, so the speed of querying information will consistently achieve the highest performance.
The unique identifier _id will always be indexed automatically, so the speed of querying information will consistently achieve the highest performance.

3. What Is Mongoose?

Mongoose is a query language to communicate between the server and the database and performs reading, writing, updating and deleting operations of the data.
It is an Object Data Modeling (ODM) library for MongoDB and NodeJS which provides schemas to model application data which therefore cleans up the ambiguity of databases.
It enforces a standard structure to all the documents in a collection using schema.
It also validates the stored data in the documents as well as allow only valid data to be saved in the database.
Overall Mongoose provides all the features to the MongoDB including
query building functions and business logic in the data.

Question about a process of building a real world web...

Saeed — Fri, 15 Apr 2022 11:04:13 +0000

I'm curious about the process of building a real web app.

because I'm trying to build a web application (management system) right now, I'm not sure I'm not certain which part of entire code i should start first.

Should I create the front-end design and interaction first, or create a back-end function first?

Or do I have to do both at the same time with each function part like login, register, upload etc.?