DEV Community: Khalil Najjar

My chatbot builder is over-engineered, and I love it

Khalil Najjar — Mon, 12 Aug 2024 19:52:12 +0000

Over the past year, I've been working on a chatbot builder called Fastmind. My goal with this project was to learn how to build a fully automated service that could scale to thousands of users without breaking the bank. Given the nature of a chatbot builder, I encountered several interesting challenges, such as exposing an AI model to the web, preventing malicious users from abusing the service (which, in contrast to traditional CRUD apps, could turn out to be insanely expensive), and handling a large number of concurrent users for the chat stream without performance bottlenecks. These challenges were in addition to the more common SaaS issues like billing, user management, and monitoring.

In this post, I'll share the lessons I learned during this long journey and offer my advice on building software as a solo developer (spoiler alert: don't do what I did). By the time you finish reading this, I hope you'll find it useful whether you're building a similar product or just curious about how over-engineered chatbots work behind the scenes.

The Architecture Behind Fastmind

When building a new project, it's important to choose the right tools for the job.

So what is the best tool for this job? Well, you guessed it! Always choose what you know. In my case, I've been working with the JavaScript ecosystem for a while, so I decided to stick with React for the frontend and Hono for the backend. I also used Convex heavily for the database, cron jobs, real-time capabilities, and more, all bundled together in a Turborepo. I'll go into more detail about each part of the architecture in the following sections.

Here is a rough overview of the folder structure:

apps
- chatbot-builder
- chat-widget
- marketing-website
- hono-api
packages
- convex
- ui
- utils

Frontend

I have three separate frontend applications. This separation was intentional to maintain a clear division between the chatbot builder (dashboard), the chatbot itself (the chat widget embedded on your website), and the marketing website. This separation makes it easier to maintain and scale the applications independently and push updates without affecting other parts of the system. For me, this setup works because it's the same one I use at work, so I feel very comfortable with this architecture. Again, this might not be your case, and that’s perfectly fine. I won’t go into too much detail about the marketing website, as it's not the focus of this post, but it’s a Next.js app hosted on Vercel.

Chatbot Builder

For the main app (chatbot builder), I used Next.js paired with Shadcn for UI components and Tailwind CSS for styling. Since I use Convex for the DB layer, I used their React library to connect to the DB to perform queries and mutations with real-time features and to call serverless functions with what they call "actions." I'll delve deeper into Convex and all of its amazing features in another post if you're interested. For authentication, I used Clerk to avoid dealing with user management and building all the different auth flows, allowing me to focus on product development. This is one of my favorite tools out there—you have to check them out if you're a React developer. This app is hosted on Vercel, which is a great platform for hosting Next.js apps.

Chat Widget

The chat widget is a basic Next.js application that connects to the long-running API to get the chatbot configuration, stream, and send messages. The chat widget is hosted on Cloudflare Workers, a serverless platform that allows you to run JavaScript code at the edge of the network. This is very useful for a chat widget, as it’s closer to the user and can significantly reduce latency and costs, especially if a large number of users start using the chat widget. This could easily be hosted alongside the chatbot builder, and it would be perfectly fine—and cheaper. But I am a bit of a performance freak, so I decided to go with Cloudflare Workers. Also, I've heard horror stories about the costs of apps hosted on Vercel being DDoSed, so I decided to go with Cloudflare Workers to prevent this from happening.

Backend

Long-Running API Server

The backend consists of a long-running Hono server that handles most of the chat widget requests, as explained above. It’s a Bun application that connects to Convex to read and write data to the database. It’s hosted on Railway, a platform that allows you to deploy and scale your applications without worrying about infrastructure. This server could be replaced with Convex and use their serverless functions or even HTTP solution, but I prefer to have a long-running API server for the chat widget to prevent DDoS attacks on my serverless functions. To achieve this, I have a local Redis instance available for the server to rate-limit requests by IP. Then I have another rate-limit layer directly on the DB (Convex) to block multiple requests for an account. Believe it or not, this helps me sleep at night.

Convex

Now the star of the show—most of the backend logic runs in Convex. It is very similar to Firebase or Supabase, but with an even better developer experience, in my opinion. I highly recommend checking them out if you're building a new project, especially in React. Caching, real-time updates, serverless functions, and cron jobs development environment are all handled by Convex. I also use their HTTP actions to handle incoming webhooks from Lemonsqueezy and Clerk.

Why did I choose Convex over Firebase, Supabase, or even a self-hosted solution? I initially started Fastmind using the T3 stack, but found it to be too much work to handle some of the real-time features I needed. I also required a lot of background jobs, for which I tried solutions like Bull, and services like Inngest and Trigger.dev. While these are great services, I wanted everything in one place, so I decided to give Convex a try, and I’m very happy with the results.

Infrastructure

As mentioned above, I use Vercel for the frontend apps, Cloudflare Workers for the chat widget, and Railway for the backend. I also use Sentry for error tracking, Lemonsqueezy for billing as my Merchant of Record so I don't have to worry about taxes, Clerk for authentication, and Convex for the database, cron jobs, and real-time features. The AI models are Command R and Command R+ from Cohere.

Another star of the show is Cloudflare. I use them heavily in Fastmind. In addition to what I’ve mentioned above, I use their AI Gateway service. This is a great product to use when you need to expose an AI model to the web; it handles rate limiting, caching, and DDoS protection for chat requests. All my domains are managed by Cloudflare—they are second to none when it comes to handling DNS and SSL certificates and preventing DDoS attacks. Lastly, their zero-egress pricing for storage is a great deal for me. I use it to store chatbot assets and scripts. Eventually, I plan to allow users to upload PDFs and other files to train their chatbots, as well as conversation logs so they can export them and analyze them later.

My Key Takeaways

It Doesn't Have to Be Perfect

I spent a lot of time trying to make everything perfect, but I realized it's better to have something that works and iterate on it. I waited too long to ship Fastmind, and I was going to postpone it even further. But my competition is already making bank while I was just building, and I must say I don't think Fastmind is missing any features; if anything, it has more. Since realizing this, I adopted a "ship fast, iterate often" mentality, and I’ve been able to launch Fastmind and get users without everything being perfect, and that is okay. I’ve been able to fix bugs and add features as I go, and users are happy. This is the beauty of building a SaaS product—you can always improve it; it’s never finished. So don't worry too much about making everything perfect—just get it out there and start getting feedback. This is something other successful founders have taught me, but being an engineer first, I always focused on the "fun" part of building the product and not on the "hard" part of getting users. I’m still learning this, but I’m getting better at it.

Focus on the User

After I launched, I started to realize what users wanted, and trust me, it's very different from what you think they want. I’ve been able to gather a lot of feedback from users and improve the product based on that feedback. For example, I thought users would want a lot of customization options for the chat widget, but it turns out they just want a simple chatbot that works and does one thing very well. I also thought users would want a lot of AI features, but it turns out they just want a chatbot that can answer their questions accurately. I’ve been able to focus on the features that users want and ignore the ones they don’t care about. This has helped me build a product that users love and are willing to pay for. Onboarding is key—I've seen a lot of users drop off because they don't know how to use the product, so I've been working on improving the onboarding process to make it easier for users to get started. I’ve built the easiest chatbot builder out there; in just 2 minutes you can have a chatbot up and running on your website (yes, I timed it).

The Tech Stack Doesn't Matter

I’ve been able to build Fastmind using the tech stack I know and love, but I’ve seen other successful founders build their products using entirely different tech stacks. What matters is that you build something that works and that users want. Don’t get too caught up in the tech stack—just build something functional and valuable. In the end, if your product is successful, you can always rewrite it in a different tech stack, but by then, you'll have the users, revenue, and motivation to do so.

Don’t Wait Too Long to Launch

I’ve been building Fastmind in the shadows since August 2023. Meanwhile, I have competitors who entered the market with a product similar to mine—not perfect, but they’re making money and collecting feedback while I was just building alone without a clear direction of what my users wanted. I should have launched Fastmind earlier; I would have received insanely valuable feedback from users and been able to improve the product based on that feedback faster. I was afraid of launching because I thought the product wasn't perfect, but I realized that it's better to launch something that works and iterate on it than to wait for it to be perfect.

I Regret Being a Hipster in Tech

Khalil Najjar — Sun, 27 Jun 2021 10:08:19 +0000

The Problem

For the last five or more years I've been doing it all wrong. I thought that learning the newest, trendiest, and hippest library, framework or language out there was the way to go. I've curated a list of some of the technologies I've learned thought-out the years:

Docker, Kubernetes, Terraform, Rancher, Helm, Angular, React, Vue, Adonis.js, Prisma, Gatsby, GraphQL, Postgres, MongoDB, Cassandra, Socket.io, Webpack, SASS, Gulp, Next.js, Nuxt.js, Meteor, Express, Koa, Elastic, Algolia, jQuery, TypeScript, Node.js, Deno.js, Go, Python, Ruby, PHP, Java, among many more.

What is the problem with this list? Well if you look closely, I learned many tools that solve similar problems, I am getting ahead of my self but pro-tip: you don't have to learn every single one out there 😉.

Also, there's been a lot of software architecture changes during this time, we've had the monolithic approach, then client-service came along, need I remind you of the complicated microservices architecture? and now we have serverless.

Look, I could keep going on... As you can see (and probably you've been in a similar situation) I've invested a lot of my free time learning different tools that achieve similar results.

Like so many developers out there, I started to burn out.

Long story short, I took some vacations and went to the German Alps 🏔 to contemplate about life and find what was I doing wrong. During my mountain retreat, Apple had the WWDC﹡, and of course, me being a nerd, I just couldn't miss the event therefore I joined.

And all of a sudden it was clear to me what do I needed... it was stability in my day-to-day as a developer. Something Apple offers with their WWDC.

Allow me to explain...

The beauty of this is that it gives guidance to the Apple developers, there are no new disruptive technologies that pop out of nowhere, it's a clear and predictable roadmap, which allows the developers to follow this guided path, and simply focus on getting better at what they do and very occasionally learn the new way of doing things.

Now... I love the web, I was not going to change my career and become a mobile developer overnight. So naturally, I started looking for alternatives in my domain. That's when I crossed paths with the well-known and battle-tested MVC frameworks 🐍 💎.

A New Hope

I've worked with different MVC frameworks in the past, like Django, but they were more like stepping stones towards my career as a wheel-reinventor engineer.

I tried Adonis.js, which is a very good JavaScript MVC framework, I truly enjoyed working with it, although at the time it had a small community and many new breaking-changes were being introduced to the framework as it was being actively developed.

That's when I decided to go with something more mature, hence boring, and what is more boring than Rails?
I immediately fell in love with the framework and the community behind it.

After a few months of learning the Rails way, I started to realize something... I felt lazy, before I used to learn new skills for at least 4 hours/day (after working 8 hours). Now, all of a sudden I was finally using my free time in a different and healthier way, but why was I feeling lazy?

Throughout the years I got used to the idea that I had to be studying every single day, like if I had some sort of homework because you know - that's the life that I chose by becoming a software engineer (although it's true to some extent).

Don't get me wrong, learning new skills during your free time is important, but it's also important to enjoy other aspects of life, while still learning occasionally in your free time or during work.

This is what I did, instead of using most of my free time to keep up to date with tools I probably will never use. I leveraged my job to introduce new technologies (when the need arises). This way I can stay relevant and scratch my itch for learning new stuff.

Nonetheless, it took me some time to stop feeling guilty and start appreciating the stability that I was looking for, and let me tell you something, it is well worth it ✌️.

Now after work, I don't open Udemy or Hacker News, I'd rather go for a stroll 🚶‍♂️, bike 🚵‍♂️, cook for my girlfriend 🧑‍🍳, do some yoga 🧘‍♂️ - you get the point. I am no longer a prisoner of tech and guess what? I enjoy my work, coding and life more now than ever.

Learnings

There will always be something new to learn, you can try and stay up-to-date, get burned out, take some vacations and repeat this never-ending cycle.

It's okay (and encouraged) to learn new and hipster tech every once in a while, I certainly still do it occasionally for fun, but it's not longer (and thankfully) part of my day-to-day life.

If you feel that you are in a similar situation don't wait to get burned out, act - make a change. I did it and it has improved my life drastically.

Being a good, happy and healthy developer it's not knowing everything, but rather understanding what you need to know.

"Life is really simple, but we insist on making it complicated."
~ Confucius

Appendix

WWDC
It is a yearly event where Apple presents to their community of developers the improvements coming to the Swift language as well as new features arriving to their operating systems.