BotTalk: Gathering requirements for our LLM-powered Telegram Chat Bot

Vignesh Sankar Iyer — Thu, 24 Aug 2023 07:38:44 +0000

Part II of the series where I chronicle the journey of building BotTalk.

Building on the motivation and vision articulated in our first entry, we now turn to a more granular level of planning. In the world of software development, the foundation of a successful project is a clear and comprehensive understanding of its requirements. These requirements can broadly be categorised into Functional and Non-functional. But what do these terms mean, and why are they pivotal?

Functional vs. Non-Functional Requirements: A Brief Primer

Functional requirements pertain to the essential functions a system must execute. Think of them as the definitive list of features or services a system needs to provide. They answer the "what" of a system — what tasks it should perform, what its input/output behaviour should be, and the interactions it must support.

On the other hand, Non-functional requirements, detail how these functions are realised, focusing on system behaviour and quality. They're often the attributes that users expect by default, such as reliability, security, and performance. In essence, they address the "how" — how fast, how secure, how scalable, and so on.

Together, these requirements give us a 360-degree view of what we aim to achieve and set the stage for future phases of our software development journey.

Now, let's cover the Functional Requirements of BotTalk

Functional Requirements of BotTalk

In the context of BotTalk, there are a couple key components that would drive the motivation for creating such a service. Of course, that would mean comparing it to the most prevalent chat assistant service right now: ChatGPT.

Dissecting the UI gives us an idea of how BotTalk could enhance its use-case.

Fig.1 - ChatGPT UI, taken on 23/08/2023

In the context of ChatGPT, conversations with the model are organised in threads, sorted by date on the left side. As for opening new chats, ChatGPT gives an option to select the preferred model above, while maintaining contexts per conversation within each tab and even providing suggestions for prompts.

Taking inspiration from ChatGPT then gives us some understanding of the user requirements for a chat bot like BotTalk. These include:

Initialisation & Individual Usage:
- The bot should initialise in a one-on-one chat setting, primarily recording user details.
- Users should be able to select the LLM model they prefer from a provided list. We're initially focusing on OpenAI's models, but a future iteration might allow user-driven customisation. This choice ensures users have control over the conversational quality and style, fostering a sense of familiarity and empowerment.
- Users should dictate the level of context retrieval, graded on a scale from 1-10. This permits dynamic adaptability, tailoring conversations to the desired depth and relevance.
- To ensure quality, only one prompt will be answered at a time. Over-zealous prompters will be kindly asked to pace their inquiries.
Group Integration:
- Bot should effortlessly integrate into group chats, answering based on the collective context offered by the group's members.
- Unregistered users must engage with the bot privately before group chat participation.
Monetisation & Subscription:
- Users will operate on a token-based system, reflective of their usage.
- Users will be able to track their usage thus far.
Input Sanitation And Moderation:
- Inputs by users will be monitored and checked for hateful language, mirroring ChatGPT's input sanitation mechanism

These user requirements give a reasonable road map for the project, and allows for extendibility such as the use and deployment of models that is fine tuned on the collection of prompts stored by BotTalk.

Next, let's take a look at some of the non-functional requirements of BotTalk

Non-functional Requirements of BotTalk

Examining some of the network connections and requests also gives us an idea of some of the system choices and non-functional requirements taken in mind while building ChatGPT, which we could use towards the development of BotTalk.

Fig.2 - ChatGPT Network Requests, taken on 23/08/2023

This is the network request of a prompt that requests an implementation of a distributed queue system.

Fig.3 - Request to generate implementation of a distributed queue system, taken on 23/08/2023

First, we see that the return content-type of the sent GET request is in fact an event stream. The model's response is streamed to the client to provide a better user experience, allowing the user to feel like the user is having a conversation with the model in real time. On top of that, while the entire content of the response in fact took 45 seconds to complete downloading, the server response only takes 2 seconds, even with millions of users around the world. This gives us an idea of the non-functional requirements that BotTalk should adopt:

Scalability: BotTalk should be built to cater to thousands of users globally. As the user base grows, the bot's architecture should gracefully handle the increased load.
Low Latency: In the world of chatbots, speed is paramount. Our bot aims for real-time communication, offering quick, concise, and pertinent responses.
Security: Conversations are personal. Our bot should ensure data integrity and prevent users from accessing other people's interaction histories. While ChatGPT achieves this with individual conversation threads with the user, we should be limiting the context to only the chat the user participates in with the bot. For example, if the user participates in 2 groups with the bot enabled Chat A and Chat B, user should not be able to get responses with the context for Chat A while participating in Chat B.

With the functional and non-functional requirements in mind, let's generate user-stories, partitioned by phases. This allows for strategic, incremental development and testing. Breaking it down phase-by-phase ensures that each part of the project is validated before moving on to the next, minimising potential issues, as well as a road map for product release.

User Stories

User stories are informal, general explanations of a software feature written from the perspective of the end user, which often follow a format of "As a [type of user], I want [an action] so that [a benefit/a value]". I have divided the user stories up into 4 phases,

Initial Setup and Single-User Interaction,
Group Integration,
Monetisation and Credit System, and
Sanitation and Moderation

At the end of each phase or iteration, I expect to have completed the user stories and have a working version of the bot that has completed the phase's user stories and maintained the status of previous phases's completed work.

Phase 1: Initial Setup and Single-User Interaction

As an alpha user, I want to initiate the bot in a private chat so that I can have personalized interactions.
As a new user, I want the bot to record and recognize my details so that my interactions are personalized.
As a user, I want to choose from available GPT models so that I can tailor the chat experience.
As an advanced user, I want the option to fine-tune or select advanced models in the future so that I can continuously improve the chat experience.
As a user, I want to set a context sensitivity level so that my chat reflects the depth of conversation history I desire.
As a user, I want to be sure my chats are isolated so that my interactions are not influenced by other users' interactions.
As a user, I want to receive a response for each of my prompts so that I feel the bot is attentive and interactive.
As a user, I want the bot to inform me if I'm sending prompts too quickly so that I can adjust my interaction speed.
As a user, I want to view my usage statistics so that I can understand my interaction patterns with the bot.

Phase 2: Group Integration

As a group admin, I want to add the bot to my group chat so that multiple users can interact with it.
As a group member, I want the bot to consider all user inputs so that it provides relevant group context in its responses.
As a new group member, I want the bot to prompt me for registration so that I can also participate in group interactions.
As a group admin, I want to ensure only fellow admins can control the bot so that its functionalities aren't misused.

Phase 3: Monetization and Credit System

As a user, I want to know about my available credits so that I can manage my usage.
As a user, I want to purchase additional credits so that I can continue using the bot extensively.
As a super admin, I want to grant certain users unlimited access so that they can have uninterrupted interactions for promotional or testing purposes.
As a user, I want my credits to be used uniformly across different chats so that I don't have to manage multiple balances.

Phase 4: Input Moderation

As a responsible developer, I want to ensure that users are not misusing the chat bot by sending inappropriate prompts, which could possibly influence the set of prompts collected by the users.

With our requirements in place, our next step is to take this blueprint and begin crafting our chatbot. The journey of BotTalk is only getting started, and there's much more ground to cover. I hope you'll continue to join me as we embark on this meticulous journey, watching an idea mature through each SDLC phase.

Thank you and see you in the next article!

BotTalk Begins: Introducing the LLM-powered Chatbot for Telegram Groups

Vignesh Sankar Iyer — Sun, 20 Aug 2023 12:34:03 +0000

This post is the first part in a series whereby I document my progress on my side project.

Large Language Models (LLMs) have been all the rage, especially since the release of OpenAI's ChatGPT: a web-based chat application that utilises Generative Pre-trained Transformer (GPT) models to provide one-on-one conversational prowess, including understanding context, generating coherent responses, and adapting to various conversational topics.

However, tools like ChatGPT currently offer one-on-one chats and often come with a price tag. This limits who can use them and how. Imagine having a chatbot in group chats on apps like Telegram or WhatsApp. It would mean more people could benefit from just one subscription. Plus, a chatbot in a group would need to respond to different topics and voices, making it even smarter and more useful. This isn't just about making chatbots available to more people; it's about making them fit better into how we talk online today, and better integrating them into our workflows.

After laying down the broader motivations for such an application, I'd like to share some personal driving forces behind embarking on this side project of building a Telegram bot powered by GPT.

Gaining System Design Experience: As I navigate my final year in university, I've recognized a gap in hands-on system design experience in many academic curriculums. Taking on this project isn't just about addressing an industry need, but also about furnishing myself with a practical understanding of building scalable systems. It's one thing to read about system design, but entirely another to implement it from scratch.
Exploring Diverse Technologies: The realm of technology is vast, and every project can be approached with myriad tools and methodologies. My aim is to craft an affordable and resource-efficient system, leaning heavily on open-source solutions. This doesn't just democratize access to models like GPT, but also grants me an invaluable learning opportunity, enabling me to familiarize myself with a range of technologies.
Diving Deeper into Natural Language Models: While the surface-level applications of models like GPT are fascinating, the underlying mechanics and intricacies intrigue me even more. This project serves as a deep dive into understanding natural language models, their workings, and most importantly, tailoring them for real-world applications.

In essence, this isn't just a side project. It's a personal journey of exploration, learning, and growth, bridging the gap between academic knowledge and practical application. So, come join me as I build a scalable, functional GPT-based chat bot that can participate and collaborate with you in your group conversations: BotTalk! This series will serve as a dev-journal as I develop my application, as well as keep me accountable for the progress that I make.

Transitioning from motivation to action, I've envisioned this series of articles to closely align with the stages of the Software Development Life Cycle (SDLC). Each instalment will not only mark a developmental milestone for BotTalk but also offer insights into the intricacies of each phase. With our next piece, we will be immersing ourselves in "Requirement Gathering", laying out foundational needs, and sketching the personas BotTalk seeks to serve. As we journey forward, upcoming articles will delve into system design, development, testing, and deployment. Through these chronicles, my goal is to provide a panoramic view of both the technical landscape and the myriad decisions and challenges that punctuate the creation journey. Whether you're deep into the tech realm, just dipping your toes, or simply curious about the evolution of a chatbot, I invite you to accompany me on this meticulous journey, watching an idea mature through each SDLC phase.