Do analytics for the bot from scratch. Part 1 - writing a bot

Maksim Lamanov — Sat, 24 Jun 2023 12:32:33 +0000

In this article, we will look at the importance of connecting basic analytics to a bot and what benefits it can bring.

Introduction

In today's digital world, where interaction with customers is increasingly happening through online platforms, creating and maintaining effective chatbots is becoming an important aspect of a successful business. However, building a bot that simply provides information or answers basic questions is no longer enough for a competitive advantage. Today, it is important to understand user behaviour and needs, analyse data and make informed decisions based on this information.

This is where connecting analytics to the bot comes to the rescue. The integration of analytical tools and methods allows you to collect, analyse and interpret data about user interaction with the bot. This gives us the opportunity to gain valuable insights that will help optimise the performance of the bot, improve the user experience and make informed decisions based on data.

Connecting analytics to a bot is an integral part of the development and improvement of chatbots in our dynamic digital age. It allows you to turn a bot from a simple communication tool into a powerful data analysis tool that will help your business form an effective strategy and achieve its goals. Let's look at all the advantages and opportunities that the connection of analytics to the bot opens up for us.

What data will we receive today

In this example, we will learn:

see the number of "live" and "dead" users;
see the number of messages sent through the bot;
see the messages themselves;
build charts.

Tools

Python 3 (aiogram - a framework for creating bots);
MongoDB - a database that will serve as a source for building a dashboard;
Telegram - is the best messenger for working with bots;
Redash - is a self-hosted service for building dashboards.

First, let's write a bot

As an example, let's write a simple Python 3 bot that allows you to exchange anonymous messages.

Principle of operation

The user is provided with a personal link to the bot, which contains the identifier of the user to whom the message needs to be sent. He can post it on his social networks. Anyone who wants to send an anonymous message to a user must follow this link. After that, the bot will receive the /start command with a payload in the form of a user ID in Telegram.

Create a bot in Telegram

Go to @BotFather and create a bot:

Connecting libraries

We connect the library for creating telegram bots:

pip3 install aiogram

And also for working with MongoDB:

pip3 install pymongo

Write the code

Create a main.py file and write the beginning for any bot:

bot = Bot(token='BotFather TOKEN HERE')
storage = MemoryStorage() # storage that allows you to remember the states of dialogs with users
dp = Dispatcher(bot, storage=storage)

Let's add one single state that we need to send messages:

class Stage(StatesGroup):
    send_message_to = State()

Let's write a handler for the /start command:

@dp.message_handler(commands=['start'], state=['*', Stage.send_message_to])
async def start(message: types.Message, state: FSMContext):
    args = message.get_args() # get payload from command /start
    payload = decode_payload(args)

    if payload: # we go here if we clicked on someone's link
        async with state.proxy() as data:
            data['user-id'] = payload # save the recipient ID in the context of the dialog

        await Stage.send_message_to.set() # set the state of sending the message to the recipient

        start_text = 'Hello! I am a bot for anonymous communication. Send a message to the person who posted this link. He won't know who sent the message.'
        await message.reply(start_text)
        return

    # if there is no payload, then the user just pressed /start
    link = await get_start_link(message.from_user.id, encode=True) # create a personal link
    content = f'Your personal link: {link}'
    await message.reply(content)

Let's write a handler for sending messages:

@dp.message_handler(state=Stage.send_message_to, content_types=ContentType.ANY)
async def process_message(message: types.Message, state: FSMContext):
    inline_btn = InlineKeyboardButton('Reply', callback_data=f'answer_{message.from_user.id}')
    inline_kb = InlineKeyboardMarkup().add(inline_btn)
    async with state.proxy() as data:
        if 'answer_user_id' in data:
            user_id = data['answer_user_id'] # if we reply to a message
            await message.answer('Your response has been delivered.')
            await message.bot.send_message(user_id, 'You have received an answer.')
        else:
            user_id = data['user-id'] # if we send a new message
            await message.answer('Your message has been sent.')
            await message.bot.send_message(user_id, 'You have received a new anonymous message.')
        if message.video:
            await message.bot.send_video(user_id, message.video.file_id, reply_markup=inline_kb)
            content = message.video.as_json()
        elif message.video_note:
            await message.bot.send_video_note(user_id, message.video_note.file_id, reply_markup=inline_kb)
            content = message.video_note.as_json()
        elif message.voice:
            await message.bot.send_voice(user_id, message.voice.file_id, reply_markup=inline_kb)
            content = message.voice.as_json()
        elif message.photo:
            await message.bot.send_photo(user_id, message.photo[-1].file_id, reply_markup=inline_kb)
            content = message.photo[-1].as_json()
        elif message.audio:
            await message.bot.send_audio(user_id, message.audio.file_id, reply_markup=inline_kb)
            content = message.audio.as_json()
        elif message.sticker:
            await message.bot.send_sticker(user_id, message.sticker.file_id, reply_markup=inline_kb)
            content = message.sticker.file_id
        elif message.document:
            await message.bot.send_document(user_id, message.document.file_id, reply_markup=inline_kb)
            content = message.document.as_json()
        else:
            await message.bot.send_message(user_id, message.text, reply_markup=inline_kb)
            content = message.text
    await state.finish()

In the example above, we implemented the ability to send various types of messages, but you can leave only text.

Let's write a click handler for the "Reply" button:

answer_regexp = re.compile("answer_(.*)")


@dp.callback_query_handler(lambda c: answer_regexp.match(c.data), state=['*', Stage.send_message_to])
async def process_callback_answer(callback_query: types.CallbackQuery, state: FSMContext):
    await bot.answer_callback_query(callback_query.id)
    await bot.send_message(callback_query.from_user.id, 'Enter your reply.')
    async with state.proxy() as data:
        data['answer_user_id'] = answer_regexp.match(callback_query.data).group(1)
    await Stage.send_message_to.set()

It remains to run the bot:

if __name__ == '__main__':
    executor.start_polling(dp, skip_updates=False)

Working with the database

Let's create a db.py file and mark up methods for working with the database.

Connecting to the database:

client = MongoClient('CONNECTION_STRING HERE')
db = client['AskFM']

users = db['users']
messages = db['messages']

Adding a user:

def add_user(user_id, username, datetime):
    user_filter = {
        'user_id': user_id
    }
    user = users.find_one(user_filter)
    if user and user['status'] == 'deleted':
        update = {
            'status': 'active'
        }
        users.update_one(user_filter, {'$set': update})
        return
    elif user:
        return
    data = {
        'user_id': user_id,
        'username': username,
        'datetime': datetime,
        'status': 'active'
    }
    users.insert_one(data)

Delete user:

def delete_user(user_id):
    user_filter = {
        'user_id': user_id
    }
    update = {
        'status': 'deleted'
    }
    users.update_one(user_filter, {'$set': update})

Adding a message:

def add_message(user_id_from, user_id_to, username_from, username_to, content, message_type, datetime):
    message = {
        "user_id_from": user_id_from,
        "user_id_to": user_id_to,
        "username_from": username_from,
        "username_to": username_to,
        "content": content,
        "message_type": message_type,
        "datetime": datetime
    }
    messages.insert_one(message)

Integrating work with the database into the bot

import db

Let's write a handler that tracks the activation and deactivation of the bot:

@dp.my_chat_member_handler()
async def add_to_channel(update: ChatMemberUpdated):
    if update.new_chat_member.status == 'member':
        db.add_user(update.from_user.id, update.from_user.username, time.time())
    elif update.new_chat_member.status == 'kicked':
        db.delete_user(update.from_user.id)

Add user registration to the /start method:

db.add_user(message.from_user.id, message.from_user.username, time.time())

Final handler code:

@dp.message_handler(commands=['start'], state=['*', Stage.send_message_to])
async def start(message: types.Message, state: FSMContext):
    args = message.get_args() # get payload from /start command
    payload = decode_payload(args)

    if payload: # we go here if we clicked on someone's link
        async with state.proxy() as data:
            data['user-id'] = payload # save the recipient ID in the context of the dialog

        await Stage.send_message_to.set() # set the state of sending the message to the recipient

        start_text = 'Hello! I am a bot for anonymous communication. Send a message to the person who posted this link. He won't know who sent the message.'
        await message.reply(start_text)

        db.add_user(message.from_user.id, message.from_user.username, time.time()) # added here

        return

    # if there is no payload, then the user just pressed /start
    link = await get_start_link(message.from_user.id, encode=True) # create a personal link
    content = f'Your personal link: {link}'
    await message.reply(content)

    db.add_user(message.from_user.id, message.from_user.username, time.time()) # and added here

Why add user registration to the /start method if we have a separate handler that looks at the activation and deactivation of the bot? Answer: that method does not always work correctly.

Add message registration to the very end of the message send handler:

user_to = await bot.get_chat_member(user_id, user_id)
if 'answer_user_id' in data:
    message_type = 'answer'
else:
    message_type = 'question'
db.add_message(message.from_user.id,
               user_id, message.from_user.username,
               user_to.user.username, content,
               message_type, time.time())

This is the end of the code. The finished code can be viewed here.

We start the bot and try to test it:

python3 main.py

In the next part, we will look at creating the dashboard itself.

Configuring Applications at Runtime: Feature Flags

Maksim Lamanov — Sat, 24 Jun 2023 11:50:03 +0000

In modern programming, more and more attention is paid to the flexibility and configurability of applications. Developers strive to create applications that can adapt to different environments and user requirements without the need for recompilation or redeployment. In this context, feature flags are an increasingly popular tool for configuring applications at runtime.

What are feature flags?

Feature flags are a mechanism that allows you to enable or disable certain parts of the application's functionality depending on configuration or runtime conditions. Instead of hard-coding functionality into an application and releasing a new version to make changes, feature flags allow you to change the behavior of an application on the fly without restarting.

Advantages

One of the benefits of feature flags is the ability to roll out new functionality in stages. Instead of enabling new functionality for all users at once, developers can use feature flags to incrementally enable new features for only certain groups of users. For example, you can enable new functionality for only a small group of users to test its stability and gather feedback before enabling it for everyone.

Another advantage of feature flags is the ability to quickly respond to changes in requirements and conditions. If you need to temporarily disable a piece of functionality or change its behaviour, this can be done simply by changing the feature flag configuration, without the need to release a new version of the application. This is especially useful when you need to quickly make changes or fix problems.

Feature flags also contribute to the division of responsibilities between developers and system operators. Developers can add feature flags to the code to define the behaviour of the functionality, but the configuration of the flags can be changed by system operators without the need for developer intervention. This allows operators to quickly respond to requirements and changes in the application environment.

Risks

However, there are some risks and challenges associated with using feature flags. As the number of flags and their combinations grows, it can become difficult to manage and keep track of all configurations. Incorrect use of feature flags can lead to difficult to track and unpredictable application behaviour. Therefore, it is important to carefully plan and manage feature flags, especially in large and complex projects.

Conclusion

Configuring an application at runtime using feature flags is a powerful tool for creating flexible and adaptive applications. They allow developers to control the behaviour of functionality and quickly respond to changing requirements and conditions without having to restart or redeploy the application. However, feature flags must be carefully planned and managed to avoid potential problems.

P.S.

I started making a tool for working with feature flags in C# (GitHub). I will be glad if you join the development :)

DEV Community: Maksim Lamanov