Do analytics for the bot from scratch. Part 1 - writing a bot

In this article, we will look at the importance of connecting basic analytics to a bot and what benefits it can bring.

Introduction

In today's digital world, where interaction with customers is increasingly happening through online platforms, creating and maintaining effective chatbots is becoming an important aspect of a successful business. However, building a bot that simply provides information or answers basic questions is no longer enough for a competitive advantage. Today, it is important to understand user behaviour and needs, analyse data and make informed decisions based on this information.

This is where connecting analytics to the bot comes to the rescue. The integration of analytical tools and methods allows you to collect, analyse and interpret data about user interaction with the bot. This gives us the opportunity to gain valuable insights that will help optimise the performance of the bot, improve the user experience and make informed decisions based on data.

Connecting analytics to a bot is an integral part of the development and improvement of chatbots in our dynamic digital age. It allows you to turn a bot from a simple communication tool into a powerful data analysis tool that will help your business form an effective strategy and achieve its goals. Let's look at all the advantages and opportunities that the connection of analytics to the bot opens up for us.

What data will we receive today

In this example, we will learn:

• see the number of "live" and "dead" users;
• see the number of messages sent through the bot;
• see the messages themselves;
• build charts.

Tools

• Python 3 (aiogram - a framework for creating bots);

• MongoDB - a database that will serve as a source for building a dashboard;

• Telegram - is the best messenger for working with bots;

• Redash - is a self-hosted service for building dashboards.

First, let's write a bot

As an example, let's write a simple Python 3 bot that allows you to exchange anonymous messages.

Principle of operation

The user is provided with a personal link to the bot, which contains the identifier of the user to whom the message needs to be sent. He can post it on his social networks. Anyone who wants to send an anonymous message to a user must follow this link. After that, the bot will receive the /start command with a payload in the form of a user ID in Telegram.

Create a bot in Telegram

Go to @BotFather and create a bot:

Connecting libraries

We connect the library for creating telegram bots:

pip3 install aiogram

And also for working with MongoDB:

pip3 install pymongo

Write the code

Create a main.py file and write the beginning for any bot:

bot = Bot(token='BotFather TOKEN HERE')
storage = MemoryStorage() # storage that allows you to remember the states of dialogs with users
dp = Dispatcher(bot, storage=storage)

Let's add one single state that we need to send messages:

class Stage(StatesGroup):
    send_message_to = State()

Let's write a handler for the /start command:

@dp.message_handler(commands=['start'], state=['*', Stage.send_message_to])
async def start(message: types.Message, state: FSMContext):
    args = message.get_args() # get payload from command /start
    payload = decode_payload(args)

    if payload: # we go here if we clicked on someone's link
        async with state.proxy() as data:
            data['user-id'] = payload # save the recipient ID in the context of the dialog

        await Stage.send_message_to.set() # set the state of sending the message to the recipient

        start_text = 'Hello! I am a bot for anonymous communication. Send a message to the person who posted this link. He won't know who sent the message.'
        await message.reply(start_text)
        return

    # if there is no payload, then the user just pressed /start
    link = await get_start_link(message.from_user.id, encode=True) # create a personal link
    content = f'Your personal link: {link}'
    await message.reply(content)

Let's write a handler for sending messages:

@dp.message_handler(state=Stage.send_message_to, content_types=ContentType.ANY)
async def process_message(message: types.Message, state: FSMContext):
    inline_btn = InlineKeyboardButton('Reply', callback_data=f'answer_{message.from_user.id}')
    inline_kb = InlineKeyboardMarkup().add(inline_btn)
    async with state.proxy() as data:
        if 'answer_user_id' in data:
            user_id = data['answer_user_id'] # if we reply to a message
            await message.answer('Your response has been delivered.')
            await message.bot.send_message(user_id, 'You have received an answer.')
        else:
            user_id = data['user-id'] # if we send a new message
            await message.answer('Your message has been sent.')
            await message.bot.send_message(user_id, 'You have received a new anonymous message.')
        if message.video:
            await message.bot.send_video(user_id, message.video.file_id, reply_markup=inline_kb)
            content = message.video.as_json()
        elif message.video_note:
            await message.bot.send_video_note(user_id, message.video_note.file_id, reply_markup=inline_kb)
            content = message.video_note.as_json()
        elif message.voice:
            await message.bot.send_voice(user_id, message.voice.file_id, reply_markup=inline_kb)
            content = message.voice.as_json()
        elif message.photo:
            await message.bot.send_photo(user_id, message.photo[-1].file_id, reply_markup=inline_kb)
            content = message.photo[-1].as_json()
        elif message.audio:
            await message.bot.send_audio(user_id, message.audio.file_id, reply_markup=inline_kb)
            content = message.audio.as_json()
        elif message.sticker:
            await message.bot.send_sticker(user_id, message.sticker.file_id, reply_markup=inline_kb)
            content = message.sticker.file_id
        elif message.document:
            await message.bot.send_document(user_id, message.document.file_id, reply_markup=inline_kb)
            content = message.document.as_json()
        else:
            await message.bot.send_message(user_id, message.text, reply_markup=inline_kb)
            content = message.text
    await state.finish()

In the example above, we implemented the ability to send various types of messages, but you can leave only text.

Let's write a click handler for the "Reply" button:

answer_regexp = re.compile("answer_(.*)")


@dp.callback_query_handler(lambda c: answer_regexp.match(c.data), state=['*', Stage.send_message_to])
async def process_callback_answer(callback_query: types.CallbackQuery, state: FSMContext):
    await bot.answer_callback_query(callback_query.id)
    await bot.send_message(callback_query.from_user.id, 'Enter your reply.')
    async with state.proxy() as data:
        data['answer_user_id'] = answer_regexp.match(callback_query.data).group(1)
    await Stage.send_message_to.set()

It remains to run the bot:

if __name__ == '__main__':
    executor.start_polling(dp, skip_updates=False)

Working with the database

Let's create a db.py file and mark up methods for working with the database.

Connecting to the database:

client = MongoClient('CONNECTION_STRING HERE')
db = client['AskFM']

users = db['users']
messages = db['messages']

Adding a user:

def add_user(user_id, username, datetime):
    user_filter = {
        'user_id': user_id
    }
    user = users.find_one(user_filter)
    if user and user['status'] == 'deleted':
        update = {
            'status': 'active'
        }
        users.update_one(user_filter, {'$set': update})
        return
    elif user:
        return
    data = {
        'user_id': user_id,
        'username': username,
        'datetime': datetime,
        'status': 'active'
    }
    users.insert_one(data)

Delete user:

def delete_user(user_id):
    user_filter = {
        'user_id': user_id
    }
    update = {
        'status': 'deleted'
    }
    users.update_one(user_filter, {'$set': update})

Adding a message:

def add_message(user_id_from, user_id_to, username_from, username_to, content, message_type, datetime):
    message = {
        "user_id_from": user_id_from,
        "user_id_to": user_id_to,
        "username_from": username_from,
        "username_to": username_to,
        "content": content,
        "message_type": message_type,
        "datetime": datetime
    }
    messages.insert_one(message)

Integrating work with the database into the bot

import db

Let's write a handler that tracks the activation and deactivation of the bot:

@dp.my_chat_member_handler()
async def add_to_channel(update: ChatMemberUpdated):
    if update.new_chat_member.status == 'member':
        db.add_user(update.from_user.id, update.from_user.username, time.time())
    elif update.new_chat_member.status == 'kicked':
        db.delete_user(update.from_user.id)

Add user registration to the /start method:

db.add_user(message.from_user.id, message.from_user.username, time.time())

Final handler code:

@dp.message_handler(commands=['start'], state=['*', Stage.send_message_to])
async def start(message: types.Message, state: FSMContext):
    args = message.get_args() # get payload from /start command
    payload = decode_payload(args)

    if payload: # we go here if we clicked on someone's link
        async with state.proxy() as data:
            data['user-id'] = payload # save the recipient ID in the context of the dialog

        await Stage.send_message_to.set() # set the state of sending the message to the recipient

        start_text = 'Hello! I am a bot for anonymous communication. Send a message to the person who posted this link. He won't know who sent the message.'
        await message.reply(start_text)

        db.add_user(message.from_user.id, message.from_user.username, time.time()) # added here

        return

    # if there is no payload, then the user just pressed /start
    link = await get_start_link(message.from_user.id, encode=True) # create a personal link
    content = f'Your personal link: {link}'
    await message.reply(content)

    db.add_user(message.from_user.id, message.from_user.username, time.time()) # and added here

Why add user registration to the /start method if we have a separate handler that looks at the activation and deactivation of the bot? Answer: that method does not always work correctly.

Add message registration to the very end of the message send handler:

user_to = await bot.get_chat_member(user_id, user_id)
if 'answer_user_id' in data:
    message_type = 'answer'
else:
    message_type = 'question'
db.add_message(message.from_user.id,
               user_id, message.from_user.username,
               user_to.user.username, content,
               message_type, time.time())

This is the end of the code. The finished code can be viewed here.

We start the bot and try to test it:

python3 main.py

In the next part, we will look at creating the dashboard itself.