Obstacles encountered in the Messenger chatbot development

I have been working on a Messenger chatbot as a side project for the last couple of months. Tech-stack I'm using on it includes Node.js with TypeScript, NestJS as a back-end framework, Bottender as a chatbot framework, Redis for session storage, and TypeORM with PostgreSQL as the primary database.

This blog post covers some of the obstacles encountered in the development process and their solutions or workarounds.

Preventing malicious requests to the webhook endpoint

Signature verification helps to prevent malicious requests. It is a mechanism that checks if requests to the Messenger webhook URL are genuine.

HTTP request should contain an X-Hub-Signature header which includes the SHA1 signature of the request payload, using the app secret as the key and prefixed with sha1=. Bottender provides signature verification out of the box.

// src/common/guards/signature-verification.guard.ts
@Injectable()
export class SignatureVerificationGuard implements CanActivate {
  constructor(private readonly configService: ConfigService) {}

  canActivate(context: ExecutionContext): boolean {
    const {
      rawBody,
      headers: { 'x-hub-signature': signature },
    } = context.switchToHttp().getRequest();
    const { sha1 } = parse(signature);
    if (!sha1) return false;

    const appSecret = this.configService.get('MESSENGER_APP_SECRET');
    const digest = createHmac('sha1', appSecret).update(rawBody).digest('hex');
    const hashBufferFromBody = Buffer.from(`sha1=${digest}`, 'utf-8');
    const bufferFromSignature = Buffer.from(signature, 'utf-8');

    if (hashBufferFromBody.length !== bufferFromSignature.length)
      return false;

    return timingSafeEqual(hashBufferFromBody, bufferFromSignature);
  }
}

// src/modules/webhook/webhook.controller.ts
@UseGuards(SignatureVerificationGuard)
@Post()
@HttpCode(HttpStatus.OK)
handleWebhook(@Body() data) {
  // ...
}

Communication between Messenger extension and chatbot

For some complicated inputs from the user, such as a datetime picker, it is recommended to use a Messenger extension with a web view, where webpages can be loaded inside the Messenger app.

Protect the extension page with a CSRF token to prevent malicious requests. Request from the extension to the chatbot should be transformed and signed inside a middle endpoint (to avoid exposing app secret in a web view webpage) and sent to the webhook endpoint.

User's location

Users can share locations as an attachment, but that doesn't guarantee the location is one where the user is located.

Messenger deprecated quick replies for sharing user's location. One workaround would be to get the user's location with the Messenger extension. This solution works only with the Messenger app since Facebook and Messenger websites don't allow sharing location within iframes.

Data can be filtered by the postgis extension for a specific radius based on the user's location.

Timezones

Showing the datetime in the right timezone

Datetimes are stored in UTC format in the database. Since chatbots can be used across different timezones, the default timezone should be set to UTC so the chatbot can show the correct datetime for the corresponding timezone.

Date object will use UTC as the default timezone if the environment variable TZ has a value UTC. The snippet below sets datetime with the right timezone. It implies that the environment variable TZ is set correctly.

import { utcToZonedTime } from 'date-fns-tz';

const zonedTime = utcToZonedTime(datetime, timezone).toLocaleDateString(locale, options );

Timezone column format

Messenger sends the user's timezone as a number relative to GMT. Most of the libraries use timezone in the IANA timezone name format.

To avoid mapping all of the timezones with their offsets, the user's timezone (when the user sends the location) can be gotten by using the geo-tz package.

import geoTz from 'geo-tz';

// ...
const timezone = geoTz(latitude, longitude);
// ...

Multi-language chatbot, internationalization

Three independent parts of the chatbot should be internationalized. The first part is the chatbot locale based on a user's language. i18n package is used in this project as a dynamic module, it supports the advanced message format which can process the messages based on gender and singular/plural words.

The other two parts are provided by Messenger API, persistent menu, and greeting text. Persistent menu and greeting text could be shown in different languages based on which language the user uses, locale property configures persistent menu and greeting text for the specific language.

export const GREETING_TEXT: MessengerTypes.GreetingConfig[] = [
  {
    locale: 'en_US',
    text: greetingText,
  },
  // ...
  {
    locale: 'default',
    text: greetingText,
  },
];
export const PERSISTENT_MENU: MessengerTypes.PersistentMenu = [
  {
    locale: 'en_US',
    callToActions: persistentMenu,
    composerInputDisabled: false,
  },
  // ...
  {
    locale: 'default',
    callToActions: persistentMenu,
    composerInputDisabled: false,
  },
];

Some of the supported locales aren't synchronized across the Facebook website and Messenger app.

If the Messenger app doesn't support some language, it will use en_US as the default locale.

Sessions

The session state is the temporary data regarding the corresponding conversation. Bottender supports several drivers for session storage (memory, file, Redis, and MongoDB) by default.

// ...
context.setState({
  counter: 0,
});
// ...
context.resetState();
// ...

Parsing payloads

A payload can contain several parameters, so it could follow a query string format and be parsed with parse function from querystring module.

import { parse } from 'querystring';
// ...
const buttons = [{
  type: 'postback',
  title,
  payload: `type=${TYPE}&id=${ID}`,
}];
// ...
handlePostback = async (context: MessengerContext) => {
  const { type, id } = parse(context.event.postback.payload);
  switch (type) {
    // ...
  }
  // ...
};

Setting up Messenger profile

Messenger profile allows you to set up the persistent menu, greeting text, get started payload, and Messenger extensions domain whitelist.

Bottender (1.4) doesn't support a custom GraphAPI version. It supports 6.0 by default, so it has some restrictions regarding persistent menu buttons number. GraphAPI version 8 allows a persistent menu with up to 20 buttons, which must be handled with a script.

// scripts/set-messenger-profile.ts
import { MessengerClient } from 'messaging-api-messenger';

const client = new MessengerClient({
  // ...
  version: '8.0',
});

client
  .setMessengerProfile({
    getStarted: {
      payload: GET_STARTED_PAYLOAD,
    },
    greeting: GREETING_TEXT,
    persistentMenu: PERSISTENT_MENU,
    whitelistedDomains: [process.env.MESSENGER_EXTENSIONS_URL],
  })
// ...

Bottender with custom NestJS server

Bottender calls handler every time the message is received. bootstrap and handler should use the same application instance across the service.

// src/index.ts
export default async function handler() {
  const app = await application.get();
  const chatbotService = app
    .select(BotsModule)
    .get(BotsService, { strict: true });

  return chatbotService.getRouter();
}

// src/main.ts
async function bootstrap(): Promise<void> {
  const app = await application.get();
  // ...
}

Setup for development environment

Ngrok creates a secure public URL pointing to the local server, while Bottender enables webhook integrations.

npx ngrok http 3000
npm run messenger-webhook:set <NGROK_URL>/<WEBHOOK_ENDPOINT>

Demo

Here is the link to the chatbot codebase.

Comments

[valery]

Interesting. Messenger chatbot development servreality.com can face various obstacles, but this is a great opportunity to overcome them and create a unique solution.