DEV Community: Irina Maximova

Continuous integration for a CPaaS for voice, video & messaging

Irina Maximova — Tue, 31 May 2022 11:45:54 +0000

Hey, devs! Long time no see! Today I’m going to tell you about VoxEngine CI – a new tool to work with Voximplant apps, scenarios, and rules.

I hope you will find this interesting even if you don’t have a Voximplant account yet. Especially if you are a continuous integration fan :)

VoxEngine CI aims to help you manage Voximplant applications, rules, and scenarios from your own environment using @voximplant/apiclient-nodejs under the hood. With it, you can switch to local development and use your favorite tools and version control systems. If you already have those favorites, it would be nice to connect them with Voximplant by means of VoxEngine CI.

We also prepared CI/CD templates for GitLab and GitHub. They will help you set up processes for uploading your code to the Voximplant platform.

So let’s begin!

Prerequisites

Voximplant account, you can create one here
Service account (we will create it together)
Node.js >= 11
Happy mood (optional, any mood is welcome)

Here I assume that you created a Voximplant account on the main page by yourself and are ready to move to the second step.

Now and later, you can also follow a small tutorial showing how to create a Voximplant cloud app to get an idea of what Voximplant is.

Create a service account

To work with VoxEngine CI, you need a JSON file with your credentials. Go to the Service accounts section of the Voximplant control panel and generate the file there. Choose the Developer role, it suits today’s tutorial best.

Configure CI

To install VoxEngine CI, run the following command from your project folder (create the project first if you do not have it yet) in the Terminal:
npm i @voximplant/voxengine-ci
Create a .env file in the project root directory and add environment-specific variables to specify the name and location of the JSON file with your account credentials and the folder with files that will be downloaded from your account in the next step:
```
VOX_CI_CREDENTIALS=/path/to/the/vox_ci_credentials.json
VOX_CI_ROOT_PATH=/path/to/the/voxengine_ci_source_files_directory
```
Initialize the project. This command downloads all files and metadata from your Voximplant account. When all the files are downloaded, you can modify them and upload them back to the platform.
npx voxengine-ci init

Create apps, scenarios, and rules with CI

It doesn’t matter whether you already have some apps, scenarios, and rules in your account or not. You can either create or just modify them with VoxEngine CI.

While using VoxEngine CI, please don’t rename or delete existing apps, scenarios, and rules; just create new ones, otherwise consistency will be affected.
If you truly need to rename or delete something, do it from the platform and run npx voxengine-ci init --force after to make your local and remote (platform) versions consistent.

Applications

Here you need to create an application.config.json file in the /path/to/the/voxengine_ci_source_files_directory/applications/your-application-name.your-account-name.voximplant.com/ directory (account name is your Voximplant username), where /path/to/the/voxengine_ci_source_files_directory is created with the npx voxengine-ci init command.
This is the default voxfiles folder or the path you specified in the VOX_CI_ROOT_PATH env variable.

Then, add the following config to the application.config.json file:

{
    "applicationName": "your-application-name.your-account-name.voximplant.com"
}

Congratulations! Your first application is ready!

Rules and scenarios

In the same directory, create a rules.config.json file with this config, for example:

[
   {
      "ruleName":"first-rule",
      "scenarios":[
         "first-scenario"
      ],
      "rulePattern":"string-with-regexp"
   },
   {
      "ruleName":"second-rule",
      "scenarios":[
         "second-scenario"
      ],
      "rulePattern":"string-with-regexp"
   }
]

Here, first-rule and second-rule are the names of your rules; first-scenario and second-scenario are the names of your scenarios; and string-with-regexp is a regular expression that a dialed number will be checked against (".*" by default).

Create as many rules and scenarios as you need. But note that the order of the scenarios attached to one rule in this file defines the order of their execution. Keep this in mind when writing the scenario code.

Here is an example of such a file:

If your application doesn’t have any rules yet, leave the JSON file empty {}.

You can modify existing scenarios and create new ones ONLY in the /voxfiles/scenarios/src directory. Only the scenarios with their names specified in rules.config.json will be uploaded to the platform. The scenario file names must match the *.voxengine.{js,ts} pattern.

In our case, the scenario names should be as follows: testscenario1.voxengine.js, testscenario2.voxengine.js. These are the files where you write the scenario code.

When everything is ready, run
npx voxengine-ci upload --application-name your-application-name

to upload a new application with all rules and scenarios to the platform.

Use the --dry-run flag in this command to build the project locally without uploading changes to the platform like this: npx voxengine-ci upload --application-name your-application-name --dry-run

Now you can find the newly created applications, rules, and scenarios on the platform:

Modify scenarios and rules with CI

Add/modify rules in rules.config.json and scenarios in /voxfiles/scenarios/src and then run the upload command again.
If you specify a rule name or rule ID in the command, only the scenarios attached to this rule will be uploaded to the platform:

npx voxengine-ci upload --application-name your-application-name --rule-name your-rule-name

It works either when you upload a new rule or when you modify an existing one.

If you modify an existing application or existing rule, you can specify --application-id and --rule-id instead of --application-name and --rule-name, but if both are specified only the ID will be taken into account.

So that’s how it all works. Find more commands and CI/CD templates for GitHub and Gitlab here. Stay tuned and in touch :)
I’ll also be happy to answer any questions you may want to ask after reading this article. Have a good day!

Masking phone numbers of couriers and customers using a key-value store

Irina Maximova — Fri, 17 Sep 2021 15:40:44 +0000

The pandemic has greatly affected delivery services, and they have become more in demand than ever before. As we all know, a courier and a customer need each other's phone numbers to make a call and discuss the order details. But what about privacy? Many delivery services have already thought this question through, and each uses its own number masking solution. I, in turn, want to tell you how to mask phone numbers with the Voximplant key-value store. So let’s get the party started!

How it works

We will create a scenario that allows a customer and a courier to make calls without knowing each other’s phone numbers.

Meanwhile, we’ll have only one so-called “neutral” phone number to call for both a courier and a customer. We’ll rent this number in the Voximplant panel. Then, we’ll create some data structure to connect a courier and a customer with each other by an order number (a key if we are referring to a key-value store).

When calling a rented number, a caller enters their order number. If such an order exists in the database, our scenario checks the numbers attached to it. Then, if it identifies a phone number as a customer’s, we put them through to the courier responsible for the order, and vice versa.

For example, a call from a courier to a customer looks like this:

If it doesn’t find the caller’s phone number in the database, we suggest they call again from the number they used when placing the order. Or we simply transfer them to an operator.

Let’s go straight to the implementation.

What you need

A Voximplant account which you can create here;
A Voximplant application with a scenario and a rule for this scenario (we’ll create it all together);
Test phone numbers: a number rented from Voximplant, a courier’s, a customer’s, and an operator’s numbers. In the test version, we can omit an operator’s number.

To begin, log into your Voximplant account: manage.voximplant.com/auth. In the menu on the left, click Applications and then click Create application in the upper right corner. Give it a name (for example, numberMasking) and click Create.
Open a newly created app and go to Scenarios. Create a scenario by clicking the “+” button and name it kvs-scenario. You will work with the code here, but everything is not quite ready yet; wait until we add code to the scenario.
Go to the Routing tab and create a rule for the scenario. Leave the pattern (regular expression) “.*” as default. This way your rule will work for all phone numbers.
Rent a real phone number. To do that, go to the Numbers section, select one and purchase it. This number will be called by a courier and a customer and will be displayed to them instead of their personal numbers.

Please note that phone numbers are offered on-demand, except in countries where regulations require additional steps. You will find more information when renting a phone number.

In Voximplant, you can also rent test numbers to see how everything works. In our case, we need a real phone number to initiate a call from the platform.
The last thing is to attach the phone number to your application. To do that, open your application, go to the Numbers → Available and click Attach. Here you can also attach your rule so it works for incoming calls, and all other rules are ignored.

Great! The structure is ready, now you just need to fill the key-value store and add some code to the scenario.

Key-value store

The scenario works properly if the key-value store isn’t empty. To fill it up, use the Voximplant Management API. I use the Python API client. It requires Python 2.x or 3.x with pip and setuptools> = 18.5 installed.

Go to your project folder and install the SDK using pip:
python -m pip install --user voximplant-apiclient

Create a .py file and write the code that adds the order details to the key-value store. The set_key_value_item will help you do that:

from voximplant.apiclient import VoximplantAPI, 
VoximplantException

if __name__ == "__main__":
    voxapi = VoximplantAPI("credentials.json")

    # SetKeyValueItem example

    KEY = 12345
    VALUE = '{"courier": "12222222222", "client": 
"13333333333"}'
    APPLICATION_ID = 1
    TTL = 864000

    try:
        res = voxapi.set_key_value_item(KEY,
            VALUE,
            APPLICATION_ID,
            ttl=TTL)
        print(res)
    except VoximplantException as e:
        print("Error: {}".format(e.message))

You can generate a credentials.json file yourself when creating a service account in the Service accounts section. Don’t forget to choose a role that will allow you to call the set_key_value_item method. Owner, for example, will do well.

Find the APPLICATION_ID in the address bar when navigating to your app.

We use a five-digit order number as a key (KEY) and phone numbers as values. TTL here is to specify the values' storage period.

Finally, run the file to save the order details:

python kvs.py

In case you no longer want a client and a courier to disturb each other, you can delete the order details from the store. Find all the available key-value store methods in our documentation: management API and VoxEngine.

Scenario code

The kvs-scenario code is below, you can copy it as is. You just need to do one more thing – specify the number you rented in the Voximplant control panel as callid in the form of "10000000000":

Full scenario code

require(Modules.ApplicationStorage);

/**
 * @param {boolean} repeatAskForInput - whether the input request was repeated
 * @param longInputTimerId - timer for the absence of input
 * @param shortInputTimerId - timer for triggering the phrase about contacting an operator
 * @param {boolean} firstTimeout - indicator of the first timeout
 * @param {boolean} wrongPhone - indicator that the caller's number matches the number from storage
 * @param {boolean} inputRecieved - whether the input was received from the user
 * 
 */

let repeatAskForInput;
let longInputTimerId;
let shortInputTimerId;
let firstTimeout = true;
let wrongPhone;
let inputRecieved;

const store = {
    call: null,
    caller: '',
    callee: '',
    callid: 'phone number rented in the panel',
    operator_call: null,
    operatorNumber: '',
    input: '',
    data: {
        call_operator: '',
        order_number: '',
        order_search: '',
        phone_search: '',
        sub_status: '',
        sub_available: '',
        need_operator: '',
        call_record: ''
    }
}

const phrases = {
    start: 'Hello. Please -- enter the five-digit order number in tone mode.',
    repeat: 'Please -- enter the five-digit order number in tone mode, or press pound to contact an operator.',
    noInputGoodbye: 'You have not chosen anything. You can look up the order number in the text message and call us again. Goodbye, have a nice day!',
    connectToOpearator: 'To contact an operator, press pound.',
    connectingToOpearator: 'Stay on the line, putting you through to an operator.',
    operatorUnavailable: 'Unfortunately, all operators are busy.. Please,,, call back later. Goodbye, have a nice day!',
    wrongOrder: 'Order number is not found. Look up the order number in the text message and enter it in tone mode. Or contact an operator by pressing pound.',
    wrongOrderGoodbye: 'You didn’t choose anything.. Goodbye, have a nice day!',
    wrongPhone: 'Phone number is not found. If you are a customer, please call back from the number you used to place your order. If you are a courier, please call back from the number that is registered in our system. Or contact an operator by pressing pound.',
    wrongPhoneGoodbye: 'You didn’t choose anything.. Goodbye, have a nice day!',
    courierIsCalling: `A courier is calling you about the order delivery, - - ${store.data.order_number}`,
    clientIsCalling: `A customer is calling you about the order delivery, - - ${store.data.order_number}`,
    courierUnavailable: 'It seems like the courier is unavailable at the moment. Please call back in a couple of minutes. Goodbye, have a nice day!',
    clientUnavailable: 'It seems like the customer is unavailable at the moment. Please call back in a couple of minutes. Goodbye, have a nice day!',
    waitForCourier: 'Stay on the line, putting you through to the courier.',
    waitForClient: 'Stay on the line, putting you through to the customer.'
}


VoxEngine.addEventListener(AppEvents.Started, async e => {
    VoxEngine.addEventListener(AppEvents.CallAlerting, callAlertingHandler);
})

async function callAlertingHandler(e) {
    store.call = e.call;
    store.caller = e.callerid;
    store.call.addEventListener(CallEvents.Connected, callConnectedHandler);
    store.call.addEventListener(CallEvents.Disconnected, callDisconnectedHandler);
    store.call.answer();
}

async function callDisconnectedHandler(e) {
    await sendResultToDb();
    VoxEngine.terminate();
}

async function callConnectedHandler() {
    store.call.handleTones(true);
    store.call.addEventListener(CallEvents.RecordStarted, (e) => {
        store.data.call_record = e.url;
    });
    store.call.record();
    store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
    await say(phrases.start);
    addInputTimeouts();
}

function dtmfHandler(e) {
    clearInputTimeouts();
    store.input += e.tone;
    Logger.write('Entered digit is ' + e.tone)
    Logger.write('Full number ' + store.input)
    if (e.tone === '#') {
        store.data.need_operator = "Yes";
        store.call.removeEventListener(CallEvents.ToneReceived);
        store.call.handleTones(false);
        callOperator();
        return;
    }

    if (!wrongPhone) {
        if (store.input.length >= 5) {
            repeatAskForInput = true;
            Logger.write(`Received number is ${store.input}. `);
            store.call.handleTones(false);
            store.call.removeEventListener(CallEvents.ToneReceived);
            handleInput(store.input);
            return;
        }
    }
    addInputTimeouts();
}

function addInputTimeouts() {
    clearInputTimeouts();
    if (firstTimeout) {
        Logger.write('Timer for the phrase about contacting an operator is triggered');
        shortInputTimerId = setTimeout(async () => {
            await say(phrases.connectToOpearator);
        }, 1500);
        firstTimeout = false;
    }

    longInputTimerId = setTimeout(async () => {
        Logger.write('Timer for no input from the user is triggered ' + longInputTimerId);
        store.call.removeEventListener(CallEvents.ToneReceived);
        store.call.handleTones(false);
        if (store.input) {
            handleInput(store.input);
            return;
        }
        if (!repeatAskForInput) {
            Logger.write('Asking the caller to re-enter the number');
            store.call.handleTones(true);
            store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
            await say(phrases.repeat);
            addInputTimeouts();
            repeatAskForInput = true;
        } else {
            Logger.write('Number is not entered. Ending the call');
            await say(inputRecieved ? phrases.wrongOrderGoodbye : phrases.noInputGoodbye);
            store.call.hangup();
        }

    }, 8000);
    Logger.write('Timer for no input from the user is triggered ' + longInputTimerId);
}

function clearInputTimeouts() {
    Logger.write(`Clearing the timer ${longInputTimerId}. `);
    if (longInputTimerId) clearTimeout(longInputTimerId);
    if (shortInputTimerId) clearTimeout(shortInputTimerId);
}

async function handleInput() {
    store.data.order_number = store.input;
    Logger.write('Looking for a match in the key-value store by the entered number: ' + store.input)
    inputRecieved = true;
    let kvsAnswer = await ApplicationStorage.get(store.input);
    if (kvsAnswer) {
        store.data.order_search = 'Order is found';
        Logger.write('Received response from kvs: ' + kvsAnswer.value)
        let { courier, client } = JSON.parse(kvsAnswer.value);

        if (store.caller == courier) {
            Logger.write('Courier is calling')
            store.callee = client;
            store.data.sub_status = 'Courier';
            store.data.phone_search = 'Phone number is found';
            callCourierOrClient();
        } else if (store.caller == client) {
            Logger.write('Customer is calling')
            store.callee = courier;
            store.data.sub_status = 'Customer';
            store.data.phone_search = 'Phone number is found';
            callCourierOrClient();
        } else {
            Logger.write('Number of the caller does not match the numbers received from kvs');
            wrongPhone = true;
            store.data.phone_search = 'Phone number is not found';
            store.input = '';
            store.call.handleTones(true);
            store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
            await say(phrases.wrongPhone);
            addInputTimeouts();
        }

    } else {
        Logger.write('No match in kvs for the entered number');
        store.data.order_search = 'Order is not found';
        store.input = '';
        store.call.handleTones(true);
        store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
        await say(phrases.wrongOrder);
        Logger.write(`Clearing the timer ${longInputTimerId}. `);
        addInputTimeouts();

    }

}

async function callCourierOrClient() {
    clearInputTimeouts();
    Logger.write('Starting a call to the courier/customer');
    await say(store.data.sub_status === 'Courier' ? phrases.waitForClient : phrases.waitForCourier, store.call);
    const secondCall = VoxEngine.callPSTN(store.callee, store.callid);
    store.call.startPlayback('http://cdn.voximplant.com/toto.mp3');
    secondCall.addEventListener(CallEvents.Connected, async () => {
        store.data.sub_available = 'Yes';
        await say(store.data.sub_status === 'Courier' ? phrases.courierIsCalling : phrases.clientIsCalling, secondCall);
        store.call.stopPlayback();
        VoxEngine.sendMediaBetween(store.call, secondCall);
    });
    secondCall.addEventListener(CallEvents.Disconnected, () => {
        store.call.hangup();
    });
    secondCall.addEventListener(CallEvents.Failed, async () => {
        store.data.sub_available = 'No';
        store.call.stopPlayback();
        await say(store.data.sub_status === 'Courier' ? phrases.clientUnavailable : phrases.courierUnavailable, store.call);
        store.call.hangup();
    });
}

async function callOperator() {
    Logger.write('Starting a call to an operator');
    await say(phrases.connectingToOpearator, store.call);
    store.call.startPlayback('http://cdn.voximplant.com/toto.mp3');
    store.operator_call = VoxEngine.callPSTN(store.operatorNumber, store.callid);
    store.operator_call.addEventListener(CallEvents.Connected, async () => {
        store.data.call_operator = 'Operator is free';
        VoxEngine.sendMediaBetween(store.call, store.operator_call);
    });
    store.operator_call.addEventListener(CallEvents.Disconnected, () => {
        store.call.hangup();
    });
    store.operator_call.addEventListener(CallEvents.Failed, async () => {
        store.data.call_operator = 'Operator is busy';
        await say(phrases.operatorUnavailable, store.call);
        store.call.hangup();
    });
}


async function sendResultToDb() {
    Logger.write('Data to be sent to the database');
    Logger.write(JSON.stringify(store.data));
    const options = new Net.HttpRequestOptions();
    options.headers = ['Content-Type: application/json'];
    options.method = 'POST';
    options.postData = JSON.stringify(store.data);
    await Net.httpRequestAsync('https://voximplant.com/', options);
}


function say(text, call = store.call, lang = VoiceList.Amazon.
en_AU_Nicole) {
    return new Promise((resolve) => {
        call.say(text, lang);
        call.addEventListener(CallEvents.PlaybackFinished, function callback(e) {
            resolve(call.removeEventListener(CallEvents.PlaybackFinished, callback));
        });
    });
};

The code is thoroughly commented on but let's go into more detail on some points.

Enter the order number

The first thing we do when a call arrives is ask a caller to enter the order number and handle it using the dtmfHandler function.

store.input += e.tone;

If the caller enters #, put them through to the operator:

if (e.tone === '#') {
    store.data.need_operator = "Yes";
    store.call.removeEventListener(CallEvents.ToneReceived);
    store.call.handleTones(false);
    callOperator();
    return;
}

If they enter a 5-digit number, call the handleInput method:

if (store.input.length >= 5) {
    repeatAskForInput = true;
    Logger.write('Received number is ${store.input}. ');
    store.call.handleTones(false);
    store.call.removeEventListener(CallEvents.ToneReceived);
    handleInput(store.input);
    return;
}

Search the order

It’s time to compare the entered number with order numbers in the store using the ApplicationStorage.get() method and the entered number as a key here:

store.data.order_number = store.input;
Logger.write('Looking for a match in the key-value store by the entered number: ' + store.input)
inputRecieved = true;
let kvsAnswer = await ApplicationStorage.get(store.input);

If the order is found, get the courier and customer phone numbers connected with it:

if (kvsAnswer) {
    store.data.order_search = 'Order is found';
    Logger.write('Received response from kvs: ' + kvsAnswer.value)
    let { courier, client } = JSON.parse(kvsAnswer.value);

Now we need to figure out to whom to call. If the caller’s number is the courier’s, forward a call to the customer, if it’s the customer’s – to the courier. The callCourierOrClient function is intended for this:

if (store.caller == courier) {
    Logger.write('Courier is calling')
    store.callee = client;
    store.data.sub_status = 'Courier';
    store.data.phone_search = 'Phone number is found';
    callCourierOrClient();
} else if (store.caller == client) {
    Logger.write('Customer is calling')
    store.callee = courier;
    store.data.sub_status = 'Customer';
    store.data.phone_search = 'Phone number is found';
    callCourierOrClient();
}

If the number is not in the store, ask a caller to call again from the number they used when placing the order:

else {
    Logger.write('Number of the caller does not match the numbers received from kvs');
    wrongPhone = true;
    store.data.phone_search = 'Phone number is not found';
    store.input = '';
    store.call.handleTones(true);
    store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
    await say(phrases.wrongPhone);
    addInputTimeouts();
}

Finally, handle what happens when the order number is not in the store. In such a case, ask the caller to make sure the number is correct and enter it again:

else {
    Logger.write('No match in kvs for the entered number');
    store.data.order_search = 'Order is not found';
    store.input = '';
    store.call.handleTones(true);
    store.call.addEventListener(CallEvents.ToneReceived, dtmfHandler);
    await say(phrases.wrongOrder);
    Logger.write(`Clearing the timer ${longInputTimerId}. `);
    addInputTimeouts();
}

Call the customer/courier

Let's go directly to the call, i.e. to the callCourierOrClient function. Here we tell the caller that we're transferring their call to the courier/client and play music on hold. We use the callPSTN method to call the client or the courier (depending on whose number was previously identified as the caller's number):

await say(store.data.sub_status === 'Courier' ? phrases.waitForClient : phrases.waitForCourier, store.call);
const secondCall = VoxEngine.callPSTN(store.callee, store.callid);
store.call.startPlayback('http://cdn.voximplant.com/toto.mp3');

At the same time, we tell the callee that the call is about clarification of information on the order:

secondCall.addEventListener(CallEvents.Connected, async () => {
    store.data.sub_available = 'Yes';
    await say(store.data.sub_status === 'Courier' ? phrases.courierIsCalling : phrases.clientIsCalling, secondCall);
    store.call.stopPlayback();
    VoxEngine.sendMediaBetween(store.call, secondCall);
});

Then, handle the disconnect event:

secondCall.addEventListener(CallEvents.Disconnected, () => {
    store.call.hangup();
});

Notify the caller if the callee is unavailable:

secondCall.addEventListener(CallEvents.Failed, async () => {
    store.data.sub_available = 'No';
    store.call.stopPlayback();
    await say(store.data.sub_status === 'Courier' ? phrases.clientUnavailable : phrases.courierUnavailable, store.call);
    store.call.hangup();
});

The say method is responsible for all the phrases the robot utters. The phrases themselves are in the phrases associative array. We use Amazon as a TTS provider, the voice of Nicole:

function say(text, call = store.call, lang = VoiceList.Amazon.
en_AU_Nicole) {
    return new Promise((resolve) => {
        call.say(text, lang);
        call.addEventListener(CallEvents.PlaybackFinished, function callback(e) {
            resolve(call.removeEventListener(CallEvents.PlaybackFinished, callback));
        });
    });
};

Among other things, our scenario records the calls using the record method and enables you to save statistics to the database. In our code, the sendResultToDb function handles this. This is very important for business because it allows you to analyze statistics, provide quality control and quickly resolve any issues that might arise during the delivery process.

Test the app

When you add the full code to the scenario and the order details to the storage, feel free to start testing.

Let's call from the customer's or courier's phone number to the number rented in the panel. Then enter the order number (in our case, it is 12345) and wait for the connection with the other party.

If we do everything correctly, the customer and the courier will be able to call each other and discuss the details of the order without knowing each other's personal numbers, and thus without any privacy issues.

I’m glad you read all the way to the end of this article. It means it was gripping, right? :) Anyway, I wish you successful development and trouble-free order delivery! Stay tuned for more helpful articles in the future.

How we make Flutter work with CallKit Call Directory

Irina Maximova — Fri, 25 Jun 2021 12:53:56 +0000

Disclaimer: This will be a long read that my colleague wrote for Habr.ru and I decided to translate to share with you guys!

Stock up on snacks, make yourself comfortable, and let’s begin!

In this long read, I will tell you how we (at Voximplant) decided to create our own Flutter plugin to use CallKit in a Flutter app. And appeared to be the first who made call blocking & identification work for Flutter using Call Directory.

What is CallKit?

Apple CallKit is a framework used for integrating calls from 3rd party apps into the system.

If a call from a 3rd party app is displayed as native, it means that CallKit is used here. If a call from a 3rd party app is in the list of system application calls (Phone) – it is also CallKit. Third-party applications that act as a caller ID – CallKit. Calls from third-party apps that can't get through Do Not Disturb mode – well, you get the idea.

CallKit provides third-party developers with a system UI for displaying calls

What’s with CallKit on Flutter?

CallKit is a part of iOS SDK, but it can be accessed from Flutter by interacting with native code. To use the framework’s functionality, you need to connect a third-party plugin that encapsulates the Flutter interaction with iOS. Or you can implement everything yourself, for example, this way:

An example of CallKit service implementation for Flutter, where the iOS application code (platform code) connects Flutter application to the system

Ready-made CallKit Flutter solutions

So, we needed to integrate our Flutter application for VoIP calls with the system. First, we looked through most of the existing third-party solutions and chose one of them to use for a while. However, this and the rest of the available options had their problems.

Existing plugins partially or completely wrap the CallKit API in their own high-level API. Because of that flexibility is lost and some features are unavailable. Due to their implementation of architecture and interfaces, such plugins contain their bugs. The documentation is incomplete or absent, and the authors of some of them stopped supporting almost immediately, which is especially dangerous on the fast-growing Flutter.

How we came up with our solution

For simple scenarios, this worked at first, but a specific case gave us some trouble. We had to study the source code to find out how this particular plugin interacted with CallKit. In the end, we discovered that we wouldn't be able to implement what we wanted because of the high-level API limitations.

We thought about implementing our solution with those disadvantages in mind.

We wanted to preserve the architecture and interfaces of CallKit. This way we would give users all the flexibility and the ability to use the original documentation, and we could protect them from potential bugs in our implementation.

Our implementation

We managed to move the entire CallKit API to Dart, preserving the hierarchy of classes and mechanisms of interaction with them.

Our plugin covers all work with the platform and provides an identical interface

The communication between Flutter and iOS is asynchronous, so it took us a while to implement some of the details. The main difficulty was the functionality that required synchronous communication on one side or the other.

For example, the CXProviderDelegate.provider(_:execute:) native CallKit API requires synchronously returning a Bool value:



optional func provider(_ provider: CXProvider, 
    execute transaction: CXTransaction) -> Bool

This method is called every time a new CXTransaction needs to be processed. You can return true to process the transaction yourself and notify the system about it. If you return false (default behavior), the corresponding handler method in CXProviderDelegate is called for each CXAction contained in the transaction.

To use this API in the plugin, we needed to declare it in the Dart code so that the user could control this behavior despite the asynchronous nature of data exchange between the platforms. By returning true in native code, we managed to move transaction control to the Dart code, where we perform manual or automatic CXTransaction processing depending on the value received from the user.

Problems with asynchrony arise in the native part as well. For example, there is the PushKit iOS framework that is not part of CallKit but they are often used together, so it was necessary to integrate it. When you receive a VoIP push, you need to immediately notify CallKit of an incoming call in native code, otherwise the application will crash. To handle this we decided to allow reporting incoming calls directly to CallKit from native code without an asynchronous "hook" in the form of Flutter. As a result, for this integration, we implemented several helpers in the native part of the plugin (available via the FlutterCallkitPlugin iOS class) and several on the Flutter side (available via the FCXPlugin Dart class).

We declared additional features of the plugin in its class to separate the plugin interface from the CallKit interface.

How to report an incoming call directly to CallKit

When a VoIP push is received, one of the PKPushRegistryDelegate.pushRegistry(_: didReceiveIncomingPushWith:) methods is called. Here you need to create a CXProvider instance and call reportNewIncomingCall to notify CallKit of the call. Since the same provider instance is required to further handle the call, we added the FlutterCallkitPlugin.reportNewIncomingCallWithUUID method from the native side of the plugin. When the method is called, the plugin reports the call to the CXProvider and also executes FCXPlugin.didDisplayIncomingCall on the Dart side to continue working with the call.



func pushRegistry(_ registry: PKPushRegistry,
                  didReceiveIncomingPushWith payload: PKPushPayload,
                  for type: PKPushType,
                  completion: @escaping () -> Void
) {
    // Retrieve the necessary data from the push
    guard let uuidString = payload["UUID"] as? String,
        let uuid = UUID(uuidString: uuidString),
        let localizedName = payload["identifier"] as? String
    else {
        return
    }

    let callUpdate = CXCallUpdate()
    callUpdate.localizedCallerName = localizedName

    let configuration = CXProviderConfiguration(
        localizedName: "ExampleLocalizedName"
    )

    // Report the call to the plugin and it will report it to CallKit
    FlutterCallkitPlugin.sharedInstance.reportNewIncomingCall(
        with: uuid,
        callUpdate: callUpdate,
        providerConfiguration: configuration,
        pushProcessingCompletion: completion
    )
}

To sum up: the main feature of our plugin is that using it on Flutter is almost the same as using the native CallKit on iOS.

One more thing

But there still is one thing about Apple CallKit that we haven't implemented (and no one has implemented in available third-party solutions). It's the Call Directory App Extension support.

Call Directory is

CallKit can block and identify calls. Developers can access these features using a special system extension – Call Directory. Read more about iOS app extensions in the App Extension Programming Guide.

Call Directory app extension allows you to block and/or identify numbers

In short, it is a separate iOS app target that runs independently of the main application at the request of the system.

For example, when receiving an incoming call, iOS tries to identify or find the caller in the list of blocked by standard means. If the number is not found, the system can request data from available Call Directory extensions to somehow handle the call. At this point, the extension has to "retrieve" these numbers from storage. The application itself can add numbers from its databases in there at any time. Thus there is no interaction between the extension and the application, the data is exchanged through the shared storage.

An example of how one can implement a Call Directory

Read more about iOS App Extensions: App Extension Programming Guide.

Call Directory Extension in Flutter

Not so long ago a user asked if we can add Call Directory support. We started to study the possibility of implementing this feature and found out that it won’t be able to provide a Flutter API without making the users write native code. The problem is that the Call Directory works in the extension. It is launched by the system, runs for a very short time, and does not depend on the application (including Flutter). Thus, to support this functionality, the user of the plugin will need to create an app extension and data storage on his own eventually.

An example of working with Call Directory in Flutter application

Decision we made

Despite the difficulties with native code, we were determined to make using Call Directory as convenient as possible for our framework's users.

Having tested the ability of such an extension to work with a Flutter app, we started designing it. The solution had to retain all the Call Directory Manager APIs, require the user to write a minimum of native code, and be easy to interact with via Flutter.

This is how we created version 1.2.0 with Call Directory Extension support.

How we implemented Call Directory for Flutter

To implement this functionality, we had to consider several aspects. We needed to:

Transfer the interface of CXCallDirectoryManager class (CallKit object that allows managing Call Directory);
Decide what to do with the app extension and its numbers storage;
Create a convenient way to transfer data from the Dart code to native code and back to manage the list of numbers from the Flutter app.

Transfer CXCallDirectoryManager interfaces to Flutter

The code presented in this article was simplified on purpose to make it easy to perceive. Find the full version of the code following the links at the end of the article. We used Objective-C to implement the plugin since it was chosen as the main language in our project earlier. The CallKit interfaces are written in Swift for simplicity.

Interface

First of all, let's see what exactly needs to be transferred:



extension CXCallDirectoryManager {  
    public enum EnabledStatus : Int {
        case unknown = 0
        case disabled = 1
        case enabled = 2
    }
}

open class CXCallDirectoryManager : NSObject {
    open class var sharedInstance: CXCallDirectoryManager { get }

    open func reloadExtension(
        withIdentifier identifier: String,
        completionHandler completion: ((Error?) -> Void)? = nil
    )

    open func getEnabledStatusForExtension(
        withIdentifier identifier: String,
        completionHandler completion: @escaping (CXCallDirectoryManager.EnabledStatus, Error?) -> Void
    )

    open func openSettings(
        completionHandler completion: ((Error?) -> Void)? = nil
    )
}

Let's recreate the equivalent of the CXCallDirectoryManager.EnabledStatus enum with Dart:



enum FCXCallDirectoryManagerEnabledStatus {
  unknown,
  disabled,
  enabled
}

Now you can declare the class and methods. There is no need for sharedInstance in our interface, so let's make a regular Dart class with static methods:



class FCXCallDirectoryManager {
  static Future<void> reloadExtension(String extensionIdentifier) async { }

  static Future<FCXCallDirectoryManagerEnabledStatus> getEnabledStatus(
    String extensionIdentifier,
  ) async { }

  static Future<void> openSettings() async { }
}

Preserving the API is important, but it is just as important to consider the platform and language code style so that the interface is clear and convenient for plugin users.
For the API in the Dart, we used a shorter name (the long name was from objective-C) and replaced the completion block with Future. Future is the standard mechanism used to get the result of asynchronous methods in Dart. We also return Future from most Dart plugin methods because communication with native code is asynchronous.
Before – getEnabledStatusForExtension(withIdentifier:completionHandler:)
After – Future getEnabledStatus(extensionIdentifier)

Implementation

To make communication between Flutter and iOS possible, we use FlutterMethodChannel.

Read more about the features of this communication channel here.

On the Flutter side

Create a MethodChannel object:



const MethodChannel _methodChannel =
  const MethodChannel('plugins.voximplant.com/flutter_callkit');

On the iOS side

The first thing we do is subscribe the iOS plugin class to the FlutterPlugin protocol to interact with Flutter:



@interface FlutterCallkitPlugin : NSObject<FlutterPlugin>

@end

When initializing the plugin, create a FlutterMethodChannel with the same identifier we used above:



+ (void)registerWithRegistrar:(NSObject<FlutterPluginRegistrar>*)registrar {
    FlutterMethodChannel *channel
        = [FlutterMethodChannel 
          methodChannelWithName:@"plugins.voximplant.com/flutter_callkit"
          binaryMessenger:[registrar messenger]];
    FlutterCallkitPlugin *instance 
        = [FlutterCallkitPlugin sharedPluginWithRegistrar:registrar];
    [registrar addMethodCallDelegate:instance channel:channel];
}

Now you can use this channel to call iOS methods from Flutter.

Let's take a closer look at the implementation of Dart methods and the native part of the plugin using the getEnabledStatus example.

On the Flutter side

The Dart implementation will be as simple as possible. We will call MethodChannel.invokeMethod with the necessary arguments and process the result of that call.

About MethodChannel MethodChannel API allows us to asynchronously get the result of a call from native code using Future but imposes restrictions on the data types that we pass.

We pass the method name (we'll use it in native code to identify the call) and the extensionIdentifier argument to MethodChannel.invokeMethod and then convert the result from the int type to FCXCallDirectoryManagerEnabledStatus. We should also handle PlatformException in case of an error in native code.



static Future<FCXCallDirectoryManagerEnabledStatus> getEnabledStatus(
  String extensionIdentifier,
) async {

  try {

    // Use MethodChannel with extensionIdentifier
    // as an argument to call the corresponding 
    // method in the platform code
    int index = await _methodChannel.invokeMethod(
      'Plugin.getEnabledStatus',
      extensionIdentifier,
    );

    // Convert the result to the
    // FCXCallDirectoryManagerEnabledStatus enum
    // and return its value to the user
    return FCXCallDirectoryManagerEnabledStatus.values[index];

  } on PlatformException catch (e) {

    // If we get an error, we pass it to FCXException
    // and then return it to the user in special type
    throw FCXException(e.code, e.message);
  }
}

Pay attention to the method identifier that we used:
Plugin.getEnabledStatus
The word before the dot is used to define the module responsible for a particular method.
getEnabledStatus is equal to the name of the method in Flutter, not in iOS (or Android).

On the iOS side

Now we move to the platform code and implement the backend for this method.

Calls through FlutterMethodChannel go straight to the handleMethodCall:result: method.

Using the previously passed identifier, we can determine what method was called, get the arguments from it and execute the main part of the code. Detailed information is in the comments here:



- (void)handleMethodCall:(FlutterMethodCall*)call
                  result:(FlutterResult)result {

    // Calls from Flutter can be initiated by name,
    // which is passed to `FlutterMethodCall.method` property
    if ([@"Plugin.getEnabledStatus" isEqualToString:call.method]) {

        // When passing arguments with MethodChannel, 
        // they are packed to `FlutterMethodCall.arguments`.
        // Extract extensionIdentifier, which we passed 
        // from the Flutter code earlier
        NSString *extensionIdentifier = call.arguments;

        if (isNull(extensionIdentifier)) {
            // If the arguments are invalid, return an error
            // using the `result` handler.
            // The error should be packed to `FlutterError`.
            // It’ll be thrown as PlatformException in the Dart code
            result([FlutterError errorInvalidArguments:@"extensionIdentifier must not be null"]);
            return;
    }

        // When the method is detected and the arguments
        // are extracted and validated,
        // we can write the logic

        // To interact with this CallKit functionality
    // we need the CallDirectoryManager instance
        CXCallDirectoryManager *manager 
            = CXCallDirectoryManager.sharedInstance;

        // Call the CallDirectoryManager method
        // and wait for the result
        [manager 
            getEnabledStatusForExtensionWithIdentifier:extensionIdentifier
            completionHandler:^(CXCallDirectoryEnabledStatus status, 
                                           NSError * _Nullable error) {

            // completion handler (containing the result of 
            // the CallDirectoryManager method) is executed, 
            // now we need to pass the result to Dart
            // But first we convert it to the в suitable type, 
            // because only certain data types can be passed 
            // through  MethodChannel
            if (error) {

                // Our errors are packed to `FlutterError`
                result([FlutterError errorFromCallKitError:error]);
            } else {

                // Numbers are packed to `NSNumber`
                // This enum is `NSInteger`, so we
                // make the required conversion
                result([self convertEnableStatusToNumber:enabledStatus]);
            }
    }];
    }
}

Implement the two remaining FCXCallDirectoryManager methods in the same way

On the Flutter side



static Future<void> reloadExtension(String extensionIdentifier) async {
  try {

    // Set an identifier, pass the argument, 
    // and call the platform method
    await _methodChannel.invokeMethod(
      'Plugin.reloadExtension',
      extensionIdentifier,
    );
  } on PlatformException catch (e) {
    throw FCXException(e.code, e.message);
  }
}

static Future<void> openSettings() async {
  try {

    // This method does not accept arguments
    await _methodChannel.invokeMethod(
      'Plugin.openSettings',
    );
  } on PlatformException catch (e) {
    throw FCXException(e.code, e.message);
  }
}

On the iOS side



if ([@"Plugin.reloadExtension" isEqualToString:call.method]) {
    NSString *extensionIdentifier = call.arguments;
    if (isNull(extensionIdentifier)) {
        result([FlutterError errorInvalidArguments:@"extensionIdentifier must not be null"]);
        return;
    }
    CXCallDirectoryManager *manager 
        = CXCallDirectoryManager.sharedInstance;
    [manager 
        reloadExtensionWithIdentifier:extensionIdentifier
        completionHandler:^(NSError * _Nullable error) {
        if (error) {
            result([FlutterError errorFromCallKitError:error]);
        } else {
            result(nil);
        }
    }];
}

if ([@"Plugin.openSettings" isEqualToString:call.method]) {
    if (@available(iOS 13.4, *)) {
        CXCallDirectoryManager *manager 
            = CXCallDirectoryManager.sharedInstance;
        [manager 
            openSettingsWithCompletionHandler:^(NSError * _Nullable error) {
            if (error) {
                result([FlutterError errorFromCallKitError:error]);
            } else {
                result(nil);
            }
        }];
    } else {
        result([FlutterError errorLowiOSVersionWithMinimal:@"13.4"]);
    }
}

That’s it! CallDirectoryManager is implemented and ready to be used.

Learn more about Platform-Flutter interaction

App Extension and number storage

Since, due to the presence of the Call Directory in the iOS extension, we will not be able to provide its implementation with the plugin and working with the platform code is usually unfamiliar for Flutter developers, we will try to help them as much as possible providing ... Documentation!

Let's create a complete app extension and storage samples and connect them to the example app of our plugin.

As the simplest version of the storage, we will use UserDefaults, which we will wrap in propertyWrapper.

This is how the interface of our storage looks like:



// Access to the storage from the iOS app
@UIApplicationMain
final class AppDelegate: FlutterAppDelegate {
    @UserDefault("blockedNumbers", defaultValue: [])
    private var blockedNumbers: [BlockableNumber]

    @UserDefault("identifiedNumbers", defaultValue: [])
    private var identifiedNumbers: [IdentifiableNumber]
}

// Access to the storage from the app extension
final class CallDirectoryHandler: CXCallDirectoryProvider {
    @UserDefault("blockedNumbers", defaultValue: [])
    private var blockedNumbers: [BlockableNumber]

    @UserDefault("identifiedNumbers", defaultValue: [])
    private var identifiedNumbers: [IdentifiableNumber]

    @NullableUserDefault("lastUpdate")
    private var lastUpdate: Date?
}

Storage implementation code: UserDefaults

iOS app code: iOS App Delegate

iOS extension code: iOS App Extension

Note that the storage and extension samples are not part of the plugin, but rather part of the example application that comes with it.

Pass numbers from Flutter to iOS and vice versa

So, the app extension is configured and connected to the storage, the necessary methods of CallDirectoryManager are implemented, the last detail – to learn how to take numbers from Flutter and put them in the platform storage or, conversely, request them from the platform storage.

The easiest way as it may seem is to make the user of the plugin deal with the data. He'll have to set up his own MethodChannel or use other third-party storage management solutions. It will certainly suit some people! :) And for the rest, we will make a simple API to pass numbers directly through our framework. This functionality will be optional so we don’t limit those who are more comfortable using their ways of transferring data.

Interface

Let’s see which interfaces we need:

Add blocked/identifiable numbers to the storage
Delete blocked/identifiable numbers from the repository
Request blocked/identifiable numbers from the repository

On the Flutter side

We previously decided to use the FCXPlugin (Flutter) and FlutterCallkitPlugin (iOS) classes for the helpers. However, Call Directory is a highly specialized functionality that is not used in every project. That's why I want to put it in a separate file but leave the access through the FCXPlugin class object. The extension will do this work:



extension FCXPlugin_CallDirectoryExtension on FCXPlugin {

  Future<List<FCXCallDirectoryPhoneNumber>> getBlockedPhoneNumbers()
    async { }

  Future<void> addBlockedPhoneNumbers(
    List<FCXCallDirectoryPhoneNumber> numbers,
  ) async { }

  Future<void> removeBlockedPhoneNumbers(
    List<FCXCallDirectoryPhoneNumber> numbers,
  ) async { }

  Future<void> removeAllBlockedPhoneNumbers() async { }

  Future<List<FCXIdentifiablePhoneNumber>> getIdentifiablePhoneNumbers()
    async { }

  Future<void> addIdentifiablePhoneNumbers(
    List<FCXIdentifiablePhoneNumber> numbers,
  ) async { }

  Future<void> removeIdentifiablePhoneNumbers(
    List<FCXCallDirectoryPhoneNumber> numbers,
  ) async { }

  Future<void> removeAllIdentifiablePhoneNumbers() async { }
}

On the iOS side

To let Flutter access numbers that are in storage on the iOS side, the user of the plugin needs to somehow connect his database of numbers and the plugin. Let’s give him the interface to do that:



@interface FlutterCallkitPlugin : NSObject<FlutterPlugin>

@property(strong, nonatomic, nullable)
NSArray<FCXCallDirectoryPhoneNumber *> *(^getBlockedPhoneNumbers)(void);

@property(strong, nonatomic, nullable)
void(^didAddBlockedPhoneNumbers)(NSArray<FCXCallDirectoryPhoneNumber *> *numbers);

@property(strong, nonatomic, nullable)
void(^didRemoveBlockedPhoneNumbers)(NSArray<FCXCallDirectoryPhoneNumber *> *numbers);

@property(strong, nonatomic, nullable)
void(^didRemoveAllBlockedPhoneNumbers)(void);

@property(strong, nonatomic, nullable)
NSArray<FCXIdentifiablePhoneNumber *> *(^getIdentifiablePhoneNumbers)(void);

@property(strong, nonatomic, nullable)
void(^didAddIdentifiablePhoneNumbers)(NSArray<FCXIdentifiablePhoneNumber *> *numbers);

@property(strong, nonatomic, nullable)
void(^didRemoveIdentifiablePhoneNumbers)(NSArray<FCXCallDirectoryPhoneNumber *> *numbers);

@property(strong, nonatomic, nullable)
void(^didRemoveAllIdentifiablePhoneNumbers)(void);

@end

Each type of interaction with the repository has its handler. It is called by our framework every time the corresponding helper is called from the Flutter side.

Handlers are optional which allows you to use only some part of this functionality or use your own solution instead.

Implementation

Now let's implement the communication between the declared helper methods in Flutter and the handlers in iOS.

There are a lot of methods but they all work almost the same. That’s why we will focus on two of them, the ones with the opposite direction of data movement.

Get identifiable numbers

On the Flutter side



Future<List<FCXIdentifiablePhoneNumber>> getIdentifiablePhoneNumbers() async {
  try {

    // Call the platform method and save the result    List<dynamic> numbers = await _methodChannel.invokeMethod(
      'Plugin.getIdentifiablePhoneNumbers',
    );

    // Type the result and return it to the user
    return numbers
      .map(
        (f) => FCXIdentifiablePhoneNumber(f['number'], label: f['label']))
      .toList();

  } on PlatformException catch (e) {
    throw FCXException(e.code, e.message);
  }
}

On the iOS side



if ([@"Plugin.getIdentifiablePhoneNumbers" isEqualToString:call.method]) {
    if (!self.getIdentifiablePhoneNumbers) {
        // Check if the handler exists,
        // if not, return an error
        result([FlutterError errorHandlerIsNotRegistered:@"getIdentifiablePhoneNumbers"]);
        return;
    }

    // Using the handler, request numbers from a user
    NSArray<FCXIdentifiablePhoneNumber *> *identifiableNumbers
        = self.getIdentifiablePhoneNumbers();

    NSMutableArray<NSDictionary *> *phoneNumbers
        = [NSMutableArray arrayWithCapacity:identifiableNumbers.count];

    // Wrap each number in the dictionary type 
    // so we could pass them via MethodChannel 
    for (FCXIdentifiablePhoneNumber *identifiableNumber in identifiableNumbers) {
        NSMutableDictionary *dictionary 
            = [NSMutableDictionary dictionary];
        dictionary[@"number"] 
            = [NSNumber numberWithLongLong:identifiableNumber.number];
        dictionary[@"label"] 
            = identifiableNumber.label;
        [phoneNumbers addObject:dictionary];
    }

    // Pass the numbers to Flutter
    result(phoneNumbers);
}

Add identifiable numbers

On the Flutter side



Future<void> addIdentifiablePhoneNumbers(
  List<FCXIdentifiablePhoneNumber> numbers,
) async {
  try {
    // Prepare the numbers to be passed via MethodChannel
    List<Map> arguments = numbers.map((f) => f._toMap()).toList();

    // Pass the numbers to native code
    await _methodChannel.invokeMethod(
      'Plugin.addIdentifiablePhoneNumbers',
      arguments
    );

  } on PlatformException catch (e) {
    throw FCXException(e.code, e.message);
  }
}

On the iOS side



if ([@"Plugin.addIdentifiablePhoneNumbers" isEqualToString:call.method]) {
    if (!self.didAddIdentifiablePhoneNumbers) {
        // Check if the handler exists,
        // if not, return an error
        result([FlutterError errorHandlerIsNotRegistered:@"didAddIdentifiablePhoneNumbers"]);
        return;
    }

    // Get the numbers passed as arguments
    NSArray<NSDictionary *> *numbers = call.arguments;
    if (isNull(numbers)) {
        // Check if they’re valid
        result([FlutterError errorInvalidArguments:@"numbers must not be null"]);
        return;
    }

    NSMutableArray<FCXIdentifiablePhoneNumber *> *identifiableNumbers
        = [NSMutableArray array];

    // Type the numbers
    for (NSDictionary *obj in numbers) {
        NSNumber *number = obj[@"number"];
        __auto_type identifiableNumber
            = [[FCXIdentifiablePhoneNumber alloc] initWithNumber:number.longLongValue
                                                                                     label:obj[@"label"]];
        [identifiableNumbers addObject:identifiableNumber];
    }

    // Pass the typed numbers to the handler
    self.didAddIdentifiablePhoneNumbers(identifiableNumbers);

    // Tell Flutter about the end of operation
    result(nil);
}

The rest of the methods are implemented the same way, here is the full code for Flutter and iOS:

Usage samples

Now we move to the user side of the plugin and learn how our users can utilize the interfaces.

Reload extension

The reloadExtension(withIdentifier:completionHandler:) method is used to reload the Call Directory extension. You may need it, for example, after adding new numbers to the storage so that they get into CallKit.

Use it the same way you use native CallKit API: call FCXCallDirectoryManager and request reload by the given extensionIdentifier:



final String _extensionID =
  'com.voximplant.flutterCallkit.example.CallDirectoryExtension';

Future<void> reloadExtension() async {
  await FCXCallDirectoryManager.reloadExtension(_extensionID);
}

Get identified numbers

On the Flutter side

We request a list of identifiable numbers using our plugin class:



final FCXPlugin _plugin = FCXPlugin();

Future<List<FCXIdentifiablePhoneNumber>> getIdentifiedNumbers() async {
  return await _plugin.getIdentifiablePhoneNumbers();
}

On the iOS side

Add the getIdentifiablePhoneNumbers handler, which the plugin uses to pass the specified numbers to Flutter. We will use it to pass the numbers from our identifiedNumbers storage:



private let callKitPlugin = FlutterCallkitPlugin.sharedInstance

@UserDefault("identifiedNumbers", defaultValue: [])
private var identifiedNumbers: [IdentifiableNumber]

// Add a phone number request event handler
callKitPlugin.getIdentifiablePhoneNumbers = { [weak self] in
    guard let self = self else { return [] }

    // Return the numbers from the storage to the handler
    return self.identifiedNumbers.map {
        FCXIdentifiablePhoneNumber(number: $0.number, label: $0.label)
    }
}

Now the numbers from the user storage will go to the handler and then to Flutter.

Add identified numbers

On the Flutter side

We pass the numbers that we want to identify to the plugin object:



final FCXPlugin _plugin = FCXPlugin();

Future<void> addIdentifiedNumber(String number, String id) async {
  int num = int.parse(number);
  var phone = FCXIdentifiablePhoneNumber(num, label: id);
  await _plugin.addIdentifiablePhoneNumbers([phone]);
}

On the iOS side

Add the didAddIdentifiablePhoneNumbers handler, which the plugin uses to notify the platform code about receiving new numbers from Flutter. In the handler, we save the received numbers to the number storage:



private let callKitPlugin = FlutterCallkitPlugin.sharedInstance

@UserDefault("identifiedNumbers", defaultValue: [])
private var identifiedNumbers: [IdentifiableNumber]

// Add an event handler for adding numbers
callKitPlugin.didAddIdentifiablePhoneNumbers = { [weak self] numbers in
    guard let self = self else { return }

    // Save the numbers to the storage 
    self.identifiedNumbers.append(
        contentsOf: numbers.map {
            IdentifiableNumber(identifiableNumber: $0)
        }
    )

    // The numbers in Call Directory must be sorted
    self.identifiedNumbers.sort()
}

Now numbers from Flutter go to the plugin, then to the event handler, and eventually to the user's number storage. Next time you reload the Call Directory extension, they will be available for CallKit for call identification.

Full code samples:

To sum it up

We managed to make an opportunity to use the CallKit Call Directory from Flutter!

The details of platform communications are still hidden in the depths of the plugin, the native API is preserved, and the custom iOS implementation is well documented.

Now it is relatively easy to block and/or identify numbers using the native Call Directory in Flutter.

An example of working with Call Directory in Flutter application using flutter_callkit_voximplant

Results:

Completely moved CallDirectoryManager interface
Created an easy way to pass the numbers from the Flutter code to iOS, leaving the option to use your data transfer solutions
Described the architecture of the solution in the README with visual diagrams for better understanding
Added a full-fledged working example app that uses all Call Directory functionality and implements platform modules samples (iOS extension and data storage)

Useful links

Optimizing graphics when working with Joint JS

Irina Maximova — Tue, 13 Apr 2021 12:44:50 +0000

What do I do if my Voximplant Kit scenario is pretty neat, but has so many elements it becomes slow? That is the question our developers asked themselves and answered with an idea to achieve optimization.

Continuing the series of articles about Kit and Joint JS, I’m going to tell you how optimization made large scenarios super fast and what problems our dev team encountered while creating this feature.

Why optimize

Many of us are familiar with the performance issue caused by a large number of elements on the page. What does it mean? In our case, the more elements in the Voximplant Kit scenario, the more they affect the rendering speed of the elements on the canvas when we drag them across it, as well as the speed of scaling and rendering the canvas itself.

We wanted to use the will-change CSS property that allows informing the browser about changes to be applied to a certain element. It means one can set optimization up beforehand to avoid launching the operations that negatively affect the responsiveness of the page.

However, in the JointJS library that we use in Voximplant Kit, transformations related to dragging and scaling the canvas are applied to the child group of the SVG element, not to the entire canvas. We didn’t manage to shift the calculations to the video card; browsers just ignored this property and redrew all the group elements with each movement, which caused a delay.

<svg ... > <!-- Canvas -->
  <g transform="matrix(1,0,0,1,224,444)"> <!-- Group of elements inside svg -->
    <rect>
    <rect>

Implementation

Our developers decided that if they wrap the SVG element in a div element they would be able to apply all transformations first to the div element, and only if necessary, to the SVG element with the canvas. Once the transformations started to apply to the div, the time came to use will-change:transform to keep track of them:

<div> <!-- div-wrapper to which we apply optimization and transformation -->
  <svg> <!-- Canvas -->
    <g> <!-- Group of elements inside svg -->
      <rect>
      <rect>

But here we face another challenge – the use of will-change initiates the creation of an additional layer. And the greater the width and height of the element to which we apply this property, the more RAM we need to store this layer. We solved this problem by reducing the scale of SVG by 10 times. For example, when the scale of the canvas is 200%, the will-change layer requires 300 megabytes of RAM. After scaling down, about 3 megabytes are required.

To achieve that, we set zoom = 0.1 and shrink the SVG canvas using the transformToCenterViewport method. Then we apply the same transformations to the div element:

if (isPerfMode) {
  this.el.classList.add('perf-mode');
  // Change scaling before enabling the performance mode
  const prevScale = this._viewportMatrix.a;
  const point = this.getViewPortCenter();
  const zoom = 0.1;
  // Shrink the original svg so that will-change uses less RAM
  this.transformToCenterViewport(point, zoom, true, false, true);
  this.initScale = this._viewportMatrix.a;
  this.createMatrix();
  this.isPerfMode = true;
  // Apply transformations to the wrapper-element
  this.startPerformance();
  this.transformToCenterViewport(point, prevScale, false, false, true);
}

When enabling optimization mode, we shrink the SVG and the canvas gets really small and hard to work with. To fix it we'll apply inverse scaling directly to the div element:

public startPerformance(force = false) {
  ...
  this.isPerformance = true;
  // Get the size of the area with blocks and the indent from the left corner of the viewport
  const { x, y, width, height } = this.layers.getBBox();
  const initScale = this.initScale;
  // Width and height for the wrapper and the x- and y-axis offsets for the area with blocks
  const wrapW = Math.floor(width * initScale) + 2;
  const wrapH = Math.floor(height * initScale) + 2;
  const layerX = -x * initScale;
  const layerY = -y * initScale;
  // this.wrapMatrix - div-element matrix containing the canvas
  this.wrapMatrix.e = +(this._viewportMatrix.e + x * this._viewportMatrix.a);
  this.wrapMatrix.f = +(this._viewportMatrix.f + y * this._viewportMatrix.d);
  this.svgWrapper.style.width = wrapW + 'px';
  this.svgWrapper.style.height = wrapH + 'px';
  this.svgWrapper.style.transform = this.wrapMatrix.toString();
  this.svgWrapper.style.willChange = 'transform';
  this.layers.style.transform = `matrix(${initScale},0,0,${initScale},${layerX} ,${layerY} )`;
}

We solved the speed problem, but it didn’t end there: when scaling down the canvas, the image details started to disappear. So when scaling up, it became blurry. We found the solution with the help of the article about re-rastering composited layers on scale change.

After scaling is stopped (the scroll event), we remove the will-change property for 0.1 seconds and then set it again. This causes the browser to re-raster the layer and return the missing image details:

// Add a 3d transformation so that the layer is not deleted
this.svgWrapper.style.transform = this.wrapMatrix.toString() + ' translateZ(0)';
this.transformFrameId = requestAnimationFrame(() => {
  // Set the will-change property to apply in the next frame
  this.svgWrapper.style.willChange = '';
  this.transformFrameId = requestAnimationFrame(() => {
    this.svgWrapper.style.willChange = 'transform';
    this.svgWrapper.style.transform = this.wrapMatrix.toString();
  });
});

The last thing we should do is always display the moved block on top of other blocks. In JointJS, there are toFront and toBack methods (analog of z-index in HTML) to move blocks and links along the Z-axis. But they have to sort elements and redraw blocks and links, thus causing delays.

Our developers came up with the following: the block we interact with is temporarily put at the end of the SVG element tree (the element with the highest z-index is at the end) on the mousedown event, and then returns to its previous position on the mouseup event.

How it works

One can test optimization mode in all Chromium-based browsers (Chrome, Opera, Edge, Yandex Browser, etc.) and the Safari browser. If scenarios contain 50 blocks or more, the optimization mode is enabled automatically. You can enable or disable it yourself by switching to the editor settings in the upper right corner:

Once you enable or disable the optimization mode, you’ll see a notification at the top of the scenario window:

Check the GIFs below to see the difference between scenarios with the optimization mode on and off. But since it's always more interesting to try it yourself, feel free to go to your Voximplant Kit scenario or, if you don't have an account yet – to the registration page.

Working with the canvas and its elements when the optimization is off looks more or less like this (computers with different characteristics may show different results):

We enable optimization and voila!

This is how we made the canvas moving and scaling smoother and faster, as well as increased the rendering speed of dragging and dropping blocks with links.

I hope you liked this article! We will continue to improve the product, so be ready for me sharing more updates and tricks with you! :)

“Live” interactive logs: call logs visualization in Voximplant Kit

Irina Maximova — Fri, 10 Jul 2020 14:35:22 +0000

Not so long ago, I posted my first article about updates to Voximplant Kit, a smart and flexible tool for customer experience automation. Our team continues to improve the product using JointJS. And today I’m happy to tell you how our newly created live call logging feature works. What does “live logs” mean? Are they somehow dangerous:)? Let’s find out!

Previously, Voximplant Kit users could only listen to call recordings to analyze the call results. We wanted to create something else to make this process plain sailing. Since we are dealing with a low-code/no-code product, text logs are not the solution we're looking for. So we decided to visualize call logs.

How it works

All call results are now available as a sequence of blocks used in the call. The visual editor animates and highlights these blocks just like it does in the demo mode. The only difference is that the path the client took is highlighted in advance.

To see a particular call log, go to the outbound or inbound call history tab or to the report of a particular campaign and click "View log info". Then click “Start scenario”, and the editor will show you what was happening in the call step by step.

Playback control

Start/stop controls (1) are to pause and resume playback. Rewind/fast forward (2) – to move you to the beginning of the previous or next block. You can also click on the timeline to start playback from a specific point in time, as is typical in most media players.

If your scenario includes an audio recording, it will be played along with moving from block to block. The audio recording is colored differently on the timeline (3).

A list of passed blocks with their timestamps is also available:

Spoiler:
Here, in the “Log” tab, we plan to show the details of each block. They will help us understand why a particular port was chosen and whether there were errors. For example, for a speech recognition block, we will see recognition results and errors.
Complex blocks, such as DialogFlowConnector, IVR, ASR, etc., will be of greatest interest here.

Variables

Changed variables are displayed on the left as pop-up notifications in chronological order. That is, if we move to the “Data modification” block, the variables that have been changed there will immediately pop up. If we move far from it (more than 4s on the timeline) – the variables will disappear until we are again in the interval where the change has occurred:

Lifehack

Live call logs remain their original form even after the scenario is changed or deleted. This means that you won't have any problems restoring the scenario from its log if you need it.

You can play around with the logs yourself after creating a Voximplant Kit account.

What’s inside?

Let's see how live logs are implemented in the code. Let's say right away, JointJS helped us with animating and highlighting only. The rest is our creativity.

Getting timepoints

When opening a specific call log, the server gives us a list of all used blocks with their timestamps (date and time when the client entered each block) and a list of variables changed during the call. In other words, there are two arrays of objects we will use: log_path and log_variables.

Also, the server response has the link to the audio and its duration, if the call was recorded.

Knowing the timestamps, we can calculate the timepoints (in milliseconds) for all used blocks and changed variables. The start point (0 ms) is the time we enter the first block. If we entered the second block in 5 seconds after the call started, then the timepoint of the second block is 5000 ms. Using these timepoints, we calculate the full log duration.

Looking for matches

After pressing the play button, the timeline starts updating every 10 ms. At each update, we check if the current time coincides with one of the timepoints:

const found = this.timePoints.find((item) => item === this.playTime);

If there is a match, we will look for all the blocks with the timepoint = current time + 600 ms (the time required to animate the transition from one block to another).

Here’s the full code of the updatePlayTime() method:

updatePlayTime(): void {
    const interval = 10;
    let expected = Date.now() + interval;

    const tick = () => {
        const drift = Date.now() - expected;
        const found = this.timePoints.find((item) => item === this.playTime);
        this.$emit('update', {
            time: this.playTime,
            found: found !== undefined
        });

        if (this.playTime >= this.duration) {
            this.isPlay = false;
            this.playTime = this.duration;
            clearTimeout(this.playInterval);
            this.$emit('end', this.playTime);
            return;
        }

        expected += interval;

        this.playTime += 0.01;
        this.playTime = +this.playTime.toFixed(2);

        this.updateProgress();

        this.playInterval = window.setTimeout(tick, Math.max(0, interval - drift));
    };

    this.playInterval = window.setTimeout(tick, 10);
}

Also, every 90 ms we check the matches for the current time and timepoints of the changed variables + 4000 ms (the time during which the variable change notification is shown).

Selecting blocks

After all the matches have been found, we add blocks to the queue to highlight them and animate the links between them.

If there are several blocks with timepoint = current time + 600 ms, we animate the transition directly to the last block:

if (i === blocks.length - 1) {
    await this.selectBlock(blocks[i], 600, true, true);

It's needed because there are blocks that are processed very quickly. For example, “Data Verification”, “Data Modification”, etc. – several blocks can be passed at a time. We still need to show them but do not want to slow down the animation too much, so we animate the transition straight to the last block.

Check the method code below:

async onUpdateTimeline({
    time,
    found
}) {
    this.logTimer = time * 1000; // convert to ms
    this.checkHistoryNotify();

    if (!found) return;

    // Select the group of blocks from the first found point + 600ms
    const blocks = this.callHistory.log_path.filter((item) => {
        return item.timepoint >= this.logTimer && item.timepoint < this.logTimer + 600;
    });

    if (blocks.length) {
        this.editor.unselectAll();

        for (let i = 0; i < blocks.length; i++) {
            if (i === blocks.length - 1) {
                await this.selectBlock(blocks[i], 600, true, true);

                const cell = this.editor.getCellById(blocks[i].idTarget);
                this.editor.select(cell);
            } else {
                await this.selectBlock(blocks[i], 0, false, true);
            }
        }
    }
}

We do it over and over again: if there is a match, we highlight the block; if the block is already in the queue, we do nothing:

async selectBlock(voxHistory, timeout = 700, animate = true, animateLink = true) {
    const inQueue = this.selectQueue.find((item) => item[0].targetId === voxHistory.idTarget);

    if (!inQueue) this.selectQueue.push(arguments);

    return this.exeQueue();
}

Rewinding

Rewinding works the same way: when the user clicks on the timeline, we get the time to rewind to and mark as not selected all blocks with the timepoint greater than the current time:

const forSelect = this.callHistory.log_path.filter((item) => {
    const time = accurate ? item.accurateTime : item.timepoint;
    return time <= this.logTimer;
});

After that, we animate the transition to the last block.

The full code of the onRewind() method:

async onRewind({
    time,
    accurate
}: RewindData, animation = true) {
    this.editor.unselectAll();
    this.stopLinksAnimation();
    this.checkHistoryNotify(true);

    const forSelect = this.callHistory.log_path.filter((item) => {
        const time = accurate ? item.accurateTime : item.timepoint;
        return time <= this.logTimer;
    });

    for (let i = 0; i < forSelect.length; i++) {
        if (i === forSelect.length - 1) {
            await this.selectBlock(forSelect[i], 600, animation, false);
            const cell = this.editor.getCellById(forSelect[i].idTarget);
            this.editor.select(cell);
        } else {
            await this.selectBlock(forSelect[i], 0, false, false);
        }
    }

    if (this.isPlay) this.restartAnimateLink();

    this.onEndTimeline();
}

Playing audio

Turning audio on/off is even easier. If the current time coincides with the start time of the recording, the audio starts playing. The updatePlayer() method is responsible for this:

updatePlayer() {
    if (this.playTime >= this.recordStart && this.player.paused && !this.isEndAudio) {
        this.player.play();
        this.player.currentTime = this.playTime - this.recordStart;
    } else if (this.playTime < this.recordStart && !this.player.paused) {
        this.player.pause();
    }
}

That’s it! JointJS methods + creativity of our developers. This is how live logs have been implemented. Test them out yourself if you haven't done it yet :)

It's great if you like this series of articles about Voximplant Kit insides. The most interesting and challenging is yet to come. Stay tuned!

How we used JointJS in Voximplant Kit

Irina Maximova — Mon, 20 Apr 2020 14:00:19 +0000

Hey, community! My name is Irina, I'm a technical writer at Voximplant. It's exciting to have an ability to share something intriguing with you guys, so let's get straight to the point – as my job includes lots and lots of "how it works" investigation, I'd like to tell you about nice tweaks that our frontend developers applied to a popular JS library, JointJS. But before we go deeper to the JavaScript jungle, let me give you a tiny onboarding to our product in which we implemented the aforementioned tweaks. I'm talking about Voximplant Kit that has recently undergone a major update. So, let's begin, shall we?

Voximplant Kit is a smart and flexible tool for customer experience automation, formerly known as Smartcalls. With Voximplant Kit you can build smart IVRs, launch automated call campaigns and make trigger outbound calls with our visual flow builder to improve customer experience in no time.

And now we’re ready to share what we changed in our interface and how we tamed JointJS, a JavaScript beast that helped us a lot in updating our visual editor.

What has actually changed?

Although the concept remains the same, this update brings a new user interface crafted by our UX experts, along with new tools and features for even smarter call processing automation.

New design of the registration page, the top menu position change, an option of grouping and sorting scenarios and audio recordings; campaign cards showing the average duration of a successful call and the total amount of money spent.

Voximplant visual editor

Demo mode (spoiler: this is the main killer feature we want to tell you about).

Аlong with real-time scenario execution, the demo mode highlights the blocks used in the current call and shows the result of the call (Flow and Log), which makes the debugging process easier and faster.

Here you can watch the video of how the demo mode works or test it yourself after creating a Voximplant Kit account.

We put our cards on the table

It's time to figure out how the block animation is implemented in the code.

The editor calls the StartScenarios HTTP API method to run a cloud scenario. The Voximplant cloud starts the scenario and gives the media_access_url to the editor. From this moment on, the editor calls the media_access_url every second, receiving information on the blocks used by the scenario. Based on this data, the editor highlights the necessary blocks and animates the lines that connect them.

And here we’ll need the movement history stored in the HISTORY object – the JSON object with the following fields:

timestamp;
idSource - initial block;
idTarget - final block;
port (there may be several outputs from 1 block).

These custom and service values help the front-end understand from which block to which the scenario moves. How exactly? When a new block is added, it immediately gets an ID, which is then used in HISTORY as idSource / idTarget.

To implement this functionality, we used JointJS and some self-written code.

Let's begin with the main method – selectBlock. It works as follows: we go through the array of movements history (idSource, idTarget) and as soon as we find the start and end points, we search for a connector between them:

const link = this.editor.getTestLink(sourceCell, portId);

If there is a connector, we animate the movement from block to block (like on the GIF at the beginning of the article):

if (link) this.setLinkAnimation(link);

The selectBlock method is called every time this.testHistory is updated. Since several blocks can be passed to this.testHistory at the same time, we recursively call selectBlock every 700 ms (this is the approximate time spent on animating the movement from block to block):

setTimeout(this.selectBlock, 700);

The full code of this method is given below. Pay attention to the methods selectTestBlock and getTestLink, lines 7 and 10 – in a moment, we will talk about each of them:

selectBlock():void {
     if (this.historyIndex < this.testHistory.length) {
       const i = this.historyIndex;
       const targetCellId = this.testHistory[i].idTarget;
       const sourceCellId = this.testHistory[i].idSource;
       const portId = this.testHistory[i].port;
       const targetCell = this.editor.selectTestBlock(targetCellId);
       const sourceCell = this.editor.getCell(sourceCellId);
       if (sourceCell && targetCell) {
         const link = this.editor.getTestLink(sourceCell, portId);
         if (link) this.setLinkAnimation(link);
       }
       this.historyIndex += 1;
       setTimeout(this.selectBlock, 700);
     }
   }

Drawing a line connector

The getTestLink method helps us get a connector between blocks. It is based on getConnectedLinks, a built-in JointJS method that receives a block and returns an array of its connectors. In this array, we look for a connector to the port with portId as the value of the source property:

link = this.graph.getConnectedLinks(cell, {outbound : true}).find(item => {
     return item.get('source').port === portId;

Then, if there is such a connector, we highlight it:

return link ? (link.toFront() && link) : null;

The full code of the method:

getTestLink(sourceCell: Cell, portId: string): Link {
  let link = null;
  if (sourceCell && sourceCell.id) {
    let cell = null;
    if (sourceCell.type === 'ScenarioStart' || sourceCell.type === 'IncomingStart') {
      cell = this.getStartCell()
    } else {
      cell = this.graph.getCell(sourceCell.id);
    }
    link = this.graph.getConnectedLinks(cell, {outbound : true}).find(item => {
      return item.get('source').port === portId;
    });
  }
  return link ? (link.toFront() && link) : null;
}

The movement from block to block is animated completely by means of JointJS (check the demo).

Moving to the current block

We call the selectTestBlock method when it is necessary to select the final block and move the canvas to it. Here we get the coordinates of the block center:

const center = cell.getBBox().center();

Then, we call the setTestCell method to color the block:

editor.tester.setTestCell(cell);

Finally, we zoom to its center using the self-written zoomToCell function (we’ll talk about it at the end of the article):

editor.paperController.zoomToCell(center, 1, false);

The full code:

selectTestBlock(id: string): Cell {
 const cell = (id === 'ScenarioStart') ? editor.tester.getStartCell() : editor.graph.getCell(id);
 if (cell) {
   const center = cell.getBBox().center();
   editor.tester.setTestCell(cell);
   editor.paperController.zoomToCell(center, 1, false);
 }
 return cell;
}

The method for coloring: it finds the SVG element of our block and adds the .is-tested CSS class to color it:

setTestCell(cell: Cell): void {
 const view = cell.findView(this.paper);
 if (view) view.el.classList.add('is-tested');
}

Smooth zoom

And finally, zoomToCell! JointJS has a built-in method for moving the canvas along the X and Y axes. At first, we wanted to work with it, however, this method uses transform as an attribute of the SVG tag. It does not support smooth animation in the Firefox browser and utilizes the CPU only.

We used a small hack – created our own zoomToCell function, which, in essence, does the same thing but transform here is an inline CSS. This way, we enable GPU rendering because WebGL is involved in the process. Thus, the problem of cross-browser compatibility has been solved.

Our function not only moves the canvas along X and Y but also allows us to zoom simultaneously by means of the transform matrix.

The will-change property of the .animate-viewport class informs the browser that the element will be changed and optimization must be applied, including the use of the GPU. And the transition property sets the smoothness of moving the canvas to the block:

.animate-viewport {
 will-change: transform;
 transition: transform 0.5s ease-in-out;

Check the full method code below:

public zoomToCell(center: g.Point, zoom: number, offset: boolean = true): void {
   this.updateGridSize();
   const currentMatrix = this.paper.layers.getAttribute('transform');
   // Get a new SVG matrix to move the canvas to a point from the center argument
    // and destructure it to set the style attribute
   const { a, b, c, d, e, f } = this.zoomMatrix(zoom, center, offset);
// For FireFox you need to set the original matrix, otherwise there is an abrupt motion of the canvas
   this.paper.layers.style.transform = currentMatrix;
   // Without the first timeout, FF skips the fact that we set the original matrix, and an abrupt motion occurs again
   setTimeout(() => {
// Add a CSS selector .animate-viewport, which has its own transition;
// Set the new matrix in the style attribute and calculate the duration of the transition
     this.paper.layers.classList.add('animate-viewport');
     this.paper.layers.style.transform = `matrix(${ a }, ${ b }, ${ c }, ${ d }, ${ e }, ${ f })`;
     const duration = parseFloat(getComputedStyle(this.paper.layers)['transitionDuration']) * 1000;
     // After the animation is completed, remove the selector and style attribute;
      // set the matrix for the canvas using JointJS
     setTimeout(() => {
       this.paper.layers.classList.remove('animate-viewport');
       this.paper.layers.style.transform = null;
       this.paper.matrix(newMatrix);
       this.paper.trigger('paper:zoom');
       this.updateGridSize();
       this.paper.trigger('paper:update');
     }, duration);
   }, 100);
 }

As it turned out, sometimes even the most advanced libraries have to be modified if your needs require that. We hope you enjoyed taking a deep dive into the library insides. We wish you successful development in general and with the Voximplant Kit in particular! Find other interesting stories on Voximplant and Voximplant Kit blogs.

P.S.

Like the article and want to know more about our frontend-related challenges? For example, how we implemented undo/redo (1), proper shortcuts processing (2), pop-up menu with the minimap, zooming, sharing, and other tools (3), and so on. Leave your comment below, let us know what would grab your attention (if any) :) Thanks for reading!