DEV Community: Jonathan Wheat

Impress your boss by giving your Rasa-bot a voice with Twilio for $0

Jonathan Wheat — Wed, 14 Jul 2021 19:59:56 +0000

Building and interacting with a chat-bot can be fun, but calling a number and actually talking to it for the first time over the phone is next level. It also gives others the ability to interact with your bot in a new way. And talk about CDD! (Conversation Driven Development. All hell breaks loose when people talk to it. It's a whole different animal.

This tutorial will walk you through setting up your Rasa-bot with Twilio, an online service that provides a host of products, one of which is called Programmable Voice.

Twilio is a paid service, however they offer a free trial (YEA!). They do this in a pretty cool way. They allocate a credit ($15 US) to your account and the trial lasts as long as you have money in your account. You can spend it on anything they offer, and obviously the more you do, the faster it goes.

I've setup voice (the main topic of this tutorial) as well as messaging (texting with my bot) and we'll hit on that at the end, it's pretty easy once you have the voice interface configured.

The best part is that you can run your bot from your own machine! You don't have to deploy it to a server or jump through any crazy hoops. This is especially good if you happen to have proprietary data that the bot utters, or collect or divulge any (PII)[https://www.dhs.gov/privacy-training/what-personally-identifiable-information].

Requirements

Here's a quick run down of the various tools we'll need, just in case you're the kind of person that has to know going in.

A phone 😆 or at least some way to make a voice call to a phone number
Rasa ^2.6.0
A bot to hook up (the out of the box rasa init will be fine)
Twilio Trial Account
NGrok

Getting Started (Twilio)

The first thing you need to do is get yourself a trial Twilio account. It's a pretty simple process, albeit there are a few steps.

Sign up
Verify your email address
Log in

Once you log in, you'll need to verify your phone number.

Next pick the following -

Which Twilio product are you here to use? - Voice
What do you plan to build with Twilio? - IVR & Bots
How do you plan to build with Twilio? - With minimal code
What is your preferred coding language? - Python
Would you like Twilio to host your code? - Yes, host my code

Finally, you'll be delivered to the console. Depending on when you're reading this, the current console may be either the "Legacy Console" or the new "Beta Console".

I want you on the Beta console, which, like I said could be the default one now. You'll know because there will be a button
in the header that reads "Try the beta Console".

Click that.
Complete the little form.

You should land on your Dashboard, and right there dead center is a blue button that says "Get a Trial Number"

Click that.

When you click it, it'll hand you a phone number. There's a big red button that reads "Choose this Number"

Click that.

You'll get a Congratulations modal with a Done button.

Click that. 😄

You'll land back on your dashboard, and you'll see your trial balance ($15.50) as well as your new trial phone number. If you call it, it's legit, it'll connect to Twilio, although not much happens after the initial message.

Also on your dashboard are 2 very important items we'll need shortly for Rasa. The account SIT and the Auth Token. Keep your screen there, and head over to your bot.

Configure your bot for Twilio Voice

The next thing you need to do is prep your bot for Twilio. I'm presuming you have a bot already setup. If you don't, create a new directory and run rasa init to spin one up.

Rasa has something called connectors. These connectors can be used to hook to different channels such as Slack, Messenger, Telegram, etc. We, as developers, can build custom connectors too, which is pretty great, they're crazy powerful. Fun fact, I actually started this adventure with a custom Twilio connector, a python file sitting in my project...

BUT... As of Rasa 2.6.0, Twilio connector is built in! That's not to say we still can't use a custom one, but to save us some angst with respect to this tutorial, we'll use the built in one. So be sure you have at least Rasa 2.6.0 installed.

credentials.yml

Open your credentials.yml, a file you probably haven't ever touched. We're going to add a configuration that will tell that internal Twilio connector we're going to use it.

First we'll configure the basic Twilio connector with the Account SID, Auth Token and phone number from our account.

Add the following to your credentials.yml file. You can add it above or below the default rest: and rasa: config that may already be there.

twilio:
  account_sid: <copy-paste-your-account-sid-here>
  auth_token: <copy-paste-your-auth-token-here>
  twilio_number: <copy-paste-your-phone-number-here>

One thing to note for the twilio_number is to just enter the digits, no - or ( and it does require a leading country code (in the USA that's a 1) So it should look something like this:

  twilio_number: 12223334444

Next we'll specify some custom parameters specifically for Twilio Voice. These are items that you can change and some will directly impact the "cost" of your account. I'll explain each one below.

Add this to your credentials.yml file as well

twilio_voice:
  initial_prompt: "hello"
  assistant_voice: "Polly.Salli"
  language: "en-GB"
  reprompt_fallback_phrase: "I didn't get that could you repeat?"
  speech_timeout: "1"
  speech_model: "default"
  enhanced: "false"

Some quick explanation here for each piece.

inital_prompt is what will get sent to your Rasa-bot in the background to trigger it. Your bot won't respond unprompted, so this gets things started. If you're using the out-of-the-box default Rasa bot, this will trigger the greet intent. You can also adjust the response specifically for Twilio using the channel specific variations Rasa allows. I'll touch on that near the end.

assistant_voice is the voice that will be used on the phone when your bot responds. There is a giant list of voices you can choose from, both male and female and from many nationalities.

What's great is you can play around with all of these to see what they sound like in the Text to Speech Console. Choose "Basic" or "Amazon Poly" at the top, and it will load the different voices. It's not very apparent, but click a voice and a modal will display. Enter some text into the box, and then click the little play arrow directly over the box and it will speak whatever you've just entered.

Once you find one you like, you can update the assistant_voice parameter accordingly. Use the Voice Name from the console (ie. Amy) and if you're using the Amazon Poly voice, add a Polly. before the voice name like this Polly.Amy. If you're using the Basic voice, you can just add the name alone, there's no need to prefix it with Basic or anything.

One thing to consider is that the Amazon Poly voices cost against your account, but it's literal fractions of a penny per character and if you're trying to impress your boss, then you want a great sounding voice, so this is worth the incremental cost.

Amazon has another voice system called Neural, these voices are more human, more robust and as you can imagine, cost more. But you have free money, so play around.

language is easy, just look at the voice list (or the Language code in the speech console) it'll be in the parenthesis ( ) and enter that. The voice names overlap, and some have the same names but different dialects and accents based on the language / nationality.

reprompt_fallback_phrase - this is functionally similar to Rasa's Default Fallback, however it is for Twilio. If for some reason it can't even attempt to understand you, or translate what you said to text, it will speak back this message instead of sending a garbled stream of characters to your bot. Because it is a different configuration you can make it a different phrase than your utter_default response.

speech_timeout Speech is different than text. When you complete a text message you hit send and the bot knows to process what you've typed. When speaking, the bot doesn't know when you're done, so this timeout is how much silence it will wait for before it attempts to send what you said to your bot. The shorter the time, the quicker response, however if you have someone that talks slow, a 1 second delay could produce a bot that interrupts that person. And if it didn't get the complete phrase it may ask them to repeat it when they never finished it. The flip side to this is that if it is too long, the system feels laggy, even though it's acting properly. I changed mine to 1 second because I talk quicker and when I demo the prototype, I want as little lag-feel as possible.

speech_model This is the model that Twilio will use when performing it's STT (Speech to Text) and TTS (Text to Speech) functions. Values here are limited to default, numbers_and_commands and phone_call. If you read the docs on these settings, default is best for talking to Rasa, unless of course you're building a number menu tree like many phone systems have in place, in which case you want the phone_call. If you are speaking short phrases and keywords, choose numbers_and_commands. These different models are trained up to specialize on the content that is being spoken.

enhanced This allows you to enable (or disable) Twilio's enhanced premium speech transcription (STT) service. This is a very good system and seems to nail the transcriptions better than when it's off. However, this will impact your account balance (costs about 50% more), so for the time being, unless you really have problems with your bot responding to your voice, I'd leave it at false most of the time. But then again, you have free money, just make sure you don't burn through it before you wow your boss 😉.

Believe it or not, that's all you have to do to prep your bot for voice, so it's now ready to respond to Twilio's API calls.

Lets fire it up and I'll show you how to get it wired up to Twilio.

Run this command -

rasa run -m models --enable-api --log-file out.log --cors "*" --debug

That may take a bit to start up, but eventually you'll see the wonderful message -

Rasa server is up and running.

I hear you saying - "But it's on my laptop! Am I going to call my laptop?"

Yes it is and no you're not, well not directly. We'll open it up to Twilio through a secure tunnel - which is WAY easier than it sounds, and infinitely easier than trying deploy it on a server.

That said - if you have one running on a server, you can certainly grab the rasa server url and skip the next section.

Getting your bot accessible from the internet

We'll use a utility called Ngrok to set this up. If you don't have it installed already, go to Ngrok's site, download and install it on your system. Once it's installed, open a new console, or Command Prompt if you're on Windows and type

ngrok http 50005

You're looking for something like this to come up -

ngrok by @inconshreveable                                                                               (Ctrl+C to quit)

Session Status                online
Session Expires               1 hour, 57 minutes
Version                       2.3.40
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://f43f711b19b9.ngrok.io -> http://localhost:5005
Forwarding                    https://f43f711b19b9.ngrok.io -> http://localhost:5005
Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

If you see this, we're golden, it's all working.
What this just did was setup a secure tunnel for port 5005 - where the Rasa server is running - and it maps it to a secure (yet publicly accessible url https://f43f711b19b9.ngrok.io

One thing to note is that the tunnel will only last for two hours. Notice the "Session Expires" time there. That will actively count down before it collapses the tunnel. And you just have to CTRL+C and run it again to generate a new tunnel.

>>>> UPDATE <<<<

You can create a free account on Ngrok's site and then your tunnel won't expire every hour!

Cool right?

All the information you need to do this is right there, however you can see there is a web interface at http://127.0.0.1:4040 If you hit that in a browser, you'll see the url's listed, but to get a crazy amount of information about the tunnel, click the status link. We don't need to know any of that for what we're doing but it's good to know it's there though. So back to the setup.

Configure the Twilio Webhook.

One more step and we're ready to call our bot. We need to Tell Twilio that our bot is temporarily living at the new Ngrok url

Go to your Twilio account, click the > next to Phone Numbers
Then click the > next to Manage
And click Active Numbers

Click your number to get to the configuration screen for that specific number.

Scroll down to the Voice & Fax section

In the row that says "When a call comes in"
Make sure it says "Webhook"

We want to change the first part of the URL to our ngrok email
The final url should look something like this

https://f43f711b19b9.ngrok.io/webhooks/twilio_voice/webhook

Make sure you copy in the url from your Ngrok terminal.

And that's it!

Time to try it out

Grab your phone, call your Twilio number. A brief trial message will play telling you to hit a number to execute your code. This will send your initial_prompt: value through Ngrok to your local instance of Rasa. When it does, you'll see the webhook hit Ngrok in the console, and then you'll see your rasa terminal spin like a top (because we launched it in debug mode).

Rasa will process what came in from Twilio, and send back text and Twilio will translate that into speech, and your bot will talk back to you! (now quickly ask for a raise)

There are times in your bot where you may have a response like this -

  utter_greet:
    - text: Hello, I am TwilBot. \nPlease let me know how I can help you

If this is read to you, Twilio (and any other TTS engine) will read back the line break \n and it will say "backslash n". But we want that line break there for our web chat component. So what to do????

Remember the channel specific responses we mentioned before? This is there those come in handy. Just change the response to this, and you're golden.

  utter_tell_greet:
    - text: Hello, I am TwilBot. \nPlease let me know how I can help you
      channel: "twilio_voice"
    - text: Hello, I am TwilBot, thank you for calling. Please let me know how I can help you

My only gripe with the channel formatting looks a bit odd, having the channel: with the same indent level as the - text: response above it, and then having the channel-specific response at that same indent level. At first I thought it was a typo in the docs, but sure enough that's how it should look.

Save that and any other channel changes you want to make, train your bot and re-run it using the same command we had before

rasa run -m models --enable-api --log-file out.log --cors "*" --debug

Wait for this message...

Rasa server is up and running.

And as long as your Ngrok tunnel hasn't expired, you can call again and it should greet you properly.

Congratulations

Great job, you made it to the end and hopefully your boss is excited and ready to give you that raise.

But wait - there's more!!!

Setting up Messaging

Speaking to your bot is cool, but texting your bot is pretty awesome too. Here's how you configure that.

The number you chose will support both voice and SMS it just needs to be configured. This was a little confusing at first, because you need to setup what they call a Messaging Service. All this is, is a configured name, it's nothing involved. I thought you'd need some other service to hook in, but really you just need to set a name and go. Let's do that.

Explore Products > Messaging

This will land you on the Programmable Messaging Dashboard. There's a blue button that reads - Start Sending Messages

Click that 😆

An Modal will open and want to know the Messaging Service Name, so name it. It's arbitrary - you can call it "My Messaging Service" or you can call it "SendIt", heck you could call it AT&T if you wanted to, not sure why you would though.

For the Messaging Service Use Case on the same modal, choose one that fits our needs - "Engage in a discussion"

The blue button there that says Create - you guessed it - Click it.

In the left nav, click Integration
Then choose - Send a Webhook

In the Request URL box, enter our NGrok url, the same one you entered for Voice.

Click Save. (yea I know I like to switch it up a bit)

A few more steps and we're done.

In the top navbar, click the "My First Twilio Account" link, then navigate back down to the phone number screen

Phone Numbers > Manage > Active numbers > click your number

Scroll down to the Messaging section and select the Messaging Service you setup.

Then in the Webhook url - paste in your NGrok URL, the same one you entered for the voice config.

Click Save.

THATS IT!!

Now jump on your phone and text hello to the number you have setup. It should reply with a message that starts with "Send from your Twilio trial account -". This text will prefix all text messages received from your bot. Obviously it will go away if you upgrade.

Whelp, that's all I got. Hope this go you that raise you were looking for #fingersCrossed

A Makefile for Rasa

Jonathan Wheat — Fri, 30 Apr 2021 15:48:44 +0000

As a programmer, I can be lazy and have a list of aliases I use when working with everything on the command line, Rasa included.

At Nearly Human we use make for a lot of different things, and I decided to utilize the muscle memory for typing make all the time for my Rasa development.

If you're curious, currently I work on Windows 10, and I do the bulk of my Rasa development using VSCode and use the integrated WSL Ubuntu terminal. I'm working on another tutorial about setting up VSCode for Rasa development, but haven't finished that one yet.

Installing make

From an WSL Ubuntu terminal install make

sudo apt-get update -y
sudo apt-get install -y make

When that's completed make sure it's all happy with

make --version

You should get something like this - possibly with a different version. The important thing here is that you don't get an error.

GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

We also use commit hooks with pre-commit to keep some formatting standards. Lets get that installed too.

This is really easy as well. It's just

pip install pre-commit

We'll need a configuration for this, so create a .pre-commit-config.yml file in the root of your project.

You can copy / paste in this generic configuration

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files

To test this lets check all your files manually first, then we'll create a make recipe for it.

Run this

pre-commit run --files *.yml data/*

IF all goes well you'll see something like this

Trim Trailing Whitespace.......................................Passed
Fix End of Files...............................................Passed
Check Yaml.....................................................Passed
Check for added large files....................................Passed

If you get a Failed, read why, it should tell you what happened. What's great about pre-commit is that it will fix most common formatting issue on it's own. So run that again if you saw a Failed line and chances are this time they'll all pass... unless of course you really botched a file badly.

I'll presume you have that all working now.

The Makefile

Now to create the fun part. In the root of your Rasa project, create a new file named Makefile <-- there's no extension, it's all one word with a lowercase 'f'

You can read online all about make and how to construct recipes, but these are the ones I use. Essentially you have a recipe name, and a set commands you want to execute, one per line

Clean

This removes past model files. If you're like me you train often and this directory can get pretty large depending on the size of your model file.

In your Makefile add this super easy recipe

clean:
    rm models/*.gz -f

Save that, and from the terminal run make clean and it'll execute and remove your model files. The -f forces the recipe to run and not fail if the command fails. So for example if there are no models to delete, that command technically fails and without the -f flag, the recipe would fail as well.

Validate

This is a crazy valuable recipe that runs the validate command. Mostly because I have a very complex data directory (I do NOT have a domain.yml file in the root of my project, so I have to specify a parameter)

Add a new line and add this recipe to your Makefile (** see the note below)

validate:
    rasa data validate --domain data

** You may need to remove the --domain data to get this to work depending on your setup.

To execute this one you can run make validate

Training

The next most important recipe is the training.
This recipe is fantastic. It'll run the clean recipe removing the model files, then check your formatting, then validate, then train.

Add this recipe to your Makefile

train: clean
    pre-commit run --files *.yml data/*
    rasa data validate --domain data
    rasa train --domain data --out models

You'll note that clean is on the same line as the recipe name, this is on purpose, it'll allow you to run another recipe from this one.
As you would expect, run make train to execute this time-saver.

Testing

Now that we have a model trained you can test it. This command typically has many parameters so it's very handy to use a recipe for.

Add this to your Makefile

test: check
    rasa test --stories tests/conversation_tests.yml --fail-on-prediction-errors

run make test for this one.

This runs a check using my custom conversational_tests.yml file. You can change these parameters to whatever you use to run your tests.

Almost done

I have two more for you. I like to run the rasa server api locally (not rasa shell) and that has many parameters. Also in conjunction with that, I like to fire up the action server in another terminal window so here's two more recipes that are useful

Action Server

runactions:
    rasa run actions --action actions --debug --auto-reload

Fire up the action server in debug mode with auto-reload enabled with make runactions There are circumstances where the auto-reload flag doesn't actually reload your changes, so just be aware of that, but it's not related to the Makefile

Rasa Server (not shell)

run:
    rasa run -m models --enable-api --log-file out.log --cors "*" --debug

To launch this recipe type make run the rasa server will fire up with the api active, create a log file for you, enable cors for external communications and do it all in debug mode (very useful). It will sit there waiting for connections from your chat interface, whether that is a web ui, slack, messenger, whatever.

That's It

Hopefully that'll save you some keystrokes and you can become a lazy developer like me :)

If you have any recipes or commands you'd like to share, do that in the comments and I just may steal ... er use them in my workflow.

Organize your Rasa 2.0 training data like a boss

Jonathan Wheat — Mon, 28 Dec 2020 16:13:42 +0000

Rasa 2.x gives us a lot of new features as conversational designers / developers. The coolest feature that isn't quite apparent is the ability to group and organize your training data any way you please.

At my company we build "abilities" for our bot - think of them similar to Alexa Skills. Essentially you have some domain information, nlu training data, stories and sometimes actions or forms to round out the ability. The more abilities the bot has, the longer and more complex your files get; which leads to a longer time to grock what you're looking at if you drop in to tweak something, let alone onboard a new developer.

The old days

Rasa 1.x allowed you to split up your nlu and stories files simply by creating a data/nlu, and data/core directory in your project and putting the individual files there. You can group your data into separate files which makes it easier to find something if/when you need to change something. For example if you needed to add new chit-chat training data, you could jump into data/nlu/chit-chat.md and add new data. Initiating the rasa train command utilizes the files in data/nlu and data/core in combination with domain.yml in the root of your project to train your model.

This was great, but not ideal for me. I built a script to let me split my domain files in a similar way; creating a data/domain directory and putting my files there. Rasa however, didn't recognize that directory, so I wrote a script to merge these files into a single domain.yml file and drop it in the root. This allowed the rasa train command to utilize my separate domain related files.

A New Organizational Paradigm

Rasa 2.x gives us the ability to split up our domain files and the benefit to that is clear; smaller files with more focused data. I also don't have to utilize my custom script now!

Why is this cool? To expand on my explanation above; if your bot can handle chit-chat, weather, restaurant search, and directions you would have a single long domain.yml file in the root of your project with ALL of your intents, slots, entities, responses, action calls, and form config. Your topical data is interlaced, and it makes it hard to find things. Being able to split this into different files just makes more sense. (Thank you Rasa!)

Your new data directory structure can now change to -

data/core
data/domain
data/nlu

And each of these can contain multiple files that make up your bot's data. You can even do this with your action files.

Kick It Up A Notch!

Here's something that is an amazing side effect / undocumented feature of the way Rasa deals with training data in 2.x You can create directories under core,domain, and nlu and Rasa will recurse down through looking for files during the training process.

I know you're asking - why is this awesome? In our case, as I said we build abilities, which are mostly isolated functions and conversational scenarios. In v1 we adopted a filename convention to differentiate between abilities. In v2, by exploiting this new directory structure we can have individual developers work on a single ability without stepping on the toes of other developers.

They can create a new ability directory - let's say they're working on a book recommendation ability. Our dev creates data/book-recommendation and in that directory creates a domain.yml, nlu.yml, stories.yml, rules.yml and works solely from that directory. Fun fact, the filename doesn't matter. Each .yml file is keyed - intents:,nlu:, even rules: so it doesn't matter how many files you have, or what the names are, it all works!

If you decide to do this, you'll need to run rasa train with the --domain parameter so it will find your domain files

rasa train --domain data

If you leave off the --domain parameter, Rasa will look for domain.yml in the directory you're running it from so be sure to delete domain.yml in the root of your project, or you may be quite confused why your latest changes aren't getting pulled in.

Don't Leave Actions Out In The Cold

You can also do this with your action.py file, albeit in a different location and there's an extra file. We create an ability directory under actions/, drop in an empty __init__.py file (making python treat it as a package) then add an actions.py file (or whatever filename you want)

In our book recommendation example we would have something like this:

actions/__init__.py
actions/book-recommendation 
actions/book-recommendation/__init__.py
actions/book-recommendation/actions.py

Doing all of this directory organization centralizes the code and lets your developer spin up a local rasa init project, and work on that ability from beginning to end, creating a very focused bot complete with tests. One caveat is if the ability being worked on is integrated with another ability in some way. Depending on the level of that reliability, you may think about whether the new code is actually a separate/new function as opposed to an extension of a current ability - but that's getting away from the main topic here.

In practice

We have plan to have a special repo of abilities, so when our devs are done they can just move their directories over issue a PR and that new ability will be available for everyone else on the team to pull down and add to their bot if needed.

But wait, there's more

Up until now, if your ability had any python related action code, you'd have 2 directories to manage. What if you could create a truly self-contained ability in one directory. Literally add one directory of files, retrain and have a new ability in your bot?

You can.

To achieve this, we'll move the action files into our data/book-recommendation directory. There's some setup to do this however.
Remember the __init__.py files we've been dropping all over? Python uses those to detect if a directory is loadable (a package).

To get our all-in-one setup we'll need to drop an __init__.py into data and then move our __init__.py and the specific actions.py file from our actions' ability folder into our data's ability directory. This way everything is 100% self-contained in one single directory like this:

data/__init__.py
data/book-recommendation
data/book-recommendation/__init__.py
data/book-recommendation/actions.py
data/book-recommendation/stories.yml
data/book-recommendation/domain.yml
data/book-recommendation/nlu.yml

The trick here is to run your action server with the --actions parameter like this:

rasa run actions --actions data

This tells rasa to load the actions files from your data directory, and it will recurse down and load any python files it finds.

As noted above, you'll also need to run rasa train with the --data parameter like this:

rasa train --domain data

That will tell Rasa where your domain files are.

Parting Thoughts

I think this is a pretty cool advancement in the ability (no pun intended) to organize our data, streamline our development process and allow a very interesting approach to developing different independent functions.

I'm not sure I like the python being intermingled in the same directory as my .yml files, it feels a little gross, but I supposed I could also create a data/book-recommendation/actions directory to move out all the python other than the __init__.py file of course. Or maybe even go crazy with

/data/book-recommendation/actions
/data/book-recommendation/data

OR even rename our data directory and create something like this:

/abilities/book-recommendation/actions
/abilities/book-recommendation/data

If you do something crazy like this be sure to alter your --data and --action parameters when firing up Rasa!

Those both feel a little over the top, but the point is the possibilities are endless and you have the ability to organize your files however makes sense to you.

I'll continue iterating on this approach. I'm interested in knowing what others are doing to organize their data. The single file system works for smaller / simpler bots, but anything with some robustness will quickly outgrow that model (pun intended).

Let me know what you think in the comments!

Debugging Rasa Errors

Jonathan Wheat — Tue, 22 Sep 2020 18:25:46 +0000

This is a short summary of the "random" Rasa and python errors (and what to look for to fix them) you will no doubt encounter when you're developing a chat bot using Rasa. Some are pretty obscure, and after some time you'll get used to identifying them, and other times no so much.

I'll admit, this post is pretty selfish, it is more for me to have a location bookmarked that I can reference from time to time when I can't remember, but I know it'll help someone new or old to the Rasa scene.

I've posted this as a repo as well (https://github.com/jwheat/rasa-errors), so if you have any updates / corrections / or new errors I've missed (now that there's a new Beta out) feel free to issue a pull-request and I'll get them merged in.

Now without further rambling, here are the errors, in no particular order, other than the order I've encountered them over the last few months when I was compiling this.

AttributeError: 'str' object has no attribute 'setdefault'

File "c:\users\jonathan\anaconda3\envs\rasa\lib\site-packages\rasa\core\domain.py", line 290, in _transform_intent_properties_for_internal_use
    properties.setdefault(USE_ENTITIES_KEY, True)
AttributeError: 'str' object has no attribute 'setdefault'

Type: Syntax Error

File: domain.yml

Where to look: intents: - specifically use_entities: []

Solution:

Make sure there is a space between the : and the []
Make sure your use_entities: [] is indented properly under the parent intent name. Here's an example of good indenting practices

intents:
  - enter_data:
    use_entities: []

AttributeError: 'bool' object has no attribute 'get'

File "c:\users\jonathan\anaconda3\envs\rasa\lib\site-packages\rasa\core\domain.py", line 265, in collect_slots
    slot_class = Slot.resolve_by_type(slot_dict[slot_name].get("type"))
AttributeError: 'bool' object has no attribute 'get'

Type: Syntax Error

File: domain.yml

Where to look: slots:

What the problem is: indenting slot properties

Solution: you need an ever increasing set of indents for slot definitions. Make sure your type and auto_fill, initial_value and any other properties of a slot are indented under the slot name

What I've done: I utilize past code a lot, and VSCode likes to automatically indent when you past a block that has indening, so many times my indenting is off and I'm in a rush and don't see it is incorrect. Needless to say, this error is my most frequent.

Here is an example of proper slot indentation

slots:
  customer_first_name:
    type: unfeaturized
    auto_fill: true

pykwalify.core - ["Value 'None' is not a dict. Value path: '/slots'"]

Type: Syntax Error

File: domain.yml

Where to look: slots:

What the problem is: indenting slot name, and/or missing - if your slot happens to be a list or array

Solution: Similar to the above error, you need an ever increasing set of indents for slot definitions with proper - hyphens as well. Make sure your slot name is properly indented under the slots: key and double check your hyphens o be sure they're all there (or not)

What I've done: Sometimes even though you have your hyphens, the indents are still off. I have some custom slots that have dictionaries (multiple hyphenated items) and sometimes I forget the -

Here is an example of proper slot indentation

slots:
  risk_level:
    type: categorical
    values:
      - low
      - medium
      - high

rasa.core.actions.action - Failed to extract slot customer_first_name with action form_change_customer_first_name

Type: spelling / naming error

File: actions.py, domain.yml

Where to look: python slot code in your actions.py and specifically the slot names in your domain.yml

What the problem is: slot name mismatch (aka typo)

Solution: - this is more than likely a naming / spelling issue. Make sure the slot name you're referencing in your actions.py file MATCHES to the slot and/or response in your domain.yml file.

What I've done: Sometimes I'll write python code for a new slot, and forget to add it to my domain.yml file, or I'll rename it in my head and forget to update it somewhere.

TypeError: 'NoneType' object is not iterable

Type:

File: actions.py

Where to look: required_slots method

What the problem is:

Your required slots does not return anything. You may have some nested if logic, and chances are whatever path you just took in chat, resulted in a path that did not return anything.

Solution:

A quick hack, not advisable is to add a return [] to the end of your required_slots method. This catch all will always return an empty list, and satisfy the actual functionality of this method.

The better solution is to trace your if logic and handle the path you took in chat, and return an actual list (even one) of requiredSlots

Check out the repo (https://github.com/jwheat/rasa-errors) for any updates to this, or send me a pull-request if you have changes, or new errors to add!

Using Docker with Rasa for development

Jonathan Wheat — Thu, 21 May 2020 15:25:48 +0000

Requirements

Docker Engine install docs
Docker Compose install docs
Rasa Open Source install docs

If you're developing a chatbot with Rasa, no doubt you've gone down the same path as me and opened multiple consoles / terminals to launch the Rasa server, Action server, Duckling, your webchat ui and possibly more depending on how advanced your bot has become.

This is a really easy way to develop, although unless your terminal app can do tabs, it becomes quite tiresome tracking down the proper one to restart.

I'm going to show you how you can setup docker to launch them all with one command, and give you the flexibility to watch the logs, connect directly to a service, and of course stop and restart them when needed.

We're going to use the docker-compose.yml file to achieve this.

Docker Compose

Briefly, the docker-compose.yml file is a configuration file that tells docker how to build a stack of containers. That means, it will fire up a single container for each service (chat server / action server / chatUI / etc.) all with one command. It's pretty sweet, especially if you're used to opening up multiple consoles to launch each service like I mentioned above.

I've attached my docker-compose.yml file to this post in case you want to just jump ahead, because who has time to actually read blog posts these days. Am I right?

If you do happen to have time, good for you, I'm going to explain how to get this working, and you'll be 30 minutes ahead of the people that skipped this section, wondering why my file didn't work straight away.

If you have a Rasa project, you can download and drop my file into the root of your project. My file assumes your actions.py (and other action files) are located in an actions directory off your project directory. If you followed my post about the actions files, then you're good to skip the next section.

Configure your actions

To use my file as-is, you just need to do a couple things

create an actions directory off your project root
move actions.py into it
copy __init.py__ from your project root into it

For more information about the actions files and the actions directory read my post here

docker-compose.yml - Some items of note

We're going to talk briefly about a few of the config options in the docker-compose.yml file.

First up is volumes. The volumes option takes an array of values, meaning you indent and use a hypen ( - ) for each item. The items here are directories you want to map to docker. In my file the volumes section looks like this for the rasa-server service:

    volumes:
      - ./:/app

You can see the value on the volumes: line is split with a :. This tells Docker what local directory (the left side of the :) to make available to the service container and where in the container (right side of the :) to mount them to.

You end up with something like ./:/app That looks like a lot of dots and slashes, but essentially it is saying to map the current local project directory ./ to /app in the container. The result if this notation is that all files and directories in the project folder (think configuration, models, nlu data, etc) are made available to the rasa server container when it's fired up.

This makes sense if you think about it because the server requires all of those config files and your models in order to run. It would be useless if you couldn't get your models or configuration to be read by Rasa in the container.

To beat a dead horse, we'll look at the volumes: value for the action-server service configuration. It is similar.

    volumes:
      - ./actions:/app/actions

I know what you may be thinking, we just told docker to use the entire project directory (including the actions directory), why do we need to do it again?

you'll notice in the docker-compose.yml file there are different sections called services:. We have a rasa-server service and an action-server service. Docker allows us to configure each one independently giving us some crazy flexibility and allows us to tune each service to use only what it needs.

You also may be thinking, why didn't you use the same ./:/app value from the rasa-server? You could and it would work, but that would bloat the action-server container because it would also have access to the models and other config data. Best practices say to limit access to just what is needed and make your containers as small as possible.

Back to the beating that horse with actual value now.

    volumes:
      - ./actions:/app/actions

The actions server is mounting the actions directory we created above (./actions) and making it available at /app/actions inside the action-server container. The action server docker image that is being pulled, knows to load the actions from it's local /app/actions directory. Make sense?

Why is this important?

The volume: parameter is important because it allows outside files into the container, meaning data can be saved from the container to the volume and will persist next time you run the container. The same in reverse as well. If you make changes to your domain.yml, nlu.md, stories.md, or any other config file, and retrain rasa, you just stop and restart the container and it will use your new models and configuration.

These volume mappings allow you to continue to develop / train, etc locally with Docker and have your data and configuration persist.

How to use Docker Compose

Provided you have the requirements mentioned at the top of this article, it is pretty easy to use docker compose.

Drop the file into your rasa project directory run docker-compose up and should be good to go.

docker-compose up will keep the terminal live, and you'll see all the logs for each of the services you're firing up. You would use CTRL+C to stop everything.

You can also run docker-compose up -d and that will launch them in the background and give you your terminal back. To then stop them all if running in the background, run docker-compose down and they'll all stop.

One thing that may (or may not be) obvious is that you need to be in the same directory as your docker-compose.yml file to run these commands. Since that is our configuration file, docker compose uses that to know what to launch or stop.

Becoming more proficient

When I first started out, I never used the -d flag because I like to watch logs as I test and develop. This is especially useful if you're starting out because you can immediately spot issues with configurations, etc. But, after a while you'll want to fire this off and you'll want your cursor back :)

The -d flag is handy because they run in the background. This allows you to be able to stop different containers of needed, so if for example you're editing your actions.py file, you can just stop and restart the action server instead of bouncing all of your containers each time. That is ok if you do this, to me it is just a waste of time.

Restarting a container

First you need to know the name of your container. To get a list of your running containers, run docker ps. You'll see something like this:

CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                               NAMES
af21abaf9997        rasa/rasa:1.9.0       "rasa run --cors * -…"   2 hours ago         Up 8 seconds        0.0.0.0:5005->5005/tcp              rasa-r2-server
f631aa0cd485        mysql:5.7             "docker-entrypoint.s…"   2 hours ago         Up 8 seconds        0.0.0.0:3306->3306/tcp, 33060/tcp   mysql-rasa-r2
e18dc702ff5b        rasa/rasa-sdk:1.9.0   "./entrypoint.sh sta…"   2 hours ago         Up 8 seconds        0.0.0.0:5055->5055/tcp              rasa-r2-action-server

If you wanted to restart the action server for example, you can take the CONTAINER ID for the action server (the bottom one) and run docker container restart e18dc702ff5b

When that's completed, you can look with docker ps again and see it was restarted under the CREATED column

CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                               NAMES
af21abaf9997        rasa/rasa:1.9.0       "rasa run --cors * -…"   2 hours ago         Up 3 minutes        0.0.0.0:5005->5005/tcp              rasa-r2-server
f631aa0cd485        mysql:5.7             "docker-entrypoint.s…"   2 hours ago         Up 3 minutes        0.0.0.0:3306->3306/tcp, 33060/tcp   mysql-rasa-r2
e18dc702ff5b        rasa/rasa-sdk:1.9.0   "./entrypoint.sh sta…"   2 hours ago         Up 29 seconds       0.0.0.0:5055->5055/tcp              rasa-r2-action-server

For those of you still reading, here's some coolness that we've implemented that you didn't even know about.

What's in a name

Those hashes change each time a container is launched, and are a pain to deal with especially if you need to bounce a container a lot during development.

You'll notice that each container has an actual name (NAME column), our action server is named rasa-r2-action-server whether you realized it or not, we defined this in our file with the parameter container-name:. Now we know what the container will be called all the time, it won't change no matter how many times you bounce it.

You can use this name to restart the container like this:
docker container restart rasa-r2-action-server

That makes it easier to restart because now you don't need to always get the new hash with docker ps

If you leave out the container-name: parameter in docker-compose.yml, docker will auto generate a name for the container which is fine, but not ideal, it is just easier to give each of our containers a name, then we dictate what it is called.

Killing a container

Occasionally you may need to forcibly kill a container, or even just stop one for a while instead of restarting it. This is very similar to restarting a container, you just use the keyword kill instead. docker kill rasa-r2-action-server If you run docker ps you'll see it is now gone.

Starting one service

Great, you've killed a service using the container name, fixed whatever the issue is and want to fire it up again? Well you'd think you can access it via the container name because that's what we've done in the other sections. Don't be fooled, there is no container with the name of the one you just stopped. We need to access it by the service name. In our example you'd run docker-compose up -d action-server where action-server is the service name defined in docker-compose.yml Check docker ps and you'll see it is fired back up.

Watch that log

It can be quite important to look at the logs in the event something does wrong. You can use the container name we defined above and watch the logs like this:

docker logs rasa-r2-action-server -f

2020-05-20 16:38:04 INFO     rasa_sdk.endpoint  - Starting action endpoint server...
2020-05-20 16:38:04 INFO     rasa_sdk.executor  - Registered function for 'action_hello_world'.

I added the -f flag to keep the logs active. This command will take over the terminal and display changes to the log in real-time. Hit CTRL+C to stop and get your terminal back.

Get me in there!

For Rasa, it would be rare for you to need to ssh into a container, but I'll round out the article with how to do that in case you need to verify some library version or something.

docker exec -it rasa-r2-server bash

This will drop you into the /app directory and you'll be inside the container. You can run something like rasa --version or python --version, or pip list whatever you need to. Just type exit to pop back out.

Docker Compose Docs

Take a look at the docker compose docs, there are some other useful commands like pause, start, top, etc that may prove useful as you get more into developing using docker.

My docker-compose.yml file

Here's a link to the file I use
docker-compose.yml

That's it

That is everything you need to know to develop Rasa using Docker. If you notice something I typed or said incorrectly, please let me know and I'll fix it right away. I tried to perform these examples as I wrote this, hopefully everything works for you. Good luck!

Splitting your Actions in Rasa

Jonathan Wheat — Thu, 30 Apr 2020 15:21:01 +0000

Rasa is an amazingly flexible open source system for building conversational chat bots. You can quite literally have the basic out-of-the-box bot working in less than 15 minutes. There are a host of tutorials and videos online that explain how to set up, extend and train your bot.

The main reason Rasa is so flexible is that you can build out actions to extend the functionality. If you are familiar with Alexa, you can think of these as added skills. These actions can be as simple as an API call to an external system (ie weather service or joke of the day) or could be setup to walk a user through a set of questions to gather information (think "form"), or you can build actions to interact with an external ERP, other complex system, knowledge base, or really whatever you can think of.

All of these actions are built with python and by default exist as classes in the actions.py file. You can imagine that this file can get quite large and unwieldy as your bot grows in abilities and functionality. Thankfully there is a way to split up and organize these actions to make development more streamlined. This method is not documented anywhere on the RASA site, although it is fully supported out of the box and requires only a little modification to your project's directory structure, which also helps with organization.

Note
For this tutorial, we'll reference my main project directory C3PO. I'm on Ubuntu, so I'll also give you the commands to run where necessary, but feel free to perform the necessary steps however you feel comfortable.

From a terminal, lets change to our project's home direcotry:
cd ~/RASA/C3PO

Setting Up Your New Actions Directory

Create a new directory off of the root of your project's directory called actions. This is where all of your new action files will live.

mkdir actions

You'll end up with something like this:
C3PO/actions

Now we want to make that actions folder a package in python. This sounds pretty involved, but really only requires the creating of one file with a special filename.

In the actions directory create an empty file named __init.py__ That may be hard to tell, but that is 2 underscores before and after the init.py.

From your project's home run
touch actions/__init.py__

That's all, you don't have to edit this file at all, it just has to exist. The actions directory is now considered a package and what that means is that when it is referenced all files in there will be loaded. Perfect.

Move and package

We can now move our actions.py file into our package directory. Be sure to not copy it, move it there or else Rasa won't check for the actions directory.

mv actions.py actions

GREAT. You can consider your actions.py file "packaged" :) That was pretty easy.

You can stop now and everything will work as it did before, but that's not the point of this article is it.

The Big Split

The first thing you'll want to do is open actions.py in your code editor.
Create a new file in your actions directory, the filename is arbitrary. Give it a filename that means something to you. Here is where the organizational part comes in, and only you can know what makes sense.

If you're like me, you have a naming convention throughout your bot, and you've split your NLU, Stories and Domain data already. If you have, you'll probably use that naming style here. For example, I've prefixed all of my actions with an a and an index number that references internal documentation about that particular element whether that is an intent, story, entity, etc.

Here's an example action filename from one of my bots: a004-ext-api-bmchelix-create-issue.py I can tell from the text portion that's an external api call to BMC Helix to create an issue (ticket) and I can look up item 004 on my internal documentation and read more about that particular call, if there are other references to entities, etc. Internal docs are super important, especially if you have other developers working with you on this. Even if you're a lone-wolf developer, these bots can get pretty complex, so I can't stress enough about taking your time and taking notes about what you've done.

Continuing on.

At the top of this new file, you'll need to copy / paste the top section from your main actions.py file. This includes all of the code down to the first Class statement. For me it is all of the import and from lines like this:

import logging
from typing import Any, Text, Dict, List, Union, Optional
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk.forms import FormAction
from rasa_sdk.events import SlotSet, AllSlotsReset, Restarted, UserUtteranceReverted, ConversationPaused

logger = logging.getLogger(__name__)

That gives your new action file access to all of the RASA related items you'll need for an action.

Then go ahead and move (Cut / Paste) the class or classes from your main actions.py file that relate to the new file you've created.

Save it.

Repeat that process as necessary as you want.

Now you'll have an actions package directory full of .py action files.

Then What?

Then nothing. You're done. There's nothing special to do. You can still launch your bot the same way you always have with rasa run actions and it'll load all of your custom action files as if they were in one file like before.

You can try this to be sure you haven't made an error. Run rasa run actions and the server should spin up. You'll see all of your action class names scroll onto the screen and it'll sit there waiting for the Rasa server to hit it.

Help Me Help You

I want this to be useful, so if you discover I've made any errors, let me know and I'll correct them asap.

Happy developing