DEV Community

Jonathan Wheat
Jonathan Wheat

Posted on • Edited on

A Makefile for Rasa

#rasa #make #makefile #recipe

As a programmer, I can be lazy and have a list of aliases I use when working with everything on the command line, Rasa included.

At Nearly Human we use make for a lot of different things, and I decided to utilize the muscle memory for typing make all the time for my Rasa development.

If you're curious, currently I work on Windows 10, and I do the bulk of my Rasa development using VSCode and use the integrated WSL Ubuntu terminal. I'm working on another tutorial about setting up VSCode for Rasa development, but haven't finished that one yet.

Installing make

From an WSL Ubuntu terminal install make

sudo apt-get update -y
sudo apt-get install -y make
Enter fullscreen mode Exit fullscreen mode

When that's completed make sure it's all happy with

make --version
Enter fullscreen mode Exit fullscreen mode

You should get something like this - possibly with a different version. The important thing here is that you don't get an error.

GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Enter fullscreen mode Exit fullscreen mode

We also use commit hooks with pre-commit to keep some formatting standards. Lets get that installed too.

This is really easy as well. It's just

pip install pre-commit
Enter fullscreen mode Exit fullscreen mode

We'll need a configuration for this, so create a .pre-commit-config.yml file in the root of your project.

You can copy / paste in this generic configuration

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files
Enter fullscreen mode Exit fullscreen mode

To test this lets check all your files manually first, then we'll create a make recipe for it.

Run this

pre-commit run --files *.yml data/*
Enter fullscreen mode Exit fullscreen mode

IF all goes well you'll see something like this

Trim Trailing Whitespace.......................................Passed
Fix End of Files...............................................Passed
Check Yaml.....................................................Passed
Check for added large files....................................Passed
Enter fullscreen mode Exit fullscreen mode

If you get a Failed, read why, it should tell you what happened. What's great about pre-commit is that it will fix most common formatting issue on it's own. So run that again if you saw a Failed line and chances are this time they'll all pass... unless of course you really botched a file badly.

I'll presume you have that all working now.

The Makefile

Now to create the fun part. In the root of your Rasa project, create a new file named Makefile <-- there's no extension, it's all one word with a lowercase 'f'

You can read online all about make and how to construct recipes, but these are the ones I use. Essentially you have a recipe name, and a set commands you want to execute, one per line

Clean

This removes past model files. If you're like me you train often and this directory can get pretty large depending on the size of your model file.

In your Makefile add this super easy recipe

clean:
    rm models/*.gz -f
Enter fullscreen mode Exit fullscreen mode

Save that, and from the terminal run make clean and it'll execute and remove your model files. The -f forces the recipe to run and not fail if the command fails. So for example if there are no models to delete, that command technically fails and without the -f flag, the recipe would fail as well.

Validate

This is a crazy valuable recipe that runs the validate command. Mostly because I have a very complex data directory (I do NOT have a domain.yml file in the root of my project, so I have to specify a parameter)

Add a new line and add this recipe to your Makefile (** see the note below)

validate:
    rasa data validate --domain data
Enter fullscreen mode Exit fullscreen mode

** You may need to remove the --domain data to get this to work depending on your setup.

To execute this one you can run make validate

Training

The next most important recipe is the training.
This recipe is fantastic. It'll run the clean recipe removing the model files, then check your formatting, then validate, then train.

Add this recipe to your Makefile

train: clean
    pre-commit run --files *.yml data/*
    rasa data validate --domain data
    rasa train --domain data --out models
Enter fullscreen mode Exit fullscreen mode

You'll note that clean is on the same line as the recipe name, this is on purpose, it'll allow you to run another recipe from this one.
As you would expect, run make train to execute this time-saver.

Testing

Now that we have a model trained you can test it. This command typically has many parameters so it's very handy to use a recipe for.

Add this to your Makefile

test: check
    rasa test --stories tests/conversation_tests.yml --fail-on-prediction-errors
Enter fullscreen mode Exit fullscreen mode

run make test for this one.

This runs a check using my custom conversational_tests.yml file. You can change these parameters to whatever you use to run your tests.

Almost done

I have two more for you. I like to run the rasa server api locally (not rasa shell) and that has many parameters. Also in conjunction with that, I like to fire up the action server in another terminal window so here's two more recipes that are useful

Action Server 

runactions:
    rasa run actions --action actions --debug --auto-reload
Enter fullscreen mode Exit fullscreen mode

Fire up the action server in debug mode with auto-reload enabled with make runactions There are circumstances where the auto-reload flag doesn't actually reload your changes, so just be aware of that, but it's not related to the Makefile

Rasa Server (not shell) 

run:
    rasa run -m models --enable-api --log-file out.log --cors "*" --debug
Enter fullscreen mode Exit fullscreen mode

To launch this recipe type make run the rasa server will fire up with the api active, create a log file for you, enable cors for external communications and do it all in debug mode (very useful). It will sit there waiting for connections from your chat interface, whether that is a web ui, slack, messenger, whatever.

That's It

Hopefully that'll save you some keystrokes and you can become a lazy developer like me :)

If you have any recipes or commands you'd like to share, do that in the comments and I just may steal ... er use them in my workflow.

Top comments (3)

Collapse
 
emmajadew profile image
Emma Jade Wightman •

Another great tutorial! :D

Collapse
 
jonathanpwheat profile image
Jonathan Wheat •

Thank you Emma. Funny, I just pulled down your helpdesk-assistant and I see a Makefile :) That commit pre-dates mine so I know I didn't influence it, but it's funny I eventually got there on my own.

Ya'll do a lot of neat things with your repos and code, the way things interact and pull configuration data from the files system. Lots of great patterns there to learn from if you have time to dive into the code.

Collapse
 
socr102 profile image
Eric •

Hello Jonathan Wheat
I am glad to see your post
I am very interesting in your post
Because I am a beginner in Rasa
I posted about my problem
Could you tell me how to install rasa in linux?
thank you
best
Mark

Read next

cheedge_lee profile image

Etcd Backup and Restore (4)

Cheedge Lee -

codebyblazej profile image

From Wrenches to Code - Why Learning to Code Is Like Fixing a Car

CodeByBlazej -

hema_latha profile image

What is Encapsulation ?

hema latha -

aditya_debnath_9348ca986b profile image

December Magic

Aditya Debnath -