DEV Community

loading...
Cover image for RASA - Testing unhappy paths

RASA - Testing unhappy paths

Petr Janik
I'm a student of software engineering at Masaryk University in Brno. I love coding, especially in React, Python and Java.
Updated on ・3 min read

In the previous chapter, we have written two unhappy paths for our newsletter_form. Now let's test that those paths work as we want them to.

# tests/test_stories.yml
# ... previous contents ...
  - story: Chitchat during email
    steps:
      - user: |
          I want to subscribe
        intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - user: |
          How's the weather?
        intent: chitchat
      - action: utter_ask_continue
      - user: |
          Yes, I do
        intent: affirm
      - action: newsletter_form
      - active_loop: null
      - action: utter_subscribed

  - story: Chitchat during frequency
    steps:
      - user: |
          I want to subscribe
        intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - user: |
          my email is [example@email.com](email)
        intent: inform_email
      - action: newsletter_form
      - user: |
          How's the weather?
        intent: chitchat
      - action: utter_ask_continue
      - user: |
          Yes, I do
        intent: affirm
      - action: newsletter_form
      - active_loop: null
      - action: utter_subscribed
Enter fullscreen mode Exit fullscreen mode

If you run the tests, they will all pass.

But if we add another test, testing both interruption during email and interruption during frequency, we will see it fail.

Failed test

We can see why it failed in results/failed_test_stories.yml.

# results/failed_test_stories.yml
version: "2.0"
stories:
- story: Newsletter form two interruptions (/var/folders/ng/z4dw_m9d1bz6m08vw7nlwwwh0000gq/T/tmpqlha244q/7bc3772cad9348a589e105ce91f29c74_test_stories.yml)
  steps:
  - intent: subscribe
  - action: newsletter_form
  - active_loop: newsletter_form
  - intent: chitchat
  - action: utter_ask_continue
  - intent: affirm
  - action: newsletter_form
  - slot_was_set:
    - email: example@email.com
  - intent: chitchat
  - action: utter_ask_continue
  - intent: affirm
  - action: newsletter_form  # predicted: action_two_stage_fallback
  - active_loop: null
  - action: utter_subscribed
Enter fullscreen mode Exit fullscreen mode

To fix this, start an interactive chat – rasa interactive.

Throughout the chat, I said what the correct actions shuld be. At the end of the chat, I chose to save this chat to data/stories.yml.
Let's inspect the story that has been created for us.

# data/stories.yml
# ... previous content ...
  - story: interactive_story_1
    steps:
      - intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - slot_was_set:
          - email: example@email.com
      - slot_was_set:
          - email: example@email.com
      - slot_was_set:
          - requested_slot: frequency
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: frequency
      - slot_was_set:
          - frequency: twice a week
      - slot_was_set:
          - requested_slot: null
      - active_loop: null
      - action: utter_subscribed
Enter fullscreen mode Exit fullscreen mode

Now let's look at new kind of steps that we have not been working with before.

This step is saying, that email is currently being requested:

      - slot_was_set:
          - requested_slot: email
Enter fullscreen mode Exit fullscreen mode

This step is saying, that email has been filled with value example@email.com. We do not need this step for our test to pass.

      - slot_was_set:
          - email: example@email.com
Enter fullscreen mode Exit fullscreen mode

This step is saying, that frequency has been filled with value twice a week. We do not need this step for our test to pass either.

      - slot_was_set:
          - frequency: twice a week
Enter fullscreen mode Exit fullscreen mode

This step means, that no more slots are being requested.

      - slot_was_set:
          - requested_slot: null
Enter fullscreen mode Exit fullscreen mode

You can learn more about requested_slot in my next article or in the documentation.

After droping the unnecessary steps, we get a bit simpler story.

# data/stories.yml
# ... previous content ...
  - story: interactive_story_1
    steps:
      - intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - slot_was_set:
          - requested_slot: frequency
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: frequency
      - active_loop: null
      - action: utter_subscribed
Enter fullscreen mode Exit fullscreen mode

I also reverted changes that have been made to domain.yml and data/nlu.yml and kept only the generated story.

If we retrain the model and run tests again, they will all pass now.

Passing tests

You can learn more about testing unhappy paths in the documentation.

In the next chapter, we will look at how to communicate with our chatbot through REST API.

Repository for this tutorial:

You can checkout the state of the repository at the end of this tutorial by running:

git clone --branch 06-testing-unhappy-paths git@github.com:petr7555/rasa-dev-tutorial.git
Enter fullscreen mode Exit fullscreen mode

Discussion (0)