I added Rasa Mantle to my VS Code extension: tool calling, memory and LLMs

I keep my project notes inside VS Code with an extension I built called NoteVs. For the Rasa Heroes program I gave it a conversational agent built on Rasa Mantle, Rasa's new agent engine. Now I can ask "what did I annotate in the auth module?" or "remind me about the refactor note tomorrow" from a chat panel next to my code.

This isn't really a post about notes, though. It covers the three parts of Mantle I spent the most time with: tool calling, memory, and the LLM layer. That includes running it on Groq, which Rasa doesn't list as a named provider. I'll cover what each part does, how I used it, and what broke.

The setup in 30 seconds

Everything runs locally. The VS Code extension owns the notes and exposes a tiny HTTP endpoint on localhost:37492. The agent is a local Rasa server whose tools call that endpoint. The extension's chat panel talks to Rasa over its REST channel.

rasa-notevs-agent/
├─ agent.yml              ← persona + global rules
├─ skills/*/skill.md      ← 4 skills
├─ tools/notevs_tools.py  ← 10 tools
├─ memory.yml             ← project memory
└─ integrations.yml       ← LLM + channels

I first built this on classic CALM: flows, slots and slot mappings, about 590 lines of YAML across five files. The Mantle version is about 210 lines, mostly plain English. Here's where that difference comes from.

1. Tool calling

Tools are typed Python functions

A Mantle tool is an async function with a decorator. The type hints and description are what the LLM sees.

from rasa.calm_v2.tools.decorator import ToolContext, tool
from rasa.calm_v2.tools.result import ToolResult

@tool(description="Search notes by keyword across title, content, tags, and annotation comments.")
async def search_notes(query: str, context: ToolContext = None) -> ToolResult:
    return await _call("notevs_search_notes", {"query": query}, context)

All 10 of my tools are thin wrappers like this around one helper that POSTs to the extension. The most useful decision was making failures readable instead of raising:

except httpx.HTTPError as exc:
    return ToolResult(llm_response={"ok": False, "error": f"notevs_extension_unreachable: {exc}"})

The LLM sees notevs_extension_unreachable and the skill tells it what to say: "NoteVs doesn't seem to be running." The user never sees a stack trace.

Pass what a tool needs as an argument

My worst bug lived here. With several VS Code windows open, every window's agent was reading the same project. The extension's HTTP server uses a fixed port, so only one window wins it, and that window answered every tool call. I found out when an agent listed notes from a completely different project.

The fix was for the helper to send the project folder explicitly as an argument on every call, taken from an env var each window sets for its own agent. If a tool needs a value to be correct, pass it in. Don't rely on the tool's surroundings to supply it.

Skills decide which tools are in play

A skill is a skill.md file. Front matter lists the tools it can use, and the body is instructions. Each skill only imports what it needs, so the model never picks from all 10 tools at once:

---
name: Note Management
import_tools:
  - create_note
  - save_note
  - add_annotation
  - delete_note
  - list_notes
  - search_notes
  - get_note
tool_constraints:
  - delete_note:
      requires_confirmation:
        enabled: true
---

If the user references a note by title only, look it up first with
`search_notes` or `list_notes` to get its real id ... never invent an id.

Confirmation is enforced by the framework

That tool_constraints block replaced an entire classic-CALM flow: collect the id, collect the title, collect a yes/no, branch, call, reset slots. In Mantle it's four lines. Deletes, reminders and Notion/Obsidian exports can't run until the user confirms, whatever the model decides. Read-only tools have no constraint, so "find my bug notes and show me the newest" can chain search → get in one turn.

What broke: the very first message of every conversation got swallowed. Mantle's built-in session-start skill greets the user and ends the turn, so "hey, can you list my notes" got a hello and no tool call. I overrode it with a skill that does nothing:

---
name: Session Start
routing:
  engine_managed: true
---
:::ordered_block id=main
steps:
  - id: skip
    noop: true
    next: END
:::

After that, the first message goes straight to the right skill and its tool.

2. Memory

Mantle memory is the set of values a skill keeps track of during a conversation. Every entry has to be declared in a memory.yml first, and tools write to it with context.memory.set(...). The LLM can only write entries you mark llm_settable, so it can't make up something like a verification result. Values shared between skills go in a project-level memory.yml, and instructions can branch on a value (if: session.project.<entry> == ...).

I only used it lightly. My project memory has one field, the workspace folder:

folder_path:
  type: text
  description: "Absolute path of the VS Code workspace this conversation is scoped to."

The tools copy the folder path into it, so I can see which project a conversation belongs to in rasa inspect. Project fields lock once they're set, which suits a value that never changes during a session.

Where I'd take it next: remembering the note we're talking about, so "delete that one" two turns later doesn't need another search, and using memory as a skill precondition, so exports only start once an integration is connected. If you want to try it, the memory docs cover the full schema.

3. The LLM layer (and running on Groq)

The model is set in integrations.yml. OpenAI and Anthropic are Rasa's named providers, but I wanted Groq for its speed in a chat panel. Groq isn't on the list.

Before giving up, I read Rasa's provider mapping in the installed package (rasa/shared/providers/mappings.py). Unnamed providers go through LiteLLM, which supports Groq, so this works:

llm:
  provider: groq
  model: qwen/qwen3.8-27b
  api_key_env: GROQ_API_KEY
  reasoning_effort: none

Getting to that config took three model changes:

1. openai/gpt-oss-120b failed Mantle's strict tool-schema validation twice in one session. Once it called a tool that doesn't exist, and once it tried to activate a skill id that wasn't in the allowed list. Both were real 400s that killed the turn.
2. llama-3.3-70b-versatile was listed in Groq's docs as supporting tool calls, but a live GET /openai/v1/models showed my account never had access to it.
3. Qwen was the first to complete a full activate → search_notes → result turn correctly. It thinks by default, though, and its reasoning leaked into chat replies, not always inside <think> tags. Setting reasoning_effort: none turns thinking mode off instead of hiding it. A month later Groq replaced qwen3.6-27b with qwen3.8-27b and I had to re-pin.

Lessons:

• An agent whose whole job is tool calling needs a model that is genuinely good at tool calling. Speed doesn't matter if the turn dies on a schema error.
• Make the provider swappable. Since it all goes through LiteLLM, I added a small script that rewrites the llm: block at startup from an LLM_PROVIDER env var. Groq, OpenAI and Anthropic are now each one setting away. Users of the extension pick one in Settings.

The catch: I'm on Groq's free tier

I run this on a free Groq API key, and you can see what that costs in the screenshots above. In the second one, I ask the agent to delete "Note of this". It asks me to confirm, I say "Yeah go ahead", and the note disappears from the sidebar. Then I ask "is it removed?" and get no answer, just the panel's fallback message. Groq had rate-limited the request; the extension's Output channel shows the response with retry-after: 9.

The delete worked. Only the LLM's next reply failed. On a free key, a few quick messages in a row can hit the limit, and the agent then misses a reply or answers late. I think the same limit explains why the first screenshot needed an "are you done?" nudge before the list came back.

That's a limit of the free tier, not of Mantle. A paid Groq key, or switching LLM_PROVIDER to OpenAI or Anthropic, removes it. NoteVs itself doesn't need an LLM at all: notes, annotations and search all work from the sidebar. The agent is an optional layer on top.

One config trap

With inspector.enabled: true in integrations.yml, rasa run printed "Rasa server is up and running" and then crashed with EADDRINUSE. The inspector is for the separate rasa inspect command. Under rasa run it tries to bind the same port the REST channel just took. I keep it off in the running config and use rasa inspect --debug when I want a turn-by-turn trace of every LLM and tool call, which is how I found most of the bugs above.

Try it

• NoteVs on the VS Code Marketplace. Add your LLM and Rasa keys in Settings - Rasa Mantle You can find the docs here.
• Build your own starting at rasa.com with a free Developer Edition licence and the examples in rasa-community-resources.

A note on versions: this is all on a pre-release build (rasa-pro==3.19.0.dev5). Pin your version and expect things to move.

This was built for Rasa Heroes, Rasa's community program for developers building with the new Rasa. If you're building with Mantle, especially on a provider other than OpenAI or Anthropic, I'd like to hear how it's going in the comments.