Intro:
As Copilot Studio agents become more capable, a simple chat transcript is rarely enough to debug them. An agent may invoke skills, tools, connectors, MCP servers, knowledge sources, workflows, and file-processing steps before returning an answer.
When something goes wrong, developers need answers to practical questions:
- Which capability was invoked?
- Which step failed or was skipped?
- Did the tool succeed but the overall task remain incomplete?
- Was the requested output file actually generated?
- Can the run be correlated with Copilot Studio Observe traces or other platform telemetry?
A lightweight Conversation Run Log, created per conversation or run, gives developers a clear, shareable execution record. It does not replace platform telemetry—but it makes troubleshooting, handover, and quality reviews far more repeatable.
This article compares two useful logging patterns: a detailed execution trace and a concise capability-centred audit summary.
Method 1: Prompt-led plan and on-demand run log:
This is the quickest approach for development and testing. At the start of a conversation, ask the agent to create a plan and log the work as it goes.
For this conversation, please create a Plan markdown file and log all the steps so that I can test and export.
The agent creates a plan that describes the intended workflow—such as receiving a file, invoking a skill, checking outputs, creating a report, and returning artifacts.
At the end of testing, prompt the agent:
Conversation Run Log
or:
Create a Conversation Run Log markdown file for this conversation.
The agent then creates a Markdown summary based on the conversation context and any execution details available during that session.
Why it is useful
- No permanent agent instruction is required.
- It is ideal for ad hoc testing and demos.
- You can test whether the agent followed its plan.
- The plan provides a baseline for checking which steps were completed, skipped, or failed.
Limitation
This method is session-dependent. The agent can only reconstruct the run from context and telemetry it can still access in the current conversation. If the generated file was temporary and is no longer present in the runtime storage location, it may need to be regenerated before you leave the session.
_Practical tip: Download the Plan and Conversation Run Log while the session is still active. Do not assume a generated file will remain available after the runtime session ends or resets.
_
Method 2: Instruction-led audit logging:
This approach adds run-log expectations directly to the agent instructions. Instead of relying on the developer to remember the starter prompt each time, the agent is guided to capture a consistent audit record whenever it runs a relevant task.
Instruction snippet
### Run Log Requirements
Capture the following, where available:
- Conversation ID, session ID, run/trace/correlation ID
- Agent name, environment, and UTC start/end timestamps
- User request and relevant follow-up instructions
- Capabilities invoked: tools, MCP servers, skills, knowledge sources, actions, connectors, workflows, and files
- A concise record of each material step, including status and output summary
- Generated artifacts and their filenames
- Warnings, errors, limitations, and recovery guidance
- Final status: `Completed`, `Completed with warnings`, `Partial`, or `Failed`
Do not invent IDs, timestamps, telemetry, capability activity, or execution results. Use:
`Not available in current session`
for information not exposed by the platform.
Never include secrets, credentials, tokens, connection strings, or sensitive user data in the log.
If processing is incomplete or fails, return the log showing completed steps, the failure point, and what is needed to continue. Only confirm successful completion when the required capability execution and requested deliverables have actually completed.
Why it is useful
- Creates a consistent logging standard across conversations.
- Works well for agents with multiple skills, MCP tools, connectors, workflows, and knowledge sources.
- Encourages accurate reporting of partial execution.
- Helps support teams compare the agent’s run log with Copilot Studio Observe traces, where IDs are available.
- Explicitly prevents the agent from inventing unavailable telemetry.
Limitation
An instruction can guide the agent to produce a log, but it cannot guarantee that the platform exposes every internal execution detail. For example, conversation IDs, run IDs, tool telemetry, or precise timestamps may not be available to the agent at runtime.
It also does not automatically solve file persistence: if the Markdown file is generated in temporary runtime storage, it must be downloaded or exported while available.
| Area | Method 1: Prompt-led | Method 2: Instruction-led |
|---|---|---|
| Setup effort | Very low | One-time instruction setup |
| Best use | Testing, demos, troubleshooting a specific run | Reusable agents and repeatable support processes |
| Plan created upfront | Yes | Optional—add it if needed |
| Logging consistency | Depends on the developer prompt | More consistent |
| Session dependency | High | Still relevant for file export and available runtime context |
| Multi-tool/MCP agents | Works, but manual | Strong fit |
| Debug detail | Good if the plan is maintained | Good if the instruction requires material steps |
| Best outcome | Plan + final run log | Automatic audit habit + final run log |
Take away:
Use a plan to test intent, use a run log to prove execution, and export both before the runtime session ends.
Top comments (0)