DEV Community: Bala Madhusoodhanan

Using Custom Metrics to Improve Agentic Experiences

Bala Madhusoodhanan — Mon, 05 Oct 2026 08:07:11 +0000

Intro:
Custom metrics in Microsoft Copilot Studio help teams understand the quality and outcomes of agent conversations—not simply how many conversations take place. They enable process owners and business teams to track the operational signals that matter for their particular agentic use case, such as whether users receive a resolved answer, where essential information is missing, when source content is insufficient, or when a request needs to move to a different support channel.

As outlined in Microsoft’s Custom metrics documentation, metrics can be configured using natural language. The metric owner describes the outcome they want to measure across conversations, then defines the result categories that the agent should identify. This makes the feature accessible to business and process teams, without requiring them to create a complex reporting data model or introduce manual tagging of every conversation.

Where custom metrics add value
For agentic use cases, standard analytics—such as number of sessions, escalation rate, or engagement—show activity, but may not explain why an agent did or did not help the user. Custom metrics add that business context.
They can help a process team or business owner understand:

Resolution quality: whether users receive a supported, complete response.
Information gaps: whether the agent needs essential facts before it can proceed.
Knowledge and content gaps: whether approved sources contain sufficient information to answer the question.
Data-quality issues: whether a lookup, mapping, or reference data prevents a tailored answer.
Demand for unsupported services: whether users expect the agent to carry out actions outside its current scope.
Improvement priorities: whether the right intervention is a conversation-design change, content update, data correction, integration enhancement, or clearer user guidance.

The important point is that each metric should answer one practical operational question. Categories should be clear, distinct, and based on outcomes that can be recognised in the conversation.

Example: Travel & Expense policy assistant

A Travel & Expense (T&E) policy assistant provides a useful example. Its role is to answer employee questions about T&E policy using approved policy documentation. For personal policy guidance, the assistant may need to establish the employee’s employment or base country, validate the applicable policy route, retrieve the relevant evidence, and determine whether the request is a policy question or an operational request such as making a booking.

This means that an apparently simple question—such as “Can I travel business class?”—may have several possible outcomes:

The assistant can provide a grounded answer from the relevant policy.
It needs a missing fact, such as the employee’s base country or band.
The country-to-policy mapping cannot be validated.

The policy documentation does not adequately answer the question.
The user is requesting an action that needs a service desk or booking channel.

Example Metric 1: Context and Routing Readiness

Element	Definition
Purpose	Identifies whether the assistant can establish the context needed to give a tailored answer.
What do you want to measure?	Whether personal policy questions can be successfully routed to the correct policy scope. Measure whether the user provides the necessary context and whether the required policy mapping or reference-data lookup is successfully validated, unavailable, disabled, duplicated, malformed, or missing.

Result category	Definition
Successfully routed	The conversation concerns a personal policy question and the assistant confirms that the required context and valid policy route are available, allowing it to provide an appropriately tailored answer.
Essential context needed	The conversation concerns a personal policy question, but the user has not provided a necessary fact required to determine their applicable policy or entitlement. The assistant asks for that fact.
Routing or reference-data issue	The user provides the required context, but the assistant cannot validate the applicable route because the required lookup or mapping is missing, disabled, duplicated, incomplete, malformed, unavailable, or returns an error.
General information request	The user asks a general policy or document-information question that does not require personal context or tailored policy routing.

Example Metric 2: Answer Resolution Outcome

Element	Definition
Purpose	Shows whether the assistant resolved the substantive user question and, where it did not, identifies the reason.
What do you want to measure?	The outcome of substantive user questions. Classify whether the assistant provides a grounded answer using applicable approved sources, needs an essential clarification, cannot answer because relevant source content is missing or incomplete, or directs the user to an appropriate operational or human-support channel for requests outside its capability.

Result category	Definition
Resolved with approved evidence	The assistant answers the substantive question using applicable approved source content, including material conditions, approvals, limitations, or citations where available.
Clarification required	The assistant cannot make a case-specific determination because an essential fact is missing. It asks a concise question for the information required to continue.
Knowledge or evidence gap	The assistant has sufficient user context, or the question is general, but cannot establish an answer because the relevant approved source content is missing, incomplete, unavailable, conflicting, or does not address the requested point.
Operational or human support needed	The user requests an action outside the assistant’s capability, or requires human assistance. The assistant directs the user to the appropriate approved support route.

What this tells the business owner

In the T&E example, the desired outcome is an increasing proportion of Resolved with approved evidence. The remaining categories indicate where improvement should be focused:

Metric outcome	Likely focus area	Example in the T&E agent
Clarification required	Conversation design and user guidance	Explain earlier that country or HR band may be needed
Knowledge or evidence gap	Policy content and knowledge governance	Update a missing local policy rule or clarify conflicting guidance
Operational or human support needed	Service design and integration roadmap	Improve booking or service-desk hand-off; assess future integration
Routing or reference-data issue	Data ownership and technical operations	Correct country-policy mappings or resolve lookup access issues

Closing remarks:

Custom metrics help process teams shift from asking, “Is the agent being used?” to asking, “Is the agent producing the intended business outcome, and what prevents it when it does not?”

The T&E policy assistant is only one example. The same pattern can be applied to procurement, order-to-cash, supply-chain, HR, IT support, customer service, or any agent that depends on user context, business rules, enterprise knowledge, and hand-offs to other services.

As the feature is currently in preview, custom metrics initially display a rolling seven-day view because the analysis is computationally intensive. Once sufficient data has been collected, and hopefully have a view expanding to show up to 90 days of metric history.

Teams should begin with a weekly review cadence, monitor meaningful changes in the category mix, and assign improvement actions to the appropriate owner—such as the agent owner, process team, policy/content owner, data steward, or service owner.

Important: Custom metrics are calculated from the conversation transcript. The outcome described in “What do you want to measure?” must therefore be visible or clearly evidenced in the conversation—for example, through the user’s request, the agent’s response, or an explicit outcome such as a resolved answer, clarification request, routing failure, or escalation.

From Career Pivot to Credible Candidate: Work IQ for Interview Stories and Your 90-Day Plan

Bala Madhusoodhanan — Mon, 14 Sep 2026 04:49:05 +0000

Intro:
A career pivot becomes real when a hiring manager looks at you and asks:

“Tell me about a time you did that.....”

This is the point where a well-written CV is no longer enough. You need evidence. You need stories. You need to be able to explain how your previous experience prepares you to solve complex problems in a completely new context.

In the first two articles in this series, we explored how to use Microsoft 365 Copilot and Work IQ to discover transferable evidence in your authorised work context, and how to turn that evidence into a stronger CV and LinkedIn profile.

Now comes the final step: proving that your experience is relevant, and being honest about what you still need to learn.

Interviews are Evidence Tests
Hiring managers are not just listening for keywords. They are trying to assess whether you can:

Learn quickly
Lead through ambiguity
Influence stakeholders
Make good decisions
Deliver outcomes
Adapt your expertise to a new environment

That is why a credible adjacent candidate—someone who can clearly articulate how their past wins apply to future challenges—can often outperform a candidate with a familiar title but weak evidence.

Turn Work Evidence into STAR Stories
The traditional STAR method is familiar:

Situation — What was happening?
Task — What were you responsible for?
Action — What did you do?
Result — What changed?

But career switchers and adjacent candidates must add one more critical dimension: Transferability.

Act as the hiring manager for the target role.

Use the Work IQ experience in Microsoft 365 Copilot to identify the most relevant examples from my authorised work context, including key projects, complex decisions, stakeholder alignments, and delivery outcomes.

Prioritise the items I attach with /, including:

/Current_CV.docx

/Target_Job_Description.docx

/[Insert Major Project Summary/Review Document here]

Based on the target role and my experience, generate the 15 interview questions I am most likely to face.

For each question, provide:

What the interviewer is actually assessing

The most relevant evidence from my work context

A tailored STAR answer based only on documented evidence

The transferable capability demonstrated

A likely follow-up question

A common mistake to avoid

Where my experience is adjacent rather than direct, help me position it honestly and confidently without overstating my expertise in areas I am still developing.

Example: An Adjacent-Candidate Answer
Interview question: “Tell me about a time you led the adoption of a new framework or operating model.”

Weak answer: “I am very passionate about transformation and have attended many sessions on enterprise change management.”

Stronger evidence-based answer:
“During a global integration initiative, I helped align disparate regional stakeholders around a unified process model. The challenge was not just the process itself, but building agreement on governance and adoption across teams with competing priorities. I facilitated the strategic discussions, translated localized concerns into global design decisions, and helped establish a repeatable governance approach. The result was a clearer decision path and a 40% drop in escalation times. While the specific methodology was different from what you use here, the core leadership challenge is exactly the same: creating trust, enforcing governance, and delivering practical business value through change.”

Run a Brutally Honest Recruiter Audit
The most powerful part of using Copilot and Work IQ for career mapping is not the flattering output. It is the honest output. You need to know, objectively, whether you are:

A direct match
A credible adjacent candidate
A career switcher who needs to build more evidence first

None of these categories are "bad." They simply require different strategies.

Prompt: Work IQ-Powered Career Transition Audit
Act as a brutally honest senior recruiter, hiring manager, and executive career coach.

Review my CV against the target job description. Then use the Work IQ experience in Microsoft 365 Copilot to identify relevant evidence from my authorised work context that is missing from, or underrepresented in, my CV.

Use the items I attach with / as priority evidence.

Score my candidacy from 1–100 across:

ATS compatibility

Role relevance

Achievement strength

Leadership evidence

Keyword alignment

Clarity of career-transition narrative

Quality of available work evidence

Then provide:

The top reasons I might be rejected

Specific actions to address each risk

Evidence from my work context that should be added to my CV

Gaps that can be addressed through better positioning

Gaps that require genuine development

Finally, classify me as a career switcher, a credible adjacent candidate, or a direct match. Explain the classification using evidence, not encouragement alone.

An honest gap is not a reason to stop. It is a reason to create a plan. The key is not just completing generic courses; the key is building visible proof that connects learning to outcomes.

Period	Goal	Practical Actions
Days 1–30	Build foundation	Complete targeted learning on the target domain; follow internal communities; map new concepts to your existing expertise.
Days 31–60	Create evidence	Join a cross-functional initiative, support a prototype, document a relevant use case, or contribute to governance thinking in the target area.
Days 61–90	Make growth visible	Share learnings internally, run a community session, document a new framework, or update your CV/LinkedIn with this newly built evidence.

Prompt: Create a Personalised 90-Day Plan:

Act as an executive career coach.

Based on my target role, my current CV, the Work IQ-supported assessment of my existing work evidence, and the genuine gaps identified in my recruiter audit, create a realistic 90-day development plan.

Structure it across:

Days 1–30: Foundation

Days 31–60: Applied practice

Days 61–90: Visible proof and positioning

For each action, provide:

the capability it develops;

why it matters for the target role;

a practical deliverable or evidence artefact;

an estimated weekly time commitment; and

how I can demonstrate progress on my CV, LinkedIn, or in an interview.

Prioritise a small number of high-value actions. Do not recommend generic learning without a practical outcome

Confidence Comes From Evidence:

Career transitions are rarely linear. Your current title might describe where you have been, but the evidence embedded in your daily work reveals where you are heading.

Work IQ does not choose your next career move. It helps you see the evidence already forming beneath it.

From Work Evidence to Career Narrative: Using Work IQ to Strengthen Your CV and LinkedIn

Bala Madhusoodhanan — Mon, 07 Sep 2026 05:12:13 +0000

Intro:
In the first article in this series, we explored a career question many of us face:

“Am I really changing careers—or am I already building toward an adjacent role?”

The answer is rarely found in a job title alone. Once you have used Microsoft 365 Copilot and Work IQ to identify credible evidence from your authorised work context, the next challenge is more practical:

“How do I translate that evidence into a CV and LinkedIn profile that makes sense to a recruiter?”

This is where many career pivots fail. People often make one of two mistakes:

They undersell themselves by describing only their old domain-specific responsibilities.
They oversell themselves by adding fashionable keywords they cannot defend in an interview.

The better approach is evidence-based positioning. Work IQ can help Copilot find relevant work evidence from permitted organisational context, while / references allow you to prioritise specific source material.

The Career Narrative Formula
A credible career-transition story has three parts:

For example:
“I am a delivery and transformation leader with deep operational expertise. My transferable strengths are enterprise governance, platform strategy, stakeholder alignment, and scaling adoption. I am now applying those strengths to lead cross-functional strategic initiatives.”

That is very different from simply claiming you are a "Strategy Executive" if you do not have the title. The first statement is credible because it connects real experience to a deliberate future direction.

Step 1: Rewrite CV Bullets with Evidence—Not Memory
Most CV bullets start with tasks:

Managed a project
Supported implementation
Participated in workshops
Led governance meetings

These statements are not wrong, but they do not tell a recruiter what changed because of your work. A stronger structure is: Action → Task → Result → Business Impact

Prompt: Rewrite CV Bullets Using Work IQ Evidence

Act as a hiring manager for the target role.

Use the Work IQ experience in Microsoft 365 Copilot to find relevant evidence from my authorised work context, including project documents, meeting recaps, emails, Teams conversations, and available business metrics.

Prioritise the files I attach with /, including:

/Current_CV.docx

/Target_Job_Description.docx

/[Insert Major Project/Review Document here]

Rewrite my CV bullet points using this structure: Action → Task → Result → Business Impact

For every rewritten bullet:

use only evidence found in the referenced items or authorised work context;

never invent metrics, technologies, outcomes, or leadership scope;

identify the evidence source;

emphasise transferable capabilities directly relevant to the target role (e.g., cross-functional leadership, specific methodologies, stakeholder influence, or technical acumen);

flag any bullet where the evidence is too weak to make a credible claim.

Return the output in a table: Original CV bullet | Evidence found | Rewritten CV bullet | Relevance to target role

Original CV bullet	Rewritten CV bullet	Relevance to target role
Managed system integration project	Led cross-functional delivery for a core transformation initiative, aligning distinct departments to establish a scalable operating model and resolving resource bottlenecks.	Demonstrates cross-functional leadership, transformation delivery, and stakeholder alignment.
Supported governance meetings	Governed strategic vendor relationships and internal compliance discussions, defining strict performance metrics that improved service delivery and accountability.	Demonstrates enterprise governance, risk management, and service level oversight.

Notice what this does not do: It does not invent a percentage saving, and it does not replace domain expertise with vague buzzwords. It simply makes the transferable value visible.

Step 2: Find Missing Keywords—But Do Not Stuff Them In
A target job description might ask for capabilities like roadmap development, change management, or platform thinking. Your CV might not contain those exact terms, but that does not automatically mean you lack the capability.

Step 3: Build an Authentic Professional Summary
A professional summary is not a list of generic claims. It should answer:

Who are you now?
What value do you bring?
Why does your experience transfer?
What direction are you moving toward?

Stakeholder feedback can be particularly valuable here because it captures how other people experience your contribution.

Prompt: Create a Professional Summary Using Work IQ Evidence

Act as a recruiter, hiring manager, and executive career coach.

Use the Work IQ experience in Microsoft 365 Copilot to review my authorised work context for evidence of leadership impact, stakeholder influence, innovation, cross-functional collaboration, technology adoption, delivery outcomes, and feedback from colleagues and leaders.

Use the items I attach with / as priority sources:

/Current_CV.docx

/Target_Job_Description.docx

/Performance_Review.docx

selected stakeholder feedback or recognition emails

Write five professional CV summaries tailored to the target role.

Each summary must:

be 3–4 sentences;

position me accurately as a career switcher, credible adjacent candidate, or direct match;

reflect transferable strengths supported by evidence;

avoid vague phrases such as “hardworking,” “results-driven,” or “team player”;

avoid claiming experience that is not evidenced.

After each summary, explain the career narrative it uses and the evidence supporting it.

Example Summary:
"Transformation and delivery leader with experience translating complex operational challenges into scalable, governed solutions. Brings strengths in strategic planning, global stakeholder alignment, platform governance, and innovation delivery. Positioned to help organisations bridge the gap between strategic intent and practical, scalable adoption."

Step 4: Translate Your Story to LinkedIn
Your LinkedIn profile should not simply copy your CV. Your CV is designed for a specific role, while your LinkedIn profile should make your direction understandable to a broader audience of recruiters, hiring managers, and professional communities.

Prompt: Build a LinkedIn Profile with Work IQ-Supported Evidence

Act as a LinkedIn personal-branding strategist and recruiter.

Using my target role, current CV, and evidence from my authorised work context available through the Work IQ experience in Microsoft 365 Copilot, create an accurate LinkedIn profile that positions me for my target career move.

Use the items I attach with / as priority sources. Identify evidence of enterprise leadership, cross-functional collaboration, transformation, governance, stakeholder influence, and continuous learning.

Then rewrite my:
LinkedIn headline
About section
Experience descriptions
Skills section
Make the language natural, credible, and recruiter-search-friendly. Do not overstate expertise if my evidence is primarily in strategy, governance, or adoption.

Example output
"Enterprise Transformation Leader | Driving Scalable Delivery, Strategic Governance & Cross-Functional Innovation"

This does not deny your current domain expertise. It simply makes the next chapter visible.

Your Next Career Move May Already Be in Your Work: Discovering Career Evidence with Work IQ

Bala Madhusoodhanan — Mon, 31 Aug 2026 07:24:24 +0000

Intro:
You see a target role that feels like a leap. Your current experience is rooted in one specific domain, but the new role asks for a blend of strategy, leadership, and cross-functional capabilities that seem just out of reach.

At first glance, the instinct is familiar to many of us:
“My current job title does not match. I am not qualified.”

But that is the wrong question to ask. Instead of asking if your current title matches the role, the better question is:

“What capabilities is this role actually hiring for—and what evidence already exists in my work?”

That is where Microsoft 365 Copilot and Work IQ become surprisingly useful. Work IQ helps Copilot use authorised work context—such as relevant files, emails, meetings, chats, people, and work relationships—to provide a grounded response. Using / references, you can focus Copilot on specific priority material, such as a performance review, a major project wrap-up, and the target job description.

The Problem with Traditional Career Planning
Most of us assess our career readiness through static artefacts:

Our job title
Our current CV
An organisational chart
A formal job description

These documents are useful, but they are fundamentally incomplete. They show what you have historically been hired to do. They rarely capture:

What complex problems people increasingly bring to you
How you manage ambiguity and align stakeholders
The cross-functional projects you are driving
The leadership capabilities you are developing before a promotion formally recognises them

Your next role may be visible in your day-to-day work patterns long before it makes it onto your CV. Job titles are just labels; capabilities are evidence.

A senior or stretch role is rarely hiring just for domain expertise. It is looking for individuals who can define roadmaps, govern risk, translate strategy into delivery, and move an organisation from experimentation to scale. Those capabilities exist across many roles, even if the job titles look completely different.

The goal of this exercise is not to use Copilot to artificially convince yourself you are ready. The goal is to objectively separate your background into three categories:

Demonstrated experience: Capabilities you have explicitly executed and delivered.
Adjacent experience: Work you have done that is highly comparable and transfers directly to the new context.
Development gaps: Genuine areas where you lack evidence and need to build skills.

Prompt: Decode the Role and Map Your Evidence
Use this prompt as the first step in your career transition journey.

Act as a recruiter and career coach.

Analyse the target job description attached below and identify:

Core responsibilities

Hiring goals and business outcomes

Technical, strategic, and leadership capabilities being assessed

Important ATS keywords

The characteristics of a successful candidate

Then, use the Work IQ experience in Microsoft 365 Copilot to analyse my authorised work context for relevant evidence. Consider relevant work files, meetings, emails, Teams conversations, project activity, and stakeholder interactions that I am permitted to access.

Use the items I attach with / as priority evidence.

Create a table with these columns:

Target capability (from the JD)

Evidence from my work context

Source or cited work item

Why the evidence is transferable

Classification: demonstrated experience, adjacent experience, or development gap

Do not invent experience or outcomes. Clearly distinguish documented evidence from reasonable inference. Be honest about gaps.

What to Attach Using / References
To get the best results, start small and specific. Good first attachments include:

Your current CV
The target job description
Your latest performance review
A major project summary, business case, or closure presentation
A documented stakeholder feedback email

Decoding the Output
When you run this prompt, the output shifts your perspective from "Do I have the title?" to "Do I have the proof?" Here is an example of what that output might look like regardless of your specific industry:

Target Capability	Evidence from Existing Work Context	Classification
Cross-Functional Leadership	Chaired the weekly steering committee, aligning three distinct departments on the Q3 delivery schedule and resolving resource bottlenecks.	Demonstrated experience
Strategic Roadmapping	Authored the business case and phased rollout plan for the internal tool migration, translating user pain points into a delivery timeline.	Adjacent experience
Enterprise Governance	Established the risk review matrix used by the wider team to evaluate third-party vendors before onboarding.	Demonstrated experience
Executive Communication	Drafted monthly status briefings for the VP group, synthesising complex project metrics into clear, decision-ready updates.	Demonstrated experience
Specific Platform Expertise	Limited evidence of direct hands-on configuration or certification in the target proprietary software.	Development gap

Using Work IQ Responsibly
A career mapping exercise should be handled carefully when using enterprise tools:

Use only information you are authorised to access.
Do not use confidential work content (like financial data or proprietary architectures) in external job applications.
Verify all evidence and citations before relying on them.
Treat Copilot as a reflection and drafting partner, not the final decision-maker.

Choose one stretch role that interests you this week. Do not start by rewriting your CV. Start by asking what capabilities the role actually requires, and let your existing work provide the evidence. In the next article, we will take the evidence you uncover and turn it into a compelling CV and LinkedIn narrative.

Designing Intelligence: From Ad-Hoc Prompts to Enterprise AI Skills

Bala Madhusoodhanan — Mon, 24 Aug 2026 05:31:33 +0000

Intro:

The pattern now as part of agentic design is, we are witnessing a shift from treating Large Language Models purely as conversational interfaces to integrating them as governed architectural components. For developers using platforms like Microsoft Copilot Studio, understanding the transition from an ephemeral prompt to a persistent AI skill is critical for building scalable, compliant solutions.

Ephemeral Prompts vs. Persistent Skills:

A prompt is how you get started; a skill is how your team scales. Prompts are reactive, natural language instructions used for one-off tasks.
Context is completely forgotten when the session ends. Conversely, an AI Skill in Copilot Studio is a reusable, persistent workflow package—often defined as a Markdown file with YAML metadata—that loads on demand. A skill functions as an auditable microservice with standing system instructions, strict operational boundaries, and embedded operating logic.

Dimension	Prompts	Enterprise AI Skills
Use Case	One-off, exploratory tasks.	Executing standardized, highly repeatable processes safely.
Composition	Pure natural language prose lacking boundaries.	Hybrid: Markdown instructions combined with executable scripts.
Governance	Unmonitored compliance gaps.	Auditable assets enabling Risk Management and Human Oversight.

Not every task warrants becoming a skill. Unchecked "skill sprawl" leads to heavy maintenance debt and unnecessary token consumption. Use this decision tree to evaluate whether an automation request requires a simple prompt action or a formal enterprise skill.

Core Design Principles for Copilot Studio
To maintain a clean architecture, evaluate potential skills against these core system design principles:

The Rule of Two: If a prompt is typed more than twice for a team workflow, formalize it into a skill to guarantee output consistency.
Hybrid Execution Logic: Separate probabilistic reasoning from deterministic execution. Let the LLM handle context while Python scripts, APIs, or Power Automate flows execute rigid calculations.
Carbon-Aware Routing: When a workflow carries high regulatory demands, a managed skill is non-negotiable. To offset the computing footprint of persistent skills, deliberately route mechanical tasks to less energy-intensive models or execute batch operations during periods of lower grid carbon intensity.
Context Management: Design skills with highly optimized descriptions. Copilot Studio uses this metadata to decide when to dynamically pull the full skill instructions into the context window, reducing token cost and latency.
Traceability for Compliance: Align with frameworks like the EU AI Act by designing skills that enforce programmatic safety boundaries, telemetry, and human-in-the-loop triggers before execution.

Closing Thoughts: The Trap of "Easy"

As platforms like Copilot Studio lower the barrier to entry, building an AI Skill has never been easier. But just because you can build a skill does not mean you should.

The ease of low-code development often tempts citizen developers into "skill sprawl"—productizing every minor prompt into a persistent agentic workflow simply because the platform makes it simple to do so. This is an architectural trap. Not every task requires a governed microservice. As Mikko Koskinen’s real-world cost testing brilliantly highlights, defaulting to premium orchestrated agents for simple tasks can rapidly multiply your operational expenditure through stacked message consumption.

True system design is about restraint. Ad-hoc prompts remain a perfectly valid, highly cost-effective tool for daily friction, personal productivity, and exploratory work. Reserve the rigorous engineering of a persistent Skill for processes that truly demand strict boundaries, regulatory compliance, and repeatable, team-wide accuracy.

Unshakeable - Ancient Wisdom for Modern Resilience

Bala Madhusoodhanan — Mon, 17 Aug 2026 07:30:52 +0000

For those who have read my book, Ctrl+Alt+Dharma: Ethical AI through an Ancient Lens), you know I spent years researching how ancient Indian philosophical frameworks—specifically the Bhagavad Gita—can serve as a moral compass for modern artificial intelligence. In enterprise architecture, we use these ancient truths to mitigate algorithmic bias, design ethical workflows, and ensure our technology serves humanity.

Where my work focuses on building a resilient target architecture for enterprise AI, the author of this book Vivek has written the ultimate field guide for building a resilient target architecture for the human mind.

What makes Unshakeable so brilliant is its practicality. It doesn't just offer philosophical platitudes; every chapter ends with a highly specific "Move" to put the concept into immediate practice.

Whether you are a tech professional trying to avoid burnout, a leader looking to build a more resilient team, or simply someone trying to find your own "Dharma" in a chaotic world, this book is a mandatory read. It is a powerful reminder that before we can build intelligent, ethical systems, we must first build unshakeable minds.

Unshakeable

Have you read it yet? Let me know your thoughts in the comments, or share what ancient philosophies you rely on to navigate the modern world!

From Clicks to Documentation: Converting D365 Task Recordings into L4 Guides

Bala Madhusoodhanan — Mon, 10 Aug 2026 08:15:39 +0000

Intro:
Documenting business processes in Dynamics 365 can be time-consuming. Functional consultants and process owners often need to turn Task Recorder recordings into step-by-step user guides, training material, and Level 4 (L4) work instructions.
The challenge is that a Task Recorder file captures the system interaction, but it is not always ready to share as a polished business document.
To help with this, I created the axtr-to-markdown skill. It converts Microsoft Dynamics 365 Task Recorder recordings into structured Markdown documents that can be reviewed, enriched, published, or used as knowledge sources for Copilot Studio agents.

What is the axtr-to-markdown skill?
The axtr-to-markdown skill takes a Dynamics 365 Task Recorder file and transforms the recorded process into a structured Level 4 process training document.
It extracts available process information such as:

Process title and description
Recorded user steps
Form and screen names
Field or control names
Recorded values
Action types such as click, input, select, navigate, validate, save, and post

The output is a Markdown file that provides a much better starting point for a user manual than a raw Task Recorder export.
Key Features

AXTR to Markdown conversion Converts Dynamics 365 Task Recorder content into an editable Markdown document.
Level 3 and Level 4 support Generates either a concise SOP-style L3 document or a detailed click-by-click L4 work instruction.
Structured process documentation Includes purpose, scope, prerequisites, roles and responsibilities, detailed steps, expected outcomes, troubleshooting, related documents, and revision history.
SIPOC analysis draft Produces a draft Suppliers, Inputs, Process, Outputs, and Customers overview based on the available recording data.
Metadata options Add a custom document title, author, department, process level, and output location.
Copilot Studio-ready output Markdown documents can be used as a starting point for SharePoint knowledge articles, controlled documentation, or agent knowledge sources.

Why AXTR files are useful for L4 documentation
A Level 4 document focuses on the exact way a user performs a task in the system.
Dynamics 365 Task Recorder captures those practical details while a user completes the process. This makes AXTR-based documentation particularly useful for:

Standard operating procedures
End-user training guides
Onboarding documentation
Process harmonisation
UAT test-script drafting
Regression-test documentation
Support and issue-reproduction steps

Instead of starting with a blank Word document and manually transcribing every click, teams can use the recording as the baseline and focus their time on validating the business context, controls, exceptions, and approvals.

axtr-to-markdown

axtr-to-markdown/
├── scripts/
│   ├── axtr_to_markdown.py
│   └── helpers.py
├── references/
├── assets/
└── SKILL.md

The axtr-to-markdown skill does not attempt to replace process experts. Instead, it removes the repetitive first step of documenting every click and screen interaction manually.
It turns a Dynamics 365 Task Recorder output into an editable Markdown draft that functional consultants, business users, and process owners can review and enrich. The result is a faster route from recorded system activity to a maintainable L4 work instruction.

Build Repeatable AI Workflows with SharePoint Skills

Bala Madhusoodhanan — Mon, 03 Aug 2026 06:56:52 +0000

Many SharePoint tasks are repetitive: checking whether a project folder contains the right files, reviewing documents against a checklist, or finding knowledge articles that are overdue for an update.

SharePoint Skills are designed for exactly this kind of work.
A Skill is a reusable, multi-step AI workflow that runs inside SharePoint. Rather than writing code or building a conventional automation flow, you describe the outcome you want in natural language, then review and refine the Skill SharePoint generates.

You can also manage Skills directly from the SharePoint site’s Agent Assets library. Navigate to:

SharePoint site → Site contents → Agent Assets → Skills

From the Skills folder, you can manually upload a Skill file or organize existing Skills alongside your team’s other agent resources. This is useful when you want to reuse a Skill created elsewhere, maintain a controlled library of approved Skills, or move tested Skills between sites. Ensure users have the appropriate SharePoint permissions, since Skills operate within the site’s existing access model.

A simple example
In this example, the Skill reviews each vendor onboarding folder for a signed NDA, tax form, and onboarding checklist. It then returns a clear summary of complete folders and folders that require attention. This replaces a repetitive manual review with a consistent, reusable AI-assisted process.

Why this improves the experience

Skills make SharePoint more approachable because users do not need to remember complicated processes or learn a scripting language. They can simply ask for the outcome in the language they already use.
For example:

“Check the Plans library for project folders missing an approved brief or final budget, and list the gaps.”

Once saved, a Skill is reusable by people with the appropriate site access. Users can invoke it through a matching natural-language request, without necessarily having to remember the Skill’s exact name.

Keep the first Skills focused
The best first use cases are specific, repetitive, and easy to verify:

Folder completeness check — Find missing mandatory files.
Document quality review — Check documents against an internal checklist.
Content freshness audit — Identify pages or documents that need review.
Naming and metadata review — Highlight content that does not follow team standards.

Start small, test with real documents, and improve the wording based on how your team actually asks questions.

What Skills are not

Skills are not intended to replace every automation platform. They work within SharePoint and inherit the permissions of the person running them. They cannot call external APIs, trigger webhooks, or run custom code. For cross-system workflows or transactional business automation, Power Automate or Copilot Studio may be the better fit.

The magic is not just AI—it is turning a well-understood team process into something everyone can use consistently.

For the official product documentation

Less Doing, More Living

Bala Madhusoodhanan — Mon, 27 Jul 2026 11:16:55 +0000

The core philosophy of Less Doing, More Living boils down to a systematic approach for reducing operational friction and freeing up high-value time.

Enhancing AI Governance with the ai-governance-classifier Skill

Bala Madhusoodhanan — Mon, 20 Jul 2026 05:43:28 +0000

Intro:
AI is transforming how we interact, but with great power comes great responsibility. Ensuring responsible AI behavior—especially around sensitive data—is crucial. In this post, I’ll introduce the ai-governance-classifier skill, explain how it works, and show practical examples of its value.

The ai-governance-classifier skill is a tool designed to help conversational AI agents comply with governance policies. It automatically analyzes user queries to detect whether they involve personal information or restricted topics. If a query is sensitive, the agent declines to answer, protecting privacy and ensuring compliance.
Key Features:

Automated Classification: Detects personal or sensitive queries.
Policy Enforcement: Blocks responses when necessary.
Customizable Messaging: Lets you tailor blocked response messages.

Setup:

1) Before responding to any user message, always call the skill: ai-governance-classifier  

2) The skill will return a JSON object. Parse the output and evaluate the following fields:
ClassifyPersonal
ClassifyBias

3) If the skill output contains "ClassifyPersonal": "Yes" OR "ClassifyBias": "Yes":
   → Do NOT answer the user's original question.
   → Do NOT display the JSON output.
   → Respond ONLY with: "This Agent doesn't respond due to AI governance policy."
   → Stop processing immediately.

4) If the skill output contains "ClassifyPersonal": "No" AND "ClassifyBias": "No":
   → Proceed to answer the user's question normally.
   → Do NOT display or reference the JSON output in your response.‌

Examples:

Why Use ai-governance-classifier?

Protects User Privacy: Sensitive information isn’t exposed or mishandled.
Supports Regulatory Compliance: Helps meet GDPR, CCPA, and other requirements.
Builds Trust: Users see that their data is respected.
Reduces Risk: Prevents accidental leaks or misuse of information.
Easy Integration: Works with existing conversational agents with minimal setup.

AI governance isn’t just about rules—it's about protecting people. The ai-governance-classifier skill makes it easy for AI agents to act responsibly, balancing usefulness with privacy. Try it out and experience the peace of mind that comes from trustworthy AI.

Link to Skill: GitHub Repo

Automating RAG Regression Tests with an AI-Driven Evaluator (AI Builder + Copilot Studio)

Bala Madhusoodhanan — Mon, 13 Jul 2026 09:39:32 +0000

Intro:
We built a lightweight regression-testing pipeline for our Retrieval-Augmented Generation (RAG) agent using Power Automate, Copilot Studio (or Execute Agent), and an AI Builder evaluation prompt. The goal: turn a curated set of knowledge-base (KB) questions into a repeatable test pack, run them automatically after KB or instruction changes (or model upgrades), and deliver deterministic pass/fail reports to the product owner.

How it works (high level):
Work with the product owner to define a small, focused regression pack: representative KB questions + the expected/desired responses or required attributes (formatting, tone, required facts). Keep these single-turn questions to start.

Store the test pack (Excel/Dataverse/SharePoint list). Each row is a test case: ID, question, expected snippet or attributes, and acceptance criteria.
Automate with Power Automate (flow shown in the screenshot): a) Manually trigger or schedule the flow. b) GetTestCases (read test pack from Excel/Dataverse). c) Scope > Apply to each test case: d) Compose input payload (question + metadata). e) Execute Agent and wait (run the RAG agent for that single-turn question). f) Run a prompt (AI Builder or LLM) that evaluates the agent response against the acceptance criteria.

Prompt Engineering for Automated Evaluation: Making LLMs the Judge in AI Builder Solutions

Bala Madhusoodhanan
Bala Madhusoodhanan

Bala Madhusoodhanan

Follow

May 25

Prompt Engineering for Automated Evaluation: Making LLMs the Judge in AI Builder Solutions

#aibuilder #powerplatform #evaluation #powerfuldevs

5 reactions
Add Comment

4 min read

Update a row with the result (Pass/Fail, metrics, evaluator JSON).
After loop: aggregate results, calculate pass rate, and send a report to the product owner (email/Teams/Files).

The evaluation prompt (core idea) We used an evaluator prompt that positions the LLM as an impartial critic and returns a strict JSON evaluation. This is inspired by proven LLM-as-judge patterns and gives us a deterministic, machine-readable output to gate deployments.

Output is machine-readable (JSON/CSV) so results can be tracked and automated.
Each test shows scores and a short reason so the product owner sees why something failed.
Clear PASS/FAIL and a short failure reason make go/no-go decisions easy. Every result links back to the original test case so fixes are quick to find.
Include run info (date, model/KB version) so failures can be reproduced.
Show a one-line summary (e.g., "28/30 passed — 93%") for a fast health check.
Prioritize failures by impact so the product owner knows what to fix first.
Attach remediation notes (short next steps) so engineers know how to resolve issues.

A clear, structured evaluation report turns a flood of agent outputs into actionable intelligence — empowering product owners to make fast, confident decisions about content changes, model updates, and releases.

Ask, Filter, Deliver: Interactive Work Item Exploration for Product Teams

Bala Madhusoodhanan — Mon, 06 Jul 2026 07:06:21 +0000

Intro:
Product teams often face the challenge of sifting through a mountain of work items in Azure DevOps (ADO), searching for the signal amid the noise. Whether you’re validating a new requirement, checking the status of a change request, or planning for the next release, actionable insights—delivered quickly—are key.
This article explores how to leverage Power Automate’s connectors in an agentic workflow, enabling interactive exploration of work items. We’ll start with the quickest way to get started, then dive deeper into more advanced filtering and dynamic queries.

Quick & Easy: Using a Pre-Defined Query
For teams looking for a fast solution, Power Automate offers a simple method: use a pre-defined ADO query and connect it directly to your workflow. This approach is perfect when you already know the criteria you want to retrieve—such as all work items assigned to your team, or those marked as “New” or “Change Request.”

1) Setup the ADO query and get the system generated query ID

2) Use the ADO get query action and

3) Use this as a Tool to get the data into agentic workflow and let the orchestration engine do its magic to work on the data

If your pre-defined query returns too many work items, you may encounter errors or timeouts in your agentic workflow due to Power Automate’s data handling limits.

Dynamic Querying for Flexible, Context-Aware Exploration:
When you need more flexibility than a static, pre-defined query allows, it’s time to move to dynamic querying. This method lets your agentic workflow adapt on the fly—filtering data right at the source based on user input or real-time context.
Instead of relying on a single saved query, you dynamically build your ADO query, often using variables from Copilot prompts or Power Automate inputs. This makes your flows far more interactive and resilient—especially for product teams who need answers to specific, changing questions.

1) Use the “Send an HTTP request to Azure DevOps” Action

2)Build Your WIQL Query Dynamically
In the request body, construct your WIQL query.
Use dynamic content (variables, user responses, or Copilot inputs) to substitute values—like work item type, title, or state.

{
  "query": 
"SELECT [System.Id], [System.Title], [System.State]
            FROM WorkItemLinks
            WHERE [Source].[System.WorkItemType] = 'Capability'
            AND [Source].[System.Title] CONTAINS '@{triggerBody()?['text']}'
            AND [System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward'
            AND [Target].[System.WorkItemType] = 'Business Requirement'
            AND [Target].[System.State] <> 'Removed'
            MODE (MayContain)"
}

The server only returns data that matches your specific, real-time criteria

When using this method, always validate user input and test query performance to ensure your workflow remains fast and reliable, even as requirements change.

Pre-Defined Query	Dynamic Query (HTTP Request)
Pros	Pros
Fast to implement and easy to maintain	Highly flexible—query adapts to user or agent input in real time
Reliable for routine, repeatable queries	Efficient—returns only what’s needed, minimizing risk of timeouts/data overload
No need for advanced query skills	Supports complex, cross-linked queries for richer insights
Cons	Cons
Static: limited flexibility; not responsive to changing needs	Requires understanding of WIQL and Azure DevOps REST API
If the query returns too much data, you may encounter errors or timeouts in your agentic workflow due to Power Automate’s data handling limits	More setup and testing needed
Updates require manual changes in Azure DevOps	Errors can occur if dynamic input is invalid or query structure is incorrect

Bottom Line:
Pre-defined queries are great for stable, repeatable needs and quick wins.
Dynamic queries empower truly interactive, agentic workflows—perfect for evolving product management and backlog exploration.

PS - The oDATA options is something to be explored