As AI systems become integral parts of applications which assist humans in classifying information, summarizing documents, suggesting actions and automating workflows.
Generating the result is not the end of the story.
When the AI system affects the decision being made, users may also have a need to know why that particular result was generated.
It does not necessarily imply that each model will have to make itself explainable. Application developers can design an application that provides the human-readable context for the output of an AI system.
Reveal the Relevant Factors
For example, an AI system labels the customer's issue as 'high priority'.
Instead of just showing:
Priority: High
it could provide relevant information about the issue:
- Customer has complained about the issue several times
- Issue falls under a critical category of support requests
- Problem remains unsolved
- Human review is advised
This will give additional context to the user without revealing the inner workings of the model itself.
Distinguish between AI Results and Application Logic
Another fundamental principle is to keep AI results distinct from application decisions.
Example workflow:
User Request
↓
AI Analysis
↓
Structured Result
↓
Application Logic
↓
Human Action/Review
AI returns data, application decides what to do based on that data.
This simplifies testing and allows developers to have more control over critical decisions.
Indicate When You Are Not Sure About The Result
Sometimes AI returns incomplete or uncertain results.
Rather than returning all the results with equal degree of confidence, applications may indicate uncertainty appropriately.
Example workflow:
Category: Support Request
Status: Needs Review
Explanation: More than one category returned
This is much more informative than presenting uncertain result as a definitive answer.
Explanations for Developers' Benefit
Human-readable explanations are not only helpful for end users.
It can be helpful in debugging as well.
Having structured information about the input data, model results, validation and final action stored in an application, developers can better understand what happened.
Learn more about the AI + IoT work of Aperture Venture Studio.
Top comments (0)