AI engineering does not end when a model performs well in testing.
Once an AI system enters a real business environment, additional questions emerge. How is the system being used? What data does it depend on? Who owns decisions around it? How are unexpected outputs handled? What happens when the model, data, vendor, or use case changes?
These questions are part of the wider AI governance problem.
The NIST AI Risk Management Framework offers a useful structure for thinking about that problem.
NIST AI RMF 1.0 is a voluntary framework from the U.S. National Institute of Standards and Technology. It is designed to help organizations identify and manage AI risks while promoting trustworthy AI.
The framework is organized around four functions:
Govern establishes organizational policies, roles, responsibilities, accountability, and oversight.
Map focuses on understanding the context of an AI system, including its intended purpose, stakeholders, potential impacts, and risk environment.
Measure focuses on evaluating risks and system characteristics through appropriate testing, analysis, and monitoring.
Manage focuses on prioritizing identified risks, determining responses, and monitoring those risks over time.
For developers and engineering leaders, an important takeaway is that AI risk is broader than application security.
Depending on the system, engineering teams may need to consider data privacy, reliability, security, explainability, fairness, harmful bias, and unexpected system behavior.
The framework also recognizes that AI risk changes over time. A model that performs appropriately in one environment may create different risks after its data, users, integrations, or intended purpose changes.
This makes ongoing monitoring an important part of AI governance.
NIST AI RMF can also provide useful context when organizations are evaluating how their AI practices connect with other frameworks and standards, including cybersecurity and AI management approaches.
For a broader introduction to the framework, NIST AI Risk Management Framework: A Starter Guide for US Enterprises explains the four functions, key trustworthy-AI characteristics, and practical considerations for organizations adopting NIST AI RMF.
Top comments (0)