Originally published at https://charz.ai/blog/generative-ai-profile-nist-ai-600-1 Char-Z AI.
Why Generative AI Needs Its Own Profile
The NIST AI RMF was designed for AI broadly, but generative AI produces distinctive risks — confabulated output, harmful content, disclosure of private training data, and threats to information integrity (NIST, 2024). In July 2024, NIST published the Generative AI Profile (NIST AI 600-1) to help organizations apply the AI RMF's Govern, Map, Measure, and Manage functions specifically to generative AI systems, including large language models.
The profile is voluntary and functions as a lens on the AI RMF. It identifies 12 action categories under the four functions and maps each to the relevant AI RMF subcategories and controls. It is among the most practical references for teams building governance around foundation models, chatbots, code assistants, and media-generation tools.
The 12 Action Categories
Govern
Action 1: Define and communicate actual and intended use cases and context. Be explicit about what your generative AI system is for and where it may be deployed or misused.
Action 2: Sufficiently document context to allow confidence in the output. Document training distribution, intended bounds, and limitations so users can reason about trustworthiness.
Action 3: Conduct red-teaming and tabletop exercises. Proactively probe the system for jailbreaks, harmful outputs, and failure modes before deployment.
Action 4: Zero-trust architecture approaches. Apply least-privilege and identity controls around the system, its data, and its integrations.
Action 5: Invest in robust statistical analyses of outputs. Track and analyze output behavior to detect degradation and drift.
Map
Action 6: Assess and disclose risks and potential harms. Identify and document the types of harm generative AI can produce in your context.
Action 7: Establish a risk-management process. Connect the profile's actions to the broader AI RMF risk process.
Action 8: Responsible sourcing and transparent sharing of AI assets. Understand provenance of models, datasets, and weights you use or publish.
Measure
Action 9: Verify assessments, evaluations, benchmarks, and claims. Test outputs against defined metrics before trusting or publishing performance claims.
Action 10: Implement effective mitigation for content identified as private, sensitive, or harmful. Put in place filtering, redaction, and refusal behaviors.
Manage
Action 11: Identify and implement effective mitigations for cybersecurity vulnerabilities. Address the system-specific security risks of generative AI, including prompt-injection and adversarial inputs.
Action 12: Establish AI technology incident response and recovery plans. Prepare to detect, respond to, and recover from generative-AI incidents.
How to Apply the Profile
Scope first. Not every action applies at full depth to every system. Start with the generative AI systems that carry the highest risk — often those with broad user access, autonomy, or sensitive data.
Map actions to your existing controls. Many of the twelve overlap with security and privacy work you already do. Use the profile to find gaps rather than to start from zero.
Pair it with the EU AI Act. For the EU market, the profile's focus on transparency, content provenance, and risk disclosure complements the Act's Article 50 transparency obligations and its GPAI requirements (European Commission, 2024). The two reinforce each other: the profile gives you the methodology, the Act gives you the deadline (December 2, 2026, when the watermarking grace period ends).
Example: A Customer-Facing Chatbot
Govern: document intended use, red-team before launch, define a risk appetite for misinformation
Map: assess harms (confabulation, brand risk, harmful content) and sources
Measure: benchmark factual accuracy, monitor refusal and harmful-output rates
Manage: apply output filtering, prompt-injection mitigations, and an incident-response plan
Sources
NIST. (2024). *Generative Artificial Intelligence Profile (NIST AI 600-1)*. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1
NIST. (2023). *Artificial Intelligence Risk Management Framework (AI RMF 1.0)*. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.100-1
European Commission. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council. *Official Journal of the European Union*. https://eur-lex.europa.eu/eli/reg/2024/1689
Top comments (0)