Problem
GenAI agents must resist prompt injection, avoid harmful outputs, and maintain provenance. Azure AI Content Safety offers moderation for text/images and safety features (Prompt Shields, groundedness detection, protected material) with configurable thresholds and blocklists.
Options and trade-offs
- Prompt Shields (direct and indirect injection): Detect jailbreak attempts and document-borne attacks; low false-positive rates; add latency; configurable block vs annotate modes.
- Groundedness detection (Preview): Detect ungrounded completions relative to sources; helps provenance and QA; preview scope limits.
- Protected material detection (text GA, code preview): Identifies copyrighted segments in completions; supports compliance; English scope.
- Blocklists and custom categories: Enforce organization-specific terms and custom detection; stored encrypted at rest with CMK options.
Minimal working example
Text moderation (Python) Requirements: requests or SDK; endpoint; subscription key or Entra auth.
import requests, json
endpoint = "https://YOUR-CONTENT-SAFETY.cognitiveservices.azure.com"
headers = {"Ocp-Apim-Subscription-Key": "KEY","Content-Type":"application/json"}
payload = {"text": "Sample text", "blocklistNames": []}
r = requests.post(f"{endpoint}/contentsafety/text:analyze?api-version=2024-09-01",
headers=headers, data=json.dumps(payload))
print(r.json())
Test: Inspect categoriesAnalysis severities; adjust thresholds; add blocklists.
Checklist
- Configure separate input/output filters; turn on Prompt Shields for inputs; add groundedness and protected material for outputs.
- Confirm API rate limits and payload size limits per modality/tier.
- Use blocklists for brand/PII terms; understand storage and deletion controls.
Where this doesn't apply
Offline moderation pipelines with proprietary classifiers; integrate but consider Content Safety for complementary coverage.
Sources
- What is Azure AI Content Safety? (GA) — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview (accessed 2026-10-13).
- Documentation hub (GA) — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/ (accessed 2026-10-13).
- Data, privacy, and security (GA) — https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/content-safety/data-privacy (accessed 2026-10-13).
Top comments (0)