Compliance teams spend hundreds of hours mapping controls between frameworks, assessing maturity gaps, and generating evidence checklists. What if an AI agent could do the lookup and analysis in seconds while you focus on the decisions?
The Secure Controls Framework
The Secure Controls Framework https://securecontrolsframework.com/ bundles up more than 200 laws, regulations, and frameworks into an organized metaframework. Controls are mapped to each other and multiple extras are included, including evidence request lists, remediation guidance, and more. It’s an incredible resource and is available in multiple formats. SCF has 1,534 controls mapped to 252+ regulations, which is still a lot of data, even with the cross referencing.
My Approach
Agents and vector databases seem like an ideal combination to process the data and recommend actions, create reports, and so on. I built an AI agent that has the full SCF 2026.2 dataset as its knowledge base. Users ask questions in plain English and the agent does the lookup and reasoning. I used Terraform and multiple components of AWS Bedrock AgentCore.
Architecture Decisions
This is where things got interesting. I stumbled around making a mess of things and learning limitations the hard way. In Lessons Learned (below) I’ll explain why these decisions were made:
-Why I used vector search for discovery + DynamoDB for full data (not one or the other)
-Why S3 Vectors didn’t work for this data (2KB metadata limit)
-The “unable to assume role” error that actually means 5 different things
-Adding web search using AgentCore Gateway connectors
Agent Capabilities
Right now, I just have this running from the command line and using IAM permissions. A better long term solution would be to create a frontend and also use something like Cognito for authentication. For the time being though, these are some of the features:
-Gap analysis: “What are my HIPAA gaps?” → shows missing controls prioritized by weight
-Maturity assessment: “Are we at Level 3 for access control?” → compares against CMM criteria
-Framework mapping: “What do we need for NIS2?” → maps SCF controls to EU requirements
-Evidence checklists: “What does an auditor need for IAC-15?” → lists artifacts and cadence
-Live web research: “Any new HIPAA enforcement actions?” → searches authoritative sources
-Memory: remembers your organization across sessions
The agent can ingest additional information related to your company or project. In https://github.com/mgbec/Secure-Controls-Framework-Agent/tree/main/sample-project, I have some files that you can reference in your agent chats. Examples include “third party vendors”, “controls inventory”, “policies and procedures”, and “organization profile”. There are some test prompts you can use, like:
“Acme HealthTech plans to expand to the EU in 2027. Assess their readiness against:
1. EU NIS2 Directive requirements
2. EU DORA (if they serve financial sector healthcare)
3. EU AI Act (for CareFlow Analytics)
Using their current SCF controls inventory, identify:
- Controls they already have that satisfy EU requirements
- New controls they must implement
- Timeline recommendations for a 2027 launch”
Lessons Learned
-This was a really lame error of mine. I populated a knowledge base/vector database and initially didn’t use it as a vector database. Instead I loaded the full SCF file into memory and iterated through the data with Python. As you can guess, the performance was extremely bad. I eventually figured that out and moved toward using things correctly. I ran into more drama- see the next point.
-S3 Vectors needed to be combined with a DynamoDB, at least in this project. My SCF data was going over the S3 Vector metadata limit of 2048 bytes. https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-limitations.html. There are a number of ways to fix this issue, including using OpenSearch and trimming the data. I have spent too much money on OpenSearch before, so I wanted to avoid that. I ended up trimming my data and using S3 Vectors for semantic search. Then DynamoDB was used for full control data for detail retrieval.
-Bedrock model IDs, legacy and inference profiles- I initially used a model id in my code instead of an inference profile. I have also unwittingly used a legacy model id in the past. Both of these cause issues when you invoke the agent runtime. You might see something like this-“Bedrock now requires inference profile IDs (us.anthropic.claude-sonnet-4–20250514-v1:0) rather than raw model IDs for on-demand invocations.” or “An error occurred (ResourceNotFoundException) when calling the ConverseStream operation: Access denied. This Model is marked by provider as Legacy and you have not been actively using the model in the last 30 days. Please upgrade to an active model on Amazon Bedrock”, “status”: “error”}”.
I now have a preflight check script https://github.com/mgbec/Secure-Controls-Framework-Agent/blob/main/scripts/preflight.py that should hopefully catch config issues immediately instead of in the middle of a long deploy cycle.
-Errors “Bedrock Knowledge Base was unable to assume the given role”- I received this error six times for five different things: missing trust conditions, IAM propagation delay, missing S3 vectors permissions, empty S3 Vectors config, or S3 Vectors bucket doesn’t exist.
“RuntimeClientError: Received error (500) from runtime” means your agent code crashed and it could be the wrong model ID, missing module, S3 load timeout, or response too large.
-AgentCore Gateway connectors make adding live web search super easy and give you a way to control the sites available to the search tool, domain filtering.
SCF Updates
The Secure Controls Framework https://securecontrolsframework.com/ is “A Living Control Set (LCS)” with a quarterly update cadence. With the warning about out-of-cycle updates above, and the relentless pace of innovation lately, I put in a check for updates weekly. A lambda(scf-agent-scf-updater) checks every Monday at 8:00 UTC. A SSM Parameter tracks current version. This is the process:
- Download new JSON from SCF website
- Upload to S3 (full + per-domain + versioned copy)
- Reload DynamoDB with all controls (full, untruncated)
- Trigger KB re-ingestion
- Update SSM version parameter
- Send SNS email notification (mentions DynamoDB reload in the message)
What Could be Next?
-Better KB indexing (OpenSearch Serverless once cost justifies)
-Streaming responses for long queries
-Web frontend for non-CLI users
-Multi-tenant support (multiple orgs sharing the infrastructure)
-Security improvements
Try It Yourself
The repo is here: https://github.com/mgbec/Secure-Controls-Framework-Agent, let me know what you think. Many thanks to the volunteers at SCF!
Top comments (0)