Ninth Wave built Compass to solve a specific problem: validating bank APIs against Financial Data Exchange (FDX) standards takes weeks of manual review, blocking open finance integrations. Their solution uses Amazon Bedrock AgentCore to coordinate specialized agents that parse API specs, check compliance, score gaps, and generate reports. The system compresses onboarding from weeks to minutes while maintaining SOC 2 and PCI DSS boundaries.
This is not a chatbot wrapper. It is a production multi-agent system handling sensitive bank credentials and regulatory compliance in a domain where mistakes trigger audits.
The Onboarding Bottleneck
Open finance platforms need to validate that third-party bank APIs conform to FDX standards before integration. Manual validation involves:
- Parsing OpenAPI specifications for endpoint structure
- Checking authentication flows against FDX security requirements
- Validating data schemas for account, transaction, and payment objects
- Scoring compliance gaps across 200+ standard requirements
- Generating audit-ready reports for compliance teams
Each bank integration requires this full cycle. Manual review takes 2-4 weeks per bank. Compass automates the entire pipeline.
Multi-Agent Architecture
Compass uses four specialized agents coordinated by Amazon Bedrock AgentCore:
Specification Parser Agent
Ingests OpenAPI/Swagger files, extracts endpoints, authentication schemes, and data models. Returns structured JSON representation of the API surface.
FDX Validation Agent
Compares parsed API structure against FDX 6.0 standard requirements. Checks endpoint naming conventions, required headers, OAuth 2.0 flows, and data field mappings.
Compliance Scoring Agent
Assigns weighted scores to validation results. Critical gaps (missing authentication, incorrect data types) score higher than cosmetic issues (documentation formatting). Outputs a 0-100 compliance score with itemized gap list.
Report Generation Agent
Transforms validation results and scores into human-readable reports. Generates executive summaries for business stakeholders and detailed technical reports for engineering teams.
AgentCore handles orchestration. When a bank API spec arrives, AgentCore routes it to the parser, waits for structured output, then fans out to validation and scoring agents in parallel. Report generation runs after both complete.
Security Boundaries in Regulated Workflows
SOC 2 and PCI DSS requirements shape the architecture in specific ways:
Credential Isolation
Bank API credentials never touch agent prompts. The parser agent receives pre-signed S3 URLs pointing to encrypted specification files. Credentials for test API calls live in AWS Secrets Manager, accessed only by isolated Lambda functions that agents invoke via tool calls.
Audit Logging
Every agent action logs to CloudWatch with structured metadata: agent ID, tool invoked, input hash, output hash, timestamp. Logs feed into AWS Security Hub for compliance monitoring.
Data Residency
All intermediate state (parsed specs, validation results, scores) stays in S3 buckets with encryption at rest and in transit. No data crosses regional boundaries. Bedrock model invocations use VPC endpoints to avoid public internet exposure.
Access Control
IAM roles enforce least privilege. The parser agent can read S3 specifications but cannot write to the scoring database. The report generator can read validation results but cannot invoke the FDX validation agent.
Orchestration Flow
# Simplified orchestration pseudocode
def process_bank_onboarding(api_spec_url):
# Step 1: Parse specification
parsed_spec = parser_agent.invoke(
input={"spec_url": api_spec_url},
tools=["s3_read", "openapi_parse"]
)
# Step 2: Parallel validation and scoring
validation_future = fdx_agent.invoke_async(
input={"spec": parsed_spec},
tools=["fdx_standard_check", "endpoint_validator"]
)
scoring_future = scoring_agent.invoke_async(
input={"spec": parsed_spec},
tools=["gap_analyzer", "weight_calculator"]
)
validation_result = validation_future.result()
compliance_score = scoring_future.result()
# Step 3: Generate report
report = report_agent.invoke(
input={
"validation": validation_result,
"score": compliance_score
},
tools=["pdf_generator", "s3_write"]
)
return report
AgentCore manages retries, timeouts, and error propagation. If the FDX validation agent times out (common with large API specs), AgentCore retries with exponential backoff. If validation fails due to malformed input, the workflow halts and escalates to human review rather than proceeding to scoring.
State Management and Failure Modes
Compass stores intermediate state in DynamoDB with TTL set to 30 days. Each onboarding session gets a unique ID. State includes:
- Parsed specification JSON
- Validation results per FDX requirement
- Compliance score breakdown
- Report generation status
Failure Mode: Parser Timeout
Large API specs (5000+ endpoints) occasionally exceed the 5-minute Lambda timeout. AgentCore detects the timeout, splits the spec into chunks, and invokes the parser agent multiple times. Results merge before validation.
Failure Mode: FDX Standard Ambiguity
Some FDX requirements have multiple valid interpretations. The validation agent flags these as "manual review required" rather than pass/fail. The compliance score excludes ambiguous items from the denominator.
Failure Mode: Credential Expiry
Test API credentials expire during validation. The system detects 401 responses, triggers a credential refresh via Secrets Manager rotation, and retries the validation step. If refresh fails, the workflow pauses and notifies the ops team.
Tool Call Patterns
Each agent has a constrained tool set. The FDX validation agent cannot write to S3 or invoke other agents. It can only:
- Read from the FDX standard reference database (RDS PostgreSQL)
- Invoke the endpoint validator Lambda
- Write validation results to DynamoDB
This constraint prevents privilege escalation. If an attacker compromises the validation agent's prompt, they cannot exfiltrate data or trigger unintended workflows.
Tool calls include input validation. The endpoint validator Lambda rejects requests with SQL injection patterns, path traversal attempts, or oversized payloads. Rejected calls log to Security Hub and trigger alerts.
Observability and Debugging
Compass exposes three observability layers:
Agent Traces
AgentCore logs every agent invocation with input/output snapshots. Traces show which tools each agent called, in what order, and how long each step took. Useful for debugging why a validation failed or a score seems wrong.
Workflow Metrics
CloudWatch tracks end-to-end latency, per-agent latency, retry counts, and failure rates. Dashboards show P50/P95/P99 latency for each agent. Alerts fire when retry rates exceed 5% or when any agent's P99 latency crosses 60 seconds.
Compliance Audit Logs
Immutable logs in S3 capture every API spec processed, every validation result, and every report generated. Logs include SHA-256 hashes of inputs and outputs for tamper detection. Compliance teams query these logs during SOC 2 audits.
Deployment Shape
Compass runs entirely on AWS:
- AgentCore: Orchestrates agent invocations, manages state, handles retries
- Bedrock Models: Claude 3.5 Sonnet for parsing and validation, Claude 3 Haiku for scoring and report generation
- Lambda: Hosts tool functions (endpoint validator, PDF generator, credential rotator)
- DynamoDB: Stores session state and validation results
- RDS PostgreSQL: Hosts FDX standard reference data
- S3: Stores API specs, intermediate results, and generated reports
- Secrets Manager: Manages bank API test credentials
- CloudWatch: Logs, metrics, and alarms
Infrastructure as code uses CDK. Deployments run through a CI/CD pipeline with automated security scanning (Checkov, Trivy) and integration tests that validate agent coordination against synthetic bank API specs.
Trade-offs and Constraints
| Dimension | Choice | Trade-off |
|---|---|---|
| Model Selection | Claude 3.5 Sonnet for validation | Higher cost per invocation but better accuracy on complex FDX rules vs. faster/cheaper models that miss edge cases |
| State Storage | DynamoDB with 30-day TTL | Fast reads/writes but no complex queries; compliance teams must export to S3 for long-term analysis |
| Orchestration | AgentCore managed service | Less control over retry logic and timeout tuning vs. self-managed Step Functions with custom error handling |
| Tool Isolation | Separate Lambda per tool | Higher cold start latency (200-500ms per tool call) vs. monolithic Lambda with all tools bundled |
| Credential Management | Secrets Manager rotation | Adds 100-200ms per validation run vs. caching credentials in memory (violates PCI DSS) |
The team chose managed services over self-hosted infrastructure to reduce operational burden. Ninth Wave has three engineers maintaining Compass. Self-hosting Kubernetes for agent orchestration would require dedicated SRE headcount.
When Validation Fails
Not all bank APIs pass validation. Compass handles three failure scenarios:
Critical Gaps
Missing required endpoints (e.g., /accounts, /transactions) or broken authentication flows. Compliance score drops below 60. Workflow generates a report but marks the integration as blocked. Human review required before production deployment.
Minor Gaps
Non-critical issues like missing optional fields or documentation errors. Compliance score 60-85. Workflow generates a report with recommended fixes. Integration can proceed with documented exceptions.
Ambiguous Requirements
FDX standard allows multiple implementations for certain features (e.g., pagination strategies). Validation agent flags these for manual review. Compliance score excludes ambiguous items. Report includes guidance on acceptable implementation patterns.
Technical Verdict
Use Compass-style multi-agent validation when:
- You need to automate compliance checking against a well-defined standard (FDX, FHIR, HL7)
- Manual review creates a multi-week bottleneck in your onboarding pipeline
- You operate in a regulated domain (finance, healthcare) where audit trails and security boundaries are non-negotiable
- You have 50+ integrations to validate and maintain over time
Avoid this pattern when:
- Your compliance requirements change frequently (agent prompts and tool logic become maintenance nightmares)
- You need real-time validation (multi-agent orchestration adds 30-90 seconds of latency vs. synchronous rule engines)
- Your team lacks AWS expertise (debugging multi-agent failures requires deep knowledge of Bedrock, Lambda, and IAM)
- You process fewer than 10 integrations per year (manual review is cheaper than building and maintaining agent infrastructure)
The value proposition scales with integration volume. At 5 integrations per year, manual review costs less than agent infrastructure. At 50+ integrations, Compass pays for itself in saved engineering time and faster time-to-market.
Top comments (0)