DEV Community: Cristopher Coronado

Measure Agent Quality and Safety with Azure AI Evaluation SDK and Azure AI Foundry

Cristopher Coronado — Tue, 31 Mar 2026 00:19:42 +0000

A practical evaluation pipeline for GraphRAG agents with quality metrics, safety scans, and observable runs.

Introduction

In Part 4, we orchestrated multiple agents. This article (Part 5) answers a harder question: can we prove that the system is reliable enough for production workloads?

For AI Engineers, answer quality alone is not enough. You also need:

Repeatable quality checks before release.
Safety evidence for security and compliance reviews.
Traceability when behavior changes after prompt, model, or tool updates.

This part adds an evaluation module under src/evaluation with three goals:

Quality: task completion, intent resolution, tool-call behavior, graph-grounded correctness.
Safety: adversarial probing with red team strategies and risk categories.
Observability: telemetry and artifacts that support debugging and regression analysis.

How the three goals are measured

Goal	Primary signals	Current evidence in this article
Quality	`task_adherence`, `intent_resolution`, `relevance`, `coherence`, `response_completeness`	Foundry quality snapshot (March 2026): 80% task adherence, 100% on other quality signals
Safety	Red team attack outcomes by risk category and strategy	Red team run section and Foundry safety screenshots
Observability	Prompt/completion token usage, OTel traces, local artifacts	Quality snapshot token counters (85,686 prompt / 5,048 completion) + OTel and JSON report references

Quality and safety runs can also be exported to Azure AI Foundry, so teams review outcomes in shared dashboards instead of only local JSON artifacts.

Why This Matters for AI Engineers

When you ship agent systems, every change can alter behavior: prompts, model versions, tool schemas, and data.

Engineering scenario	Typical failure mode	Why this pipeline helps
Prompt or model update	Fluent but lower-quality answers	Batch baselines expose quality regressions before release
Tool contract changes	Wrong tool or wrong arguments	Tool-call evaluators detect routing and schema drift
Knowledge graph refresh	Unsupported entities/relationships in answers	Custom graph evaluators detect grounding errors
Safety hardening	Unknown risk exposure under adversarial inputs	Red team runs provide repeatable safety evidence
Incident debugging	Hard to explain why behavior changed	OTel traces and result artifacts reduce investigation time

What You Build

Layer	Component	Purpose
Dataset	`golden_questions.jsonl`	Controlled test set with expected outcomes
Generation	`generate_eval_data.py`	Runs the agent and writes evaluation data
Quality Eval	`run_batch_evaluation.py`	Runs built-in and custom evaluators
Safety Eval	`run_redteam.py`	Runs red team scans against the agent/model
Reporting	`evaluation_report.md`, JSON outputs, Foundry run links	Human and machine-readable results

Module Layout

src/evaluation/
├── config.py
├── datasets/
│   ├── golden_questions.jsonl
│   └── eval_data.jsonl
├── evaluators/
│   ├── builtin.py
│   ├── entity_accuracy.py
│   └── relationship_validity.py
├── monitoring/
│   └── otel_setup.py
├── results/
└── scripts/
    ├── generate_eval_data.py
    ├── run_batch_evaluation.py
    └── run_redteam.py

What Is Evaluated, and Why

Built-in quality evaluators

Evaluator	What it checks	Why it matters
`TaskAdherenceEvaluator`	Does the response complete the requested task?	Detects incomplete or off-target answers
`IntentResolutionEvaluator`	Does the response resolve user intent?	Detects responses that are fluent but irrelevant
`RelevanceEvaluator`	Is the response relevant to the query?	Detects answers that drift away from the user request
`CoherenceEvaluator`	Is the response logically consistent?	Detects contradictions and weak reasoning flow
`ResponseCompletenessEvaluator`	Does the response cover expected content?	Detects partial answers against expected coverage

Built-in tool-behavior evaluator (conditional)

Evaluator	What it checks	Why it matters
`ToolCallAccuracyEvaluator`	Were tools/arguments appropriate?	Detects wrong routing, wrong parameters, unnecessary calls

ToolCallAccuracyEvaluator is included when structured tool_call payloads exist in eval_data.jsonl.

Custom graph evaluators

Evaluator	What it checks	Why it matters
`EntityAccuracyEvaluator`	Mentioned entities exist in Parquet graph data	Detects unsupported entities and weak grounding
`RelationshipValidityEvaluator`	Co-mentioned entity pairs match graph relationships	Detects fabricated links between entities

Safety scanning

Component	What it checks	Why it matters
`RedTeam` scan	Attack outcomes by risk category and strategy	Produces safety evidence and failure patterns

Pipeline Steps

Step 1: Start MCP Server

poetry run python run_mcp_server.py

Step 2: Generate Evaluation Data

poetry run python -m evaluation.scripts.generate_eval_data

This runs the Knowledge Captain against 10 golden questions and writes eval_data.jsonl.

Step 3: Run Batch Evaluation

poetry run python -m evaluation.scripts.run_batch_evaluation

Optional variants:

# Skip custom graph evaluators
poetry run python -m evaluation.scripts.run_batch_evaluation --no-custom

# Publish quality run to New Foundry (dashboard + report URL)
poetry run python -m evaluation.scripts.run_batch_evaluation --foundry

Figure. Batch quality run in Azure AI Foundry, including evaluator metrics and row-level evidence.

Step 4: Run Red Team Scan

poetry run python -m evaluation.scripts.run_redteam --flow cloud-model

Use cloud-model as default for predictable behavior.

Figure. Red team safety run in Azure AI Foundry, showing risk-category outcomes and attack results.

Where Azure AI Foundry Fits

Part 5 uses Azure AI Foundry as the shared visualization layer for evaluation operations:

Step 3 (--foundry): publishes a New Foundry quality run and returns a report URL for dashboard review. The default Foundry quality set emphasizes semantic signals (relevance, coherence, response_completeness) plus agent checks (task_adherence, intent_resolution).
Step 4 (run_redteam): runs the red team scan and publishes a New Foundry reference run for safety visibility.
Custom graph evaluators: execute in the same batch pipeline and are persisted in local artifacts (evaluation_results.json, evaluation_report.md) that are reviewed alongside Foundry run links.

By design, lexical overlap metrics such as F1 are not the default in Foundry export for this agent workflow, because they can under-score correct but paraphrased answers.

This gives one operational workflow: Foundry for centralized run visibility, local custom metrics for graph-specific grounding checks.

Latest Foundry quality snapshot (March 2026)

Most recent quality run summary (10 rows):

Metric	Score	Rows
Task adherence	80%	8/10
Intent resolution	100%	10/10
Relevance	100%	10/10
Coherence	100%	10/10
Response completeness	100%	10/10
Prompt tokens	85,686	-
Completion tokens	5,048	-

How to interpret this snapshot:

Semantic quality is stable across the full set.
task_adherence is the primary optimization target.
ToolCallAccuracyEvaluator is emitted only when eval_data.jsonl includes structured tool_call payloads.

Figure. Azure AI Foundry evaluations list used as the central run registry for Part 5.

Key Snippets That Matter

1. Message conversion (MAF to evaluator schema)

The SDK expects OpenAI-style tool_call and tool_result. MAF internally uses function messages.

from evaluation.evaluators.builtin import convert_to_evaluator_messages

messages = convert_to_evaluator_messages(all_msgs)

Without this conversion, tool-focused evaluators are unreliable.

2. Correct `evaluator_config` mapping shape

evaluator_config = {
    "task_adherence": {
        "column_mapping": {
            "query": "${data.query}",
            "response": "${data.response}",
        }
    }
}

Flattened mappings break field binding.

3. Deployment compatibility guard

Some deployments reject max_tokens and require max_completion_tokens.

if "intent_resolution" in evaluators and not _supports_legacy_max_tokens(config):
    evaluators.pop("intent_resolution", None)

This keeps the run operational while preserving the rest of the evaluation set.

4. Red team semantic success guard

total_attacks = _extract_total_evaluated_attacks(result_payload)
if total_attacks == 0:
    raise RuntimeError("Red team scan completed but produced zero evaluated attacks.")

This prevents false-success runs in unsupported regions.

Interpreting Results for Release Decisions

Do not treat one score as the whole truth. Use a small gate matrix.

Figure. Foundry evaluation evidence used as release-gate input, not only as post-run reporting.

Dimension	Signal to watch	Practical release question
Task quality	TaskAdherence, IntentResolution	Are answers complete and aligned with user goal?
Tool behavior	ToolCallAccuracy	Is orchestration stable after changes?
Graph grounding	EntityAccuracy, RelationshipValidity	Are claims supported by the knowledge graph?
Safety	Red team ASR and risk outcomes	Did risk exposure improve, regress, or stay flat?
Traceability	OTel traces + run artifacts	Can we explain failures quickly?

Recommended practice:

Compare against previous baseline, not isolated absolute values.
Block release on clear regressions in critical metrics.
Keep known exceptions documented and time-boxed.
Treat missing ToolCallAccuracy in a run as dataset-shape not-applicable (no structured tool calls), not as an automatic failure.

Common Failure Patterns

Pattern	Typical cause	Mitigation
Missing or wrong evaluator columns	Incorrect `evaluator_config` shape	Use nested `column_mapping`
Intermittent evaluator failure	Deployment incompatibility for token params	Use evaluator-only deployment override
Red team run shows `0/0`	Region capability mismatch	Move Foundry project to supported region
Good text but poor grounding	Response not constrained by graph evidence	Add graph checks and update prompts
Hard-to-debug regressions	Missing traces/artifacts	Keep OTel + result JSON in every run

Environment Variables

Variable	Required	Purpose
`AZURE_OPENAI_ENDPOINT`	Yes	Azure OpenAI endpoint
`AZURE_OPENAI_API_KEY`	Yes	Azure OpenAI key
`AZURE_OPENAI_CHAT_DEPLOYMENT`	No	Default evaluator/chat deployment
`AZURE_OPENAI_EVAL_CHAT_DEPLOYMENT`	No	Evaluator-only deployment override
`AZURE_AI_PROJECT`	Step 4	New Foundry project endpoint
`APPLICATIONINSIGHTS_CONNECTION_STRING`	No	Production telemetry sink
`OTEL_TRACING_ENDPOINT`	No	Local OTLP endpoint

Recommended Foundry Screenshots

Evaluations list view for portfolio-level evidence.
Batch quality run details with summary and row-level metrics.
Red team run details with risk categories and outcomes.
Release-gate evidence view for decision making.
Prefer detailed run views over reduced pages that only show token counts.

Validation Snapshot

Current module tests:

File	Tests
`tests/evaluation/test_config.py`	15
`tests/evaluation/test_builtin_evaluators.py`	21
`tests/evaluation/test_custom_evaluators.py`	14
`tests/evaluation/test_monitoring.py`	6
`tests/evaluation/test_run_redteam.py`	7
Total	63

Why This Part Is a Milestone

Part 5 turns the project from a functional demo into an engineering-grade evaluable system.

You can compare behavior over time.
You can detect regressions before production.
You can produce safety evidence in a repeatable way.
You can keep a traceable path from query to metric.

This is the baseline for Part 6 (Human-in-the-Loop) and production quality gates.

Key Takeaways

Agent quality in production is not a single score. You need quality, safety, and traceability together.
Built-in evaluators and custom graph evaluators solve different problems and should be used as a combined gate.
Azure AI Foundry gives shared visibility for runs, while local artifacts preserve GraphRAG-specific evidence.
Missing ToolCallAccuracy is often a dataset-shape condition, not automatically a regression.
Red team outcomes should be treated as release evidence, not as an optional post-check.

What's Next

In Part 6, we will add Human-in-the-Loop controls to the same pipeline:

Approval gates for sensitive tool actions.
Explicit escalation paths for low-confidence responses.
Audit-friendly checkpoints connected to the same evaluation workflow.

Resources

Project repository: https://github.com/cristofima/maf-graphrag-series

Serverless ML on AWS: How I Cut Inference Costs by 99% with Lambda & ONNX

Cristopher Coronado — Mon, 29 Dec 2025 01:39:39 +0000

TL;DR: I built an AutoML platform on AWS that handles training and inference for pennies. By moving from always-on SageMaker endpoints to a Serverless Lambda + ONNX architecture, I eliminated idle costs completely—bringing the total bill for a side project down to ~$3-25/month vs ~$36-171/month with SageMaker.

As an AWS Community Builder, I'm always looking for ways to leverage cloud-native services to solve problems efficiently. In my previous article, I shared how to train models cheaply using AWS Fargate Spot. But training is only half the battle.

To turn a trained model into a product, you need Inference—the ability to make predictions on new data. The industry standard is Amazon SageMaker Real-time Endpoints. They are powerful, scalable, and enterprise-ready. But for a personal project or sporadic workload, they have one major downside: You pay for them 24/7, even when no one is using them.

In this post, I'll walk you through how I evolved AWS AutoML Lite into a full-stack ML platform by adding Serverless Inference, Dark Mode, and robust Model Comparison, all while keeping costs at rock bottom.

1. The Inference Challenge: $0 vs $36-171/month

The standard "easy" path on AWS is deploying a SageMaker Endpoint.

Production Standard: An ml.c5.xlarge instance costs ~$147/month.
Budget Option: An ml.t3.medium instance costs ~$36/month.

But what about traffic?
Let's compare costs for a Side Project (100k reqs) vs Startup Scale (10M reqs).

Scenario	Monthly Requests	SageMaker (ml.t3.medium)	Lambda (Serverless)
Side Project	100,000	$36.00	$0.35
Startup Scale	10,000,000	$36.00	~$35.00

Transparent Accounting: "Serverless" isn't magic. You also pay for S3 Storage (to hold the model) and S3 GET Requests (every time a cold Lambda downloads it).

Storage: 50MB model = ~$0.001/month.

Access: 100k cold starts = ~$0.04 in request fees.

Even adding these "hidden" costs, the total remains ~$35.04 vs $36.00. The math holds up.

The Verdict: You would need over 10 million predictions/month just to match the cost of the cheapest, least reliable SageMaker instance ($36). For the production-grade ml.m5.xlarge ($171), the break-even point is closer to 48 million requests.

Until you hit that scale, Serverless is vastly cheaper.

The Solution: ONNX Runtime on Lambda

I decided to move the inference logic to AWS Lambda. Since Lambda is event-driven, you only pay when code runs.

Cost: ~$0.20 per 1 million requests.
Idle Cost: $0.00.

To make this work efficiently, I used the ONNX (Open Neural Network Exchange) format.

Export: During training (on Fargate), we convert models (Scikit-Learn, LightGBM) to .onnx.
Deploy: The deployment process simply flags the job in DynamoDB—no server provisioning required.
Predict: A Python Lambda function loads the model from S3 into memory (creating a "warm" start) and uses onnxruntime to generate predictions in milliseconds.

This architecture is the definition of Cloud Native Efficiency: maximum value, minimum waste.

We added a "Deploy" button to the results page. Under the hood, this doesn't spin up a server. It simply flags the job as "deployed" in DynamoDB and ensures the model artifacts are ready in S3. The API then knows to route prediction requests for that Job ID to the inference engine.

Bonus: Portable Models

If you don't even want to run Lambda, you can just download the model. We provide both the raw .pkl (Pickle) file and the .onnx file. This means you can run your trained model locally, on your own server, or even inside a browser using ONNX Runtime Web.

2. Comparing Models: Data-Driven Decisions

Training one model is rarely enough. You usually train 4-5 variations with different time budgets or datasets to see what sticks.

Version 1.1.0 introduces a Compare Page. You can select up to 4 training runs and see them side-by-side.

Once selected, the platform visualizes the differences. This is crucial for spotting trade-offs. Maybe Model A has slightly better accuracy (94%) but took 20 minutes to train, while Model B is close enough (92%) but finished in 2 minutes.

3. UI Polish: Dark Mode & UX

A modern developer tool isn't complete without Dark Mode. We implemented a system-aware theme switcher using next-themes and Tailwind CSS. It respects your OS preference by default but lets you toggle it manually.

We also moved from a "developer console" look to a cleaner design, optimizing the history table to show just the key stats you need.

4. Engineering Lessons Learned

Building features is fun, but fixing bugs teaches you more. Here are two big technical hurdles we overcame in this release.

Lesson A: The "Deleted" Resource that Wasn't

We encountered a frustrating bug: users would delete a training job, but if they clicked "Back" in their browser, the Job Details page would still load perfectly—serving stale data.

The Culprit: Browser Caching.
Even though the DELETE API call was successful, the browser had cached the previous GET request for the Job Details. When the user navigated back, the browser served the cached "200 OK" response from disk without asking the server.

The Fix:
We had to implement a strict "Trust No One" caching strategy:

Backend: The DELETE response now sends aggression anti-cache headers (no-store, max-age=0).
Frontend: The critical getJobDetails fetch call now uses cache: 'no-cache'. This forces the browser to send an ETag check to the server.
- If the job exists: Server says "304 Not Modified" (fast).
- If the job is deleted: Server says "404 Not Found" -> UI updates immediately to show the error state.

Takeaway: Deleting a resource on the server doesn't automatically purge it from your user's browser cache. You have to be explicit.

Lesson B: Preserving State via Polling

Our UI polls the backend every 5 seconds to update the training progress bar. However, we found that sometimes valid presigned URLs (for downloading models) would disappear or break during these updates.

Ideally, you shouldn't re-fetch the whole object if you only need the status. But since we do fetch the full state, we implemented a client-side merger. It takes the new status from the API but preserves any existing valid URLs from the old state. This prevents the "flickering" download buttons and ensures a smooth user experience.

What's Next? (Roadmap)

We aren't stopping here. The roadmap for next version includes:

Multi-user Authentication (Cognito)
Email Notifications (SNS) when long training jobs finish.
Hyperparameter Tuning UI for advanced users.

Try It Yourself

AutoML Lite is open source and can be deployed to your own AWS account in about 15 minutes.

GitHub Repository

If you have questions or want to contribute, drop a comment or open an issue on GitHub!

Building a Cost-Effective AutoML Platform on AWS: Serverless Training at ~$0.02/job

Cristopher Coronado — Tue, 02 Dec 2025 22:14:22 +0000

TL;DR: I built a serverless AutoML platform that trains ML models for ~$10-25/month (20 jobs). Upload CSV, select target column, get a trained model. No ML expertise required. Training costs ~$0.02/job vs $0.03-0.16/job on SageMaker—but the real savings come from avoiding always-on infrastructure.

Prerequisites

To deploy this project yourself, you'll need:

AWS Account with admin access
AWS CLI v2 configured (aws configure)
Terraform >= 1.9
Docker installed and running
Node.js 20+ and pnpm (for frontend)
Python 3.11+ (for local development)

⏱️ Deployment time: ~15 minutes from clone to working platform

The Problem

AWS SageMaker Autopilot is powerful but the total cost of ownership can be high for prototyping:

Training costs: $0.034-0.16/job (10 min) depending on instance type—reasonable for occasional use
Real-time endpoints: A single ml.c5.xlarge endpoint costs ~$150/month running 24/7
Setup overhead: SageMaker Studio requires initial configuration and learning curve

For side projects where I train occasionally and don't need real-time inference, I wanted a simpler, cheaper alternative with portable models I could use anywhere.

Goals:

Upload CSV → Get trained model (.pkl) - portable, not locked to AWS
Auto-detect classification vs regression
Generate EDA reports automatically
Training cost < $0.05/job for small-medium datasets
Total cost under $25/month for moderate usage (20 jobs)

Architecture Decision: Why Lambda + Batch (Not Containers Everywhere)

The key insight: ML dependencies (265MB) exceed Lambda's 250MB limit, but the API doesn't need them.

Split architecture benefits:

Lambda: Fast cold starts (~200ms), cheap ($0.0000166/GB-sec)
Batch/Fargate Spot: 70% cheaper than on-demand, handles 15+ min jobs
No always-on containers = no idle costs

Data Flow

Tech Stack

Component	Technology	Why
Backend API	FastAPI + Mangum	Async, auto-docs, Lambda-ready
Training	FLAML + scikit-learn	Fast AutoML, production-ready
Frontend	Next.js 16+ Tailwind	SSR support via Amplify
Infrastructure	Terraform	Reproducible, multi-env
CI/CD	GitHub Actions + OIDC	No stored AWS credentials

Key Implementation Details

1. Smart Problem Type Detection

The UI automatically detects if a column should be classification or regression:

# Classification if: <20 unique values OR <5% unique ratio
def detect_problem_type(column, row_count):
    unique_count = column.nunique()
    unique_ratio = unique_count / row_count

    if unique_count < 20 or unique_ratio < 0.05:
        return 'classification'
    return 'regression'

2. Environment Variable Cascade (Critical Pattern)

Training container runs autonomously on Batch. It receives ALL context via environment variables:

Terraform → Lambda env vars → batch_service.py → containerOverrides → train.py

If you add a parameter to train.py, you MUST also add it to containerOverrides in batch_service.py.

# batch_service.py
container_overrides = {
    'environment': [
        {'name': 'DATASET_ID', 'value': dataset_id},
        {'name': 'TARGET_COLUMN', 'value': target_column},
        {'name': 'JOB_ID', 'value': job_id},
        {'name': 'TIME_BUDGET', 'value': str(time_budget)},
        # ... all S3/DynamoDB configs
    ]
}

3. Auto-Calculated Time Budget

Based on dataset size:

Rows	Time Budget
< 1K	2 min
1K-10K	5 min
10K-50K	10 min
> 50K	20 min

4. Training Progress Tracking

Real-time status via DynamoDB polling (every 5 seconds):

5. Generated Reports

EDA Report - Automatic data profiling:

Training Report - Model performance and feature importance:

CI/CD with GitHub Actions + OIDC

No AWS credentials stored in GitHub. Uses OIDC for secure, temporary authentication.

Required IAM Permissions (Least Privilege)

{
  "Statement": [
    {
      "Sid": "CoreServices",
      "Effect": "Allow",
      "Action": ["s3:*", "dynamodb:*", "lambda:*", "batch:*", "ecr:*"],
      "Resource": "arn:aws:*:*:*:automl-lite-*"
    },
    {
      "Sid": "APIGatewayAndAmplify",
      "Effect": "Allow",
      "Action": ["apigateway:*", "amplify:*"],
      "Resource": "*"
    },
    {
      "Sid": "IAMRoles",
      "Effect": "Allow",
      "Action": ["iam:*Role*", "iam:*RolePolicy*", "iam:PassRole"],
      "Resource": "arn:aws:iam::*:role/automl-lite-*"
    },
    {
      "Sid": "ServiceLinkedRoles",
      "Effect": "Allow",
      "Action": "iam:CreateServiceLinkedRole",
      "Resource": "arn:aws:iam::*:role/aws-service-role/*"
    },
    {
      "Sid": "Networking",
      "Effect": "Allow",
      "Action": ["ec2:Describe*", "ec2:*SecurityGroup*", "ec2:*Tags"],
      "Resource": "*"
    },
    {
      "Sid": "Logging",
      "Effect": "Allow",
      "Action": "logs:*",
      "Resource": "arn:aws:logs:*:*:log-group:/aws/*/automl-lite-*"
    }
  ]
}

Removed (not needed): CloudFront, X-Ray, ECS (Batch manages it internally).

Deployment Flow

Push to dev  → Auto-deploy to DEV
Push to main → Plan → Manual Approval → Deploy to PROD

Granular deployments save time:

Lambda only: ~2 min
Training container: ~3 min
Frontend: ~3 min
Full infrastructure: ~10 min

Cost Breakdown (20 jobs/month)

Service	Monthly Cost
AWS Amplify	$5-15
Lambda + API Gateway	$1-2
Batch (Fargate Spot)	$2-5
S3 + DynamoDB	$1-2
Total	$10-25

Fair comparison with SageMaker:

Training only (SageMaker): ~$0.68-3.20/month for 20 jobs—actually comparable!
Training + Endpoint (SageMaker): ~$150-300/month (ml.c5.xlarge 24/7)
AutoML Lite (all-in): $10-25/month (includes frontend, API, storage)

The headline cost difference comes from infrastructure model: AutoML Lite is fully serverless with no always-on components, while SageMaker real-time endpoints run 24/7.

Training Cost by Time: Detailed Comparison

Important context: The following comparison focuses on training costs only. SageMaker's value proposition includes managed infrastructure, model registry, A/B testing, and enterprise compliance—features that justify higher costs for production workloads.

The real cost difference lies in training time costs. Here's a detailed breakdown:

AWS AutoML Lite (Fargate Spot - 2 vCPU, 4GB RAM)

Using Fargate Spot prices for US East (N. Virginia) - December 2025:

vCPU: $0.000011244/vCPU-second → $0.0405/vCPU-hour
Memory: $0.000001235/GB-second → $0.00445/GB-hour
Fargate Spot discount: Up to 70% off on-demand prices

Training Time	vCPU Cost	Memory Cost	Total Cost
2 min (<1K rows)	$0.0027	$0.0006	$0.003
5 min (1K-10K rows)	$0.0067	$0.0015	$0.008
10 min (10K-50K rows)	$0.0135	$0.0030	$0.017
20 min (>50K rows)	$0.0270	$0.0059	$0.033
1 hour (complex model)	$0.0810	$0.0178	$0.099

SageMaker AI Training (ml.m5.xlarge - 4 vCPU, 16GB RAM)

Using SageMaker Training prices for US East (N. Virginia) - December 2025:

ml.m5.xlarge: $0.23/hour (4 vCPU, 16GB RAM)
ml.m4.4xlarge: $0.96/hour (16 vCPU, 64GB RAM)
ml.c5.xlarge: $0.204/hour (4 vCPU, 8GB RAM)
Free Tier: 50 hours of m4.xlarge or m5.xlarge (first 2 months only)

Training Time	ml.c5.xlarge	ml.m5.xlarge	ml.m4.4xlarge
2 min	$0.007	$0.008	$0.032
5 min	$0.017	$0.019	$0.080
10 min	$0.034	$0.038	$0.160
20 min	$0.068	$0.077	$0.320
1 hour	$0.204	$0.230	$0.960

Cost Comparison Summary (20 training jobs/month)

Assuming average 10 min training time per job:

Solution	Per-Job Cost	20 Jobs/Month	Annual Cost
AutoML Lite (Fargate Spot)	$0.017	$0.34	$4.08
SageMaker (ml.c5.xlarge)	$0.034	$0.68	$8.16
SageMaker (ml.m5.xlarge)	$0.038	$0.76	$9.12
SageMaker (ml.m4.4xlarge)	$0.160	$3.20	$38.40

Key insight: For pure training costs, AutoML Lite is 50-90% cheaper than equivalent SageMaker training instances due to Fargate Spot pricing. However, SageMaker training alone is affordable—the $150+/month figure refers to always-on inference endpoints, not training.

The Real Cost Driver: Inference Endpoints

Scenario	SageMaker	AutoML Lite
Training only (20 jobs, 10 min each)	$0.68-3.20/month	$0.34/month
+ Real-time endpoint (24/7)	+$150-300/month	N/A (batch only)
+ EDA reports	Manual/extra cost	Included
+ Model portability	SageMaker-locked	Download .pkl

💡 When SageMaker wins: If you need real-time inference with auto-scaling and SLA guarantees, SageMaker endpoints are worth the cost. AutoML Lite is optimized for training and batch inference scenarios.

When SageMaker Makes Sense

Despite higher costs, SageMaker excels when you need:

GPU training (ml.p3, ml.g4dn instances)
Built-in HPO (Hyperparameter Optimization)
Model Registry and versioning
A/B testing for production models
Enterprise compliance requirements

💡 Pro tip: SageMaker offers 50 free training hours on m4.xlarge/m5.xlarge for the first 2 months. Great for evaluation!

Prices as of December 2025. Always check AWS Pricing Calculator for current rates.

Feature Comparison: SageMaker vs AutoML Lite

Feature	SageMaker Autopilot	AWS AutoML Lite
Training Cost (10 min job)	$0.034-0.16	~$0.02
Real-time Inference	✅ Yes ($150+/mo)	❌ Batch only
Total Cost (20 jobs/mo)	$0.68-3.20 training only	$10-25 (all-in)
Setup Time	30+ min (Studio setup)	~15 min
Portable Models	❌ SageMaker format	✅ Download .pkl
ML Expertise Required	Medium	None
Auto Problem Detection	✅ Yes	✅ Yes
EDA Reports	❌ Manual	✅ Automatic
Infrastructure as Code	❌ Console-heavy	✅ Full Terraform
GPU Training	✅ Yes (ml.p3, ml.g4dn)	❌ CPU only
Model Registry	✅ Built-in	❌ Manual
A/B Testing	✅ Built-in	❌ Not available
Free Tier	50h training (2 months)	Fargate Spot only
Best For	Production ML pipelines	Prototyping & side projects

Using Your Trained Model

Download the .pkl file and use Docker for predictions:

# Build prediction container
docker build -f scripts/Dockerfile.predict -t automl-predict .

# Show model info
docker run --rm -v ${PWD}:/data automl-predict /data/model.pkl --info

# Predict from CSV
docker run --rm -v ${PWD}:/data automl-predict \
  /data/model.pkl -i /data/test.csv -o /data/predictions.csv

Lessons Learned

Container size matters: 265MB ML deps forced the Lambda/Batch split
Environment variable cascade: Document your data flow or debugging becomes painful
Fargate Spot is great: 70% savings, rare interruptions for short jobs
FLAML over AutoGluon: Smaller footprint, faster training, similar results

What's Next? (Future Roadmap)

[ ] ONNX Export - Deploy models to edge devices
[ ] Model Comparison - Train multiple models, compare metrics side-by-side
[ ] Real-time Updates - WebSocket instead of polling
[ ] Multi-user Support - Cognito authentication
[ ] Hyperparameter UI - Fine-tune FLAML settings from the frontend
[ ] Email Notifications - Get notified when training completes

Contributions welcome! Check the GitHub Issues for good first issues.

Try It Yourself

GitHub: cristofima/AWS-AutoML-Lite

git clone https://github.com/cristofima/AWS-AutoML-Lite.git
cd AWS-AutoML-Lite/infrastructure/terraform
terraform init && terraform apply

Building Rich Domain Models: A Practical Guide to DDD in .NET

Cristopher Coronado — Wed, 06 Aug 2025 00:25:28 +0000

Introduction

Have you ever found yourself lost in a maze of anemic domain models, where your business logic is scattered across services, and your entities are nothing more than data containers? If you're nodding along, you're not alone. Many .NET developers struggle with organizing complex business logic in a way that's both maintainable and reflects the real-world domain they're modeling.

Domain-Driven Design (DDD) offers a solution to this common problem. Rather than focusing on technical concerns first, DDD puts the business domain at the center of your application. It's not just a set of patterns—it's a way of thinking about software that aligns your code with the business reality it represents.

In this article, we'll dive deep into building rich domain models using .NET 9 and Clean Architecture principles. We'll use a real banking microservice as our example, showing you how to create entities that aren't just data holders, but intelligent objects that protect their own integrity and encapsulate business behavior.

By the end of this piece, you'll understand how to build domain models that business experts can relate to, developers can maintain, and that actually model the complexities of real-world business scenarios.

What Makes a Domain Model "Rich"?

Before we jump into code, let's understand what distinguishes a rich domain model from its anemic counterpart. An anemic domain model is essentially a collection of getters and setters with little to no business logic—the kind of code that makes you question whether you're building a business application or a simple CRUD system.

A rich domain model, on the other hand, encapsulates business rules, maintains its own consistency, and provides meaningful operations that reflect how the business actually works. Think of it this way: if you show your domain code to a business expert, they should be able to understand what's happening without needing a computer science degree.

Core Building Blocks of DDD

In DDD, we work with several key building blocks:

Entities: Objects with a distinct identity that threads through time and different states
Value Objects: Objects that describe characteristics but have no conceptual identity
Aggregate Roots: Entities that serve as entry points to aggregates and maintain consistency boundaries
Domain Events: Objects that capture something significant that happened in the domain
Domain Services: Operations that don't naturally fit within an entity or value object

Let's see how these concepts come together in a practical banking scenario.

Building the Account Aggregate: A Real-World Example

Let's start with the foundation of our banking domain—the Account aggregate. In traditional applications, you might see something like this:

// ❌ Anemic model - just data with no behavior
public class Account
{
    public Guid Id { get; set; }
    public string AccountNumber { get; set; }
    public decimal Balance { get; set; }
    public string Status { get; set; }
    public Guid CustomerId { get; set; }
}

This approach forces business logic into service classes, making it hard to ensure consistency and scattered across multiple places. Instead, let's build a rich domain model that encapsulates the real behavior of a bank account.

Creating the Account Entity

Here's how we can model an Account as a rich domain entity:

using BankSystem.Shared.Domain.Common;
using BankSystem.Shared.Domain.ValueObjects;

namespace BankSystem.Account.Domain.Entities;

/// <summary>
/// Represents a bank account aggregate root in the banking domain.
/// An account is the primary entity for managing customer finances and transactions.
/// </summary>
public class Account : AggregateRoot<Guid>
{
    /// <summary>
    /// Gets the unique account number for this account.
    /// </summary>
    public AccountNumber AccountNumber { get; } = null!;

    /// <summary>
    /// Gets the customer who owns this account.
    /// </summary>
    public Guid CustomerId { get; }

    /// <summary>
    /// Gets the current balance of the account.
    /// </summary>
    public Money Balance { get; private set; } = null!;

    /// <summary>
    /// Gets the current status of the account.
    /// </summary>
    public AccountStatus Status { get; private set; }

    /// <summary>
    /// Gets the type of account (Checking, Savings, etc.).
    /// </summary>
    public AccountType Type { get; private set; }

    /// <summary>
    /// Gets the date and time when the account was closed (if applicable).
    /// </summary>
    public DateTime? ClosedAt { get; private set; }

    // Private constructor for EF Core
    private Account() { }

    /// <summary>
    /// Private constructor that ensures accounts are created through factory methods
    /// </summary>
    private Account(
        AccountNumber accountNumber,
        Guid customerId,
        AccountType type,
        Currency currency)
    {
        Guard.AgainstNullOrEmpty(accountNumber, nameof(accountNumber));
        Guard.AgainstNullOrEmpty(currency, nameof(currency));
        Guard.AgainstEmptyGuid(customerId, nameof(customerId));

        Id = Guid.NewGuid();
        AccountNumber = accountNumber;
        CustomerId = customerId;
        Type = type;
        Balance = Money.Zero(currency);
        Status = AccountStatus.PendingActivation;
    }

    /// <summary>
    /// Factory method to create a new account for a customer.
    /// This is the only way to create a new account, ensuring all business rules are followed.
    /// </summary>
    public static Account CreateNew(Guid customerId, AccountType type, Currency currency)
    {
        var accountNumber = AccountNumber.Generate();
        var account = new Account(accountNumber, customerId, type, currency);

        // Raise domain event - this is important for event-driven architecture
        account.AddDomainEvent(
            new AccountCreatedEvent(
                account.Id,
                customerId,
                account.AccountNumber,
                type.ToString(),
                DateTime.UtcNow
            )
        );

        return account;
    }

    /// <summary>
    /// Business operation: Activates the account
    /// Notice how business rules are encapsulated within the entity
    /// </summary>
    public Result Activate()
    {
        if (Status == AccountStatus.Active)
            return Result.Failure("Account is already active");

        if (Status == AccountStatus.Closed)
            return Result.Failure("Cannot activate a closed account");

        Status = AccountStatus.Active;

        AddDomainEvent(
            new AccountActivatedEvent(Id, CustomerId, AccountNumber.Value, DateTime.UtcNow)
        );

        return Result.Success();
    }

    /// <summary>
    /// Business operation: Suspends the account with business logic validation
    /// </summary>
    public Result Suspend(string reason)
    {
        Guard.AgainstNullOrEmpty(reason, nameof(reason));

        if (Status == AccountStatus.Closed)
            return Result.Failure("Cannot suspend a closed account");

        Status = AccountStatus.Suspended;

        AddDomainEvent(new AccountSuspendedEvent(Id, reason, DateTime.UtcNow));
        return Result.Success();
    }

    /// <summary>
    /// Business operation: Closes the account with business constraints
    /// </summary>
    public Result Close(string reason)
    {
        Guard.AgainstNullOrEmpty(reason, nameof(reason));

        if (Status == AccountStatus.Closed)
            return Result.Failure("Account is already closed");

        // Business rule: Cannot close account with outstanding balance
        if (!Balance.IsZero)
            return Result.Failure("Cannot close account with non-zero balance");

        Status = AccountStatus.Closed;
        ClosedAt = DateTime.UtcNow;

        AddDomainEvent(new AccountClosedEvent(Id, AccountNumber, CustomerId, reason));
        return Result.Success();
    }

    /// <summary>
    /// Business operation: Processes a deposit transaction
    /// </summary>
    public Result<Transaction> Deposit(Money amount, string description, string reference)
    {
        if (Status != AccountStatus.Active)
            return Result<Transaction>.Failure("Account must be active to process deposits");

        if (amount.Amount <= 0)
            return Result<Transaction>.Failure("Deposit amount must be positive");

        if (amount.Currency != Balance.Currency)
            return Result<Transaction>.Failure("Currency mismatch");

        Balance = Balance.Add(amount);

        var transaction = Transaction.CreateDeposit(Id, amount, description, reference);

        AddDomainEvent(new DepositProcessedEvent(Id, amount, Balance, transaction.Id));

        return Result<Transaction>.Success(transaction);
    }

    /// <summary>
    /// Business operation: Processes a withdrawal with overdraft protection
    /// </summary>
    public Result<Transaction> Withdraw(Money amount, string description, string reference)
    {
        if (Status != AccountStatus.Active)
            return Result<Transaction>.Failure("Account must be active to process withdrawals");

        if (amount.Amount <= 0)
            return Result<Transaction>.Failure("Withdrawal amount must be positive");

        if (amount.Currency != Balance.Currency)
            return Result<Transaction>.Failure("Currency mismatch");

        // Business rule: Overdraft protection
        if (Balance.Amount < amount.Amount)
            return Result<Transaction>.Failure("Insufficient funds");

        Balance = Balance.Subtract(amount);

        var transaction = Transaction.CreateWithdrawal(Id, amount, description, reference);

        AddDomainEvent(new WithdrawalProcessedEvent(Id, amount, Balance, transaction.Id));

        return Result<Transaction>.Success(transaction);
    }
}

Notice how this entity is vastly different from our anemic example. It:

Protects its invariants: Private setters and factory methods ensure the object is always in a valid state
Encapsulates business logic: Operations like Activate(), Suspend(), and Close() contain the actual business rules
Communicates intent: Method names reflect business operations, not just data manipulation
Publishes domain events: When important business events occur, the entity announces them

Value Objects: The Building Blocks of Rich Models

Value objects are crucial for creating expressive domain models. They represent concepts that have no identity but are important to the domain. Let's look at two key value objects in our banking domain:

AccountNumber Value Object

namespace BankSystem.Account.Domain.ValueObjects;

/// <summary>
/// Represents a bank account number with validation and formatting logic.
/// This is a value object - it has no identity, just value.
/// </summary>
public record AccountNumber
{
    private const int AccountNumberLength = 10;
    private static readonly Random Random = new();
    private static readonly Lock RandomLock = new();

    public string Value { get; }

    public AccountNumber(string value)
    {
        if (string.IsNullOrWhiteSpace(value))
            throw new DomainException("Account number cannot be null or empty");

        var cleanValue = value.Trim();

        if (!IsValidFormat(cleanValue))
            throw new DomainException($"Invalid account number format: {value}. Expected 10 digits");

        Value = cleanValue;
    }

    /// <summary>
    /// Factory method to generate a new unique account number
    /// This encapsulates the business logic of how account numbers are created
    /// </summary>
    public static AccountNumber Generate()
    {
        var digits = new char[AccountNumberLength];

        lock (RandomLock)
        {
            for (var i = 0; i < AccountNumberLength; i++)
            {
                digits[i] = (char)('0' + Random.Next(0, 10));
            }
        }

        return new AccountNumber(new string(digits));
    }

    /// <summary>
    /// Validates the account number format according to business rules
    /// </summary>
    private static bool IsValidFormat(string value)
    {
        return value.Length == AccountNumberLength &&
               value.All(char.IsDigit);
    }

    /// <summary>
    /// Provides formatted display for different contexts
    /// </summary>
    public string ToFormattedString()
    {
        return $"{Value[..3]}-{Value[3..6]}-{Value[6..]}";
    }

    // Record types automatically provide value-based equality
    public override string ToString() => Value;

    // Implicit conversion for convenience
    public static implicit operator string(AccountNumber accountNumber) => accountNumber.Value;
}

Money Value Object

namespace BankSystem.Shared.Domain.ValueObjects;

/// <summary>
/// Represents monetary amounts with currency information.
/// This prevents mixing currencies and provides financial operations.
/// </summary>
public record Money
{
    public decimal Amount { get; init; }
    public Currency Currency { get; init; }

    // Private constructor for EF Core
    private Money() { }

    public Money(decimal amount, Currency currency)
    {
        ArgumentNullException.ThrowIfNull(currency);

        // Enforce currency precision rules
        if (decimal.Round(amount, currency.DecimalPlaces) != amount)
            throw new DomainException($"Amount has too many decimal places for currency {currency.Code}");

        Amount = amount;
        Currency = currency;
    }

    public static Money Zero(Currency currency) => new(0, currency);

    /// <summary>
    /// Adds two Money values. Both must have the same currency.
    /// This business rule is enforced at the domain level.
    /// </summary>
    public Money Add(Money other)
    {
        if (!Currency.Equals(other.Currency))
            throw new DomainException($"Cannot add {Currency.Code} to {other.Currency.Code}");

        return new Money(Amount + other.Amount, Currency);
    }

    /// <summary>
    /// Subtracts two Money values with currency validation
    /// </summary>
    public Money Subtract(Money other)
    {
        if (!Currency.Equals(other.Currency))
            throw new DomainException($"Cannot subtract {other.Currency.Code} from {Currency.Code}");

        return new Money(Amount - other.Amount, Currency);
    }

    /// <summary>
    /// Multiplies money by a factor (useful for interest calculations)
    /// </summary>
    public Money Multiply(decimal factor)
    {
        return new Money(Amount * factor, Currency);
    }

    /// <summary>
    /// Business method to check if the amount is zero
    /// </summary>
    public bool IsZero => Amount == 0;

    /// <summary>
    /// Business method to check if the amount is positive
    /// </summary>
    public bool IsPositive => Amount > 0;

    /// <summary>
    /// Business method to check if the amount is negative
    /// </summary>
    public bool IsNegative => Amount < 0;

    /// <summary>
    /// Formatted string representation for display
    /// </summary>
    public override string ToString()
    {
        return $"{Amount:C} {Currency.Code}";
    }
}

Domain Events: Capturing What Happened

Domain events are crucial for building event-driven systems and maintaining loose coupling between bounded contexts. They represent something significant that happened in the domain.

namespace BankSystem.Account.Domain.Events;

/// <summary>
/// Domain event raised when a new account is created.
/// This allows other parts of the system to react to account creation.
/// </summary>
public record AccountCreatedEvent(
    Guid AccountId,
    Guid CustomerId,
    AccountNumber AccountNumber,
    string AccountType,
    DateTime CreatedAt) : IDomainEvent
{
    public Guid Id { get; } = Guid.NewGuid();
    public DateTime OccurredOn { get; } = DateTime.UtcNow;
}

/// <summary>
/// Domain event for account activation
/// </summary>
public record AccountActivatedEvent(
    Guid AccountId,
    Guid CustomerId,
    string AccountNumber,
    DateTime ActivatedAt) : IDomainEvent
{
    public Guid Id { get; } = Guid.NewGuid();
    public DateTime OccurredOn { get; } = DateTime.UtcNow;
}

/// <summary>
/// Domain event for deposit transactions
/// </summary>
public record DepositProcessedEvent(
    Guid AccountId,
    Money Amount,
    Money NewBalance,
    Guid TransactionId) : IDomainEvent
{
    public Guid Id { get; } = Guid.NewGuid();
    public DateTime OccurredOn { get; } = DateTime.UtcNow;
}

The Aggregate Root Pattern

The AggregateRoot base class provides the infrastructure for handling domain events:

namespace BankSystem.Shared.Domain.Common;

/// <summary>
/// Base class for aggregate roots that manage domain events
/// </summary>
public abstract class AggregateRoot<TId> : Entity<TId>
{
    private readonly List<IDomainEvent> _domainEvents = [];

    /// <summary>
    /// Domain events produced by this aggregate.
    /// These will be published after successful persistence.
    /// </summary>
    public IReadOnlyCollection<IDomainEvent> DomainEvents => _domainEvents.AsReadOnly();

    /// <summary>
    /// Adds a domain event to be published later
    /// </summary>
    protected void AddDomainEvent(IDomainEvent domainEvent)
    {
        _domainEvents.Add(domainEvent);
    }

    /// <summary>
    /// Clears all domain events (typically called after publishing)
    /// </summary>
    public void ClearDomainEvents() => _domainEvents.Clear();
}

/// <summary>
/// Base class for all entities in the domain
/// </summary>
public abstract class Entity<TId>
{
    public TId Id { get; protected set; } = default!;

    public override bool Equals(object? obj)
    {
        if (obj is not Entity<TId> entity || GetType() != entity.GetType())
            return false;

        return EqualityComparer<TId>.Default.Equals(Id, entity.Id);
    }

    public override int GetHashCode()
    {
        return EqualityComparer<TId>.Default.GetHashCode(Id);
    }
}

How Other Layers Interact with the Domain

One of the beautiful aspects of Clean Architecture is how the domain layer remains independent while other layers depend on it. Let's see how this works in practice:

Application Layer Usage

// Command handler in the Application layer
public class CreateAccountCommandHandler : IRequestHandler<CreateAccountCommand, Result<AccountDto>>
{
    private readonly IAccountRepository _accountRepository;
    private readonly ICurrentUser _currentUser;

    public async Task<Result<AccountDto>> Handle(CreateAccountCommand request, CancellationToken cancellationToken)
    {
        // Use domain factory method
        var account = Account.CreateNew(
            _currentUser.UserId,
            request.AccountType,
            new Currency(request.Currency));

        // Persist using repository abstraction
        await _accountRepository.AddAsync(account, cancellationToken);

        // Domain events will be published by infrastructure
        return Result<AccountDto>.Success(AccountDto.FromDomain(account));
    }
}

Infrastructure Layer Implementation

The key to maintaining clean separation of concerns is to automatically publish domain events when Entity Framework saves changes, rather than coupling this responsibility to repositories.

// Clean repository implementation - no event publishing concerns
public class AccountRepository : IAccountRepository
{
    private readonly AccountDbContext _context;

    public AccountRepository(AccountDbContext context)
    {
        _context = context;
    }

    public async Task<Account?> GetByIdAsync(Guid id, CancellationToken cancellationToken)
    {
        return await _context.Accounts
            .FirstOrDefaultAsync(a => a.Id == id, cancellationToken);
    }

    public async Task AddAsync(Account account, CancellationToken cancellationToken)
    {
        _context.Accounts.Add(account);
        // No event publishing here - it's handled automatically by EF interceptors
        await _context.SaveChangesAsync(cancellationToken);
    }

    public async Task UpdateAsync(Account account, CancellationToken cancellationToken)
    {
        _context.Accounts.Update(account);
        await _context.SaveChangesAsync(cancellationToken);
    }
}

Automatic Domain Event Publishing with EF Core Interceptors

To achieve true separation of concerns, we use Entity Framework interceptors to automatically publish domain events whenever changes are saved:

// Domain Event Publishing Interceptor
public class DomainEventPublishingInterceptor : SaveChangesInterceptor
{
    private readonly IDomainEventPublisher _eventPublisher;
    private readonly ILogger<DomainEventPublishingInterceptor> _logger;

    public DomainEventPublishingInterceptor(
        IDomainEventPublisher eventPublisher,
        ILogger<DomainEventPublishingInterceptor> logger)
    {
        _eventPublisher = eventPublisher;
        _logger = logger;
    }

    public override async ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            await PublishDomainEventsAsync(eventData.Context, cancellationToken);
        }

        return await base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private async Task PublishDomainEventsAsync(DbContext context, CancellationToken cancellationToken)
    {
        // Find all aggregate roots with domain events
        var aggregateRoots = context.ChangeTracker.Entries<AggregateRoot<Guid>>()
            .Where(e => e.Entity.DomainEvents.Any())
            .Select(e => e.Entity)
            .ToList();

        // Collect all domain events
        var domainEvents = aggregateRoots
            .SelectMany(ar => ar.DomainEvents)
            .ToList();

        // Clear events from aggregates before publishing to prevent duplicate publishing
        foreach (var aggregateRoot in aggregateRoots)
        {
            aggregateRoot.ClearDomainEvents();
        }

        // Publish each domain event
        foreach (var domainEvent in domainEvents)
        {
            try
            {
                await _eventPublisher.PublishAsync(domainEvent, cancellationToken);
                _logger.LogInformation("Successfully published domain event {EventType} with ID {EventId}",
                    domainEvent.GetType().Name, domainEvent.Id);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed to publish domain event {EventType} with ID {EventId}",
                    domainEvent.GetType().Name, domainEvent.Id);

                // Re-add events to aggregate if publishing fails
                // This ensures events aren't lost if there's a publishing failure
                foreach (var aggregateRoot in aggregateRoots)
                {
                    aggregateRoot.AddDomainEvent(domainEvent);
                }

                throw; // Re-throw to prevent saving if event publishing fails
            }
        }
    }
}

DbContext Configuration

The DbContext configuration registers the interceptor to automatically handle event publishing:

public class AccountDbContext : DbContext
{
    public DbSet<Account> Accounts { get; set; } = null!;

    public AccountDbContext(DbContextOptions<AccountDbContext> options) : base(options) { }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        // Configure Account entity
        modelBuilder.Entity<Account>(entity =>
        {
            entity.HasKey(a => a.Id);

            // Configure value objects
            entity.OwnsOne(a => a.AccountNumber, an =>
            {
                an.Property(x => x.Value)
                  .HasColumnName("AccountNumber")
                  .HasMaxLength(10)
                  .IsRequired();
            });

            entity.OwnsOne(a => a.Balance, money =>
            {
                money.Property(m => m.Amount)
                     .HasColumnName("Balance")
                     .HasPrecision(18, 2);

                money.OwnsOne(m => m.Currency, currency =>
                {
                    currency.Property(c => c.Code)
                           .HasColumnName("Currency")
                           .HasMaxLength(3);
                });
            });

            // Ignore domain events - they're not persisted
            entity.Ignore(a => a.DomainEvents);
        });
    }
}

Dependency Injection Setup

Finally, register everything in your DI container:

// In Program.cs or ServiceCollectionExtensions
public static IServiceCollection AddInfrastructure(
    this IServiceCollection services,
    IConfiguration configuration)
{
    // Register the domain event publisher
    services.AddScoped<IDomainEventPublisher, MassTransitDomainEventPublisher>();

    // Register the EF interceptor
    services.AddScoped<DomainEventPublishingInterceptor>();

    // Register DbContext with the interceptor
    services.AddDbContext<AccountDbContext>((serviceProvider, options) =>
    {
        var connectionString = configuration.GetConnectionString("DefaultConnection");
        var interceptor = serviceProvider.GetRequiredService<DomainEventPublishingInterceptor>();

        options.UseSqlServer(connectionString)
               .AddInterceptors(interceptor);
    });

    // Register repositories
    services.AddScoped<IAccountRepository, AccountRepository>();

    return services;
}

Benefits You'll Experience

By implementing rich domain models with automatic event publishing, you'll start noticing several significant benefits:

1. Business Logic Centralization

All rules related to accounts live in the Account entity. There's no scattered business logic across services, making the codebase more maintainable and easier to understand.

2. Improved Testability

You can unit test business logic without any infrastructure concerns. Domain entities can be tested in isolation, and the event publishing mechanism can be tested separately.

[Test]
public void Account_Deposit_Should_Raise_DepositProcessedEvent()
{
    // Arrange
    var account = Account.CreateNew(Guid.NewGuid(), AccountType.Checking, Currency.USD);
    account.Activate();

    // Act
    var result = account.Deposit(new Money(100, Currency.USD), "Test deposit", "REF123");

    // Assert
    Assert.That(result.IsSuccess, Is.True);
    Assert.That(account.DomainEvents.Count, Is.EqualTo(2)); // AccountCreated + DepositProcessed
    Assert.That(account.DomainEvents.OfType<DepositProcessedEvent>().Count(), Is.EqualTo(1));
}

3. Better Communication with Business Experts

Business experts can read and understand the domain code. The ubiquitous language is embedded directly in the code structure.

4. Automatic Consistency Guarantees

Invariants are always maintained within aggregates
Domain events are published automatically after successful persistence
No risk of forgetting to publish events or publishing them incorrectly

5. True Event-Driven Capabilities

The EF interceptor approach enables:

Automatic event publishing: No manual intervention required
Transactional consistency: Events are only published if database changes succeed
Clean separation: Repository logic remains focused on data persistence
Loose coupling: Domain events enable eventual consistency across bounded contexts

6. Resilient Event Publishing

The interceptor pattern provides built-in resilience:

Events are cleared before publishing to prevent duplicates
Failed event publishing prevents database commits
Events are re-added to aggregates if publishing fails, ensuring no loss

Common Pitfalls to Avoid

As you start implementing rich domain models, watch out for these common mistakes:

Overly Large Aggregates: Keep aggregates focused and small. If an aggregate is doing too much, consider splitting it
Missing Business Rules: Don't let business logic leak into application services
Anemic Events: Domain events should be meaningful business events, not just "data changed" notifications
Ignoring Ubiquitous Language: Use the same terms that business experts use

Conclusion

Rich domain models are the heart of any successful Domain-Driven Design implementation. By encapsulating business logic within entities and value objects, we create code that not only works but also communicates intent clearly and maintainably.

The Account aggregate we've built demonstrates how to:

Protect business invariants through careful encapsulation
Express business operations as meaningful methods
Use value objects to create type-safe, expressive APIs
Leverage domain events for loose coupling and event-driven architecture

In our next article, we'll explore how to publish these domain events using MassTransit and Azure Service Bus, enabling true event-driven microservices architecture. We'll see how the events generated by our rich domain models can trigger workflows across different bounded contexts, maintaining consistency while keeping services loosely coupled.

Remember, DDD isn't just about patterns—it's about creating software that reflects and supports the business domain it serves. Rich domain models are your first step toward achieving that goal.

Next in the series: "Publishing Domain Events with MassTransit and Azure Service Bus" - Learn how to build event-driven microservices that react to domain changes.

📋 Resources

Generating Images with Amazon Bedrock

Cristopher Coronado — Sun, 02 Feb 2025 20:10:25 +0000

Introduction

Amazon Bedrock is a powerful AI service that enables developers to generate text, images, and more using foundation models. This guide walks through the implementation of an API and a simple UI to generate images using Amazon Bedrock.

Development

Prerequisites

Before starting, ensure you have:

Node.js 18+
AWS IAM permissions (AmazonBedrockFullAccess)
Access to available Bedrock foundation models (FMs)

In order to make sure you have been granted access to the Amazon models, go to Amazon Bedrock / Request model access. Choose the models from the Amazon tab and follow the steps in the wizard to save the configuration.

Clone this GitHub repository.

In the .env file paste your AWS credentials.

AWS_REGION=your-region
AWS_ACCESS_KEY=your-access-key
AWS_SECRET_KEY=your-secret-key

In the API/src/index.ts file you can modify the existing configuration to generate the images.

You can find documentation about it in the links below:

const input = {
      contentType: "application/json",
      accept: "*/*",
      modelId: "amazon.titan-image-generator-v2:0",
      body: JSON.stringify({
        taskType: "TEXT_IMAGE",
        textToImageParams: { text: prompt },
        imageGenerationConfig: {
          numberOfImages: 1,
          quality: "standard",
          cfgScale: 8.0,
          height: 512,
          width: 512,
          seed: 0,
        },
      }),
    };

To test the demo. Run npx ts-node src/index.ts from API folder, and npm run dev from UI.

Here are some examples of the images generated and their prompts.

Conclusion

Amazon Bedrock provides a scalable and efficient way to generate images using advanced AI models. By integrating it with Node.js and React, developers can quickly build applications with AI-powered image generation capabilities. With further enhancements, such as fine-tuning models or expanding UI functionalities, this solution can be adapted to various creative and business needs.

Thanks for reading

Thank you very much for reading. I hope you found this article interesting and may be useful in the future. If you have any questions or ideas you need to discuss, it will be a pleasure to collaborate and exchange knowledge.

Building an Image Moderation System with AWS Rekognition, Nest.js, and React

Cristopher Coronado — Mon, 30 Dec 2024 16:33:21 +0000

Introduction

Images abundant on web platforms in today's digital environment. Images can be an effective means of expression and communication, but they can also be used to spread harmful content such hate speech, violence, and nudity. Image moderation is useful in this situation. Automatic picture analysis is used by image moderation systems to find and mark potentially offensive or dangerous content.
This post will discuss how to use AWS Rekognition, Nest.js, and React to create a reliable image moderation system.Conclusion

Development

We created a demo application to demonstrate the capabilities of AWS Rekognition. A React frontend serves as the user interface in this demo, while a Nest.js backend API manages the laborious task of content analysis.

For this demo, we will use this GitHub repository where you can set the settings such as API Key, connection string, etc.

Prerequisites:

AWS Credits
Node.js
MongoDB database

First, we need to create a S3 Bucket and enable the public access. After that, create a folder called images.

Once it's created, add the following Bucket policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>/images/*"
        }
    ]
}

Then go to Security Credentials and create two users (in the same AWS Region) with the following permissions:

User 1 (AWS-Rekognition): AmazonRekognitionFullAccess and AmazonS3ReadOnlyAccess
User 2 (AWS-S3): AmazonS3FullAccess

Generate an access key and secret key for both and paste them in the .env file from the API project.

MONGO_URI=

AWS_S3_ACCESS_KEY=
AWS_S3_SECRET_ACCESS_KEY=
AWS_S3_REGION=
AWS_S3_BUCKET_NAME=
AWS_S3_BUCKET_FOLDER=

AWS_REKOGNITION_REGION=
AWS_REKOGNITION_ACCESS_KEY=
AWS_REKOGNITION_SECRET_ACCESS_KEY=

Create a MongoDB and add the connection string in the MONGO_URI key.
In the UI project you need to add base url for the API in REACT_APP_API_BASE_URL key.

Rekognition Integration:
Use the AWS SDK to call the Rekognition DetectModerationLabels API. This API analyzes the image and returns a list of detected labels, including confidence scores.
Define rules based on the detected labels and confidence scores to determine if an image is flagged as inappropriate. Check the constants/category-thesholds.contstant.ts file in API project to see the thresholds per category.

See this documentation to know more about the label categories.

This is an example for the response returned from AWS Rekognition:

{
    "ModerationLabels": [
        {
            "Confidence": 99.44782257080078,
            "Name": "Smoking",
            "ParentName": "Drugs & Tobacco Paraphernalia & Use",
            "TaxonomyLevel": 3
        },
        {
            "Confidence": 99.44782257080078,
            "Name": "Drugs & Tobacco Paraphernalia & Use",
            "ParentName": "Drugs & Tobacco",
            "TaxonomyLevel": 2
        },
        {
            "Confidence": 99.44782257080078,
            "Name": "Drugs & Tobacco",
            "ParentName": "",
            "TaxonomyLevel": 1
        }
    ],
    "ModerationModelVersion": "7.0",
    "ContentTypes": [
        {
            "Confidence": 99.9999008178711,
            "Name": "Illustrated"
        }
    ]
}

To test the demo, you submit some images.

If the image content is categorized as harmful (based on threshold categories) in any category, it will be blurred by default. But you can turn off the blur by clicking the button displayed in the center for each image.

Images are rejected if the rejection threshold is equal to or greater than the confidence score of a category.

Conclusion

A strong and effective picture moderation system can be created by utilizing the capabilities of AWS Rekognition, Nest.js, and React. By proactively identifying and eliminating dangerous information, this approach can assist online platforms in making the internet a safer and more enjoyable place for all users.
This is a basic example, and you can further enhance this system by adding features like:

Customizable moderation rules: Allow administrators to define and adjust moderation rules based on their specific needs.
Human review: Implement a human review process for images that are flagged by the system.

Thanks for reading

Detecting harm content in text and images using Azure AI Content Safety

Cristopher Coronado — Thu, 26 Dec 2024 15:59:48 +0000

Introduction

Social media platforms have become double-edged swords in the last years. They provide us with worldwide connections, but they can also act as havens for dangerous content. Tech companies have resorted to artificial intelligence to protect online communities from this escalating problem. Azure AI Content Safety, a service made to recognize and flag hazardous content, is a potent tool.

Development

We created a demo application to demonstrate the capabilities of Azure AI Content Safety. An Angular frontend serves as the user interface in this demo, while a.NET backend API manages the laborious task of content analysis.

For this demo we will use this GitHub repository where you can set the settings such as API Key, connection string, etc.

Prerequisites:

Azure subscription
.NET SDK
Angular SDK
SQL Server database

First, let’s create the Azure resources. Go to the Azure portal and create the Content Safety resource.

Once the resource is created, go to Keys and Endpoint under Resource Management and copy the key and endpoint.

These credentails must be pasted in the appsettings.Development.json file in the AzureAIContentSafety.API project.

{
   "AzureAIContentSafety":{
      "Endpoint":"https://<SERVICE_NAME>.cognitiveservices.azure.com/",
      "ApiKey":"",
      "TextSeverityThreshold":{
         "Blur":3,
         "Reject":5
      },
      "ImageSeverityThreshold":{
         "Blur":2,
         "Reject":4
      }
   }
}

Now, create the Storage Account. Follow this configuration in the Basics tab.

Once the resource is created, go to Configuration under Settings and enable the blob anonymous access.

Then, go to Containers under Data storage and create the container images and chose the Anonymous access level to Blob.

Now, you must copy the connection string to your Storage account. So, go to Access keys under Security + networking. Click on Show for the connection string key1.

In the same appsettings.Development.json file, paste it.

{
   "AzureStorage":{
      "BlobCacheControl":"max-age=21600",
      "BlobContainerName":"images",
      "ConnectionString":""
   }
}

Here's how the solution works:

User Interaction: A user can upload an image or enter text to interact with the Angular frontend.
API Request: The user-submitted content is sent to the.NET API by the Angular application.
Content Analysis: To check for possible harm, the .NET API then makes use of the Azure AI Content Safety service. Four main categories are covered in this analysis:
- Hate: Content that incites hatred or discrimination against groups
- Sexual: Content that is exploitative, explicit, or sexually suggestive
- Violence: Information that promotes or shows violence
- Self-Harm: Information that encourages or promotes self-harm
Flagging and Moderation: The AI marks content that it believes to be harmful so that human moderators can examine it further. This guarantees that suitable measures, like deletion or user suspension, can be implemented.

The severities levels for the harm categories are in a scale of 0 to 7, but for images the classifier only returns severities 0, 2, 4, and 6.

Also, for texts, if the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.

[0, 1] -> 0
[2, 3] -> 2
[4, 5] -> 4
[6, 7] -> 6

You can adjust the severities thresholds based on your needs in the appsettings.Development.json file.

For testing the demo, you can either submit text, image or both.

If the image content was categorized as harmful in any category, it will be blurred by default. But you can turn on/off the blur from the button displayed at the top right corner for each image.

Conclusion

Even though AI-powered solutions like Azure AI Content Safety are a big improvement, it's crucial to keep in mind that they are not panacea. To ensure accurate and moral content control, human oversight remains required. We can make the internet a safer and more welcoming place by combining human knowledge with AI's ability. We might expect even more advanced tools to appear as technology develops further, assisting us in managing the challenges of the digital age.

Thanks for reading

Mastering NLP with AWS Comprehend and Python

Cristopher Coronado — Wed, 21 Feb 2024 17:46:25 +0000

AWS Comprehend is a natural language processing (NLP) service that uses machine learning to analyze text and extract insights. It can perform tasks such as named entity recognition, key phrase extraction, sentiment analysis, topic modeling, and language detection.

In this article, we will use Python and the AWS SDK for Python (Boto3) to interact with AWS Comprehend and perform some common NLP tasks. We will need an AWS account and an IAM user with the appropriate permissions to access AWS Comprehend.

Overview of NLP tasks

Named Entity Recognition

Named entity recognition (NER) is the process of identifying and categorizing entities in a text, such as people, places, organizations, dates, etc. AWS Comprehend can recognize up to 12 types of entities, and return their text, type, and score.

Key Phrase Extraction

Key phrase extraction is the process of identifying and extracting the most important phrases in a text, such as nouns and noun phrases. AWS Comprehend can return the text and the score of the key phrases.

Sentiment Analysis

Sentiment analysis is the process of determining a text's emotional tone or attitude, such as positive, negative, neutral, or mixed. AWS Comprehend can return the overall sentiment of the text, as well as the scores of each sentiment category.

Getting started

From your AWS Console go to Security Credentials and create a user attaching the ComprehendReadOnly permission policy.

Once the user is created go to Security Credentials for this user and create an access key. Copy the access and secret keys.

Now, use these keys in the Python Notebook:

import boto3

boto_session = boto3.Session(
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    region_name='us-west-2')

comprehend_client = boto_session.client('comprehend')

text = "Nestled in Kyoto, the Kinkaku-ji Temple, covered in shimmering gold leaf, exudes cultural richness and serenity. Visitors find profound beauty in this iconic site, appreciating its historical significance and tranquil surroundings."

# Perform Named Entity recognition
response = comprehend_client.detect_entities(Text=text, LanguageCode='en')

# Display the detected entities
print("Detected Entities:")
for entity in response['Entities']:
    print(f"Type: {entity['Type']}, Text: {entity['Text']}")

# Perform the Key Phrases detection
response = comprehend_client.detect_key_phrases(
    Text= text,
    LanguageCode='en'
)

# Display the detected Key Phrases
print("Detected Key Phrases:")
for keyPhrase in response['KeyPhrases']:
    print(f"Text: {keyPhrase['Text']}, Score: {round(keyPhrase['Score'], 2) * 100} %")

# Perform the Sentiment detection
response = comprehend_client.detect_sentiment(
    Text= text,
    LanguageCode='en'
)

# Display the detected Sentiment
print("Detected Sentiments:")
main_sentiment = response['Sentiment'].capitalize()
score = response['SentimentScore'][main_sentiment]
print(f"Main Sentiment: {main_sentiment}, Score: {round(score, 2) * 100} %")
print("Other sentiments:")

sentiments = ["Positive", "Negative", "Neutral", "Mixed"]
for sentiment in sentiments:
  if sentiment == main_sentiment:
    continue

  score = response['SentimentScore'][sentiment]
  score = round(score, 2) * 100
  if score > 0:
    print(f"Sentiment: {sentiment}, Score: {score} %")

The output is the following:

Named Entity Recognition

Type: LOCATION, Text: Kyoto
Type: LOCATION, Text: Kinkaku-ji Temple

Key Phrase Extraction

Text: Kyoto, Score: 100.0 %
Text: the Kinkaku-ji Temple, Score: 100.0 %
Text: gold leaf, Score: 100.0 %
Text: cultural richness and serenity, Score: 99.0 %
Text: Visitors, Score: 100.0 %
Text: profound beauty, Score: 100.0 %
Text: this iconic site, Score: 100.0 %
Text: its historical significance, Score: 100.0 %
Text: tranquil surroundings, Score: 95.0 %

Sentiment Analysis

Main Sentiment: Positive, Score: 66.0 %
Other sentiments:
Sentiment: Neutral, Score: 34.0 %

Conclusion

In this article, we learned how to use AWS Comprehend with Python to perform some common NLP tasks, such as named entity recognition, keyphrase extraction, and sentiment analysis. We used the Boto3 library to create a Comprehend client and call the relevant methods with the text and the language code as parameters. We saw that AWS Comprehend can return useful and accurate information about the text, such as the entities, key phrases, and sentiment.

AWS Comprehend is a powerful and easy-to-use service that can help us analyze text and extract insights. It can be used for various applications, such as content analysis, customer feedback, social media monitoring, and more. We encourage you to explore more of its features and capabilities and see how it can enhance your projects.

Thanks for reading

Introducing Amazon Polly: How to Use Text-to-Speech service

Cristopher Coronado — Thu, 18 Jan 2024 17:48:44 +0000

In today's digital age, businesses are increasingly looking for ways to improve their customer experience and operational efficiency. One way to do this is through the use of natural language processing (NLP) services, such as Amazon Polly. Amazon Polly is a text-to-speech service that allows you to convert text into lifelike speech. In this blog post, we'll provide an overview of these these services and show you how to get started using it.

Overview of Amazon Polly

Amazon Polly is a powerful service that allows you to create lifelike speech from text. It supports a wide variety of languages and voices, and you can customize the pronunciation, pitch, and speed of the speech to suit your needs. This makes it an ideal choice for businesses that want to create personalized voice messages or voice-overs for videos.
Some of the key features and benefits of Amazon Polly include:

High-quality speech synthesis: Amazon Polly uses advanced deep learning algorithms to create lifelike speech that is almost indistinguishable from human speech.
Wide language support: Amazon Polly supports more than 60 languages and dialects, including English, Spanish, French, German, and Japanese.
Customizable speech: You can customize the pronunciation, pitch, and speed of the speech to suit your needs. This allows you to create a unique voice that represents your brand.
Easy integration: Amazon Polly is easy to integrate with your applications and workflows, thanks to its API and SDKs.

Getting started

From your AWS Console go to Security Credentials and create a user attaching the AmazonPollyFullAccess permission policy.

Once the user is created go to Security Credentials for this user and create access key. Copy the access and secret keys.

Now, use these keys in the Python Notebook:

import boto3
from contextlib import closing
import os
import sys
import subprocess

boto_session = boto3.Session(
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    region_name='us-west-2')

# Create a new Polly client
polly_client = boto_session.client('polly')

# Set the text that you want to synthesize
text = "Hello, welcome to Amazon Polly!"

# Synthesize speech with Amazon Polly
response = polly_client.synthesize_speech(
    Text=text,
    OutputFormat="mp3",
    VoiceId="Joanna"
)

# Access the audio stream from the response
if "AudioStream" in response:
    with closing(response["AudioStream"]) as stream:
        output = os.path.join(os.getcwd(), "speech.mp3")

        try:
            # Open a file for writing the output as a binary stream
            with open(output, "wb") as file:
                file.write(stream.read())
        except IOError as error:
            # Could not write to file, exit gracefully
            print(error)
            sys.exit(-1)

else:
    # The response didn't contain audio data, exit gracefully
    print("Could not stream audio")
    sys.exit(-1)

# Play the audio using the platform's default player
if sys.platform == "win32":
    os.startfile(output)
else:
    # The following works on macOS and Linux. (Darwin = mac, xdg-open = linux).
    opener = "open" if sys.platform == "darwin" else "xdg-open"
    subprocess.call([opener, output])

Conclusion

Amazon Polly is a powerful tool that can help you generate high-quality speech for your applications. With just a few lines of code, you can synthesize speech from text. These tool can be used to create voiceovers for videos, generate audio versions of articles or blog posts.

Thanks for reading

Python-Powered Number Plate Recognition: Unleashing the Potential of Azure Custom Vision and Azure Computer Vision

Cristopher Coronado — Wed, 14 Jun 2023 19:45:38 +0000

Introduction

Number Plate Recognition (NPR) has become increasingly significant in numerous domains, including law enforcement, traffic management, and parking systems. By automating the process of extracting and identifying license plate numbers, NPR systems streamline operations, enhance security, and enable efficient data analysis. In this blog post, we will explore a comprehensive approach to Number Plate Recognition that combines the power of Azure Custom Vision and Azure Computer Vision technologies.

Approach

Our approach to Number Plate Recognition involves two fundamental components: Azure Custom Vision and Azure Computer Vision. Azure Custom Vision, a cloud-based image recognition service, enables us to create a robust and accurate license plate detection model. Conversely, Azure Computer Vision, powered by Azure Cognitive Services, empowers us to extract the plate number from the detected license plate regions.

1. Training the License Plate Detection Model using Azure Custom Vision

The first step in our approach is to train a license plate detection model using Azure Custom Vision. This process entails collecting a diverse dataset of labeled license plate images, covering various license plate sizes, styles, lighting conditions, and angles. With Azure Custom Vision, we can easily upload the dataset, train a machine learning model, and fine-tune it to accurately detect license plates within images.

From the Azure Portal, search Cognetive Services multi service account and create an account.

For this example, the instance name is CarPlateDetection.

Open another tab and go to the Custom Vision Portal. You must sign in with the Microsoft account that used earlier. Click on New Project.

Type the project name, description and select the resource. Click on Object Detection as Project Type and General [A1] as Domain, then Create Project.

Click on Add images to upload the car images where the plate number is visible. Search at least 20 images for this example.

You will see the images you uploaded.

Now, click on each one to select the region of the plate number you want the Machine Learning model detects. You must type the tag name after you select the region.

After you finished to tag all the images, click on Train.

Chosse Quick Training an click on Train.

This can take between 10 and 15 minutes based on the number of images you tagged.

You will see three metrics:

Precision: It's the percentage of identified classifications that were correct. For example, if the model identified 100 images as dogs and 99 of them were actually dogs, then the precision is 99 percent
Recall: It's the percentage of actual classifications that were correctly identified. For example, if there were actually 100 images of apples, and the model identified 80 as apples, the recall is 80 percent.
Mean average precision (mAP): It's the average value of the average precision (AP). AP is the area under the precision/recall curve (precision plotted against recall for each prediction made).

The Probability Threshold is the desired level of confidence that a prediction needs to have in order to be considered correct. When you interpret prediction calls with a high probability threshold, they tend to return results with high precision at the expense of recall. That is, the detected classifications are correct, but many remain undetected. A low probability threshold does the opposite: most of the actual classifications are detected, but there are more false positives within that set. With this in mind, you should set the probability threshold according to the specific needs of your project. By default, the probability threshold is 50% and can be set between 0% and 100%.

You can test the model by clicking on Quick Test. Upload any car image to test the model.

Once you are comfortable with the testing results, click on Publish and type the model name, then select the prediction resource. Finally, click on Publish.

Click on Prediction URL to see the URL and Prediction Key.

2. Preprocessing and Localization

After training the license plate detection model, we need to preprocess the input images to enhance their quality and reduce noise. Azure Cognitive Services provide powerful image preprocessing capabilities, such as image resizing, contrast adjustment, and noise reduction, which can be applied to optimize the image for subsequent processing. Once the preprocessing is complete, the license plate detection model is utilized to locate license plate regions within the images.

We'll create a Python Notebook to call the Azure Custom Vision and Azure Computer Vision APIs.

import requests
import cv2
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from PIL import Image
from numpy import asarray

Take a look to Custom Vision Prediciton 3.0 for further details.

custom_vision_imgurl = 'https://<resource_name>.cognitiveservices.azure.com/customvision/v3.0/Prediction/<project_id>/detect/iterations/<published_name>/image'
api_key = '<YOUR_PREDICTION_KEY>'

data = open('Cars326.png', 'rb').read()
# decode the image file as a cv2 image, useful for later to display results
img = cv2.imdecode(np.array(bytearray(data), dtype='uint8'), cv2.IMREAD_COLOR)

custom_vision_headers = {
    'Content-Type': 'application/octet-stream', 
    'Prediction-Key': api_key
    }

custom_vision_resp = requests.post(url = custom_vision_imgurl, 
    data = data, 
    headers = custom_vision_headers).json()

# inspect the top result, based on probability 
hit = pd.DataFrame(custom_vision_resp['predictions']).sort_values(by='probability',ascending=False).head(1).to_dict()

print(hit)

{
   "probability":{
      "0":0.99115235
   },
   "tagId":{
      "0":"c5b7e0da-7c90-4e63-bf6f-b299f1c6492a"
   },
   "tagName":{
      "0":"Plate"
   },
   "boundingBox":{
      "0":{
         "left":0.3620827,
         "top":0.6698199,
         "width":0.2792757,
         "height":0.08204675
      }
   }
}

# extract the bounding box for the detected number plate 
boundingbox = list(hit['boundingBox'].values())[0]
l, t, w, h = (boundingbox['left'], 
    boundingbox['top'], 
    boundingbox['width'], 
    boundingbox['height'])

# extract bounding box coordinates and dimensions are scaled using image dimensions 
polylines1 = np.multiply([[l,t],[l+w,t],[l+w,t+h],[l,t+h]], 
    [img.shape[1],img.shape[0]])

# draw polylines based on bounding box results
temp_img = cv2.polylines(img, np.int32([polylines1]), 
    isClosed = True, color = (255, 255, 0), thickness = 2)

# display the original image with the plate region
plt.imshow(cv2.cvtColor(temp_img, cv2.COLOR_BGR2RGB))

# crop the image to the bounding box of the plate region
crop_x = polylines1[:,0].astype('uint16')
crop_y = polylines1[:,1].astype('uint16')

img_crop = img[np.min(crop_y):np.max(crop_y), 
    np.min(crop_x):np.max(crop_x)]

# display the detected plate region
plt.imshow(cv2.cvtColor(img_crop, cv2.COLOR_BGR2RGB))

3. License Plate Extraction using Azure Computer Vision

Once the license plate regions are identified, we employ Azure Computer Vision techniques to extract the actual plate numbers. This step involves applying image segmentation algorithms to isolate the characters and digits on the license plates. With Azure Computer Vision's OCR capabilities, we can accurately extract the alphanumeric information from the segmented regions.

Before calling the Azure Computer Vision API, we must ensure the image we send has at least 50px of height.

img_crop_height = img_crop.shape[0]
if img_crop_height < 50:
      pil_image = Image.fromarray(img_crop)
      img_crop_width = img_crop.shape[1]
      difference = 50 / img_crop_height
      resized_dimensions = (int(img_crop_width * difference), int(img_crop_height * difference))
      pil_image_resized = pil_image.resize(resized_dimensions)
      img_crop_resized = asarray(pil_image_resized)

      plt.imshow(cv2.cvtColor(img_crop_resized, cv2.COLOR_BGR2RGB))

computer_vision_imgurl = 'https://<resource_name>.cognitiveservices.azure.com/computervision/imageanalysis:analyze?api-version=2023-02-01-preview&features=read'

crop_bytes = bytes(cv2.imencode('.png', img_crop_resized)[1])

# make a call to the computer_vision_imgurl
computer_vision_resp = requests.post(
    url=computer_vision_imgurl, 
    data=crop_bytes, 
    headers={
        'Ocp-Apim-Subscription-Key': api_key, 
        'Content-Type': 'application/octet-stream'}).json()

{
   "readResult":{
      "stringIndexType":"TextElements",
      "content":"DL8CX 48.50",
      "pages":[
         {
            "height":50.0,
            "width":224.0,
            "angle":0.5536,
            "pageNumber":1,
            "words":[
               {
                  "content":"DL8CX",
                  "boundingBox":[
                     5.0,
                     12.0,
                     103.0,
                     13.0,
                     102.0,
                     43.0,
                     4.0,
                     42.0
                  ],
                  "confidence":0.677,
                  "span":{
                     "offset":0,
                     "length":5
                  }
               },
               {
                  "content":"48.50",
                  "boundingBox":[
                     127.0,
                     14.0,
                     208.0,
                     14.0,
                     207.0,
                     44.0,
                     127.0,
                     43.0
                  ],
                  "confidence":0.668,
                  "span":{
                     "offset":6,
                     "length":5
                  }
               }
            ],
            "spans":[
               {
                  "offset":0,
                  "length":11
               }
            ],
            "lines":[
               {
                  "content":"DL8CX 48.50",
                  "boundingBox":[
                     4.0,
                     11.0,
                     211.0,
                     13.0,
                     211.0,
                     43.0,
                     4.0,
                     41.0
                  ],
                  "spans":[
                     {
                        "offset":0,
                        "length":11
                     }
                  ]
               }
            ]
         }
      ],
      "styles":[

      ],
      "modelVersion":"2022-04-30"
   },
   "modelVersion":"2023-02-01-preview",
   "metadata":{
      "width":224,
      "height":50
   }
}

In the above response, we can see that the plate number is DL8CX 48.50 which is present in readResult.content.

You can find the source code here.

4. Post-processing and Analysis

After successfully extracting the license plate numbers, additional post-processing techniques can be applied to refine the results. These may include noise removal, character recognition validation, and formatting adjustments to ensure the extracted plate numbers are accurate and formatted correctly. Once the data is ready, it can be further analyzed, stored, or used for various purposes, such as database integration or real-time monitoring systems.

Conclusion

Number Plate Recognition using Azure Custom Vision and Azure Computer Vision technologies offers a comprehensive solution for accurate license plate detection and extraction. By leveraging the capabilities of Azure's cloud-based services, we can automate the process of recognizing and extracting license plate numbers, opening up a wide range of possibilities for applications in traffic management, parking systems, and law enforcement. In the upcoming sections of this blog series, we will delve deeper into each step of the approach, exploring the underlying techniques and demonstrating their implementation using Azure Custom Vision and Azure Computer Vision. Stay tuned to unlock the potential of Number Plate Recognition and revolutionize your image processing workflows with Azure's advanced AI capabilities.

Thanks for reading

Thank you very much for reading, I hope you found this article interesting and may be useful in the future. If you have any questions or ideas that you need to discuss, it will be a pleasure to be able to collaborate and exchange knowledge together.

Facial recognition using Amazon Rekognition on Python

Cristopher Coronado — Mon, 20 Mar 2023 21:21:06 +0000

AWS offers a powerful facial recognition service called Amazon Rekognition. In this tutorial, we'll use Python and the AWS SDK to create a simple app that uses the Rekognition service to detect faces in an image.

To get started, you'll need an AWS account and access to the AWS Console. Once you're logged in, navigate to the Security Credentials in the right nav bar and create an user with the AmazonRekognitionFullAccess permission. Then, create the access key and copy the secret key as well.

Next, you'll need to install the AWS SDK for Python (Boto3) using the following command:

pip install boto3

Once you've installed Boto3, you can import the necessary libraries in your Python script:

import boto3
from PIL import Image, ImageDraw, ImageFont

To use the Rekognition service, you'll need to create a Boto3 client and specify the region:

rekognition = boto3.client('rekognition', aws_access_key_id='<YOUR_ACCESS_KEY>',
aws_secret_access_key='<YOUR_SECRET_KEY>',
                           region_name='us-east-1')

For this example, we're going to use the Arial font. You can use whatever you want, first you must download the TTF file for that font.

font = ImageFont.truetype('arial.ttf', 30)

Now, we're going to read the image and call the detect_faces method.

with open(image_name, 'rb') as image_data:
        response_content = image_data.read()

rekognition_response = rekognition.detect_faces(
        Image={'Bytes': response_content}, Attributes=['ALL'])

This is the summarized response from above request.

{
   "FaceDetails":[
      {
         "BoundingBox":{
            "Width":0.1575244814157486,
            "Height":0.26773321628570557,
            "Left":0.7044062614440918,
            "Top":0.35157522559165955
         },
         "AgeRange":{
            "Low":22,
            "High":30
         },
         "Smile":{
            "Value":true,
            "Confidence":96.33894348144531
         },
         "Eyeglasses":{
            "Value":false,
            "Confidence":97.71277618408203
         },
         "Sunglasses":{
            "Value":false,
            "Confidence":99.9967269897461
         },
         "Gender":{
            "Value":"Male",
            "Confidence":99.95927429199219
         },
         "Beard":{
            "Value":true,
            "Confidence":99.96766662597656
         },
         "Mustache":{
            "Value":true,
            "Confidence":88.09671783447266
         },
         "EyesOpen":{
            "Value":true,
            "Confidence":98.2445297241211
         },
         "MouthOpen":{
            "Value":true,
            "Confidence":95.4747314453125
         },
         "Emotions":[
            {
               "Type":"HAPPY",
               "Confidence":99.83971405029297
            },
            {
               "Type":"SURPRISED",
               "Confidence":6.266087532043457
            },
            {
               "Type":"FEAR",
               "Confidence":5.88023567199707
            },
            {
               "Type":"SAD",
               "Confidence":2.1496529579162598
            },
            {
               "Type":"CONFUSED",
               "Confidence":0.06452298164367676
            },
            {
               "Type":"CALM",
               "Confidence":0.018650680780410767
            },
            {
               "Type":"ANGRY",
               "Confidence":0.012820698320865631
            },
            {
               "Type":"DISGUSTED",
               "Confidence":0.011646679602563381
            }
         ],
         "Pose":{
            "Roll":-27.842042922973633,
            "Yaw":-6.863317012786865,
            "Pitch":6.0284576416015625
         },
         "Quality":{
            "Brightness":93.38902282714844,
            "Sharpness":92.22801208496094
         },
         "Confidence":99.99991607666016
      }
   ],
   "ResponseMetadata":{
      "RequestId":"dd18647f-736c-44a9-aaa3-6c9782c657c7",
      "HTTPStatusCode":200,
      "HTTPHeaders":{
         "x-amzn-requestid":"dd18647f-736c-44a9-aaa3-6c9782c657c7",
         "content-type":"application/x-amz-json-1.1",
         "content-length":"23339",
         "date":"Mon, 20 Mar 2023 20:50:38 GMT"
      },
      "RetryAttempts":0
   }
}

With the above response, we're going to draw the faces and write the emotions in the image.

image = Image.open(image_name)
image_width, image_height = image.size
draw = ImageDraw.Draw(image)

line_width = 3
for item in rekognition_response.get('FaceDetails'):
    bounding_box = item['BoundingBox']
    width = image_width * bounding_box['Width']
    height = image_height * bounding_box['Height']
    left = image_width * bounding_box['Left']
    top = image_height * bounding_box['Top']

    left = int(left)
    top = int(top)
    width = int(width) + left
    height = int(height) + top

    draw.rectangle(((left, top), (width, height)),
                    outline='red', width=line_width)

    face_emotion_confidence = 0
    face_emotion = None
    for emotion in item.get('Emotions'):
        if emotion.get('Confidence') >= face_emotion_confidence:
            face_emotion_confidence = emotion['Confidence']
            face_emotion = emotion.get('Type')

    draw.text((left, top), face_emotion, 'white', font=font)

This is the result calling image variable:

This is the full code:

import boto3
from PIL import Image, ImageDraw, ImageFont

rekognition = boto3.client('rekognition', aws_access_key_id='<YOUR_ACCESS_KEY>',
                           aws_secret_access_key='<YOUR_SECRET_KEY>',
                           region_name='us-east-1')

font = ImageFont.truetype('arial.ttf', 30)

def detect_faces_and_emotions(image_name: str):
    with open(image_name, 'rb') as image_data:
        response_content = image_data.read()

    rekognition_response = rekognition.detect_faces(
        Image={'Bytes': response_content}, Attributes=['ALL'])

    print(rekognition_response)

    image = Image.open(image_name)
    image_width, image_height = image.size
    draw = ImageDraw.Draw(image)

    line_width = 3
    for item in rekognition_response.get('FaceDetails'):
        bounding_box = item['BoundingBox']
        width = image_width * bounding_box['Width']
        height = image_height * bounding_box['Height']
        left = image_width * bounding_box['Left']
        top = image_height * bounding_box['Top']

        left = int(left)
        top = int(top)
        width = int(width) + left
        height = int(height) + top

        draw.rectangle(((left, top), (width, height)),
                    outline='red', width=line_width)

        face_emotion_confidence = 0
        face_emotion = None
        for emotion in item.get('Emotions'):
            if emotion.get('Confidence') >= face_emotion_confidence:
                face_emotion_confidence = emotion['Confidence']
                face_emotion = emotion.get('Type')

        draw.text((left, top), face_emotion, 'white', font=font)

    return image

result_1 = detect_faces_and_emotions('people.jpg')
result_1

In this repository you can find the code, font file and the images.

And that's it! With just a few lines of Python code and the power of AWS Rekognition, you can create a facial recognition app that is capable of detecting faces with its emotions in an image.

Thanks for reading

Deploy a Regression model through AWS SageMaker studio using Autopilot

Cristopher Coronado — Tue, 07 Feb 2023 03:34:16 +0000

In this article we are going to create a Machine Learning model on AWS SageMaker using Amazon SageMaker Studio and Autopilot to predict house price.

1. Upload dataset to Amazon S3

First, we need to upload the dataset for house pricing. You can download the dataset for train and test, here.

From the search bar in our AWS Management Console, search S3.

Create a bucket and upload the train dataset.

2. Setup SageMaker domain

Now, from the search bar, search for SageMaker Studio.

We must have a SageMaker domain, so create one. Click on "Create a SageMaker domain".

Choose Quick setup and type a domain name.

Below the same page, you must specify a profile name and create a Execution role.

Select the before role created a click on Submit.

Once the domain and profile are created, now you can launch SageMaker Studio.

3. Import the dataset on SageMaker and add some preprocessing steps

From SageMaker Studio, click on Data Wrangler from the left sidebar.

Click on Import data.

From Data sources, choose Amazon S3.

Select the bucket where you uploaded the dataset.

Select the dataset and then click on Import.

In the Data types * Transform section, we can add some steps for preprocessing. Click on + Add step.

In the search bar, type "remove col" and click on the first result.

Select Drop column in Transform and Id in Columns to drop. Then, click on Add.

Click on Export and train.

In the Training tab, click on Export and train.

4. Create an Autopilot experiment

Type an experiment name, toggle on Auto split data and click on Next.

Select SalePrice as Target and the rest of the columns as Features. Then, click on Next.

Choose Auto and click on Next.

Type the endpoint name, select Regression as problem type and MSE as objective metric. Then, click on Next.

Review if the configuration details are correct and click on Create experiment.

Now, wait until the experiment finishes. You can see the best trial at the top. In this case, the experiment made 100 trials.

In the Job profile tab you can see how much time it took the experiment to finish. In this case, it took 46 minutes.

You can see the model details for the best trail. In Trails tab, click on View model details. On this page, there is a chart about the feature importance.

5. Deploy ML model

Go back to Trials tab, check the best model from the table and click on Deploy model.

Choose Make real-time predictions and type the endpoint name. Then, click on Deploy.

Now, wait until the endpoint is deployed. In this case, it took 19 minutes to finish the deployment.

6. Invoke deployed ML model

In AWS Settings tab you can copy the deployed URL.

In order to invoke the url, we need the Access Key and Secret Key. So, we can create an user and asign the AmazonSageMakerFullAccess permission policy. To do that, follow these steps:

Go to IAM Management Console
Users
Click on Add Users
Type de username
In Set permissions step, check Use a permissions boundary to control the maximum permissions and search AmazonSageMakerFullAccess in the permissions policies list
Review the details for the user and click on Create user

Once the user is created, from the users list, click on this user. Then, go to Security credentials tab and create an access key. Copy the access key and secret key.

We're going to invoke the url from Postman, but you can do it from other similar software as well.

Select POST as HTTP method. In the Authorization tab, select AWS Signature as type. Then, paste the access key and secret key. Also, type sagemaker as Service Name.

In the Headers tab, change the Content-Type to text/csv.

In the Body tab, select raw and Text. Paste any record from the test dataset. Don't forget to remove the id and sale price from the record. Now it's ready to invoke the url.

NOTE: After this lab if you want to delete the created endpoint, visit this page.

Thanks for reading

DEV Community: Cristopher Coronado

Measure Agent Quality and Safety with Azure AI Evaluation SDK and Azure AI Foundry

Introduction

How the three goals are measured

Why This Matters for AI Engineers

What You Build

Module Layout

What Is Evaluated, and Why

Built-in quality evaluators

Built-in tool-behavior evaluator (conditional)

Custom graph evaluators

Safety scanning

Pipeline Steps

Step 1: Start MCP Server

Step 2: Generate Evaluation Data

Step 3: Run Batch Evaluation

Step 4: Run Red Team Scan

Where Azure AI Foundry Fits

Latest Foundry quality snapshot (March 2026)

Key Snippets That Matter

1. Message conversion (MAF to evaluator schema)

2. Correct evaluator_config mapping shape

3. Deployment compatibility guard

4. Red team semantic success guard

Interpreting Results for Release Decisions

Common Failure Patterns

Environment Variables

Recommended Foundry Screenshots

Validation Snapshot

Why This Part Is a Milestone

Key Takeaways

What's Next

Resources

Serverless ML on AWS: How I Cut Inference Costs by 99% with Lambda & ONNX

1. The Inference Challenge: $0 vs $36-171/month

The Solution: ONNX Runtime on Lambda

Bonus: Portable Models

2. Comparing Models: Data-Driven Decisions

3. UI Polish: Dark Mode & UX

4. Engineering Lessons Learned

Lesson A: The "Deleted" Resource that Wasn't

Lesson B: Preserving State via Polling

What's Next? (Roadmap)

Try It Yourself

Building a Cost-Effective AutoML Platform on AWS: Serverless Training at ~$0.02/job

Prerequisites

The Problem

Architecture Decision: Why Lambda + Batch (Not Containers Everywhere)

Data Flow

Tech Stack

Key Implementation Details

1. Smart Problem Type Detection

2. Environment Variable Cascade (Critical Pattern)

3. Auto-Calculated Time Budget

4. Training Progress Tracking

5. Generated Reports

CI/CD with GitHub Actions + OIDC

Required IAM Permissions (Least Privilege)

Deployment Flow

Cost Breakdown (20 jobs/month)

Training Cost by Time: Detailed Comparison

AWS AutoML Lite (Fargate Spot - 2 vCPU, 4GB RAM)

SageMaker AI Training (ml.m5.xlarge - 4 vCPU, 16GB RAM)

Cost Comparison Summary (20 training jobs/month)

The Real Cost Driver: Inference Endpoints

When SageMaker Makes Sense

Feature Comparison: SageMaker vs AutoML Lite

Using Your Trained Model

Lessons Learned

What's Next? (Future Roadmap)

Try It Yourself

Building Rich Domain Models: A Practical Guide to DDD in .NET

Introduction

What Makes a Domain Model "Rich"?

Core Building Blocks of DDD

Building the Account Aggregate: A Real-World Example

Creating the Account Entity

Value Objects: The Building Blocks of Rich Models

AccountNumber Value Object

Money Value Object

2. Correct `evaluator_config` mapping shape