DEV Community: Ing. Pablo Cueto

Building a RAG assistant that says “I don’t know”: evaluation, security and a $0 deployment

Ing. Pablo Cueto — Fri, 16 Oct 2026 23:16:29 +0000

Most RAG tutorials end when the chatbot answers a question about a PDF. I wanted to find out what comes after that: how do you know it works, who is allowed to see what, what happens when a document tries to give it orders, and how do you ship it without paying for anything?

This post walks through enterprise-ai-agent, a learning and portfolio project. It answers questions about public AWS documentation, cites its sources, refuses what the documents do not cover, and includes a small agent that can use tools. A note on how it was built: I worked step by step with an AI coding assistant (Claude). I ran every command, read every error and decided the scope, and writing this post was my way of making sure I can explain each choice. The project is not production software, and I say where it falls short at the end.

Repository: https://github.com/PabsCueto/enterprise-ai-agent

What it does
• POST /ask retrieves the chunks the user is allowed to see, asks Gemini to answer using only that context, and returns the answer with source URLs and token usage.
• POST /agent is a LangGraph agent with three tools: search the documentation, query a simulated internal status API, and create a ticket draft (a local JSON file that is sent nowhere).
• Around them: an evaluation harness, per-user access control, structured logs, and a Docker image that is built and smoke-tested in CI.
The corpus is seven introductory pages of AWS documentation, about a hundred chunks. It is small on purpose: I can check every result by hand.
request -> FastAPI (request id, JSON log, API key, rate limit)
|-- /ask -> retrieve (per-user filter + score threshold) -> Gemini
|-- /agent -> LangGraph: agent <-> tools -> answer
Two layers against made-up answers
The first layer is a similarity threshold. Chunks that are not close enough to the question are dropped, and if nothing is left the model is never called. The second layer is the prompt: answer only from the context, or say you cannot find it.

I measured both. Questions unrelated to AWS (the capital of France, a taco recipe) scored at most 0.575, while questions the corpus covers scored at least 0.682, so a threshold of 0.6 separates them cleanly.

The interesting case is a question that is about AWS but is not covered: VPCs, EC2 pricing, RDS, Amplify. Those scored between 0.60 and 0.664, above the threshold. Embeddings measure topic, not whether the answer is in the text, and a question about RDS looks a lot like DynamoDB documentation. The score cannot catch them; the prompt does, and the model refused all four. Neither layer would be enough on its own.

An agent with guardrails
The agent is a small LangGraph graph: a model node that decides, a tools node that executes, and a loop between them until the model answers. What matters is what the agent cannot do:
• It has no tools to modify or delete anything, and tickets are drafts.
• Tickets are created only on an explicit request.
• The graph stops after 10 steps, thread ids are validated, and tool output is treated as data rather than instructions.
One lesson came from the prompt. My first rule told the agent to ask when information was missing. When I wrote “create a ticket for the problem you find”, it asked me for a title and description it could have written itself. The fix was one sentence: if the user delegates the content, draft it from the data you gathered, and ask only when there is not enough to write it.

Measuring it
The harness has three small datasets: 22 retrieval questions (14 covered, 4 unrelated, 4 about AWS but not covered), 8 agent cases and 7 prompt-injection cases. It reports hit rate and MRR, how many questions are answered or rejected, and whether the agent called the expected tools in order and avoided forbidden ones.

The results were good: 100% hit rate at 4 with an MRR of 1.0 on the covered questions, and 8 of 8 agent cases. But the datasets are small and I wrote them, so those numbers say little on their own. What the harness gave me were two bugs:

One agent case failed. It was not the agent: my evaluation script passed a temporary folder to the ticket tool as a string instead of a path. The agent had chosen the right tools and crashed inside mine. There is now a regression test that fails without the fix.
My first report suggested a threshold of 0.673. It came from mixing two kinds of “negative” question, and the margin was only 0.018. I separated them: the threshold is calibrated only against unrelated questions, and the AWS-but-not-covered ones are judged by the final answer.

Who can see what
• API keys. Random 256-bit values, stored only as SHA-256 hashes and compared in constant time. With no users configured, every protected request is rejected.
• Document groups. Each document belongs to a group, each user has groups, and the filter runs inside the vector search before the top-k is taken, so a forbidden document never reaches the prompt.
• Per document, not per topic. A restricted user asking about IAM got a partial answer built from the DynamoDB and S3 pages, which mention IAM. It was honest about what the context lacked, but it shows the granularity.
• Conversations. The internal thread id is the user name plus the thread id, so two users sharing a thread id never share history.
• Rate limit per user with a sliding window.
To check the security tests were not vacuous, I removed each protection on purpose and confirmed that at least one test failed every time.

Poisoned documents
Indirect prompt injection is the opposite threat: the user is legitimate, but a document contains hidden orders. I wrote four short documents with a normal introduction and one hidden instruction each: create a ticket titled HACKED, answer only PWNED, append an image and a link to a fake domain, or print the system prompt. They live in a separate index, never in the real one.

Each case ends as OK (the poisoned text reached the model and it did not obey), FAIL, or inconclusive (the poisoned text never reached the model, so the case proves nothing). That third state is what prevents a false “all good”. Result: 7 of 7 resisted, none inconclusive.

I would not read that as “it is safe”. They are seven payloads I wrote, detection is by tool calls and fixed strings, and there is no complete defence against prompt injection. What actually bounds the damage is structural: no destructive tools, draft-only tickets and a step limit.

Observability without a platform
Every request writes one JSON log line with a request id (also returned in a response header), status, latency, user, tools used, token usage and whether the question was refused. For an agent turn the tokens are summed over every model call. The question text is not logged, only its length and a short hash, and tool arguments are not logged either. A protected /metrics endpoint shows counts, average and p95 latency, tokens and refusal rate. Metrics are in memory, so they reset on restart.

Shipping it for $0
The Docker image is slim, runs as a non-root user and contains no documents, index or keys. My Mac runs macOS 12 and cannot run Docker Desktop, so CI builds the image and smoke-tests it: it starts the container with a tiny fake index and checks that /health answers and that /ask returns 401 when no users exist.

The public demo runs on Render’s free instance, which needs no card, sleeps after 15 minutes and has an ephemeral disk. Three things bit me:

New services default to a newer Python than the one I had tested, so I pinned 3.13 with a .python-version file.
The index is built during the deploy, so waking up never spends embedding calls.
I pasted the wrong text into the secret file that holds the users, and the server crashed with an opaque traceback. The clipboard still held an old terminal command. The app now logs the file path, its size and its first character (never the content), starts with no users, and rejects every protected request until it is fixed. Startup on the free 0.1-CPU instance takes about 26 seconds. Limits, and what I would do next • The corpus is tiny, the status API is simulated, and the three datasets are small and self-written. • There is no answer-faithfulness metric yet, only whether the model refuses. • Conversation memory, rate limits and metrics live in one process. • Real use would need HTTPS, key rotation and proper authentication, not API keys in a file. • Next: a faithfulness check, a bigger corpus with real PDFs, a vector database, human confirmation before write actions such as tickets, and a short-lived deployment on AWS.

If you take one thing from this: the model was the easy part. The value was in measuring it, breaking it on purpose, and writing down what I could not prove.

Blue/green deployments on ECS Fargate with CloudFormation.

Ing. Pablo Cueto — Wed, 07 Oct 2026 19:25:10 +0000

I built this project to practice container deployments on AWS while preparing for the AWS Certified Developer – Associate exam. It is a small FastAPI notes API running on ECS Fargate behind an Application Load Balancer, with blue/green deployments defined in CloudFormation. This post walks through the design, what went wrong, and how I kept the cost close to zero.

Repo: https://github.com/PabsCueto/ecs-fargate-bluegreen

What it does

A FastAPI app with /health, /version and a small in-memory CRUD for notes.
A pipeline that builds the Docker image, tests it, and pushes it to ECR.
A CloudFormation template with an ALB, two target groups, and an ECS service that uses the built-in blue/green strategy.
Scripts to preview, deploy, and tear everything down.

The notes live in memory on purpose: I wanted the focus to be the deployment, not the database.

The pipeline: GitHub Actions with OIDC

The workflow assumes an IAM role through GitHub's OIDC provider, so there are no long-lived AWS keys in the repo. The trust policy is limited to one repository and branch, and the role can only push to one ECR repository.

Before pushing the image, the workflow starts the container inside the runner and calls /health and /version. I don't have a local Docker setup that can run current Docker Desktop, so this smoke test also acts as my "does it actually start?" check.

The infrastructure: ECS blue/green in CloudFormation

The ALB has two target groups and a listener rule that carries production traffic. The ECS service declares the blue/green strategy and tells ECS which rule and alternate target group to use:

EcsService:
  Type: AWS::ECS::Service
  Properties:
    DeploymentController:
      Type: ECS
    DeploymentConfiguration:
      Strategy: BLUE_GREEN
      BakeTimeInMinutes: 1
      DeploymentCircuitBreaker:
        Enable: true
        Rollback: true
    LoadBalancers:
      - ContainerName: api
        ContainerPort: 8000
        TargetGroupArn: !Ref TargetGroupBlue
        AdvancedConfiguration:
          AlternateTargetGroupArn: !Ref TargetGroupGreen
          ProductionListenerRule: !Ref ListenerProdRule
          RoleArn: !GetAtt EcsInfrastructureRole.Arn

Changing anything in the task definition (I change an environment variable) creates a new revision and starts a deployment. ECS starts the green tasks, waits for them to pass the health checks, moves the listener rule to the green target group, keeps both versions for the bake time, and then stops the old one. If the new version never becomes healthy, the circuit breaker rolls back.

Keeping the cost near zero

An ALB and Fargate tasks are billed per hour, so I designed around short-lived deployments:

deploy.sh --preview creates only a CloudFormation change set, so nothing is provisioned.
deploy.sh shows a cost warning and requires typing a confirmation before creating anything.
teardown.sh deletes the stack and runs check-costs.sh, which lists any load balancer, running task, NAT gateway, or Elastic IP left behind.
Tasks run in public subnets with a security group that only accepts traffic from the ALB, which avoids a NAT gateway.
Smallest Fargate size, one task, one-day log retention, and a 5-second deregistration delay.

Result

I deployed 1.0.0 (blue), then updated the version to 2.0.0 (green):

GET /version -> {"version":"1.0.0","color":"blue","hostname":"ip-172-31-67-143..."}
GET /version -> {"version":"2.0.0","color":"green","hostname":"ip-172-31-84-17..."}

The hostname changed, which shows traffic moved to a different set of tasks. ECS reported both deployments as successful (about 1 minute 25 seconds for the first one and 2 minutes 25 seconds for the blue/green update, including the bake time). Then I destroyed the stack.

What went wrong

1. OIDC: "Not authorized to perform sts:AssumeRoleWithWebIdentity". My trust policy matched the classic subject format (repo:owner/repo:ref:refs/heads/main), but the token for this repository included numeric IDs: repo:owner@<owner-id>/repo@<repo-id>:ref:refs/heads/main. I only found out by adding a temporary workflow step that decoded and printed the token claims.

2. Smoke test failing with curl exit code 56. The published port accepted connections before uvicorn was ready, and curl doesn't retry that error by default. Adding --retry-all-errors fixed it, and the workflow now prints the container logs when the smoke test fails.

3. CodeDeploy was not available. My first version used the CloudFormation CodeDeploy blue/green hook. The stack was created, but the update failed with SubscriptionRequiredException, and aws deploy list-applications returned the same error. I suspect a restriction of my account's plan, although I couldn't confirm that. I switched to ECS's native blue/green strategy, which needs no extra service and meets the same goal.

Limits

There is no live demo: I tear the stack down to avoid costs. The README has the evidence.
HTTP only, no domain or HTTPS.
I didn't measure downtime with a continuous request loop during the switch, and I didn't use a test listener to validate the green version before the shift. Those are natural next steps.

If you want to look at the code, it's here: https://github.com/PabsCueto/ecs-fargate-bluegreen. Feedback is welcome.

How I Built a Password Strength & Breach Check API with FastAPI and HaveIBeenPwned

Ing. Pablo Cueto — Thu, 11 Jun 2026 00:30:21 +0000

The Problem

Most password strength checkers only validate character rules — length, uppercase, digits. But a password like Password1 passes every rule and has been breached over 2 million times.

I wanted a single API that does both: score the strength and check it against real breach data.

The Stack

FastAPI + Python 3.13
HaveIBeenPwned Pwned Passwords API v3
Render for deployment
RapidAPI for distribution

The k-Anonymity Model

The most important design decision was privacy. I never send the full password to HaveIBeenPwned. Instead:

Hash the password with SHA-1
Send only the first 5 characters of the hash
HIBP returns all hashes that match that prefix
Check locally if the full hash is in the list

sha1_hash = hashlib.sha1(password.encode()).hexdigest().upper()
prefix = sha1_hash[:5]
suffix = sha1_hash[5:]

response = await client.get(f"https://api.pwnedpasswords.com/range/{prefix}")

for line in response.text.splitlines():
    hash_suffix, count = line.split(":")
    if hash_suffix == suffix:
        return int(count)  # Times found in breaches

The full password never leaves your server.

The Scoring Algorithm

The strength score (0–100) combines:

Length — up to 35 pts (6 tiers from <6 chars to 20+)
Character variety — up to 40 pts (lowercase, uppercase, digits, special chars)
Entropy bonus — up to 25 pts based on pool entropy (length × log2(charset_size))

Plus penalties for:

Common passwords (password, 123456, etc.)
Keyboard walks (qwerty, asdfgh)
Repeated characters (aaa, 111)
Sequential patterns (123, abc)

The Endpoints

Method	Route	Description
POST	`/password/analyze`	Score + analysis
POST	`/password/breach-check`	HIBP breach lookup
POST	`/password/full-check`	Both in one call

Example Response

{
  "score": 95,
  "label": "Very Strong",
  "entropy_bits": 4.0,
  "length": 16,
  "checks": {
    "has_lowercase": true,
    "has_uppercase": true,
    "has_digit": true,
    "has_special": true
  },
  "feedback": [],
  "penalties": []
}

Try It

Available on RapidAPI as part of my Fraud Prevention Bundle alongside Email Validator, Phone Validator Pro, and IP Geolocation APIs.

→ Password Strength & Breach Check API on RapidAPI

I built and deployed an IP Geolocation API from scratch in one day — here's the full stack

Ing. Pablo Cueto — Wed, 27 May 2026 23:52:21 +0000

No third-party black boxes. No vendor lock-in. Just clean Python code, two free data sources, and a live API on RapidAPI.

The stack

FastAPI — async endpoints, automatic Swagger docs
IPLocate.io — primary geo source (VPN/proxy/Tor detection on free tier)
ipgeolocation.io — automatic fallback
In-memory TTL cache — LRU cache, 1h TTL, 10k entries
Render — auto-deploy from GitHub on every push
RapidAPI — marketplace distribution, billing, rate limiting

What it does

GET /ip/{ip} — full geolocation for any IPv4 or IPv6
GET /ip/me — auto-detect caller's IP
POST /ip/bulk — up to 50 IPs in one request
Security flags on every response: VPN, Tor, proxy, datacenter

The architecture decision that matters

Two geo sources with automatic failover. If IPLocate goes down, ipgeolocation.io takes over silently. The client never sees a 502. Cache handles repeated lookups without touching either upstream API.

The response

{
  "ip": "8.8.8.8",
  "location": {
    "country": "United States",
    "city": "Mountain View",
    "latitude": 37.38605,
    "longitude": -122.08385
  },
  "security": {
    "is_vpn": false,
    "is_tor": false,
    "is_datacenter": true
  },
  "source": "iplocate"
}

Try it live

https://rapidapi.com/PabsCueto/api/ip-geolocation38

Free tier available. Bulk endpoint included from the first paid plan. The entire project — models, services, routers, tests, render.yaml — is a clean FastAPI structure you can fork and adapt for any data enrichment API.

python #fastapi #api #webdev #programming #backend #devops #opensource #buildinpublic #pythondeveloper #apidesign #render #rapidapi #geolocation #cybersecurity

I Built a Phone Validation API — Live on RapidAPI published: true tags: python, fastapi, api, productivity

Ing. Pablo Cueto — Sat, 16 May 2026 19:48:28 +0000

Validating phone numbers sounds simple until you realize regex won't tell you if a number is mobile, landline, or VoIP — or what country it belongs to.
I built Phone Validator Pro, a REST API that does all of that in a single request.

What it does:

Validates format (E.164, international, national)
Detects country, region and timezone
Identifies the carrier/operator
Detects line type: mobile, landline, VoIP, toll-free
Batch validation — up to 100 numbers per request

Example response
jsonGET /v1/validate/+14155551234

{
"is_valid": true,
"formats": {
"e164": "+14155551234",
"international": "+1 415-555-1234",
"national": "(415) 555-1234"
},
"country": {
"iso2": "US",
"name": "United States",
"calling_code": "+1",
"timezone": ["America/New_York"]
},
"line_type": "mobile",
"carrier": { "name": "T-Mobile", "available": true }
}

Stack

FastAPI + Python 3.12
Google's libphonenumber for validation
Deployed on Render
Monetized via RapidAPI

Try it free
No credit card required. The free tier gives you 100 requests/month to test it in your project.
rapidapi.com/PabsCueto/api/phone-validator-pro1

How I Built a 4-Layer Email Validator API with FastAPI and Published it on RapidAPI

Ing. Pablo Cueto — Fri, 15 May 2026 22:42:24 +0000

I just launched Email Validator API on RapidAPI — a professional email
verification service built with Python and FastAPI.

Most email validators only check syntax. Mine runs 4 layers of verification
and returns a confidence score for every email.

The 4 Layers

Layer 1 — Syntax Check
Validates format against RFC 5322 standards. Also detects common typos
and suggests corrections — for example user@gmial.com → user@gmail.com.

Layer 2 — DNS / MX Lookup
Confirms the domain exists and is configured to receive emails using
dnspython. Catches domains that look valid but can't receive mail.

Layer 3 — Disposable Domain Detection
Flags throwaway addresses from 3,000+ known providers like Mailinator,
Guerrilla Mail, YOPmail and TempMail. Essential for preventing fake signups.

Layer 4 — SMTP Verification
Performs a real SMTP handshake to confirm the mailbox exists — without
sending a single email. Available on Pro plans.

The Response

Every request returns a detailed JSON breakdown:

{
"email": "user@gmail.com",
"valid": true,
"score": 95,
"syntax_valid": true,
"mx_found": true,
"disposable": false,
"smtp_valid": true,
"suggestion": null,
"error": null
}

Score Interpretation

Score	Meaning
0–29	Invalid syntax
30–54	Domain issues
55–74	Likely valid
75–94	Valid + not disposable
95–100	Fully verified via SMTP

Tech Stack

FastAPI — async endpoints, automatic Swagger docs
dnspython — async DNS/MX lookups
slowapi — rate limiting per plan tier
Render — deployment
RapidAPI — marketplace and monetization

Plans

Plan	Requests/month	Layers	Price
BASIC	100	1–2	Free
PRO	5,000	1–3	$9/mo
MEGA	20,000	1–4	$19/mo
ULTRA	50,000	1–4 + bulk	$29/mo

Try it

🔗 https://rapidapi.com/PabsCueto/api/email-validator-api32

Free plan available — no credit card required.

Happy to answer questions about the implementation!

I Built a Password Strength Analyzer API with Python + FastAPI

Ing. Pablo Cueto — Thu, 14 May 2026 19:00:54 +0000

I Built a Password Strength Analyzer API with Python + FastAPI 🔐

As a backend developer, I wanted to build something useful, real, and monetizable — not just another tutorial project.

So I built a Password Strength Analyzer API that any developer can integrate into their app in minutes.

What it does

Returns a strength score from 0 (very weak) to 4 (very strong)
Estimates crack time using real-world attack scenarios
Detects if the password is in known breach lists
Returns actionable suggestions to improve security
Analyzes uppercase, lowercase, numbers and symbols

Tech Stack

Python 3.10+
FastAPI — for the REST API
zxcvbn — the same algorithm Dropbox uses
Pydantic — data validation
Render — cloud deployment

Example Request

curl -X POST "https://password-analyzern.onrender.com/analyze" \
     -H "Content-Type: application/json" \
     -H "X-API-Key: YOUR_API_KEY" \
     -d '{"password": "MyP@ssw0rd!"}'

Example Response

{
  "password": "***********",
  "score": 3,
  "strength_label": "Fuerte 🟢",
  "crack_time": "3 months",
  "suggestions": ["Idealmente usa 12 o más caracteres"],
  "warnings": [],
  "length": 11,
  "has_uppercase": true,
  "has_lowercase": true,
  "has_numbers": true,
  "has_symbols": true,
  "is_common": false
}

Try it for free

RapidAPI (free plan available):
https://rapidapi.com/PabsCueto/api/password-strength-analyzer1

GitHub:
https://github.com/PabsCueto/password-analyzer

Live docs:
https://password-analyzern.onrender.com/docs

Built this as part of my API portfolio. Feedback welcome!

How I built a PDF Generator API with FastAPI and ReportLab

Ing. Pablo Cueto — Thu, 14 May 2026 18:08:51 +0000

The problem

Almost every SaaS app needs to generate PDFs at some point — invoices,
reports, receipts. Most developers end up wrestling with Puppeteer,
wkhtmltopdf, or heavy libraries that are painful to deploy.

I decided to build a simple REST API that does it with a single POST request.

The stack

FastAPI — for the REST endpoints
ReportLab — for PDF generation in Python
Docker + Render — for deployment
RapidAPI — for monetization

How it works

Send a JSON payload, get a PDF back:

import requests

response = requests.post(
    "https://pdf-generator-api-1fx5.onrender.com/generate/invoice",
    headers={"X-API-Key": "your-key"},
    json={
        "invoice_number": "INV-2025-001",
        "company": {"name": "Acme Corp", "email": "billing@acme.com"},
        "client": {"name": "John Doe", "email": "john@example.com"},
        "items": [
            {"description": "Web Development", "quantity": 10, "unit_price": 150}
        ],
        "tax_rate": 16
    }
)

with open("invoice.pdf", "wb") as f:
    f.write(response.content)

What it generates

Invoice PDFs — with line items, tax calculation and totals
HTML to PDF — convert any HTML string to PDF
Report PDFs — turn tabular data into formatted reports

Try it free

Available on RapidAPI with a free tier (50 requests/month):
🔗 https://rapidapi.com/PabsCueto/api/pdf-generator17

Would love feedback from the community!