Originally published at AgentBadge.
When someone tells you that an API has an 87/100 Agent Readiness score, the first question should not be:
"Is 87 a good score?"
The better question is:
"Why is it 87?"
And the question after that is even more important:
"Can I reproduce the result myself?"
This is the problem AgentBadge is designed to solve.
Agent Readiness should not be an opinion generated by an LLM. It should be a measurable property of a service, calculated from explicit rules and supported by evidence.
The core idea is simple:
Rules → Evidence → Assertions → Score → Report
This article explains what happens inside that pipeline.
1. A scanner should measure, not guess
Imagine two tools scanning the same API.
Tool A says:
"Your API appears to be highly suitable for AI agents."
Tool B says:
"AB-004 passed because
https://example.com/openapi.jsonreturned HTTP 200 and contained a valid OpenAPI document."
Which result would you trust?
The second one is much more useful.
It tells you:
- what was checked;
- what rule was applied;
- what evidence was found;
- why the rule passed or failed;
- and where the evidence came from.
This is the fundamental design principle behind AgentBadge:
Every meaningful score should be explainable through evidence.
The scanner should not ask an AI model:
"How agent-ready does this API feel?"
It should ask deterministic questions such as:
"Does this URL exist?"
"Does it return the expected content type?"
"Does the response contain an OpenAPI document?"
"Does the declared authentication mechanism contain the required information?"
The difference may look subtle, but architecturally it is enormous.
2. Rules are the measurement instrument
A scanner is only as trustworthy as its rules.
Instead of hiding the evaluation logic inside application code, AgentBadge treats rules as explicit, versioned measurement definitions.
A simplified rule might look conceptually like this:
AB-001
Name: OpenAPI discoverability
Given:
target = https://example.com
Check:
GET /.well-known/openapi.json
Pass when:
HTTP status = 200
AND response is valid OpenAPI
Evidence:
URL
HTTP status
content type
content hash
Severity:
medium
The important property is that another implementation should be able to understand the same rule.
The rule is not:
"The API looks well documented."
It is:
"This specific machine-readable artifact was found and passed these specific checks."
That makes the scanner much easier to test, audit and reproduce.
3. Deterministic before intelligent
This leads to one of the most important architectural principles of AgentBadge:
Deterministic before intelligent.
If something can be established deterministically, use deterministic logic.
For example:
| Question | Preferred method |
|---|---|
| Does robots.txt exist? | HTTP request |
| Does sitemap exist? | HTTP request + parser |
| Does OpenAPI exist? | HTTP request + schema validation |
| Is JSON valid? | JSON parser |
| Does declared endpoint exist in another document? | Exact matching |
| What does an undocumented endpoint mean? | AI-assisted inference |
| What does an API capability actually mean? | Human confirmation / assisted review |
AI has a role, but it should not become the judge of facts that can be verified directly.
An LLM can help interpret ambiguous documentation.
It should not silently decide that an API supports refunds simply because a paragraph mentions the word "refund."
This is why AgentBadge treats AI as a copilot, not as the authority responsible for the score.
4. Evidence is the missing layer
A score without evidence is difficult to trust.
Consider this finding:
Documentation: 18/25
There is almost nothing you can do with it.
Now consider:
AB-007 OpenAPI discoverability
STATUS: VERIFIED
Evidence:
GET https://api.example.com/openapi.json
HTTP 200
Content-Type: application/json
OpenAPI version:
3.1.0
Confidence:
1.00
Now the developer knows what happened.
They can inspect the same resource themselves.
This creates a chain:
HTTP response → Evidence → Assertion → Rule result → Category score → Overall score
The evidence is therefore not an optional explanation attached to the report.
Evidence is part of the measurement itself.
5. Assertions connect evidence and scoring
A useful internal abstraction is an assertion.
An assertion answers one concrete question.
For example:
{
"rule_id": "AB-007",
"status": "VERIFIED",
"target": "https://api.example.com/openapi.json",
"evidence": {
"http_status": 200,
"content_type": "application/json"
},
"confidence": 1.0
}
The important states are intentionally explicit.
VERIFIED
The scanner found direct evidence supporting the assertion.
INFERRED
The scanner has a reasonable interpretation, but the evidence is not sufficient to treat it as fact.
CONFLICT
Two sources disagree.
For example:
Guide says: POST /refund
OpenAPI says: POST /refund-request
MISSING
The expected capability or artifact could not be found.
These states are more informative than a simple pass/fail system.
They tell us not only what the scanner thinks, but also how strongly it knows it.
6. Confidence is not the same as verification
This distinction is important.
A scanner may infer something with high confidence.
That does not automatically make it verified.
For example, an API documentation page might strongly suggest that a service supports refunds.
An LLM may assign a confidence of 0.94 to that interpretation.
But unless there is machine-readable evidence supporting the capability, the assertion should not magically become VERIFIED.
Instead: INFERRED, confidence: 0.94
The user can then: Confirm, Edit, Reject
This is the boundary between automatic fixes and assisted fixes.
Safe, deterministic changes can be automated.
Semantic claims require human confirmation.
7. Reproducibility matters
Now we arrive at the second major property of the scanner.
Suppose you scan https://api.example.com today and receive 76/100.
Someone else runs the same rules against the same captured state and should be able to understand how the result was produced.
That requires more than storing the final number.
The report needs to describe the measurement context.
Conceptually:
{
"target": "https://api.example.com",
"ruleset": "agent-readiness-v1.0",
"scanner_version": "0.1.0",
"timestamp": "...",
"assertions": [...],
"score": {
"total": 76,
"categories": {
"discovery": 18,
"documentation": 20,
"authentication": 17,
"machine_readability": 21
}
}
}
This allows the score to be understood as the output of a defined measurement process rather than a mysterious number.
8. Rules must be versioned
Rules change. New standards appear. New machine-readable formats emerge.
Some checks eventually turn out to be too strict or too weak.
Therefore: Agent Readiness v1.0 must not silently become v1.1 while pretending the results are identical.
Each ruleset should have an explicit version.
Now a report can say: Score: 82/100, Ruleset: Agent Readiness v1.2
This gives us an important property:
Same target + same measurement state + same ruleset = reproducible result.
9. Why open rules do not destroy the product
If AgentBadge publishes its rules, can't someone simply copy them?
Yes. And that is intentional.
The goal is not to create a secret scoring algorithm. The goal is to establish a useful measurement standard.
The long-term value comes from the workflow around that standard: open specification, open scanner, GitHub Action, README badge, continuous monitoring, regression alerts, fix workflow, developer adoption.
A competitor can copy AB-001, AB-002, AB-003.
They cannot instantly copy: thousands of repositories displaying the badge, existing GitHub Actions, developer habits, historical scan data, integrations, workflow configuration, trust built around independently verifiable reports.
The moat is not secret rules. It is a standard installed inside the developer workflow.
10. The score should explain itself
A single number is useful for quick comparison. But it should never be the only information available.
Suppose a developer's score changes: 76 to 72.
The product should explain the delta: +8 OpenAPI documentation detected, -12 New authentication issue detected, +0 Discovery unchanged. Result: 76 to 72.
This turns measurement into an improvement loop.
11. From Measure to Prove to Improve
The architecture becomes a simple loop: Measure, Prove, Improve, Measure again.
The score is the beginning of the workflow, not the end.
12. What AgentBadge should never claim
AgentBadge measures Agent Readiness.
It does not certify: API security, business correctness, service reliability, legal compliance, quality of business logic, whether an agent should trust the company.
A high score does not mean "This API is safe." It means "This API satisfied these measurable Agent Readiness criteria."
Don't certify. Measure.
13. What this enables
Once the measurement layer exists, many higher-level products become possible.
A developer can run npx @agentbadge/cli scan https://api.example.com.
A CI pipeline can enforce a minimum score. A README can display the current measurement. A platform can query AgentBadge programmatically. An organization can compare vendors using the same ruleset.
14. The bigger idea
The web has spent years developing tools for measuring websites. Performance has metrics. Accessibility has automated checks. Security has scanners. TLS has analyzers. SEO has crawlers and validators.
The emerging agentic web needs something similar.
AgentBadge's approach is deliberately conservative: Define the rules. Collect the evidence. Show the reasoning. Version the rules. Make the result reproducible.
That is what makes Agent Readiness a measurement discipline rather than another AI-generated checklist.
Read more:
- What Is Agent Readiness? — Article 1
- Why AI Agents Fail to Use APIs — Article 5
- What Does an AI Agent Need to Understand an API? — Article 6
- Why Your OpenAPI Spec Isn't Enough — Article 7
- How Do You Measure Agent Readiness? — Article 8
Agent Knowledge Layer: agentbadge.xyz/agent-guide
Top comments (0)