Building a Privacy-First Face-Analysis API with REST, MCP, and A2A

Teams building image-aware products usually discover the same problem: the demo is easy, but the contract around the demo is not. A useful system has to explain exactly what it measures, what it refuses to infer, how clients can integrate it, and how a result can be inspected later.

This post describes the design I am using for iLook, a privacy-first browser tool for exploring visible facial geometry in a user-provided photo. The goal is not identity, ranking, or diagnosis. The goal is a clear, bounded report that a person can understand and a developer can consume.

Start with a narrow, testable scope

A face-analysis service should begin with an explicit scope statement. For iLook, the service works with visible geometry in the supplied image and can explain:

• approximate face-shape proportions
• symmetry signals and measurement confidence
• relative distances and ratios
• Golden Ratio phi context as an educational reference
• structured presentation notes that are not medical advice

It does not identify a person, build a biometric profile, infer sensitive traits, or make a medical or high-impact decision. These boundaries are product requirements, not just copy for a landing page. They affect the data model, error handling, retention policy, and review process.

A practical threat model asks four questions before implementation:

1. What data is received, and is it necessary for the requested result?
2. What is retained after the result is returned?
3. Which outputs could be misunderstood as a judgment rather than a measurement?
4. How can a client tell that a result is partial or low confidence?

Writing these answers down makes later API and UI decisions much easier.

A useful response is explainable, not merely numeric

A single score is difficult to audit. A better response separates observations from interpretation. An illustrative response shape might look like this:

{
  "schema_version": "2026-01",
  "status": "complete",
  "signals": {
    "face_shape": { "label": "oval", "confidence": 0.82 },
    "symmetry": { "summary": "balanced", "confidence": 0.74 },
    "proportions": [
      { "name": "width_to_height", "value": 0.78, "confidence": 0.81 }
    ],
    "phi_context": {
      "available": true,
      "note": "Educational comparison; not a quality score"
    }
  },
  "limitations": [
    "Lighting and camera angle can change visible measurements",
    "Results describe the supplied image, not a person's identity"
  ]
}

The important detail is the presence of confidence and limitations beside every meaningful signal. If the image is tilted, heavily shadowed, or cropped, the system should say so instead of hiding uncertainty behind a precise-looking number.

Keep transport separate from the analysis model

REST and OpenAPI are a useful baseline because they work in browsers, scripts, and conventional backend systems. The transport layer should accept an image reference or upload, validate size and type, and return a job or a result with a stable schema version.

The analysis model should not know whether the request came from REST, an MCP client, or an A2A message. That separation makes it possible to add new clients without duplicating safety checks. It also makes testing simpler: the same fixture image can be sent through every adapter and compared against the same expected structure.

For a production integration, document:

• authentication and rate limits
• accepted image formats and maximum dimensions
• synchronous versus asynchronous behavior
• error codes for invalid, unsupported, or low-quality images
• retention and deletion behavior
• schema versioning and deprecation policy

MCP and A2A are adapters, not shortcuts

MCP can make a bounded analysis capability discoverable to an assistant. A good tool definition states the input, output, and limits in plain language. For example, a tool might be named analyze_visible_face_geometry and return the same structured result as REST. The description should explicitly say that the tool reports visible geometry from the supplied image and does not identify people or infer sensitive attributes.

A2A is useful when one agent needs to request work from another service. The agent-facing contract should carry the same schema version, confidence values, and limitations. An orchestrator can then decide whether to show the result, ask for a better image, or stop because the requested operation is outside scope.

The rule I use is simple: protocol adapters may change the envelope, but never the safety contract.

Test the edges, not only the happy path

The most valuable tests are usually not the perfect front-facing image. Include fixtures for:

• low light and strong backlight
• partial occlusion and side angles
• multiple faces when the workflow expects one
• very small or very large images
• unsupported formats and corrupt uploads
• repeated requests and network retries
• prompt injection text embedded in an image or filename

For every fixture, assert both the result and the refusal behavior. A system that returns a confident-looking answer for an unusable image is harder to trust than one that clearly reports a limitation.

Make the user-facing explanation part of the API

Developers often treat explanations as UI copy added at the end. That is backwards. The explanation should be generated from the same structured fields that the API exposes, so the browser view, REST client, MCP tool, and A2A agent do not drift apart.

That is the design direction behind iLook: a privacy-first way to explore visible facial geometry with a readable report and machine-readable integration paths. The product is free to try, and the useful constraint is that every result should remain understandable, bounded, and easy to question.

Closing checklist

Before shipping an image-analysis capability, verify that:

• the scope is written in one paragraph and reflected in the schema
• uncertainty is visible beside measurements
• the same result model is used by REST, MCP, and A2A adapters
• deletion and retention behavior are documented
• unsafe or ambiguous inputs produce explicit limitations
• examples teach users how to integrate without implying more certainty than the system has

A small, honest contract is more valuable than a large list of impressive outputs. It gives developers something they can test, users something they can understand, and reviewers a clear way to see whether the product is behaving as designed.