DEV Community

Karthi Mahadevan
Karthi Mahadevan

Posted on

Step 2 — Architecture Evidence Validation

#softwaredevelopment #softwareengineering #architecture

Purpose

Validate the Architecture Evidence Model produced by Step 1.

This is a validation task.

Do not generate the final architecture documentation.

Do not improve the architecture.

Do not silently correct unsupported claims.

Determine whether the extracted architecture evidence is sufficiently accurate, complete, consistent, and traceable to the source repository.

The output will be used to decide whether the repository can proceed to Step 3 — Documentation.

1. Validation Principles

Apply these principles throughout validation.

Principle 1 — Source evidence is authoritative

The source repository is the primary authority.

The Architecture Evidence Model must not contain claims that cannot be supported by repository evidence.

Principle 2 — Do not validate by plausibility

A statement being technically plausible is not sufficient.

For example:

Kafka is present.
Therefore the system is event-driven.
Enter fullscreen mode Exit fullscreen mode

This is not valid evidence.

The validator must identify the actual producer, event/topic, consumer and processing behaviour.

Principle 3 — Distinguish fact from inference

Every significant architectural claim must be classified as:

  • Verified
  • Inferred
  • Unknown

Check that the classification matches the evidence.

Principle 4 — Do not silently repair

If the extraction contains an error:

  • identify the error
  • provide the evidence
  • state the required correction

Do not silently modify the Architecture Evidence Model.

Principle 5 — Prefer missing information over invented information

If the repository does not provide enough evidence:

Unknown

is preferable to an unsupported conclusion.

2. Validation Result

Return one overall status:

PASS
PASS_WITH_WARNINGS
FAIL
Enter fullscreen mode Exit fullscreen mode

PASS

The evidence model is sufficiently accurate and complete for documentation generation.

PASS_WITH_WARNINGS

The model can proceed, but there are known limitations that must appear in the documentation.

FAIL

The model contains significant errors, unsupported claims, contradictions, or missing evidence that could make the architecture documentation misleading.

3. Repository Coverage Check

Confirm that Step 1 examined the relevant repository artefacts.

Check for:

  • application source code
  • configuration
  • build files
  • dependency definitions
  • tests
  • deployment manifests
  • Dockerfiles
  • Helm charts
  • infrastructure definitions
  • CI/CD configuration
  • repository documentation
  • scripts
  • messaging configuration
  • API definitions

For each category report:

Area Found Analysed Evidence Status

If a category does not exist, record:

Not present

Do not treat absence as a validation failure.

4. Evidence Traceability Check

For every significant architectural claim, verify that evidence exists.

Check:

  • source file path
  • class/function/configuration
  • relevant implementation
  • configuration
  • relationship evidence

Example:

Claim:
Payment Service consumes order.created.

Evidence:
src/messaging/OrderCreatedConsumer.java
Kafka consumer configuration
Topic configuration: order.created

Classification:
Verified
Enter fullscreen mode Exit fullscreen mode

Flag any claim that has:

  • no evidence
  • vague evidence
  • incorrect source location
  • evidence that does not support the claim

5. Classification Check

Check whether each significant item has the correct classification.

Verified

Directly supported by repository evidence.

Inferred

Reasonably derived from multiple pieces of evidence but not explicitly established.

Unknown

Cannot be established from the repository.

Flag cases where:

  • Verified is actually only an inference
  • Inferred should be Unknown
  • Unknown is incorrectly presented as fact

6. Component Validation

Validate the extracted application components.

For each significant component:

  • Does it exist?
  • Does the source location exist?
  • Does its described responsibility match the implementation?
  • Are relationships supported?
  • Is the component actually architecturally significant?

Flag:

  • invented components
  • duplicated components
  • incorrect names
  • incorrect responsibilities
  • components described at the wrong abstraction level

Do not require every source-code class to appear as an architecture component.

7. Processing Flow Validation

Validate every extracted major processing flow.

For each flow check:

Trigger
   ↓
Entry Point
   ↓
Processing
   ↓
External Interaction
   ↓
Persistence / Messaging
   ↓
Output
Enter fullscreen mode Exit fullscreen mode

Confirm that each significant step has source evidence.

Check that:

  • sequence is plausible from actual code
  • synchronous interactions are not labelled asynchronous
  • asynchronous interactions are not labelled synchronous
  • error paths are supported
  • persistence is supported
  • external calls are supported

Flag flows where the extraction jumps from:

A → D

without evidence for:

A → B → C → D

8. API Validation

For every extracted API validate:

  • endpoint/path
  • HTTP method where applicable
  • implementation
  • request/input
  • response/output
  • authentication
  • downstream calls

Confirm that API claims match actual source definitions.

Flag:

  • endpoints that do not exist
  • incorrect HTTP methods
  • incorrect request/response descriptions
  • unsupported security claims

9. Messaging Validation

This is a high-priority validation area.

For every messaging relationship validate:

Producer
    ↓
Topic / Queue
    ↓
Consumer
Enter fullscreen mode Exit fullscreen mode

Check:

  • technology
  • topic/queue name
  • producer
  • consumer
  • message/event
  • configuration
  • serialization/schema if available
  • retry behaviour
  • acknowledgement behaviour
  • dead-letter handling

Do not accept:

Kafka dependency exists
→ Kafka integration exists
→ Event-driven architecture
Enter fullscreen mode Exit fullscreen mode

without evidence of actual messaging behaviour.

Flag orphaned relationships such as:

Topic exists
but no producer or consumer evidence was found.
Enter fullscreen mode Exit fullscreen mode

Unless the repository genuinely only contains one side of the integration.

10. Persistence Validation

For every database or datastore:

Check:

  • connection/configuration evidence
  • actual usage
  • read/write behaviour
  • relevant repositories or adapters
  • transaction evidence where claimed

Do not claim:

transactional boundary

unless source evidence supports it.

Do not claim:

database is the system of record

unless the repository provides evidence sufficient to support that statement.

11. External Dependency Validation

For every external system check:

  • source/configuration evidence
  • protocol
  • endpoint
  • interaction direction
  • authentication
  • actual usage

Distinguish:

Configured dependency
Enter fullscreen mode Exit fullscreen mode

from:

Actively used dependency
Enter fullscreen mode Exit fullscreen mode

A URL in configuration does not by itself prove that the application actively calls the system.

12. Deployment Validation

Validate deployment claims against actual repository artefacts.

Check:

  • Dockerfile
  • Kubernetes manifests
  • Helm charts
  • Terraform
  • CloudFormation
  • AWS SAM
  • CI/CD pipelines
  • deployment scripts

Validate claims about:

  • runtime platform
  • Kubernetes resources
  • AWS resources
  • storage
  • networking
  • secrets
  • configuration
  • health checks
  • scaling
  • deployment strategy

Do not infer runtime infrastructure that is not represented in the repository.

13. Security Validation

Validate security claims against implementation or configuration.

Examples:

mTLS
OAuth
OIDC
JWT
IAM
Vault
Secrets Manager
certificate authentication
Enter fullscreen mode Exit fullscreen mode

Check whether the evidence demonstrates actual use.

Do not treat a library dependency as proof that the security mechanism is active.

For example:

Spring Security dependency exists
Enter fullscreen mode Exit fullscreen mode

does not automatically prove:

OAuth authentication is enforced.
Enter fullscreen mode Exit fullscreen mode

14. Architecture Pattern Validation

This is a critical quality gate.

For every claimed architecture pattern, verify the required evidence.

Event-Driven Architecture

Look for:

  • event producers
  • event definitions
  • messaging transport
  • asynchronous consumers
  • event-driven processing

Kafka alone is insufficient.

Saga

Require evidence of:

  • distributed business transaction
  • multiple participating services
  • state progression
  • orchestration or choreography
  • compensation/recovery behaviour where applicable

Multiple services and Kafka alone are insufficient.

Outbox

Require evidence of:

  • business data persistence
  • event persistence
  • atomic or transactionally coupled write
  • relay/publisher mechanism

Database + Kafka is insufficient.

CQRS

Require evidence of:

  • distinct command and query responsibilities
  • separate models or processing paths
  • meaningful separation of write and read behaviour

Separate controller classes are insufficient.

API Gateway

Require evidence that a gateway is acting as an entry point for multiple downstream services or equivalent gateway behaviour.

Circuit Breaker

Require actual circuit-breaker configuration or implementation.

Idempotent Consumer

Require evidence of duplicate detection, idempotency keys, deduplication state, or equivalent behaviour.

For each pattern return:

Pattern Evidence Found Required Evidence Result
Event-Driven ... ... Confirmed/Possible/Not established
Saga ... ... Confirmed/Possible/Not established
Outbox ... ... Confirmed/Possible/Not established

15. Dependency Graph Validation

Validate every dependency relationship.

For each relationship:

Source
    ↓
Relationship
    ↓
Target
Enter fullscreen mode Exit fullscreen mode

Confirm that both endpoints exist or are explicitly classified as external.

Examples:

Order Service
    → produces
order.created
Enter fullscreen mode Exit fullscreen mode 
Payment Service
    → consumes
order.created
Enter fullscreen mode Exit fullscreen mode 
Payment Service
    → calls
Customer API
Enter fullscreen mode Exit fullscreen mode

Check for:

  • missing endpoints
  • impossible relationships
  • duplicate relationships
  • contradictory relationships
  • incorrect direction

16. Cross-Section Consistency

Compare information across the entire Architecture Evidence Model.

Check for contradictions such as:

Section A:
Payment Service consumes order.created.

Section B:
Payment Service produces order.created.
Enter fullscreen mode Exit fullscreen mode

Or:

Dependency Model:
SQS

Messaging Model:
Kafka
Enter fullscreen mode Exit fullscreen mode

Or:

Deployment:
Kubernetes

Infrastructure:
EC2 standalone deployment
Enter fullscreen mode Exit fullscreen mode

Do not assume one is correct.

Report the contradiction and identify the evidence required to resolve it.

17. Naming Consistency

Check that the same entity is not represented using inconsistent names.

Examples:

PaymentService
payment-service
Payment Service
Payments Service
Enter fullscreen mode Exit fullscreen mode

Determine whether these are:

  • the same entity
  • different entities
  • unresolved

Do not automatically merge entities unless evidence supports the relationship.

18. Completeness Check

Check whether the extraction contains enough information for the next stage.

Required areas:

  • application identity
  • significant components
  • major processing flows
  • APIs
  • messaging
  • persistence
  • external dependencies
  • deployment
  • security where identifiable
  • observability where identifiable
  • resilience where identifiable
  • architecture pattern evidence
  • dependency relationships
  • unknowns

A missing area is not automatically a failure.

Determine whether it is:

Not present

or:

Not analysed

or:

Missing evidence

These have different meanings.

19. Hallucination Check

Actively search for unsupported claims.

Ask:

"Could this statement have been produced from general software knowledge rather than repository evidence?"

If yes, inspect the evidence.

Flag claims such as:

  • assumed business purpose
  • assumed architecture pattern
  • assumed runtime behaviour
  • assumed scalability
  • assumed high availability
  • assumed disaster recovery
  • assumed security controls
  • assumed data ownership
  • assumed transaction boundaries

Unless supported by evidence.

20. Unknowns Check

Ensure uncertainty has not been hidden.

The validation must identify important unanswered questions.

Examples:

Business purpose unknown.

Runtime deployment environment not established.

Kafka topic ownership not established.

External system contract not available.

Production scaling configuration not present.
Enter fullscreen mode Exit fullscreen mode

These must remain explicit.

21. Validation Findings

Produce a findings table.

ID Severity Category Finding Evidence Required Action

Severity:

  • Critical
  • High
  • Medium
  • Low
  • Informational

Examples:

V001 | High | Evidence | Saga claimed without compensation/orchestration evidence | ... | Remove claim or provide evidence

V002 | Medium | Completeness | Deployment strategy not established | ... | Mark Unknown

V003 | Low | Naming | PaymentService and payment-service may refer to same component | ... | Confirm canonical identity
Enter fullscreen mode Exit fullscreen mode

22. Correction Instructions

If validation fails, produce explicit correction instructions for Step 1.

Format:

CORRECTION REQUEST

Finding:
...

Problem:
...

Required re-analysis:
...

Source locations to inspect:
...

Expected correction:
...

Do not modify unrelated areas.
Enter fullscreen mode Exit fullscreen mode

The correction request must be actionable.

23. Final Quality Gate

The Architecture Evidence Model may proceed to Step 3 only when:

  • no Critical findings remain
  • no High unsupported architectural claims remain
  • significant claims have evidence
  • classifications are reasonable
  • major relationships are traceable
  • contradictions are resolved or explicitly documented
  • unknowns are clearly identified
  • architecture patterns are evidence-based
  • naming is sufficiently consistent
  • the dependency model is usable for later aggregation

Return:

VALIDATION STATUS:
PASS
Enter fullscreen mode Exit fullscreen mode

or:

VALIDATION STATUS:
PASS_WITH_WARNINGS
Enter fullscreen mode Exit fullscreen mode

or:

VALIDATION STATUS:
FAIL
Enter fullscreen mode Exit fullscreen mode

Do not generate the final architecture documentation.

Step 3 will consume the validated Architecture Evidence Model.

Top comments (0)