Heka-Insights-Agent: Milestone M3-4: Fail-Fast Exporter Validation at Startup

Milestone M3-4: Fail-Fast Exporter Validation at Startup

On April 24, 2026, I completed M3-4 in heka-insights-agent:

• M3-4: Add configuration validation for exporter settings

This milestone closed a critical gap in the exporter foundation by removing silent fallback behavior and enforcing explicit startup failures for invalid exporter configuration.

Context

In Milestone M3-3, we routed output through the exporter interface and lifecycle:

• initialize()
• export(metrics)
• shutdown()

That established the architecture.

M3-4 focused on correctness and operational safety: startup must fail early when exporter configuration is invalid or points to an unimplemented adapter.

The Problem Before M3-4

Prior behavior was permissive:

• invalid EXPORTER_TYPE values defaulted back to console
• unimplemented exporter types also downgraded to console

Why this was risky:

• misconfiguration could go unnoticed in production
• users could think they were exporting to one backend while actually exporting to console
• behavior violated milestone acceptance criteria requiring explicit errors

M3-4 Goals

The implementation targeted three concrete outcomes:

1. invalid EXPORTER_TYPE must fail fast with a clear startup error
2. configured but unimplemented exporter adapters must fail fast with a clear startup error
3. docs must reflect strict validation (remove pre-M3-4 fallback messaging)

Implementation Summary

1) Strict validation in runtime config

Updated src/config/runtime.py in get_exporter_type(...).

Behavior now:

• missing EXPORTER_TYPE still defaults to console
• unsupported value now raises RuntimeError with explicit supported values list
• optional logger records an error message before raising

This changed exporter selection from “best-effort fallback” to deterministic validation.

2) Removed fallback from exporter factory

Updated src/exporters/factory.py in create_exporter(...).

Behavior now:

• console returns ConsoleExporter
• other configured values currently raise RuntimeError because adapters are not implemented yet

This is intentional. It prevents false confidence and makes readiness of each exporter explicit.

3) Documentation updated to M3-4 semantics

Updated:

• docs/configuration.md
• README.md

Both now state that:

• unsupported exporter values fail fast
• unimplemented exporters fail fast
• only missing value defaults to console

This removed the “fallback to console with warning” wording from the pre-M3-4 behavior.

Validation Performed

Validation included compile checks and runtime behavior checks.

Compile validation

• python3 -m compileall src passed

Behavior validation

Tested startup resolution paths for:

• missing EXPORTER_TYPE
• EXPORTER_TYPE=console
• EXPORTER_TYPE=invalid_value
• EXPORTER_TYPE=otlp_http (declared but not implemented adapter)

Observed results:

• missing value -> resolves to console
• console -> exporter creation succeeds
• invalid_value -> immediate RuntimeError with supported-values message
• otlp_http -> immediate RuntimeError stating exporter not implemented

These outcomes align with M3-4 requirements.

Why this change matters operationally

Fail-fast startup validation improves reliability in real deployments:

• misconfigurations are caught immediately
• no hidden routing to fallback output
• deployment behavior is explicit and auditable
• future exporter rollout can be gated by implementation readiness

This is especially important when teams automate deployments and rely on env-based configuration.

Architectural Impact

After M3-4, exporter behavior is now strictly layered:

• config validates selector
• factory enforces implementation availability
• runtime starts only on valid+implemented exporter path

That gives us a clean foundation for adding real transports (otlp_http, datadog_native, newrelic_otlp) without changing collector logic.

What’s Next

The natural next step after M3-4 is M3-5:

• document exporter lifecycle and responsibilities in architecture docs
• include startup validation expectations as part of operational guidance
• capture adapter implementation contract for future backend integrations

M3-4 makes sure the system fails loudly when exporter config is wrong, which is exactly what a transport foundation should do before adding real outbound integrations.