How to Build a Cybersecurity Metric Data Dictionary That Executives Can Actually Trust

Cybersecurity Metric Data Dictionary: Make Every Dashboard Number Reproducible

Audience: security program managers, GRC leads, control owners, audit coordinators, and consultants

A dashboard can look precise while hiding a basic operating problem: two people cannot reproduce the same number from the same evidence. The usual cause is not a charting tool. It is an undefined metric contract—scope, formula, denominator, source, timestamp, owner, validation, and exception handling live in different notes or in nobody’s notes.

NIST describes a measurement program as a consistent structure for collecting, analyzing, and communicating data used to monitor information security risk.[2] NIST also recommends documenting scope, a numeric measure, formula, target, implementation evidence, responsible parties, data source, time reference, and reporting format.[1] A practical data dictionary turns those ideas into one reviewable record per metric.

Boundary: This is an operational design guide, not legal advice, an audit opinion, certification, or evidence that a control is effective.

The failure pattern: one label, several calculations

Consider a dashboard tile named Critical vulnerabilities remediated on time. It might mean:

• closed findings divided by all findings created this month;
• findings closed within SLA divided by findings due this month;
• affected assets fixed divided by affected assets discovered;
• scanner instances marked resolved, even when exceptions are open;
• only production assets, or every inventoried asset.

All five can produce a percentage. They do not answer the same question. A target does not fix this ambiguity; the denominator and inclusion rules must be explicit first.

The minimum metric contract

Use one stable ID and capture these fields before a metric enters a recurring report.

Field	What to record	Acceptance test
Metric ID	Immutable identifier, not the display title	Renaming does not break history
Decision question	The decision this metric should inform	Reviewer can name the action it may trigger
Scope	Included/excluded systems, teams, environments, severities	Edge cases are listed
Grain	One row represents asset, finding, control, event, or period	No mixed grains in one calculation
Numerator	Exact qualifying records	Independently filterable
Denominator	Exact eligible population	Cannot silently be zero or shrink
Formula	Calculation, units, rounding, null handling	Two reviewers reproduce the value
Target and direction	Threshold and whether higher/lower is better	Target is not confused with forecast
Source	System/table/report and authoritative owner	Source can be located without tribal knowledge
As-of rule	Event time, extract time, timezone, reporting period	Late-arriving records have a rule
Freshness SLA	Maximum acceptable source age	Stale status is machine-testable
Validation	Schema, range, uniqueness, reconciliation tests	Failed checks stop or label publication
Exception policy	Waivers, accepted risks, duplicates, reopened items	Exceptions do not vanish from scope
Owner / approver	Calculation owner and business reviewer	Escalation path exists
Version	Definition version and effective date	Historical numbers remain interpretable

NIST treats formula as the calculation that results in a numeric expression and identifies implementation evidence as what computes and validates a measure.[1] NIST also identifies data sources broadly—including databases, tools, logs, organizations, and roles—so “from the dashboard” is not a sufficient source description.[1]

A worked example

Display name: Percentage of due critical findings remediated within approved SLA

Metric ID: VM-SLA-CRIT-001

Decision question: Where should remediation escalation focus next week?

Population: Critical findings whose SLA due date falls within the reporting period on in-scope production assets.

Numerator: Population records with validated remediation timestamp on or before SLA due date.

Denominator: Population records, including approved risk acceptances in a separately labeled exception count.

Formula: numerator ÷ denominator × 100; one decimal place; denominator zero returns N/A, not 100%.

As-of: 23:59 Asia/Seoul on the final day of the reporting period.

Freshness: scanner export and ticket snapshot each no older than the declared cutoff.

Validation: unique finding key; valid asset join; due date not before discovery date; exception expiry present; aggregate reconciled to source totals.

Version: 1.0, effective from the named reporting period.

This example intentionally keeps approved exceptions visible. Whether they belong in the numerator, denominator, or a separate companion metric is a governance choice; the important control is that the choice is documented and versioned.

Four quality gates before publication

1. Definition gate

Reject the metric if the scope, grain, numerator, denominator, or zero-denominator behavior is missing. NIST notes that scope distinguishes total risk from the risks currently measured.[1]

2. Data gate

Validate schema, required fields, allowed values, timestamps, uniqueness, joins, and totals before calculation. NIST states that checking data integrity, accuracy, and structure before analysis can address potential errors.[1]

3. Reproduction gate

Give the definition and a frozen input extract to a second reviewer. The metric passes only if the reviewer reproduces the published value within the stated rounding rule and records the same exceptions.

4. Communication gate

Publish the value with period, as-of timestamp, definition version, scope note, source freshness, and quality status. Regular measurement reporting can support accountability and continuous improvement, but only if recipients know what the result means.[2]

Versioning without rewriting history

Do not overwrite a metric definition when the population, formula, severity logic, source, or exception policy changes. Instead:

1. close the old version with an end date;
2. create a new version with an effective date and change reason;
3. decide whether historical backfill is valid and label it explicitly;
4. keep old dashboard values linked to the definition used at that time;
5. run both versions in parallel for one review cycle when the change is material.

A display-title correction may be non-material. A denominator change is material even when the title is unchanged.

A compact review checklist

Before approving a recurring metric, ask:

• Can two people independently calculate it from the named source?
• Is the denominator observable and protected from silent filtering?
• Are timezone, period boundaries, and late arrivals defined?
• Are stale, partial, and failed-source states different from zero?
• Are exceptions visible and expiration-aware?
• Does every published value point to a definition version?
• Is there a named decision or action when the target is missed?
• Can the team retain only necessary evidence while protecting sensitive operational data?

NIST’s program workflow moves from evaluating the security program through identifying measures, collecting and analyzing data, identifying corrective actions, and applying those actions.[2] A dictionary is useful only when it preserves that connection from number to decision.

What to automate—and what not to automate

Good automation candidates: schema checks, freshness checks, deduplication, denominator-zero handling, formula execution, reconciliation, version stamping, evidence hashes, and report generation.

Keep accountable review: deciding scope, accepting an exception, changing a target, interpreting material risk, and approving external claims. Automation can make a metric reproducible; it cannot make an ambiguous metric meaningful.

Related practical kit

If you need a spreadsheet-based workflow for defining cybersecurity metrics, recording review decisions, and preserving executive-reporting evidence, see the Cybersecurity Metrics, Executive Reporting & Decision Kit. The linked product is a related operational template, not evidence of demand, NIST endorsement, certification, audit assurance, or guaranteed compliance.

Sources

[1] https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v1.pdf — NIST SP 800-55v1: Measurement Guide for Information Security — Volume 1
> "Scope: Definition of what is considered in and out of scope. The scope helps explain aggregated risks and distinguish the total risk from the risks currently being measured."
> "Formula: Calculation that results in a numeric expression of a measure."
> "Data source: Location of the data to be used in calculating the measure, including databases, tracking tools, logs, organizations, and specific roles within the organization that can provide the required information."
> "Having a validation process in place to check the integrity, accuracy, and structure of the data provides a way to address potential errors before any analysis is done."
[2] https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v2.pdf — NIST SP 800-55v2: Measurement Guide for Information Security — Volume 2
> "A measurement program can provide a consistent and defined structure for collecting, analyzing, and communicating about data to monitor information security risks."
> "Regularly reporting on measurement and assessment results fosters transparency, promotes accountability for meeting performance targets, and encourages continuous improvements."
> "The workflow of implementing an information security measurement program consists of five major activities, as shown in Fig. 2:"

---

Disclosure: AI assisted with drafting and verification. The cited control logic was checked against the linked official sources before publication.