DEV Community: Omobolaji Adeyan

SPF, DKIM, and DMARC in Phishing Detection: Useful Signals, Not Magic Answers

Omobolaji Adeyan — Tue, 23 Jun 2026 01:11:05 +0000

Email authentication is valuable evidence, but it is not a verdict.

SPF, DKIM, and DMARC can help a receiving system decide whether a message came
through authorized infrastructure and whether authenticated identities align
with the visible sender. But in phishing detection, treating those checks as
magic answers can create the wrong kind of confidence.

That distinction shaped the email-authentication support I added to
PhishGuard AI, my
open-source Python phishing-detection project.

PhishGuard parses a trusted receiver's Authentication-Results header and
treats authentication failures as supporting evidence. It does not independently
query DNS, evaluate SPF policy, verify DKIM signatures, or perform DMARC
alignment. That means the caller must provide authentication results produced by
a receiver they trust.

This is deliberate. The goal is explainable phishing detection, not hidden
security theater.

What SPF, DKIM, and DMARC Prove

SPF can show whether a sending IP was authorized by the domain's SPF policy.

DKIM can show whether a message has a valid cryptographic signature from a
domain and whether signed content remained intact.

DMARC can show whether SPF or DKIM passed in a way that aligns with the visible
sender domain, according to the domain owner's policy.

Together, these checks are powerful. They make spoofing harder, support domain
protection, and give defenders useful evidence when investigating suspicious
email.

But they do not answer every question.

What They Do Not Prove

An SPF, DKIM, or DMARC pass does not prove that a message is safe.

Attackers can send malicious content from infrastructure they control. A
compromised mailbox, abused email service, or maliciously registered domain may
still pass authentication checks.

An authentication failure also does not automatically prove phishing.

Forwarding, mailing lists, and message transformations can break SPF or DKIM in
ways that are not malicious. If a detector treats every authentication failure
as a phishing verdict, it can punish legitimate mail and create avoidable false
positives.

This is why PhishGuard treats authentication results as evidence, not proof.

The Scoring Boundary

The implementation is intentionally conservative:

Authentication failures can increase risk when combined with suspicious content.
Authentication passes do not reduce risk, because authenticated infrastructure can still send malicious messages.
Missing, malformed, and unsupported authentication values remain unknown.
A single SPF failure is not treated as proof of phishing.

That last point matters. Security tools should preserve uncertainty when the
input signal is incomplete.

Regression Examples

Two regression cases demonstrate the boundary.

A legitimate forwarded message with SPF failure remained SAFE, moving from:

0.3149 -> 0.3595

A synthetic credential lure with SPF, DKIM, and DMARC failures moved from:

0.6525 SUSPICIOUS -> 0.8220 PHISHING

The difference is context. Authentication failure is more meaningful when other
features also look suspicious.

Explainable JSON Output

PhishGuard can export the feature set as JSON so the decision is inspectable:

{
  "verdict": "PHISHING",
  "probability": 0.8134,
  "features": {
    "spf_result": "fail",
    "dkim_result": "fail",
    "dmarc_result": "fail",
    "spf_auth_risk": 1.0,
    "dkim_auth_risk": 1.0,
    "dmarc_auth_risk": 1.0
  }
}

The same analysis can be exported as SARIF 2.1.0 for GitHub Code Scanning and
security automation workflows.

That is important because security findings should be reviewable by people and
usable by machines.

Why SARIF Matters

SARIF gives security tools a standard way to describe findings, rules,
locations, severity, fingerprints, and properties.

For PhishGuard, SARIF output means a phishing finding can carry the same
explainable feature evidence into CI and code-scanning workflows. A reviewer can
see not only that something was flagged, but why it was flagged.

This fits the larger design goal: make detection understandable enough that a
human can challenge it.

Tests and Safety Checks

The email-authentication work is covered by:

Case-insensitive parser tests
Forwarding false-positive regression coverage
CLI support
JSON and SARIF output examples
Repository policy checks
CodeQL
Packaging verification
Tests across Python 3.10 through 3.13

The project also documents what the current metrics do and do not establish.
Small fixtures and synthetic examples are useful regression evidence, but they
are not the same thing as population-level accuracy or production adoption.

Engineering Evidence

Project: github.com/omobolajiadeyan/phishguard-ai
GitHub Marketplace Action: PhishGuard AI Phishing Detector
Authentication implementation: PR #21
JSON and SARIF examples: EMAIL_OUTPUT_EXAMPLES.md
Current release: v0.5.1

The Lesson

Explainable security software should preserve uncertainty.

SPF, DKIM, and DMARC are useful signals. They can strengthen a phishing
assessment, especially when combined with suspicious content, URL structure, and
other behavioral features.

But they should not become shortcuts for thinking.

The more important engineering principle is this: a security tool should show
its work. It should make the signal, trust boundary, limitation, and scoring
impact visible enough for another person to test, question, and improve.

That is the direction I am building with PhishGuard AI.

Feedback I Would Value

If you work with email security, SOC workflows, GitHub Actions, or phishing
analysis, I would welcome technical feedback on three questions:

Should authentication failures influence risk differently for forwarded messages, mailing lists, and direct sender-to-recipient mail?
What additional safe regression examples would make this kind of scoring easier to trust?
Would SARIF output from a phishing detector be useful in your CI or security review workflow, or would another format be more practical?

Comments, issues, and small test cases are welcome. The most useful feedback is
specific: a false-positive case, a scoring concern, a documentation gap, or a
workflow where this output would or would not help.

From Single Files to Scenario Suites: Batch Validation in the OWASP Agent Security Regression Harness

Omobolaji Adeyan — Thu, 18 Jun 2026 17:33:45 +0000

Security regression testing is most valuable when it can cover your entire suite of scenarios in one command - not just a single file at a time. I recently contributed a change to the OWASP Agent Security Regression Harness that extends the validate command to accept files, directories, and glob patterns. The PR was merged with green CI. Here is what the problem was, what changed, and what it teaches about scoped open-source contributions.

What the Harness Does

The OWASP Agent Security Regression Harness is a framework for writing and running executable security regression tests for agentic applications and MCP-integrated systems. You define scenarios as YAML files describing security-relevant goals, expected behaviors, and assertions. The harness runs those scenarios against your agent and reports pass or fail results.

The validate command checks that a scenario file is structurally correct before you try to run it. Catching a malformed scenario early - before it silently passes CI - is exactly the kind of hygiene that prevents false confidence in a security regression suite.

The Problem: One File at a Time

Before this change, validate accepted a single file path:

agent-harness validate scenarios/goal_hijack/basic.yaml

That works fine with one scenario. Real suites grow. A project might have dozens of scenario files across multiple subdirectories, covering prompt injection, goal hijacking, secret disclosure, and other attack patterns. Validating each file individually does not scale - and more importantly, a CI job that only validates one file gives you incomplete coverage.

The practical risk: an invalid scenario sitting in a directory that was never validated can produce misleading results. A scenario that fails to parse might be skipped silently, reducing the effective coverage of your regression suite without any visible signal.

What Changed

PR #150 extended validate to accept one or more files, directories, or glob patterns:

# Validate one file
agent-harness validate scenarios/goal_hijack/basic.yaml

# Validate every scenario in a directory (recursive)
agent-harness validate scenarios/

# Validate a glob pattern
agent-harness validate "scenarios/**/*.yaml"

The command prints one line per scenario, a summary, and exits non-zero if any scenario is invalid - making it usable as a CI gate.

valid: goal_hijack_basic
valid: prompt_injection_system
invalid: secret_disclosure_draft - field 'assertions' is required
---
2 valid, 1 invalid

The core of the implementation is a _discover_scenario_files function that handles the three input cases cleanly:

def _discover_scenario_files(patterns: list[str]) -> list[Path]:
    """Return unique scenario files matched by files, directories, or globs."""
    scenario_files: list[Path] = []
    seen: set[Path] = set()

    for pattern in patterns:
        path = Path(pattern)
        if path.is_dir():
            matches = sorted(
                matched
                for suffix in ("*.yaml", "*.yml")
                for matched in path.rglob(suffix)
                if matched.is_file()
            )
        else:
            glob_matches = sorted(
                Path(match) for match in glob.glob(pattern, recursive=True)
            )
            matches = glob_matches if glob_matches else [path]

        for match in matches:
            normalized = match.resolve()
            if normalized in seen or not match.is_file():
                continue
            seen.add(normalized)
            scenario_files.append(match)

    return scenario_files

Deduplication via resolved paths prevents a scenario from being validated twice when a glob pattern and an explicit file path both match the same file.

The nargs="+" change to the argument parser is the smallest part of the diff, but it is what makes the whole thing composable in CI:

- name: Validate all security scenarios
  run: agent-harness validate "scenarios/**/*.yaml"

The Review

The maintainer accepted the change with one round of review. The scope was deliberate: no unrelated refactors, no new dependencies, tests for each input mode (single file, recursive directory, mixed-validity glob), and documentation in the GitHub Actions guide. Keeping a contribution scoped to what the issue actually asked for is what makes a PR reviewable quickly.

CI ran lint (ruff), type checking (mypy), and the full test suite - 333 tests, 1 skipped, all scenario validation checks passing.

The Lesson

A validation command that only accepts one file at a time is not a CI-ready tool. The change is small - under 100 lines including tests - but it makes the harness meaningfully more useful for teams running scenario suites in automated pipelines.

If you work on security regression testing for AI systems or MCP-integrated applications, the OWASP Agent Security Regression Harness is worth looking at. The project has open issues for SARIF output, suite-level runners, and additional adapter support - all practical contributions with clear scope.

PR #150 on GitHub

PhishGuard AI

Omobolaji Adeyan — Tue, 09 Jun 2026 19:28:21 +0000

I have added explainable SPF, DKIM, and DMARC signals to PhishGuard AI, my open-source Python phishing-detection project.

The implementation parses a trusted receiver's Authentication-Resultsheader and treats authentication failures as supporting evidence rather thanproof of phishing.

Measured regression examples:

A legitimate forwarded message with SPF failure remained SAFE, moving from 0.3149 to 0.3595.
A synthetic credential lure with SPF, DKIM, and DMARC failures moved from 0.6525 SUSPICIOUS to 0.8220 PHISHING.

Pass results do not reduce the risk score because authenticated infrastructure can still send malicious messages. Missing, malformed, and unsupported values remain unknown.

The change includes case-insensitive parsing, forwarding false-positive coverage, CLI support, documentation, packaging verification, repository policy checks, CodeQL, and tests across Python 3.10 through 3.13.

Pull request and engineering evidence:
https://github.com/omobolajiadeyan/phishguard-ai/pull/21

The project now also has a one-minute safe demo and a guide for first-time contributors:

https://github.com/omobolajiadeyan/phishguard-ai

I welcome technically grounded feedback, testing, documentation improvements, and focused open-source contributions.

Cybersecurity #Python #EmailSecurity #OpenSource #DevSecOps

DEV Community: Omobolaji Adeyan

SPF, DKIM, and DMARC in Phishing Detection: Useful Signals, Not Magic Answers

What SPF, DKIM, and DMARC Prove

What They Do Not Prove

The Scoring Boundary

Regression Examples

Explainable JSON Output

Why SARIF Matters

Tests and Safety Checks

Engineering Evidence

The Lesson

Feedback I Would Value

From Single Files to Scenario Suites: Batch Validation in the OWASP Agent Security Regression Harness

What the Harness Does

The Problem: One File at a Time

What Changed

The Review

The Lesson

PhishGuard AI

Cybersecurity #Python #EmailSecurity #OpenSource #DevSecOps

Suggested Featured Link