The White House has directed NSA-led agencies to create classified benchmarks for advanced AI cyber capabilities, but the public record does not verify a claim that the NSA is spending billions on the work. The distinction matters because government testing of models is now an official security policy, while the claimed scale, participating labs, and actual operational program remain opaque. Treating a single unnamed-source cost claim as settled fact would obscure the more important and verifiable governance change.
Key facts
- Executive Order 14409 was signed June 2, 2026 and directs classified benchmarking for advanced cyber capabilities.
- The NSA director is to determine when a system qualifies as a “covered frontier model.”
- The envisioned pre-release access framework is voluntary and can provide government access for up to 30 days before broader trusted-partner release.
- Primary source: Executive Order 14409.
The phrase “testing AI models” can mean several different things, which is why the executive order deserves more attention than a speculative price tag. Its Section 3 instructs the Treasury secretary, acting through the NSA director, together with the Department of Homeland Security acting through CISA, to build and maintain a classified process for evaluating advanced cyber capabilities. It also directs them to define the threshold for a covered frontier model and to share assessments with developers and researchers when appropriate.
The order contemplates early access, not a public leaderboard. Developers may voluntarily seek a determination and provide a covered model to the government for up to 30 days before releasing it more broadly to other trusted partners. The government could use that window to probe the system in classified environments and feed back findings. Crucially, the order says this framework must not be construed as mandatory licensing, pre-clearance, or permitting. A lab is not publicly required to obtain an NSA stamp before releasing a model.
This is an AI-cybersecurity story because the relevant capability is not merely whether a chatbot writes code. Advanced models can help find vulnerabilities, reason through system configurations, generate exploit components, and coordinate tool use. Testing has to consider model behavior, scaffolding, access, and guardrails together. The NSA Artificial Intelligence Security Center describes its mission as detecting and countering AI vulnerabilities and improving security across training data, models, abilities, and the machine-learning lifecycle.
A useful analogy is aviation certification, with an important difference. A public aircraft certification process has published rules and visible documentation. The classified AI process will necessarily conceal many test cases, thresholds, and results because disclosure could teach attackers or enable evasion. That secrecy may be legitimate for cyber evaluation, but it makes outside accountability harder. Citizens and companies can know a process exists without knowing which models failed which test or whether a threshold is sensible.
Congressional intelligence materials sketch a broader architecture. A Senate report describes an AISC pilot for information sharing with frontier developers and a secure research test bed supporting pre-deployment testing. Earlier authorization language discusses vendor-consented access to proprietary models, cost-recovery access for agencies, independent review, ongoing validation, and human oversight for high-impact intelligence uses. These are public descriptions of a proposed or directed capability, not proof that all elements are funded, built, or being used on named models today.
The disputed “billions” claim comes from a September 24 Washington Sun story citing two unnamed people familiar with classified intelligence estimates. It gives no public budget justification, appropriation, award, program element, vendor, model, contract vehicle, or exact figure, then says the exact amount is unclear. No primary source in the dossier corroborates it. Public Congressional Budget Office estimates attached to different legislative proposals are tens of millions across years, not evidence of NSA spending. The credible reporting line is therefore: substantial classified evaluation is officially directed; the price tag is unverified.
That correction is more than pedantry. “Billions” implies a mature, operational program with large compute purchases and major vendor arrangements. The official materials support a different picture: a security institution directed to create a capability, a voluntary access path, and a research/test-bed architecture. Those facts could eventually entail expensive compute and staffing, but public documents do not say that they already do, how much they cost, or which labs participate.
The policy case for independent evaluation is strong. Senator Maria Cantwell has said federal agencies and national laboratories should lead frontier-model testing for national-security purposes. AVERI, an independent evaluation organization, argues that lab safety and security claims are still heavily self-reported. An external test bed can reduce the conflict in which a vendor both makes a model and grades its most dangerous capabilities.
The strongest counterargument concerns secrecy and mission creep. A classified test regime can become an opaque channel through which government gets privileged access, influences release timing, or pressures suppliers without public standards. It may also be hard to distinguish defensive red teaming from preparation for offensive use. The reviewed public materials do not establish live-target operations or a requirement that vendors hand over weights; claims of that kind should not be inferred from “classified benchmarking.”
The immediate lesson for developers is practical. Cyber risk evaluation increasingly includes more than jailbreaking a chat window. It includes whether an agent can find alternate network paths, exploit tools, handle credentials, coordinate with other systems, and evade an evaluator. The OpenAI DNS incident shows why that broader test surface matters. The government's new framework says the same concern has moved from lab policy into national-security infrastructure—though its actual scale remains a question the public record cannot yet answer.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)