DEV Community: Suny Choudhary

System Prompt Leakage: Why Hidden AI Instructions Are Not a Security Boundary

Suny Choudhary — Tue, 16 Jun 2026 09:53:52 +0000

Most developers treat system prompts like hidden configuration.

That is the mistake.

In an LLM application, a system prompt is not source code sitting safely behind access controls. It lives inside the model’s context, where user instructions, external content, and conversation history can influence what the model does next.

That is why system prompt leakage is not just an edge case. It is a design risk.

System prompts are often treated as confidential assets within LLM applications. They define the behavior of the model, establish safety boundaries, restrict topics, specify workflows, and determine how external tools should be used. Many developers assume these instructions remain hidden from end users and therefore provide a reliable control mechanism.

In practice, however, that assumption is increasingly being challenged. The growing prevalence of system prompt leakage demonstrates that prompts are not equivalent to source code or traditional secrets. Unlike configuration files stored behind access controls, prompts exist within the model's context window and ultimately become part of the information the model processes. As a result, attackers can attempt to manipulate conversations in ways that expose or reconstruct those hidden instructions.

This distinguishes prompt leakage from prompt injection. Prompt injection focuses on influencing model behavior, whereas prompt leakage seeks to reveal the instructions governing that behavior. Access to those instructions can provide valuable information about safety mechanisms, tool usage, and application logic, allowing attackers to craft more effective adversarial inputs.

Ultimately, prompts are inputs, not security boundaries. Treating them as inherently secret creates assumptions that modern AI systems cannot always guarantee.

How Prompt Extraction Attacks Work

A prompt extraction attack aims to reveal the hidden instructions that govern an LLM application's behavior. Rather than exploiting software vulnerabilities, these attacks rely on manipulating the model through carefully crafted inputs and conversational techniques.

Common approaches include direct requests, role-playing scenarios, context switching, translation prompts, and multi-turn conversations designed to gradually expose internal instructions. In many cases, attackers do not ask the model to violate its rules explicitly. Instead, they reframe the interaction in ways that encourage the model to reveal information unintentionally.

The underlying challenge is that language models do not possess an inherent distinction between system instructions and user instructions. Both ultimately exist within the same context window and are processed together. This makes prompt secrecy a relatively weak security boundary.

Frameworks such as the responsible AI security framework emphasize that protecting AI systems requires securing prompts, context, and runtime interactions rather than relying solely on hidden instructions.

Ultimately, the effectiveness of a prompt extraction attack does not depend on breaking the model. It depends on persuading it. As a result, organizations should assume that prompts may eventually be exposed and design their systems accordingly.

Common Techniques Used to Leak System Prompts

Prompt extraction attacks rarely rely on sophisticated exploits. More often, they take advantage of the model's tendency to interpret instructions conversationally. Attackers employ a variety of techniques to persuade the model into revealing information that was intended to remain hidden.

Some of the most common approaches include:

Roleplay Attacks

The attacker reframes themselves as a developer, security auditor, or administrator and asks the model to disclose the instructions it was supposedly given for review purposes.

Instruction Hierarchy Manipulation

User prompts attempt to reinterpret or override hidden instructions by introducing new contexts or priorities.

Translation and Summarization Tricks

The model is asked to paraphrase, explain, or translate its own operating rules, often exposing portions of the system prompt in the process.

Context Window Exploitation

Long conversations can gradually weaken earlier instructions, increasing the likelihood of unintended disclosures.

Indirect Prompt Injection

External content such as web pages, documents, or emails influences the model into revealing internal instructions without the attacker's prompt directly requesting them.

The challenge with system prompt leakage is that these attacks rarely involve compromising the underlying model. Instead, they exploit the way language models interpret and prioritize natural language.

Why Prompt Security Requires More Than Hiding Prompts

Relying on secrecy alone is not a sustainable defense against prompt extraction. Hidden instructions may provide some friction for attackers, but they should not be treated as the primary mechanism protecting an AI application. Effective prompt security requires a broader, defense-in-depth approach.

Several principles are particularly important:

Prompts Should Not Be Treated as Secrets

System instructions may eventually be disclosed through adversarial interactions. Organizations should assume that prompt exposure is possible and design systems accordingly.

Layered Security Provides Greater Resilience

Runtime inspection, policy enforcement, and input validation offer stronger protections than secrecy alone.

Least-Privilege Tool Access Limits Impact

Even if prompts are revealed, restricting permissions prevents attackers from escalating the consequences of prompt exposure.

Continuous Monitoring Improves Detection

Observing prompt interactions and anomalous behavior helps identify extraction attempts before they result in broader compromise.

Organizations seeking to secure homegrown AI applications should recognize that prompt confidentiality cannot be guaranteed indefinitely. Instead, systems should be designed so that exposing internal instructions does not automatically compromise security.

System Prompt Leakage Is a Design Problem, Not a Model Problem

Ultimately, system prompt leakage is not a failure of the model itself. Nor is a prompt extraction attack necessarily evidence that the underlying LLM is defective. In most cases, prompt disclosure reflects architectural assumptions that treat hidden instructions as security controls rather than operational guidance.

Organizations should instead adopt a defense-in-depth approach based on permission boundaries, runtime controls, monitoring, and least-privilege access. The objective is not to guarantee that prompts remain secret indefinitely, but to ensure that their exposure does not compromise the application.

As AI applications continue to evolve, the challenge will not be preventing prompts from ever being revealed. It will be designing systems that remain secure even when they are.

This is also where tools like LangProtect become relevant. The goal is not to pretend prompts will stay hidden forever. The goal is to add runtime inspection, policy enforcement, monitoring, and audit visibility around AI interactions so that prompt exposure does not automatically become system compromise.

This Meta incident is a good reminder that once AI can trigger account recovery or access changes, it is no longer “just support.” It is security infrastructure.

Suny Choudhary — Mon, 08 Jun 2026 10:35:44 +0000

Suny Choudhary

Jun 8

Meta’s AI Support Hack Is a Warning for Every Team Automating User Access

#meta #ai #security #cybersecurity

4 min read

Meta’s AI Support Hack Is a Warning for Every Team Automating User Access

Suny Choudhary — Mon, 08 Jun 2026 10:35:15 +0000

The recent Meta AI support incident should make every engineering and security team pause.

Not because Meta got hacked in some cinematic way.

But because the attack looks painfully simple from the outside.

Attackers reportedly abused Meta’s AI-powered support flow to take over Instagram accounts. The system was meant to help users recover access. Instead, it became a shortcut for attackers to change account access and reset credentials.

That is the real lesson here.

AI did not need to be “evil.”
It just needed too much authority with too little verification.

The problem is not AI support

AI support is not the problem.

Most companies are moving in this direction anyway. Support teams are overloaded. Users expect instant help. Account recovery, onboarding, refunds, compliance requests, and internal IT helpdesk tickets are all obvious places where AI can reduce wait time.

The issue starts when AI is allowed to act on sensitive workflows without strong guardrails.

There is a big difference between:

“Explain how account recovery works.”

and

“Change the recovery email for this account.”

The first is information.
The second is privilege.

Once an AI system can trigger privileged actions, it becomes part of your security boundary.

Most teams are still not treating it that way.

AI workflows need the same security thinking as APIs

Developers would never expose an API endpoint that lets someone reset another user’s account without authentication, rate limits, logging, and authorization checks.

But when the same action is wrapped inside a chatbot, teams sometimes treat it as a UX feature instead of an access control surface.

That is dangerous.

An AI support agent can be manipulated through prompts, incomplete context, weak verification, confusing instructions, or social engineering. If the agent has access to tools, it can do real damage.

Not theoretical damage.

Real actions:

Reset passwords
Change email addresses
Reveal account details
Approve refunds
Modify permissions
Pull internal data
Trigger workflows
Create support escalations

The more useful the agent becomes, the more dangerous it becomes if controls are weak.

The missing layer is enforcement

A lot of companies think AI safety means writing better prompts.

That helps, but it is not enough.

A system prompt that says “never change account access unless verified” is not a security control. It is guidance.

Security needs enforcement outside the model.

For sensitive AI workflows, teams need to ask:

Can the AI perform privileged actions?

What identity checks happen before those actions?

Can the model be tricked into skipping those checks?

Are risky prompts inspected before tool execution?

Are responses scanned before they reach the user?

Are all AI decisions logged?

Can security teams replay what happened?

Is there a human approval path for high-risk actions?

This is where AI security has to become more practical.

At LangProtect, this is the exact direction we think enterprises need to move toward. Not blocking AI. Not slowing teams down. But putting a security layer around prompts, responses, files, and AI-triggered workflows before they become incidents.

Because once AI is connected to real business actions, visibility alone is not enough. You need policy enforcement.

AI agents should not be trusted by default

The big mistake is assuming an AI agent is safe because it works well in normal cases.

Security failures do not happen in normal cases.

They happen when someone intentionally pushes the edge of the system.

A good AI support agent may handle 99 percent of users correctly. But the 1 percent edge case can be expensive if the agent has access to account recovery, financial data, admin functions, or internal tools.

That means AI agents need least privilege.

They should only access the data they need.
They should only call the tools they are allowed to call.
They should escalate high-risk actions instead of completing them automatically.
They should be monitored like production infrastructure, not treated like a help widget.

What teams should fix before shipping AI support

If your team is building AI support, AI agents, or AI copilots, do not wait for a public incident.

Start with these basics:

Separate advice from action
Let AI explain steps, but require strong verification before executing anything sensitive.
Put policy checks before tool calls
The model should not be the final judge of whether an action is safe.
Log every prompt, response, and action
If something goes wrong, you need evidence, not guesses.
Add human review for high-impact workflows
Account recovery, permission changes, payments, refunds, and data exports should not be fully autonomous by default.
Test for prompt injection and social engineering
Do not just test happy paths. Test manipulation attempts.
Scan AI inputs and outputs in real time
Sensitive data, malicious instructions, credential exposure, and unsafe actions should be detected before they move further.

The real takeaway

This Meta incident is not only about Instagram.

It is about where AI is going next.

AI is moving from answering questions to taking actions. That changes the risk model completely.

When AI only generates text, a bad answer is embarrassing.

When AI controls workflows, a bad answer becomes an account takeover, data leak, payment fraud, or compliance failure.

That is the shift every engineering team needs to understand.

AI automation is powerful. But without security controls around it, you are not just scaling support.

You are scaling trust in a system that attackers are already learning how to manipulate.

Google Patched an Actively Exploited Android Flaw. Enterprises Should Treat This as an AI Security Problem Too

Suny Choudhary — Wed, 03 Jun 2026 12:17:17 +0000

Google recently patched an actively exploited Android flaw affecting millions of devices.

Most teams will read that sentence and treat it as a mobile patching issue.

That is not wrong.

But it is incomplete.

In 2026, a compromised mobile device is not just a device problem. It can become an AI security problem too.

Employees use Android phones for work email, SaaS dashboards, MFA approvals, browser sessions, file access, chat apps, and AI tools. They paste work data into ChatGPT, Gemini, Claude, Copilot, and other AI platforms. They approve logins from mobile devices. They read internal documents on mobile browsers. They move between corporate apps and personal tools all day.

So when an Android vulnerability is actively exploited, security teams should not only ask:

“Did we patch the device?”

They should also ask:

“What enterprise data, AI activity, SaaS access, and prompt workflows could that device expose if compromised?”

That is the part most organizations still miss.

What Google Patched

According to Security Affairs, Google released its June 2026 Android security updates, fixing 124 vulnerabilities across Android.

The most important one is CVE-2025-48595.

It is an Android Framework vulnerability with a CVSS score of 8.4. It affects Android 14, Android 15, Android 16, and Android 16 QPR2.

Google said there are indications that CVE-2025-48595 may be under limited, targeted exploitation.

The issue is caused by an integer overflow that can lead to code execution and privilege escalation on a vulnerable device. That matters because privilege escalation can allow an attacker to move from limited access to deeper control over the system.

Security Affairs also noted that Google has not publicly disclosed the attacker, delivery method, or victim count.

That lack of detail is normal in actively exploited vulnerability cases. But it also means enterprises should not wait for perfect information before acting.

If a vulnerability is already being exploited, the patch window is not theoretical anymore.

It is live.

Why Privilege Escalation Matters

Privilege escalation is not always the first step in an attack.

Often, it is the step that makes the first foothold dangerous.

A malicious app, phishing link, exploit chain, or compromised device session may start with limited access. But if privilege escalation succeeds, the attacker may gain deeper access to device resources, app data, tokens, files, clipboard activity, browser sessions, or enterprise applications.

To be clear, there is no public evidence that CVE-2025-48595 is being used to steal AI prompts or SaaS data.

That is not the claim.

The real point is that a flaw like this can become part of a broader attack chain.

And that chain can reach enterprise AI activity if the compromised device is used for AI tools, work data, SaaS apps, authentication, and browser-based workflows.

This is why mobile endpoint security now overlaps with AI security.

Not because the vulnerability itself is an AI flaw.

But because the device is where enterprise data meets AI.

Why This Becomes an AI Security Problem

AI adoption has changed the value of endpoint compromise.

A few years ago, a compromised phone might expose email, files, contacts, or login sessions.

That was already serious.

Now add AI usage into the same environment.

Employees may use mobile devices to:

Access ChatGPT, Gemini, Claude, Copilot, or Perplexity

Paste customer data into AI tools

Summarize internal documents

Draft sales emails from CRM notes

Analyze screenshots or files

Use AI through mobile browsers

Approve logins through MFA apps

Open SaaS dashboards from unmanaged networks

Move data between personal and corporate accounts

This creates a wider risk surface.

Employees now use Android devices to access AI tools, SaaS apps, browser sessions, work email, and authentication workflows. That means mobile compromise can expose more than files. It can expose the data employees send into AI systems.

This is why AI data leakage prevention needs to include the endpoint layer, not just the AI model or chatbot interface.

The risk is not only “someone pasted sensitive data into AI.”

The risk is:

Who pasted it?

From which device?

Was the device managed?

Was the session protected?

Was the data classified before submission?

Was the AI tool approved?

Was the prompt logged?

Was the action blocked, warned, or allowed?

Most organizations cannot answer those questions clearly today.

That is the AI security gap.

The Patch Gap Is the Real Enterprise Risk

Android patching has a known structural problem.

Pixel devices usually receive updates quickly. Other manufacturers often require additional testing, customization, and carrier or OEM rollout time before patches reach users.

Security Affairs pointed out that this fragmented update model can leave some users exposed for weeks or months after a vulnerability becomes public.

Attackers understand this.

Once a patch is released, defenders get a fix. But attackers also get a signal. They can reverse engineer patches, identify vulnerable code paths, and hunt for devices that have not yet updated.

For enterprises, the risk is not just whether Google released the patch.

The real risk is whether every employee device that touches business systems has actually received it.

This is where security posture becomes messy.

Some devices are corporate-managed.

Some are BYOD.

Some access work apps through personal profiles.

Some have outdated OS versions.

Some use unmanaged browsers.

Some access AI tools through personal accounts.

Some are invisible to IT.

That last part matters most.

You cannot protect what you cannot see.

Mobile Devices Are Becoming Shadow AI Gateways

Shadow AI is usually discussed as a web or SaaS issue.

Employees use unapproved AI tools. They paste sensitive data. They create personal accounts. Security teams lose visibility.

But mobile devices make the problem harder.

An employee may use an approved AI tool on a managed laptop during the day, then use a personal Android phone at night to continue the same work. They may paste notes into a mobile AI app. They may upload a screenshot. They may summarize customer information. They may ask AI to rewrite confidential internal content.

From the employee’s perspective, this feels harmless.

From a security perspective, it creates a blind spot.

The organization may have no visibility into:

Which AI tools are being used

Which accounts are being used

Which data is being pasted

Whether prompts contain PII, credentials, source code, or financial data

Whether the activity happens from a patched or unpatched device

Whether the AI tool is approved or unmanaged

This is why Shadow AI discovery is becoming a real requirement, not just a governance nice-to-have.

The Android flaw is a reminder of the same bigger issue.

Enterprise data does not only move through managed laptops anymore.

It moves through browsers, mobile devices, AI tools, personal accounts, copied text, files, screenshots, chats, and prompts.

If security teams only monitor the old paths, they will miss the new ones.

Why Traditional Endpoint Thinking Is Not Enough

Traditional endpoint security focuses on device health, malware detection, patch status, and access control.

Those still matter.

But AI workflows introduce a different question:

What data is leaving the endpoint through AI interactions?

A device can be patched and still leak data into AI tools.

A user can pass MFA and still paste confidential information into an unmanaged chatbot.

A browser session can be legitimate and still move sensitive content into an unapproved AI assistant.

That is why endpoint security and AI security need to work together.

Security teams need to know not only whether the device is secure, but also what the user is doing with enterprise data once access is granted.

That means AI security cannot start at the model.

It has to start at the point of interaction.

Prompt fields.

File uploads.

Copy paste actions.

Browser sessions.

Mobile AI apps.

SaaS workflows.

Agent actions.

Anywhere enterprise data touches AI, security needs visibility and enforcement.

What Security Teams Should Do Now

The first step is obvious.

Patch Android devices quickly.

But stopping there is lazy security.

Enterprises should treat this kind of vulnerability as a trigger to review mobile AI exposure more broadly.

Here is what security teams should do.

First, identify which Android devices can access work email, SaaS apps, cloud storage, AI tools, and authentication workflows.

Second, prioritize high-risk users. Executives, engineers, finance teams, legal teams, security teams, HR, and anyone with access to customer data or source code should be patched and checked first.

Third, enforce device posture checks. Sensitive apps should not be accessible from outdated or non-compliant devices.

Fourth, review BYOD access. If personal devices can access AI tools and enterprise SaaS systems, the organization needs clear policy and visibility.

Fifth, monitor AI prompt and file flows. Security teams need visibility into what AI tools employees use, what data they paste, and whether that activity happens through managed or unmanaged devices.

This is where an enterprise AI security firewall becomes useful because it gives teams a control layer around AI interactions, not just network access.

Sixth, classify sensitive data before it enters AI tools. PII, PHI, credentials, secrets, source code, financial data, and internal documents should be detected before submission.

Seventh, log AI activity for audit readiness. If a sensitive prompt is blocked or allowed, there should be a record.

Eighth, align AI usage policies with endpoint policy. AI governance cannot sit in a PDF while device access remains unmanaged.

The practical goal is simple:

Do not let an unpatched or unmanaged endpoint become the easiest path into enterprise AI data.

The Real Lesson From CVE-2025-48595

CVE-2025-48595 is an Android vulnerability.

But the lesson goes beyond Android.

Every modern enterprise runs on connected workflows.

A mobile device connects to SaaS.

SaaS connects to identity.

Identity connects to MFA.

MFA connects to account recovery.

Browsers connect to AI tools.

AI tools receive prompts, files, screenshots, code, notes, and customer data.

That means security teams cannot treat mobile, SaaS, identity, and AI as separate risk categories anymore.

Attackers do not care about your internal categories.

They care about paths.

A compromised endpoint is a path.

A personal AI account is a path.

A pasted customer list is a path.

An unmonitored prompt is a path.

An unpatched Android device used for work is a path.

The question is whether your security program can see the path before it becomes an incident.

AI Security Starts Before the Prompt

Many teams still think AI security starts when a prompt reaches the model.

That is too late.

AI security starts earlier.

It starts with the device.

The browser.

The identity session.

The file.

The clipboard.

The app.

The account.

The user action.

By the time sensitive data reaches an AI tool, the organization has already lost part of the control battle.

That does not mean AI adoption should be blocked.

Blocking AI usually pushes employees into worse behavior. They use personal accounts, personal devices, and unapproved apps. That creates even less visibility.

The better answer is controlled enablement.

Let employees use AI.

But enforce security where data moves.

Monitor prompt fields.

Classify sensitive content.

Warn or block risky submissions.

Detect unmanaged AI tools.

Log activity.

Tie AI access to device posture.

Treat mobile AI usage as part of the enterprise security surface.

That is how AI adoption becomes safer without killing productivity.

Final Takeaway

Google patched an actively exploited Android flaw.

Security teams should patch fast.

But they should also zoom out.

The bigger issue is that employee devices are now connected to AI workflows, SaaS apps, identity systems, and sensitive enterprise data. A mobile endpoint compromise can become more than a device incident. It can become a data leakage incident, an identity incident, or an AI governance failure.

AI security does not start at the model.

It starts wherever enterprise data touches AI.

In many organizations, that place is now the employee’s phone.

[Boost]

Suny Choudhary — Wed, 27 May 2026 08:00:50 +0000

Suny Choudhary

May 27

AI Adoption Security: The Missing Layer in Every Enterprise Security Stack

#ai #cybersecurity #security #devops

3 min read

AI Adoption Security: The Missing Layer in Every Enterprise Security Stack

Suny Choudhary — Wed, 27 May 2026 08:00:44 +0000

Most enterprise security stacks were designed around predictable infrastructure. DLP monitors files, SIEM tracks logs, IAM governs identities, and endpoint tools inspect devices and applications.

AI systems change how all of those layers behave. Prompts, retrieval pipelines, copilots, plugins, memory layers, and AI agents introduce entirely new operational workflows inside enterprise environments. Sensitive data now moves conversationally, context is retrieved dynamically, and AI systems increasingly make decisions or trigger downstream actions during runtime.

That is why enterprise AI adoption security is becoming a separate security challenge rather than simply an extension of existing controls. The issue is not that current enterprise security tooling is obsolete. It is that most of it was never designed to observe AI interaction layers deeply.

And as AI adoption accelerates across organizations, that visibility gap is becoming increasingly difficult to ignore.

AI Introduced A New Runtime Layer Most Security Tools Don’t Inspect

AI systems introduced a runtime interaction layer that most traditional enterprise controls still inspect only partially. Prompts move through browsers, copilots, retrieval systems, APIs, plugins, and orchestration layers continuously during execution.

That changes how enterprise data moves operationally. Sensitive information is no longer limited to documents or structured transfers. It now flows through prompts, contextual memory, AI-generated outputs, and connected workflow systems that interact dynamically during runtime. In many environments, these interactions happen invisibly from the perspective of traditional monitoring tools.

This is why modern AI security architecture increasingly focuses on runtime visibility rather than static infrastructure inspection alone. Organizations need visibility into how prompts move, what context gets retrieved, which systems AI interacts with, and where enterprise data travels after inference begins.

That is also where frameworks like practical enterprise AI security framework) become important. AI adoption security is no longer just about controlling access to AI tools. It is about governing the operational interaction layer forming around them.

Why Existing Enterprise Controls Miss AI Risk Structurally

The problem is not that enterprise security tools are poorly designed. The problem is that AI systems changed the operational model underneath them.

Traditional controls were built around infrastructure events, while AI systems operate through contextual interactions happening dynamically during runtime.

In practice:

DLP monitors files, not prompts
SIEM tracks logs, not conversational reasoning
IAM governs identities, not autonomous AI actions
CASB sees applications, not AI interaction flows
Existing controls rarely inspect retrieval-layer context movement

This is also why discussions around why traditional controls fail at the AI layer are becoming increasingly relevant. AI systems continuously retrieve context, trigger workflows, interact with external tools, and move enterprise data across operational layers that many traditional controls cannot fully observe.

That creates entirely new enterprise AI governance controls challenges, especially once AI systems become deeply integrated into everyday enterprise workflows.

The Missing Layer Is Operational AI Visibility And Governance

The missing layer in most enterprise environments is operational AI governance during runtime itself. Organizations already monitor infrastructure heavily. What they often lack is visibility into how AI systems interact with enterprise data while workflows are actively executing.

That requires controls around:

Prompt and response inspection
Monitor sensitive information before prompts reach models and before outputs move into workflows or downstream systems.
Context governance
Control how retrieval systems, memory layers, plugins, and AI agents access enterprise context during execution.
Runtime policy enforcement
Apply security and governance controls dynamically while AI interactions are happening instead of relying only on static policies.
Continuous AI activity logging
Create visibility into prompts, outputs, tool calls, and cross-system AI interactions operationally.

This is also why resources like complete enterprise guide to AI adoption security are becoming more important. AI security increasingly depends on governing interactions, context movement, and runtime workflows rather than only protecting infrastructure boundaries.

AI Adoption Security Will Become A Core Enterprise Security Layer

AI systems are no longer experimental tooling sitting outside enterprise operations. They are increasingly becoming embedded into customer workflows, internal productivity systems, decision-making pipelines, and operational infrastructure itself.

That shift is why enterprise AI adoption security is becoming a foundational security layer rather than an optional add-on. Organizations are realizing that traditional controls still matter, but they are no longer sufficient on their own once AI systems begin interacting dynamically with enterprise data and workflows.

The future enterprise security stack will not replace DLP, SIEM, IAM, or existing governance systems. It will add an AI interaction and governance layer above them, one focused on prompts, context movement, runtime behavior, retrieval systems, and AI-driven operational workflows.

Because the missing layer in modern enterprise security is no longer visibility into infrastructure alone. It is visibility into how AI systems interact, retrieve context, and make decisions operationally.

AI governance is not just a policy problem anymore. CISOs now need runtime visibility, prompt inspection, enforcement, context governance, and audit logs before AI adoption gets ahead of control. Shared a practical breakdown here.

Suny Choudhary — Thu, 21 May 2026 11:04:39 +0000

Suny Choudhary for Langprotect

May 21

The 5 AI Adoption Security Controls Every CISO Needs Before Q3 2026

#ai #cybersecurity #security #devops

4 min read

The 5 AI Adoption Security Controls Every CISO Needs Before Q3 2026

Suny Choudhary — Thu, 21 May 2026 11:03:52 +0000

Enterprise AI adoption is already operational. Copilots, AI assistants, workflow automations, and internal agents are now embedded into daily workflows across most organizations.

The problem is that governance has not kept pace. Most enterprises are still writing policies and inventorying tools while employees continue integrating AI into operational systems faster than security teams can realistically monitor. By the time governance discussions happen, AI usage is often already widespread across the organization.

That is why enterprise AI security controls are becoming increasingly important heading into Q3 2026 and the EU AI Act enforcement timeline. The challenge is no longer whether enterprises will adopt AI. It is whether they can build enough visibility and operational control around AI usage before regulatory expectations catch up.

Control #1 and #2: Visibility and Prompt-Level Inspection

The first control CISOs need is visibility. Organizations cannot govern AI systems they cannot see. That means understanding which AI tools employees use, which workflows interact with AI systems, and how enterprise data moves during execution.

*In practice, this requires visibility into: *

Browser-based AI usage
AI plugins and connected SaaS tools
Internal AI workflows and copilots
Prompt and response activity across systems

This is where many AI governance controls still struggle. Traditional inventories were built for applications and infrastructure, not dynamic AI interactions happening across operational workflows.

The second control is prompt-level inspection. Sensitive information increasingly moves through conversational workflows rather than traditional files or databases. Enterprises need runtime inspection before prompts, uploads, or outputs reach the model itself.

*This includes controls around: *

Sensitive data detection
Prompt injection inspection
Output validation
Unauthorized context sharing

Without these two controls, organizations lose visibility at the exact layer where most AI operational risk now exists.

Control #3 and #4: Runtime Enforcement and Context Governance

Visibility alone is not enough. Organizations also need controls that actively govern AI behavior during execution, not just after deployment. This is where enterprise AI risk management becomes operational rather than policy-driven, and also where organizations begin realizing why existing security controls fail AI systems.

The third control is runtime enforcement. AI systems should be monitored and controlled while prompts, outputs, and tool calls are actively happening.

*This includes: *

Sensitive data filtering
Prompt injection detection
Output moderation
Tool-call restrictions

The fourth control is context governance. Modern AI systems continuously retrieve, retain, and reuse information across workflows, which creates entirely new data exposure paths.

Organizations need governance around:

Session memory retention
Retrieval-layer access
Plugin context boundaries
Context expiration policies

Most enterprise AI risk now emerges from how context moves between systems, not just from the model itself.

Control #5: Continuous Logging and Auditability

The fifth control is continuous logging and auditability. As AI systems become operational infrastructure, enterprises need a reliable record of how AI interactions occur across workflows, systems, and users. This is becoming a foundational part of any effective enterprise AI security framework.

*This requires visibility into: *

Prompt and response activity
Tool calls and downstream actions
Context retention and retrieval behavior
Cross-system AI interactions

Operational logging is becoming critical not just for investigations, but for governance and regulatory readiness as well. Organizations increasingly need evidence showing how AI systems handled data, what decisions were influenced, and which controls were active during execution.

Without continuous auditability, AI governance becomes difficult to prove operationally, especially as enterprise AI environments grow more dynamic and interconnected.

Why Traditional Security And Governance Models Break Down With AI

Traditional security models were designed around predictable systems. Applications had defined behaviors, users had scoped permissions, and data movement followed relatively structured paths. AI systems operate very differently.

Prompts dynamically change context, AI agents interact with external tools, and retrieval systems continuously pull information from multiple sources during execution. A single interaction may involve APIs, vector databases, plugins, logging systems, and downstream workflows simultaneously.

*This creates several governance gaps: *

Static policies struggle to govern dynamic AI behavior
Traditional DLP tools miss conversational data movement
Existing IAM systems were not built for autonomous AI actions
Security reviews often stop at deployment instead of runtime behavior

That is why AI governance is increasingly shifting from documentation-driven processes to operational control layers. The challenge is no longer simply approving AI systems. It is continuously governing how they behave after deployment across real enterprise workflows.

AI Governance Will Become Operational, Not Policy-Based

Most organizations already have AI policies. Very few have operational AI governance.

That distinction will become increasingly important as AI adoption scales and regulatory expectations tighten heading into Q3 2026. Policies can define acceptable usage, but they cannot control prompts, monitor context movement, inspect outputs, or govern runtime behavior across connected AI systems.

That is why enterprise AI security controls are becoming foundational to enterprise AI adoption itself. The organizations best prepared for the next phase of AI governance will not necessarily be the ones with the longest policy documents. They will be the ones with visibility, enforcement, logging, and runtime controls already embedded into their AI infrastructure.

Because AI adoption is already happening. The real question is whether operational governance will arrive before regulators do.

Why OAuth Tokens Are Becoming the New API Keys for Attackers

Suny Choudhary — Thu, 14 May 2026 09:15:19 +0000

OAuth was originally adopted because it solved a practical problem for developers. It reduced password sharing, simplified third-party authentication, and made integrations easier to manage. Over time, it became the default trust layer for modern SaaS applications, cloud platforms, developer tools, and AI systems.

What changed is the role these tokens now play inside infrastructure.

An OAuth token is no longer just an authentication artifact tied to a single application. In modern environments, especially across AI platforms and workflow automation systems, tokens inherit delegated permissions that extend across multiple services simultaneously. A single approved integration can gain access to repositories, internal documentation, messaging systems, cloud storage, customer data, and AI workflows without requiring repeated authentication.

That shift matters because tokens now behave much more like infrastructure credentials than application credentials. They carry trust between systems automatically, often with broad scopes and long-lived access patterns that developers rarely revisit after onboarding.

This is becoming increasingly important in conversations around the OAuth supply chain attack AI platform 2026 landscape. Attackers are starting to recognize that compromising a trusted token can be more valuable than exploiting an application directly. Instead of breaking into systems, they can move through existing trust relationships that organizations already approved themselves.

The result is a security model where access is no longer defined only by who authenticates, but by which systems are connected, what scopes were granted, and how far delegated trust extends once a token enters the ecosystem.

Why OAuth Tokens Are More Valuable Than API Keys Now

For years, API keys were considered one of the most sensitive assets inside modern applications. They granted direct access to services, infrastructure, and developer environments. But in many AI-driven systems today, OAuth tokens have become significantly more valuable from an attacker’s perspective.

The reason is simple: API keys usually grant application-level access. OAuth tokens increasingly grant ecosystem-level access.

Unlike traditional API keys, OAuth tokens often inherit:

Delegated user permissions
Tokens operate with the authority of the user or system that approved them, allowing access to workflows, documents, repositories, and communication systems.
*Context-aware access across platforms *
A single token may connect AI tools to CRMs, cloud storage, Slack workspaces, GitHub repositories, and internal knowledge bases simultaneously.
Dynamic workflow permissions
AI systems use tokens to trigger actions automatically, retrieve context, and interact with external services in real time.
*Long-lived trust relationships *
Refresh tokens and persistent integrations can quietly maintain access long after the original onboarding event is forgotten.

This is what makes an OAuth token compromise so powerful in modern AI ecosystems. Compromising a token no longer means gaining access to one application. It can mean inheriting an entire chain of trusted interactions between connected systems.

AI platforms amplify this further because they aggregate multiple integrations into a single operational layer. An AI assistant connected to several tools effectively becomes a centralized access point into a much larger environment.

How Modern AI Platforms Expand the Attack Surface

Modern AI platforms are built around connectivity. They pull information from multiple systems, trigger workflows automatically, and continuously exchange data between services. That flexibility is what makes them useful, but it also creates a much larger attack surface than most teams initially realize.

In many environments, AI systems now operate as orchestration layers sitting between several connected platforms at once. Every integration introduces another trust relationship, another token, and another path through which information can move.

*In practice, this often looks like: *

Tokens reused across multiple integrations
Excessive OAuth scopes granted during onboarding
Background refresh tokens extending access lifetimes
AI agents triggering downstream API calls automatically
Plugins inheriting permissions from connected systems
Internal context being passed between tools silently

This is why concerns around AI platform supply chain security are growing quickly. AI systems do not operate in isolation. They continuously interact with SaaS platforms, developer tools, cloud services, and internal data sources, often with very little runtime visibility into how those interactions evolve over time.

That is also where solutions like AI security for applications become increasingly relevant. The challenge is no longer just securing the application itself, but understanding how AI systems behave across the broader ecosystem of connected services.

The result is a trust chain that becomes increasingly difficult to visualize. A single compromised token can quietly move through several connected systems without ever looking like a traditional intrusion.

The Problem Isn’t OAuth. It’s Invisible Delegated Trust

OAuth itself is not the problem. In fact, the protocol solves many important security and usability challenges. The real issue is how delegated trust behaves once OAuth tokens begin moving across interconnected AI systems and SaaS workflows.

Most teams understand authentication reasonably well. They know how users log in, how permissions are granted, and how integrations are approved. What becomes much harder to track is how those trust relationships evolve after deployment, especially when AI systems begin interacting with multiple services dynamically.

*In practice, the gaps usually appear in areas like: *

No runtime visibility into how tokens are actually being used
Over-permissioned scopes remaining active long after they are needed
Third-party integrations inheriting broader access than expected
Token revocation and lifecycle management happening inconsistently

The problem becomes even more complex in AI environments because integrations are rarely static. AI assistants, plugins, and orchestration systems continuously exchange context and trigger downstream actions automatically. Over time, small trust relationships accumulate into much larger access chains that few teams fully map or audit.

This is where solutions like AI security services become increasingly important. The challenge is no longer limited to authentication itself. It is understanding and governing how delegated trust behaves across systems after access has already been granted.

Most organizations still evaluate integrations as isolated tools. Attackers increasingly view them as connected trust networks.

OAuth Tokens Are Becoming the New Attack Path of AI Infrastructure

OAuth tokens are starting to function less like temporary authentication mechanisms and more like persistent infrastructure credentials. In AI-driven environments, they carry trust across systems automatically, often with access levels that extend far beyond what teams initially intended. As AI platforms become more interconnected, these tokens increasingly sit at the center of how applications, workflows, and services communicate with one another.

That shift changes how supply chain attacks evolve. Future incidents will likely rely less on exploiting software vulnerabilities directly and more on abusing trusted integrations that already exist inside the environment. A compromised token can quietly inherit permissions, move between connected systems, and access valuable context without triggering the kinds of signals traditional security models were built to detect.

This is why conversations around the OAuth supply chain attack AI platform 2026 landscape are becoming more important. The attack surface is no longer defined only by code. It is defined by delegated trust, connected workflows, and invisible interaction paths between AI systems and SaaS infrastructure.

The most dangerous credential in modern AI environments may no longer be the API key. It may be the OAuth token everyone already approved.

How to Implement AI Governance in LLM Systems Without Slowing Development

Suny Choudhary — Thu, 07 May 2026 10:46:01 +0000

Most teams treat governance as something that slows development down. It shows up as extra reviews, stricter controls, and additional steps before anything can go live. Developers see it as friction. Product teams see it as delay. So governance gets pushed to later stages, often after the system is already built. That is where the real problem begins.

Because governance introduced late is almost always restrictive. It tries to control a system that is already moving fast, already integrated, already in use. At that point, the only way to enforce it is by adding blockers, approvals, and manual checks. Naturally, it feels like it is slowing everything down.

But that is not a problem with governance itself. It is a problem with how it is implemented. In LLM systems, where behavior changes with every prompt and interaction, governance cannot be something you layer on after development. It has to be part of how the system is designed from the start. When done correctly, governance does not slow teams down. It removes uncertainty. It allows developers to move faster because the system itself enforces what is safe and what is not.

The tradeoff between speed and governance is not real. It only exists when governance is treated as an afterthought.

Why Traditional AI Governance Frameworks Break in LLM Systems

Most existing approaches to an AI governance framework were not designed for how LLM systems behave.

They are built around predictable systems, where inputs are structured, outputs are constrained, and behavior can be validated at specific checkpoints. Governance, in that model, happens through policy documents, manual reviews, and compliance processes that sit around the system rather than inside it.

LLM systems do not operate that way. Every interaction is dynamic. Prompts change based on user intent. Context is pulled from multiple sources. Outputs are generated in ways that cannot always be anticipated in advance. This makes it difficult to rely on static rules or one-time validations. The result is a growing gap between governance and execution.

Policies may define what should happen, but they do not control what actually happens at runtime. A model can process sensitive data, generate unintended outputs, or trigger downstream actions without violating any predefined rule in a way that gets detected.

This is where governance begins to fail. From a leadership perspective, especially for roles focused on AI security governance at the CISO level, this creates a difficult situation. There is an expectation of control, but no direct visibility into how AI systems are behaving in real time.

What a Dev-Friendly LLM Governance Policy Actually Looks Like

A practical LLM governance policy cannot feel like an external approval system. If it interrupts workflows or adds manual steps, developers will either bypass it or delay it. For governance to work in LLM systems, it has to be embedded into how the system already operates.

That means shifting from rigid controls to adaptive, low-friction mechanisms that run alongside development rather than against it.

In practice, a dev-friendly governance policy looks like this:

Prompt-level checks that evaluate inputs before they reach the model, without requiring manual review
Output validation that ensures responses are safe before they are returned or reused
Context-aware enforcement that adapts based on data sensitivity, user role, and use case
Automated policy application so developers define rules once and the system enforces them continuously
Minimal friction within workflows, allowing developers to build without waiting on approvals

The goal is not to restrict how developers use LLMs. It is to make safe usage the default behavior of the system.

When governance operates this way, it becomes almost invisible. Developers do not have to think about enforcement because it is already happening in the background. That is what makes it effective.

How to Implement Governance Without Slowing Down Development

Implementing governance in LLM systems does not require adding more checkpoints. It requires choosing the right layer to enforce control.

Most teams try to implement governance at the edges, either before deployment through reviews or after deployment through monitoring. Both approaches introduce delay and still miss what happens during actual usage. The more effective approach is to operate at the interaction layer, where prompts, context, and outputs are continuously flowing.

This is where governance becomes part of execution instead of a separate process. Rather than relying on manual reviews, teams can introduce real-time inspection of prompts and responses. Policies are defined once and then enforced automatically every time the system is used. This removes the need for constant oversight while still maintaining control over how data is handled and how outputs are generated.

Integrating governance into existing workflows is also critical. It should fit naturally into development pipelines, APIs, and application layers without requiring teams to change how they build. When governance is embedded this way, it does not interrupt velocity. It supports it.

This is the shift that approaches like AI security for AI applications enable. They focus on enforcing governance at runtime, where decisions are actually made, rather than relying on assumptions defined earlier in the process.

What Changes When Governance Is Done Right

When an AI governance framework is implemented at the right layer, the impact is immediate. Governance stops feeling like a constraint and starts functioning as an enabler for both development and security.

The difference shows up in how teams build, deploy, and operate LLM systems:

Development cycles move faster because safety checks are automated, not manual
Risk is reduced without slowing down experimentation or iteration
Developers gain confidence in using real data within controlled boundaries
Security teams get visibility into how AI is actually being used
Audit readiness improves with clear logs of prompts, decisions, and outputs

This is also where leadership priorities align more clearly. For roles focused on AI security governance at the CISO level, governance is no longer abstract. It becomes measurable, enforceable, and visible across systems.

Capabilities like AI security services support this shift by enabling continuous enforcement and visibility, rather than relying on periodic checks or assumptions.

The outcome is not just better governance. It is a system where development speed and control exist together, without tradeoffs.

Also Read: What Does AI Governance Actually Require in 2026?

Governance Should Accelerate, Not Restrict

AI governance is often positioned as a control mechanism, something that limits how systems are built and used. But in LLM environments, that framing does not hold for long. When governance is added late or enforced manually, it creates friction. It slows teams down, introduces delays, and often leads to workarounds. That is why many teams hesitate to implement it early.

But when governance is designed as part of the system, the outcome changes. It removes uncertainty instead of adding constraints. Developers can move faster because they do not have to constantly question whether something is safe or compliant. Security teams gain visibility without interrupting workflows. Governance becomes something that supports execution, not something that blocks it.

Why Developers Trust AI Code More Than They Should

Suny Choudhary — Tue, 05 May 2026 09:34:06 +0000

Most developers don’t trust AI.

Until it writes code that works.

Then suddenly… they do.

The Shift That’s Happening Quietly

You paste a prompt.
It generates a function.
You test it.
It works.

You move on.

No deep review. No second guessing.

Because it looks right.

That’s the moment trust creeps in.

The Problem Isn’t AI Code

AI-generated code isn’t the real issue.
The issue is how quickly we stop questioning it.

We assume:

the logic is correct
the inputs are handled safely
the dependencies are fine
the security is “good enough” But AI doesn’t know your system.

It doesn’t know:

your access controls
your data sensitivity
your internal architecture
your compliance requirements It predicts patterns.

That’s it.

Why This Is Getting Risky

Modern AI security research is already pointing this out.

The OWASP Foundation highlights risks like insecure outputs, prompt injection, and unsafe integrations in its LLM security guidance.

And it’s not just theory.

The GitGuardian reports that millions of secrets are still leaking through codebases, with AI-assisted development accelerating the problem.

So this isn’t about “AI might be risky.”

It already is.

Where Developers Get It Wrong

Most AI-generated code failures don’t come from obvious bugs.

They come from things like:

missing input validation
over-permissive access
unsafe API usage
weak error handling
hidden dependency risks
logging sensitive data Nothing breaks immediately.

Which is exactly why it slips through.

The Real Issue: Trust Without Verification

Here’s the pattern:
AI explains the code → it feels correct
Code runs → it feels safe
Tests pass → it feels done

But none of that guarantees security.

That’s the gap.

This Is Bigger Than Just Code

Attackers are already shifting toward exploiting system complexity instead of single vulnerabilities.

The CrowdStrike 2025 Threat Hunting Report shows how modern attacks move across systems, APIs, identities, and cloud layers instead of targeting one weak point .

That’s exactly what AI-generated code creates:

More connections
More paths
More surface area

What You Should Actually Do

Not “stop using AI.”

That’s unrealistic.

Instead:

Treat AI-generated code as untrusted
Review logic, not just syntax
Validate inputs explicitly
Check dependencies
Watch how outputs are used
Understand what the code actually touches If you didn’t write it, you still own it.

The Bigger Pattern

Developers don’t blindly trust AI.

They trust working results.

AI just happens to produce those faster.

That’s why this is dangerous.

Because it doesn’t feel risky.

If You Want a Deeper Breakdown

We went deeper into how this expands attack surface and why it’s becoming a real security problem:
👉 AI-generated code is expanding your attack surface

And if you want the legal + product risk angle (especially in legal tech):
👉 Vibe coding security risks explained

Question for Devs Here

Be honest:

Do you fully review AI-generated code before shipping it?

Or do you trust it once it works?

Sources

OWASP Top 10 for LLM Applications
GitGuardian State of Secrets Sprawl Report
CrowdStrike 2025 Threat Hunting Report

Your “AI-Powered” Fintech App Might Not Survive an Audit

Suny Choudhary — Fri, 01 May 2026 07:24:24 +0000

**Most fintech apps say they use AI.

Few can prove it.

And that gap is starting to get companies fined.**

Everyone says their product uses AI.

AI-powered fraud detection
AI-driven underwriting
AI-based trading signals

Sounds familiar.

But here’s the problem:

If your system can’t prove those claims, you don’t just have a marketing issue.

You have a system design flaw.

This Isn’t About “Fake AI”

AI washing is rarely fake AI.

It’s overstated AI.

You say:

“Our AI detects fraud in real time”

Reality:

model runs on batch data
rules engine handles most decisions
humans review high-risk cases

AI exists.

But your claim describes something else.

That mismatch is the risk.

What an Audit Actually Looks Like

Regulators don’t ask:

“Do you use AI?”

They ask:

Which model is used?
Which version was active?
What data was processed?
Where are the logs?
Can you reproduce the output?

If you can’t answer this cleanly, your claim falls apart.

The Real Problem: No Evidence Layer

Most systems today lack:

model-to-feature mapping
prompt + output logging
decision traceability
visibility into fallback logic

So when someone asks:

“Show me how your AI made this decision”

You don’t have a clean answer.

Why This Is Getting Risky Now

The SEC has already penalized firms for misleading AI claims.

They called it AI washing.

Source:
https://www.sec.gov/newsroom/press-releases/2024-36

This isn’t theoretical anymore.

Where Developers Get Caught Off Guard

Your architecture probably looks like:

User → API → Model → Output

But reality is:

User → API → Rules → Model → Human Review → Output

And your marketing only mentions the model.

That’s the gap.

What You Should Fix (Practical)

1. Map every AI claim to a real system

If it doesn’t map, remove it.

2. Add observability

Log:

inputs
outputs
decision paths

Not for debugging. For proof.

3. Track model versions

Know exactly:

what changed
when it changed
how behavior changed

4. Be honest about human involvement

If humans are in the loop, say it.

5. Test your own claims

Ask:

“Can we prove this today?”

If not, fix it.

The Bigger Insight

AI washing is not a marketing problem.

It’s a visibility problem.

A system problem.

A traceability problem.

Final Thought

Most teams focus on building AI.

Very few focus on defending AI claims.

In fintech, that’s the difference between scaling and getting flagged.

Full Breakdown

https://www.langprotect.com/blog/ai-washing-sec-fintech-enforcement-risk?utm_source=Sahil&utm_medium=Medium&utm_campaign=Information