I remember the first time I saw a security vendor demo an "AI-powered SOC copilot."
The analyst typed a plain English question. The tool spat out a clean investigation summary in under three seconds. The room was genuinely impressed.
Someone in the back muttered: "This changes everything."
They were right. Just not in the way they meant.
Because that same week, a threat research team published findings showing attackers were using the exact same foundation models to draft phishing campaigns. Not some secret military AI. Not a custom dark-web tool. The same commercially available LLMs. The same cloud APIs.
That's the reality nobody prepared us for.
The "Defenders Finally Win" Narrative Was Always Wrong
When AI got woven into security tools, the marketing story wrote itself: defenders now had an asymmetric advantage. Attackers have to succeed every time; defenders only need to catch them once. AI tips that scale further in defenders' favor.
Reasonable logic. Wrong conclusion.
The actual advantage AI creates isn't about which side has it. It's about who converts an idea into action faster. And both sides figured that out at roughly the same time.
What we're watching now isn't defenders getting ahead. It's the baseline level of capability rising for everyone — and that includes people trying to break into your systems.
What Attackers Are Actually Using AI For
Let me be specific here, because the vague "AI helps hackers" framing doesn't help anyone.
Reconnaissance got dramatically faster. The early stages of an attack used to be tedious. Searching LinkedIn for org charts. Combing GitHub for exposed secrets. Reading through job postings to infer the tech stack. An attacker could spend days on this before they even had a target list.
With an LLM, you can synthesize publicly available information about a company in minutes. Not because the AI is "hacking" anything — it's not. It's just connecting dots across sources faster than a human can.
Phishing quality jumped. The classic tell used to be bad grammar. Weird phrasing. That slightly-off tone that made you double-check the sender. LLMs wiped that out. Now a non-native English speaker can write a convincing HR policy update that reads like it came from your actual HR team. And the same model can translate it into twelve languages on request.
Social engineering got personal at scale. This one bothers me more than the others. Old phishing was spray-and-pray — send the same email to fifty thousand people and hope a few click. Modern campaigns generate individualized messages based on the target's role, recent company news, their LinkedIn activity, even their writing style scraped from public forums. The attack feels personal. It is personal, just automated.
Development cycles shortened. I want to be honest here: LLMs aren't churning out sophisticated malware on their own. Experienced malware developers aren't being replaced. But they are being made more productive. Debugging, API documentation, refactoring, porting code across platforms — the grunt work that used to eat hours now takes minutes. That compounds.
What Defenders Are Doing With the Same Technology
Security operations was already a capacity problem before AI. Most enterprise environments generate more events per day than any team could meaningfully review. Alert fatigue wasn't a morale problem — it was a math problem.
AI has genuinely helped here. Not because it's magic, but because it's good at specific things defenders needed:
Summarizing alerts into readable narratives. Correlating events across logs that don't naturally talk to each other. Mapping suspicious behavior to MITRE ATT&CK techniques without analysts having to memorize the entire framework. Flagging the same pattern across thousands of endpoints simultaneously.
Threat hunting in particular has changed. Analysts used to write complex query syntax to search through telemetry. Now they can ask plain questions — "show me devices that talked to newly registered domains this week" — and get usable results without being fluent in whatever query language your SIEM happens to use.
The result isn't that humans do less work. It's that humans do different work. Less log-reading. More actual investigation.
The Uncomfortable Part
Here's what I keep coming back to: both sides are sometimes using the exact same model, from the same vendor, running on the same infrastructure.
One organization asks Claude or GPT-4 to explain what suspicious PowerShell activity looks like. Another asks it to explain how PowerShell can be used to move laterally.
Same model. Same training data. Different intent.
The technology is genuinely neutral in a way that feels philosophically strange if you sit with it long enough. We've built intelligence — or something that looks a lot like intelligence — that serves whoever is holding the prompt.
That's not an argument against using AI in security. It's an argument against treating it as an inherently defensive tool.
Where I Think This Actually Goes
Most "AI arms race in cybersecurity" takes you'll read focus on the technology itself. Who has the smarter model. Who can automate more of the kill chain.
I think that's the wrong frame.
The real competition is about three things that have nothing to do with which model you're running:
Data quality. An AI that knows your environment — your asset inventory, your normal traffic patterns, your user behavior baselines, your historical incidents — is dramatically more useful than a generic one. The same applies on the offensive side; attackers with more context about a target make better decisions. Whoever has better data wins, regardless of the model.
Feedback loops. Security is iterative. Attackers that can learn quickly from failed attempts improve faster. Defenders that can run retrospectives and update detections improve faster. AI can shorten both loops. The side that uses it to actually learn — not just to move faster — has the advantage.
Human judgment. I know this sounds like the obligatory "don't worry, humans are still important" section. It's not that. It's that the decisions AI genuinely can't make well are often the most consequential ones. What's the business risk of this incident? How do we communicate this to the board? Do we take down this system now, during a critical business process, or wait two hours? Those calls require context that lives in people, not models.
AI will keep getting better at the rest. The human part isn't going away.
The Mistake That Will Cost Organizations the Most
Buying an AI-powered security platform and calling it a strategy.
AI doesn't fix weak authentication. It doesn't patch unpatched systems. It doesn't clean up overprivileged accounts or secure misconfigured storage buckets. What it does is help you find those problems faster — and potentially help attackers exploit them faster too.
The fundamentals still matter more than the tooling. They always have.
The best-resourced SOC in the world, running the most sophisticated AI, still loses if the basics aren't covered. And the basics are still mostly unsexy: MFA enforcement, timely patching, least-privilege access, secrets management, logging that actually captures what you need.
Get those right first. Then talk about AI.
Where This Leaves Us
Every major shift in how computing works has changed the threat landscape. The internet created attack surface. Cloud changed how infrastructure was compromised. Mobile changed identity. Each time, both attackers and defenders adapted.
AI is that shift for this decade.
What's different this time is how fast the adaptation is happening on both sides simultaneously, and how low the barrier is to entry. A junior attacker with good prompt skills can now do things that used to require years of experience. That's genuinely new.
But defenders have access to the same leverage.
The teams that will come out ahead aren't the ones with the biggest AI budget. They're the ones that build good processes around these tools, invest in the quality of their data, and keep humans meaningfully in the loop on decisions that actually matter.
Security has always been a human problem with a technology dimension. AI made the technology dimension more interesting. It didn't change which part is harder.
Curious where you stand: do you think AI ultimately favors attackers or defenders over the next five years — and what's your reasoning?
Top comments (1)
what's your thought on this ?🤔