Project Glasswing: What AI Doesn't Tell You When It Writes Your Code

A local corner store won't let you into the back room. Doesn't matter how much you trust the owner — some things aren't for everyone. The stock, the suppliers, the real prices. There's a line between what's on display and what actually keeps the business running.

Software supply chain is exactly that back room. And for years we've treated it like the storefront. Open, well-lit, welcoming. Trusting that vendors are who they say they are.

Then AI came along and we expanded the back room without adding any cameras.

Project Glasswing software supply chain security AI: what we're actually talking about

Glasswing is a research initiative focused on the problem the industry is choosing not to look at directly: when you use AI to generate code, review dependencies, suggest architectures — who audits the auditor?

The premise is simple, and it landed right where it hurts. We have CI/CD pipelines running automated checks. We have Dependabot, Snyk, Trivy. We have SBOMs. But now we also have:

• Code generated by LLMs that suggest libraries with plausible names that don't actually exist (hallucinated packages)
• AI agents with access to our repos that can execute commands
• Models fine-tuned on code of questionable origin
• Business context we're passing to external APIs without thinking too hard about it

Every single one of those is a door to the back room. And most of us haven't put a lock on any of them yet.

I was already thinking about this when I wrote about Scion, Google's framework for orchestrating agents. I framed it as an interesting tool back then. Today I look at it differently: what's the permission model when one agent can call another? Who audits the actions of the full chain?

The real problem: what changed with AI-assisted coding

I'm going to be honest about my own practice before I sound like a security evangelist.

I use AI on real projects. Every day. I broke it down in some detail in vibe-coding vs stress-coding: there are moments where AI accelerates everything and moments where it walks you straight into a dead end. But what I hadn't properly mapped until now was the specific attack vector.

These are the three that Glasswing puts front and center:

1. Package hallucination

LLMs make up package names. Not always, but they do it. And when an attacker notices that GPT-4 consistently suggests react-auth-utils for a specific use case — a package that doesn't exist — they can register that name on npm and wait.

It's called dependency confusion with a new twist: instead of exploiting private packages, they exploit the model's hallucinations.

# Before installing ANYTHING an AI suggested, verify it:
npm view package-name --json | grep -E '"name"|"version"|"author"|"downloads"'

# If the package is less than a month old with zero downloads, ask yourself why
# If it doesn't exist, the command will throw an error — that's already useful information

2. Context leakage

When you pass context to an LLM to generate code — database schema, sample environment variables, system architecture — that context leaves your perimeter. With commercial APIs, data retention policies vary. With models fine-tuned on third-party code, the problem is different but just as real.

This isn't paranoia. It's attack surface.

3. AI-generated code with vulnerabilities traditional scanners won't catch

This is the one that worries me most. SAST scanners look for known patterns. AI-generated code can have semantically correct vulnerabilities — code that compiles, passes tests, does what it's asked — but with broken authorization logic or race conditions that no regex is ever going to catch.

// Code an LLM might generate that looks correct
async function getDocument(userId: string, docId: string) {
  const doc = await db.documents.findOne({ id: docId });
  // The LLM forgot to verify that userId === doc.ownerId
  // The scanner won't catch it because there's no known vulnerability pattern
  // It's logically wrong, not syntactically wrong
  return doc;
}

// What it should look like:
async function getDocument(userId: string, docId: string) {
  const doc = await db.documents.findOne({ 
    id: docId,
    ownerId: userId // Always filter by ownership in the query, not after
  });

  if (!doc) {
    throw new Error('Document not found or access denied');
  }

  return doc;
}

What I actually changed in my pipeline

After reading the Glasswing research and sitting with it for a few days, I made concrete changes. Not dramatic ones, but concrete.

1. Lock files as first-class citizens

I always used lock files, but now I actively review them in code review. A PR that touches pnpm-lock.yaml in more places than expected makes me stop. When we migrated from npm to pnpm last year — that migration that took installs from 14 minutes down to 90 seconds — I realized how many transitive dependencies exist without anyone having consciously chosen them.

# Compare the lockfile before and after AI suggests changes
git diff pnpm-lock.yaml | grep '^+' | grep 'resolution' | wc -l
# If the number is way higher than the packages you explicitly added, dig in

2. SBOM generated on every build

# In my GitHub Actions workflow
- name: Generate SBOM
  uses: anchore/sbom-action@v0
  with:
    artifact-name: sbom.spdx.json
    format: spdx-json

- name: Scan SBOM against known vulnerabilities
  uses: anchore/scan-action@v3
  with:
    sbom: sbom.spdx.json
    fail-build: true
    severity-cutoff: high

3. Manual review of any AI-suggested package I don't recognize

Sounds obvious. It wasn't in practice. When Copilot or Claude suggest an import, I'd developed the habit of just writing it out without thinking. Now I have a personal rule: if I don't recognize the package from memory, I open npm before I install anything.

This connects to something I touched on when I built the SSL certificate extension: implicit trust is the enemy. A certificate can look valid and not be. A package can look legitimate and not be.

4. Minimum necessary context to external APIs

I've started being more intentional about what I pass to an LLM. Full production schema: no. Anonymized or example schema: yes. Real environment variables: never. Full folder structure with internal service names: also no.

It's the principle of least privilege, but for the context you share.

The gotchas nobody talks about

The false positive of "I already have Dependabot"

Dependabot is necessary. It's not sufficient. Dependabot knows about known, published vulnerabilities in databases. It doesn't know about freshly created packages mimicking names that LLMs consistently hallucinate. It doesn't know about logical vulnerabilities in generated code. You need those layers, but you can't stop there.

The team context problem

When you work solo, you can control what you pass to AI. When you work on a team, someone else might be pasting the prod schema into a Claude chat without you knowing. That requires policy, not just individual practice.

Confusing security tools with security posture

I have the HAProxy extension I documented here and the certificate extension. I'm careful with infrastructure. But careful with infrastructure ≠ careful with supply chain. They're different layers.

The score that gives you false confidence

Just like Lighthouse's accessibility score can lie to you because it measures what it can measure mechanically, your security tooling's score measures what's been catalogued. The new attack surface that AI introduces doesn't have mature metrics yet.

FAQ: Project Glasswing and AI supply chain security

What exactly is Project Glasswing?

It's a security research initiative focused on the new attack vectors that AI introduces into the software development lifecycle. The name references the Glasswing butterfly — transparent, delicate, more resilient than it looks. The focus is on how language models affect the integrity of the software supply chain: from package hallucination to semantic vulnerabilities in automatically generated code.

Is package hallucination a real risk or a theoretical one?

It's real and there are already documented cases. Security researchers have shown it's possible to register packages with names that popular LLMs consistently suggest for common use cases. Hallucination rates vary by model and context, but none of them are at zero. The risk scales with model popularity: the more people using the same LLM, the more predictable the names it'll invent.

Is it enough to have Snyk or Dependabot in the pipeline?

No. Those tools are essential but they cover known, catalogued vulnerabilities. The AI-assisted coding vector introduces two problems that escape that model: malicious packages not yet in any vulnerability database, and logical vulnerabilities in generated code that have no recognizable pattern for static analysis. You need those layers, but you can't stop there.

How do I know if AI-generated code has logical vulnerabilities?

That's the hard question. Traditional SAST tools aren't optimized for this. What works today: human review with specific focus on authorization logic and sensitive data handling, functional security tests (not just unit tests), and — paradoxically — using another AI to review the code generated by the first one, with a prompt specifically aimed at finding security problems. It's not perfect, but it adds a layer.

What information should I never pass to an external LLM?

Real credentials, tokens, API keys — obvious. But also: production database schemas with real table and column names, internal service names and infrastructure architecture, user data even if partially anonymized, and anything that under your threat model would be valuable to an attacker who knew your system's internal structure. The practical rule: if you wouldn't publish it in a public README, don't paste it into an AI chat.

Do locally-run models (Ollama, LM Studio) have the same risks?

The context leakage risk to external APIs disappears. The package hallucination and logical vulnerability risks persist — they're properties of the model, not the deployment. If you use local models, you gain control over where your context goes, but you still need to review suggestions with the same scrutiny. It's not security through obscurity, it's reducing the attack surface you actually control.

What won't change: the responsibility is still ours

I've been in tech for 30 years. A lot of those in infrastructure — servers, networks, the healthy paranoia of someone who took down a production server with rm -rf in their first week and learned the hard way that implicit trust has real costs.

What hits me about Glasswing isn't the alarmism. It's the reminder of something that should be obvious but that the speed of the AI ecosystem makes us forget: every new tool that accelerates development also expands the surface we have to defend.

I'm not saying stop using AI to code. I'm not stopping. I'm saying that the same energy you put into learning to prompt well needs to go into understanding where the new doors to the back room are.

The lock file is yours. The generated code is yours. The responsibility is yours.

If this made you think of something you've been meaning to review in your pipeline, start today. It doesn't have to be all at once. An SBOM in CI, a context policy for your team, the habit of verifying packages before installing them. One door at a time.