Trying LLM-Based Threat Modeling

Introduction

Hi, I’m JJ (yuasa), a security engineer.

In this post, I’ll try out Threat Thinker, an automated threat modeling tool that uses LLMs, on several different systems. From an AWS-based system to a smart home, we’ll see how an LLM surfaces threats from architecture diagrams. My goal is to give you a feel for what LLM-based threat modeling looks like, with real outputs included.

https://github.com/melonattacker/threat-thinker

What is Threat Thinker?

Threat Thinker is a tool that performs automatic threat modeling from system architecture diagrams using an LLM. It can parse various diagram formats such as Mermaid, draw.io, screenshots of architecture diagrams, and OWASP Threat Dragon. From the relationships between components, it infers potential threats.

In traditional threat modeling, once you have an architecture diagram, developers and security engineers have to manually go through and identify threats one by one. In my experience, there are broadly two types of threats:

• “Basic threats” that can appear in almost any system
• “System-specific threats” that you only notice if you deeply understand the specs and implementation details

By using Threat Thinker, you can automate the initial identification of those “basic threats.” Humans can then focus on deeper analysis of “system-specific threats” and on designing countermeasures. It supports both a CLI and a web UI, so even non-security specialists can use it without much friction.

Automatically identifying basic threats from an architecture diagram

Trying Out Threat Thinker

Let’s use Threat Thinker to identify threats for three cases: an AWS-based system, a corporate network, and a smart home. For each one, we’ll feed the architecture diagram into Threat Thinker and see which threats it extracts.

AWS-Based System

System Architecture

Here is the architecture diagram of the target AWS-based system, created in Mermaid. It’s a fairly common web application architecture consisting of CloudFront → ALB → ECS → RDS/S3. We’ll use this to see how Threat Thinker extracts threats from a relatively simple stack.

graph LR
  %% Trust boundaries
  subgraph Internet
    user[User]
  end

  subgraph AWS_Edge[Edge]
    cf[CloudFront]
  end

  subgraph VPC[VPC]
    subgraph PublicSubnet[Public subnet]
      alb[ALB]
    end
    subgraph PrivateSubnet[Private subnet]
      ecs[ECS Service]
      rds[(Customer RDS<br>PII)]
      s3[(S3 Bucket<br>Logs/Uploads)]
    end
  end

  %% Data flows
  user -- sends HTTPS request --> cf
  cf -- forwards HTTPS request --> alb
  alb -- routes HTTP request --> ecs
  ecs -- reads/writes data (SQL/TLS) --> rds
  ecs -- stores/reads objects (S3 API) --> s3

Running Threat Thinker

Below is a CLI-based execution example. You specify the Mermaid diagram file path via --diagram. With --infer-hints enabled, the LLM will also infer auxiliary information not explicitly written in the diagram and use that to reason about threats. In this example, we use OpenAI’s gpt-4.1 model and ask it to output up to 5 threats.

threat-thinker think \
    --diagram path/to/diagram/system.mmd \
    --infer-hints \
    --topn 5 \
    --llm-api openai \
    --llm-model gpt-4.1 \
    --out-dir path/to/report/dir

Extracted Threats

The top 5 threats in the generated Markdown report are as follows. These are all typical risks you might see when building a web application on AWS: issues around authentication/authorization, lack of encryption, insufficient logging and monitoring, and S3 misconfiguration.

ID	Threat	Severity	STRIDE	Affected Components	Score
T001	Potential Lack of Authentication/Authorization on ALB to ECS Path	High	Spoofing / EoP	ALB → ECS	8.0
T002	Unencrypted Traffic Between ALB and ECS Allows Tampering and Disclosure	High	Tampering / Info Disc.	ALB → ECS	8.0
T003	Exposure of PII in RDS Without Explicit Encryption at Rest	High	Info Disclosure	ECS ↔ RDS	7.0
T004	Insufficient Logging and Monitoring for Sensitive Operations	Medium	Repudiation	ECS / RDS / S3	6.0
T005	Potential S3 Bucket Misconfiguration Exposing Internal Data	Medium	Info Disclosure	S3	6.0

In the HTML report, you can visually see which part of the architecture each threat maps to.

Extracted architecture graph

Visualizing where threats exist in the architecture

Corporate Network

System Architecture

Here is the architecture diagram for the target corporate network, created in draw.io. It’s a simple small/medium business network split into three zones: Internet, DMZ, and internal network.

Corporate network architecture diagram

Running Threat Thinker

This time, we’ll use the web UI. You can start the web UI with threat-thinker webui.

$ threat-thinker webui
ℹ️ Starting Threat Thinker Web UI
* Running on local URL:  http://127.0.0.1:7860
* To create a public link, set `share=True` in `launch()`.

Copy and paste the draw.io (XML) diagram source, and set Diagram Format to drawio.

Configure the options as needed and click Generate Report.

Extracted Threats

The top 5 threats in the Markdown report are as follows. They include risks such as attacks on the public-facing web server, injection vulnerabilities due to insufficient input validation, lateral movement from the DMZ to the internal network, and exposure of sensitive data stored on internal servers. Overall, it points out a balanced set of risks that you would typically expect in a corporate network.

ID	Threat	Severity	STRIDE	Affected Components	Score
T001	External Attackers Can Reach Public-Facing Web Server	High	Spoofing / Tampering	Internet → Web Server	9.0
T002	Insufficient Input Validation on Public Web Server	High	Tampering / Info Disc.	Web Server	8.0
T003	Potential Lateral Movement from DMZ to Internal Network	High	EoP / Info Disc.	Web Server → Internal Net	8.0
T004	Sensitive Data Exposure on File and Directory Servers	High	Info Disclosure	File Server / AD	8.0
T005	VPN Gateway Exposed to Credential Attacks	High	Spoofing / EoP	Internet → VPN GW	8.0

Smart Home

System Architecture

Here is the architecture diagram for the target smart home system, created with the threat modeling tool OWASP Threat Dragon. It represents a typical cloud-connected smart home environment: residents use a mobile app to access a cloud control service, which in turn controls home devices such as IP cameras, smart locks, and smart speakers via the home router.

Smart home architecture diagram

Running Threat Thinker

We’ll use the web UI again for this example.

Threat Thinker has a RAG feature that lets you upload Markdown, HTML, and other documents to build a Knowledge Base that the LLM can reference during threat reasoning. Since our target is a smart home system, we’ll build a Knowledge Base based on the OWASP IoT Top 10.

Building a Knowledge Base based on OWASP IoT Top 10

Then we configure the threat inference settings so that the model uses that Knowledge Base.

Using the Knowledge Base during threat inference

Extracted Threats

The top 5 threats in the Markdown report are as follows. The model points out issues such as potentially unencrypted communication between the mobile app and the cloud service, lack of assurance that commands sent from the cloud to home devices are authentic, and insufficient protection for video/logs stored in the cloud. It also notes that if user authentication is weak, third parties could control devices.

Overall, the results feel reasonable for a threat analysis that references OWASP IoT Top 10.

ID	Threat	Severity	STRIDE	Affected Components	Score
T001	Insecure Communication Between Mobile App and Cloud Control Service	High	Tampering / Info Disc.	Mobile App ↔ Cloud	9.0
T002	Lack of Authentication for Device Commands from Cloud to Home Network	High	Spoofing / EoP	Cloud → Home Devices	9.0
T003	Insecure Storage of Sensitive Video/Logs in Cloud	High	Info Disc. / Repudiation	Cloud Storage	8.0
T004	Unencrypted Video/Telemetry Data in Transit	High	Info Disc. / Tampering	Devices ↔ Cloud	8.0
T005	Weak or Missing Authentication for Mobile App User Actions	High	Spoofing / EoP	User ↔ Mobile App	8.0

When you use Threat Dragon as the input format, Threat Thinker can output the results back in Threat Dragon format with the inferred threats added to the diagram.

Threats added and linked to the relevant elements

Conclusion

In this post, we used Threat Thinker to perform LLM-based threat modeling on three different systems: an AWS-based system, a corporate network, and a smart home. We saw that just by feeding in an architecture diagram, Threat Thinker can automatically identify a solid set of basic threats.

At the same time, risks tied to business logic or to organization-specific operations still need human review. LLM-based threat modeling is not a replacement for expert review. Rather, it’s best used as a way to quickly generate an initial draft or as a safety net to prevent oversights. By importing guidelines like the OWASP IoT Top 10 into the Knowledge Base, you can also steer it toward more domain-specific reasoning.

If this sounds interesting, I encourage you to try running Threat Thinker against your own architecture diagrams and see what threats it surfaces.