How to Build an AI Model That Classifies Phishing Emails

Phishing emails account for over 90% of confirmed breaches according to the Verizon DBIR. The core problem with rule-based and blacklist-based filters is that they are reactive — you have to have seen an attack before you can block it. An ML-based classifier trained on email content generalizes to new campaigns based on linguistic and structural patterns, not signatures.

This post walks through building a working phishing email classifier in Python: feature engineering, a TF-IDF baseline you can ship in an afternoon, and a fine-tuned language model for when precision really matters. All code is runnable.

What Makes an Email "Phishy"

Before writing a line of code, understand your feature space. Phishing emails cluster around a handful of detectable patterns:

• Urgency and fear language: "Your account will be suspended in 24 hours", "Immediate action required"
• Impersonation signals: sender display name doesn't match the actual domain, lookalike domains (paypa1.com, amazon-support.com)
• Credential harvesting intent: requests for passwords, social security numbers, card details
• Suspicious URLs: IP-based links, URL shorteners, domains with suspicious TLDs (.tk, .ml, .ga)
• Generic salutations: "Dear Customer", "Dear User" rather than a real name

A model trained on enough labeled examples will learn these patterns without you explicitly coding each one — but knowing the signal space helps you evaluate whether the model has learned the right things.

Dataset Selection

For this project, combine publicly available datasets:

• CEAS 2008 phishing corpus (~5k phishing emails)
• SpamAssassin public corpus (~6k emails — keep the legitimate subset)
• Enron email dataset (~30k legitimate corporate emails)

The class imbalance matters. Real inboxes have roughly 0.5–2% phishing, so train with class_weight="balanced" and measure recall on the phishing class specifically. A model that achieves 99% accuracy by predicting everything as legitimate is worthless in practice.

Baseline: TF-IDF + Logistic Regression

A TF-IDF pipeline is fast to train, interpretable, and often good enough for internal tooling. Start here before reaching for transformers.

import re
import email
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report


def extract_text(raw_email: str) -> str:
    # Extract subject and body text from a raw email string.
    try:
        msg = email.message_from_string(raw_email)
        subject = msg.get('Subject', '') or ''
        body = ''
        if msg.is_multipart():
            for part in msg.walk():
                if part.get_content_type() == 'text/plain':
                    payload = part.get_payload(decode=True)
                    body += payload.decode('utf-8', errors='ignore') if payload else ''
        else:
            payload = msg.get_payload(decode=True)
            body = payload.decode('utf-8', errors='ignore') if payload else ''
        return f'{subject} {body}'
    except Exception:
        return raw_email


def url_heuristics(text: str) -> str:
    # Append URL-derived pseudo-tokens that signal phishing patterns.
    import re
    urls = re.findall(r'https?://[^\s<>"\']]+', text)
    domains = [re.sub(r'^https?://', '', u).split('/')[0] for u in urls]
    tokens = []
    if any(re.match(r'\d+\.\d+\.\d+\.\d+', d) for d in domains):
        tokens.append('FEAT_IP_URL')
    if any(d.endswith(('.tk', '.ml', '.ga', '.cf')) for d in domains):
        tokens.append('FEAT_SUSPICIOUS_TLD')
    if any(d in {'bit.ly', 't.co', 'goo.gl', 'ow.ly', 'tinyurl.com'} for d in domains):
        tokens.append('FEAT_URL_SHORTENER')
    return text + ' ' + ' '.join(tokens)


# emails.csv has columns: raw (str), label (0=legit, 1=phishing)
df = pd.read_csv('emails.csv')
df['text'] = df['raw'].apply(extract_text).apply(url_heuristics)

X_train, X_test, y_train, y_test = train_test_split(
    df['text'], df['label'], test_size=0.2, stratify=df['label'], random_state=42
)

pipeline = Pipeline([
    ('tfidf', TfidfVectorizer(
        sublinear_tf=True,
        max_features=50_000,
        ngram_range=(1, 2),
        min_df=2,
    )),
    ('clf', LogisticRegression(C=1.0, max_iter=1000, class_weight='balanced')),
])

pipeline.fit(X_train, y_train)
print(classification_report(y_test, pipeline.predict(X_test), target_names=['legitimate', 'phishing']))

On a balanced 10k dataset this typically yields F1 ~0.92–0.94 on the phishing class. The model is interpretable: inspect the top coefficients with pipeline.named_steps['tfidf'].get_feature_names_out() combined with pipeline.named_steps['clf'].coef_ to understand exactly what it learned.

Fine-Tuning a Language Model for Production

When false negatives carry real cost — a missed phishing email that leads to a breach — a fine-tuned small language model consistently outperforms TF-IDF by 3–5% F1. DistilBERT (66M parameters) runs comfortably on a CPU inference endpoint and trains in under 10 minutes on a GPU.

import numpy as np
from datasets import Dataset
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    TrainingArguments,
    Trainer,
)
from sklearn.metrics import f1_score, accuracy_score

MODEL_NAME = 'distilbert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)


def tokenize_batch(batch):
    return tokenizer(batch['text'], truncation=True, max_length=512, padding='max_length')


train_ds = Dataset.from_dict({'text': X_train.tolist(), 'label': y_train.tolist()})
test_ds  = Dataset.from_dict({'text': X_test.tolist(),  'label': y_test.tolist()})

train_ds = train_ds.map(tokenize_batch, batched=True)
test_ds  = test_ds.map(tokenize_batch,  batched=True)

model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME, num_labels=2)


def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    return {
        'accuracy': accuracy_score(labels, preds),
        'f1': f1_score(labels, preds, average='binary'),
    }


args = TrainingArguments(
    output_dir='./phishing-clf',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    evaluation_strategy='epoch',
    save_strategy='epoch',
    load_best_model_at_end=True,
    metric_for_best_model='f1',
    learning_rate=2e-5,
    weight_decay=0.01,
    report_to='none',
)

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_ds,
    eval_dataset=test_ds,
    compute_metrics=compute_metrics,
)

trainer.train()
trainer.save_model('./phishing-clf-final')

Three epochs on a V100 takes around 8 minutes for 30k examples. On CPU inference, expect roughly 50ms per email — acceptable for a mail gateway integration.

Serving the Classifier

Wrap the trained model in a FastAPI endpoint. Keep the preprocessing pipeline identical to what you used at training time; skipping this step is the most common source of silent accuracy regression in production.

from fastapi import FastAPI
from pydantic import BaseModel
from transformers import pipeline as hf_pipeline

app = FastAPI()
_clf = hf_pipeline(
    'text-classification',
    model='./phishing-clf-final',
    tokenizer=MODEL_NAME,
    truncation=True,
    max_length=512,
)


class EmailRequest(BaseModel):
    subject: str
    body: str


@app.post('/classify')
def classify(req: EmailRequest):
    text = url_heuristics(f'{req.subject} {req.body}')
    result = _clf(text)[0]
    return {
        'phishing': result['label'] == 'LABEL_1',
        'confidence': round(result['score'], 4),
    }

Wire this behind your mail gateway, an IMAP IDLE listener, or a milter integration and you have a classifier that updates with each monthly retrain cycle.

Production Considerations

A few things that are not obvious from the training code:

Retraining cadence. Phishing campaigns evolve continuously. Retrain monthly at minimum; weekly if analysts are feeding labeled examples from reported emails back into the pipeline.

Threshold tuning. The default 0.5 decision threshold is arbitrary. Plot the precision-recall curve for the phishing class and pick a threshold based on your organization's acceptable false-positive rate. A 0.3 threshold that catches 98% of phishing at the cost of 2% false positives may be better than a 0.5 threshold that catches 94% at 0.5%.

Header-based features. SPF/DKIM/DMARC pass/fail status, reply-to address mismatch, and routing anomalies are strong signals the body model cannot see. Add them as supplementary binary features fed into a stacked ensemble on top of the classifier's probability output.

Adversarial robustness. Attackers obfuscate text with Unicode lookalikes, zero-width characters, or image-only content. Normalize Unicode to NFKC before tokenization. For image-only attachments, run OCR before classification.

The classifier handles content analysis well, but it does not replace infrastructure controls. If you are building this as part of a broader email security program, the security hardening checklists at ayinedjimi-consultants.fr cover the SPF/DKIM/DMARC enforcement, attachment sandboxing, and link rewriting controls that the model alone cannot enforce.

The Takeaway

The TF-IDF baseline gets you to F1 ~0.93 in an afternoon and is worth shipping while you train the transformer. The fine-tuned language model pushes recall on phishing to ~0.97 and generalizes better to novel campaigns. In both cases: measure recall on the phishing class specifically, not overall accuracy; tune your decision threshold against a realistic false-positive budget; and plan for monthly retraining. A classifier trained six months ago without updates is losing ground to adversaries every day.

---

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.