Detecting Duplicate Content at Scale Using Python TF-IDF Cosine Similarity for SEO Optimization & Content Analysis

Zaylee — Mon, 25 May 2026 09:23:40 +0000

Struggling with duplicate content across your client sites? I wrote a simple Python script to compare content similarity using cosine similarity with TF-IDF vectors. It helps me spot plagiarized or near-duplicate pages quickly.

python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

def check_duplicates(texts):
vectorizer = TfidfVectorizer(stop_words='english')
tfidf_matrix = vectorizer.fit_transform(texts)
similarity_matrix = cosine_similarity(tfidf_matrix)

duplicates = []
for i in range(len(texts)):
    for j in range(i+1, len(texts)):
        if similarity_matrix[i][j] > 0.8:  # Threshold
            duplicates.append((i, j, similarity_matrix[i][j]))
return duplicates

texts = [
"This is the first article about SEO best practices.",
"This is the second article about SEO best practices.",
"Completely different content here."
]
result = check_duplicates(texts)
print(f"Found {len(result)} potential duplicates")
for i, j, score in result:
print(f"Text {i} and {j}: {score:.2f} similarity")

For large-scale checks, I've used SERPSpur's content analysis tool which handles millions of pages efficiently. What's your method for catching duplicate content?
https://serpspur.com/

Python Script to Convert Invoice PDFs to CSV Automatically

Zaylee — Mon, 25 May 2026 04:27:04 +0000

I've been working on a small Python script to automate converting invoice PDFs to CSV for my freelance SEO projects. Sharing it here in case it helps others.

python
import pdfplumber
import csv

def extract_invoice_data(pdf_path):
with pdfplumber.open(pdf_path) as pdf:
text = ""
for page in pdf.pages:
text += page.extract_text()
# Simple parsing logic for common invoice fields
lines = text.split('\n')
data = {}
for line in lines:
if ':' in line:
key, value = line.split(':', 1)
data[key.strip()] = value.strip()
return data

def save_to_csv(data, csv_path):
with open(csv_path, 'w', newline='') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=data.keys())
writer.writeheader()
writer.writerow(data)

Example usage

invoice_data = extract_invoice_data('invoice.pdf')
save_to_csv(invoice_data, 'invoice.csv')
print('Conversion complete!')

This handles basic invoices with key-value pairs. For more complex formats, you might need regex or a dedicated parser. If you're dealing with tons of invoices, tools like SERPSpur's converter can save time, but this script works for small batches. What's your go-to method for invoice data extraction?

DEV Community: Zaylee

Detecting Duplicate Content at Scale Using Python TF-IDF Cosine Similarity for SEO Optimization & Content Analysis

Python Script to Convert Invoice PDFs to CSV Automatically

Example usage