I like dashboards.
I just do not trust myself to check them consistently.
That was the whole reason I built a daily competitor digest.
I had all the data: competitor ads, landing pages, CTA shifts, campaign launches. But the insight was trapped inside tools that only became useful if someone remembered to open them.
That is the flaw with a lot of competitive intelligence stacks.
They optimize for exploration, not attention.
If you want a team to notice important competitor changes, the system should show up where they already work. For most teams, that means Slack or email. Not another tab.
So instead of building yet another competitor monitoring dashboard, I built a daily digest that answers one question every morning:
what changed yesterday that is actually worth knowing?
This post walks through the architecture, the diff logic, the data sources, and the JavaScript and Python code I would use to build it.
What Belongs in a Competitor Digest
The digest should not be a dump of every signal you can collect.
It should be a shortlist of changes that matter.
The categories I care about most are:
- new ads launched
- new landing pages detected
- meaningful CTA changes
- platform expansion or reactivation
- unusually large jumps in active creative count
That is enough to tell a growth team, founder, or product marketer where to look next.
Why a Digest Beats a Dashboard for Busy Teams
Dashboards are good for exploration.
Digests are good for follow-through.
A dashboard says, "go look around."
A digest says, "here are the three things that changed, and here is why they matter."
That difference is huge in practice.
Most teams do not have a competitor data problem. They have a prioritization problem.
The Architecture I Like Best
Keep it boring.
That is a compliment.
Scheduler
-> Collect today's snapshots
-> Compare with yesterday's snapshots
-> Build a short summary
-> Send to Slack or email
That is it.
You do not need an overbuilt analytics system to get value from this.
JavaScript Version: Daily Digest From Ad Library Snapshots
This version uses public ad data from Facebook, Google, LinkedIn, and Reddit, then compares today's snapshot with yesterday's stored JSON.
import fs from 'fs/promises';
const headers = { 'X-API-Key': process.env.SOCIAVAULT_API_KEY };
const SLACK_WEBHOOK = process.env.SLACK_WEBHOOK_URL;
const competitors = [
{ name: 'HubSpot', domain: 'hubspot.com', redditQuery: 'HubSpot' },
{ name: 'Notion', domain: 'notion.so', redditQuery: 'Notion' },
];
async function fetchJson(url) {
const response = await fetch(url, { headers });
if (!response.ok) {
throw new Error(`Request failed with ${response.status}: ${url}`);
}
return response.json();
}
function normalizeAds(items = []) {
return (items || []).map(item => ({
headline: item.headline || item.title || item.snapshot?.title || '',
body: item.body || item.text || item.snapshot?.body?.markup || '',
cta: item.cta || item.call_to_action || item.snapshot?.cta_text || '',
url: item.url || item.landingPageUrl || item.snapshot?.link_url || '',
}));
}
async function collectSnapshot(competitor) {
const [facebook, google, linkedin, reddit] = await Promise.all([
fetchJson(
`https://api.sociavault.com/v1/scrape/facebook-ad-library/company-ads?companyName=${encodeURIComponent(competitor.name)}&status=ACTIVE&trim=true`
),
fetchJson(
`https://api.sociavault.com/v1/scrape/google-ad-library/company-ads?domain=${encodeURIComponent(competitor.domain)}®ion=US`
),
fetchJson(
`https://api.sociavault.com/v1/scrape/linkedin-ad-library/search?company=${encodeURIComponent(competitor.name)}`
),
fetchJson(
`https://api.sociavault.com/v1/scrape/reddit/ads/search?query=${encodeURIComponent(competitor.redditQuery)}`
),
]);
return {
capturedAt: new Date().toISOString(),
competitor: competitor.name,
facebook: normalizeAds(facebook.data),
google: normalizeAds(google.data),
linkedin: normalizeAds(linkedin.data),
reddit: normalizeAds(reddit.data),
};
}
function uniqueValues(items, field) {
return new Set(items.map(item => item[field]).filter(Boolean));
}
function diffSnapshots(today, yesterday) {
const lines = [];
for (const platform of ['facebook', 'google', 'linkedin', 'reddit']) {
const current = today[platform] || [];
const previous = yesterday?.[platform] || [];
const currentUrls = uniqueValues(current, 'url');
const previousUrls = uniqueValues(previous, 'url');
const currentCtas = uniqueValues(current, 'cta');
const previousCtas = uniqueValues(previous, 'cta');
const newLandingPages = [...currentUrls].filter(url => !previousUrls.has(url));
const newCtas = [...currentCtas].filter(cta => !previousCtas.has(cta));
const countDelta = current.length - previous.length;
if (countDelta > 0) {
lines.push(`${platform}: ${countDelta} new active ads`);
}
if (newLandingPages.length > 0) {
lines.push(`${platform}: ${newLandingPages.length} new landing page${newLandingPages.length > 1 ? 's' : ''}`);
}
if (newCtas.length > 0) {
lines.push(`${platform}: new CTA signals -> ${newCtas.slice(0, 3).join(', ')}`);
}
if (previous.length === 0 && current.length > 0) {
lines.push(`${platform}: platform reactivated or newly detected`);
}
}
return lines;
}
async function saveSnapshot(filePath, snapshot) {
await fs.writeFile(filePath, JSON.stringify(snapshot, null, 2));
}
async function loadSnapshot(filePath) {
try {
const raw = await fs.readFile(filePath, 'utf8');
return JSON.parse(raw);
} catch {
return null;
}
}
async function sendSlackDigest(text) {
await fetch(SLACK_WEBHOOK, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text }),
});
}
for (const competitor of competitors) {
const filePath = `./snapshots/${competitor.name.toLowerCase().replace(/\s+/g, '-')}.json`;
const yesterday = await loadSnapshot(filePath);
const today = await collectSnapshot(competitor);
const lines = diffSnapshots(today, yesterday);
const digest = lines.length
? `Daily competitor digest: ${competitor.name}\n\n- ${lines.join('\n- ')}`
: `Daily competitor digest: ${competitor.name}\n\n- No meaningful public ad changes detected.`;
console.log(digest);
await sendSlackDigest(digest);
await saveSnapshot(filePath, today);
}
This is intentionally simple.
You can make it smarter later with scoring, AI summaries, ownership routing, or landing-page HTML diffs.
But this version is already useful.
Python Version: Same Digest, Same Core Logic
If you prefer Python for automation scripts and scheduled jobs, the same design works well there too.
import json
import os
from pathlib import Path
from datetime import datetime, timezone
import requests
HEADERS = {'X-API-Key': os.environ['SOCIAVAULT_API_KEY']}
SLACK_WEBHOOK = os.environ['SLACK_WEBHOOK_URL']
SNAPSHOT_DIR = Path('./snapshots')
SNAPSHOT_DIR.mkdir(exist_ok=True)
COMPETITORS = [
{'name': 'HubSpot', 'domain': 'hubspot.com', 'reddit_query': 'HubSpot'},
{'name': 'Notion', 'domain': 'notion.so', 'reddit_query': 'Notion'},
]
def fetch_json(url):
response = requests.get(url, headers=HEADERS, timeout=30)
response.raise_for_status()
return response.json()
def normalize_ads(items=None):
items = items or []
normalized = []
for item in items:
normalized.append({
'headline': item.get('headline') or item.get('title') or item.get('snapshot', {}).get('title', ''),
'body': item.get('body') or item.get('text') or item.get('snapshot', {}).get('body', {}).get('markup', ''),
'cta': item.get('cta') or item.get('call_to_action') or item.get('snapshot', {}).get('cta_text', ''),
'url': item.get('url') or item.get('landingPageUrl') or item.get('snapshot', {}).get('link_url', ''),
})
return normalized
def collect_snapshot(competitor):
facebook = fetch_json(
f"https://api.sociavault.com/v1/scrape/facebook-ad-library/company-ads?companyName={competitor['name']}&status=ACTIVE&trim=true"
)
google = fetch_json(
f"https://api.sociavault.com/v1/scrape/google-ad-library/company-ads?domain={competitor['domain']}®ion=US"
)
linkedin = fetch_json(
f"https://api.sociavault.com/v1/scrape/linkedin-ad-library/search?company={competitor['name']}"
)
reddit = fetch_json(
f"https://api.sociavault.com/v1/scrape/reddit/ads/search?query={competitor['reddit_query']}"
)
return {
'capturedAt': datetime.now(timezone.utc).isoformat(),
'competitor': competitor['name'],
'facebook': normalize_ads(facebook.get('data')),
'google': normalize_ads(google.get('data')),
'linkedin': normalize_ads(linkedin.get('data')),
'reddit': normalize_ads(reddit.get('data')),
}
def diff_snapshots(today, yesterday):
lines = []
for platform in ['facebook', 'google', 'linkedin', 'reddit']:
current = today.get(platform, [])
previous = yesterday.get(platform, []) if yesterday else []
current_urls = {item['url'] for item in current if item.get('url')}
previous_urls = {item['url'] for item in previous if item.get('url')}
current_ctas = {item['cta'] for item in current if item.get('cta')}
previous_ctas = {item['cta'] for item in previous if item.get('cta')}
new_urls = current_urls - previous_urls
new_ctas = current_ctas - previous_ctas
count_delta = len(current) - len(previous)
if count_delta > 0:
lines.append(f'{platform}: {count_delta} new active ads')
if new_urls:
lines.append(f'{platform}: {len(new_urls)} new landing pages')
if new_ctas:
lines.append(f"{platform}: new CTA signals -> {', '.join(list(new_ctas)[:3])}")
if not previous and current:
lines.append(f'{platform}: platform reactivated or newly detected')
return lines
def load_snapshot(path):
if not path.exists():
return None
return json.loads(path.read_text())
def save_snapshot(path, snapshot):
path.write_text(json.dumps(snapshot, indent=2))
def send_slack_digest(text):
requests.post(SLACK_WEBHOOK, json={'text': text}, timeout=30).raise_for_status()
for competitor in COMPETITORS:
path = SNAPSHOT_DIR / f"{competitor['name'].lower().replace(' ', '-')}.json"
yesterday = load_snapshot(path)
today = collect_snapshot(competitor)
lines = diff_snapshots(today, yesterday)
digest = (
f"Daily competitor digest: {competitor['name']}\n\n- " + '\n- '.join(lines)
if lines else
f"Daily competitor digest: {competitor['name']}\n\n- No meaningful public ad changes detected."
)
print(digest)
send_slack_digest(digest)
save_snapshot(path, today)
What I Would Add Next
Once the basic digest works, the next useful upgrades are usually:
- HTML diffs for landing pages
- scoring so only the highest-signal changes get included
- AI summarization for longer daily reports
- separate digests for paid media and organic content
- weekly rollups for leadership
But do not start there.
Get the daily change detection working first.
That is where the value lives.
Honest Alternatives
There are cases where a digest is not the best default.
Real-time alerts
Better for crisis monitoring, live launch tracking, or brand safety.
Dashboards
Better for analysts who want to explore everything.
Weekly reports
Better for leadership teams that do not want daily noise.
For most growth and competitive-intel teams, though, the daily digest is the sweet spot.
It keeps the system useful without forcing everyone to become a dashboard tourist.
Where SociaVault Fits
This workflow gets a lot easier when you are not manually checking four different public ad libraries yourself.
That is why I like using SociaVault as the public data layer here. It lets me spend my time on the snapshot, diff, and digest logic instead of the collection layer.
That is the split I want as an engineer.
Collection is table stakes. Delivery is where the product becomes useful.
Final Take
The best competitor monitoring system is not the one with the fanciest dashboard.
It is the one your team actually reads.
That is why I keep coming back to a daily digest. It turns competitor data into a habit instead of a homework assignment.
If you want to build one without wiring up every ad library manually, SociaVault is a solid place to start.
Then keep the rest boring: collect, diff, summarize, deliver.
That is enough to make competitor monitoring much more useful than most teams expect.
Top comments (0)