Build an Instagram Highlights Archiver to Save Any Brand's Best Content

Instagram Highlights are the most curated content a brand publishes. They pick their best stories, organize them into categories, and pin them permanently to their profile. It's basically their content strategy, organized and labeled for you.

And yet almost nobody is systematically archiving competitor Highlights.

Let's build an Instagram Highlights Archiver that:

1. Pulls all Highlights from any public profile
2. Archives the stories inside each Highlight
3. Tracks when they add, remove, or reorganize Highlights
4. Analyzes what content categories they prioritize

Why Highlights Are Strategic Intelligence

Think about what Instagram Highlights represent:

• Product launches they're most proud of
• Customer testimonials they want visitors to see first
• FAQs that reveal what customers ask most
• Behind-the-scenes content that shows brand personality
• Sales/promotions that drive the most conversions

When someone lands on an Instagram profile, Highlights are the first thing they scroll through. Brands know this. They curate them carefully.

If your competitor reorganized their Highlights last week, that wasn't an accident. They're testing new positioning.

The Stack

• Node.js: Runtime
• SociaVault API: Instagram Highlights + Highlight Detail endpoints
• better-sqlite3: Archive tracking
• OpenAI: Content strategy analysis

Step 1: Setup

mkdir ig-highlights-archiver
cd ig-highlights-archiver
npm init -y
npm install axios better-sqlite3 openai dotenv

Create .env:

SOCIAVAULT_API_KEY=your_key_here
OPENAI_API_KEY=your_openai_key

Step 2: Archive Database

Create db.js:

const Database = require('better-sqlite3');

const db = new Database('highlights.db');

db.exec(`
  CREATE TABLE IF NOT EXISTS profiles (
    username TEXT PRIMARY KEY,
    display_name TEXT,
    total_highlights INTEGER DEFAULT 0,
    last_checked TEXT
  );

  CREATE TABLE IF NOT EXISTS highlights (
    id TEXT PRIMARY KEY,
    username TEXT,
    title TEXT,
    cover_url TEXT,
    story_count INTEGER DEFAULT 0,
    first_seen TEXT DEFAULT (datetime('now')),
    last_seen TEXT DEFAULT (datetime('now')),
    is_active BOOLEAN DEFAULT 1,
    position INTEGER,
    FOREIGN KEY (username) REFERENCES profiles(username)
  );

  CREATE TABLE IF NOT EXISTS stories (
    id TEXT PRIMARY KEY,
    highlight_id TEXT,
    media_type TEXT,
    media_url TEXT,
    thumbnail_url TEXT,
    timestamp TEXT,
    archived_at TEXT DEFAULT (datetime('now')),
    FOREIGN KEY (highlight_id) REFERENCES highlights(id)
  );

  CREATE TABLE IF NOT EXISTS changes (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    username TEXT,
    change_type TEXT,
    detail TEXT,
    detected_at TEXT DEFAULT (datetime('now'))
  );
`);

module.exports = db;

Step 3: Fetch Highlights

Create archiver.js:

require('dotenv').config();
const axios = require('axios');
const db = require('./db');
const OpenAI = require('openai');

const API_BASE = 'https://api.sociavault.com';
const headers = { 'Authorization': `Bearer ${process.env.SOCIAVAULT_API_KEY}` };
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function getHighlights(username) {
  console.log(`📸 Fetching Highlights for @${username}...\n`);

  const { data } = await axios.get(
    `${API_BASE}/v1/scrape/instagram/highlights`,
    { params: { username }, headers }
  );

  const highlights = data.data?.highlights || data.data || [];

  console.log(`  Found ${highlights.length} Highlights:\n`);
  highlights.forEach((h, i) => {
    const title = h.title || h.name || 'Untitled';
    const count = h.storyCount || h.mediaCount || '?';
    console.log(`  ${i + 1}. "${title}" (${count} stories)`);
  });
  console.log();

  // Detect changes from previous check
  const existing = db.prepare(
    'SELECT * FROM highlights WHERE username = ? AND is_active = 1'
  ).all(username);

  const existingTitles = new Set(existing.map(h => h.title));
  const currentTitles = new Set(highlights.map(h => h.title || h.name));

  // New highlights
  for (const title of currentTitles) {
    if (!existingTitles.has(title)) {
      db.prepare('INSERT INTO changes (username, change_type, detail) VALUES (?, ?, ?)')
        .run(username, 'highlight_added', `New Highlight: "${title}"`);
      console.log(`  🆕 NEW: "${title}"`);
    }
  }

  // Removed highlights
  for (const title of existingTitles) {
    if (!currentTitles.has(title)) {
      db.prepare('INSERT INTO changes (username, change_type, detail) VALUES (?, ?, ?)')
        .run(username, 'highlight_removed', `Removed Highlight: "${title}"`);
      console.log(`  ❌ REMOVED: "${title}"`);

      db.prepare('UPDATE highlights SET is_active = 0 WHERE username = ? AND title = ?')
        .run(username, title);
    }
  }

  // Position changes
  for (let i = 0; i < highlights.length; i++) {
    const h = highlights[i];
    const title = h.title || h.name;
    const prev = existing.find(e => e.title === title);

    if (prev && prev.position !== i) {
      db.prepare('INSERT INTO changes (username, change_type, detail) VALUES (?, ?, ?)')
        .run(username, 'position_changed', `"${title}" moved from position ${prev.position + 1} to ${i + 1}`);
      console.log(`  🔄 MOVED: "${title}" (${prev.position + 1} → ${i + 1})`);
    }
  }

  // Store highlights
  const upsert = db.prepare(`
    INSERT INTO highlights (id, username, title, cover_url, story_count, position)
    VALUES (?, ?, ?, ?, ?, ?)
    ON CONFLICT(id) DO UPDATE SET
      last_seen = datetime('now'),
      is_active = 1,
      story_count = ?,
      position = ?
  `);

  db.prepare('UPDATE profiles SET total_highlights = ?, last_checked = datetime("now") WHERE username = ?')
    .run(highlights.length, username);
  db.prepare('INSERT OR IGNORE INTO profiles (username, total_highlights) VALUES (?, ?)')
    .run(username, highlights.length);

  const tx = db.transaction(() => {
    for (let i = 0; i < highlights.length; i++) {
      const h = highlights[i];
      const id = h.id || h.highlightId || `hl_${username}_${i}`;
      const title = h.title || h.name || 'Untitled';
      const count = h.storyCount || h.mediaCount || 0;

      upsert.run(id, username, title, h.coverUrl || '', count, i, count, i);
    }
  });

  tx();

  return highlights;
}

Step 4: Archive Individual Highlight Stories

Dig into each highlight and save every story:

async function archiveHighlightStories(highlightId) {
  console.log(`  📥 Archiving stories from Highlight ${highlightId}...`);

  const { data } = await axios.get(
    `${API_BASE}/v1/scrape/instagram/highlight-detail`,
    { params: { highlight_id: highlightId }, headers }
  );

  const stories = data.data?.stories || data.data?.items || data.data || [];

  const upsert = db.prepare(`
    INSERT OR IGNORE INTO stories (id, highlight_id, media_type, media_url, thumbnail_url, timestamp)
    VALUES (?, ?, ?, ?, ?, ?)
  `);

  let newCount = 0;
  const tx = db.transaction(() => {
    for (const story of stories) {
      const result = upsert.run(
        story.id || story.mediaId || `st_${Math.random().toString(36).slice(2)}`,
        highlightId,
        story.mediaType || (story.videoUrl ? 'video' : 'image'),
        story.mediaUrl || story.videoUrl || story.imageUrl || '',
        story.thumbnailUrl || story.imageUrl || '',
        story.timestamp || story.takenAt || ''
      );
      if (result.changes > 0) newCount++;
    }
  });

  tx();
  console.log(`    → ${stories.length} stories (${newCount} new)\n`);

  return stories;
}

async function fullArchive(username) {
  const highlights = await getHighlights(username);

  console.log(`\n📦 Archiving all stories...\n`);

  for (const h of highlights) {
    const id = h.id || h.highlightId;
    if (!id) continue;

    const title = h.title || h.name || 'Untitled';
    console.log(`  📁 "${title}"`);
    await archiveHighlightStories(id);

    await new Promise(r => setTimeout(r, 1500));
  }

  // Summary
  const totalStories = db.prepare(
    `SELECT COUNT(*) as count FROM stories s
     JOIN highlights h ON s.highlight_id = h.id
     WHERE h.username = ?`
  ).get(username);

  console.log(`\n✅ Archive complete: ${highlights.length} Highlights, ${totalStories.count} total stories`);
}

Step 5: Content Strategy Analysis

async function analyzeContentStrategy(username) {
  const highlights = db.prepare(
    'SELECT * FROM highlights WHERE username = ? AND is_active = 1 ORDER BY position'
  ).all(username);

  if (highlights.length < 2) {
    console.log('Need more data. Run archive first.');
    return;
  }

  const changes = db.prepare(
    'SELECT * FROM changes WHERE username = ? ORDER BY detected_at DESC LIMIT 20'
  ).all(username);

  console.log(`\n🧠 Analyzing @${username}'s Highlights strategy...\n`);

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{
      role: 'user',
      content: `Analyze this brand's Instagram Highlights for content strategy insights.

Highlights (in order of display position):
${JSON.stringify(highlights.map((h, i) => ({
  position: i + 1,
  title: h.title,
  storyCount: h.story_count,
  daysActive: Math.floor((Date.now() - new Date(h.first_seen).getTime()) / 86400000),
})), null, 2)}

Recent changes:
${JSON.stringify(changes.map(c => ({
  type: c.change_type,
  detail: c.detail,
  date: c.detected_at,
})), null, 2)}

Return JSON:
{
  "content_pillars": ["main content themes based on Highlight titles"],
  "priority_analysis": "What they prioritize based on position and story count",
  "brand_positioning": "How they want visitors to perceive them",
  "customer_journey": "The story they're telling as someone scrolls through Highlights",
  "strategic_changes": "Analysis of recent changes (if any)",
  "category_breakdown": {
    "product_focused": "% of Highlights about products",
    "social_proof": "% featuring testimonials/reviews",
    "educational": "% teaching/informing",
    "brand_personality": "% showing culture/behind-the-scenes",
    "promotional": "% about sales/offers"
  },
  "competitive_insights": [
    "What you can learn from their Highlights strategy",
    "What they're doing well",
    "What they're missing"
  ],
  "recommendations": [
    "How to build better Highlights than theirs",
    "Categories they haven't covered",
    "Ways to differentiate"
  ]
}`
    }],
    response_format: { type: 'json_object' }
  });

  const analysis = JSON.parse(completion.choices[0].message.content);

  console.log('🎯 HIGHLIGHTS STRATEGY ANALYSIS');
  console.log('═'.repeat(55));

  console.log(`\n📌 Content Pillars: ${analysis.content_pillars.join(', ')}`);
  console.log(`\n🎯 Priority: ${analysis.priority_analysis}`);
  console.log(`\n💡 Brand Positioning: ${analysis.brand_positioning}`);
  console.log(`\n📖 Customer Journey: ${analysis.customer_journey}`);

  if (analysis.strategic_changes) {
    console.log(`\n🔄 Recent Strategy Shifts: ${analysis.strategic_changes}`);
  }

  console.log('\n📊 Category Breakdown:');
  Object.entries(analysis.category_breakdown).forEach(([cat, pct]) => {
    console.log(`  ${cat.replace(/_/g, ' ')}: ${pct}`);
  });

  console.log('\n💡 Competitive Insights:');
  analysis.competitive_insights.forEach((i, n) => console.log(`  ${n+1}. ${i}`));

  console.log('\n🚀 Recommendations:');
  analysis.recommendations.forEach((r, n) => console.log(`  ${n+1}. ${r}`));

  return analysis;
}

Step 6: Competitor Comparison

async function compareHighlights(usernames) {
  console.log(`\n📊 Comparing Highlights across ${usernames.length} brands...\n`);

  const profiles = [];

  for (const username of usernames) {
    const highlights = await getHighlights(username);

    profiles.push({
      username,
      highlightCount: highlights.length,
      titles: highlights.map(h => h.title || h.name || 'Untitled'),
      totalStories: highlights.reduce((sum, h) => sum + (h.storyCount || h.mediaCount || 0), 0),
    });

    await new Promise(r => setTimeout(r, 2000));
  }

  console.log('\n📋 HIGHLIGHTS COMPARISON');
  console.log('═'.repeat(60));

  for (const p of profiles) {
    console.log(`\n  @${p.username}`);
    console.log(`  Highlights: ${p.highlightCount} | Stories: ${p.totalStories}`);
    console.log(`  Categories: ${p.titles.join(' | ')}`);
  }

  // Find common themes
  const allTitles = profiles.flatMap(p => p.titles.map(t => t.toLowerCase()));
  const titleCounts = {};
  allTitles.forEach(t => titleCounts[t] = (titleCounts[t] || 0) + 1);

  const common = Object.entries(titleCounts)
    .filter(([_, count]) => count > 1)
    .sort((a, b) => b[1] - a[1]);

  if (common.length > 0) {
    console.log('\n\n🔄 Common Highlight Categories:');
    common.forEach(([title, count]) => {
      console.log(`  "${title}" — used by ${count}/${usernames.length} brands`);
    });
  }

  return profiles;
}

Step 7: CLI

async function main() {
  const command = process.argv[2];
  const target = process.argv[3];

  switch (command) {
    case 'check':
      await getHighlights(target.replace('@', ''));
      break;

    case 'archive':
      await fullArchive(target.replace('@', ''));
      break;

    case 'analyze':
      await analyzeContentStrategy(target.replace('@', ''));
      break;

    case 'compare': {
      const usernames = target.split(',').map(u => u.trim().replace('@', ''));
      await compareHighlights(usernames);
      break;
    }

    case 'changes': {
      const changes = db.prepare(
        'SELECT * FROM changes WHERE username = ? ORDER BY detected_at DESC LIMIT 20'
      ).all(target.replace('@', ''));
      console.log(`\n📜 Recent changes for @${target}:\n`);
      changes.forEach(c => {
        console.log(`  ${c.detected_at} | ${c.change_type}: ${c.detail}`);
      });
      break;
    }

    default:
      console.log('Instagram Highlights Archiver\n');
      console.log('Commands:');
      console.log('  node archiver.js check <username>             - List Highlights');
      console.log('  node archiver.js archive <username>           - Full archive with stories');
      console.log('  node archiver.js analyze <username>           - AI strategy analysis');
      console.log('  node archiver.js compare user1,user2,user3    - Compare brands');
      console.log('  node archiver.js changes <username>           - View change history');
  }
}

main().catch(console.error);

Running It

# Quick check
node archiver.js check glossier

# Full archive
node archiver.js archive glossier

# Analyze strategy
node archiver.js analyze glossier

# Compare competitors
node archiver.js compare "glossier,fentybeauty,rfrfrfrfrfre"

Why This Matters

Highlights are the only Instagram content that:

• Doesn't disappear (unlike stories)
• Is manually curated (unlike the algorithm-sorted feed)
• Shows clear categories (unlike posts)
• Reveals priority order (position = importance)

When a brand reorganizes their Highlights, it means they tested something and changed direction. When they add a new category, it means they're expanding into a new content pillar. When they remove one, it means it didn't convert.

This is competitive intelligence hiding in plain sight.

Cost Comparison

Method	Cost	Coverage
Manual checking	Free	Once a week if you remember
Social media monitoring tools	$100-300/mo	Don't track Highlights
Competitive intelligence platforms	$500+/mo	May track some visual elements
This tool	~$0.05/check	Full archive + change tracking

Get Started

1. Get your API key at sociavault.com
2. Pick 3 competitors and archive their Highlights
3. Set up weekly checks to catch repositioning moves

The most strategic content on Instagram is the content brands choose to pin permanently. Pay attention to it.

---

Instagram Highlights are a brand's curated pitch to new visitors. Reading your competitor's Highlights is reading their strategy deck.

javascript #instagram #marketing #webdev