Olamide Olaniyan

Posted on Jan 12

Build a Hashtag Research Tool That Finds Hidden Gems

#webdev #ai #programming #tutorial

Most hashtag tools show you what everyone already knows: #fyp, #viral, #trending.

Useless.

The real wins come from finding niche hashtags with high engagement and low competition. The ones your competitors haven't discovered yet.

In this tutorial, we'll build a Hashtag Research Tool that:

Analyzes hashtag performance across platforms
Finds related hashtags with untapped potential
Scores hashtags by competition vs. reach ratio

Stop copying hashtags. Start finding hidden gems.

The Hashtag Opportunity Formula

A good hashtag isn't about total posts. It's about:

Opportunity Score = (Avg Engagement / Post Count) × Recency Factor

A hashtag with 10K posts averaging 50K views each is BETTER than a hashtag with 1M posts averaging 500 views.

What we're looking for:

High Engagement: Posts using this tag actually perform
Low Competition: Not oversaturated
Active Growth: Posts are recent (not a dead tag)
Niche Relevance: Related to your content

The Stack

Node.js: Runtime
SociaVault API: Hashtag data across platforms
OpenAI API: For generating related hashtag ideas

Step 1: Setup

mkdir hashtag-research
cd hashtag-research
npm init -y
npm install axios openai dotenv

Create .env:

SOCIAVAULT_API_KEY=your_sociavault_key
OPENAI_API_KEY=your_openai_key

Step 2: Fetch Hashtag Data

Create index.js:

require('dotenv').config();
const axios = require('axios');
const OpenAI = require('openai');

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const SOCIAVAULT_BASE = 'https://api.sociavault.com';
const headers = { 'Authorization': `Bearer ${process.env.SOCIAVAULT_API_KEY}` };

async function getTikTokHashtagData(hashtag) {
  console.log(`📱 Fetching TikTok data for #${hashtag}...`);

  try {
    const response = await axios.get(`${SOCIAVAULT_BASE}/v1/scrape/tiktok/hashtag`, {
      params: { hashtag, limit: 30 },
      headers
    });

    const data = response.data.data;
    const videos = data.videos || data;

    // Calculate metrics from the posts
    const stats = analyzeHashtagPosts(videos, 'tiktok');

    return {
      platform: 'tiktok',
      hashtag,
      totalPosts: data.totalPosts || data.postCount || videos.length * 100, // Estimate if not available
      ...stats
    };
  } catch (error) {
    console.error(`TikTok hashtag error for #${hashtag}:`, error.message);
    return null;
  }
}

async function getInstagramHashtagData(hashtag) {
  console.log(`📸 Fetching Instagram data for #${hashtag}...`);

  try {
    const response = await axios.get(`${SOCIAVAULT_BASE}/v1/scrape/instagram/hashtag`, {
      params: { hashtag, limit: 30 },
      headers
    });

    const data = response.data.data;
    const posts = data.posts || data.items || data;

    const stats = analyzeHashtagPosts(posts, 'instagram');

    return {
      platform: 'instagram',
      hashtag,
      totalPosts: data.mediaCount || data.media_count || posts.length * 100,
      ...stats
    };
  } catch (error) {
    console.error(`Instagram hashtag error for #${hashtag}:`, error.message);
    return null;
  }
}

async function searchTikTokByKeyword(keyword) {
  console.log(`🔍 Searching TikTok for "${keyword}"...`);

  try {
    const response = await axios.get(`${SOCIAVAULT_BASE}/v1/scrape/tiktok/search`, {
      params: { query: keyword, limit: 30 },
      headers
    });

    return response.data.data || [];
  } catch (error) {
    console.error('Search error:', error.message);
    return [];
  }
}

function analyzeHashtagPosts(posts, platform) {
  if (!posts || posts.length === 0) {
    return {
      postsAnalyzed: 0,
      avgViews: 0,
      avgLikes: 0,
      avgEngagement: 0,
      topPerformers: 0,
      recentPosts: 0,
      commonHashtags: []
    };
  }

  const metrics = posts.map(post => {
    if (platform === 'tiktok') {
      return {
        views: post.playCount || post.stats?.playCount || 0,
        likes: post.diggCount || post.stats?.diggCount || 0,
        comments: post.commentCount || post.stats?.commentCount || 0,
        shares: post.shareCount || post.stats?.shareCount || 0,
        created: post.createTime ? new Date(post.createTime * 1000) : new Date(),
        hashtags: extractHashtags(post.desc || post.description || ''),
        authorFollowers: post.author?.followerCount || post.authorStats?.followerCount || 0
      };
    } else {
      return {
        views: post.play_count || post.video_view_count || (post.like_count || 0) * 10,
        likes: post.like_count || post.likes || 0,
        comments: post.comment_count || post.comments || 0,
        created: post.taken_at ? new Date(post.taken_at * 1000) : new Date(),
        hashtags: extractHashtags(post.caption || ''),
        authorFollowers: post.user?.follower_count || 0
      };
    }
  });

  const totalViews = metrics.reduce((sum, m) => sum + m.views, 0);
  const totalLikes = metrics.reduce((sum, m) => sum + m.likes, 0);
  const totalComments = metrics.reduce((sum, m) => sum + m.comments, 0);

  const avgViews = totalViews / metrics.length;
  const avgLikes = totalLikes / metrics.length;
  const avgEngagement = avgViews > 0 ? ((totalLikes + totalComments) / totalViews) * 100 : 0;

  // Count posts with above-average performance
  const topPerformers = metrics.filter(m => m.views > avgViews * 1.5).length;

  // Count posts from last 7 days
  const weekAgo = Date.now() - (7 * 24 * 60 * 60 * 1000);
  const recentPosts = metrics.filter(m => m.created.getTime() > weekAgo).length;

  // Find common co-occurring hashtags
  const hashtagCounts = {};
  metrics.forEach(m => {
    m.hashtags.forEach(tag => {
      hashtagCounts[tag] = (hashtagCounts[tag] || 0) + 1;
    });
  });

  const commonHashtags = Object.entries(hashtagCounts)
    .filter(([tag, count]) => count >= 2)
    .sort((a, b) => b[1] - a[1])
    .slice(0, 20)
    .map(([tag, count]) => ({ tag, count, percentage: Math.round((count / metrics.length) * 100) }));

  return {
    postsAnalyzed: metrics.length,
    avgViews: Math.round(avgViews),
    avgLikes: Math.round(avgLikes),
    avgEngagement: avgEngagement.toFixed(2),
    topPerformers,
    recentPosts,
    commonHashtags
  };
}

function extractHashtags(text) {
  const matches = text.match(/#[\w\u4e00-\u9fa5]+/g) || [];
  return matches.map(tag => tag.toLowerCase().replace('#', ''));
}

Step 3: Score Hashtags

function calculateHashtagScore(hashtagData) {
  const {
    totalPosts,
    avgViews,
    avgLikes,
    avgEngagement,
    topPerformers,
    recentPosts,
    postsAnalyzed
  } = hashtagData;

  let score = 0;
  const factors = [];

  // Competition Score (lower posts = higher score, but too low means dead)
  let competitionScore = 0;
  if (totalPosts < 1000) {
    competitionScore = 15; // Very niche but might be dead
    factors.push({ name: 'Competition', value: 'Very Low', impact: '+15', note: 'Niche opportunity' });
  } else if (totalPosts < 10000) {
    competitionScore = 25;
    factors.push({ name: 'Competition', value: 'Low', impact: '+25', note: 'Sweet spot' });
  } else if (totalPosts < 100000) {
    competitionScore = 20;
    factors.push({ name: 'Competition', value: 'Medium', impact: '+20', note: 'Moderate competition' });
  } else if (totalPosts < 1000000) {
    competitionScore = 10;
    factors.push({ name: 'Competition', value: 'High', impact: '+10', note: 'Saturated' });
  } else {
    competitionScore = 5;
    factors.push({ name: 'Competition', value: 'Very High', impact: '+5', note: 'Oversaturated' });
  }
  score += competitionScore;

  // Engagement Quality Score
  let engagementScore = 0;
  const eng = parseFloat(avgEngagement);
  if (eng > 10) {
    engagementScore = 30;
    factors.push({ name: 'Engagement Rate', value: `${avgEngagement}%`, impact: '+30', note: 'Excellent' });
  } else if (eng > 5) {
    engagementScore = 25;
    factors.push({ name: 'Engagement Rate', value: `${avgEngagement}%`, impact: '+25', note: 'Very Good' });
  } else if (eng > 2) {
    engagementScore = 15;
    factors.push({ name: 'Engagement Rate', value: `${avgEngagement}%`, impact: '+15', note: 'Average' });
  } else {
    engagementScore = 5;
    factors.push({ name: 'Engagement Rate', value: `${avgEngagement}%`, impact: '+5', note: 'Below Average' });
  }
  score += engagementScore;

  // View Potential Score
  let viewScore = 0;
  if (avgViews > 100000) {
    viewScore = 25;
    factors.push({ name: 'Avg Views', value: formatNumber(avgViews), impact: '+25', note: 'Viral potential' });
  } else if (avgViews > 50000) {
    viewScore = 20;
    factors.push({ name: 'Avg Views', value: formatNumber(avgViews), impact: '+20', note: 'High reach' });
  } else if (avgViews > 10000) {
    viewScore = 15;
    factors.push({ name: 'Avg Views', value: formatNumber(avgViews), impact: '+15', note: 'Good reach' });
  } else if (avgViews > 1000) {
    viewScore = 10;
    factors.push({ name: 'Avg Views', value: formatNumber(avgViews), impact: '+10', note: 'Moderate reach' });
  } else {
    viewScore = 5;
    factors.push({ name: 'Avg Views', value: formatNumber(avgViews), impact: '+5', note: 'Low reach' });
  }
  score += viewScore;

  // Recency Score (is this hashtag still active?)
  let recencyScore = 0;
  const recencyRate = postsAnalyzed > 0 ? (recentPosts / postsAnalyzed) * 100 : 0;
  if (recencyRate > 70) {
    recencyScore = 15;
    factors.push({ name: 'Recency', value: `${recentPosts}/${postsAnalyzed} recent`, impact: '+15', note: 'Very active' });
  } else if (recencyRate > 40) {
    recencyScore = 10;
    factors.push({ name: 'Recency', value: `${recentPosts}/${postsAnalyzed} recent`, impact: '+10', note: 'Active' });
  } else if (recencyRate > 20) {
    recencyScore = 5;
    factors.push({ name: 'Recency', value: `${recentPosts}/${postsAnalyzed} recent`, impact: '+5', note: 'Moderate activity' });
  } else {
    recencyScore = 0;
    factors.push({ name: 'Recency', value: `${recentPosts}/${postsAnalyzed} recent`, impact: '+0', note: 'Possibly dead' });
  }
  score += recencyScore;

  // Top Performer Consistency
  const consistencyRate = postsAnalyzed > 0 ? (topPerformers / postsAnalyzed) * 100 : 0;
  let consistencyScore = 0;
  if (consistencyRate > 30) {
    consistencyScore = 5;
    factors.push({ name: 'Top Performers', value: `${topPerformers} viral posts`, impact: '+5', note: 'Consistent hits' });
  }
  score += consistencyScore;

  // Calculate opportunity rating
  const opportunityRating = getOpportunityRating(score);

  return {
    score,
    maxScore: 100,
    factors,
    opportunityRating,
    recommendation: generateRecommendation(score, hashtagData)
  };
}

function getOpportunityRating(score) {
  if (score >= 80) return { rating: 'A+', emoji: '🔥', description: 'Exceptional opportunity' };
  if (score >= 70) return { rating: 'A', emoji: '⭐', description: 'Great opportunity' };
  if (score >= 60) return { rating: 'B+', emoji: '✅', description: 'Good opportunity' };
  if (score >= 50) return { rating: 'B', emoji: '👍', description: 'Decent opportunity' };
  if (score >= 40) return { rating: 'C', emoji: '➖', description: 'Average' };
  return { rating: 'D', emoji: '⚠️', description: 'Not recommended' };
}

function generateRecommendation(score, data) {
  if (score >= 70) {
    return `Strong hashtag! Use this regularly. ${data.avgViews > 50000 ? 'High viral potential.' : 'Good engagement rates.'}`;
  } else if (score >= 50) {
    return `Solid hashtag for your mix. ${data.totalPosts > 100000 ? 'Include with niche tags.' : 'Good niche option.'}`;
  } else if (score >= 40) {
    return 'Use sparingly. Consider finding related alternatives.';
  } else {
    return 'Skip this one. Too competitive or low engagement.';
  }
}

function formatNumber(num) {
  if (num >= 1000000) return (num / 1000000).toFixed(1) + 'M';
  if (num >= 1000) return (num / 1000).toFixed(1) + 'K';
  return num.toString();
}

Step 4: Find Related Hashtags

async function findRelatedHashtags(hashtag, platform = 'tiktok') {
  console.log(`\n🔍 Finding related hashtags for #${hashtag}...`);

  // Get hashtags that commonly appear with our target
  let hashtagData;
  if (platform === 'tiktok') {
    hashtagData = await getTikTokHashtagData(hashtag);
  } else {
    hashtagData = await getInstagramHashtagData(hashtag);
  }

  if (!hashtagData) return [];

  // Use AI to expand with related ideas
  const aiSuggestions = await generateRelatedHashtags(hashtag, hashtagData.commonHashtags);

  // Combine co-occurring hashtags with AI suggestions
  const allRelated = [
    ...hashtagData.commonHashtags.map(h => h.tag),
    ...aiSuggestions
  ];

  // Remove duplicates and the original hashtag
  const unique = [...new Set(allRelated)]
    .filter(tag => tag.toLowerCase() !== hashtag.toLowerCase())
    .slice(0, 15);

  return unique;
}

async function generateRelatedHashtags(hashtag, existingTags) {
  const prompt = `Given the hashtag #${hashtag} and these related hashtags that often appear with it:
${existingTags.slice(0, 10).map(t => `#${t.tag}`).join(', ')}

Generate 10 additional related hashtag ideas that:
1. Are in the same niche/topic
2. Would attract similar audiences
3. Mix between popular and niche options
4. Don't include obvious generic tags like #fyp, #viral, #trending

Return ONLY a JSON array of hashtag strings without the # symbol.
Example: ["fitness", "workout", "homegym"]`;

  try {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: prompt }],
      response_format: { type: 'json_object' }
    });

    const result = JSON.parse(response.choices[0].message.content);
    return result.hashtags || result.tags || Object.values(result).flat();
  } catch (error) {
    console.error('AI suggestion error:', error.message);
    return [];
  }
}

Step 5: Research Multiple Hashtags

async function researchHashtags(hashtags, platform = 'tiktok') {
  console.log('\n═══════════════════════════════════════════════════════════');
  console.log('🏷️  HASHTAG RESEARCH TOOL');
  console.log('═══════════════════════════════════════════════════════════\n');

  const results = [];

  for (const hashtag of hashtags) {
    let data;
    if (platform === 'tiktok') {
      data = await getTikTokHashtagData(hashtag);
    } else {
      data = await getInstagramHashtagData(hashtag);
    }

    if (data) {
      const score = calculateHashtagScore(data);
      results.push({
        hashtag,
        ...data,
        ...score
      });
    }

    // Small delay between requests
    await new Promise(resolve => setTimeout(resolve, 500));
  }

  // Sort by score
  results.sort((a, b) => b.score - a.score);

  return results;
}

function displayResults(results) {
  console.log('\n═══════════════════════════════════════════════════════════');
  console.log('📊 HASHTAG ANALYSIS RESULTS');
  console.log('═══════════════════════════════════════════════════════════\n');

  results.forEach((r, index) => {
    console.log(`${index + 1}. #${r.hashtag}`);
    console.log('─────────────────────────────────────────────────────────');
    console.log(`   ${r.opportunityRating.emoji} Score: ${r.score}/100 (${r.opportunityRating.rating})`);
    console.log(`   📊 ${formatNumber(r.totalPosts)} total posts | ${formatNumber(r.avgViews)} avg views`);
    console.log(`   💬 ${r.avgEngagement}% engagement rate`);
    console.log(`   📈 ${r.recommendation}`);
    console.log('');

    // Show scoring breakdown
    console.log('   Scoring Breakdown:');
    r.factors.forEach(f => {
      console.log(`   • ${f.name}: ${f.value} (${f.impact}) - ${f.note}`);
    });
    console.log('\n');
  });

  // Summary table
  console.log('═══════════════════════════════════════════════════════════');
  console.log('📋 QUICK REFERENCE');
  console.log('═══════════════════════════════════════════════════════════\n');

  console.log('TIER A (Use These):');
  results.filter(r => r.score >= 60).forEach(r => {
    console.log(`  ${r.opportunityRating.emoji} #${r.hashtag} (${r.score}/100)`);
  });

  console.log('\nTIER B (Mix In):');
  results.filter(r => r.score >= 40 && r.score < 60).forEach(r => {
    console.log(`  ${r.opportunityRating.emoji} #${r.hashtag} (${r.score}/100)`);
  });

  console.log('\nTIER C (Skip):');
  results.filter(r => r.score < 40).forEach(r => {
    console.log(`  ${r.opportunityRating.emoji} #${r.hashtag} (${r.score}/100)`);
  });
}

Step 6: Generate Hashtag Sets

async function generateHashtagSet(niche, platform = 'tiktok') {
  console.log(`\n🎯 Generating optimized hashtag set for: ${niche}\n`);

  // Generate seed hashtags from niche
  const seedHashtags = await generateSeedHashtags(niche);
  console.log(`Seed hashtags: ${seedHashtags.join(', ')}`);

  // Research all seed hashtags
  const research = await researchHashtags(seedHashtags, platform);

  // Display results
  displayResults(research);

  // Find related hashtags from top performers
  const topHashtag = research[0]?.hashtag;
  if (topHashtag) {
    const related = await findRelatedHashtags(topHashtag, platform);
    console.log('\n🔗 RELATED HASHTAGS TO EXPLORE:');
    console.log('─────────────────────────────────────────────────────────');
    console.log(related.map(t => `#${t}`).join('  '));
  }

  // Generate final recommended set
  const finalSet = generateOptimalSet(research);

  console.log('\n═══════════════════════════════════════════════════════════');
  console.log('🏆 YOUR OPTIMIZED HASHTAG SET');
  console.log('═══════════════════════════════════════════════════════════\n');
  console.log('Copy this for your next post:\n');
  console.log(finalSet.map(t => `#${t}`).join(' '));
  console.log('\n');

  return { research, finalSet };
}

async function generateSeedHashtags(niche) {
  const prompt = `Generate 10 hashtags for the niche: "${niche}"

Include a mix of:
- 2 broad/popular hashtags (100K+ posts typically)
- 4 medium-sized niche hashtags (10K-100K posts)
- 4 specific/long-tail hashtags (under 10K posts)

Return ONLY a JSON array of hashtag strings without the # symbol.
Don't include generic tags like fyp, viral, trending, foryou.`;

  try {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: prompt }],
      response_format: { type: 'json_object' }
    });

    const result = JSON.parse(response.choices[0].message.content);
    return result.hashtags || result.tags || Object.values(result).flat();
  } catch (error) {
    console.error('Seed generation error:', error.message);
    return [niche.toLowerCase().replace(/\s+/g, '')];
  }
}

function generateOptimalSet(research) {
  // Select hashtags for an optimal mix
  const tierA = research.filter(r => r.score >= 60).slice(0, 4);
  const tierB = research.filter(r => r.score >= 40 && r.score < 60).slice(0, 3);
  const tierC = research.filter(r => r.score >= 30 && r.score < 40).slice(0, 2);

  const selected = [...tierA, ...tierB, ...tierC].map(r => r.hashtag);

  // Aim for 8-10 hashtags total
  return selected.slice(0, 10);
}

Step 7: Run It

async function main() {
  const niche = process.argv[2] || 'fitness motivation';
  const platform = process.argv[3] || 'tiktok';

  // Full analysis with set generation
  await generateHashtagSet(niche, platform);
}

main();

Run with:

node index.js "fitness motivation" tiktok
node index.js "cooking recipes" instagram
node index.js "tech reviews" tiktok

Sample Output

🎯 Generating optimized hashtag set for: fitness motivation

Seed hashtags: fitnessmotivation, gymlife, workout, homeworkout, fitnessjourney, gains, fitfam, strengthtraining, personaltrainer, fitnesstips

📱 Fetching TikTok data for #fitnessmotivation...
📱 Fetching TikTok data for #gymlife...
📱 Fetching TikTok data for #workout...
...

═══════════════════════════════════════════════════════════
📊 HASHTAG ANALYSIS RESULTS
═══════════════════════════════════════════════════════════

1. #homeworkout
─────────────────────────────────────────────────────────
   ⭐ Score: 78/100 (A)
   📊 45.2K total posts | 89.3K avg views
   💬 6.7% engagement rate
   📈 Strong hashtag! Use this regularly. High viral potential.

   Scoring Breakdown:
   • Competition: Low (+25) - Sweet spot
   • Engagement Rate: 6.7% (+25) - Very Good
   • Avg Views: 89.3K (+20) - High reach
   • Recency: 24/30 recent (+15) - Very active
   • Top Performers: 8 viral posts (+5) - Consistent hits

2. #strengthtraining
─────────────────────────────────────────────────────────
   ⭐ Score: 72/100 (A)
   📊 78.5K total posts | 62.1K avg views
   💬 5.8% engagement rate
   📈 Strong hashtag! Use this regularly. Good engagement rates.

   ...

═══════════════════════════════════════════════════════════
📋 QUICK REFERENCE
═══════════════════════════════════════════════════════════

TIER A (Use These):
  ⭐ #homeworkout (78/100)
  ⭐ #strengthtraining (72/100)
  ✅ #fitnesstips (68/100)
  ✅ #personaltrainer (64/100)

TIER B (Mix In):
  👍 #fitnessjourney (56/100)
  👍 #fitfam (52/100)
  ➖ #gains (48/100)

TIER C (Skip):
  ⚠️ #workout (35/100)
  ⚠️ #fitnessmotivation (32/100)
  ⚠️ #gymlife (28/100)

🔗 RELATED HASHTAGS TO EXPLORE:
─────────────────────────────────────────────────────────
#bodytransformation  #fitover40  #kettlebellworkout  #dumbbellworkout  #functionalfitness  #calisthenics  #noexcuses  #progressnotperfection

═══════════════════════════════════════════════════════════
🏆 YOUR OPTIMIZED HASHTAG SET
═══════════════════════════════════════════════════════════

Copy this for your next post:

#homeworkout #strengthtraining #fitnesstips #personaltrainer #fitnessjourney #fitfam #gains #bodytransformation #kettlebellworkout

What You Just Built

Hashtag tools are expensive:

Flick: $14/month
HashtagsForLikes: $25/month
RiteTag: $49/month
Keyhole: $79/month

Your version analyzes REAL engagement data for cents per search.

Pro Tips

Refresh monthly: Hashtag performance changes
Mix tiers: Don't use all high-competition tags
Platform-specific: TikTok ≠ Instagram hashtag strategies
Track performance: Note which tags drive YOUR engagement

Get Started

Get your SociaVault API Key
Run the tool for your niche
Use the optimized hashtag set

Stop copying competitors. Start finding untapped hashtags.

The right hashtag finds the right audience.

DEV Community