Building an AI-Powered Social Listening Platform with Next.js 15 and Cloudflare Workers

How I solved the "what should I build next?" problem for 10,000+ developers using modern web technologies

---

The Developer's Eternal Question

We've all been there. 3 AM, empty project folder, cursor blinking in the terminal, asking ourselves: "What should I build next?"

As developers, we're great at the "how" but often struggle with the "what." I spent months building projects that nobody wanted, analyzing competitors until analysis paralysis set in, and browsing endless "project ideas" lists that felt disconnected from real user needs.

That frustration led me to build builtwhat.app - a platform that uses AI to turn social media complaints into actionable product opportunities for developers.

Here's how I built it, the technical challenges I faced, and what I learned along the way.

The Core Idea

Instead of hunting for ideas, what if ideas could find us? Social platforms are goldmines of user frustrations:

• "Why doesn't GitHub have feature X?"
• "I hate using tool Y for workflow Z"
• "Am I the only one who thinks app A is overengineered?"

These complaints represent unmet needs - potential product opportunities. The challenge was building a system to systematically discover, analyze, and curate them.

Tech Stack Overview

graph TD
    A[Social Media APIs] --> B[Data Collection Worker]
    B --> C[AI Analysis Pipeline]
    C --> D[Content Generation]
    D --> E[Cloudflare KV Cache]
    E --> F[Next.js Frontend]
    F --> G[Users]

    H[Cloudflare Workers] --> I[API Endpoints]
    I --> F

Frontend: Next.js 15 + React 19 + TypeScript

Backend: Cloudflare Workers + Hono framework

Database: PostgreSQL + Redis

Caching: Cloudflare KV

AI: OpenAI GPT-4 + custom sentiment analysis

Hosting: Cloudflare Pages

Why This Stack?

Next.js 15 + React 19

{
  "next": "15.3.3",
  "react": "19.0.0",
  "react-dom": "19.0.0"
}

React 19's concurrent features improved user experience significantly, especially for the daily inspiration loading states. Next.js 15's app router made SEO optimization much easier.

Cloudflare Workers + Hono

import { Hono } from 'hono'
import { cors } from 'hono/cors'

const app = new Hono()

app.use('*', cors({
  origin: ['https://builtwhat.app'],
  allowMethods: ['GET', 'POST'],
}))

app.get('/api/inspiration/daily', async (c) => {
  const cached = await c.env.KV.get('daily_inspiration')
  if (cached) {
    return c.json(JSON.parse(cached))
  }

  const fresh = await generateDailyInspiration()
  await c.env.KV.put('daily_inspiration', JSON.stringify(fresh), {
    expirationTtl: 86400 // 24 hours
  })

  return c.json(fresh)
})

export default app

Cloudflare Workers solved our global latency problem. With users in both US and China, edge computing was essential.

The AI Pipeline

1. Data Collection

class SocialDataCollector {
  async collectComplaints() {
    const sources = [
      this.collectFromReddit(),
      this.collectFromTwitter(),
      this.collectFromHackerNews()
    ]

    const results = await Promise.allSettled(sources)
    return results
      .filter(result => result.status === 'fulfilled')
      .flatMap(result => result.value)
  }

  async collectFromReddit() {
    // Rate-limited API calls with exponential backoff
    const subreddits = ['webdev', 'programming', 'startups']
    const complaints = []

    for (const sub of subreddits) {
      try {
        const posts = await this.fetchWithRetry(
          `https://reddit.com/r/${sub}/hot.json?limit=25`
        )

        const filtered = posts.data.children
          .filter(post => this.containsComplaint(post.data.title))
          .map(post => ({
            content: post.data.title,
            score: post.data.score,
            comments: post.data.num_comments,
            source: 'reddit'
          }))

        complaints.push(...filtered)
      } catch (error) {
        console.error(`Failed to fetch r/${sub}:`, error)
      }
    }

    return complaints
  }
}

2. AI Analysis

class AIAnalyzer {
  async analyzeComplaints(complaints) {
    const prompt = `
    Analyze these user complaints and identify product opportunities:

    ${complaints.slice(0, 10).map(c => `- ${c.content}`).join('\n')}

    For each opportunity, provide:
    1. Problem statement
    2. Proposed solution
    3. Target audience
    4. Technical feasibility (1-10)
    5. Market potential (1-10)

    Return as JSON array.
    `

    const response = await fetch('https://api.openai.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: 'gpt-4',
        messages: [{ role: 'user', content: prompt }],
        temperature: 0.7,
      })
    })

    return await response.json()
  }
}

3. Quality Scoring

class QualityAssessment {
  scoreOpportunity(opportunity) {
    let score = 0

    // Problem clarity (0-25 points)
    score += this.assessProblemClarity(opportunity.problem)

    // Solution feasibility (0-25 points) 
    score += this.assessSolutionFeasibility(opportunity.solution)

    // Market demand (0-25 points)
    score += this.assessMarketDemand(opportunity.audience)

    // Technical implementation (0-25 points)
    score += this.assessTechnicalFeasibility(opportunity.technical_feasibility)

    return {
      score,
      shouldPublish: score >= 70,
      categories: this.categorizeOpportunity(opportunity)
    }
  }
}

Frontend Implementation

Daily Inspiration Component

'use client'

import { useState, useEffect } from 'react'
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'

interface Inspiration {
  id: string
  title: "string"
  problem: string
  solution: string
  difficulty: 'Easy' | 'Medium' | 'Hard'
  tags: string[]
  marketPotential: number
}

export function DailyInspiration() {
  const [inspiration, setInspiration] = useState<Inspiration | null>(null)
  const [isLoading, setIsLoading] = useState(true)

  useEffect(() => {
    fetchDailyInspiration()
  }, [])

  const fetchDailyInspiration = async () => {
    try {
      const response = await fetch('/api/inspiration/daily')
      if (!response.ok) throw new Error('Failed to fetch')

      const data = await response.json()
      setInspiration(data)
    } catch (error) {
      console.error('Error fetching inspiration:', error)
    } finally {
      setIsLoading(false)
    }
  }

  if (isLoading) {
    return (
      <div className="flex items-center justify-center h-64">
        <div className="animate-spin rounded-full h-8 w-8 border-b-2 border-blue-600" />
      </div>
    )
  }

  if (!inspiration) {
    return <div className="text-center text-gray-500">No inspiration available</div>
  }

  return (
    <Card className="max-w-2xl mx-auto">
      <CardHeader>
        <CardTitle className="flex items-center justify-between">
          {inspiration.title}
          <span className={`px-2 py-1 rounded text-sm ${
            inspiration.difficulty === 'Easy' ? 'bg-green-100 text-green-800' :
            inspiration.difficulty === 'Medium' ? 'bg-yellow-100 text-yellow-800' :
            'bg-red-100 text-red-800'
          }`}>
            {inspiration.difficulty}
          </span>
        </CardTitle>
      </CardHeader>

      <CardContent className="space-y-4">
        <div>
          <h3 className="font-semibold text-red-600 mb-2">Problem</h3>
          <p className="text-gray-700">{inspiration.problem}</p>
        </div>

        <div>
          <h3 className="font-semibold text-blue-600 mb-2">Proposed Solution</h3>
          <p className="text-gray-700">{inspiration.solution}</p>
        </div>

        <div className="flex items-center justify-between">
          <div className="flex gap-2">
            {inspiration.tags.map(tag => (
              <span key={tag} className="px-2 py-1 bg-gray-100 rounded-full text-sm">
                {tag}
              </span>
            ))}
          </div>

          <div className="text-sm text-gray-600">
            Market Potential: {inspiration.marketPotential}/10
          </div>
        </div>
      </CardContent>
    </Card>
  )
}

Performance Optimizations

Multi-Layer Caching Strategy

class CacheManager {
  constructor() {
    this.memoryCache = new Map()
    this.TTL = 24 * 60 * 60 * 1000 // 24 hours
  }

  async get(key) {
    // Layer 1: Memory cache
    if (this.memoryCache.has(key)) {
      const { data, timestamp } = this.memoryCache.get(key)
      if (Date.now() - timestamp < this.TTL) {
        return data
      }
      this.memoryCache.delete(key)
    }

    // Layer 2: Cloudflare KV
    const kvData = await KV.get(key)
    if (kvData) {
      const data = JSON.parse(kvData)
      this.memoryCache.set(key, { data, timestamp: Date.now() })
      return data
    }

    return null
  }

  async set(key, data) {
    // Store in both layers
    this.memoryCache.set(key, { data, timestamp: Date.now() })
    await KV.put(key, JSON.stringify(data), { 
      expirationTtl: this.TTL / 1000 
    })
  }
}

Rate Limiting & Resilience

class RateLimitedFetcher {
  constructor() {
    this.queue = []
    this.processing = false
    this.retryDelays = [1000, 2000, 5000, 10000]
  }

  async fetchWithRetry(url, options = {}, maxRetries = 4) {
    for (let attempt = 0; attempt < maxRetries; attempt++) {
      try {
        const response = await fetch(url, options)

        if (response.status === 429) {
          // Rate limited - wait and retry
          await this.delay(this.retryDelays[attempt] || 10000)
          continue
        }

        if (!response.ok) {
          throw new Error(`HTTP ${response.status}`)
        }

        return response
      } catch (error) {
        if (attempt === maxRetries - 1) {
          throw error
        }
        await this.delay(this.retryDelays[attempt])
      }
    }
  }

  delay(ms) {
    return new Promise(resolve => setTimeout(resolve, ms))
  }
}

Challenges & Solutions

1. API Rate Limits

Problem: Social platforms heavily rate-limit API access.

Solution:

• Distributed collection across multiple time windows
• Intelligent backoff strategies
• Multiple data sources for redundancy

2. AI Accuracy

Problem: GPT-4 sometimes generated irrelevant or low-quality opportunities.

Solution:

• Multi-stage filtering pipeline
• Human review for edge cases
• Continuous prompt engineering

3. Global Performance

Problem: Users in China experiencing slow load times.

Solution:

• Cloudflare's global edge network
• Aggressive caching at multiple layers
• Optimized asset delivery

Results After 6 Months

Growth:

• 12,000+ registered users
• 10,000+ email subscribers
• 2,000+ daily active users
• 500+ community members

Technical Performance:

• 98.5% uptime
• <200ms average response time globally
• 0.02% error rate
• $47/month infrastructure costs

User Engagement:

• 3.2 minutes average session time
• 15% 30-day retention rate
• 4.6/5 user satisfaction score

Lessons Learned

Technical

1. Edge computing is a game-changer for global applications
2. Caching strategy is crucial for AI-powered apps
3. TypeScript saves time in the long run, especially for data processing
4. Monitor everything - performance issues compound quickly

Product

1. Real user problems beat clever ideas every time
2. Community input improves quality dramatically
3. Daily habit formation is powerful for engagement
4. International users bring unexpected perspectives

What's Next

Technical Roadmap

• Migrate to Cloudflare D1 for simpler data management
• Implement real-time collaboration features
• Add personalized recommendation engine
• Open-source core analysis pipeline

Product Evolution

• AI market validation for user-submitted ideas
• Project collaboration spaces
• Success story tracking
• Enterprise solutions

Key Takeaways for Fellow Developers

1. Start with a problem you personally face - authenticity shows
2. Modern tools make global deployment easier than ever
3. AI amplifies good ideas but can't replace product intuition
4. Community feedback is invaluable for iteration
5. Performance matters more than perfect features

Want to Try It?

If you're curious about the platform or facing similar "what to build" challenges, check out builtwhat.app. It's completely free, and I'd love to get feedback from the dev.to community!

Also happy to answer any technical questions about the implementation - always learning from fellow developers.

---

What's your biggest challenge when deciding what to build next? Drop a comment below! 👇

Follow me for more posts about building in public and AI-powered development tools.