<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: jackmeng</title>
    <description>The latest articles on DEV Community by jackmeng (@jackmeng).</description>
    <link>https://dev.to/jackmeng</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4252348%2F1955c52a-d565-4939-9467-31dbdc267391.png</url>
      <title>DEV Community: jackmeng</title>
      <link>https://dev.to/jackmeng</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jackmeng"/>
    <language>en</language>
    <item>
      <title>Stop Recommending Songs. Start Understanding Intent.</title>
      <dc:creator>jackmeng</dc:creator>
      <pubDate>Wed, 18 Nov 2026 17:32:57 +0000</pubDate>
      <link>https://dev.to/jackmeng/stop-recommending-songs-start-understanding-intent-3f8f</link>
      <guid>https://dev.to/jackmeng/stop-recommending-songs-start-understanding-intent-3f8f</guid>
      <description>&lt;p&gt;Why the next generation of music recommendation systems is multimodal, agentic, and no longer “click-prediction”&lt;/p&gt;

&lt;p&gt;By Jack Meng · AI Systems &amp;amp; Generative Audio Engineer&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The old music recommender is dead&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For a decade, music recommendation looked like this:&lt;/p&gt;

&lt;p&gt;• Log user plays&lt;/p&gt;

&lt;p&gt;• Train a two-tower model&lt;/p&gt;

&lt;p&gt;• Retrieve similar songs&lt;/p&gt;

&lt;p&gt;• Sort by CTR&lt;/p&gt;

&lt;p&gt;• Call it “personalization”&lt;/p&gt;

&lt;p&gt;It worked. But it was shallow.&lt;/p&gt;

&lt;p&gt;Because users never actually say:&lt;/p&gt;

&lt;p&gt;“Recommend me three tracks with a 124 BPM groove, minor-key melancholy, 2010s R&amp;amp;B texture, and a little tape saturation.”&lt;/p&gt;

&lt;p&gt;They say:&lt;/p&gt;

&lt;p&gt;“Something for late-night driving. Not too sad. A bit nostalgic.”&lt;/p&gt;

&lt;p&gt;That sentence has no tag, no genre, no clean label.&lt;br&gt;&lt;br&gt;
And traditional recommender systems choke on it.&lt;/p&gt;

&lt;p&gt;The future of music discovery is not a better classifier.&lt;br&gt;&lt;br&gt;
It’s a system that can hear, read, reason, and generate.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The architecture: Retrieve → Understand → Rerank → Explain&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A modern music recommendation system is no longer one model.&lt;br&gt;&lt;br&gt;
It’s a pipeline.&lt;br&gt;
User Intent&lt;br&gt;
   │&lt;br&gt;
   ▼&lt;br&gt;
┌──────────────────────────────┐&lt;br&gt;
│ 1. Query Understanding (LLM) │  ← rewrite vague prompts&lt;br&gt;
└──────────────┬───────────────┘&lt;br&gt;
               ▼&lt;br&gt;
┌────────────────────────────────────────────┐&lt;br&gt;
│ 2. Multi-Modal Retrieval                    │&lt;br&gt;
│  - Collaborative embeddings (BPR / Two-Tower)&lt;br&gt;
│  - Text embeddings (lyrics, tags, mood)&lt;br&gt;
│  - Audio embeddings (CLAP / VGGish)&lt;br&gt;
│  - Visual embeddings (album art)&lt;br&gt;
│  - Lexical search (BM25)&lt;br&gt;
└──────────────┬─────────────────────────────┘&lt;br&gt;
               ▼&lt;br&gt;
┌────────────────────────────────────────────┐&lt;br&gt;
│ 3. Fusion &amp;amp; Candidate Pooling (RRF / Router)│&lt;br&gt;
└──────────────┬─────────────────────────────┘&lt;br&gt;
               ▼&lt;br&gt;
┌────────────────────────────────────────────┐&lt;br&gt;
│ 4. Reranking                                │&lt;br&gt;
│  - Transformer / LambdaMART&lt;br&gt;
│  - Diversity constraints&lt;br&gt;
│  - Skip-rate penalty&lt;br&gt;
│  - Session-level coherence&lt;br&gt;
└──────────────┬─────────────────────────────┘&lt;br&gt;
               ▼&lt;br&gt;
┌────────────────────────────────────────────┐&lt;br&gt;
│ 5. Agentic Explainer (LLM)                  │&lt;br&gt;
│  “Because you liked X, but want calmer now”&lt;br&gt;
└────────────────────────────────────────────┘&lt;/p&gt;

&lt;p&gt;This is not theoretical.&lt;br&gt;&lt;br&gt;
This is basically what top RecSys 2026 teams converged on. &lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Step 1: Don’t trust the raw query&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Users are vague. The LLM’s job is not to generate music—it’s to translate feeling into structure.&lt;/p&gt;

&lt;p&gt;Example:&lt;/p&gt;

&lt;p&gt;“I want rain, neon, and a heartbeat”&lt;/p&gt;

&lt;p&gt;The intent parser outputs:&lt;br&gt;
{&lt;br&gt;
  "mood": ["melancholic", "cinematic"],&lt;br&gt;
  "tempo_range": [80, 110],&lt;br&gt;
  "instruments": ["synth", "pads", "soft drums"],&lt;br&gt;
  "avoid": ["heavy metal", "hyperpop"],&lt;br&gt;
  "context": "late night, focus, urban"&lt;br&gt;
}&lt;/p&gt;

&lt;p&gt;Now we have something a retrieval system can use.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Step 2: Multi-modal embeddings are non-negotiable&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A song is not a row in a database.&lt;br&gt;&lt;br&gt;
It’s:&lt;/p&gt;

&lt;p&gt;• sound&lt;/p&gt;

&lt;p&gt;• language&lt;/p&gt;

&lt;p&gt;• image&lt;/p&gt;

&lt;p&gt;• culture&lt;/p&gt;

&lt;p&gt;• memory&lt;/p&gt;

&lt;p&gt;So we encode it multiple ways.&lt;/p&gt;

&lt;p&gt;Text tower&lt;/p&gt;

&lt;p&gt;• Lyrics&lt;/p&gt;

&lt;p&gt;• Tags&lt;/p&gt;

&lt;p&gt;• Editorial descriptions&lt;/p&gt;

&lt;p&gt;• User-generated mood words&lt;/p&gt;

&lt;p&gt;Model: Qwen3-Embedding, BGE-M3&lt;/p&gt;

&lt;p&gt;Audio tower&lt;/p&gt;

&lt;p&gt;• Mel-spectrogram&lt;/p&gt;

&lt;p&gt;• Chroma&lt;/p&gt;

&lt;p&gt;• Tempo / key / timbre&lt;/p&gt;

&lt;p&gt;Model: CLAP, VGGish, fine-tuned HTS-AT&lt;/p&gt;

&lt;p&gt;Collaborative tower&lt;/p&gt;

&lt;p&gt;• User–item interaction graph&lt;/p&gt;

&lt;p&gt;• BPR loss&lt;/p&gt;

&lt;p&gt;• Artist / session co-occurrence&lt;/p&gt;

&lt;p&gt;Visual tower&lt;/p&gt;

&lt;p&gt;• Album cover semantics&lt;/p&gt;

&lt;p&gt;• Color palette / aesthetic clustering&lt;/p&gt;

&lt;p&gt;Then we store all of them in vector indexes:&lt;br&gt;
FAISS / Milvus&lt;br&gt;
├── text_vectors&lt;br&gt;
├── audio_vectors&lt;br&gt;
├── collab_vectors&lt;br&gt;
└── fused_vectors&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Step 3: Hybrid retrieval, not “one vector to rule them all”&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Pure vector search lies.&lt;br&gt;&lt;br&gt;
Pure collaborative filtering forgets new music.&lt;br&gt;&lt;br&gt;
Pure metadata search can’t feel vibe.&lt;/p&gt;

&lt;p&gt;So we do heterogeneous retrieval:&lt;br&gt;
candidates = set()&lt;/p&gt;

&lt;p&gt;candidates |= bm25_search(query_text, k=200)&lt;br&gt;
candidates |= faiss_search(text_vec, index="text", k=200)&lt;br&gt;
candidates |= faiss_search(audio_vec, index="audio", k=200)&lt;br&gt;
candidates |= collab_neighbors(user_id, k=200)&lt;br&gt;
candidates |= artist_expansion(query)&lt;/p&gt;

&lt;p&gt;Then merge with Weighted Reciprocal Rank Fusion:&lt;br&gt;
def rrf(lists, k=60):&lt;br&gt;
    scores = {}&lt;br&gt;
    for lst in lists:&lt;br&gt;
        for rank, item in enumerate(lst):&lt;br&gt;
            scores[item] = scores.get(item, 0) + 1 / (k + rank)&lt;br&gt;
    return sorted(scores.items(), key=lambda x: -x[1])&lt;/p&gt;

&lt;p&gt;Simple. Robust. Beats most “fancy” end-to-end models in production.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Step 4: Reranking is where the money is&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Retrieval gets you “plausible”.&lt;br&gt;&lt;br&gt;
Reranking gets you “right”.&lt;/p&gt;

&lt;p&gt;A real reranker looks at:&lt;/p&gt;

&lt;p&gt;• click probability&lt;/p&gt;

&lt;p&gt;• skip probability&lt;/p&gt;

&lt;p&gt;• completion rate&lt;/p&gt;

&lt;p&gt;• genre balance&lt;/p&gt;

&lt;p&gt;• artist repetition penalty&lt;/p&gt;

&lt;p&gt;• time-of-day fit&lt;/p&gt;

&lt;p&gt;• novelty vs familiarity tradeoff&lt;/p&gt;

&lt;p&gt;Sketch:&lt;br&gt;
class Reranker(nn.Module):&lt;br&gt;
    def forward(self, seq_features, item_features, context):&lt;br&gt;
        x = torch.cat([seq_features, item_features, context], dim=-1)&lt;br&gt;
        score = self.mlp(x)&lt;br&gt;
        return score  # P(listen &amp;gt; 30s)&lt;/p&gt;

&lt;p&gt;Then apply business rules:&lt;br&gt;
if same_artist_in_last_3:&lt;br&gt;
    score *= 0.6&lt;/p&gt;

&lt;p&gt;if item_novelty_high and user_explorer:&lt;br&gt;
    score *= 1.15&lt;/p&gt;

&lt;p&gt;Precision@10 jumps. Fatigue drops.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Step 5: Make the system explain itself&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A recommendation without explanation is a black box.&lt;/p&gt;

&lt;p&gt;Modern UX needs:&lt;/p&gt;

&lt;p&gt;“Because you listened to Lowlight last week, but tonight your prompt sounds calmer—so I pulled ambient R&amp;amp;B with softer attack.”&lt;/p&gt;

&lt;p&gt;That’s an LLM reranker + constrained generation, not free-form hallucination.&lt;/p&gt;

&lt;p&gt;Rules:&lt;/p&gt;

&lt;p&gt;• Only talk about retrieved items&lt;/p&gt;

&lt;p&gt;• No fake artist names&lt;/p&gt;

&lt;p&gt;• No invented lyrics&lt;/p&gt;

&lt;p&gt;• Always ground explanation in metadata&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Cold start? Solve it with content, not hacks&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;New song. Zero plays.&lt;br&gt;&lt;br&gt;
Old systems panic.&lt;/p&gt;

&lt;p&gt;New system says:&lt;/p&gt;

&lt;p&gt;“I don’t know who listens yet, but acoustically this sits between Bonobo and Tycho, lyrically it’s abstract, visually it’s dusk-toned.”&lt;/p&gt;

&lt;p&gt;So we bootstrap with:&lt;/p&gt;

&lt;p&gt;• audio embedding&lt;/p&gt;

&lt;p&gt;• lyric embedding&lt;/p&gt;

&lt;p&gt;• mood taxonomy&lt;/p&gt;

&lt;p&gt;• creator-provided brief&lt;/p&gt;

&lt;p&gt;Cold start becomes a content understanding problem, not a statistics problem.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Engineering reality: the stack I’d ship&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If I were building this today:&lt;br&gt;
Backend API        : Go (low latency, concurrency)&lt;br&gt;
Realtime Gateway   : WebSocket + gRPC&lt;br&gt;
ML Inference       : Python + PyTorch + ONNX&lt;br&gt;
Embeddings         : Qwen3 / CLAP / BGE-M3&lt;br&gt;
Vector DB          : Milvus or FAISS&lt;br&gt;
Cache              : Redis (session vectors)&lt;br&gt;
Orchestration      : LangGraph / custom agent loop&lt;br&gt;
Frontend           : Vue 3 + Web Audio preview&lt;/p&gt;

&lt;p&gt;Target latency:&lt;/p&gt;

&lt;p&gt;• p95 retrieval: &amp;lt; 40ms&lt;/p&gt;

&lt;p&gt;• rerank: &amp;lt; 25ms&lt;/p&gt;

&lt;p&gt;• generation/explanation: &amp;lt; 300ms streaming&lt;/p&gt;

&lt;p&gt;That’s production-grade, not demo-grade.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The bigger shift: from recommender to sonic agent&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We’re moving from:&lt;/p&gt;

&lt;p&gt;“What song should I play next?”&lt;/p&gt;

&lt;p&gt;to:&lt;/p&gt;

&lt;p&gt;“Build me a 40-minute sonic environment.”&lt;/p&gt;

&lt;p&gt;That means the system can:&lt;/p&gt;

&lt;p&gt;• generate missing tracks&lt;/p&gt;

&lt;p&gt;• remix existing ones&lt;/p&gt;

&lt;p&gt;• adapt BPM to heart rate&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;shift mood mid-session
• respect brand, scene, and emotion&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Recommendation is no longer search.&lt;br&gt;&lt;br&gt;
It’s intent-driven audio synthesis with guardrails.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Lessons I’ve learned shipping AI music systems&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Don’t fine-tune first. Architect first.&lt;br&gt;&lt;br&gt;
Most teams waste months on models before defining the retrieval boundary.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;LLMs are planners, not oracles.&lt;br&gt;&lt;br&gt;
Let the LLM call tools. Don’t let it invent songs.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Diversity &amp;gt; perfection.&lt;br&gt;&lt;br&gt;
A slightly imperfect but varied playlist beats 10 near-duplicates.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Implicit feedback lies less than you think.&lt;br&gt;&lt;br&gt;
Skips, replays, and abandon-time are gold.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Generative + retrieval hybrid is the endgame.&lt;br&gt;&lt;br&gt;
Pure generation hallucinates. Pure retrieval stagnates.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Final thought&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The next Spotify-killer won’t be a better playlist algorithm.&lt;/p&gt;

&lt;p&gt;It’ll be a system that understands:&lt;/p&gt;

&lt;p&gt;• what you say&lt;/p&gt;

&lt;p&gt;• what you feel&lt;/p&gt;

&lt;p&gt;• what you heard yesterday&lt;/p&gt;

&lt;p&gt;• what you didn’t know you wanted tonight&lt;/p&gt;

&lt;p&gt;Music recommendation is no longer about predicting the next click.&lt;br&gt;&lt;br&gt;
It’s about modeling human atmosphere.&lt;/p&gt;

&lt;p&gt;And that’s a much harder—and much more beautiful—problem.&lt;/p&gt;

&lt;p&gt;If you build AI music, generative audio, or recommendation infra:&lt;/p&gt;

&lt;p&gt;I write about real-world architecture for multimodal ML, agentic systems, and production-grade generative audio.&lt;br&gt;&lt;br&gt;
No hype. Just pipes, vectors, and signal.&lt;/p&gt;

&lt;p&gt;Tags: #MusicAI #RecSys #MultimodalML #LLM #AgenticSystems #PyTorch #VectorSearch #GenerativeAudio&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>machinelearning</category>
    </item>
  </channel>
</rss>
