DEV Community: Kate

My Spotify playlist takes pull requests

Kate — Sat, 17 Oct 2026 18:44:18 +0000

What if a playlist worked like a repository?

You'd propose a song with a pull request. CI would check that it's a real track and not a duplicate. Review would mean listening to it. Merging would publish it. And git log would tell you who added every song, and why.

So I built that. It's two repos:

🎧 Open Source Mixtape: a public Spotify playlist that anyone with a GitHub account can add to. Listen on Spotify.
🛠 spotify-playlist-as-code: the template it runs on. Click Use this template and your own playlist takes PRs too.

No server, no database, no dependencies beyond the Python standard library. Just a text file, GitHub Actions, and the Spotify Web API.

The whole playlist is one text file

That's it. One line per song, in playlist order. The link can be a spotify:track: URI, an open.spotify.com link (with or without ?si=… or /intl-ja/), or a bare 22-character ID. The # Artist - Title after it is for humans, so reviewers know what they're about to hear.

The # @playlist line says which Spotify playlist the file syncs to. A file without it still gets validated, it just never touches Spotify, which is handy for trying things out.

From a pasted link to a live playlist

Asking strangers to fork a repo and edit a file to suggest a song is a lot. So the front door is an issue form that asks for a single thing: a Spotify link.

Someone opens a "🎵 Add a song" issue and pastes a link. "Why this one?" and "Listening from" are optional.
A workflow turns the issue into a PR. It looks the song up on Spotify, appends <link> # Artist - Title to the file, and comments back on the issue with the PR link. If the link is an album, or the song is already in the playlist, it says so on the issue and closes it.
CI validates the PR. Malformed IDs and duplicates fail the check with an annotation on the exact line. Every added track gets looked up, and the job summary reads like a tracklist: ➕ KiiiKiii - Pop Off Pop Off.
Review is listening. I play the song, then merge, comment, or politely pass.
Merging is publishing. A push to main that touches playlists/ runs the sync, and the Spotify playlist now matches the file.

Five things I learned building it

1. Make the sync declarative

The sync doesn't figure out "add this, remove that". It replaces the playlist with whatever the file says:

def replace_playlist(self, playlist_id: str, uris: list[str]) -> None:
    """Make the playlist contain exactly `uris`, in order."""
    first, rest = uris[:BATCH], uris[BATCH:]
    self.request("PUT", f"/playlists/{playlist_id}/items", {"uris": first})
    for i in range(0, len(rest), BATCH):
        self.request("POST", f"/playlists/{playlist_id}/items", {"uris": rest[i : i + BATCH]})

PUT sets the first 100 tracks and POST appends the rest in batches of 100. Reorders, removals and reverts all come for free, and a failed run is fixed by just running it again. The trade-off is in the README in a big warning box: anything added straight in the Spotify app disappears on the next sync. The file wins. That's the point.

(If you've used the Spotify API before: these are the /items endpoints from the February 2026 Web API changes. The old /tracks paths are gone.)

2. PRs opened by `GITHUB_TOKEN` don't trigger your other workflows

This one cost me a while. The bot opens its PR with the built-in GITHUB_TOKEN, and GitHub deliberately doesn't start new workflow runs from events caused by that token, so on: pull_request never fired and the bot's PRs had no check at all.

The fix is to give the validate workflow a workflow_dispatch trigger and have the bot start it by hand:

url=$(gh pr create --base "$DEFAULT_BRANCH" --head "$branch" \
  --title "Add $LABEL" --body-file "$PR_BODY_FILE")
# Pull requests opened with GITHUB_TOKEN don't trigger other workflows, so start the check here.
gh workflow run validate.yml --ref "$branch"

You also need Settings → Actions → General → Allow GitHub Actions to create and approve pull requests, or gh pr create fails.

3. Give the human credit, not the bot

The commit is made by github-actions[bot], but the song came from whoever opened the issue. So the commit message carries a co-author trailer:

git commit -m "Add $LABEL" -m "Suggested in #$NUMBER." \
  -m "Co-authored-by: $AUTHOR <$AUTHOR_ID+$AUTHOR@users.noreply.github.com>"

With the ID+login@users.noreply.github.com form, GitHub links the commit to their profile. Adding a song to a playlist shows up as a real contribution.

4. Git history makes great liner notes

I didn't build any of this. git log -p playlists/ already answers "who added this song, when, and why", and git revert undoes a bad pick. Version control turns out to be a pretty good music library.

5. Plan for forks without secrets

Pull requests from forks don't get repository secrets, so CI can't call Spotify for them. Rather than failing, the validator skips the online lookup and still checks the format and duplicates. The sync does the same: no secrets, so it runs as a dry run. A fresh copy of the template is green on day one before you've set anything up.

Make your own

Want this for your team's office playlist, a party queue, or a road trip with friends?

Click Use this template on spotify-playlist-as-code.
Create a Spotify app, get a refresh token with python scripts/get_refresh_token.py, and add three repository secrets.
Put your playlist link in a # @playlist line and merge.

The full steps are in the README. One heads-up: Spotify's development mode needs the app owner to have Premium. People adding songs don't need it.

Add a song to the mixtape 🎶

The mixtape is small right now, so your pick will stand out. Any genre, any language, any decade.

👉 Open an "Add a song" issue and paste a Spotify link. The bot does the rest.

Then tell me in the comments what you added and why. And if you have an idea for the tool itself, issues and PRs are welcome on spotify-playlist-as-code. ⭐ Stars help other people find it.

Why my from-scratch Japanese LLM says "That sounds tough" to good news

Kate — Thu, 08 Oct 2026 16:12:18 +0000

TL;DR

Lilas is a ~40M-parameter Japanese chat model I built from scratch, tokenizer included. It kept answering cheerful remarks with "That sounds tough, please don't overdo it." A raw count of the training data pointed at one over-used phrase. Weighting the counts by how the data is actually mixed at training time changed the picture. The real gap is that the plain positive reply, "I'm glad to hear that" (それは良かった), is the rarest positive reaction the model sees, 3–5x rarer than the other positive replies and 4x rarer than "That sounds tough."

This post covers the diagnosis and the fix plan. The retraining has not been run yet, so there are no after-numbers.

Code and data: github.com/keito118/lilas-llm-from-scratch
Me: github.com/keito118

Background: what Lilas is

Lilas is a hobby project to build a Japanese conversational AI without any pretrained model. Everything runs on one consumer GPU (RTX 3060, 12 GB).

Part	Details
Model	GPT-style decoder, ~40M parameters (n_embd=512, n_layer=10, n_head=8, block_size=1024)
Tokenizer	Own byte-level BPE, vocab 16,000
Pretraining	Aozora Bunko + Wikipedia excerpts + public instruction data, ~960M characters (~610M tokens)
Fine-tuning	`User:` / `Lilas:` dialogue files in `index/`
Tools	Calculator, date, holidays, weather and Wikipedia lookups run in code and are injected into the prompt as reference text (RAG)

One detail matters for this post. The hand-made persona and dialogue files are tiny next to the public instruction data, so lilas/data.py copies them persona_repeat times into the training mix. The production model ("round2") used persona_repeat=21. Without this, the model forgot its own name.

The production model was picked after eight rounds of experiments in one day, logged in SCALE_UP_PLAN.md. The common lesson of those rounds: adding data helps only near the training examples and rarely generalizes to new phrasings.

The symptom

A typical failure:

User:  今日は天気がいいから洗濯物がよく乾きそう
       (The weather's nice today, so the laundry should dry well.)
Lilas: それは大変そうですね、無理なさらないでください
       (That sounds tough. Please don't overdo it.)

It was not a one-off. The held-out evaluations had been showing the same pattern:

Emotion-matching accuracy rose from 30–40% to 66.7%, but about a third still failed. The most common failure was a flipped polarity: a cheerful reply to bad news, or the reverse.
Adding more emotion data (round 8) did not help with close paraphrases. Training on "They sent my draft back for a rewrite" did nothing for "They told me to rewrite it again." Asked "Looking forward to the party?", the model started talking about curry.

So instead of adding more data again, I counted what was already there.

First pass: raw counts

The first pass took three steps and no training at all.

Find where the stock phrases come from. Grepping the data-generation scripts for common reaction phrases turned up five generators (emotion, greetings, long-context, arithmetic and seed dialogues) that all reuse the same handful of replies.
Count them. grep -o <phrase> index/*.txt | wc -l for each phrase over the full local training set (20 files, 68,955 Lilas: lines).
Check how the files are mixed. Reading lilas/data.py confirmed which files are copied persona_repeat times.

The headline number: "That sounds tough" (それは大変) appeared 364 times, while "I'm glad to hear that" (それは良かった) appeared only 25 times, about 15x fewer.

The first hypothesis followed directly. "That sounds tough" is the model's default escape hatch, and persona_repeat multiplies it further. That turned out to be half right.

Second pass: count what the model actually sees

Raw file counts are not what the model trains on, so I redid the count with each file multiplied by its real factor. The result overturned half of the first hypothesis.

The public repo holds only the curated dialogue files (16 files, 3,971 Lilas: lines). The large public instruction files stay local. Of the 364 "That sounds tough" hits, only 132 are in the curated files. 102 of those 132 come from a single template generator (generated_longcontext_ja.txt). The other 232 sit in the large instruction files, which are not upsampled.

Raw: grep over the full local training set (20 files). Weighted: repo files × their factor in lilas/data.py, plus the 232 non-repo occurrences ×1.

After weighting, "That sounds tough" drops from 31% of the reaction phrases to 17% and is no longer the most common. "I'm glad to hear that" stays the rarest positive reply. The upsampling did amplify something, just not the phrase I first suspected.

Revised hypothesis

The problem is not that one negative phrase dominates. It is that the model has almost no plain way to say "good for you."

The small-good-news reply is missing. "I'm looking forward to it" belongs to future plans. "That's wonderful" belongs to achievements. "I'm glad to hear that" fits everyday small wins like dry laundry, and it is the rarest positive reply in the training mix.
"That sounds tough" is a fixed two-sentence chunk. 43 of the 132 Lilas: lines containing it also say "please don't overdo it" (無理なさらないで). The failing reply is exactly that chunk.
The templates attach it to anything that isn't clearly happy. In the long-context generator, 65 of the 102 "tough" replies follow just two user lines (spilling coffee, a late train). The other 37 are the same "sounds tough, don't overdo it" line attached to mostly neutral remarks such as "I took my watch in for repair" or "I'm thinking of getting a massage." A small model can learn "anything that isn't clearly happy gets sympathy" without learning polarity at all.
When polarity is unclear, the model falls back to a high-frequency chunk. With no generic positive acknowledgement to fall back on, the negative chunk wins. This step is still a guess, and the retraining is where it gets tested.

This also explains why round 8 failed. More emotion examples written in the same style reproduce the same gap.

The plan

Five steps, roughly 6–9 hours. The order comes from what failed in earlier rounds.

Inventory conversational moves (0.5–1 h). A script that counts reactions, back-channel replies, topic changes, clarifying questions and follow-up questions per file, so gaps are visible as numbers.
Add polarity-matching data (2–3 h). Examples where the reply depends on whether the input is positive, negative or neutral, with "I'm glad to hear that" filled in across many topics. Neutral remarks get neutral replies, not sympathy.
Fill other thin conversational skills (1–2 h) found in step 1.
Add a polarity-mismatch check to the eval (1 h). Flag a negative reaction to a positive input automatically, not just by eye.
Retrain from the clean pretrained checkpoint and compare on held-out conversations (1.5–3 h) against the production model, with a fixed seed.

Lessons carried over from earlier rounds

What happened before	How it shapes this plan
Testing on conversations close to the training data made memorization look like generalization	Decide only on a frozen held-out set that never enters training (`eval/`)
Widening a recall task from 8 to 32 topics made it work 8 times in a row on unseen names	Use the same topic × slot recipe, with wide topic coverage, for polarity data
Training on 27, then 97, then 70 fixed facts kept blending entities together	Avoid a few memorized "answers"; spread reactions over many phrasings
Making only the model bigger made it worse	Keep the model size fixed and change only the data distribution
Run-to-run noise hid real effects	Fix the random seed for every comparison

Takeaways

A small model repeats the habits of its data. Before blaming model capacity, grep the corpus. The whole diagnosis took minutes of counting and zero GPU time.
Count what the model actually trains on. Raw file counts and the weighted training mix told different stories. Upsampling changes which phrases dominate.
Look at the shape, not the totals. Positive replies were never scarce overall: they match negative-leaning ones in the raw files and outnumber them in the training mix. Stopping at "too much negativity" would have led to deleting sympathy data, which fixes nothing.
Template generators leak their defaults. Most of the "tough" replies come from one generator, often attached to neutral remarks.

Limits

The counts are substring matches. They do not separate User: lines from Lilas: lines, except where noted.
The 232 occurrences outside the repo files are assumed to be used once (they are not upsampled by persona_repeat in lilas/data.py).
The repo holds the latest version of the dialogue files, which may differ slightly from what the production model trained on. That this distribution causes the bug is still a hypothesis. Step 5 is the test. I will add the results to this post.

Reproduce

From a clone of the repo, the curated files can be counted like this:

for p in それは大変 それは良かった それは楽しみ; do
  echo "$(cat index/*.txt | grep -o "$p" | wc -l) $p"
done

Multiply each file's count by its factor from lilas/data.py (persona_repeat=21 for the persona files, ×3 extra for knowledge files, ×5 extra for greetings) to get the weighted numbers.