DEV Community: Amandeep Singh

Making a dog talk with zero backend. 🐶✨ Built an entirely client-side pipeline to animate talking pets: - ✂️ In-browser image segmentation - 👄 Real-time WebGL lip-syncing - 🎬 Native MP4 export (all in the browser) No servers, no GPU compute bills

Amandeep Singh — Mon, 17 Aug 2026 04:47:46 +0000

DEV Weekend Challenge: Dog Days Edition Submission 🐕

Amandeep Singh

Aug 16

Making a dog talk with zero backend: in-browser segmentation, WebGL lip sync and real MP4 export

#devchallenge #weekendchallenge #ai

6 min read

Making a dog talk with zero backend: in-browser segmentation, WebGL lip sync and real MP4 export

Amandeep Singh — Sun, 16 Aug 2026 14:55:43 +0000

This is a submission for Weekend Challenge: Dog Days Edition

What I Built

My dog has opinions. She has never once shared them.

So I built BarkReels: you upload a photo, and it gives her a voice. The mouth actually moves with the audio, she blinks, her ears twitch, subtitles burn in, and you get a real MP4 at the end.

Demo

The goal was to see how far a serious video pipeline could go with no backend at all. No upload, no render queue, no "your video will be ready in 4 minutes" email. Your photo goes to the vision API you picked and nowhere else. Cutting your dog out of the background, animating her, and encoding the MP4 all happen on your own machine, with your own GPU.

What you get to play with

Everything here is a control in the UI, not a config file.

Where the words come from. You pick before analysis, so the vision model only does the job you need:

Let the AI write the monologue from the photo
Write the script yourself, pick a voice
Record your own voice and use it directly

Voice personas: Deep & Wise 🧙, Playful Pup 🎾, Dramatic 🎭, Sassy Diva 💅.

Animation styles, least to most motion:

Preset	What it does
🗿 Locked Off	Only the face moves. The default, and most convincing.
🍃 Barely There	A whisper of drift
🎙️ Portrait	Gentle sway
🙃 Belly Roll	Lazy sideways flop
🏀 Bouncy	Springy on the beat
💨 Zoomies	Maximum chaos

Then sliders for mouth movement, head motion, emphasis nods, camera push-in, handheld shake, background parallax, blink rate and ear twitch.

Backgrounds: blurred, original, your own uploaded image, plus procedural Sunset, Studio, Park, Neon and Solid. Your dog gets segmented out of the photo, so the background genuinely sits behind her and moves independently.

Subtitles: Karaoke (rolling window, highlighted word), Pop-up (two words, hard cuts) or Minimal (full sentence). Size, position, uppercase and colour are yours. The colour picker comes pre-loaded with shades sampled from your actual photo, so a brown dog gets warm captions and a pink collar gets pink ones.

Export: MP4 or WebM at 1080x1920, 720x1280 or 1080x1080, 24 to 60fps.

Getting the best results

The animation moves specific anatomical points, so the photo matters more than you would expect. This took a lot of bad outputs to work out:

✅ Head-on portrait. Profile shots hide one eye and one ear, and the rig has nothing to animate on that side.
✅ Face fills the frame. Head and shoulders. In a full-body shot the muzzle is too small to deform cleanly.
✅ Eyes, nose, mouth and ears clearly visible. These are literally the coordinates the animation moves.
✅ One dog. Segmentation keeps a single subject. Two dogs become one blob.
✅ Sharp and well lit. Blur and deep shadow make the cutout edge mushy.
❌ Nothing over the face. Sunglasses, hands, toys. Anything covering an anchor gets deformed along with it.

Two more things I learned the hard way:

Start with Locked Off. It is counterintuitive, but motion is what gives away a puppeted photo. Hold the frame completely still and the viewer reads it as real video of a dog sitting calmly, leaving only the mouth to judge. And the mouth is the one part driven by real audio.

Short scripts win. Two or three punchy sentences. Long monologues drift out of sync with the face and burn your free tier faster.

Demo

🔗 Live app: https://barkreels.vercel.app/

Seriously, go try it. It is faster than finishing this post.

Step	Time
Sign up at ElevenLabs, copy the API key	~3 min
Sign up at Google AI Studio with your Google account, copy the key	~2 min
Open the app, paste both keys, drop in a dog photo	~1 min
Hit analyse, hit generate, hit export	~4 min

Both free tiers are enough. No card required for either. The keys live in your browser's local storage and are sent only to the provider you picked, because there is no server of mine to send them to.

⏳ What to expect on your first run. Because background removal happens on your machine, the browser fetches the segmentation model before it can cut your dog out. That is roughly 25MB of ONNX runtime and neural network weights, and on a slow connection it can take a minute or two. You get a progress bar, and the download starts the moment you drop a photo in so it overlaps with everything else. It is a one-time cost. Your browser caches it and every run after that goes straight to work. If the first one feels slow, that is what is happening, and it is not stuck.

No dog? Steal one from Unsplash. It works on any dog and I will not tell anyone.

Code

singhAmandeep007 / barkreels

Give your dog a voice 🐕

BarkReels 🐕 - client-only AI dog video generator

Upload a photo of your dog. Get back a vertical short-form video where they talk, breathe, blink, and nod - with burned-in subtitles and a real MP4 at the end. Every pixel is rendered in your browser. There is no backend.

Built for the DEV Weekend Challenge: Dog Days Edition.

How it works

photo ──┬─→ vision model ──→ breed / mood / monologue / facial anchors
        └─→ ISNet (ONNX, in-browser) ──→ dog cutout + alpha

monologue ──→ ElevenLabs ──→ audio + word timestamps
                               │
                               ├─→ RMS envelope (asymmetric smoothing)
                               └─→ subtitle timing

envelope + anchors + cutout ──→ animation rig ──→ WebGL warp ──→ frames
                                                                  │
                                          WebCodecs H.264 + AAC ──┴──→ MP4

The one architectural rule

renderFrame(t) is a pure function of time. It reads no clock, consults no playback state, and mutates nothing that affects a later call.

That…

View on GitHub

MIT licensed. Fork it, enable Pages in repo settings, and push: there is a GitHub Actions workflow that builds and deploys on every push to main. Since there is no backend, static hosting is all it has ever needed.

The interesting files:

src/render/rig.ts               the animation formula (pure functions)
src/render/glRenderer.ts        WebGL2 renderer, renderFrame(t)
src/render/backgroundPlate.ts   pull-push inpainting
src/render/exporter.ts          offline WebCodecs encode + muxing
src/services/vision.ts          Gemini and Ollama behind one interface
src/services/audioAnalysis.ts   envelope, onsets, PCM decode

How I Built It

photo ─┬─→ Gemini ──→ breed, mood, monologue, facial anchor coordinates
       └─→ ISNet (ONNX, in-browser) ──→ dog cutout with alpha

monologue ──→ ElevenLabs ──→ audio + word-level timestamps
                                 ├─→ loudness envelope  ──→ drives the jaw
                                 └─→ word timings       ──→ drives subtitles

envelope + anchors + cutout ──→ animation rig ──→ WebGL ──→ WebCodecs ──→ MP4

Google AI does two jobs in one call. Gemini Flash writes the monologue in the dog's voice and acts as a precise vision annotator, returning normalised bounding boxes for the nose, mouth, eyes and ears. Those coordinates are what the shader deforms. When you supply your own script the prompt drops the writing job entirely, so the model spends all its attention on the coordinates.

ElevenLabs is doing more than narration. The /with-timestamps endpoint returns character-level timings alongside the audio. Those timings drive the karaoke subtitles, and the loudness envelope derived from the same audio drives the jaw. The voice is not decoration here, it is the animation input.

The one rule that made everything else work: renderFrame(t) is a pure function of time. It reads no clock and consults no playback state. Preview is requestAnimationFrame(() => renderFrame(audio.currentTime)). Export is a plain for loop over frame indices. Same code, same pixels, so the preview is an actual promise about the output. Export runs offline at roughly 2 to 3 times real time, and backgrounding the tab cannot corrupt it.

The lip sync is not a model. It is the loudness envelope with deliberately asymmetric smoothing: 15ms attack, 90ms release. Muscles snap a jaw open and tissue eases it closed. Symmetric smoothing makes the mouth flutter shut between syllables like a VU meter, and that one asymmetry did more for perceived realism than anything else I tried.

Prize Categories

Best Use of ElevenLabs and Best Use of Google AI. Both are wired into the animation rather than bolted on top of it: Gemini's bounding boxes are what the shader deforms, and ElevenLabs' word timings and loudness envelope are what drive the mouth and the subtitles. See the section above for the details.

What I would build next

Multi-photo angle switching. Upload and thumbnails work, but the renderer still uses one image. Cross-fading between angles on speech beats is the obvious next move.
Better anchors for flat faces. Pugs and bulldogs have a short muzzle and my fallback jaw hinge lands too close to the chin. A breed-aware hinge would fix it.
Head turn. The head is a flat layer today. A depth estimate would allow a small yaw and sell the 3D far harder.
Cross-origin isolation. ONNX runs single-threaded because enabling COOP/COEP would break the Gemini and ElevenLabs calls. Moving segmentation into an isolated worker would make it roughly 4 times faster.
Tail wag and a real ground shadow for full-body shots, which currently get no love at all.

Ten minutes, start to finish, and both free tiers cover it. If you make one, post it in the comments. I genuinely want to see what your dog has been holding back.

Hey everyone! I recently wrapped up a project migrating 6 separate Go microservice repositories into a unified monorepo setup. I documented the architecture decisions, pipeline setup, and lessons learned here.

Amandeep Singh — Sun, 09 Aug 2026 03:22:07 +0000

Amandeep Singh

Aug 7

Multi-Repo to Monorepo: How I Automated 6 Go Microservice Releases and Then Made It 15x Faster

#go #devops #automation #monorepo

10 min read

Multi-Repo to Monorepo: How I Automated 6 Go Microservice Releases and Then Made It 15x Faster

Amandeep Singh — Fri, 07 Aug 2026 17:47:23 +0000

Last month I spent more than an hour cutting a release across six Go microservice repos. Tag log, wait for CI. Update sdk's go.mod to point at the new log SHA, push, wait for CI. Repeat for utils. Then do api, cli, and worker in parallel - except I forgot to bump cli's dependency and the build broke at 11pm.

That was the last manual release I did.

This is the story of automating that entire workflow with Jenkins + Python + GitLab, then realizing the multi-repo architecture was the real problem, and collapsing everything into a Go monorepo that's 15x faster at cutting releases.

The full setup runs on my laptop. You can fork it and try it yourself.

The Six Modules
The Stack
Phase 1: Multi-Repo Automation
Phase 2: The Monorepo Pivot
The Unified CI Pipeline
Real Numbers
Caveats and Gotchas
Try It Yourself

The Six Modules

The project simulates a real production system with six Go modules that have strict dependency ordering:

Module	Role	Tag Scheme	Depends On
`log`	Logger (leaf, no deps)	`v0.x.0`	-
`sdk`	API client	`v0.x.0`	log
`utils`	Shared utilities	`v0.x.0`	log, sdk
`api/backend`	Backend	`APP-x.y.z`	log, utils
`cli`	CLI	`cli-x.y.z`	log, sdk
`worker`	Background	`v0.x.0`	log, utils

The first three modules are sequential - sdk can't tag until log is tagged, utils can't tag until sdk is tagged. The last three are terminal - they can process in parallel once the sequential chain is done.

Every module lives on three long-lived branches: develop → release → master. A release means moving code through all three, in all six repos, in the right order.

That's the problem. Do it manually and you're juggling 6 repos × 3 branches × dependency ordering. One forgotten go mod tidy and you're debugging at midnight.

The Stack

Everything runs on a MacBook. No cloud CI, no SaaS - just local tools wired together.

 MacBook                                         GitLab.com
+-------------------+    ngrok tunnel    +------------------------+
|  Jenkins LTS      | <===============> |  Webhooks (push / MR)  |
|  (brew service)   |                   |  Commit status API     |
|  :8080            | ================> |  6 repos (multi-repo)  |
|                   |   git push,       |  - or -                |
|  Python 3 + Go    |   API calls       |  1 repo  (monorepo)    |
+-------------------+                   +------------------------+

The key pieces:

Jenkins LTS via brew install jenkins-lts, running as a launched service
ngrok exposing localhost:8080 → https://enclosure-version-moocher.ngrok-free.dev/ so GitLab integration/webhook can reach Jenkins
GitLab.com hosting the repos
Python (python-gitlab + GitPython) for the automation engine
Go 1.24 with workspace support (go.work)

Jenkins ↔ GitLab Integration

Configured using GitLab’s native Jenkins Integration combined with ngrok for local tunneling:

GitLab → Jenkins: Native integration automatically triggers local Jenkins pipelines on push and MR events.
Jenkins → GitLab: Reports build progress and status back via API, displaying green/red status checks directly on commits and merge requests.

Phase 1: Multi-Repo Automation

The Setup: 6 Repos, 7 Jenkins Jobs

In the multi-repo world, each module is its own GitLab repo under the multirepo-release/ group:

multirepo-release/
  ├── log/          → gitlab.com:multirepo-release/log.git
  ├── sdk/          → gitlab.com:multirepo-release/sdk.git
  ├── utils/        → gitlab.com:multirepo-release/utils.git
  ├── api/          → gitlab.com:multirepo-release/api.git
  ├── cli/          → gitlab.com:multirepo-release/cli.git
  ├── worker/       → gitlab.com:multirepo-release/worker.git
  └── pipelines/    → gitlab.com:multirepo-release/pipelines.git

Each repo gets its own Jenkins CI job (6 jobs), plus one orchestration job that runs the automation Python script. That's 7 Jenkins jobs to maintain.

The Automation Engine: 1,800 Lines of Python

The pipelines/ repo contains the brain: a Python script (poc_automation.py) that talks to the GitLab API and performs git operations. One Jenkins job, five actions:

Action	What it does
`release_cut`	Merges develop → release for all 6 repos, in dependency order
`backmerge`	Merges release → develop (or master → release)
`rc_create`	Merges release → master, tags each module, bumps go.mod versions
`version_bump`	Bumps version constants in api and cli on develop
`compute_versions`	Calculates next semver tags based on existing tags

How a Release Cut Actually Works

When you trigger release_cut, the script:

Sequential phase - processes log, sdk, utils one at a time:
- Creates an intermediate branch from release
- Merges develop into it (handling go.mod conflicts automatically)
- Creates a GitLab MR
- Waits for approval (or auto-merges if configured)
- Records the merged SHA - the next repo needs it for go mod tidy
Parallel phase - processes api, cli, worker simultaneously:
- Same merge flow, but all three run concurrently since they're terminal nodes

The hardest part: cross-repo dependency tracking. When log merges to release, sdk needs to run go get gitlab.com/multirepo-release/log@<new-SHA> to pin to the exact commit. The script tracks these SHAs in self.merged_target_shas and passes them forward through the chain.

Conflict Handling

When code conflicts happen (not just go.mod), the pipeline doesn't crash. It:

Creates a conflict branch cut from the source (so GitLab renders the real diff)
Opens an MR with [CONFLICT] in the title
Polls every 30 seconds for manual resolution + approval
Auto-merges once both conditions are met
Continues the pipeline

The HTML Report

Every automation run generates a styled HTML report archived in Jenkins. It logs every action taken: branches created, MRs opened, approvals received, tags applied.

Phase 2: The Monorepo Pivot

The multi-repo automation worked. But maintaining it was its own job:

7 Jenkins jobs to configure and monitor
6 Jenkinsfile.ci files (103 lines each, identical except for the module name)
1,800 lines of Python managing 6 separate GitLab projects, tracking SHAs across repos, handling per-repo cloning
6 MRs per release cut - one per repo, each needing approval

The question became: what if all six modules lived in one repo?

go.work: The Go Workspace

Go 1.18 introduced workspaces (go.work), which let multiple modules in a single repo resolve each other locally during development while still being independently importable by external consumers.

// go.work - the workspace file at the repo root
go 1.26

use (
    ./api/backend
    ./cli
    ./log
    ./sdk
    ./utils
    ./worker
)

replace (
    gitlab.com/singhamandeep007-group/monorepo/log v0.3.0 => ./log
    gitlab.com/singhamandeep007-group/monorepo/sdk v0.3.0 => ./sdk
    gitlab.com/singhamandeep007-group/monorepo/utils v0.3.0 => ./utils
)

The replace directives are the key insight: during local development and CI, Go resolves internal dependencies from the local directory. But when an external project runs go get gitlab.com/.../monorepo/log@v0.3.0, Go fetches the tagged version from GitLab. Best of both worlds.

Per-Module Tagging in a Monorepo

Go's module system supports directory-prefixed tags. In a monorepo, each module gets its own tag with a path prefix:

# Each module tagged independently, on the same commit
git tag log/v0.3.0
git tag sdk/v0.3.0
git tag utils/v0.3.0
git tag api/backend/APP-1.3.0
git tag cli/cli-1.3.0
git tag worker/v0.3.0

When someone runs go get gitlab.com/singhamandeep007-group/monorepo/sdk@v0.3.0, Go looks for the tag sdk/v0.3.0 - the module path prefix matches the tag prefix. Each module can version independently while living in the same repo.

What Simplified

The monorepo automation (monorepo-pipelines/poc_automation.py) dropped from 1,762 to 1,143 lines - a 35% reduction. Here's what went away:

Multi-repo complexity	Monorepo equivalent
6 separate GitLab project objects	1 project object
Clone + manage 6 local git repos	No local cloning needed
Track `merged_target_shas` across repos	Not needed - same repo, same SHA
`go get module@SHA` + `go mod tidy` per repo	GitLab file API to edit go.mod directly
6 MRs per release cut	1 MR
`sync_dependencies` action	Not needed

Third-Party Consumption

An external project can import any module from the monorepo individually:

// third-party project's go.mod
module gitlab.com/singhamandeep007-group

go 1.26

require gitlab.com/singhamandeep007-group/monorepo/log v0.3.0

// main.go
package main

import "gitlab.com/singhamandeep007-group/monorepo/log"

func main() {
    logger := log.NewLogger()
    logger.Info("Using log from the monorepo")
}

The consumer doesn't need the entire monorepo - Go fetches only the log module at the tagged version.

The Unified CI Pipeline

One Pipeline, All Modules, In Parallel

The monorepo CI (jenkins/ci.Jenkinsfile, 324 lines) replaces 6 identical per-repo Jenkinsfile.ci files (636 lines total). It fans out all six modules in parallel:

For each module, in parallel:

go vet ./...
go test -count=1 -race -covermode=atomic → JUnit XML + coverage HTML
govulncheck → JSON + human-readable
go build (only for api/backend and cli) → multi-arch binaries + SHA256SUMS

stage("All Packages") {
    parallel pkgStages  // 6 modules run simultaneously
}

All six run with failFast: false - a failure in one module doesn't mask problems in others. The pipeline produces an aggregated ci-report.html showing every module's test results, coverage percentage, and vulnerability count in one page.

Multi-Arch Builds

Terminal binaries (api/backend, cli) build for three targets:

# CGO_ENABLED=0 for cross-compilation (no C dependencies)
GOOS=darwin GOARCH=arm64 CGO_ENABLED=0 go build -o bin/api-darwin-arm64
GOOS=linux  GOARCH=amd64 CGO_ENABLED=0 go build -o bin/api-linux-amd64
GOOS=linux  GOARCH=arm64 CGO_ENABLED=0 go build -o bin/api-linux-arm64

# SHA256 checksums for verification
shasum -a 256 bin/* > bin/SHA256SUMS

The Release Automation Pipeline

The release automation lives in a separate repo (monorepo-pipelines) - keeping release tooling isolated from the application code. Same five actions as before, but operating on a single repo:

After an RC, the script tags each module and updates go.mod + go.work via the GitLab file API - no local cloning needed.

Real Numbers

These come from 451 multi-repo builds and 36 monorepo builds on the same Jenkins instance, running the same Go modules.

Release Automation Speed

Action	Multi-Repo (avg)	Monorepo (avg)	Speedup
`release_cut`	1,170s (~19 min)	75s (~1 min)	15.5x
`backmerge`	688s (~11 min)	35s	19.4x
`rc_create`	501s (~8 min)	73s (~1 min)	6.9x

The multi-repo release_cut takes ~19 minutes because it processes 6 repos sequentially (3 sequential + 3 parallel), creating 6 MRs, waiting for each CI to pass, and tracking SHAs across repos. The monorepo version creates 1 MR and is done in ~75 seconds.

CI Pipeline

Metric	Multi-Repo	Monorepo
Jenkins jobs	7 (6 CI + 1 orchestration)	2 (1 CI + 1 automation)
CI wall time	~132s (sequential across 6 jobs)	~55s (parallel in 1 job)
Jenkinsfile maintenance	6 × 103 lines (identical)	1 × 324 lines
MRs per release cut	6	1

Codebase

Metric	Multi-Repo	Monorepo	Change
Automation Python	1,800 lines	1,143 lines	-36%
CI pipeline config	618 lines (6 files)	324 lines (1 file)	-48%
Jenkins orchestration	274 lines	231 lines	-16%
Total pipeline code	2,692 lines	1,698 lines	-37%

Total Build History

	Multi-Repo	Monorepo
CI builds	385	24
Automation builds	66	12
Total builds	451	36
Jenkins jobs to maintain	7	2

Caveats and Gotchas

A few things that bit me along the way - saving you the debugging time:

go.work replace versions must match go.mod exactly

If sdk/go.mod says require log v0.5.0 but go.work has replace log v0.3.0 => ./log, Go ignores the replace and tries to fetch log v0.5.0 from the remote. The versions must match exactly. This caused CI failures on all three branches when our RC automation bumped go.mod versions without creating matching tags.

Jenkins doesn't load shell profiles

If you install Go via asdf or a version manager, Jenkins won't find it - it runs a non-login, non-interactive shell. Set GOROOT and PATH explicitly in the pipeline environment:

environment {
    GOROOT = '/usr/local/go'
    PATH   = "/usr/local/go/bin:${env.HOME}/go/bin:${env.PATH}"
}

Private modules need three environment variables

For private GitLab-hosted Go modules, GOPRIVATE alone isn't enough. You also need GONOSUMCHECK and GONOSUMDB to prevent checksum database lookups that will fail:

GOPRIVATE    = 'gitlab.com/your-group/*'
GONOSUMCHECK = 'gitlab.com/your-group/*'
GONOSUMDB    = 'gitlab.com/your-group/*'

CGO and cross-compilation don't mix (on macOS)

CGO_ENABLED=1 is needed for -race in tests, but cross-compiling to linux/amd64 on macOS with CGO on fails because there's no Linux C toolchain. Solution: CGO_ENABLED=1 globally for tests, CGO_ENABLED=0 per build command:

# Tests: CGO on (for -race)
CGO_ENABLED=1 go test -race ./...

# Builds: CGO off (for cross-compilation)
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/api-linux-amd64

Groovy triple-quoted strings mangle Python escape sequences

A \n inside a Groovy """...""" string becomes a literal newline before the embedded Python sees it, causing SyntaxError: EOL while scanning string literal. Use chr(10) and chr(9) instead:

# Breaks: Groovy converts \n to a real newline
line.startswith("\nFAIL\t")

# Works: chr() bypasses Groovy's string processing
line.startswith(chr(10) + "FAIL" + chr(9))

Try It Yourself

Both repos are on GitLab:

Monorepo (Go source + CI): gitlab.com/singhamandeep007-group/monorepo
Monorepo Pipelines (release automation): gitlab.com/singhamandeep007-group/monorepo-pipelines
Multi-repo version (for comparison): gitlab.com/multirepo-release/

Quick Start

# 1. Install Jenkins
brew install jenkins-lts
brew services start jenkins-lts

# 2. Install ngrok and start tunnel
ngrok http 8080 --url your-subdomain.ngrok-free.dev

# 3. Clone and explore
git clone git@gitlab.com:singhamandeep007-group/monorepo.git
cd monorepo
go work sync
make test-all

What's Next

This is a POC - production would need:

Secrets management: rotate GitLab tokens, use Jenkins credential rotation
Agent scaling: Jenkins agents instead of running everything on the controller
Notification: Slack/email integration for merge approvals and failures
Caching: Go module cache persistence across builds

Wanted Claude to read my OneNote notebooks. Grabbed an MCP server, hit a silent 401 on every call. Wrote up the full debugging. Check it out

Amandeep Singh — Sat, 25 Jul 2026 17:45:15 +0000

Scope pitfalls and silent token renewal

Amandeep Singh

Jul 19

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

#mcp #typescript #tutorial #ai

6 min read

Onenote MCP server rewritten - awesome

Amandeep Singh — Sun, 19 Jul 2026 11:59:01 +0000

Scope pitfalls and silent token renewal

Amandeep Singh

Jul 19

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

#mcp #typescript #tutorial #ai

6 min read

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

Amandeep Singh — Sun, 19 Jul 2026 11:57:24 +0000

If you use Claude, Cursor, or any MCP-compatible AI assistant, you've probably noticed how useful it is when your AI can actually read your notes. I wanted that for OneNote — so I grabbed an existing MCP server, hit a cryptic 401 error, and ended up rewriting the whole thing in TypeScript.

Here's what I learned.

What's an MCP Server?

Model Context Protocol (MCP) is a standard that lets AI assistants call external tools over JSON-RPC. You write a server that exposes tools (like listNotebooks or createPage), and any MCP client — Claude Desktop, Cursor, Claude Code — can invoke them.

The server communicates over stdio: JSON-RPC on stdout, diagnostics on stderr. This is a critical detail that will matter later.

The Starting Point

I started with danosb/onenote-mcp, a JavaScript MCP server that uses Microsoft Graph to access OneNote. It had the right idea — device-code auth so you don't need to register an Azure app — but I hit a wall immediately.

The 401 That Broke Everything

After authenticating with my personal Microsoft account, every Graph API call returned:

HTTP 401 — "The request does not contain a valid authentication token"
(error code 40001)

The token looked fine. Authentication completed successfully. But Graph rejected every request.

The Root Cause: `.All` Scopes

The original server requested these scopes:

const scopes = ['Notes.Read.All', 'Notes.ReadWrite.All', 'User.Read'];

Those .All scopes are application-level permissions. Personal Microsoft accounts (MSA) cannot consent to them — only Azure AD work/school accounts can. When a personal account tries to use them, Azure does issue a token, but it's a token that Microsoft Graph won't accept.

No error during auth. No warning. Just a silent 401 on every subsequent call.

The Fix

Replace .All scopes with resource-qualified delegated scopes:

export const SCOPES = [
  'https://graph.microsoft.com/Notes.Read',
  'https://graph.microsoft.com/Notes.ReadWrite',
  'https://graph.microsoft.com/User.Read',
  'offline_access', // needed for refresh token
  'openid',         // needed for id_token (username)
];

These are delegated scopes that work for both personal and work/school accounts. They grant access to the signed-in user's own OneNote content, which is exactly what you want for an MCP server. The offline_access scope is critical — without it, you won't get a refresh token and users will have to re-authenticate every hour.

Bonus Gotcha: Non-JWT Tokens

Personal Microsoft accounts return compact (non-JWT) tokens — opaque strings that don't have the three-segment header.payload.signature structure. If your code validates token format by checking for JWTs, it will incorrectly reject perfectly valid personal-account tokens.

The right approach: don't validate token format at all. Let Microsoft Graph be the authority on whether a token is valid.

Why a Full TypeScript Rewrite?

Once I fixed the auth bug, I looked at the codebase and found:

~10 standalone JavaScript files doing overlapping things (simple-onenote.js, list-sections.js, list-pages.js, get-page.js, get-page-content.js, get-all-page-contents.js...)
MCP tools with no parameter schemas — everything came in through params.random_string
A dependency on jsdom just for HTML-to-text conversion
A dependency on node-fetch (unnecessary with Node 18+ native fetch)
The deprecated Client.init callback pattern instead of Client.initWithMiddleware

So I rewrote it.

The New Architecture

Everything lives in src/ with clear separation:

src/
  config.ts         — Client ID, tenant, scopes, paths
  logger.ts         — stderr-only logging (stdout = JSON-RPC)
  token-store.ts    — Token normalization utilities
  auth.ts           — OAuth 2.0 device-code auth + silent token refresh
  graph-client.ts   — Graph SDK client factory
  html.ts           — HTML→text (dependency-free)
  onenote.ts        — OneNoteClient class
  mcp-server.ts     — MCP server with Zod-typed tools
  cli.ts            — Unified CLI (replaces 10 scripts)
  index.ts          — Barrel export

Key Design Decisions

1. Zod schemas on every tool

The old server used the SDK's tool() method without schemas, so parameters arrived as params.random_string. The new server defines explicit Zod schemas:

server.tool(
  'getPage',
  'Get page content by ID or title search.',
  { query: z.string().min(1).describe('Page ID or title substring') },
  async ({ query }) => {
    // query is typed as string, validated by Zod
  },
);

This gives AI clients proper parameter descriptions and type information, so they know exactly what to pass.

2. One client class instead of scattered scripts

OneNoteClient encapsulates all Graph API operations:

const client = OneNoteClient.create();
const notebooks = await client.listNotebooks();
const page = await client.findPage("meeting notes");
const content = await client.getPageContent(page.id);
console.log(content.text); // HTML already converted to plain text

3. Dependency-free HTML→text

OneNote's Graph API returns page content as HTML. The old code used jsdom (a heavy dependency) to parse it. The new htmlToText() function handles it with regex — strip script/style blocks, convert block elements to newlines, decode entities, collapse whitespace:

export function htmlToText(html: string): string {
  return html
    .replace(/<(script|style)[^>]*>[\s\S]*?<\/\1>/gi, ' ')
    .replace(/<br\s*\/?>/gi, '\n')
    .replace(/<\/(p|div|h[1-6]|li|tr|table|ul|ol)>/gi, '\n')
    .replace(/<[^>]+>/g, ' ')
    // ... entity decoding, whitespace collapsing
    .trim();
}

For the structured HTML that OneNote returns, this is more than sufficient and eliminates a large dependency.

4. stderr-only logging

This one is subtle but critical. An MCP stdio server uses stdout exclusively for JSON-RPC messages. Any console.log() call corrupts the protocol stream and crashes the connection. All logging goes through a log() helper that writes to stderr:

export function log(...args: unknown[]): void {
  console.error('[onenote-mcp]', ...args);
}

5. Direct OAuth 2.0 — no auth library needed

The original code saved a raw access token to a file. Access tokens expire in ~1 hour, so users had to re-authenticate constantly. I initially reached for @azure/msal-node to handle token persistence and silent renewal, but hit a v5 regression where acquireTokenSilent() sent refresh token requests without client_id in the POST body — causing AADSTS900144 errors after the first hour.

The OAuth 2.0 device-code flow is two HTTP calls. Token refresh is one. I replaced the library with ~180 lines of direct fetch calls:

// Silent refresh — the entire implementation
const res = await fetch(TOKEN_URL, {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: new URLSearchParams({
    client_id: CLIENT_ID,
    grant_type: 'refresh_token',
    refresh_token: cachedRefreshToken,
    scope: SCOPES.join(' '),
  }).toString(),
});

Tokens are cached in a simple JSON file (chmod 600, gitignored) with access_token, refresh_token, and expires_at. When the access token expires, the server silently refreshes it using the stored refresh token — no user interaction required. Sign in once, never think about it again.

Testing

The test suite uses Vitest with mocked Graph clients — no real API calls needed:

vi.mock('../src/graph-client.js', () => ({
  createGraphClient: vi.fn(),
}));

// Mock returns controlled data
mockGraphClient({
  '/me/onenote/pages': {
    value: [{ id: 'p1', title: 'Meeting Notes 2024' }]
  },
});

const client = new OneNoteClient(async () => 'fake-token');
const found = await client.findPage('meeting notes');
expect(found?.id).toBe('p1');

29 tests cover HTML conversion, token handling, and all OneNote client operations.

How to Use It

git clone https://github.com/singhAmandeep007/onenote-mcp.git
cd onenote-mcp
npm install
npm run auth     # sign in once — tokens auto-renew after this
npm run verify   # confirm it works

Add to Claude Desktop's config — this runs the TypeScript source directly via tsx, no build step needed:

{
  "mcpServers": {
    "onenote": {
      "command": "npx",
      "args": ["tsx", "/path/to/onenote-mcp/src/mcp-server.ts"]
    }
  }
}

Restart Claude Desktop to pick up the new config, then ask: "What's in my OneNote notebooks?"

TL;DR

If you're building an MCP server that talks to Microsoft Graph:

Use delegated, non-.All scopes — personal accounts silently fail with .All scopes
Don't validate token format — personal accounts return non-JWT tokens that are valid
Request offline_access explicitly — without it you won't get a refresh token, and users re-authenticate every hour
Think twice before reaching for an auth library — the device-code flow is two HTTP calls; token refresh is one. Direct OAuth is fewer lines, zero dependencies, and you can read every line that runs
Never write to stdout — MCP uses it for JSON-RPC; all logging goes to stderr
Add Zod schemas to your tools — AI clients need parameter descriptions to use them correctly

The full source:

singhAmandeep007 / onenote-mcp

MCP server for Microsoft OneNote

OneNote MCP Server

A TypeScript Model Context Protocol (MCP) server that gives AI assistants (Claude, Cursor, etc.) read/write access to your Microsoft OneNote notebooks via the Microsoft Graph API.

Zero Azure setup required. Authentication uses the device-code flow with a first-party Microsoft client ID, so you only need a Microsoft account. Sign in once and tokens renew automatically — you won't be asked to authenticate again.

Fork of azure-onenote-mcp-server by Zubeid Hendricks — rewritten in TypeScript with proper auth, typed tools, and a unified CLI.

Features

One-time authentication — sign in once via device-code flow; tokens silently refresh using a stored refresh token (access tokens expire hourly, but this is invisible to you)
Works with both personal Microsoft accounts and work/school (Entra ID) accounts
List notebooks, sections, and pages
Read page content as plain text (HTML is stripped automatically)
Create pages with HTML content
Search pages by title across all…

View on GitHub

PRs welcome.

Stop wasting hours on React setup. 🛑 I just launched Version 2 of my production-ready React-Vite-TS boilerplate. It’s now a single-command scaffold to get you from zero to production.

Amandeep Singh — Thu, 16 Apr 2026 05:41:01 +0000

Amandeep Singh

Apr 4

Building a Production-Ready React + Vite + TypeScript Boilerplate: Architecture, Choices & DX Deep-Dive

#react #typescript #vite #webdev

12 min read

From college assignment to portfolio centerpiece. 📂

Amandeep Singh — Thu, 16 Apr 2026 05:28:54 +0000

Amandeep Singh

Apr 5

Building an Interactive N-Queens Visualizer with React + TypeScript

#react #typescript #algorithms #visualization

5 min read

How would you rate my portfolio out of 10 from a UX/UI and performance perspective? The good, the bad, and the ugly - tell me in the comments.

Amandeep Singh — Thu, 16 Apr 2026 05:26:05 +0000

Amandeep Singh

Apr 5

How I Built My Portfolio Website with React + Vite: Themes, Particles, GitHub Visualizations, and a Dev.to-Powered Posts Page

#react #typescript #portfolio #webdev

5 min read

Building an Interactive N-Queens Visualizer with React + TypeScript

Amandeep Singh — Sun, 05 Apr 2026 06:21:54 +0000

Building an Interactive N-Queens Visualizer with React + TypeScript

I rebuilt this write-up after a full master-branch audit and focused it on what the code actually ships today: a single-page interactive algorithm visualizer with real-time constraint feedback, simulation playback, and responsive controls.

Live demo: https://singhAmandeep007.github.io/eight-queens-problem-visualizer/

Repository: https://github.com/singhAmandeep007/eight-queens-problem-visualizer

What the App Does
Architecture
Design System
Theming
Motion and Animations
Particles and Background Effects
Special Interaction
Projects Module
Posts and Blog Module
About and Profile Module
Responsiveness
Performance
Deployment
Key Implementation Details
Lessons Learned and Next Steps

What the App Does

The N-Queens puzzle asks us to place N pieces on an N x N board with zero conflicts.

This project turns that into an interactive engineering playground:

Piece-rule switching: queen, bishop, rook, knight
Manual mode for direct placement and conflict feedback
Simulation mode for animated backtracking exploration
Variable board size from 4x4 to 8x8
Solution list replay to inspect discovered arrangements

Architecture

The app is intentionally compact but still modular in behavior:

App shell in App.tsx
State domains in ControlContext and AlertContext
Algorithm and rendering orchestration in Chessboard
Rule engine utilities in constants
Reusable control primitives in ControlSelect and common/button

Flowchart image (static for Dev.to compatibility):

Design System

The design system is lightweight and practical:

CSS variables define core color tokens
Styled-components handle composable UI surfaces
Reusable button primitive keeps action affordances consistent
Board and controls share clear visual hierarchy and spacing

Theming

Even without a dark/light toggle, the app is tokenized:

Primary and secondary hues drive control and action zones
Surface and background tokens support layered feedback
Piece and board contrast remains readable across interactions

Because tokens are centralized, runtime theme switching can be introduced later with minimal component churn.

Motion and Animations

Motion is used to explain logic rather than decorate UI:

Simulation playback updates board state at user-selected speed
Alerts animate in and out for lightweight status communication
Solved-state celebration gives immediate completion feedback

Particles and Background Effects

This branch intentionally avoids heavy particle layers and uses informative effects instead:

Conflict squares use striped overlays
Board container shadows emphasize focus
Celebration overlay marks solved states

That keeps the experience fast while preserving strong visual state signaling.

Special Interaction

The standout interaction is simulation control + replayability:

Play/Stop orchestrates async search
Simulation auto-interrupt logic handles tab visibility changes
Solved boards can be reset and replayed rapidly

GIF fallback:

Projects Module

In this single-page architecture, the solution explorer acts as the project/results module:

Every valid solution is persisted
Entries are clickable to preview placements instantly
Duplicate solutions are deduplicated using serialized keys

Posts and Blog Module

The in-app information modal works as the educational content module:

Problem statement and interaction model are available inline
Portal rendering avoids z-index and stacking issues
Users can enter and exit context quickly without navigation

About and Profile Module

Profile attribution is implemented in the footer module:

Author identity is visible in the app shell
External profile link supports discoverability and ownership

Responsiveness

Responsive behavior is handled through global breakpoints and adaptive layouts:

Root font scaling across screen-size bands
Control bar wrapping under smaller widths
Chessboard + side panel stack behavior for mobile ergonomics

Performance

Performance decisions visible in the code:

useMemo for board-grid generation by board size
Cached solved-state checks for repeated position sets
Fast simulation mode for rapid enumeration
Input controls disabled while simulation runs to reduce race conditions

Known trade-off:

Position encoding uses row*10+col, which is compact and efficient for current board ranges (4-8), but not ideal for larger generalized boards.

Deployment

Deployment pipeline is straightforward and production-safe:

npm run build creates dist output
npm run deploy uses gh-pages
Vite base path is derived from GITHUB_PAGES_REPO for GitHub Pages correctness

Key Implementation Details

Context-driven state architecture
ControlContext owns mode, speed, piece type, board size, simulation state
AlertContext centralizes user feedback and timeout cleanup
Rule engine abstraction
checkConflictMethods encapsulates row/column, diagonal, and knight checks
checkIsAttacking and checkIsSolved compose these rules for both manual and simulation paths
Async simulation loop
Recursive backtracking-style search enumerates candidate positions
Board state updates are interleaved with delays for visual progression
Stop conditions are managed with refs and side effects for safe interruption

Lessons Learned and Next Steps

Lessons:

Explicit state boundaries make even single-page apps easier to reason about
Algorithm visualizers benefit from controls that expose execution speed and mode
Visual effects should carry semantic meaning, not just aesthetics

Next improvements:

Extract solver logic to a dedicated module for stronger unit testing
Add symmetry reduction for mathematically equivalent solutions
Move from row*10+col encoding to coordinate tuples for long-term extensibility
Add keyboard-first interaction flow for accessibility

How I Built My Portfolio Website with React + Vite: Themes, Particles, GitHub Visualizations, and a Dev.to-Powered Posts Page

Amandeep Singh — Sun, 05 Apr 2026 06:00:06 +0000

How I Built My Portfolio Website with React + Vite: Themes, Particles, GitHub Visualizations, and a Dev.to-Powered Posts Page

I recently rebuilt my personal portfolio to reflect how I like to build products: clear architecture, expressive UI, strong performance defaults, and data-backed sections that stay fresh.

Live website: https://singhamandeep007.github.io/

Repository: https://github.com/singhAmandeep007/singhAmandeep007.github.io

Project Overview
Design Direction
Architecture At A Glance
Theme Engine (Multi Theme)
Particles Background With Floating Tech Logos
Resume Open Animation
Projects Page
Posts Page (Powered By Dev.to API)
About Page + GitHub Contributions + GitHub Showcase
Responsive UX + Sidebar Navigation
Performance + Developer Experience
What I Would Improve Next
Final Thoughts

Project Overview

This portfolio is built with:

React 18 + TypeScript
Vite 5
styled-components
React Router (lazy route loading)
tsParticles for animated background motion
Dev.to API integration for articles
GitHub GraphQL API integration for contribution and profile showcase widgets

The goal was simple:

Keep the UI visually alive without becoming noisy.
Keep content dynamic where possible (GitHub + Dev.to).
Keep the codebase modular and easy to personalize.

Design Direction

I wanted the experience to feel like a blend of developer identity and product polish:

bright accent colors that can switch at runtime,
subtle glassmorphism where it helps depth,
playful details (handwave animation, envelope-style resume interaction),
high-density information modules on About (contributions, profile metrics, repo cards).

The whole layout stays section-based, with each page having a clear visual job:

Home: first impression + CTA
Projects: proof of work
Posts: writing and thought process
About: profile depth + GitHub signal

Architecture At A Glance

I use a shared layout route that always renders:

sidebar menu,
active route outlet,
theme toggler,
particle background.

flowchart LR
  A[ThemeProvider + GlobalStyle] --> B[Shared Layout]
  B --> C[SidebarMenu]
  B --> D[Route Outlet]
  B --> E[ToggleTheme]
  B --> F[ParticlesContainer]

  D --> H[Home]
  D --> I[Projects]
  D --> J[Posts]
  D --> K[About]

  K --> L[GitHub Contributions GraphQL]
  K --> M[GitHub Showcase GraphQL]
  J --> N[Dev.to Articles API]

This structure keeps global behavior centralized while pages remain focused.

Theme Engine (Multi Theme)

One of my favorite parts is the theme system.

Themes are created through a small theme factory (createTheme) using primary/background/font values.
The app generates CSS variables for each theme class.
Theme switching is class-based on document.documentElement for instant variable swapping.
A small transition class smooths the color shift.

I also map theme colors into GitHub contribution chart palettes, so the data visuals feel native to whichever theme is active.

Particles Background With Floating Tech Logos

The background uses tsParticles in two layers:

baseline moving particles,
floating image particles sourced from my tech logo list.

A batched loader progressively injects logo particles so the effect ramps up instead of appearing all at once. It gives the page ambient motion while keeping content readable.

Resume Open Animation

Instead of a normal download button, I built a small envelope interaction:

a cover flap rotates with rotateX,
the letter slides out,
click opens/closes,
click on letter triggers resume download.

It is intentionally playful, but still functional and accessible.

Projects Page

The Projects page is data-driven from a single projects data source:

markdown-supported descriptions,
per-project stack icons,
image/video slide support,
fullscreen preview support.

I alternate layout direction between odd/even project items for visual rhythm on larger screens, then collapse to a single-column flow on mobile.

Posts Page (Powered By Dev.to API)

The Posts page fetches articles directly from my Dev.to account and renders them as cards with:

cover image,
tags,
publish date,
comments count,
reactions count.

I filter out repost-like items by checking reading time so the page emphasizes full technical pieces.

About Page + GitHub Contributions + GitHub Showcase

This page combines static profile context with live GitHub data.

1) Contributions Calendar

I fetch contribution years and calendars from GitHub GraphQL, then render a custom SVG chart.

Features include:

virtualized week rendering for smoother scroll,
multiple visual modes (Heatmap, Forest, Garden, Skyline),
year jump controls,
auto-scroll playback,
palette adaptation by active theme.

2) GitHub Showcase

I also built a profile showcase module using GraphQL to display:

follower and repo metrics,
contribution metrics (commits/PRs),
language distribution bar,
pinned/top repositories with topic badges.

Full About page context:

Responsive UX + Sidebar Navigation

Responsiveness is handled through centralized breakpoints and component-level media rules.

Key UX behaviors:

focus-trapped sidebar menu,
click-outside to close overlays,
blur treatment on background content when overlays are active,
typography scaling by breakpoint,
horizontal overflow handling for dense visual modules.

This keeps keyboard and touch interactions practical across sizes.

Performance + Developer Experience

Some implementation decisions that helped:

route-level lazy imports,
manual chunk grouping for vendor and particle libraries,
optional dev-time TypeScript/ESLint checking via environment toggles,
build visualizer output for bundle inspection,
strict TypeScript config and lint/format scripts.

For deployment, GitHub Actions builds and ships to GitHub Pages.

What I Would Improve Next

If I take this further, I would like to add:

richer skeleton/loading states instead of blank transitions,
offline support + cache strategy for API-backed sections,
more accessibility audits (focus indicators, reduced-motion pathways),
analytics on which projects/posts get most engagement.

Final Thoughts

This portfolio became more than a profile page. It is now a living frontend playground where I can experiment with UI systems, data visualizations, and interaction patterns in production.

If you want to explore or fork it:

I would love feedback on the architecture and UX decisions.

DEV Community: Amandeep Singh

Making a dog talk with zero backend. 🐶✨ Built an entirely client-side pipeline to animate talking pets: - ✂️ In-browser image segmentation - 👄 Real-time WebGL lip-syncing - 🎬 Native MP4 export (all in the browser) No servers, no GPU compute bills

Making a dog talk with zero backend: in-browser segmentation, WebGL lip sync and real MP4 export

Making a dog talk with zero backend: in-browser segmentation, WebGL lip sync and real MP4 export

What I Built

Demo

What you get to play with

Getting the best results

Demo

Code

singhAmandeep007 / barkreels

Give your dog a voice 🐕

BarkReels 🐕 - client-only AI dog video generator

How it works

The one architectural rule

How I Built It

Prize Categories

What I would build next

Hey everyone! I recently wrapped up a project migrating 6 separate Go microservice repositories into a unified monorepo setup. I documented the architecture decisions, pipeline setup, and lessons learned here.

Multi-Repo to Monorepo: How I Automated 6 Go Microservice Releases and Then Made It 15x Faster

Multi-Repo to Monorepo: How I Automated 6 Go Microservice Releases and Then Made It 15x Faster

Table of Contents

The Six Modules

The Stack

Jenkins ↔ GitLab Integration

Phase 1: Multi-Repo Automation

The Setup: 6 Repos, 7 Jenkins Jobs

The Automation Engine: 1,800 Lines of Python

How a Release Cut Actually Works

Conflict Handling

The HTML Report

Phase 2: The Monorepo Pivot

go.work: The Go Workspace

Per-Module Tagging in a Monorepo

What Simplified

Third-Party Consumption

The Unified CI Pipeline

One Pipeline, All Modules, In Parallel

Multi-Arch Builds

The Release Automation Pipeline

Real Numbers

Release Automation Speed

CI Pipeline

Codebase

Total Build History

Caveats and Gotchas

go.work replace versions must match go.mod exactly

Jenkins doesn't load shell profiles

Private modules need three environment variables

CGO and cross-compilation don't mix (on macOS)

Groovy triple-quoted strings mangle Python escape sequences

Try It Yourself

Quick Start

What's Next

Wanted Claude to read my OneNote notebooks. Grabbed an MCP server, hit a silent 401 on every call. Wrote up the full debugging. Check it out

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

Onenote MCP server rewritten - awesome

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

I Rewrote a OneNote MCP Server in TypeScript — Here's What I Learned About Microsoft Graph Auth

What's an MCP Server?

The Starting Point

The 401 That Broke Everything

The Root Cause: .All Scopes

The Fix

Bonus Gotcha: Non-JWT Tokens

Why a Full TypeScript Rewrite?

The New Architecture

Key Design Decisions

Testing

How to Use It

TL;DR

singhAmandeep007 / onenote-mcp

MCP server for Microsoft OneNote

OneNote MCP Server

Features

Stop wasting hours on React setup. 🛑 I just launched Version 2 of my production-ready React-Vite-TS boilerplate. It’s now a single-command scaffold to get you from zero to production.

Building a Production-Ready React + Vite + TypeScript Boilerplate: Architecture, Choices & DX Deep-Dive

From college assignment to portfolio centerpiece. 📂

Building an Interactive N-Queens Visualizer with React + TypeScript

How would you rate my portfolio out of 10 from a UX/UI and performance perspective? The good, the bad, and the ugly - tell me in the comments.

The Root Cause: `.All` Scopes