DEV Community

Cover image for I wish AI Agents just knew how I work without me explaining - so I made something that quietly observes me, learns and teaches it.
Sandro
Sandro

Posted on

I wish AI Agents just knew how I work without me explaining - so I made something that quietly observes me, learns and teaches it.

Every time I start a new Claude Code/OpenClaw/Codex session I find myself typing the same context. Here's how I review PRs. Here's my tone for client emails. Here's why I pick this approach over that one. Claude just doesn't have a way to learn these things from watching me actually do them.

So I built AgentHandover.

Mac menu bar app. Watches how you work, turns that into structured Skills, and makes them available to Claude or any agent that speaks MCP. Instead of explaining your workflow, the agent already has it. Your strategy, decision logic, guardrails, voice, which apps are required for different workflows and what to do in these apps, etc. All captured from your real behavior, your workflows end to end that you do on your Max. And it self-improves.

Two ways to use it.

Focus Record: hit record, do the task once, answer a couple clarifying questions, Skill generated. For stuff you know you want to hand over. "This is how I onboard a new client" or "this is my PR review process."

Passive Discovery: let it run in the background. It watches your screen over days, figures out what's work versus noise (activity classifier), clusters similar actions even across different days with interruptions, and after three or more observations synthesizes the pattern into a Skill. It found workflows I didn't realize I had a system for. My Monday metrics routine. How I triage GitHub issues. Stuff I was doing on autopilot that I never would have written down.

The pipeline has 11 stages, all local. Screen capture with deduplication. A local VLM (Qwen 3.5 via Ollama, you can choose different model ofc) annotating every frame with what app you're in, what you're doing, what you'll probably do next. Semantic embeddings to group similar workflows even when they look different on the surface. Cross-session linking so an interrupted task on Tuesday connects to when you finished it Thursday. Then behavioral synthesis that extracts not just steps but the why behind your decisions.

Output is a Skill file (+ knowledge base). Not a prompt, not a summary. A structured playbook with your strategy, steps, guardrails, and writing voice extracted from your own text. Each Skill has a confidence score that improves with every successful execution. If something goes wrong, the Skill adapts. (self-improving)

Safety: screenshots get deleted after. PII, API keys auto-redacted, etc.. Encrypted at rest. Zero telemetry. Nothing leaves your machine. Every Skill goes through lifecycle gates before any agent can touch it.

Pairs with Claude Code out of the box. Also OpenClaw, Codex, etc.

Repo: https://github.com/sandroandric/AgentHandover
Website: https://www.agenthandover.com/

If you've ever wished Claude just knew how you do things, that's what this is for. Happy to answer anything. <3 and ofc credits to Claude Code for being my partner in crime.

Top comments (0)