Okay so a little context first,
I work in an SDE role and work in multiple windows at once like i have many apps open in my mac.
I faced a problem of screenshotting everything and explaining the context and their relations to claude code again and again like for little UI changes and such or any bug.
I got this idea and started working on it , made a swift app which will track cursor and when your press a hotkey it starts only then and when you hold a hotkey it will draw over the areas your point with your cursor and you can keep talking while explaining what you need like you would to a real colleague. The voice timings come from whisper and they get attached with the cursor positions and screenshots and form a full event files.
those get stored as context everytime you do some actions like draw a circle , point to anything or something like that.
and at the end we generate a context using those events that you can just drop to any AI agent of your choice like claude,cursor,codex etc. and it just starts working on it right off the bat.
A bit on how it works
The hard part was never the screenshot. It's the word "this".
"Add a status key to this response" means nothing on its own. It only works if you know what "this" was aimed at. So while a session runs I keep two streams: where the cursor settled, and what you were saying at the time. Then I have to figure out which settles were actually pointing at something and which were just my hand sitting on the trackpad. Cursor position alone can't tell you that.
What worked was watching for the pointing words. If "this" or "here" lands on a settle, that settle was a pointing act. If a settle sits in the middle of silence, it was nothing. That binding is most of the app.
For the screenshot side I use AXUIElementCopyElementAtPosition against a system-wide element, which gives me whatever is under the cursor plus its accessibility label and window title. SCScreenshotManager crops the pixels and Vision runs OCR on the crop. So the brief ends up carrying the name of the thing you pointed at and not just a picture of it, which is usually what the agent was guessing at anyway.
There are no accounts. The trial is keyed to a hash of the machine's platform UUID rather than anything in preferences, so deleting a plist doesn't get you a fresh one. Screenshots, OCR text and the brief never leave the Mac. The audio is the only thing that goes anywhere and it's gone as soon as the text exists.
Here is the producthunt page for more context on it:
And below is the site to get it:
Deiko: show, don’t type
Point at anything on your screen and say what should change. Deiko throws your coding agent a brief with the screenshot attached.
deiko.app
I am thinking of providing a free version with BYO option and a lifetime subscription for around 40ish dollars.
Need some advice and people who face this problem and what do they generally use to solve it?
Top comments (5)
Claude Code downscales whatever you paste, a tall page like a long docs scroll shrinks from something like 2540x18410 down to roughly 355x2576 on current models, so body text ends up a couple pixels tall. Splitting it into text plus image instead of one flat screenshot actually fixes that. I build SlimSnap.ai, free for Mac and Windows, it does that split automatically before handing off to Claude Code or Cursor. Might save you from building a whole voice and cursor pipeline for the same resolution problem.
Deiko doesn't really take those kind of long screenshots or anything
The cap only bites when one edge is much longer than the other, so a normal window capture sits under it fine.
The "this" binding is the genuinely hard part, and pairing pointing words with cursor settles is a neat way to solve it. Two thoughts on pricing: a free BYO-key tier makes sense, but check your per-user transcription cost before fixing a ~$40 lifetime price, because a lifetime deal with a per-minute cost underneath is the classic trap. whisper.cpp runs well on Apple Silicon and would remove that cost, along with the one piece of data that currently leaves the Mac. That's a stronger privacy line than any policy.
Hey thanks...you are right about whisper.cpp I will look into it.