The hardest part of a local AI writing assistant is not the model

Sergey Levine — Sat, 26 Sep 2026 19:58:46 +0000

A lot of people assume the hard part of a local AI writing app is picking the model.

That matters, but it has not been the whole problem.

The harder product problem is making the assistant feel like it belongs in the text field you are already using.

If writing help forces you to copy text into a separate chat window, wait, copy the result back, then clean it up, the model can be good and the workflow can still feel bad.

For a Mac writing assistant, the boring details matter more than they look:

How quickly suggestions show up
Whether rewriting works on selected text
Whether the app stays out of the way when you do not need it
Whether the text stays private instead of being sent to a server
Whether the assistant works in normal macOS text fields instead of only inside one editor

That is the direction I have been taking with DraftKey.

DraftKey is a native Mac app for local AI autocomplete and instruction-based rewriting inside existing macOS text fields. It runs on-device on Apple Silicon, so the writing workflow is closer to "keep typing" than "open another AI tab."

I am not trying to turn every writing moment into a chat session. I want the AI part to feel like a small layer on top of the text box you already picked.

There are real tradeoffs with that approach. Local models need a download. Performance depends on the Mac. Some apps expose text fields better than others. But for private writing assistance, those tradeoffs feel worth it.

If you are curious, the app is here: https://www.getdraftkey.com/

The bigger lesson for me: with AI products, raw output quality is only one part of the product. The workflow around the output decides whether people actually use it.

DEV Community: Sergey Levine

The hardest part of a local AI writing assistant is not the model