I'm curious if you specifically have a sense of where you draw the line in terms of your interest in AI assistants in your editor and/or CLI that work fast, or are maximally smart.
It's hard to pass up the smartest model, but speed has to matter too.
Can you articulate this?
Top comments (5)
I think speed is what gives the user productivity. I think it's based on how you define productivity. Is it how fast you can finish the product or is it something that you implement and learn as you go?
I tend to see productivity as working faster in a way that it reduces redundancy. For example, if I wrote a specific block of code before, instead of typing it out, I ask the AI to write it for me. That way, it saves me time.
The honest answer is: it depends on where in the workflow the agent is sitting.
For inline autocomplete and small edits speed wins completely. Any latency breaks the flow. I'd rather have a fast, slightly dumber suggestion than a brilliant one that takes 3 seconds to appear.
But for anything involving reasoning across files, architecture decisions, or debugging something non-obvious I'll wait. The cost of a fast wrong answer is higher than the cost of a slow right one. A quick confident hallucination in a complex debugging session can send you down the wrong path for 30 minutes.
So I've started thinking of it less as speed vs smarts and more as: reversibility of the task. Easy to undo? Give me speed. Hard to undo? Give me the smartest model available.
My day-to-day tasks don't require too much intelligence so I'm generally optimizing for speed to get things out the door. Perhaps one day I'll have something more complex to work on.
Speed matters until it doesn't. For scaffolding, boilerplate, anything I already know the answer to. fast is fine. For architecture decisions, anything that touches how the system is structured or how components talk to each other — I want the smartest model in the room, full stop.
The expensive mistakes aren't in the code that's obviously wrong. They're in the code that looks right but made a decision I didn't authorize. A faster model gets you to that mistake quicker.
It seems to me that the most important thing about AI is its ability to process long context. Speed may vary. AI largely depends on the hardware it runs on, so this factor may be ambiguous.