Thanks for the tip! I can't confirm the exact number, but lower prompt overhead definitely helps small models since the system prompt eats into the context window.
Great question! In this demo everything ran locally, so I haven't tested a split yet. Local for reading and boilerplate, and a cloud model for the final review, sounds like the sweet spot. I am planning on trying it and cover it in a follow-up.
@mohammed_sebaa_2610 Local for the reads, cloud for the final review. A missed error costs the most at the review, so that is where the better model earns its price.
2.1.300 reportedly cut 19,045 prompt tokens, handy for 9b models.
Thanks for the tip! I can't confirm the exact number, but lower prompt overhead definitely helps small models since the system prompt eats into the context window.
Local is the right place for the cheap steps. A lot of a session is reads, so the split matters. Does the review step still go to the cloud model?
Great question! In this demo everything ran locally, so I haven't tested a split yet. Local for reading and boilerplate, and a cloud model for the final review, sounds like the sweet spot. I am planning on trying it and cover it in a follow-up.
@mohammed_sebaa_2610 Local for the reads, cloud for the final review. A missed error costs the most at the review, so that is where the better model earns its price.
Great job, Mohammed! Your explanation is so clear and helpful for beginners. Keep it up, and best of luck! 😊
Thank you! Means a lot, and glad you found it helpful!
Clear and simple
Thank You !
Well done, bro! 👏
Thank you !
I found this very useful. Thank you for sharing this clear and practical guide! 🙌
You're very welcome! Glad you found the guide useful! 🙌
Great tutorial. A practical way to explore local AI-assisted coding with Ollama and Claude Code. Thanks for sharing
You're welcome! Thanks for reading, glad it was useful!