Yep, the model has performance near gemini-1.5 I think, and it can run easily on a single GPU, which is a great advantage for local models!
I ran the 1 billion parameter model on my laptop at about 20 TPS (my laptop is weak) and for a 1b model is does really good.
wow, this benchmark is crazy. from what i heard it is about gemini-1.5 level but open source?
Yep, the model has performance near
gemini-1.5I think, and it can run easily on a single GPU, which is a great advantage for local models!I ran the 1 billion parameter model on my laptop at about 20 TPS (my laptop is weak) and for a 1b model is does really good.
1 billion sounds really big lol, idk if i can run that
1b is actually fairly small. 70b is really large, and 671b is massive (for example, DeepSeek or ChatGPT).
ok, maybe i will try it :D
Awesome! I'd recommend Ollama for running it.
ollama.com
It makes it pretty easy to run it in your terminal.
Curious what you all think!
good post :)
Thank you! ❤️