So after I saw a video of Llama3 8b doing 800-1000 tokens/s I instantly thought it would be phenominal to have that on my laptop instantly giving me responses that I could select or discard with the added side benefit would be to learn some new tech.
I have once coded a small AI trained on like 100.000 items trying to predict housing prices, let me say it was not great.
But again I don't have billions upon billions of records to train on nor the compute to train a decent model but I like the control over my coding environment.
The first goal
Having llama 8b running locally autocompleting code snippets.
Yeah I know the barrier are low but is most def a useful starting point. I personally dont believe in setting unobtainable goals because failing at something hard is not as great as succeeding at small increments.
Lastly I intent to make a guide on my complete journey sub and like to get notified when new content is published
Banner video is from groq.com a great service where you can test these open source models like llama not sponsored by them just wanted to credit
Top comments (0)