I built a small language model from scratch. Here's how it works.

player6 — Sat, 17 Oct 2026 07:53:09 +0000

I've been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.

The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.

I made a video walking through the project and showing the model running.

YouTube breakdown:

https://www.youtube.com/watch?v=zraZKknc7LQ

GitHub repository:

https://github.com/Harper-16/Orin_AI

I'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch.

What would you improve first?

DEV Community: player6

** I built a small language model from scratch. Here's how it works.**

I built a small language model from scratch. Here's how it works.