In this video, you will learn about Arcee-Lite, a small yet powerful 1.5B model created with Distilkit, an open-source project for model distillation. Arcee-Lite outperforms Qwen2 1.5B and is currently the best 1.5B model.
First, I run an 8-bit version on my M3 MacBook with ollama and OpenWeb UI. Then, I deploy the model on AWS with Amazon SageMaker. I run both synchronous and streaming inference. I also show you how to use the OpenAI Messages API, allowing you to invoke the model with the OpenAI prompting format.
Top comments (0)