DEV Community

SelfHost Pilot
SelfHost Pilot

Posted on Originally published at selfhostpilot.com

MoE VRAM requirements: total vs active parameters

Sparse MoE models have two sizes. Active parameters set speed; total parameters set memory. How to size any MoE model, using Step 5 Preview as an example.

Originally published on SelfHost Pilot - practical self-hosted software and private AI, tested on a real server.

Top comments (0)