Tried jingyaogong/minimind: Train a 64M LLM in Two Hours

jingyaogong/minimind is gaining serious attention today, with +472 GitHub stars. The reason is simple: it turns LLM training from an opaque, multi-node infrastructure problem into a readable, runnable project.

MiniMind is a compact educational implementation for training a roughly 64M-parameter language model from scratch in about two hours. It covers the practical pipeline: tokenizer preparation, pretraining, supervised fine-tuning, preference optimization, inference, and a lightweight web UI. Instead of hiding everything behind frameworks, the repository exposes the pieces developers actually need to inspect when a loss curve explodes or generation quality collapses.

The useful part is not claiming that 64M parameters will replace frontier models. It is that MiniMind gives you a fast local lab for understanding model behavior, validating datasets, and testing training changes before spending real GPU budget.

For a practical setup, use MiniMind locally for training and route stronger-model evaluation or synthetic-data generation through an OpenAI-compatible gateway.

export OPENAI_BASE_URL="https://b-lost.com/v1"
export OPENAI_API_KEY="your_b_lost_key"
export OPENAI_MODEL="claude-fable-5"

curl "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-fable-5",
    "messages": [
      {"role": "user", "content": "Create 20 concise instruction-answer examples for a tiny Python coding model."}
    ]
  }'

That output can become seed material for MiniMind SFT experiments. Keep the generated dataset small, inspect it manually, and avoid poisoning a tiny model with repetitive synthetic examples.

If your workflow uses Anthropic-style clients, B-Lost also supports the native /v1/messages format. That matters for repeated long system prompts: Prompt Caching can reduce cache-hit prompt cost by up to 90%, which is useful when running the same evaluator across many MiniMind checkpoints.

The relay’s standard API shape also makes it straightforward to use from Cursor, Cline, Roo Code, Aider, NextChat, or LibreChat through a custom base URL. For quick iteration, that is a cleaner split: train the small model locally, then use claude-fable-5 as the external judge, data generator, or debugging assistant.