Following up on my concept for a Lightweight AI Tutor for NCERT Class 12 — a compact (15–25M parameter) GPT-style model trained from scratch on NCERT textbooks, notes, and Q&A for offline-friendly personalized learning.
Here’s the recommended training setup I’m planning to use:
ByteLevelBPETokenizer with 30,000 vocab size
Model: 6 layers, 6 attention heads, embedding size 384
Dataset: custom size text, block size 128 tokens
Training args: 100 epochs, batch size 2, gradient accumulation 2, checkpoint every 1000 steps, logging every 100 steps
On my current system, I tested a smaller run with 5 KB of text for 60 epochs in about 1.5 minutes. While these settings are ideal, running larger datasets and epochs takes significantly longer.
These settings can be adjusted as needed, and contributions or resource support are always welcome to help this project grow.
Excited to continue developing this AI-powered learning tool for offline accessibility.
hashtag#AI hashtag#NLP hashtag#MachineLearning hashtag#DeepLearning hashtag#Transformers hashtag#EducationTech hashtag#HuggingFace hashtag#LightweightAI
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)