Forem

Cover image for 🧠Generative AI - 3
Abheeshta P
Abheeshta P

Posted on

🧠Generative AI - 3

How Are Generative AI Models Trained? 🏋️‍♂️

Generative AI models like GPT are trained in two main stages:

1. Unsupervised Pretraining 📚:

  • The model is fed massive amounts of text data (e.g., 45TB of text for GPT models).
  • The model learns patterns, language structures, grammar, semantics and general knowledge by predicting the next word/token in a sentence without labeled data.
  • This results in 175 billion parameters for models like GPT-3.

2. Supervised Fine-Tuning 🎯:

  • After pretraining, the model is fine-tuned on smaller, labeled datasets for specific tasks (e.g., summarization, sentiment analysis).
  • Fine-tuning ensures the model generates more accurate and task-relevant outputs. Eg : text summarization, Language translation etc

GPT-3

📝 Stay tuned in this learning journey for more on generative AI! I'd love to discuss this topic further – special thanks to Guvi for the course!

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (1)

Collapse
 
abheeshta profile image
Abheeshta P

Next will be on RAG stay tuned!!

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay