Fine-tuning and how it works(The Core "Why" and "How")

Fine-tuning refers to the process of taking a pre-trained language model and further training it for a specific task or domain-specific dataset. It basically improves the performance of a foundation model by allowing the model to adapt its knowledge and capabilities to better suit the requirements of the business use case.

Use Cases

Let's consider these use cases. If you are working on a task that requires industry knowledge, you can take a pre-trained model and fine-tune it with industry data. If the task involves medical research, for example, the pre-trained model can be fine-tuned with articles from medical journals to achieve more contextualized results.

There are two ways to fine-tune a model;

👉 Instruction fine-tuning, uses a dataset of prompt-response pairs on how the model should respond to a specific instruction.

👉 Reinforcement learning from human feedback (RLHF), provides human feedback data, resulting in a model that is better aligned with human performance.

How fine-tuning works

Step 1: Start with a pre-trained language model that already understands language at a high level.

Step 2: Gather and clean the data relevant to your specific domain.

Step 3:Add task-specif layers

Step 4: Fine-tune the model

Step 5: Evaluate and iterate: Test the model against a "held-out" set of data to ensure it has improved without losing its general capabilities.