Introduction
Firstly, it is important to understand why we need to adapt a model and what adapting a model actually means.
Here, a model refers to a foundation model, which is trained on a very large amount of general-purpose data. These models are capable of handling a wide variety of tasks, but they are not optimized for any specific use case by default.
Adapting a model means customizing a foundation model for a specific use case so that we can leverage its capabilities more effectively and fulfill a defined purpose.
Let us assume a use case where we need to build a customer support chatbot backed by an LLM. For this, we would start by selecting a foundation model and then customizing it so that it can directly interact with customers, resolve their issues, or take appropriate actions.
To achieve this, there are two main approaches. These approaches are not alternatives, but instead serve different use cases, depending on the requirements.
Prompt-Based Techniques
The first technique used to adapt a model is prompt-based adaptation. In this approach, there is a middle layer between the user’s query and the LLM.
When a user submits a question, this middle layer adds:
- Additional context
- Instructions
- Constraints or rules
along with the original user query. This combined prompt is then sent to the LLM. Based on these instructions and context, the model generates a response that is more aligned with the expected behavior.
There are multiple prompt-based techniques, including but not limited to:
- Zero-shot Prompting
- Few-shot Prompting
- Role Prompting
- Retrieval-Augmented Generation (RAG)
- Tool or Function Calling
Limitations of Prompt-Based Techniques
One major limitation of prompt-based techniques is inconsistency. This approach does not reliably enforce behavior, which means the output can vary even when similar instructions are provided. As a result, the response is not always guaranteed to follow the expected structure or tone.
Another limitation is the reduction of the available context window. Since instructions and additional context need to be sent with every request, they consume a good part of the model’s context window.
Fine-Tuning
Another strategy to adapt a model is fine-tuning. This approach requires more technical knowledge and high-quality data compared to prompt-based methods.
In fine-tuning, the weights of the foundation model are updated so that the model learns to behave in a specific way by default. Instead of guiding the model through instructions at runtime, we change how the model responds internally.
Prompt-based techniques can be compared to giving instructions to a smart generalist, whereas fine-tuning is like sending that generalist back to school to become a specialist. Instead of reminding them what to do every time, their training changes how they think and respond by default.
Some common fine-tuning methods include:
- Supervised Fine-Tuning (SFT)
- Reinforcement Learning from Human Feedback (RLHF)
- Instruction Fine-Tuning
With fine-tuning, the model becomes highly reliable. It consistently follows a fixed tone, response structure, and style. Additionally, it can learn company-specific jargon, terminology, and phrasing, making it more suitable for long-term and stable use cases.
Conclusion
In general, prompt-based techniques make more sense when the instructions given to the model change frequently. For example, product FAQs or dynamic content that evolves over time can be efficiently handled using prompt-based methods.
On the other hand, fine-tuning is more suitable for behaviors that remain consistent over long periods. This includes company policies, tone of communication, customer interaction rules, and compliance requirements.
In practice, a hybrid approach often works best. Parameters that are expected to remain stable for a long time can be used to fine-tune the foundation model. At the same time, variables that evolve more frequently can be provided dynamically through prompts at inference time.
This further reinforces the idea that prompt-based adaptation and fine-tuning are not alternatives, but complementary techniques, each with specific and well-defined use cases.
Citation
This blog is inspired by the book “AI Engineering” by Chip Huyen. If you want to go beyond high-level concepts and understand how real-world AI systems are designed, adapted, evaluated, and deployed, this book is an excellent resource. It covers model adaptation, system design, data considerations, production AI workflows, etc., making it valuable for developers building practical AI applications.
Top comments (0)