DEV Community: neha

Long Long Ago — The History of Generative AI

neha — Fri, 17 Oct 2025 16:11:56 +0000

We have all seen it and are mesmerised by it. ChatGPT can write new essays, generate images, create stories, create art, write code, be our friend, and give advice as a mentor — It is doing it all. It seems like magic, but this is a result of decades of ideas, technological evolutions and small steps taken behind the scenes — while the Generative AI models emerged as winners on the grand stage of AI.

How did we get here? I will trace through the story of how we got to these powerful LLMs. The journey of how we got here draws a lot of parallels from trying to mimic how the actual human mind works.

Symbolic AI: In the Beginning [1950s - 1980s]
The very first experiment with AI was not smart but they were the front benchers — the very obedient set of students who followed rules exactly as told.

The early version of AI followed statements which seemed more like the If-Else clause — “If X happens, do Y.” This is referred to as — Symbolic AI.

For example —
We give the robot some rules to follow:

If it is yellow and curved → it is a banana.
If it is red and round → it is an apple.
If it is orange and round → it is an orange.

Now,
If the fruit is a yellow and curved → “It’s a banana!”
If the fruit is red and round → “It’s an apple!”

Symbolic AI uses two concepts — Symbolic representation and Rule-based Inference. Some real-world examples of Symbolic AI include —

Expert systems for medical diagnosis
Knowledge graphs and Ontologies
The initial version of Chatbots — ELIZA (1960s) was the first chatbot designed to mimic a therapist.

Machine Learning — Learning Takesover the Rules [1990–2010s]
The next major shift came from the question — “Can machines learn from examples instead of a definitive set of rules? “

This question and experimentation led to Machine learning — Instead of giving rules, data was fed to systems to figure out patterns and learn from them. Emails labelled as Spam or Brain Images labelled as having a tumour. These resulted in task-specific intelligent systems like —

Spam filters
Recommendation engines
Image Recognition
Speech Recognition

Why is this better than before? These systems are more adaptive of real-world scenarios.
But what was missing here? These models were very specific to the task and could navigate only their area of expertise. They do not have any general intelligence.

The Big Shift: Learning From Everything [2017s — 2020]
The next paradigm shift was to learn from everything, but what changed now? Why weren't we training models on everything before?

With time technology evolved there were two aspects to LLMs — The brain and the Muscle.
The muscle — GPUs became faster, and thousands of cores for parallel computation possible.
The brain — In 2017, Google researchers published — “Attention is All You Need”. This paper introduced the concept of transformers.

Enter — Transformers

Before 2017, the models which relied on RNNs (Recurrent Neural Networks) read everything sequentially — word by word . Transformers “transformed” this approach, they could look at sentences.

Let us look at the key concepts which made this possible —

Self-attention
Multi-head Attention
Positional Encoding
Feedforward Layers
Layer Normalization

Let's get into the details of each of these. I will use the below sentence to walk through each of these

“ Jim Halpert loves the mountains but Dwight Schrute loves the beach”

Self-Attention: Understanding the importance
In the context of each word, which word is more important? Self-Attention, allows the model to understand which word is important in a particular word's context. It computes the “attention” score for every other word in a sentence in relation to itself.

Let's see this with our example sentence now. Here, the attention score for “Jim” is high for the word “mountains” and the attention score for “Dwight” is high for the word “beach”. The model gives more weight to the word “mountain” in the context of “Jim” and more weight to the word “beach” in the context of “Dwight”. The model at every word asks the question — “Which other words matter the most to me right now”. With this, the model concludes — “Jim Halpert → mountains” and “Dwight Schrute → beach”

Multi-head attention: Many perspectives one problem

Multi-head attention is self-attention done multiple times with a different aspect as the focus point for each iteration. This brings different patterns into the model. It is similar to different people interpreting the sentence based on their expertise, one could focus on syntax, one could focus on meaning etc. Let's use the same example —

The first head might focus on “Jim Halpert → loves → mountains”
The second head might focus on the symmetry between the two phrases —_ “Jim Halpert → loves → mountains”_ and “Dwight Schrute → loves → beach”
The third head might focus on the “but” clause “but Dwight Schrute loves the beach” clause and the contrast in the sentence is trying to bring it.

This can go on, more the heads more the viewpoints the model has. GPT-3 used around 96 attention heads.

Positional Encoding: Giving a sense of order

The above concepts and models by themselves don't understand the sequence of the words in a sentence. The model initially doesn't understand the difference between — “Jim Halpert loves mountains” and “mountains love Jim Halpert”. The positional encoding makes sure that Jim comes first then the mountains and then Dwight and finally the beach.

Feedforward layers: Generation is here

After attention layers, *Feed“forward” * layer registers the learnings as abstractions which get used for generations, summarization etc. The feedforward layer leaves out the details and captures the essence. In our example something as simple as — people with different tastes. The model arrives at these abstractions after learning from many examples.

Layer normalization

Training with so many layers can be extremely unstable and unpredictable. A specific set of data can add bias into a layer which can percolate and increase exponentially down the newer layers. Layer normalization normalizes the learning for each layer. This layer is applied after attention and feedforward layers.

Here is the final sequence in which LLMs are trained —

*Residual connection adds the output for the step back to the input.

Conclusion: The Journey Has Just Begun
From the obedient rule-followers as the Symbolic AI to machines that can learn from data, and now to models that can Generate and Create — the journey of Generative AI has been a series of small, powerful steps coming together.

Transformers have changed the game. With self-attention, multi-head perspectives, and an understanding of context, these models can now write stories, answer questions, generate art, and mimic creativity.

We are just at the beginning, the systems are evolving fast, there is a lot more coming next Agentic AI, AI and human in the loop, Personalized AI and more.

The story so far was been magical and what awaits next are yet to be imagined.

Top 5 Prompt Engineering Techniques for LLMs in 2025

neha — Thu, 16 Oct 2025 15:40:08 +0000

As LLMs continue to get better, the debate — Is prompt engineering dead or becoming even more important is totally on. With today’s models, the extent to which prompting is needed is drastically reducing, but no matter how smart LLMs become, they still need directions on various ‘whats’, ‘hows’, 'whys’, and 'whens.' The guidelines become important to get the outputs you are looking for. We all have experiences with LLMs where the results are outright unacceptable. *Here are 5 prompt engineering techniques that have always worked and are very much relevant with the newest models, be it GPT, Gemini or Claude : *

1. Few Shots
Few-shot is a prompting technique where you include examples as part of your prompt. This provides LLM guidelines on how to approach a task and the format in which we expect the output. Few-shot prompting is 80% more efficient than zero-shot prompts (without any examples).

Sample Template

<Task> Use examples below as ref :
<Example1>
<Example2>

How many examples are the right number of examples? — The goal with few-shot prompts is to cover enough so that LLM can respond to a similar new task. Trying out and experimenting is the best approach here. Starting with 2 or 3 examples is a good idea, and depending on the complexity and different use cases that have to be covered, more can be included. More complex tasks, which need to cover a lot of cases, can use techniques like dynamic few-shot. For dynamic few-shot inclusion, refer to —

Dynamic few-shot prompting | Technology Radar | Thoughtworks United States

Dynamic few-shot prompting builds upon few-shot prompting by dynamically including specific examples in the prompt to guide the model's responses. Adjusting the number and relevance [...]

thoughtworks.com

With dynamic few-shots, we provide LLM just the relevant examples instead of overloading it with a lot of examples and leaving LLM to figure out which one to use as a reference. Dynamic few-shot approach is great of complex tasks, resulting in better outputs at the same time, reducing cost and latency.

2. Persona / Role-Based Prompting

In Persona or Role-based prompting, you direct the LLM to act as an expert or assume the role of a domain expert, for example, a potential client or an employer and then respond to the task. This allows LLMs to wear a specific hat and focus on the task.

Sample Template

You are an expert in <Persona>. <Task>

Without the persona, based LLMs are free to attempt the task from multiple directions, of which not all might be relevant to you. Defining the persona pushes the LLM to attempt the task with a more focused and relevant lens and, hence, gets better outputs.

3. Chain of Thought
With Chain of Thought (COT) prompting, you prompt LLMs to do step-by-step thinking while generating the output of a task. This technique is very efficient for logic-based tasks, which is usually a weak area for LLMs. COT allows LLMs to break down a problem into smaller problems before solving them. Explicitly prompting LLMs to think through each step allows LLMs to engage in logical thinking at each step, making the final output literally “Add up”. COT is great for tasks which need reasoning and logic, like SQL generation or maths problems or tasks which might need multiple steps to arrive at the correct output. The most common way of making your prompt COT-driven is by prompting the LLM to explain each step or walk through each step.

Sample Template

1. <Task prompt >. Explain your answer step by step.
2. <Task prompt >. Think through the output step by step.

While COT is great for complex/reasoning tasks, including COT can also give a deeper insight into LLM's thought process while arriving at the output. Experiment with mathematical problems to see how LLMs tend to give wrong answers without COT.
COT can be used with Few-Shot prompting — Few-Shot COT. Few-shot COTs are usually more consistent with outputs.

4. ReAct — Reasoning and Acting
ReAct prompting combines reasoning and action. ReAct allows LLMs to reason and then take an action based on the reasoning. ReAct prompting technique becomes extremely important when a task needs a series of tools or functions to be invoked. While LLMs have natively become enabled with tool/function calling, making ReAct part of your prompt has many advantages. The reasoning and actions are explicit with ReAct. Making these steps explicit can make debugging easier. ReAct with COT is a powerful combination that allows reasoning at each sub-problem level.

Sample Template

Task
Follow this exact loop:
Thought: <brief reasoning about next step>
Action: <ToolName>[Arguments]
Observation: <result returned by the operator>

Tools available
<tools>

Multiple iterations of thought-action-observation can be enforced till a task is completed. I have not included an example screenshot, as the comparison here needs to trigger the LLM function calling. I will be attaching a sample code link here soon.

5. Tree of Thoughts (TOT)
Tree of Thoughts, or TOT, is a supercharged COT. While COT follows a single chain of reasoning, TOT branches out and weighs multiple possible options before arriving at the output. TOT is extremely useful for tasks which need to consider multiple variations.

Sample Template

<Task>
Consider multiple options to approach the task
Option 1: <Option>
Option 2: <Option>
Evaluate the options and decide on the best one.
<Result>

TOT is ideal for examining multiple scenarios, weighing pros and cons and picking output which is apt for the given task. TOT combined with Persona/Role-based prompting can also be used to approach a task from multiple personas to get a holistic picture before arriving at the output.

Summary Snapshot
Below is the snapshot of the 5 prompting techniques, along with when to use them and which techniques together yield even better results. Always keep the prompt clear and concise. Provide enough context in the prompt for LLMs to respond better. And always Verify— Fix — Repeat.