DEV Community: NelimaL

Window Functions vs Aggregate Functions Made Easy

NelimaL — Sun, 20 Sep 2026 17:38:19 +0000

The first time in a SQL class, aggregate functions and window functions can seem confusing because both can perform calculations such as COUNT(), SUM(), and AVG().

The key difference is simple:

Aggregate functions combine rows and return a summary, while window functions perform calculations across related rows without removing the individual rows.

What Are Aggregate Functions?

Aggregate functions perform calculations on multiple rows and return a single result.
Common aggregate functions include:

COUNT()
SUM()
AVG()
MIN()
MAX()

For example, suppose we want to calculate the average salary in each industry.

SELECT
    industry,
    AVG(salary) AS average_salary
FROM job_salary_prediction_dataset
GROUP BY industry;

The GROUP BY divides the data into groups based on industry.
The AVG() function then calculates the average salary for each group.
The result might look like this:

industry	average_salary
Healthcare	145760
Finance	145802
Technology	145864
Retail	145400

The important thing to notice is that the original employee-level rows are no longer displayed.
Instead, we get one row for each industry.

What Is a Window Function?

A window function performs a calculation across a set of related rows while keeping the individual rows in the result.
Window functions use the OVER() clause.
The basic syntax is:

function_name() OVER (
    PARTITION BY column
    ORDER BY column
)

For example, we can calculate the average salary for each industry while still displaying each individual record.

SELECT
    job_title,
    industry,
    salary,
    AVG(salary) OVER (
        PARTITION BY industry
    ) AS industry_average
FROM job_salary_prediction_dataset;

The result could look like:

job_title	industry	salary	industry_average
AI Engineer	Healthcare	109413	145760
Data Analyst	Healthcare	150000	145760
Software Engineer	Technology	180000	145864
Data Scientist	Technology	160000	145864

Notice something important.
The individual salary is still available, but we have also added the average salary for that employee's industry.

This is the major advantage of a window function.

Understanding the OVER() Clause

The OVER() clause tells SQL that we want to perform a calculation across a group of rows without collapsing them.
For example:

AVG(salary) OVER (
    PARTITION BY industry
)

The PARTITION BY industry means:

Calculate the average separately for each industry.

It is similar to GROUP BY, but it behaves differently.
With:

GROUP BY industry

SQL produces one result for each industry.
With:

PARTITION BY industry

inside a window function, SQL calculates the value for each industry but keeps the individual rows.

GROUP BY vs PARTITION BY

Let's compare them.

Using GROUP BY

SELECT
    industry,
    AVG(salary) AS average_salary
FROM job_salary_prediction_dataset
GROUP BY industry;

This produces:

Healthcare    145760
Technology    145864
Finance       145802

The employee-level records are grouped together.

Using a Window Function

SELECT
    job_title,
    industry,
    salary,
    AVG(salary) OVER (
        PARTITION BY industry
    ) AS average_salary
FROM job_salary_prediction_dataset;

This produces something like:

AI Engineer          Healthcare    109413    145760
Data Analyst         Healthcare    150000    145760
Software Engineer    Technology    180000    145864
Data Scientist       Technology    160000    145864

The original rows remain.

Why Are Window Functions Useful?

Window functions are particularly useful when we want to compare an individual row with a group or calculate values across rows.
For example, we can compare an employee's salary with the average salary of their industry.

SELECT
    job_title,
    industry,
    salary,
    AVG(salary) OVER (
        PARTITION BY industry
    ) AS industry_average,
    salary - AVG(salary) OVER (
        PARTITION BY industry
    ) AS difference_from_average
FROM job_salary_prediction_dataset;

Now we can see:

Job Title	Industry	Salary	Industry Average	Difference
AI Engineer	Healthcare	109413	145760	-36347
Data Analyst	Healthcare	150000	145760	4240

The window function allows us to perform the calculation while keeping the employee's original salary.

Ranking with Window Functions

Window functions are not limited to AVG().
One of their most common uses is ranking.
For example, we can rank salaries from highest to lowest within each industry.

SELECT
    job_title,
    industry,
    salary,
    RANK() OVER (
        PARTITION BY industry
        ORDER BY salary DESC
    ) AS salary_rank
FROM job_salary_prediction_dataset;

Here:

PARTITION BY industry

creates a separate ranking for each industry.
And:

ORDER BY salary DESC

puts the highest salary first.
The result could look like:

Job Title	Industry	Salary	Salary Rank
Data Scientist	Technology	200000	1
Software Engineer	Technology	180000	2
Data Analyst	Technology	160000	3
AI Engineer	Healthcare	190000	1
Data Scientist	Healthcare	170000	2

Notice that the ranking starts again when we move to another industry.

RANK(), DENSE_RANK() and ROW_NUMBER()

SQL provides several window functions for ranking.

RANK()

RANK() OVER (ORDER BY salary DESC)

If two employees have the same salary, they receive the same rank.
For example:

Salary     Rank
200000       1
190000       2
190000       2
180000       4

Notice that rank 3 is skipped.

DENSE_RANK()

DENSE_RANK() OVER (ORDER BY salary DESC)

DENSE_RANK() also gives employees with the same salary the same rank, but it does not skip the next rank.

Salary     Rank
200000       1
190000       2
190000       2
180000       3

ROW_NUMBER()

ROW_NUMBER() OVER (ORDER BY salary DESC)

ROW_NUMBER()` gives every row a unique number.

text Salary Row Number 200000 1 190000 2 190000 3 180000 4
Even if two employees have the same salary, they receive different row numbers.

Window Functions for Running Totals

Another common use of window functions is calculating a running total. Suppose we have a table containing monthly salary payments.
We could calculate the cumulative salary using:

sql SELECT month, salary, SUM(salary) OVER ( ORDER BY month ) AS running_total FROM salary_payments;
The result could look like:

Month	Salary	Running Total
January	10,000	10,000
February	12,000	22,000
March	11,000	33,000
April	13,000	46,000

The SUM() calculates the total, while the OVER() clause turns it into a window calculation.

Aggregate Function vs Window Function

The easiest way to remember the difference is:

Aggregate function

sql SELECT industry, AVG(salary) FROM job_salary_prediction_dataset GROUP BY industry;
Many rows → fewer rows
It summarizes the data.

Window function

sql SELECT job_title, industry, salary, AVG(salary) OVER ( PARTITION BY industry ) AS industry_average FROM job_salary_prediction_dataset;
Many rows → same rows + additional calculation
It analyzes the data without removing the individual rows.

When Should You Use Each?

Use aggregate functions when you want a summary.
For example:

What is the average salary by industry?
How many employees are in each industry?
What is the highest salary?
What is the total salary?
What is the minimum salary?

Use window functions when you want to analyze individual rows in relation to other rows.
For example:

What is each employee's salary rank?
How does an employee's salary compare with the industry average?
What is the running total?
What was the previous employee's salary?
What is the difference between the current and previous value?
Who is the highest-paid employee in each industry?

Conclusion

The difference between aggregate functions and window functions becomes much easier when you remember what happens to the rows.
Aggregate functions summarize data and usually reduce the number of rows.( Keyword 'Group by')
Window functions perform calculations across related rows while keeping the original rows.( Keyword 'OVER()')

Once you understand this difference, functions such as RANK(), ROW_NUMBER(), LAG(), LEAD(), SUM() OVER() and AVG() OVER() become much easier to understand and apply to real-world data.

Neural Networks Explained: From Inspiration to Implementation

NelimaL — Sun, 13 Sep 2026 06:38:19 +0000

Neural networks are one of the most important ideas behind modern AI. They power applications such as image recognition, speech assistants, recommendation systems, translation tools, and many generative AI systems.

But despite their impressive capabilities, the basic idea is surprisingly simple. In this article, we'll explore:

What a neural network actually is
The main components of a neural network
How neurons and layers work
What weights and biases do
Activation functions
How a network learns
Backpropagation and gradient descent
A simple example

1. The Inspiration: The Human Brain

The idea behind artificial neural networks was inspired by the human brain. Our brains contain billions of biological neurons. These neurons receive signals, process them, and pass signals to other neurons.
For example, when you see a cat:

Eyes
  ↓
Visual processing
  ↓
Features detected
  ↓
Brain combines the information
  ↓
"That's a cat!"

The brain doesn't have a single "cat detector."
Instead, many neurons work together to recognize different patterns such as shapes, edges, colors, and textures.
Researchers wondered:

Could we build a mathematical system that learns patterns in a similar way?

This question helped lead to the development of artificial neural networks.
It's important to note that artificial neural networks are not accurate simulations of the human brain. They are mathematical and computational models that were inspired by some aspects of biological neurons.

1. What Is a Neural Network?

A neural network is a machine learning model made up of interconnected mathematical units called neurons.
At a high level, it looks something like this:

Input Layer       Hidden Layers        Output Layer

   x₁ ───────┐
             ├──> ○ ───┐
   x₂ ───────┤         │
             │    ○ ───┼──> ○
   x₃ ───────┤         │
             ├──> ○ ───┘
   x₄ ───────┘

The network receives some information as input, processes it through one or more hidden layers, and produces an output.
For example, suppose we're building a system that determines whether an image contains a cat.
The input could be:

Image → Neural Network → "Cat"

The network doesn't receive the concept of "cat" directly.
It receives numerical data representing the image and learns useful patterns from examples.

2. The Three Main Types of Layers

A basic neural network consists of layers.

Input Layer

This layer receives data.
For example, if we're predicting whether a student will pass an exam, our inputs might be:

Hours studied
Attendance
Previous test score

These values become the input to the network.
For an image, the inputs might be pixel values.

Hidden Layers

Between the input and output are the hidden layers. These layers transform the information received from the previous layer.
A network might look like:

Input → Hidden Layer → Hidden Layer → Output

A neural network with many hidden layers is commonly called a deep neural network. That's where the term deep learning comes from.

Output Layer

The output layer produces the final prediction.
For example:

Input:
Hours studied = 8
Attendance = 90%

       ↓

Neural Network

       ↓

Output:
Probability of passing = 0.94

The model might interpret 0.94 as a 94% estimated probability of passing.

3. What Is a Neuron?

A neuron is essentially a small mathematical function. It receives several inputs, gives each input a different importance, combines them, and produces an output.

Imagine:

Input 1 ──┐
Input 2 ──┼──> Neuron ──> Output
Input 3 ──┘

A simplified neuron works like this:

output = activation(weighted inputs + bias)

Let's break that down.

4. Weights: How Important Is Each Input?

A weight controls how strongly an input influences a neuron.
Suppose we're predicting whether someone will buy a product.

We might have:

Age
Income
Previous purchases

The network could assign different weights to these inputs.
For example:

Age              × 0.2
Income           × 0.7
Previous purchase × 1.1

The actual numbers aren't chosen by us manually. During training, the neural network learns them.
A weight can be thought of as:

"How important is this input for my prediction?"
A large positive weight means an input tends to increase the neuron's result.
A negative weight can decrease it.

5. Bias: An Extra Adjustment

Neurons also have something called a bias.

The simplified equation for a neuron is:

z = w₁x₁ + w₂x₂ + w₃x₃ + b

Where:

x = input
w = weight
b = bias
z = combined result

The bias gives the neuron an additional value that allows it to shift its output.

You can think of it as an adjustable starting point.
Without biases, neural networks would be more limited in the functions they could learn.

6. Activation Functions

After calculating the weighted sum, a neuron usually applies an activation function.
Why?
Because without activation functions, stacking many layers wouldn't give us the powerful nonlinear behavior we want from neural networks.

An activation function takes a number and transforms it.
One popular activation function is ReLU.

ReLU

ReLU stands for Rectified Linear Unit.
Its rule is simple:

ReLU(x) = max(0, x)

So:

ReLU(-5) = 0
ReLU(-1) = 0
ReLU( 0) = 0
ReLU( 2) = 2
ReLU( 7) = 7

This simple function is widely used in neural networks.
Other activation functions include:

Sigmoid
Tanh
Softmax
GELU

Different architectures and tasks may use different activation functions.

7. Putting a Neuron Together

Let's combine everything.
Suppose we have two inputs:

x₁ = 2
x₂ = 3

And our neuron has:

w₁ = 0.5
w₂ = 0.2
b  = 1

First, calculate the weighted sum:

z = (2 × 0.5) + (3 × 0.2) + 1

Which gives:

z = 1 + 0.6 + 1
z = 2.6

Now apply ReLU:

ReLU(2.6) = 2.6

So the neuron outputs:

2.6

That's the basic building block of a neural network.

8. Connecting Neurons

One neuron isn't very useful for complex problems. The power comes from connecting many neurons together.

For example:

Input Layer

○   ○   ○   ○
 \  |  / \  |
  \ | /   \ |
   ○ ○ ○ ○
    \ | /
     \|/
      ○

Each connection has a weight. The neurons in one layer send their outputs to neurons in the next layer.
This allows the network to build increasingly useful representations of the input.

9. How Does a Neural Network Learn?

This is where things get interesting. Suppose we want a neural network to recognize cats.
We give it thousands of labeled images:

Image 1 → Cat
Image 2 → Not Cat
Image 3 → Cat
Image 4 → Cat
Image 5 → Not Cat
...

Initially, the network's weights are usually not useful. Its predictions might be terrible:

Correct answer: Cat
Network prediction: Not Cat

The network needs a way to measure how wrong it was.That's the job of a loss function.

10. Loss Functions

A loss function measures the difference between the model's prediction and the desired answer.

For example:

Expected:   1.0
Predicted:  0.2

The model is quite wrong.The loss will therefore be relatively high.
If instead:

Expected:   1.0
Predicted:  0.95

the loss should be much smaller.
The general goal of training is:

Minimize the loss.
In simple terms:

Make a prediction
      ↓
Measure the error
      ↓
Adjust the network
      ↓
Make another prediction
      ↓
Repeat

11. Backpropagation

But how does the network know which weights should change?
This is where backpropagation comes in. Backpropagation calculates how much each parameter contributed to the error.

The process roughly looks like this:

Input
  ↓
Forward pass
  ↓
Prediction
  ↓
Calculate loss
  ↓
Backpropagation
  ↓
Calculate gradients
  ↓
Update weights

The word "backpropagation" can sound intimidating, but the basic idea is straightforward:

Start with the error and work backward through the network to determine how the parameters should change.

13. Gradient Descent

Once we know how the parameters should change, we need a method for changing them. One common approach is gradient descent.
Imagine you're standing on a mountain and want to reach the lowest point.
You can't see the entire mountain, but you can determine which direction slopes downward. So you take a small step downhill.
Then another.
And another.
Eventually, you hopefully reach a low point.

Training a neural network works somewhat similarly.
The "height" represents the loss. We want to move toward lower loss.

Loss
 ^
 |\
 | \
 |  \
 |   \       ●
 |    \     /
 |     \   /
 |      \_/
 +----------------> Parameters

The gradient tells us the direction in which the loss changes. The optimizer uses this information to update the weights.

13. Learning Rate

The size of each update is controlled by the learning rate.
A very small learning rate might look like:

Step → Step → Step → Step → Step

Learning can be slow.
A very large learning rate might look like:

      ↗
   ↙     ↗
      ↙

The model could jump around and fail to settle into a good solution.
So choosing an appropriate learning rate is important.
Modern optimizers such as Adam can adapt the updates in useful ways and are widely used in practice.

14. The Training Loop

Putting the pieces together, training often follows this pattern:

       ┌──────────────┐
       │ Training Data│
       └──────┬───────┘
              ↓
        Forward Pass
              ↓
          Prediction
              ↓
        Calculate Loss
              ↓
       Backpropagation
              ↓
       Update Parameters
              │
              └───────────┐
                          ↓
                    Repeat many times

One complete pass through the training dataset is called an epoch.
A model might train for:

1 epoch
2 epochs
3 epochs
...
50 epochs

The exact number depends on the problem and the training setup.

15. What Does "Learning" Actually Mean?

It's easy to imagine that a neural network "understands" data in the same way humans do.
That's not quite what's happening.
The network is learning numerical parameters that help it make useful predictions.

For example, when training an image classifier, early layers might learn representations related to simple visual patterns, while deeper layers can combine those representations into more complex patterns.
The exact behavior depends heavily on the architecture, training data, objective, and optimization process.

16. Parameters vs Hyperparameters

You'll often hear these two terms.

Parameters

Parameters are values learned by the model during training.
Examples:

Weights
Biases

Hyperparameters

Hyperparameters are settings chosen by the developer or training process.
Examples include:

Learning rate
Number of layers
Number of neurons
Batch size
Number of training epochs

A simple way to remember it:

Parameters are learned. Hyperparameters configure the learning process.

17. What Is a Batch?

Training a model on millions of examples at once can be expensive.
Instead, training data is usually divided into smaller groups called batches.

For example:

Dataset = 10,000 examples

Batch 1 → 64 examples
Batch 2 → 64 examples
Batch 3 → 64 examples
...

The model processes a batch, calculates the loss, and updates its parameters.
The number of examples in each batch is called the batch size.

18. Why Do We Need GPUs?

Neural networks involve a huge number of mathematical operations.
Many of these operations can be performed in parallel.
GPUs are particularly good at this type of computation.
That's why modern deep learning often relies on GPUs or other specialized accelerators.
A simplified comparison is:

CPU
Good at many different types of tasks

GPU
Excellent at performing many similar numerical operations in parallel

This makes GPUs extremely useful for training large neural networks.

19. Different Types of Neural Networks

"Neural network" is a broad term. There are many architectures designed for different kinds of problems.

Feedforward Neural Networks

Information generally moves from input toward output without recurrent connections.

They're useful for many basic prediction tasks.

Convolutional Neural Networks (CNNs)

CNNs became especially important for computer vision.

They are designed to work effectively with spatial patterns such as those found in images.

They can learn features such as:

Edges
  ↓
Shapes
  ↓
Object parts
  ↓
Objects

Recurrent Neural Networks (RNNs)

RNNs were designed for sequential data.

Examples include:

Text
Speech
Time series

They process information while maintaining a form of state from previous steps.
While important historically, many modern language applications use transformer-based architectures instead.

Transformers

Transformers have become one of the most influential neural network architectures in modern AI.
They are heavily used in:

Large language models
Machine translation
Text generation
Vision systems
Multimodal AI

A key idea behind transformers is attention, which allows the model to determine which parts of the input are especially relevant when processing information.

20. Neural Networks and Large Language Models

Modern language models are built using neural networks, particularly transformer architectures.
Suppose you type:

The sky is usually

A language model processes the context and estimates likely continuations.
For example:

blue

The model doesn't simply retrieve a sentence from a database. During training, it learns statistical patterns and representations from enormous amounts of data. At inference time, it uses those learned parameters to generate predictions.

21. A Simple Mental Model

If you're just starting, you can think about a neural network like this:

              Neural Network

Input ──> Process patterns ──> Prediction
             ↑
             │
       Learned weights
             │
             ↑
        Training adjusts
        those weights

The most important concepts are:

Concept	Simple meaning
Neuron	Small mathematical processing unit
Weight	Controls the influence of an input
Bias	Additional adjustable value
Layer	Group of neurons
Activation	Adds nonlinear behavior
Loss	Measures prediction error
Gradient	Indicates how parameters affect loss
Optimizer	Updates parameters during training
Epoch	One pass through the training dataset
Batch	Small group of training examples
Parameter	Value learned during training

22. Why Neural Networks Are So Powerful

The real power comes from combining many simple operations.
A single neuron can perform a relatively simple calculation. Millions or billions of parameters arranged in layers can represent much more complicated relationships. This gives neural networks the ability to learn patterns that would be extremely difficult to program manually.

Instead of writing:

IF this condition
AND this condition
AND that condition
THEN predict X

we can provide data and an objective and allow the model to learn useful parameters.

A neural network can perform extremely well while still making mistakes.
Problems can include:

Poor-quality training data
Biased data
Overfitting
Distribution shifts
Incorrect predictions
High computational costs
Difficulty explaining some decisions

A model is only as reliable as the problem setup, data, evaluation, and deployment practices surrounding it.

24. The Big Picture

Let's summarize the complete process.

                 TRAINING

Training Data
     ↓
Neural Network
     ↓
Prediction
     ↓
Compare with expected answer
     ↓
Loss
     ↓
Backpropagation
     ↓
Gradients
     ↓
Optimizer
     ↓
Update weights
     ↓
Repeat

After training, we can use the learned model on new data:

New Data
   ↓
Trained Neural Network
   ↓
Prediction

That's the core idea behind many neural-network-based machine learning systems.

Conclusion

Neural networks can seem complicated because modern AI systems contain enormous numbers of parameters and sophisticated architectures.

But the fundamental ideas are approachable.

A neural network:

Receives numerical inputs.
Processes them through layers of neurons.
Uses weights and biases to transform information.
Applies activation functions.
Produces a prediction.
Measures how wrong the prediction was.
Uses backpropagation to calculate gradients.
Updates its parameters using an optimizer.
Repeats this process many times.

The result is a system that can learn useful patterns from data rather than requiring every rule to be explicitly programmed.

And that's the central idea behind neural networks:

Give a model data, define what you want it to optimize, and use optimization to learn parameters that make useful predictions.

Once these fundamentals make sense, topics like CNNs, transformers, attention mechanisms, embeddings, and large language models become much easier to understand.

What's Next?

If you're learning neural networks for the first time, a good progression is:

Machine Learning Basics
        ↓
Neural Networks
        ↓
Backpropagation
        ↓
PyTorch / TensorFlow
        ↓
CNNs
        ↓
Transformers
        ↓
Large Language Models

Using Machine Learning to Direct Limited HIV Programme Resources to Communities with the Greatest Need

NelimaL — Tue, 11 Aug 2026 09:24:01 +0000

Imagine working as a Data Analyst in a healthcare Non-Governmental Organization (NGO) implementing HIV and AIDS programmes across several communities.

The organization has limited resources. There may not be enough funding, healthcare workers, testing kits, transport, outreach teams, or community programmes to serve every community at the same intensity.

This creates an important question:

How can we use data and machine learning to direct limited programme resources to communities with the greatest need?

This is where Machine Learning (ML) can become valuable.
Rather than distributing resources equally across all communities, an NGO can use historical programme data to identify communities experiencing greater HIV-related service gaps or higher levels of need. Resources can then be prioritized based on evidence.

What Is Machine Learning?

Machine Learning is a branch of Artificial Intelligence that enables computers to learn patterns from data and use those patterns to make predictions or support decisions.

Instead of manually creating rules for every situation, you provide the algorithm with historical data and allow it to identify relationships within that data.
For example, the NGO could have this information about different communities:

Community	HIV Testing Coverage	ART Coverage	Missed Appointments	Outreach Activities
Community A	85%	90%	5%	High
Community B	52%	61%	25%	Low
Community C	70%	75%	15%	Medium
Community D	40%	55%	32%	Low

Looking at this data, Community D appears to have greater programme gaps than Community A. However, in a real programme, the decision should not be based on one indicator alone.
Machine learning can analyse many variables simultaneously to identify communities that may require greater attention.

Why Resource Allocation Matters in HIV Programmes

HIV programmes operate in environments where resources are often limited.
An NGO may have:

A limited number of community health workers
A fixed outreach budget
Limited HIV testing supplies
Limited transportation
Limited clinic staff
Limited funding for community campaigns
Limited time for follow-up activities If these resources are distributed equally across all communities, the approach may appear fair, but it does not necessarily address differences in need.

For example:

Community A may have 90% ART coverage and strong appointment retention, while Community B may have 55% ART coverage and substantial treatment interruptions.
Giving both communities exactly the same level of support may not produce the greatest programme impact.

A data-driven approach asks:

Where will an additional unit of resources potentially make the greatest difference?

What Data Can Be Used?

Before building a machine learning model, an one needs appropriate data.
WHO's HIV strategic information guidance emphasizes the importance of using routine programme data to understand service access, coverage and gaps across the HIV prevention, testing and treatment cascade.

Potential community-level variables could include:

HIV Prevention

HIV prevention service coverage
PrEP uptake
Condom distribution
HIV prevention outreach
Number of people reached

HIV Testing

Number of HIV tests conducted
HIV testing coverage
HIV positivity yield
Testing among priority populations
HIV self-testing uptake

HIV Treatment

Number of people living with HIV receiving ART
ART coverage
Treatment initiation
Treatment retention
Treatment interruption

Viral Load Monitoring

Viral load testing coverage
Viral suppression
Clients overdue for viral load monitoring

Programme Access

Distance to health facilities
Availability of outreach services
Number of healthcare workers
Availability of HIV services
Transport accessibility

Demographic and Geographic Information

Population size
Age distribution
Gender
Geographic location
Population density
Socioeconomic indicators

The purpose is not to collect every possible variable. The organization should identify variables that are relevant to the programme question and ethically appropriate to use.

Turning Programme Data into a Resource Allocation Problem

The first step is to convert the broad question into a measurable problem. Instead of asking:

"Which communities need help?"
we could ask:
"Which communities have the greatest combination of HIV service gaps and unmet programme needs?"

We could then create a target variable such as:

Community Need Level

Low
Medium
High

The machine learning model will then learn from historical programme data and classify communities into these categories.
For example:

Community A → Low Need
Community B → High Need
Community C → Medium Need
Community D → High Need

This provides programme managers with a starting point for prioritization.

Using Machine Learning to Identify High-Need Communities

One possible approach is supervised learning.
Suppose the NGO has historical data from 100 communities.
For each community, we know:

HIV testing coverage
ART coverage
Treatment interruption
Viral load coverage
Outreach coverage
Distance to healthcare services
Previous programme interventions

Programme experts could define historical need categories based on agreed programme criteria.
The model then learns patterns associated with:

High Need
Medium Need
Low Need

When new programme data becomes available, the model can estimate the likely need category for each community.

An Example

Imagine an NGO has funding for only 10 community outreach campaigns, but there are 30 communities that could potentially receive them.
A simple approach would be to distribute the campaigns equally.
However, machine learning could help identify communities where programme gaps are greatest.

Suppose the model produces:

Community	Predicted Need	Priority Score
A	Low	22
B	High	91
C	Medium	58
D	High	87
E	Low	18
F	High	94

The programme team could prioritize communities F, B and D for additional assessment and potential intervention.
This does not mean that the model automatically decides where resources go.
Instead, it provides evidence to support programme managers in making better decisions.

An Alternative Approach: Clustering Communities

Machine learning does not always need to predict a known outcome.
An NGO could also use unsupervised learning, particularly clustering. Clustering groups communities that have similar characteristics.
For example, a clustering model could identify:

Cluster 1
High HIV testing
High ART coverage
Low treatment interruption
Strong outreach

Cluster 2
Low HIV testing
Low ART coverage
High treatment interruption
Limited outreach

Cluster 3
Moderate testing
Moderate ART coverage
Moderate treatment interruption
Moderate outreach

Cluster 2 may become an important focus for further programme investigation.
The organization could then ask:

What is happening in these communities that is contributing to these service gaps?
This is an important point: machine learning identifies patterns, but programme teams still need to understand the reasons behind those patterns.

Creating a Community Priority Score

Another practical approach is to develop a priority score.
For example, an NGO could combine several programme indicators:

Priority Score =
Testing Gap
+ ART Coverage Gap
+ Viral Load Monitoring Gap
+ Treatment Interruption
+ Outreach Gap
+ Access Challenges

The exact weighting would need to be agreed upon by programme specialists and validated using local evidence.

For example:

Indicator	Community A	Community B
Testing gap	Low	High
ART coverage gap	Low	High
Viral load gap	Low	Medium
Treatment interruption	Low	High
Outreach gap	Low	High
Overall priority	Low	High

Machine learning could then improve this approach by learning from historical outcomes and identifying which combinations of factors are most strongly associated with poor programme performance.

From Prediction to Action

A machine learning model is useful only if its predictions lead to appropriate action.
The process could look like this:

Programme Data
       ↓
Data Cleaning
       ↓
Exploratory Data Analysis
       ↓
Machine Learning Model
       ↓
Identify High-Need Communities
       ↓
Validate With Programme Teams
       ↓
Prioritize Resources
       ↓
Implement Intervention
       ↓
Monitor Results
       ↓
Update Model and Strategy

For example:

Community B

The model identifies Community B as high priority because of:

Low HIV testing coverage
Low ART coverage
High treatment interruption
Limited outreach
Poor access to health services

The NGO could respond by providing:

Additional community outreach
HIV testing services
Community health worker follow-up
Appointment support
Improved linkage to treatment
Additional health education

The organization would then monitor whether these interventions improve programme outcomes.

Why Data Quality Matters

Machine learning cannot solve poor-quality data. Suppose one community reports HIV testing correctly while another community consistently under-reports testing. The model may incorrectly conclude that the second community has a greater need.

Therefore, before using machine learning for resource allocation, the NGO should assess:

Missing data
Duplicate records
Incorrect values
Inconsistent reporting
Changes in reporting systems
Differences between facilities
Data completeness
Data timeliness

WHO's recent guidance on HIV surveillance emphasizes the importance of improving the quality, consistency and reliability of routine HIV data so that it can support public health action.

Better data leads to more reliable analysis and more defensible decisions.

Machine Learning Should Not Replace Programme Expertise

One of the biggest mistakes would be to assume:

"The model says Community B is high priority, therefore Community B automatically receives all the resources."
That is not how responsible machine learning should work.
The model provides evidence, but programme managers should consider additional information.

For example:

Is there a recent outbreak or programme change?
Has the community recently received an intervention?
Is the data complete?
Are there local barriers not captured in the dataset?
Are there vulnerable populations that require attention?
Is the predicted need clinically or programmatically meaningful?

The final decision should combine:

Data + Machine Learning + Programme Expertise + Community Knowledge

Ethical Considerations

HIV data requires strong privacy and confidentiality protections.
WHO guidance emphasizes data security, privacy, confidentiality, appropriate access and governance when using digital and person-centred HIV data.

1. Protect Confidential Information

Where possible, resource allocation models should work with aggregated community-level information rather than unnecessary individual-level identifiers.

For example, instead of exposing individual client records:

Client ID: 12345
HIV Status: Positive

the resource allocation model may work with:

Community: District A
ART Coverage: 62%
Treatment Interruption: 21%

2. Avoid Bias

A machine learning model can reproduce biases present in historical data. For example, if certain communities have historically received fewer services, the data may make their programme outcomes look different for reasons that are not entirely related to underlying need.
The model should therefore be evaluated carefully across different geographic and population groups.

3. Do Not Use Predictions to Discriminate

A community being classified as "high need" should lead to more appropriate support, not stigma or punishment.
The purpose should be:

Identify gaps → provide support → improve outcomes.

Measuring Whether Resources Were Actually Effective

Resource allocation should not stop after resources are distributed.
The NGO should evaluate whether the intervention produced improvement.
For example, before intervention:

HIV Testing Coverage = 52%
ART Coverage = 61%
Treatment Interruption = 25%

After six months:

HIV Testing Coverage = 70%
ART Coverage = 74%
Treatment Interruption = 15%

The organization can then assess whether the intervention appears to have improved programme outcomes.
This creates a feedback loop:

Predict
  ↓
Prioritize
  ↓
Intervene
  ↓
Measure
  ↓
Learn
  ↓
Improve

This is where machine learning can become part of a continuous programme improvement cycle.

How This Helps an NGO Make Better Decisions

Without data-driven prioritization, a programme manager may ask:

"Where should we send our outreach team next month?"
With machine learning and programme data, the question becomes:
"Which communities currently show the greatest combination of service gaps, access challenges and unmet need, and what intervention is most appropriate for each community?"

That is a much stronger decision-making process.
For example:

Limited Resources
       ↓
Identify Programme Gaps
       ↓
Use Historical Data
       ↓
Apply Machine Learning
       ↓
Rank/Group Communities
       ↓
Validate Results
       ↓
Prioritize High-Need Areas
       ↓
Deploy Resources
       ↓
Measure Impact

This approach helps an NGO move from equal distribution of resources to evidence-informed prioritization based on need and potential programme impact.

Important Limitations

Machine learning is powerful, but it is not a perfect solution.
A model may fail because:

Data is incomplete
Historical patterns change
Some important factors are not recorded
The model is biased
Communities differ in ways not captured by the dataset
Programme priorities change
Predictions may not translate into real-world impact

Therefore, machine learning should be treated as a decision-support tool, not an automatic resource allocation system.

References

World Health Organization (WHO). (2022). Consolidated guidelines on person-centred HIV strategic information: Strengthening routine data for impact. WHO. The guidelines emphasize using routine data to identify service gaps and improve HIV programme decision-making.
World Health Organization (WHO). (2020). Consolidated HIV strategic information guidelines: Driving impact through programme monitoring and management. WHO. The guidance highlights the importance of collecting, analysing and using data to identify gaps in HIV service access, coverage and quality.
UNAIDS. (2026). Resource Needs Estimation Tool. UNAIDS. The tool demonstrates how data-driven resource estimation can support HIV programme planning and allocation across prevention, testing, treatment and other programme areas.

Regression Isn’t Regularization: A Simple Guide to Understanding Both

NelimaL — Mon, 27 Jul 2026 06:44:11 +0000

Regression and regularization are both important concepts in machine learning and statistics, but they solve different problems.

Regression is primarily used to model relationships and make predictions.
Regularization is used to improve a model's ability to generalize by controlling its complexity.

Regression

This is a statistical and machine learning technique used to predict a continuous numerical outcome based on one or more input variables.
For example, we might want to predict:

A house's price based on its size and location
A student's exam score based on study hours
A company's sales based on advertising spending

Simple Linear Regression

In simple linear regression, we model the relationship between an input variable (x) and an output (y):
$$
y = \beta_0 + \beta_1x + \epsilon
$$

Where:

(y) is the predicted outcome
(\beta_0) is the intercept
(\beta_1) is the coefficient or slope
(x) is the input variable
(\epsilon) represents the error The model learns values for (\beta_0) and (\beta_1) that make its predictions as close as possible to the actual values.

Multiple Linear Regression

In multiple linear regression, several predictors are used:
$$
y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_px_p + \epsilon
$$

The goal is typically to minimize the sum of squared errors (SSE):
$$
\text{SSE} = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2
$$
This approach is known as Ordinary Least Squares (OLS).

Regularization

Regularization is a technique used to prevent a machine learning model from becoming too complex.
A model can perform extremely well on training data but poorly on new, unseen data. This problem is called overfitting.
Regularization addresses overfitting by adding a penalty for large model coefficients to the model's objective function.

Instead of minimizing only the prediction error, the model minimizes:
$$

\text{Prediction Error} + \text{Complexity Penalty}
$$
The penalty discourages the model from relying too heavily on individual features.

The Main Types of Regularization

1. Ridge Regression: L2 Regularization

Ridge regression adds a penalty based on the squared values of the coefficients:
$$

\text{Loss}

\sum_{i=1}^{n}(y_i - \hat{y}i)^2
+
\lambda\sum{j=1}^{p}\beta_j^2
$$
Here, (\lambda) controls the strength of the regularization.

If (\lambda = 0), the model is equivalent to ordinary linear regression.
A larger (\lambda) creates a stronger penalty.
Coefficients become smaller, but generally do not become exactly zero. Ridge regression is particularly useful when predictors are highly correlated.

2. Lasso Regression: L1 Regularization

Lasso regression uses the absolute values of the coefficients:
$$

\text{Loss}

\sum_{i=1}^{n}(y_i - \hat{y}i)^2
+
\lambda\sum{j=1}^{p}|\beta_j|
$$
Unlike Ridge regression, Lasso can shrink some coefficients exactly to zero.

This means Lasso can perform a type of feature selection by effectively removing less important variables from the model.

3. Elastic Net

Elastic Net combines both L1 and L2 regularization:
$$

\text{Loss}

\text{SSE}
+
\lambda_1\sum_j|\beta_j|
+
\lambda_2\sum_j\beta_j^2
$$

It combines:

Lasso's ability to perform feature selection
Ridge's ability to handle correlated features

Key Difference Between Regression and Regularization

The simplest way to understand the difference is:

Regression builds a model to explain or predict an outcome. Regularization modifies the learning process to control the model's complexity.

For example, ordinary linear regression might optimize the following objective:
$$
\min_{\beta} \text{SSE}
$$
Ridge regression changes it to:
$$
\min_{\beta}
\left(
\text{SSE}
+
\lambda\sum_j\beta_j^2
\right)
$$
The underlying task is still regression. Regularization simply adds a constraint or penalty to make the model less likely to overfit.

A Practical Example

Imagine you are predicting house prices using 100 features, including:

Square footage
Number of bedrooms
Location
Age of the house
Distance to schools
Various highly correlated measurements

An ordinary regression model may fit the training data very closely. However, if there are too many features or strong correlations between them, the model may overfit.
Regularization can help:

Ridge regression shrinks the coefficients of correlated variables.
Lasso regression may reduce some coefficients to zero.
Elastic Net combines both strategies. As a result, the model may perform better on new, unseen houses.

Comparison Table

Here is the corrected and clearly formatted Markdown table:

Feature	Ordinary Regression	Regularized Regression
Main purpose	Model relationships and make predictions	Reduce overfitting and control complexity
Objective	Minimize prediction error	Minimize prediction error plus a penalty
Coefficients	Can become very large	Penalized and typically smaller
Feature selection	Usually no	Lasso can select features
Handles multicollinearity	Can be sensitive	Ridge handles it well
Generalization	May overfit	Often improves performance on unseen data

The Bias-Variance Tradeoff

Regularization works by introducing a small amount of bias in exchange for reducing variance.

An Unregularized Model May Have:

Low training error
High variance
Poor performance on new data

A Strongly Regularized Model May Have:

Higher training error
Lower variance
Better generalization

The goal is to find the right balance.
The regularization parameter (\lambda) is commonly selected using cross-validation.

Important Practical Note: Feature Scaling

Regularization is sensitive to the scale of features.
For example:

Age might range from 0 to 100.
Income might range from $20,000 to $200,000.

Because regularization penalizes coefficient sizes, variables with different scales can be treated unfairly. Therefore, features are often standardized before applying Ridge, Lasso, or Elastic Net.

Takeaway

Regression and regularization are not competing concepts.
Regression answers:

How can we model the relationship between inputs and a numerical outcome?
Regularization answers:
How can we prevent that model from becoming unnecessarily complex and overfitting the training data?

Ordinary linear regression focuses on minimizing prediction error. Regularized regression adds a penalty that discourages overly large coefficients. Ridge, Lasso, and Elastic Net are common examples of regularized regression techniques.

A useful way to remember the distinction is:

Regression learns the relationship. Regularization controls the complexity of what is learned.

Why Statistics is the Secret Weapon Behind Data Science- Healthcare analytics

NelimaL — Fri, 19 Jun 2026 08:05:14 +0000

"Without statistics, data is just a collection of numbers. With statistics, data becomes knowledge."

When many people first hear the word statistics, they imagine complicated formulas, endless calculations, and confusing graphs. It often feels like a subject reserved for mathematicians and researchers. However, after exploring the world of data science, one thing becomes clear: statistics is not just important, it is the foundation upon which data science is built.

For instance, a healthcare organization that serves thousands of patients every day. Every hospital visit, laboratory test, prescription, and diagnosis generates data. By the end of a single month, the organization may have collected millions of records. While this data is valuable, it is useless unless someone can extract meaningful insights from it. This is where statistics steps in.

Lindah is a data scientist working for a large healthcare organization. The hospital has noticed an increase in patient readmissions(patients who return to the hospital within 30 days after being discharged). These readmissions are costly for both the hospital and the patients, so management wants to understand why they are happening.

Lindah is given access to patient data that includes:

Age
Medical history
Length of hospital stay
Medications prescribed
Number of previous admissions
Recovery outcomes

At first glance, the dataset looks overwhelming. Thousands of rows and dozens of columns contain information that appears impossible to interpret manually. Instead of guessing, Lindah turns to statistics.

Finding Patterns Through Descriptive Statistics

Her first step is to summarize the data using descriptive statistics. She calculates:

Average patient age
Average hospital stay duration
Most common medical conditions
Readmission rates by age group

These simple statistical measures immediately reveal interesting patterns. She discovers that patients above the age of 65 have significantly higher readmission rates than younger patients.
Without statistics, this important trend might remain hidden inside thousands of records.

Using Probability to Predict Risk

The next question is even more important:
"Can we predict which patients are likely to return to the hospital?"
Lindah uses statistical probability techniques to estimate the likelihood of readmission for different patient groups.
For example, she finds that:

Patients with diabetes have a higher probability of returning within 30 days.
Patients prescribed multiple medications are at greater risk.
Patients with previous admissions are more likely to be readmitted again. These probabilities help healthcare providers identify high-risk patients before they leave the hospital.

Testing Assumptions with Statistical Analysis

Hospital administrators believe that extending discharge counseling sessions will reduce readmissions.
But is that actually true?
Rather than relying on opinions, Lindah conducts a statistical study that compares two groups:

Patients who received standard discharge instructions.
Patients who received extended counseling and follow-up support. Using statistical hypothesis testing, she determines whether the observed difference in readmission rates is significant or simply due to chance. The results show that patients receiving additional counseling are 18% less likely to return to the hospital. This evidence allows hospital leadership to confidently invest in improved discharge programs.

Building Predictive Models

Statistics also powers machine learning models.
Using historical patient data, Lindah develops a predictive model that identifies patients at high risk of readmission before they are discharged. The model analyzes numerous factors simultaneously and produces a risk score for each patient.

Doctors and nurses can then:

Schedule additional follow-up appointments.
Provide extra education about medications.
Arrange home healthcare services.
Monitor high-risk patients more closely.

As a result, patient outcomes improve and healthcare costs decrease.

Why Statistics Matters in Data Science

Lindah's story demonstrates a powerful truth: statistics transforms raw data into actionable insights.
In data science, statistics helps professionals:

Summarize large datasets.
Discover hidden patterns.
Measure uncertainty.
Test hypotheses.
Make predictions.
Support evidence-based decision-making. Whether in healthcare, banking, retail, education, or technology, every successful data science project relies on statistical thinking.

Conclusion

Data science is often associated with programming languages, machine learning, and artificial intelligence. Yet beneath all these advanced technologies lies statistics—the discipline that gives meaning to data.
For a healthcare data scientist like Lindah, statistics is more than a classroom subject. It is a tool that helps hospitals reduce readmissions, improve patient care, and save lives.

The next time statistics seems intimidating, remember that behind every life-saving healthcare prediction, every accurate forecast, and every data-driven decision, statistics is quietly doing the heavy lifting. It is not merely a part of data science, it is the language that allows data science to speak.

Python and Its Role in Data Analytics: A Beginner-Friendly Guide Using Logistics Data

NelimaL — Sun, 10 May 2026 11:01:00 +0000

Introduction

In today’s digital economy, organizations generate huge amounts of data every single day. However, raw data alone is not useful unless businesses can analyze it and turn it into meaningful insights.

One of the most powerful tools used in data analytics today is Python. Python has become one of the most popular programming languages in the world because it is simple, powerful, flexible, and beginner-friendly.

Unlike some programming languages that are difficult for beginners to understand, Python uses readable syntax that looks almost like normal English.

What Is Python?

Python is a high-level programming language created by Guido van Rossum and released in 1991. It was designed to be simple, readable, and easy to learn.

A high-level language means programmers can write code using human-friendly commands instead of complicated machine instructions.

For example, displaying a message in Python only requires one line of code:

print("Hello world")

This simplicity is one of the main reasons Python is popular among beginners.

Python is also versatile. It can be used in many industries and fields such as: Data analytics, Data science, Artificial intelligence, Machine learning, Web development, Cybersecurity
and Software development

Why Python Is Popular in Data Analytics

Python has become one of the most widely used tools in the data analytics space for several reasons.

1. Python Is Easy to Learn

Python syntax is clean and readable.

For example:

age = 21

if age >= 18:
    print("Adult")
else:
    print("Minor")

Even beginners can understand what this code is doing.

2. Python Has Powerful Data Libraries

Python has many built-in libraries and external packages that make data analysis easier.

Instead of writing long and complicated code, analysts can use specialized libraries to complete tasks quickly.

Popular libraries include: Pandas, NumPy, Matplotlib, Seaborn and Scikit-learn
These libraries help analysts to:

Clean data
Analyze data
Create charts
Perform calculations
Build predictive models

3. Python Has a Large Community

Millions of people use Python worldwide. This means beginners can easily find: Tutorials, Documentation, YouTube videos, Coding forums and Online courses -If a beginner encounters an error, there is a high chance someone else has already solved the same problem online.

4. Python Handles Large Amounts of Data

Businesses often work with thousands or millions of rows of data.
Python can process large datasets efficiently and integrate with:

SQL databases
APIs
Excel files
Cloud platforms
Machine learning tools This makes Python valuable in modern organizations.

Python Libraries Used in Data Analytics

Python libraries are collections of pre-written code that help developers complete tasks faster.

1. Pandas

Pandas is one of the most important Python libraries for data analytics. It is mainly used for:

Reading data
Cleaning data
Filtering records
Grouping information
Performing calculations
Working with tables

Example: Loading a Logistics Dataset

Below is an example of loading the logistics dataset using Pandas.

import pandas as pd
url =
"https://raw.githubusercontent.com/NelimaL/LOGISTICS_DATA/refs/heads/main/LOGISTICS_MOCK_DATA%20(1).json"
df = pd.read_json(url)

Display first 5 rows by slicing

print(logistics_df[:5])

This code imports the logistics dataset and displays the first five rows.

2. NumPy

NumPy is used for numerical operations and mathematical calculations.

Example: Calculating Average Delivery Time

import numpy as np

delivery_times = [2, 3, 24, 13, 4]

average_time = np.mean(delivery_times)

print("Average Delivery Time:", average_time)

Output

Average Delivery Time: 9.2

3. Matplotlib

Matplotlib is used for creating charts and graphs.

Example: Visualizing Delivery Ratings

import matplotlib.pyplot as plt
ratings = [2, 5, 2, 3, 4]
plt.hist(ratings)
plt.title("Delivery Ratings")
plt.xlabel("Ratings")
plt.ylabel("Frequency")
plt.show()

This creates a histogram showing how delivery ratings are distributed.

4. Seaborn

Seaborn is another visualization library that creates more attractive statistical charts.

Example: Delivery Time vs Package Weight

import seaborn as sns
import matplotlib.pyplot as plt
sns.scatterplot(
    x=logistics_df["package_weight"],
    y=logistics_df["delivery_time"]
)
plt.title("Package Weight vs Delivery Time")
plt.show()

This chart helps analysts determine whether heavier packages take longer to deliver.

How Python Is Used to Clean Data

Real-world datasets are rarely perfect.
Data often contains:

Missing values
Duplicate records
Incorrect formats
Typing errors
Invalid values Data cleaning is one of the most important steps in analytics.

Checking for Missing Values

# Check missing values
print(logistics_df.isnull().sum())

This code checks how many missing values exist in each column.

Removing Duplicate Records

# Remove duplicates
logistics_df = logistics_df.drop_duplicates()
print("Duplicates removed")

Duplicate records can affect business reports and calculations.

Converting Date Columns

Dates are very important in analytics.

# Convert dates
logistics_df["delivery_date"] = pd.to_datetime(logistics_df["delivery_date"])
logistics_df["pickup_date"] = pd.to_datetime(logistics_df["pickup_date"])

This converts the columns into proper date format.

Checking Data Types

print(logistics_df.dtypes)

Understanding data types helps analysts know which operations can be performed.

How Python Is Used to Analyze Data

Once data is cleaned, analysts can start extracting insights.

Example 1: Counting Delivery Statuses

A logistics company may want to know how many deliveries were:

Delivered
Returned
In Transit
Out for Delivery

status_count = logistics_df["delivery_status"].value_counts()
print(status_count)

Example Output

Delivered           6
Returned            5
In Transit          4
Out for Delivery    5

This helps management monitor operational performance.

Example 2: Average Delivery Rating

average_rating = logistics_df["delivery_rating"].mean()

print("Average Delivery Rating:", average_rating)

Example Output

Average Delivery Rating: 3.4

This helps companies evaluate customer satisfaction.

Example 3: Finding the Fastest Deliveries

fastest_delivery = logistics_df.sort_values(
    by="delivery_time"
)

print(fastest_delivery[["tracking_number", "delivery_time"]].head())

This identifies packages delivered in the shortest time.

Example 4: Analyzing Delivery Companies

company_count = logistics_df["delivery_company"].value_counts()

print(company_count)

This shows how many deliveries each company handled.

Example 5: Average Package Weight

average_weight = logistics_df["package_weight"].mean()

print("Average Package Weight:", average_weight)

This helps logistics companies understand shipment trends.

How Python Is Used to Visualize Data

Data visualization helps analysts communicate findings clearly.

Instead of reading large tables, decision-makers can quickly understand charts and graphs.

Example 1: Bar Chart of Delivery Status

status_count = logistics_df["delivery_status"].value_counts()
status_count.plot(kind="bar")
plt.title("Delivery Status Distribution")
plt.xlabel("Delivery Status")
plt.ylabel("Count")
plt.show()

This chart shows the number of packages in each delivery status.

Example 2: Pie Chart of Delivery Ratings

rating_count = logistics_df["delivery_rating"].value_counts()
rating_count.plot(kind="pie", autopct="%1.1f%%")
plt.title("Delivery Ratings")
plt.ylabel("")
plt.show()

This chart helps visualize customer satisfaction levels.

Example 3: Line Graph of Delivery Time

logistics_df["delivery_time"].plot(kind="line")
plt.title("Delivery Time Trend")
plt.xlabel("Shipment Number")
plt.ylabel("Delivery Time")

plt.show()

This graph helps analysts identify delivery trends.

Real-World Uses of Python in Data Analytics

Python is used in many industries worldwide.

1. Logistics and Supply Chain

Logistics companies use Python to:

Track deliveries
Optimize delivery routes
Predict delays
Analyze delivery performance
Monitor customer satisfaction

Using the logistics dataset, analysts can identify common delivery failure reasons.

failure_reason = logistics_df[
    "delivery_failure_reason"
].value_counts()

print(failure_reason)

This helps companies reduce failed deliveries.

2. Banking Industry

Banks use Python for:

Fraud detection
Credit scoring
Customer analysis
Financial forecasting

Example:

transactions = [500, 12000, 300, 15000]

for amount in transactions:
    if amount > 10000:
        print("Suspicious transaction:", amount)

3. Healthcare Industry

Hospitals use Python to:

Analyze patient records
Predict diseases
Monitor patient recovery
Manage hospital operations

Example:

patients = {
    "John": 120,
    "Mary": 150,
    "James": 180
}

for patient, pressure in patients.items():
    if pressure > 140:
        print(patient, "has high blood pressure")

4. Retail Industry

Retail businesses use Python to:

Analyze customer purchases
Predict sales trends
Manage inventory
Recommend products Example:

products = [
    "Milk",
    "Bread",
    "Milk",
    "Eggs",
    "Milk"
]

print(products.count("Milk"))

To manageinventory

Why Beginners Should Learn Python

There are many reasons beginners should consider learning Python.

1. Python Is Beginner-Friendly

Python has simple syntax that makes programming easier to understand. Beginners can focus on solving problems instead of struggling with complicated rules.

2. Python Is Highly Marketable

Python skills are highly demanded in industries such as:

Data analytics
Artificial intelligence
Software engineering
Cybersecurity
Data science Professionals with Python skills often have strong career opportunities. ## 3. Python Supports Automation Python can automate repetitive tasks which saves time and increases efficiency. Example:

for report in range(1, 6):
    print("Generating Report", report)

4. Python Encourages Problem Solving

Learning Python improves logical thinking and problem-solving skills. This skill is valuable in many careers.

Challenges Beginners May Face

Although Python is beginner-friendly, learners may still experience challenges.

1. Understanding Programming Logic

Programming requires critical thinking and logical reasoning.

2. Learning Libraries

Libraries such as Pandas and NumPy can feel overwhelming at first.

3. Debugging Errors

Beginners often encounter syntax errors.
Example:

print("Hello"

The closing bracket is missing.
However, with practice, debugging becomes easier.

Tips for Learning Python Successfully

Practice Regularly

The more you practice, the more comfortable you become.

Build Real Projects

Projects help beginners apply what they learn.
Examples include:

Delivery tracking dashboards
Sales analysis projects
Expense trackers
Student performance analysis

Learn Data Visualization

Visualization is one of the most valuable skills in analytics.
Practice creating:

Bar charts
Pie charts
Histograms
Scatter plots

Read Error Messages Carefully

Python error messages usually explain the problem. Understanding these messages helps learners improve faster.

Use Real Data

Working with real datasets improves learning.
The logistics dataset used in this article demonstrates how businesses analyze operational data using Python.

Conclusion

As businesses continue generating larger amounts of data, the demand for Python skills will continue growing. Beginners who start learning Python today are preparing themselves for exciting opportunities in the future of data analytics.

SQL Without Confusion: Subqueries vs CTEs (Visual + Practical Guide)

NelimaL — Wed, 22 Apr 2026 09:07:36 +0000

Understanding subqueries and CTEs (Common Table Expressions) becomes much easier when you can see how data flows. This guide uses simple diagrams and real-style datasets.

What is a Subquery?
A subquery is a query inside another query.

Visual Representation
Outer Query
↓
[ Uses result from ]
↓
(Subquery)

Example: Students scoring above average

SELECT first_name, marks
FROM students s
JOIN results r ON s.student_id = r.student_id
WHERE marks > (
    SELECT AVG(marks) FROM results
);

**How it Works
Step 1: Subquery runs
SELECT AVG(marks) → returns 81.6

Step 2: Outer query runs
Returns students with marks > 81.6**

Types of Subqueries
1. Scalar Subquery (Single Value)
Returns exactly one value (one row and one column). It is commonly used in SELECT lists or in WHERE clauses with simple comparison operators like = or >.

SELECT first_name
FROM students
WHERE student_id = (SELECT student_id FROM results WHERE marks = 90);

2. Correlated Subquery (Row-by-Row Execution)
Depends on the outer query and executes once for each row in the outer query
FOR each row in outer query:run subquery again

SELECT s.first_name
FROM students s
WHERE marks > (
    SELECT AVG(marks)
    FROM results r
    WHERE s.student_id = r.student_id
);

This can be slow because it runs multiple times.

3. Multi-row Subquery
Returns one or more rows. These usually require set operators such as IN, ANY, or ALL in the outer query.

SELECT first_name
FROM students
WHERE city IN (
    SELECT city FROM patients
);

This finds students living in the same cities as patients.

What is a CTE?
A CTE is a temporary result set defined using the WITH clause. It improves readability and allows you to reuse query logic.

Basic Syntax:
WITH cte_name AS (
SELECT column1, column2
FROM table
)
SELECT * FROM cte_name;

Types of CTEs

Simple CTE-Used to simplify complex queries

WITH avg_marks AS (SELECT AVG(marks) AS avg_score FROM results)
SELECT s.first_name, r.marks
FROM students s
JOIN results r ON s.student_id = r.student_id
JOIN avg_marks a ON 1=1
WHERE r.marks > a.avg_score;

Flow
Step 1: avg_marks created
Step 2: Main query uses it

Multiple CTEs-Define more than one CTE in a single query

WITH student_marks AS (
    SELECT s.first_name, r.marks
    FROM students s
    JOIN results r ON s.student_id = r.student_id
),
top_students AS (SELECT * FROM student_marks WHERE marks > 80)
SELECT * FROM top_students;

Recursive CTE (Hierarchy) A CTE that references itself to repeatedly execute a query until a specific condition is met. It consists of an anchor member (initial result) and a recursive member (iterative logic). Example: Employee hierarchy (Hospital)

WITH RECURSIVE staff_hierarchy AS (
    SELECT patient_id, full_name
    FROM patients
    WHERE patient_id = 201
    UNION ALL
    SELECT p.patient_id, p.full_name
    FROM patients p
    JOIN staff_hierarchy sh ON p.patient_id = sh.patient_id)
SELECT * FROM staff_hierarchy;

Subqueries vs CTEs
Comparison Table

📊 Comparison Table

Feature	Subqueries	CTEs
Readability	Can become hard to read when nested	Much cleaner and easier to follow
Reusability	Cannot be reused easily	Can be referenced multiple times
Performance	Faster for simple queries	Better for complex logic
Execution	Often executed multiple times	Executed once (in many cases)
Debugging	Harder to debug	Easier to test step-by-step
Recursion	Not supported	Supported

When to Use What
Use Subqueries When:

The logic is simple and short
You only need the result once
You want a quick inline filter Use CTEs When:
The query is complex or layered
You need to reuse logic
You want cleaner, more maintainable SQL
You are working with recursive data

Final Thoughts
Both subqueries and CTEs are essential tools in SQL. While subqueries are great for quick, simple tasks, CTEs shine when dealing with complex logic and improving readability.

As a data analyst, knowing when to use each can significantly improve both your query performance and the clarity of your code.

Mastering SQL Joins: A Practical Guide for beginners.

NelimaL — Tue, 21 Apr 2026 07:23:20 +0000

Introduction

SQL joins allow you to combine data from multiple tables based on relationships between them. In this article, we’ll explore different types of joins using this two schemas:

City Hospital Schema
Nairobi Academy Schema

What is a SQL Join?

A join retrieves data from two or more tables using a related column.

Basic Syntax

SELECT columns
FROM table1
JOIN table2
ON table1.column = table2.column;

1. INNER JOIN
Returns only records that exist in both tables.

Example (City Hospital schema)

SELECT p.full_name AS patient_name,
    d.full_name AS doctor_name,
    a.appointment_date,
    a.diagnosis
FROM appointments a
INNER JOIN patients p 
    ON a.patient_id = p.patient_id
INNER JOIN doctors d 
    ON a.doctor_id = d.doctor_id;

Only matching records appear while missing relationships are excluded

2. LEFT JOIN
Returns all records from the left table and matches from the right.
Example

SELECT 
    p.full_name,a.appointment_date,a.diagnosis
FROM patients p
LEFT JOIN appointments a 
    ON p.patient_id = a.patient_id;

All patients are shown while missing appointments appear as NULL
This shows all primary records, even if incomplete

3. RIGHT JOIN
Returns all records from the right table.
Example

SELECT d.full_name,a.appointment_date,a.diagnosis
FROM appointments a
RIGHT JOIN doctors d 
    ON a.doctor_id = d.doctor_id;

All doctors appear while missing appointments show NULL
The Focus on the right-side table.

4. FULL OUTER JOIN
Returns all records from both tables, matched or not.

SELECT 
    p.full_name,
    a.appointment_date
FROM patients p
FULL OUTER JOIN appointments a 
    ON p.patient_id = a.patient_id;

Combines LEFT + RIGHT JOIN behavior
You will see everything, including gaps.

5. CROSS JOIN
Returns all possible combinations.

Example (Nairobi Academy)

SELECT 
    s.student_name,
    e.exam_name
FROM students s
CROSS JOIN exams e;

Produces Cartesian product
This Generates combinations

6. SELF JOIN
A table joins with itself.
Example

SELECT 
    s1.student_name AS student_1,
    s2.student_name AS student_2,
    s1.class
FROM students s1
JOIN students s2 
    ON s1.class = s2.class
    AND s1.student_id <> s2.student_id;

Useful for comparisons or hierarchies
This Compares rows within the same table.

7. LEFT ANTI JOIN (Find Missing Data)
Returns records in the left table with no match.

SELECT p.full_name
FROM patients p
WHERE NOT EXISTS (
    SELECT 1 
    FROM appointments a 
    WHERE a.patient_id = p.patient_id
);

This finds missing relationships.

8. RIGHT ANTI JOIN

SELECT d.full_name
FROM doctors d
WHERE NOT EXISTS (
    SELECT 1
    FROM appointments a
    WHERE a.doctor_id = d.doctor_id
);

Finds unused or inactive records

Nairobi Academy: Combining Data
Student Exam Results

SELECT s.student_name,e.exam_name,r.marks
FROM results r
INNER JOIN students s 
    ON r.student_id = s.student_id
INNER JOIN exams e 
    ON r.exam_id = e.exam_id;

Ranking Students

SELECT s.student_name,e.exam_name,r.marks,
    RANK() OVER (PARTITION BY e.exam_name ORDER BY r.marks DESC) AS rank
FROM results r
JOIN students s ON r.student_id = s.student_id
JOIN exams e ON r.exam_id = e.exam_id;

Here is a shortcut to choosing the Right Join
| *Join Type * | *When to Use * |
| ------------------- | ------------------------ |
| INNER JOIN | Only matching data |
| LEFT JOIN | All left + matches |
| RIGHT JOIN | All right + matches |
| FULL OUTER JOIN | Everything |
| CROSS JOIN | All combinations |
| SELF JOIN | Compare within table |
| ANTI JOIN | Find missing data |

Conclusion
SQL joins are the foundation of relational data analysis.
Mastering them allows you to:

Combine datasets effectively
Identify missing or inconsistent data
Build powerful reports

The real skill isn’t just writing joins,it’s understanding when to use each one.

Understanding SQL Basics: DDL, DML, Filtering, and Data Transformation

NelimaL — Sun, 12 Apr 2026 05:30:54 +0000

This week I was focused on building a solid foundation in SQL by working with student, subject, and exam result data. Along the way, I explored key concepts like DDL, DML, filtering using WHERE, and transforming data using CASE WHEN. Here’s a summary of what I learned.

What are DDL and DML?

SQL commands are broadly categorized into two main types:

1. DDL (Data Definition Language)

DDL is used to define and manage the structure of a database.

Examples include:

CREATE – used to create tables
ALTER – used to modify table structure
DROP – used to delete tables

Example:

CREATE TABLE students (
    student_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    gender CHAR(1),
    date_of_birth DATE,
    class VARCHAR(10),
    city VARCHAR(50)
);

In simple terms, DDL defines how the database looks.

2. DML (Data Manipulation Language)

DML is used to manage and manipulate the data inside the tables.

Examples include:

INSERT – add new data
UPDATE – modify existing data
DELETE – remove data

Example:

INSERT INTO students (student_id, first_name, last_name)
VALUES (1, 'Amina', 'Wanjiku');

DML focuses on the data itself.

USING CREATE, INSERT, UPDATE, and DELETE
I used CREATE to define tables like students, subjects, and exam_results.
I used INSERT to populate tables with records extracted from provided data.
Although not heavily used, UPDATE can be applied to change values,
for example:

UPDATE students
SET city = 'Nairobi'
WHERE student_id = 1;
DELETE removes records:
DELETE FROM students
WHERE student_id = 10;

These commands helped me understand how databases are built and maintained step by step.

Filtering Data Using WHERE
The WHERE clause is used to filter records based on conditions.

These are some of operators used:
= → exact match

→ greater than
BETWEEN → range filtering
IN → multiple values
LIKE → pattern matching

Examples:

-- Exact match
SELECT * FROM students WHERE class = 'Form 4';

-- Range
SELECT * FROM exam_results WHERE marks BETWEEN 50 AND 80;

-- Multiple values
SELECT * FROM students WHERE city IN ('Nairobi', 'Mombasa', 'Kisumu');

-- Pattern matching
SELECT * FROM students WHERE first_name LIKE 'A%';

This is one of the most powerful parts of SQL because it allows precise data retrieval.

Transforming Data Using CASE WHEN
CASE WHEN allows us to create new columns based on conditions. It works like an "if-else" statement in programming.

Example:

SELECT 
    marks,
    CASE 
        WHEN marks >= 80 THEN 'Distinction'
        WHEN marks >= 60 THEN 'Merit'
        WHEN marks >= 40 THEN 'Pass'
        ELSE 'Fail'
    END AS performance
FROM exam_results;

This helped convert raw numeric data into meaningful categories.
CASE WHEN is very useful for reporting and data interpretation.

Reflection
This week was both challenging and interesting.
What I found challenging:
Understanding when to use operators like IN vs BETWEEN
Writing clean and error-free queries, most of my queries were to be corrected before running.

Seeing how raw data can be transformed into meaningful insights on the other hand was very interesting.

Conclusion
This week helped me understand the basics of working with databases:

DDL defines structure
DML manipulates data
WHERE filters results
CASE WHEN transforms data

These are essential building blocks for any developer working with databases. I’m looking forward to learning more advanced SQL concepts like joins and aggregations next and I will share it all here.

How to Publish a Power BI Report and Embed it into a Website

NelimaL — Mon, 06 Apr 2026 12:30:02 +0000

Introduction
As s student data plumber, there are very high chances are you've come across Power BI, Microsoft’s powerful business intelligence tool that transforms raw data into meaningful insights through interactive dashboards and reports.
So far, as a data student I’ve explored Power BI queries, DAX, data modeling, joins, charts, dashboards, and reporting. But creating a report is only half the job. The real value comes when you share it with others.

In this article, I’ll walk you through how to:

Publish a Power BI report
Generate an embed code
Embed the report into a website By the end, you’ll be able to make your reports accessible and interactive for users beyond Power BI.

What is Power BI Publishing?
Publishing in Power BI means uploading your report from Power BI Desktop (.pbix file) to the Power BI Service (cloud platform). Once published, your report can be shared, collaborated on, or embedded into websites.

Step 1: Create a Workspace
A workspace is like a container where your reports, dashboards, and datasets live.
Steps:
Go to Power BI Service (https://app.powerbi.com)
Sign in with your account
On the left sidebar, click Workspaces
Click + New workspace
Enter: Workspace name(e.g., Linda Nelima Workspace)
Description (optional)
Click Save

Step 2: Upload and Publish Your Report
Now that your workspace is ready, let’s upload the .pbix file.
Steps:
Open your project in Power BI Desktop
Click File → Publish → Select Workspace
Choose the workspace you created
Click Select

Alternatively (via browser):
Open your workspace in Power BI Service
Click Upload → Browse
Upload your .pbix file
Once uploaded, your report and dataset will appear in the created workspace.

Step 3: Generate the Embed Code
Power BI allows you to embed reports using an iframe.
iframe:(short for inline frame) is an HTML element that embeds another document, such as a video, map, or webpage, within the current HTML webpage
Steps:
Open your report in Power BI Service
Click File → Embed report → Publish to web (public)
Click Create embed code
Copy the generated iframe code
Note:“Publish to web” makes your report publicly accessible. Avoid using this for sensitive data.

Step 4: Embed the Report on a Website
Now let’s embed the report into a simple HTML page.
Example:
<!DOCTYPE html>

Power BI Report

Kenya Crops Dashboard

<iframe width="800" height="600"
    src="YOUR_EMBED_LINK_HERE"
    frameborder="0"
    allowFullScreen="true">
</iframe>

Steps:
Replace YOUR_EMBED_LINK_HERE with your copied embed URL
Save the file as index.html
Open it in a browser

You should now see your interactive Power BI report embedded on the page.

Key Insights
Power BI makes it easy to move from data analysis to sharing insights
Workspaces help organize and manage reports efficiently
Embedding reports allows integration with web apps and dashboards
Always be cautious with data privacy when using public embed options

Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained

NelimaL — Mon, 30 Mar 2026 14:40:50 +0000

Introduction

When I first started working with Power BI at LuxedevHQ, I quickly realized that building reports isn’t just about visuals, it’s about how well your data is structured underneath. That structure is called data modeling, and it’s the backbone of every meaningful dashboard.

What is Data Modeling
Data modeling is the process of organizing data into tables and defining how those tables relate to each other so that analysis becomes efficient, accurate, and scalable. In Power BI, data modeling involves:
• Structuring tables (Fact & Dimension)
• Creating relationships
• Defining cardinality and filter behavior
• Optimizing for performance and usability
Think of it as designing a map before starting a journey. Without it, you’ll get lost in your data.

SQL Joins
Before Power BI relationships, you need to understand joins, because they define how tables combine.
Let’s use this simple scenario
Customers
| CustomerID | Name |
|------------|------|
| 1 | John |
| 2 | Mary |
Orders
| OrderID | CustomerID | Amount |
|---------|------------|--------|
| 101 | 1 | 200 |
| 102 | 1 | 150 |

1. INNER JOIN
This returns only matching records.
Result
| Name | Amount |
|------|--------|
| John | 200 |
| John | 150 |
Mary is excluded because she has no orders and this only shows customers who made a purchase
2. LEFT JOIN
This returns all records from the left table and matches.
Result
| Name | Amount |
|------|--------|
| John | 200 |
| John | 150 |
| Mary | NULL |
Keeps all customers, even without orders.
3. RIGHT JOIN
This join returns all records from the right table and matches.
Use case: Show all orders, even if customer data is missing.
4. FULL OUTER JOIN
This returns everything from both tables.
Use case:Data reconciliation between systems.
5. LEFT ANTI JOIN
The Left Anti Join returns rows in left table with NO match in right.
Result
| Name |
|------|
| Mary |
Use case:Customers who never purchased.
6. RIGHT ANTI JOIN
Returns rows in right table with NO match in left.
Use case:Orders with missing customer records.

How to Create Joins in Power BI
In Power Query:

Click Transform Data
Select a table
Click Merge Queries
Select second table
Choose join type:
Inner
Left Outer
- Right Outer
- Full Outer
- Left Anti
- Right Anti
Click OK
Expand columns This is where SQL joins happen inside Power BI.

Power BI Relationships
Unlike joins, relationships don’t merge tables—they connect them.
Types of Relationships
1. One-to-Many (1:M)
This is the Most common and the default best practice.
Here One value in Table A relates to many in Table B.
Example
One customer → many orders
One product → many sales

Many-to-Many (M:M) Both tables have duplicate values. **Example: Students ↔ Courses
A student can take many courses
A course can have many students** Problem: This can create ambiguous calculations It is harder to control filtering 3. One-to-One (1:1) Here each value appears once in both tables. Example: User table ↔ User profile table This is rare in real-world analytics

Active vs Inactive Relationships
Active Relationship This is a default relationship used automatically
It is represented by a solid line in Model View
Inactive Relationship Exists but is NOT used unless explicitly activated
Represented by a dashed line

Cardinality
Cardinality defines the structure of the relationship
Power BI usually auto-detects this, but you should always verify it.

Cross-Filter Direction
Defines how filters flow:

Single direction: One table filters another
Both direction: Tables filter each other

How to Create Relationships in Power BI
Method 1: Model View

Go to Model View
Drag one column onto another
Relationship is automatically created Method 2: Manage Relationships
Go to Home → Manage Relationships
Click New
Select:
Tables
Columns
Cardinality
Cross-filter direction
Click OK

Joins vs Relationships (Key Difference)
| Feature | Joins | Relationships |
| ----------- | -------------- | --------------- |
| Where used | Power Query | Data Model |
| Output | Combined table | Separate tables |
| Performance | Heavier | More efficient |
| Flexibility | Less | More |
TIP: Use relationships for analysis and joins for data cleaning

Data Modeling Schemas

Star Schema: A central fact table surrounded by denormalized dimension tables, designed for high-performance querying in business intelligence and data warehouses.
Structure

Customers
|
Products — Sales — Date
|
Stores
Why it’s great?
Fast
Easy to understand
Recommended in Power BI
2. Snowflake Schema
An extension of the star schema where dimension tables are normalized into multiple related tables, reducing redundancy but increasing complexity.
Example:
Customer → City → Country
3. Flat Table (Denormalized / DLAT)
Everything is in one table.
Pros: it is simple
Cons: Large in size, Hard to maintain

Common Data Modeling Issues

Many-to-Many Confusion, Leads to incorrect totals.
Circular Relationships: Power BI blocks these, avoid loops.
Ambiguous Paths: Multiple filter paths lead to wrong calculations.
Poor Naming: Bad names leads confusion later.
Overusing “Both” Filters

Final Thoughts
Data modeling isn’t just technical, it’s strategic.
When you: Choose the right schema, Use relationships properly and
Understand joins, Your Power BI reports become faster, cleaner, and more reliable.

How Excel is Used in Real-World Data Analysis

NelimaL — Tue, 24 Mar 2026 10:46:33 +0000

When I first started learning data analytics, I thought tools like Python or advanced software would dominate everything. But I quickly realized something surprising: Microsoft Excel is still one of the most powerful and widely used tools in real-world data analysis. It’s not just a spreadsheet application it’s a practical, flexible, and accessible platform for exploring, cleaning, and interpreting data.

So, what is Excel?
Excel is a spreadsheet tool that lets you organize data into rows and columns, perform calculations, and create visual reports. That sounds simple, but it goes much deeper than that You can clean messy data, analyze patterns, automate repetitive tasks, and even build small data models all in one place. And the best part is, you don’t need to be a programmer to start using it effectively.

Where Excel shows up in the real world
a) In business and finance, Excel is used to track income, expenses, and profits. People build forecasts and budgets using spreadsheets, helping companies plan ahead.
b) In marketing, teams use Excel to track campaign performance. By analyzing that data, they can figure out what’s working and what’s not.
c) Human resources departments use it to manage employee records, monitor attendance, and even analyze staff perfomance.
d) Even in operations, Excel plays a role in managing inventory and tracking supply chains. It helps businesses stay organized and avoid costly mistakes.

*Excel Features and formulas *

As I’ve been practicing, a few Excel features have really stood out to me because of how practical they are.

1. SUM and AVERAGE
These are basic but essential functions.
=SUM(A1:A10)adds up values.
=AVERAGE(A1:A10) calculates the mean.
In real-world scenarios, these are used to calculate total , average scores, or overall performance metrics.
2. IF Statements
The IF function allows conditional logic:
=IF(A1>50, "Pass", "Fail")
This is useful when categorizing data. For example, a company could use it to label customers as “High Value” or “Low Value” based on their spending.
3. VLOOKUP (and XLOOKUP)
These functions help retrieve data from tables:
=VLOOKUP(A2, A10:B20, 2, FALSE)
In practice, this is used to match and merge datasets like finding a customer’s details based on their ID.
4. Pivot Tables
Pivot tables are one of the most powerful tools in Excel. They allow you to summarize large datasets quickly. For example, you can take thousands of rows of sales data and instantly see:
• Total sales per region
• Sales by product category
• Monthly performance trends This is something analysts use regularly to turn raw data into insights.
5. Charts and Visualization
Excel also allows you to create charts like bar graphs, line charts, and pie charts. These help communicate findings clearly. Instead of just showing numbers, you can visualize trends making it easier for others to understand the data.

Applying this to actual data
What I’m starting to understand is that Excel is not just about knowing formulas, it’s about how you think about data. Before analyzing anything, you often need to clean it. Remove duplicates, fix errors, and make sure everything is consistent. Then you organize it in a way that makes sense.
From there, you can start asking questions:
What’s increasing? What’s decreasing? Is there a pattern? Excel helps answer those questions step by step.

My personal reflection
Learning Excel has definitely changed how I see data. Before, I would look at numbers and not think much about them. Now, I find myself trying to understand what those numbers are actually saying. I pay more attention to patterns, trends, and differences. It’s also made me more curious. Instead of just accepting information, I want to explore it and understand where it comes from. I used to think data analysis was something very technical and complicated. But Excel showed me that it actually starts with simple tools and the ability to think critically.
For me, learning Excel hasn’t just been about spreadsheets, it’s been about learning how to make sense of information in a completely different way.