DEV Community: Mark Glemba

Statistics, Parametric and Non-Parametric Tests in Python

Mark Glemba — Tue, 07 Jul 2026 20:01:44 +0000

Introduction
Statistics is one of the fundamental pillars of data science, machine learning, artificial intelligence, business intelligence, scientific research, healthcare analytics, finance, engineering, and many other disciplines. Every day, organizations collect enormous amounts of data from customers, sensors, websites, social media platforms, financial transactions, and business operations. However, raw data alone provides little value unless it can be analyzed and interpreted correctly. Statistics provides the mathematical framework that enables analysts and researchers to extract meaningful insights from data, identify patterns, test hypotheses, and make evidence-based decisions.
Python has become one of the world’s most popular programming languages for statistical analysis due to its simplicity, extensive libraries, and powerful data manipulation capabilities. Libraries such as NumPy, Pandas, SciPy, Statsmodels, and Scikit-learn provide researchers and analysts with comprehensive tools for performing descriptive statistics, inferential statistics, hypothesis testing, regression analysis, and machine learning.
Among the most important concepts in statistical analysis are parametric and non-parametric statistical tests. Choosing the correct statistical test is essential because using an inappropriate test may produce misleading conclusions. Parametric tests assume that the underlying data follow specific statistical distributions, usually the normal distribution, while non-parametric tests make fewer assumptions and are particularly useful when those assumptions are violated.
This article explores statistics, parametric tests, non-parametric tests, their assumptions, advantages, disadvantages, practical applications, and demonstrates how these tests can be implemented using Python.

Understanding Statistics
Statistics is the science of collecting, organizing, analyzing, interpreting, and presenting data. It helps transform raw numbers into meaningful information that supports decision-making.
Statistics is generally divided into two major branches:

1. Descriptive Statistics
Descriptive statistics summarize and describe the characteristics of a dataset without making conclusions beyond the data collected.
Common descriptive statistics include:
• Mean
• Median
• Mode
• Range
• Variance
• Standard deviation
• Quartiles
• Percentiles
• Skewness
• Kurtosis
For example, a company may calculate the average monthly salary of employees or the average customer spending per transaction.
Python Example:

2. Inferential Statistics
Inferential statistics goes beyond describing data by drawing conclusions about an entire population using a sample.
Inferential statistics helps answer questions such as:
• Is a new medicine more effective than the old one?
• Does education level influence salary?
• Is customer satisfaction different between two stores?
• Does gender affect purchasing behavior?
Inferential statistics relies heavily on probability theory and hypothesis testing.

Hypothesis Testing
Hypothesis testing is a statistical method used to determine whether there is sufficient evidence to support a claim.
Every hypothesis test begins with two hypotheses.

Null Hypothesis (H₀)
The null hypothesis states that there is no significant difference or relationship.
Example:
“There is no difference in average salaries between male and female employees.”
Alternative Hypothesis (H₁)
The alternative hypothesis states that there is a significant difference or relationship.
Example:
“There is a significant difference in average salaries between male and female employees.”

Understanding the p-value
The p-value measures how likely the observed data would occur if the null hypothesis were true.
General interpretation:
• p ≤ 0.05 → Reject H₀
• p > 0.05 → Fail to reject H₀
For example:

Parametric Statistical Tests
Parametric tests are statistical methods that make assumptions about the population distribution.
The most common assumption is that the data follow a normal distribution.
Because they use more information from the data, parametric tests are generally more powerful than non-parametric tests when their assumptions are satisfied.

Assumptions of Parametric Tests
Before applying a parametric test, several assumptions should be checked.

1. Normality
Data should approximately follow a normal distribution.
Python example:

If
p > 0.05
the data are considered approximately normal.

2. Homogeneity of Variance
Groups should have similar variances.
Python example:

3. Independence
Observations should be independent of one another.

4. Continuous Data
Most parametric tests require interval or ratio-level data.

Common Parametric Tests
1. Independent Samples t-Test
Purpose:
Compare the means of two independent groups.
Example:
Compare salaries of male and female employees
Python:

2. Paired t-Test
Purpose:
Compare measurements taken from the same individuals before and after an intervention.
Example:
Employee productivity before and after training.
Python:

3. One-Sample t-Test
Purpose:
Determine whether a sample mean differs from a known population mean.
Python:

4. ANOVA (Analysis of Variance)
Purpose:
Compare means of three or more groups.
Example:
Compare salaries across departments.
Python:

5. Pearson Correlation
Purpose:
Measure the linear relationship between two continuous variables.
Python:

Correlation values range from -1 to +1.

6. Linear Regression
Purpose:
Model the relationship between independent and dependent variables.
Python:

Regression is widely used for prediction and identifying significant predictors.

Advantages of Parametric Tests
Some benefits include:
• Higher statistical power
• More precise estimates
• Efficient with normally distributed data
• Suitable for predictive modeling
• Widely accepted in scientific research

Disadvantages of Parametric Tests
Limitations include:
• Sensitive to outliers
• Require normality assumptions
• Require continuous data
• Can produce misleading results when assumptions are violated

Non-Parametric Statistical Tests
Non-parametric tests are statistical procedures that do not assume a specific population distribution.
They are often called distribution-free tests.
These tests are especially useful when:
• Data are skewed
• Sample size is small
• Data contain outliers
• Data are ordinal
• Normality assumptions fail

Advantages of Non-Parametric Tests
Advantages include:
• Few assumptions
• Handle skewed data
• Robust against outliers
• Suitable for ordinal data
• Useful with small sample sizes

Disadvantages of Non-Parametric Tests
Disadvantages include:
• Lower statistical power than parametric tests
• May ignore some information contained in the data
• Less effective for normally distributed datasets

Common Non-Parametric Tests
1. Mann-Whitney U Test
Equivalent to the independent t-test.
Used for comparing two independent groups.
Python:

2. Wilcoxon Signed-Rank Test
Equivalent to the paired t-test.
Python:

3. Kruskal-Wallis Test
Equivalent to one-way ANOVA.
Python:

4. Friedman Test
Equivalent to repeated-measures ANOVA.
Python:

5. Spearman Rank Correlation
Equivalent to Pearson correlation when data are not normally distributed.
Python:

6. Chi-Square Test of Independence
Used for categorical variables.
Example:
Determine whether education level is associated with employment status.
Python:

Choosing Between Parametric and Non-Parametric Tests
Selecting the appropriate statistical test depends on several factors, including the type of data, the number of groups, whether observations are independent or paired, and whether the assumptions of parametric tests are satisfied.

Before choosing a test, analysts should perform exploratory data analysis (EDA), inspect visualizations such as histograms and box plots, and conduct assumption checks including tests for normality and homogeneity of variance. If assumptions hold, parametric tests are generally preferred because they provide greater statistical power. When assumptions are violated or the data are ordinal or heavily skewed, non-parametric tests offer a more reliable alternative.

Python Libraries for Statistical Analysis
Python offers a rich ecosystem of libraries that simplify statistical analysis and hypothesis testing.

NumPy
Used for numerical computations and array operations.

Pandas
Provides powerful data structures for data manipulation and descriptive statistics.

SciPy
Contains numerous statistical functions and hypothesis tests.

Statsmodels
Used for regression analysis, ANOVA, generalized linear models, and other advanced statistical techniques.

Matplotlib
Creates visualizations that aid in understanding data distributions and statistical results.

These libraries work seamlessly together, making Python one of the most comprehensive environments for statistical computing.

Practical Applications of Statistical Tests
Parametric and non-parametric tests are widely used across industries to solve real-world problems.
In healthcare, researchers use t-tests and ANOVA to compare treatment effectiveness between patient groups, while non-parametric tests are applied when medical data are skewed or involve ordinal pain scores.
In finance, analysts use correlation and regression to examine relationships between stock prices, interest rates, and economic indicators. Statistical tests help determine whether observed trends are significant or due to random variation.
Businesses rely on hypothesis testing to evaluate marketing campaigns, compare customer satisfaction across branches, and assess employee performance after training programs. A/B testing, a common practice in digital marketing, is fundamentally based on statistical hypothesis testing.
Educational institutions analyze examination results to compare teaching methods or determine whether interventions improve student performance. Government agencies apply statistical methods to census data, unemployment rates, and public health studies to guide policy decisions.

Best Practices for Statistical Testing in Python
To ensure valid and reliable results, analysts should follow several best practices:

Understand the research question before selecting a statistical test.
Perform exploratory data analysis to identify missing values, outliers, and unusual patterns.
Check assumptions such as normality, independence, and equal variances before applying parametric tests.
Choose non-parametric alternatives when assumptions are violated or when working with ordinal data.
Report both the test statistic and the p-value, and where appropriate include confidence intervals and effect sizes to provide a more complete interpretation of the findings.
Interpret statistical significance alongside practical significance, recognizing that a statistically significant result may not always have meaningful real-world implications.
Document the methodology and Python code to ensure that analyses are transparent, reproducible, and easy to verify.

Conclusion
Statistics plays an indispensable role in transforming raw data into meaningful knowledge. Through descriptive statistics, analysts summarize and understand datasets, while inferential statistics enables them to make predictions and draw conclusions about larger populations from sample data. Hypothesis testing forms the foundation of inferential analysis by providing an objective framework for evaluating claims and determining whether observed differences or relationships are statistically significant.
Parametric tests, including t-tests, ANOVA, Pearson correlation, and linear regression, are powerful methods when their underlying assumptions—such as normality, homogeneity of variance, and independence—are satisfied. Their higher statistical power makes them the preferred choice for many research applications involving continuous, normally distributed data. In contrast, non-parametric tests such as the Mann-Whitney U test, Wilcoxon signed-rank test, Kruskal-Wallis test, Friedman test, Spearman correlation, and Chi-square test provide flexible and robust alternatives when data violate these assumptions or are measured on an ordinal scale.
Python has revolutionized statistical analysis by offering an accessible and efficient ecosystem of libraries, including NumPy, Pandas, SciPy, Statsmodels, and Matplotlib. These tools enable users to clean data, perform sophisticated statistical tests, visualize results, and build predictive models with relatively little code. As organizations increasingly rely on data-driven decision-making, proficiency in statistical methods and Python programming has become an essential skill for data scientists, researchers, analysts, and professionals across numerous fields. Understanding when to apply parametric or non-parametric methods ensures that analyses are both statistically valid and practically meaningful, ultimately leading to more accurate conclusions and better-informed decisions.

Understanding DDL, DML, and Key SQL Concepts

Mark Glemba — Tue, 14 Apr 2026 07:11:57 +0000

Introduction
Structured Query Language (SQL) is essential for managing and manipulating data in relational databases. In this article, we explore two important categories of SQL commands—Data Definition Language (DDL) and Data Manipulation Language (DML)—along with practical operations such as CREATE, INSERT, UPDATE, DELETE, filtering with WHERE, and the use of CASE WHEN for data transformation.

What DDL and DML Are (and Their Differences)

Data Definition Language (DDL) refers to SQL commands used to define and manage the structure of a database. These commands deal with creating, modifying, and deleting database objects such as tables.

Common DDL commands include

CREATE – used to create new tables or databases
ALTER – used to modify existing structures
DROP– used to delete tables or databases

For example, using "CREATE TABLE" allows you to define columns, data types, and constraints for storing data.

Data Manipulation Language (DML), on the other hand, is used to manage the data within those structures. It focuses on inserting, updating, retrieving, and deleting records.

Common DML commands include:

INSERT – adds new records
UPDATE– modifies existing data
DELETE – removes records
SELECT– retrieves data

Key Difference:
DDL deals with the structure of the database, while DML deals with the data inside the database.

Using CREATE, INSERT, UPDATE, and DELETE

In a typical database assignment, these commands are used as follows:

CREATE: I used this command to define tables such as students, subjects, or exam results. It involved specifying column names, data types (e.g., INTEGER, VARCHAR), and primary keys.
INSERT: This command was used to add records into the tables. For example, inserting student names, subject details, and exam scores into the respective tables.
UPDATE: I used UPDATE to modify existing records. For instance, correcting a student’s score or updating a subject name.
DELETE: This command helped remove unwanted or incorrect records from the database, such as deleting a student entry or clearing outdated data.

These commands are fundamental for maintaining accurate and up-to-date data in any database system.

Filtering Data with WHERE

The WHERE clause is used in SQL to filter records based on specific conditions. It helps retrieve only the data that meets certain criteria.

Some commonly used operators include:

= (Equal to): Selects records that match a specific value
Example: "WHERE score = 80"
> (Greater than): Selects values greater than a given number
Example: "WHERE score > 70"
BETWEEN: Filters values within a range
Example: "WHERE score BETWEEN 50 AND 90"
IN: Matches values within a list
Example: "WHERE subject IN ('Math', 'Science')"
LIKE: Used for pattern matching
Example: "WHERE name LIKE 'J%'" (names starting with J)

The WHERE clause is powerful because it allows precise data retrieval, making queries more meaningful and efficient.

How CASE WHEN Helps in Transforming Data The CASE WHEN statement in SQL is used to perform conditional logic within queries. It allows you to transform data by assigning values based on conditions.

For example, you can categorize student performance:

If score ≥ 70 → “Pass”
If score < 70 → “Fail”

This helps in:

Creating calculated columns
Categorizing or grouping data
Improving readability of query results

CASE WHEN is especially useful in reports, where raw data needs to be interpreted into meaningful insights.

Conclusion

Understanding DDL and DML is crucial for working with databases effectively. While DDL defines the structure of the database, DML allows you to interact with the data itself. Commands like CREATE, INSERT, UPDATE, and DELETE form the backbone of database operations. Additionally, tools like the WHERE clause and CASE WHEN statement enhance your ability to filter and transform data, making SQL a powerful language for data management and analysis.

HOW TO PUBLISH A POWER BI REPORT AND EMBED IT INTO A WEBSITE

Mark Glemba — Mon, 06 Apr 2026 14:10:43 +0000

INTRODUCTION
Power BI is an analytics tool developed by Microsoft that lets a user visualize and share insights on certain data. Power BI is also used transform and clean data using Power Query Editor. Data transformation occurs through; shaping data by filtering, sorting and grouping, adding columns and changing data types through conversion to; numeric, text etc. Data cleaning is done through; handling blanks, standardizing and removing errors.

PUBLISHING PROCESS IN POWER BI

Create a workspace A workspace is like a folder where one stores reports, dashboards and datasets.

. Go to [https://app.powerbi.com/]
. Sign into your account
. Once your logged in, on the left far side click on workspaces

. Click on; + New workspace

2 .Uploading and publishing
. In the power bi desktop in the home page click on publish on the far right.

. Select your workspace where the report will be uploaded
. Once uploaded the dataset will appear on your workspace.

Generating embedding code . Open the report you want embedded . Click on File ~~~~ Embed report . Choose embedding option; either Public (Publish to Web), OR Secure Embed; which requires log in and is more secure. . After clicking on Publish to web, Power BI will generate an embed link + iframe code.

4.Embedding code into website

. Copy entire code
. Open your website HTML file
. Paste the iframe code where you want your report to appear

. Save and open your website where your report will display.

CONCLUSION
Using Power BI with embedding capabilities promotes better collaboration, transparency and efficiency. This empowers users to embrace data-driven solutions that support smarter strategies and improved outcomes.

HOW EXCEL IS USED IN REAL-WORLD DATA ANALYSIS

Mark Glemba — Tue, 31 Mar 2026 19:39:06 +0000

INTRODUCTION
Excel is a tool one can use to organize data for work or any other specific software functions. It allows one to develop graphs for comparison of data and create charts to organize and visualize data.

HOW EXCEL IS USED IN REAL WORLD SCENARIOS

DATA CLEANING
Excel is in various ways for data cleaning by performing functions such as; removing duplicates, trimming spaces, fixing formatting, finding errors, standardizing data, splitting data and filling blanks.
DATA SUMMARIZATION
Excel summarizes large data sets to offer quick insights by use of; pivot tables, formulas and functions, instead of users manually going through large data sets.
VISUALIZING TRENDS
Done by the use of charts; pie, line and bar charts.

FORMULAS USED IN EXCEL
Basic formulas in excel include:

Add: Used to add values of two or more cells. The (+) is used.
Subtract: Used to subtract values of two or more cells. The (-) is used.
Multiply: Used to multiply values of two or more cells. The (*) is used.
Divide: Used to divide values of two or more cells. The (/) is used.

BASIC FUNCTIONS USED IN EXCEL
Functions are used to automate tasks normally performed by formulas in excel. Some popular and basic functions include:

AVERAGE: It sums up and calculate the mean of values within a range given. For example,=AVERAGE(A1:A15).
SUM: It adds up a range of cells. For example,=SUM(A1:A15).
IF: Used to arrange values based on a logical test. The syntax for IF ; =IF(logical_test, value_if_true, [value_if_false]). For example, =IF(N2>30, "young", "old").
COUNTIF: Used to return the number of cells that meeet a certain criteria. The syntax is; =COUNTIF(range, criteria). For example, =COUNTIF(A1:A27, "Nairobi"). NOTE: COUNTIFS function counts the number of cells that meet multiple criteria.
VLOOKUP: Allows one to search for a value on a spreadsheet. Its syntax ; =VLOOKUP(lookup_value, table_array, col_index_num, [range_lookup]). For example, =VLOOKUP(300,A1:A20, 3, "False").

CONCLUSION
In conclusion Excel's versatility makes it an efficient tool for quick analysis and reporting. Its availability allows users to efficiently handle large data sets through processes such as performing calculations and creating meaningful visuals.

Understanding Data Modelling in Power BI: Joins, Relationships and Schemes Explained

Mark Glemba — Tue, 31 Mar 2026 19:13:12 +0000

DATA MODELLING
This is a detailed process that involves creating a visual representation of data and its relationships.

TYPES OF TABLES
FACT TABLE
It contains quantitative data for analysis.
DIMENSIONS TABLE
Provides context to fact table data.
STAR SCHEMA
It contains a single fact table in the center that connects to multiple other dimensiontables.

SNOW FLAKE
Is an extension of a star schema where dimension tables are broken down into subdimensions.

FLAT TABLE
A table that displays data in a simple, two-dimensional format without any relationships to other tables.

SQL JOINS IN POWER BI

INNER JOIN: Returns rows present in both tables if there is a match.
LEFT JOIN: Returns all rows present in the left table and matching rows from the right table.
RIGHT JOIN: Returns matching rows from the left table and all rows present in the SQL right table.
FULL OUTER: Returns all rows present in both right and left tables.
LEFT ANTI: Returns rows from the left table that don't have matches in the right table.
RIGHT ANTI:Returns rows from the right table that don't have matches in the left table.

POWER BI RELATIONSHIPS

One-to-Many (1:M)
One row in table A matches many in table B. For example,

Many-to Many (M:M)
Here one is recommended to use a bridge table.

0ne-to-One (1:1)
One row in table A matches one in table B. For example,

Many-to-One (M:M)
Many rows in table A match one in table B. For example,

Active and Inactive relationships
The active relationship is used by default in reports and calculations, while the inactive relationship is not used unless specified. It is also useful for alternate paths such as multiple dates.

DIFFERENCE BETWEEN JOINS AND RELATIONSHIPS
Joins; combine tables based on a condition, result in new tables with combined columns and are used in power query steps. Relationships; define connections between tables in a data model and are used for; filtering, calculations and visuals across tables.