DEV Community: kinuthia john

Understanding Your Data: The Essentials of Exploratory Data Analysis

kinuthia john — Sun, 11 Aug 2024 17:14:19 +0000

Exploratory Data Analysis commonly known as EDA is a crucial step in data analysis as it helps examine and explore the characteristics of a dataset and through this, we are able to gain an understanding of the data
This step has different processes which are:

Data visualization which is the physical representation of the dataset using graphs, charts, and other forms of visualization. This helps us to identify trends in the data and how data in different columns relates to each other. Python Libraries such as seaborn and Matplotlib are greatly used to visualize the data.

sns.displot(df, x='Temp_C')
sns.displot(df, x='Dew_Point_Temp_C',kde=True)

The snippet above helps us create bar graphs to visualize the temperature and dew point temperature columns.

Descriptive analysis of the data is the statistical analysis of the different columns to get values such as the mean, standard deviation, range, the max and minimum values of the columns.This is made possible by the following code:

df.describe(include ="all")

Cleaning the dataset- removal of errors in the dataset is done in this stage. It helps us deal with missing and inconsistent values thus making it easier for us to get accurate information from the data.
Identifying and removing outliers-outliers are anomalous data points in the dataset and they deviate significantly from the other values.It is important to identify them because they can make it difficult to draw meaningful conclusions as they skew the results.There are various methods to identify outliers and the most common one is the box model.Points outside the box are considered as outliers and need to be removed.
Understanding the distribution of data in the dataset-This helps to determine the statistical methods to be used in analysis.

In conclusion, EDA is very important as it helps us to get more information and identify patterns from the data and this helps us in making accurate choices in the statistical methods used in analysis.

Data science:Wanna master the art of storytelling using data?

kinuthia john — Sun, 04 Aug 2024 16:52:58 +0000

I know we all have come across the term data science, but have you ever thought about what it is and its impacts on the tech world?

Well, today we'll dive deeper into what the field of data science entails

What is data science?

Data science is a tech-related field that uses various scientific methods to identify a problem in society, analyze it, collect meaningful data concerning the domain, clean the data, and structure it so that we can get some meaningful insights and knowledge to solve the specific problem.
Data analyst
This person specializes in the analysis of complex datasets from various sources and can extract information that can be used to make decisions in an organization.

What it takes to be a data scientist

The following are among the few key skill requirements for having a successful data science career:

Basic computer science knowledge

To be a data scientist, you need to have the fundamental knowledge of computer science which helps you to understand various concepts such as core maths, computer basics, database management and artificial intelligence.

programming

One also needs to understand programming languages such as Python, R and SQL which are mostly used by data scientists in their day-to-day life. The knowledge is practically applied in data manipulation and performing complex mathematical functions by use of simple libraries such as numpy in python.

Data collection and preparation

A data scientist must have solid skills in collecting and cleaning the collected data for easier analysis.This skill is key because clean data is easier to work with and also make accurate decisions from the data.

Data examining

The collected data should be examined to understand its flow and the overall characteristics which helps the data scientist to understand his data well.

Data analysis

In order to get insights from data,some data analysis skills must be applied by the scientist so as deep knowledge of statistics and mathematics is required.

Data modeling

Data modeling is the process of fitting and developing machine learning models that can predict possible outcomes and future patterns from the data. This requires one to have skills in the usage of various machine-learning models based on the type of outcome they need and the type of data present.

Data visualization

Visualization skills are key as they help us create visual presentations of findings and insights gained from the data. The tools used include Tableau and powerbi.

These are just but a few skills required to get you started in the field. Remember that the field of data is very wide and one requires to practice and learn daily so as to get better and always remember that Rome was never built in one day.