DEV Community: Fiona Amolo Awuor

Mastering Data Analytics: The Ultimate Guide to Data Analysis

Fiona Amolo Awuor — Sun, 13 Oct 2024 09:02:09 +0000

Introduction
Data is compelling to organizations as it helps optimize their growth and profit. What is data analytics then? Data analytics is the process of gathering data and using data, techniques, and various tools to identify trends and generate useful information that helps in decision-making. Most workspaces utilize data to develop strategic decisions that drive the organization. A deeper insight into customer satisfaction, availability of new business opportunities, and optimized processes is achieved then.

Data Lifecycle
a) Data collection and storage
b) Data processing
c) Data analysis and visualization

Types of data analytics

Prescriptive Analysis
This type of analysis combines the insight from all analyses done previously. This helps to determine which action to take in a current problem. This analysis is more efficient as data performance is improved and it helps to analyze current problems and make decisions. Examples of prescriptive analysis are: Lead scoring in Sales, fraud detention in banks, and email automation in marketing.
Diagnostic Analysis
This analysis answers the question 'Why?'. This helps to find the cause from the insight found in Statistical Analysis. This analysis is useful for identifying behavior patterns. If a new problem arises, then one can have a look at this analysis to help find similar patterns of that problem.
Statistical Analysis
This type of analysis is done using past data in the form of dashboards and it answers "What happens?". The process includes collection, analysis, interpretation, presentation, and data modeling. A set or sample of data can be analyzed using this method. In statistical analysis, there are two main categories used i.e., descriptive and inferential. In descriptive analysis, a sample of or complete numerical data is summarized and used to calculate the mean and deviation for continuous data and frequency and percentages for categorical data. In inferential analysis, a sample from complete data is used and different summaries are made by selecting different samples.
Predictive Analysis
This type of analysis answers the question 'What will happen?'. This question is answered using previously collected data. Predictions of future outcomes are made using current or past data. For instance, current data indicates that the use of AI is high and is expected to increase over the years. Most people have found AI tools efficient and time-saving. This accuracy is measured and it depends on how much detailed the information collected is.

Benefits of data analytics

Helps in understanding how data is compiled.
Data analysis aids in recognizing patterns and trends which help in informed decision-making.
Improves productivity.
Helps in mitigating any risk that might occur.
Aids various organizations to reach their targets and goals.
Identifying sources for a competitive advantage.

Data analytics skills
This field needs one to have a specific set of skills to be a successful data professional. These skills comprise both technical and soft skills. Soft skills are essential in every field of work and they include communication, story-telling, and presentation, attention to detail, organization, time management, and problem-solving.
Technical skills include Python, data analytics and visualization software such as SQL, Excel and Power BI, machine learning algorithms and models, and database management.

Introduction to Python as a Data Analytics Tool

Fiona Amolo Awuor — Mon, 07 Oct 2024 19:08:29 +0000

As I delved into my week two learning at Lux Tech Academy, we were introduced to Python as a beginner language. Here is an overview of an introduction to Python as a data analytics tool. I've gone through numerous videos of data analysts and most of them recommend using Python for data analysis, why? Python is a popular programming language for data analytics as it offers versatility, flexibility, vast libraries, and visualization capabilities to work with large datasets. It's also suitable for data analytics as it blends well with BI tools and databases.

Advantages of Python in Data Analytics

Python is a preferred choice in data analytics since:

It has a syntax that is simple and easy to learn.
Numerous libraries such as NumPy, Pandas, Seaborn, SciPy, and Matplotlib aid in data analysis.
The available libraries such as Matplotlib and Seaborn help in data visualization, aiding in understanding data trends and patterns.
It is flexible and works well with various data sources and databases, allowing for analysis.
It helps to deploy analytical solutions into production using frameworks such as Django and Flask. This helps to apply theoretical concepts to the real business world.

Below are some of the ways in which Python is applied in data analytics

Data Wrangling What exactly is data wrangling? This is the process of gathering raw data, assessing and cleaning it to make it useful. Python is used to import data from various sources, handle missing values, and process large datasets. This helps to reshape data for analysis. Pandas Library is mostly used for data wrangling and manipulation. To use pandas, an environment has to be available i.e., Vs Code or Jupyter Notebook. If pandas isn't installed, run the command:

Pip install pandas

Then run:

import pandas as pd

From there, load a dataset into a pandas data frame.

Exploratory Data Analysis (EDA)
Python helps in data exploration and visualization using libraries like Matplotlib, Seaborn, and Pandas. Coding with Python helps to understand data distribution and relationships using descriptive statistics, pivot tables etc.

Machine Learning
Python offers libraries that aid in machine learning such as Scikit-Learn and Tensor Flow.

How do data analysts use Python in every day life?

Importing data
Various Python libraries such as NumPy, Pandas, BeautifulSoup help in data importation from various sources such as: Csv files and excel spreadsheets, SQL databases, Web APIs and scraping HTML/XML pages and cloud storage. The choice of a library to be used depends on the type and structure of data.

EDA in Python
In the real world, data often contain errors such as missing values and sometimes can be inaccurate thus the need for some cleaning. Exploratory Data Analysis in Python helps in data cleaning by: Identifying anomalies, handling missing data, checking and fixing data types, transforming raw data into a useful and reliable set for analysis.

Transforming data for insights
After cleaning data, python enables various transformation like Pivot tables, concatenating datasets and changing data types and formats.

Descriptive Analysis using Python
SciPy Library has descriptive and inferential statistical functions whereas Matplotlib and Seaborn aid in data visualization.
Python is an interesting language and I can attest it's beginner-friendly. Feel free to leave any question/comment in the comment section.

"SQL 101: Introduction to Structured Query Language."

Fiona Amolo Awuor — Mon, 30 Sep 2024 04:30:55 +0000

As a beginner, I found my first class in SQL intriguing. Let's have a summary of what was taught.
What is SQL?
SQL stands for Structured Query Language and it's a standard programming language used for accessing and manipulating databases, especially in a relational database management system.
A relational database stores information in tabular form, with rows and columns each designated with different values.
SQL is used generally to maintain and optimize database performance. This language has different queries that are used to store, search, update, remove, delete, and retrieve information to help in data analysis.

How is SQL important?
It is considered versatile as it is used in all types of applications. Data analysts and developers use SQL as it blends well with other programming languages.
Components of an SQL system
SQL Table
This is the basic element of a relational database, and the tables usually consist of rows and columns. Database Engineers usually use tables to create relationships between different and many database tables to optimize data storage. For example, a table can be created using the following. Say we are creating a table called Users.

CREATE TABLE Users(
Id INT PRIMARY KEY NOT NULL,
Name VARCHAR (255) UNIQUE,
Email Address VARCHAR (255) UNIQUE
),

SQL Statements
SQL statements, also referred to as SQL queries are instructions that a relational database understands.
Let's dive into SQL commands
These commands are specific keywords or SQL statements that data and systems analysts use to manipulate the data stored in various databases. The commands are categorized as:
Data Definition Language(DDL)
These commands design the database structure. Data Engineers and Analysts use DDL commands to create and modify database objects based on the requirements of a particular project. The CREATE command is used to create database objects such as databases, tables, and views. A data analyst must first create a database that will store data i.e

CREATE DATABASE db_name

The above command will create a database that will be used to store tables and all types of data, enabling data analysis.

Data Query Language(DQL)
These commands are used to retrieve data stored in relational databases. The most common command for retrieving data is the SELECT statement, used for filtering and returning specific results from a database table.

Data Manipulation Language (DML)
These statements are used to add new information or modify the existing records in a table. The INSERT command is used to add and store new records in a database table. For instance, we would like to add new values to a table named Patients.

INSERT INTO Patients (Id, Name, Diagnosis)
VALUES (1, 'Awuor', 'Malaria')

This command can be used to insert one or numerous records in a table.
The UPDATE command is a DML used to modify existing records in a table.

UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

The DELETE statement deletes an unwanted record. The WHERE condition must be present when wanting to delete a record. If absent, all records will be deleted from the table.

DELETE FROM table_name WHERE condition;

Data Control Language (DCL)
These commands are used by database administrators to manage or authorize database access for other users. The GRANT command is used to permit certain applications to manipulate one or multiple tables.

Data Analysis

Fiona Amolo Awuor — Sun, 11 Aug 2024 12:26:54 +0000

Data Analysis is an important aspect of every sector. Organizing and thinking about data is important as one understands what the data does and doesn't contain. What exactly is data analysis? This entails scrutinizing raw data to draw some conclusions from the information retrieved.

Types of data analysis

Prescriptive Analysis
Diagnostic Analysis
Statistical Analysis
Predictive Analysis
Text Analysis

1. Prescriptive Analysis

This type of analysis combines the insight from all analyses done previously. This helps to determine which action to take in a current problem. This analysis is more efficient as data performance is improved and it helps to analyze current problems and make decisions.

2. Diagnostic Analysis

This analysis answers the question 'Why?'. This helps to find the cause from the insight found in Statistical Analysis. This analysis is useful for identifying behavior patterns. If a new problem arises, then one can have a look at this analysis to help find similar patterns of that problem.

3. Statistical Analysis

This type of analysis is done using past data in the form of dashboards and it answers "What happens?". The process includes collection, analysis, interpretation, presentation, and data modeling. A set or sample of data can be analyzed using this method. In statistical analysis, there are two main categories used i.e., descriptive and inferential. In descriptive analysis, a sample of or complete numerical data is summarized and used to calculate the mean and deviation for continuous data and frequency and percentages for categorical data. In inferential analysis, a sample from complete data is used and different summaries are made by selecting different samples.

4. Predictive Analysis

This type of analysis answers the question 'What will happen?'. This question is answered using previously collected data. Predictions of future outcomes are made using current or past data. For instance, current data indicates that the use of AI is high and is expected to increase over the years. Most people have found AI tools efficient and time-saving. This accuracy is measured and it depends on how much detailed the information collected is.

5. Text Analysis

This is also called data mining. This is mostly used to analyze a pattern in large datasets utilizing databases or data mining tools such as SQL. This analysis helps in transforming raw data into useful data i.e., business information.

Data Analysis Process
The process of analyzing data comprises data collection, data processing, data cleaning, data analysis and finally communicating the results. From the first process, data collection entails collecting information from all relevant sources to help in answering a problem and evaluating outcomes. Data cleaning is crucial in data analysis as it helps to identify and correct errors. This process is the detection of coding errors and correcting them. Two types of data cleaning processes are applied, code cleaning and contingency cleaning. These are very important and shouldn't be ignored as this might lead to the production of misleading data. Then, data analysis is done after data cleaning.

Data Analysis Tools

Data Analysis tools help in interpreting data for easy understanding. Different tools are used to analyze data, some of which require little coding such as R in data mining and Python, Tableau, and Power BI in data visualization.
R programming language is used for statistical computing and graphics supported by the R Foundation for Statistical Computing. This language is widely used by statisticians and data miners for software development and data analysis. This language helps in manipulating data and helps in presenting information in numerous ways.
Tableau offers free software that links information sources, which is then communicated with a customer or social media.
Python is a user-friendly and open-source language that can be read, written, and maintained. It was created by Guido Van Rossum in the 1980s and it supports both functional and structured techniques of programming. It contains several libraries for machine learning, such as Keras, Tensor Flow, Scikitlearn, etc. Besides, data visualization is possible using Seaborn, Box, and Scatter Plots.

Importance of data analytics for businesses

Positive impact on businesses has been noted as data analysis improves efficiency. This helps to evaluate employee performance.
A deeper understanding of the market is enabled. Huge datasets are collected and analyzed due to the development of algorithms today. Data collection is conducted in a raw form from a wide variety of people, which helps with feedback for the formation of better marketing strategies.
Data analytics helps in faster and better decision-making. Deadlines are met with ease as high-speed in-memory analytics save time.
Data analytics is powerful as new products/services are formed. With the data collected, it is easier to decipher the needs of consumers and this ensures the formation of new products/ services that align with the values of the target market.
Industry knowledge is enabled. Data analysis helps to predict the future trend of a certain industry as it shows the kind of present economy. Also, new opportunities for businesses are developed as brands are built.