<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Hews</title>
    <description>The latest articles on DEV Community by Hews (@kamau001).</description>
    <link>https://dev.to/kamau001</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1174565%2Fadcd9c18-1658-43d1-a4f6-c04d31ffe80b.png</url>
      <title>DEV Community: Hews</title>
      <link>https://dev.to/kamau001</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/kamau001"/>
    <language>en</language>
    <item>
      <title>Data Engineering For Beginners</title>
      <dc:creator>Hews</dc:creator>
      <pubDate>Thu, 09 Nov 2023 19:07:26 +0000</pubDate>
      <link>https://dev.to/kamau001/data-engineering-for-beginners-17m3</link>
      <guid>https://dev.to/kamau001/data-engineering-for-beginners-17m3</guid>
      <description>&lt;p&gt;&lt;strong&gt;Who is a Data Engineer?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fw4pow9ybny6xzxydv6b5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fw4pow9ybny6xzxydv6b5.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Imagine you're opening a small store, and the first thing you'd do is decide what stuff you want to sell and how you're going to get it. Well, big companies with lots of data also need a plan to get their information out of storage and use it effectively. That's where data engineers come in—they're like the behind-the-scenes experts who bring in raw data from different places and get it ready for big business applications. Before we dive into the nitty-gritty of what they do, let's first understand why there's such a demand for these jobs in various industries.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;In-Demand Data Engineering Jobs&lt;/strong&gt;&lt;br&gt;
       Ten years ago, being a Data Scientist was considered the hottest job of the century. But here's the twist: the buzz around it might not be as real as we think. According to data from the Data Science Interview Report by interviewquery, the number of job interviews for Data Scientist roles only went up by 10% in 2020. Surprisingly, the interviews for Data Engineering roles shot up by a whopping 40% in the same year. Glassdoor, a job-search website, even kicked Data Scientist jobs from their top spot for the first time since 2016.&lt;br&gt;
        In 2019 and 2020, the number of job positions for Data Scientists stayed pretty much the same. However, other data-related jobs like Data Engineers, Business Analysts, Machine Learning Engineers, and Data Analysts are becoming more in-demand to make up for this leveling off.&lt;/p&gt;

&lt;p&gt;As per the report by DICE in 2020, Data Engineer emerged as the fastest-growing job in 2015, with a growth rate of 50% year-on-year&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fuhhcd2rgl0avkjcd5l0x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fuhhcd2rgl0avkjcd5l0x.png" alt="Image description"&gt;&lt;/a&gt;&lt;br&gt;
The other websites also suggest something similar, as can be noted from the mentions below:&lt;/p&gt;

&lt;p&gt;Burning Glass Nova Platform reported 88% year-on-year growth.&lt;/p&gt;

&lt;p&gt;Hired State of Software Engineer Report revealed a 45% increase in data engineer job roles, again year-on-year.&lt;/p&gt;

&lt;p&gt;LinkedIn’s Emerging Job Report for 2020 also presented 33% year-on-year growth stats for data engineer jobs.&lt;/p&gt;

&lt;p&gt;Additionally, as more and more companies rely on cloud solutions, there is an urgent need to hire many data engineers to provide essential support to the team of data scientists. According to the website comakeit, the big data and data engineering services market is estimated to grow from 18% per annum in 2017 to 31% p.a. in 2025.&lt;/p&gt;

&lt;p&gt;Thus, now is the right time if you plan to transition to a data engineering career from your current job. To get more clarity on the role of data engineers, continue reading the next section that highlights the roles and responsibilities of data engineers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What does a Data Engineer do?&lt;/strong&gt;&lt;br&gt;
      A data engineer is like the frontline hero when it comes to dealing with a company's most valuable asset—data. Their main job is to make sure that all the different teams in the company can easily dig into the data and use it for whatever they need. They do this by sourcing the data using something called ETL pipelines and making it all nice and easy for everyone in the organization to understand. But that's not all they do—data engineers usually have a bunch of other tasks up their sleeves to keep things running smoothly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Role and Responsibilities of a Data Engineer&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdruwa9x369bnk973yb19.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdruwa9x369bnk973yb19.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Prepare, Manage, and Supervise Data Pipelines:&lt;/em&gt;&lt;br&gt;
Get things ready, handle, and oversee the pathways that let data move around efficiently.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Construct and Launch ETL/ELT Pipelines:&lt;/em&gt;&lt;br&gt;
Build and set in motion these pipelines that start by bringing in data and then handle various data tasks.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Gather and Manage Data from Various Sources:&lt;/em&gt;&lt;br&gt;
Collect and handle data from different places based on what the business needs.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Team Up to Create Algorithms for Various Data Stuff:&lt;/em&gt;&lt;br&gt;
Work with a group to come up with step-by-step instructions for how data is stored, collected, accessed, checked for quality, and maybe even used for data analysis.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Collaborate with Data Scientists and Set Up the Tools for Making Things Better:&lt;/em&gt;&lt;br&gt;
Connect with the data experts and set up the systems needed to figure out, plan, and put into action improvements to how things work inside the company.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Use Tools like SQL and Big Data Tech to Get Data from Different Places:&lt;/em&gt;&lt;br&gt;
Access different data sources using tools like SQL and fancy Big Data tech to build smart pipelines that move data around.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Bonus Points for Knowing Tools Like Snowflake:&lt;/em&gt;&lt;br&gt;
Having experience with tools like Snowflake is like having an extra skill in your pocket.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Create Solutions Focused on Good Data, Smooth Operations, and Other Cool Features:&lt;/em&gt;&lt;br&gt;
Build answers that make s**ure the data is top-notch, everything runs smoothly, and other special features that describe the data.&lt;br&gt;
Write Scripts and Solutions to Move Data Between Different Spaces:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Data Engineer Salary&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fifz6xsiwr8xls20ffxgl.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fifz6xsiwr8xls20ffxgl.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;As the demand for data engineers keeps going up, so do the salary expectations for this role. Data engineering is not just rewarding in terms of job satisfaction but also financially. Let's take a look at the average annual salaries for data engineers in some major countries around the world:&lt;/p&gt;

&lt;p&gt;In the United States, the average annual salary for a data engineer is approximately $115,157. This is notably higher than the average earnings of a Data Scientist ($101,995) or a Software Engineer ($93,965).&lt;/p&gt;

&lt;p&gt;In India, the average annual salary for a data engineer is ₹10,70,746.&lt;/p&gt;

&lt;p&gt;Data engineers in the United Kingdom earn an average annual salary of £48,481.&lt;/p&gt;

&lt;p&gt;Down under in Australia, a data engineer can expect an average yearly compensation of A$110,000.&lt;/p&gt;

&lt;p&gt;Over in Germany, data engineers bring in an average income of €64,702 per year.&lt;/p&gt;

&lt;p&gt;In Russia, a Data Engineer can anticipate an average yearly income of 2,24,492 PP.&lt;/p&gt;

&lt;p&gt;After learning about the enticing job description of a data engineer and the attractive salary figures, you might be curious about what skills you need to jump onto the data engineering bandwagon. We'll delve into that in the next section.&lt;br&gt;
&lt;strong&gt;Data Engineer Skills&lt;/strong&gt;&lt;br&gt;
      Here is a concise list of technical skills required to become a big data engineer. You will also find a sample project idea to help you grab these skills in the most practical manner and ace your next data engineering interview.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqdorp4iydxfegbo5u3te.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqdorp4iydxfegbo5u3te.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Passion/Enthusiasm for Data-Driven Decision Making&lt;/strong&gt;&lt;br&gt;
Fall in love with your data; your data will love you back. Yes, it’s that simple. To start with data engineering, you need the right mindset to learn it. And by the right mindset, we simply mean the desire to learn something new and challenging. The art of curating valuable inferences using data is not that old and has only recently reached an exciting peak. So, it is likely that you will encounter problems that will demand extra effort, but if you have strong willpower, you can easily ace this domain.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Structured Query Language or SQL (A MUST!!):  Learn to Interact with the DBMS Systems&lt;/strong&gt;&lt;br&gt;
Many companies keep their data warehouses far from the stations where data can be accessed. The role of a data engineer is to use tools for interacting with the database management systems. And one of the most popular tools, which is more popular than Python or R, is SQL. So, ensure that you are well-versed in various SQL commands, syntax, and use-cases for deducing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Knowledge of a Programming/Scripting Language&lt;/strong&gt;&lt;br&gt;
You won't have to spare extra time, but you must practice at least one programming language - Java or Python as most data engineers require them in their day-to-day activities. The role of a big data engineer involves analyzing data with simple statistics and graphs. A data engineer relies on Python and other programming languages for this task. &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Understand the Fundaments of Cloud Computing&lt;/strong&gt;&lt;br&gt;
Eventually, every company will have to shift its data-related operations to the cloud. And data engineers are the ones that are likely to lead the whole process. Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure are the three top-most competitors in cloud computing service platforms. So, if you are aiming for a cloud data engineer job, spend time learning about the fundamentals of cloud computing and work on projects that give you a hint of how to utilize at least one of the three platforms for real problems.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Know-How of Data Warehousing and ETL Tools&lt;/strong&gt;&lt;br&gt;
The previous section of this article precisely highlighted that data engineers are required to build efficient ETL/ELT pipelines. These data pipelines are fundamental to any organization that wants to source data organized and efficiently. And to achieve that, there are tools like Snowflake, Star, etc., for working on cloud data warehouses. Whether an aspiring data engineer or database administrator, data warehousing skills are essential to building a successful data engineering career.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Big Data Skills&lt;/strong&gt;&lt;br&gt;
We are living in the age of information, that too of the size of petabytes. And for handling such large datasets, the Hadoop ecosystem and related tools like Spark, PySpark, Hive, etc., are prevalent in the industry. So, as a data engineer who is required to interact with large datasets, having experience with such Big Data tools is a must. &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;New-Age Data Engineering Tools&lt;/strong&gt;&lt;br&gt;
So far, we have discussed common data engineering skills, but recently, many new tools have come into use, for example, Snowflake for warehousing, dbt for ELT, Airflow for orchestration, etc. Make sure you always look for such tools and practice a few projects around them.&lt;/p&gt;

&lt;p&gt;Apart from acquiring the essential skills, you can also sign up for any data engineer course that will help you better understand the fundamental data engineering concepts and make the best use of ProjectPro platform to work on real-world data science projects to master those skills.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How to Become a Data Engineer&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Now that you have learned all about the skills and responsibilities of a data engineer role, you are likely to be curious about the steps to start learning data engineering. So, here are a few basic steps you must follow to start your career in the data engineering field.&lt;/p&gt;

&lt;p&gt;The first step is to obtain a degree in a relevant discipline related to Big Data, such as computer science, software engineering, etc.&lt;/p&gt;

&lt;p&gt;Focus on building skills specifically in computer science programming, data analysis, data modeling, machine learning, etc.&lt;/p&gt;

&lt;p&gt;Complete a few relevant certifications for various big data and cloud computing tools.&lt;/p&gt;

&lt;p&gt;Learn more about these tools by working on real-world problems.&lt;/p&gt;

&lt;p&gt;Start applying for a few data engineering jobs to understand the industry demands and plan your path accordingly.&lt;/p&gt;

&lt;p&gt;If you are willing to know how to become a data engineer without a degree, the below section will help you understand the steps you need to follow.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How to Become a Data Engineer Without a Degree?&lt;/strong&gt;&lt;br&gt;
Even without a degree, one can still work as a data engineer because there is no specific university degree for the profession.&lt;/p&gt;

&lt;p&gt;Suppose you decide not to get a degree. In that case, you can still get certified as a software engineer through an online course and gain valuable experience as a developer. Becoming a skilled software engineer is the first step toward becoming a good data engineer.&lt;/p&gt;

&lt;p&gt;Another option is to learn data engineering fundamentals if you don't have a degree. You should be familiar with the basics of computer science to explore the field of data engineering easily. To become a data engineer, one must have a solid understanding of programming languages and mathematics.&lt;/p&gt;

&lt;p&gt;You should also look for volunteer work and internships since many organizations provide these alternatives and long- or short-term projects on data engineering to develop employees' skills. A data engineer's career can progress rapidly in the freelance and open-source markets. These places don't require professional degrees, only skills.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;FAQs&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;em&gt;Is Data Engineering a Good Career?&lt;/em&gt;&lt;br&gt;
Yes, Data engineering is one of the hottest careers right now. It can be verified by the 2020 report from DICE, which revealed that Data Engineer emerged as the fastest-growing job in 2015 with a growth rate of 50% year-on-year.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;em&gt;How can I start a career in data engineering?&lt;/em&gt;&lt;br&gt;
To start your career in data engineering, first, look at the roles and responsibilities of a data engineer and the skills required to become one. After that, focus on honing the skills and working on real-world data engineering projects.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;em&gt;How long does it take to become a data engineer?&lt;/em&gt;&lt;br&gt;
It takes around four to six months to become a data engineer after pursuing a bachelor's or master's in data engineering. You need to work hard and stay focused on acquiring the right skills and industry-level expertise to launch your career in data engineering.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;em&gt;Is it hard to become a data engineer?&lt;/em&gt;&lt;br&gt;
It is not hard to become a data engineer. Anyone can master the necessary skills to become a data engineer with hard work, time, and dedication.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

</description>
    </item>
    <item>
      <title>A Complete Guide to Get a Grasp of Time Series Analysis</title>
      <dc:creator>Hews</dc:creator>
      <pubDate>Fri, 27 Oct 2023 14:57:02 +0000</pubDate>
      <link>https://dev.to/kamau001/a-complete-guide-to-get-a-grasp-of-time-series-analysis-3387</link>
      <guid>https://dev.to/kamau001/a-complete-guide-to-get-a-grasp-of-time-series-analysis-3387</guid>
      <description>&lt;p&gt;&lt;strong&gt;What Is Time Series Analysis?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Time-series analysis is a method of analyzing a collection of data points over a period of time. Instead of recording data points intermittently or randomly, time series analysts record data points at consistent intervals over a set period of time.&lt;/p&gt;

&lt;p&gt;While time-series data is information gathered over time, various types of information describe how and when that information was gathered. For example:&lt;/p&gt;

&lt;p&gt;Time series data: It is a collection of observations on the values that a variable takes at various points in time.&lt;br&gt;
Cross-sectional data: Data from one or more variables that were collected simultaneously.&lt;br&gt;
Pooled data: It is a combination of cross-sectional and time-series data.&lt;br&gt;
The variable varies according to the probability distribution, showing which value Y can take and with which probability those values are taken.&lt;/p&gt;

&lt;p&gt;Yt = μt + εt &lt;/p&gt;

&lt;p&gt;Each instance of Yt  is the result of the signal μt&lt;/p&gt;

&lt;p&gt;εt is the noise term here.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why Do We Need Time-Series Analysis?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Time series analysis has a range of applications in statistics, sales, economics, and many more areas. The common point is the technique used to model the data over a given period of time.&lt;/p&gt;

&lt;p&gt;The reasons for doing time series analysis are as follows:&lt;/p&gt;

&lt;p&gt;Features: Time series analysis can be used to track features like trend, seasonality, and variability.&lt;br&gt;
Forecasting: Time series analysis can aid in the prediction of stock prices. It is used if you would like to know if the price will rise or fall and how much it will rise or fall.&lt;br&gt;
Inferences: You can predict the value and draw inferences from data using Time series analysis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Time Series Analysis Example&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Non-stationary data—that is, data that is constantly fluctuating over time or is affected by time—is analyzed using time series analysis. Because currency and sales are always changing, industries like finance, retail, and e-commerce frequently use time series analysis. Stock market analysis, especially when combined with automated trading algorithms, is an excellent example of time series analysis in action.&lt;/p&gt;

&lt;p&gt;Time series analysis can be used in - &lt;/p&gt;

&lt;p&gt;Rainfall measurements&lt;br&gt;
Automated stock trading&lt;br&gt;
Industry forecast&lt;br&gt;
Temperature readings&lt;br&gt;
Sales forecasting&lt;br&gt;
Consider an example of railway passenger data over a period of time. &lt;/p&gt;

&lt;p&gt;On the X-axis, we have years, and on the Y-axis, you have the number of passengers.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--j9JYTEVQ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mmm8mdjvwlcln2g3f4nk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--j9JYTEVQ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mmm8mdjvwlcln2g3f4nk.png" alt="Image description" width="512" height="381"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The following observations can be derived from the given data.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Trend: Over time, an increasing or decreasing pattern has been observed. The total number of passengers has risen over time.&lt;br&gt;
Seasonality: Cyclic patterns are the ones that repeat after a certain interval of time. In the case of the railway passenger, you can see a cyclic pattern with a high and low point that is visible throughout the interval.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Time Series Analysis Types&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Some of the models of time series analysis include - &lt;/p&gt;

&lt;p&gt;Classification: It identifies and assigns categories to the data.&lt;br&gt;
Curve Fitting: It plots data on a curve to investigate the relationships between variables in the data.&lt;br&gt;
Descriptive Analysis: Patterns in time-series data, such as trends, cycles, and seasonal variation, are identified.&lt;br&gt;
Explanative analysis: It attempts to comprehend the data and the relationships between it and cause and effect.&lt;br&gt;
Segmentation: It splits the data into segments to reveal the source data's underlying properties&lt;br&gt;
&lt;strong&gt;ARIMA&lt;/strong&gt;&lt;br&gt;
ARIMA is an acronym for Autoregressive Integrated Moving Average. The Box-Jenkins method is another name for this method.&lt;/p&gt;

&lt;p&gt;_Now you will explore the ARIMA parameters in detail:&lt;br&gt;
_&lt;br&gt;
Autoregressive Component: AR stands for autoregressive, and is denoted by p. When the value of p is 0, it means there is no correlation in the series. When the value of p is 1, it means that the auto-correlation is up to one lag.&lt;br&gt;
Moving Average: Moving average is denoted by q. When q=1, it means that there is an error term.&lt;br&gt;
Integration: Integration is denoted by d. When the value of d is 0, the series is stationary. When the value of d is 1, the series is not stationary, and you can make it stationary by taking the difference. &lt;br&gt;
&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;br&gt;
Time series analysis has a wide range of applications and is one of the most important areas of study. It plays an important role in forecasting models and meaningful statistical characteristics.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Exploratory Data Analysis using Data Visualization Techniques!</title>
      <dc:creator>Hews</dc:creator>
      <pubDate>Fri, 13 Oct 2023 14:24:20 +0000</pubDate>
      <link>https://dev.to/kamau001/exploratory-data-analysis-using-data-visualization-techniques-18gf</link>
      <guid>https://dev.to/kamau001/exploratory-data-analysis-using-data-visualization-techniques-18gf</guid>
      <description>&lt;p&gt;&lt;strong&gt;What is Exploratory Data Analysis?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We can define exploratory data analysis as the essential data investigation process before the formal analysis to spot patterns and anomalies, discover trends, and test hypotheses with summary statistics and visualizations. It gives an idea about the data we will be digging deep into while analyzing. It aids in formulating how we can handle data during analysis, like choosing models, handling outliers, deciding model accuracy parameters, etc. Visualization helps to infer insights easily from massive datasets.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Exploratory Data Analysis Example&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Suppose you decide to go out for dinner on holiday in order to enjoy good quality food and a peaceful time with your family. What would you do? You would see how far you want to travel, what kind of cuisine you are craving, can you book a table so that you don’t have to wait, and of course the price range. So, based on these parameters, you will explore available options and finally pick the desired restaurant. This is nothing but exploratory data analysis. So, The only difference is, data scientists perform it on a much larger scale, with complicated parameters, statistical parameters, and much more. &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Types of Exploratory Data Analysis&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Univariate Plots&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Univariate plots show the frequency or the distribution shape of a variable.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Histograms&lt;/strong&gt;&lt;br&gt;
Histograms are two-dimensional plots in which the x-axis divide into a range of numerical bins or time intervals. The y-axis shows the frequency values, which are counts of occurrences of values for each bin. Bar graphs have gaps between the bars to indicate that they compare distinct groups, but there are no gaps in histograms. Hence, They tell us if the distribution is left/positively skew  (most of the data falls to the right side), right/negatively skewed (most of the data falls to the left side), bi-modal (graphs having two distinct peaks), normal (perfectly symmetrical without skew), or uniform (almost all the bins have similar frequency).&lt;/p&gt;

&lt;p&gt;For example, this histogram shows the maximum wind speed of all the hurricanes. Since, We can observe that most of the distribution is saturate on the left side between 30 – 40 Kt. This tells us that the maximum wind speeds are concentrate on the lower side, indicating that not many storms are severe.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Probability Distribution Plots&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Probability distributions are mathematical functions that describe all the possible values that a random variable can assume within a given range. They help model random phenomena, allowing us in order to estimate the probability of a particular event. This type of distribution is helpful to know the likely outcomes and the spread of potential values.&lt;/p&gt;

&lt;p&gt;For a single random variable, probability distributions can be divided into two types:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Types&lt;/strong&gt;&lt;br&gt;
-Discrete Probability Distributions for Discrete Variables: Also known as probability mass functions, the random variable can assume a discrete number of values like the number of reviews; it can be 100 or 101, but nothing in between. It returns probabilities; hence the output is between 0 and 1. There are a variety of discrete probability distributions that you can use to model different types of data.&lt;/p&gt;

&lt;p&gt;Binomial Distribution&lt;br&gt;
Let us look at the binomial distribution. So, There are two possible outcomes in this distribution – success or failure, and multiple trials are carried out. The probability of success and failure is the same for all trials. The sum of all probabilities must equal one.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Success Probability&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So, let’s say that there is a success probability of 0.8 of manufacturing a perfect car engine part. What is the probability of having seven successes in 10 attempts? The probability of success is 0.8, and failure is 0.2. The number of trials is ten, and the number of successes is 7. The figure shows the probability of success (getting the perfect engine part 0 times, one time, two times, and so on). From the graph, we can conclude that the probability of seven successes in 10 attempts is around 0.20.&lt;/p&gt;

&lt;p&gt;-Probability Density Functions for Continuous Variables: Also known as probability density functions, the random variable can assume an infinite number of values between any two values; like weight can take any value like 45.3, 45.36. 45.369, or 45.3698, and so on.&lt;br&gt;
Probabilities for continuous distributions are measured over ranges of values rather than single points. A probability indicates the likelihood that a value will fall within an interval. The entire area under the distribution curve equals 1. For instance, the proportion of the area under the curve that falls within a range of values along the X-axis is the likelihood that a value will fall within that range.&lt;/p&gt;

&lt;p&gt;Probability Density Function&lt;br&gt;
Suppose we have a dataset of adults’ heights in a town, and the data follows a normal distribution. The mean equals 5.5, and the standard deviation is 1. The shaded area shows the probability that a randomly picked person’s height will be smaller than 4.5 ft and is approximately equal to 0.15 or 15%.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Run Sequence Plots&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A run chart, also known as a run-sequence plot, displays observed data in a time sequence. So, Often, the data displayed represents some aspect of a business process’s output or performance. It is, therefore, a form of a line chart. They are often analyzed in order to locate anomalies in data that suggest shifts in a process over time. Changes in location and scale and outliers can easily be detected.&lt;/p&gt;

&lt;p&gt;sales vary&lt;br&gt;
Run Chart&lt;br&gt;
The graph shows how the sales vary across months and years with the average displayed by a line.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bivariate Plots&lt;/strong&gt;&lt;br&gt;
Bivariate plots display the relationship between two variables in exploratory data analysis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bar Graphs&lt;/strong&gt;&lt;br&gt;
Bar charts can be used to compare nominal or ordinal data. They are helpful for recognizing trends.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scatter Plots&lt;/strong&gt;&lt;br&gt;
Scatter plots are commonly used in statistical analysis in order to visualize numerical relationships. So, They are use in order to determine whether two measures are correlate by plotting them on the x and y-axis. They are suitable for recognizing trends.&lt;/p&gt;

&lt;p&gt;For instance, you can see a scatter plot of two measures in the figure – the house’s area against price and the trend line. The data points are concentrated in the lower price and lower area range. A few outliers are indicating larger area houses available for lower prices.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Box Plots&lt;/strong&gt;&lt;br&gt;
These charts show the distribution of values along an axis. Rectangular boxes are used in order to bucket the data, giving us an idea of how the data points are spread out. These boxes are also called quartiles which represent a quarter of a data set. Boxes can be drawn vertically or horizontally. We can also easily spot the outliers, which are usually treat as abnormal values and affect the data set’s overall observation due to their very high or low values.&lt;/p&gt;

&lt;p&gt;Box plots are suitable for identifying outliers. The below figure shows the structure of a box plot.&lt;/p&gt;

&lt;p&gt;For instance, this example shows the maximum wind speed of different hurricanes with their names that occurred in 2014. We can conclude that Gonzalo has a median of 85, which is the highest as compare to other storms. Hence, This indicates that this was the strongest hurricanes in 2014; Edouard has the maximum spread of data points in Q3, Bertha has some outliers which need to be investigated.&lt;/p&gt;

&lt;p&gt;This gives us quite a few ideas about storms in 2014.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Correlation Plots (Heat Maps)&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For instance, correlation heat maps show the interrelationship between variables—areas as shaded as per the data’s values. So, Color differences can easily spot similar and different values and make sense of the data variation. They are usually helpful when you have a large amount of data. They are used during A/B testing to see which parts of a web page are accessed by users on a website. The number of reviews generated every hour, or to analyze a cricket match to understand where a batsman is scoring the bulk of his runs or where the bowler is pitching his ball.&lt;/p&gt;

&lt;p&gt;We can see that the arrival delay in SFO is maximum from Chicago O’Hare (ORD) origin. Delta(UA) and Sky West (OO) carriers have maximum arrival delays to SFO from different origins than other carriers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Special Purpose Plots&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Pair Plots&lt;br&gt;
Pair plots are a simple way in order to visualize relationships between multiple variables. So, It produces a matrix of relationships between variables in the data for a direct examination of the data. &lt;/p&gt;

&lt;p&gt;This plot shows how registered and casual users are using bike rentals. It also shows the effect of temperature, humidity, and wind speed on bike rentals. This gives you an overview of the correlation between multiple variables.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Contour Plots&lt;/strong&gt;&lt;br&gt;
The contour plot can be use for representing a 3D surface in a 2D format. Contour plots are generally use for continuous variables rather than categorical data.&lt;/p&gt;

&lt;p&gt;The Contour maps are inspired by seismic data analysis. They can explain where the data density is high, explore deep learning error functions or gradient analysis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Density Plots&lt;/strong&gt;&lt;br&gt;
A density plot is a smoothed, continuous version of a histogram estimated from the data. The most common form of estimation is the kernel density plot. In this method, a continuous curve (the kernel) is drawn at every individual data point. All of these curves are then combine to make a single smooth density estimation.&lt;/p&gt;

&lt;p&gt;So, The y-axis in a density plot is the probability density function for the kernel density estimation and not a probability. The difference is that the probability density is the probability per unit on the x-axis.&lt;/p&gt;

&lt;p&gt;While comparing the distributions of one variable across multiple categories, histograms have issues with readability. Density plots are useful in this scenario.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Spider/Radar Charts&lt;/strong&gt;&lt;br&gt;
A spider chart is a graphical way of displaying multivariate data of three or more quantitative variables represented on axes starting from the same point. It helps demonstrate a dominant variable.&lt;/p&gt;

&lt;p&gt;This spider/radar chart is showing the variation in profit for each state across quarters. We can see the steepest drop in profits for Tamil Nadu from Q1 to Q2.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lag Plots&lt;/strong&gt;&lt;br&gt;
A relationship between an observation and the previous observation is beneficial in time series modeling. Previous observations in a time series are lags, with the observation at one previous time step. It is known as lag1, the observation at two previous steps lag2, and so on &lt;/p&gt;

&lt;p&gt;A lag plot is a useful type of plot in order to explore each observation’s relationship, and a lag of that observation and is display as a scatter plot. If the points cluster along a diagonal line from the bottom-left to the plot’s top-right, it suggests a positive correlation relationship. If the points cluster along a diagonal line from the top-left to the bottom-right, it means a negative correlation relationship.&lt;/p&gt;

&lt;p&gt;Lag plots can help compare observations simultaneously in the last week or last month, or the previous year by using corresponding lag values.&lt;/p&gt;

&lt;p&gt;The plot here shows the count of bike rentals compared to the previous day’s count, and it displays a relatively strong positive correlation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Auto-Correlation Plots&lt;/strong&gt;&lt;br&gt;
The correlation between observations and their lag values in a time series name autocorrelation. Correlation coefficients are plotted on an autocorrelation plot.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Auto-Correlation Plot&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A correlation coefficient is a correlation value between observations and their lag1 values and results in a number between -1 and +1. A value close to zero suggests a weak correlation, whereas a value closer to -1 or 1 indicates a strong correlation. It helps better understand how this relationship changes over the lag. It shows the lag on the x-axis and the correlation on the y-axis. &lt;/p&gt;

&lt;p&gt;In the graph, we can see a strong positive correlation of the count of bike rentals.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lognormal Plots&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A normal distribution can convert to a lognormal distribution using logarithmic mathematics. The lognormal distribution plots the log of random variables from a normal distribution curve. It displays the probability density function (pdf) and is of particular interest when the variable must be positive as log values are always positive.&lt;/p&gt;

&lt;p&gt;Many examples follow lognormal distribution like the concentration of elements and their radioactivity in the Earth’s crust, latent periods of infectious diseases, the distribution of particles, chemicals, and organisms in the environment, the length of comments posted on social media website discussion forums, or fluctuations in the stock markets.&lt;br&gt;
A normal distribution can convert to a lognormal distribution using logarithmic mathematics. The lognormal distribution plots the log of random variables from a normal distribution curve. It displays the probability density function (pdf) and is of particular interest when the variable must be positive as log values are always positive.&lt;/p&gt;

&lt;p&gt;Many examples follow lognormal distribution like the concentration of elements and their radioactivity in the Earth’s crust, latent periods of infectious diseases, the distribution of particles, chemicals, and organisms in the environment, the length of comments posted on social media website discussion forums, or fluctuations in the stock market. &lt;/p&gt;

&lt;p&gt;Conclusion&lt;br&gt;
Exploratory Data Analysis is just a key in order to have a better understanding and representing your data, which helps you build a stronger, more generalized model. So, The visualization of the data is easy to achieve, which facilitates the comprehension of our analysis by others.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Complete Guide to becoming a Data Scientist 2023/2024</title>
      <dc:creator>Hews</dc:creator>
      <pubDate>Mon, 02 Oct 2023 19:10:39 +0000</pubDate>
      <link>https://dev.to/kamau001/complete-guide-to-becoming-a-data-scientist-20232024-1fb5</link>
      <guid>https://dev.to/kamau001/complete-guide-to-becoming-a-data-scientist-20232024-1fb5</guid>
      <description>&lt;p&gt;Data science is a powerful and rapidly growing field that uses a variety of techniques and tools to extract knowledge and insights from data. Data scientists are the wizards of this field, using their analytical skills and domain knowledge to transform raw data into actionable insights that can be used to solve complex problems and make better decisions&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Requirements:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Education: A bachelor's degree in a related discipline, such as computer science, mathematics, statistics, engineering, or a pertinent domain (such as economics, biology), can be beneficial. However, there are no strict educational requirements. Additionally, many data scientists hold advanced degrees (master's or doctoral) in these disciplines.&lt;br&gt;
Programming: Learn a programming language commonly used in data science, such as Python or R. Master the basics of data manipulation, control structures, and functions.&lt;br&gt;
Statistics: Learn the basics of statistics, such as probability, hypothesis testing, and regression analysis. This knowledge is essential for understanding and interpreting data.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Foundational Skills:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Data Handling: Familiarize yourself with libraries like NumPy, Pandas (for Python), or data frames (for R) to handle and manipulate data efficiently.&lt;br&gt;
Machine Learning: Learn the theories and techniques of machine learning. Start with decision trees, random forests, and linear regression before moving on to support vector machines and deep learning. Caret (R) and Scikit-learn (Python) are two excellent libraries for practicing machine learning algorithms.&lt;br&gt;
SQL: Learn SQL to manipulate and retrieve data from relational databases. This is an essential skill for any data scientist, as most businesses use relational databases to store their data.&lt;br&gt;
Version Control: Learn how to use version control systems like Git and collaborate on projects using platforms like GitHub or GitLab. Version control is essential for tracking changes to code and data, and for collaborating with other data scientists.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Advanced Skills:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Big Data: Familiarize yourself with big data technologies like Apache Hadoop and Spark for handling large datasets efficiently.&lt;br&gt;
Cloud Computing: Learn about cloud platforms like AWS, Azure, or Google Cloud, which provide services for computation, data storage, and machine learning.&lt;br&gt;
Deep Learning: For complex machine learning tasks, delve deeper into deep learning frameworks like TensorFlow and PyTorch.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Real-World Experience:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Data Science Competitions: Participate in data science competitions on websites like Kaggle to put your skills to the test and learn from real-world datasets and challenges.&lt;br&gt;
Personal Projects: Start your own data science projects to develop your portfolio. This could involve exploring interesting datasets or solving problems relevant to your domain of expertise.&lt;br&gt;
Internships and Freelance Work: Apply for data science internships or freelance opportunities to gain real-world experience.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Ongoing Education:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Online Courses: Continue your education by taking online courses, reading books, and enrolling in bootcamps. There are many excellent data science courses available on websites like Coursera, Simplilearn, edX, and Udacity.&lt;br&gt;
Conferences and Meetups: Attend data science conferences, webinars, and local meetups to stay up-to-date on the latest trends and connect with industry experts.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Specialization:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;As you gain experience, you may decide to specialize in a particular area of data science, such as natural language processing (NLP), computer vision, reinforcement learning, or a particular industry domain like healthcare, finance, or marketing.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Portfolio:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Create and showcase your personal data science projects on platforms like GitHub to demonstrate your skills to potential employers.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Networking and Job Search:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Prepare for data science interviews by practicing coding challenges, case studies, and behavioral questions. LinkedIn is a great platform for networking and applying for data science jobs.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Soft Skills:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In addition to technical skills, data scientists also need strong soft skills, such as communication, collaboration, and problem-solving. Develop your communication skills so that you can effectively present your findings and insights to non-technical stakeholders.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Stay Updated:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Data science is a rapidly evolving field. It is important to stay up-to-date on the latest tools, libraries, and best practices. This can be done by reading articles and blog posts, attending conferences and meetups, and taking online courses.&lt;/p&gt;

&lt;p&gt;Tips for Becoming a Data Scientist Without Experience&lt;br&gt;
Start by building a strong foundation in the fundamentals of data science, such as mathematics, programming, and statistics.&lt;br&gt;
Gain real-world experience by working on personal projects, participating in data science competitions, or completing internships.&lt;br&gt;
Network with other data scientists and professionals in your field. Attend conferences, meetups, and online communities to learn from others and build relationships.&lt;br&gt;
Continue your education by taking online courses, reading books, and attending bootcamps.&lt;br&gt;
Be prepared to demonstrate your skills and knowledge to potential employers. Create a portfolio of your work and practice interviewing skills.&lt;/p&gt;

&lt;p&gt;Conclusion&lt;/p&gt;

&lt;p&gt;Data science is a challenging but rewarding field. With hard work and dedication, you can become a data scientist and use your skills to make a real difference in the world.&lt;/p&gt;

</description>
    </item>
  </channel>
</rss>
