<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Artik Blue</title>
    <description>The latest articles on DEV Community by Artik Blue (@artikblue).</description>
    <link>https://dev.to/artikblue</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F360602%2Ff91ec9af-627a-44c4-9fa6-4b9ad414e465.png</url>
      <title>DEV Community: Artik Blue</title>
      <link>https://dev.to/artikblue</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/artikblue"/>
    <language>en</language>
    <item>
      <title>Intro to normal distributions</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Tue, 28 Apr 2020 15:12:22 +0000</pubDate>
      <link>https://dev.to/artikblue/intro-to-normal-distributions-1c68</link>
      <guid>https://dev.to/artikblue/intro-to-normal-distributions-1c68</guid>
      <description>&lt;p&gt;Today I'm going to drop you a few relevant notes on normal distributions, so grab a coffee and follow me.&lt;/p&gt;

&lt;h4&gt;
  
  
  What is a distribution?
&lt;/h4&gt;

&lt;p&gt;A distribution, or a probability distribution is a mathematical function that provides the probabilities of the occurrence of various possible outcomes in an experiment. Probability distributions are used to define different types of random variables in order to make decisions based on these models.&lt;/p&gt;

&lt;p&gt;In other terms it is a function that shows the possible values for a variable and how often they occur.&lt;/p&gt;

&lt;p&gt;Distributions can be both &lt;/p&gt;

&lt;p&gt;If we are in a university campus and study the height of the people, we will probably see that most of the students are in the same height range (mean) we'll also see few students who are below that mean and few others above it. We can plot the histogram on that and what we'll see will be the distribution of the students! &lt;/p&gt;

&lt;p&gt;We'll see something like this (height measured in cm)&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--rAnvLutd--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/21db31amv2739vpysvnq.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--rAnvLutd--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/21db31amv2739vpysvnq.png" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That's the distribution and what it tells us is what we see, that most of the students are located in the same height range, other heights sure exist but on lower frequency so it is less probable to see students on those other ranges.&lt;/p&gt;

&lt;p&gt;...and by the way that is also a normal distribution!&lt;/p&gt;

&lt;h4&gt;
  
  
  Normal distributions
&lt;/h4&gt;

&lt;p&gt;In common words, normal distributions or bell-form distributions are those that look similar to the one that just presented. On those distributions most of the values fall within the mean&lt;/p&gt;

&lt;p&gt;aproximately 68% of the values  fall within one standard deviation (we already presented this one on our first posts) and 95% fall within 2 stdevs, like you can see here:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--K6IPuJDp--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/rq3eq6g63ai07bah08iy.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--K6IPuJDp--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/rq3eq6g63ai07bah08iy.png" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The first key point on this one is that, if your data follows a normal distribution, we only need the mean and the stdev to compute the whole graph or even better to get the actual probability/frequency on a particular value.&lt;/p&gt;

&lt;p&gt;We can use the following formula&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--yIJfVSvZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/ozn1ii1ni7y5ke1dfox6.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--yIJfVSvZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/ozn1ii1ni7y5ke1dfox6.jpg" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h4&gt;
  
  
  Other distributions
&lt;/h4&gt;

&lt;p&gt;Other distributions exist as well such as the poisson distribution that looks like a normal distribution somehow moved to the left, the skellam distribution that looks like a normal one but with a peak on the centre and of course uniform distributions that present the same value all the way up.&lt;/p&gt;

&lt;p&gt;Most of the time the type of data distribution won't matter that much in our experiments because of the central limit theorem&lt;/p&gt;

&lt;h4&gt;
  
  
  Central limit theorem and normal distributions
&lt;/h4&gt;

&lt;p&gt;In the study of probability theory, the central limit theorem (CLT) states that the distribution of sample means approximates a normal distribution (also known as a “bell curve”), as the sample size becomes larger, assuming that all samples are identical in size, and regardless of the population distribution shape.&lt;/p&gt;

&lt;p&gt;Said another way, CLT is a statistical theory stating that given a sufficiently large sample size from a population with a finite level of variance, the mean of all samples from the same population will be approximately equal to the mean of the population. Furthermore, all the samples will follow an approximate normal distribution pattern, with all variances being approximately equal to the variance of the population, divided by each sample's size.&lt;/p&gt;

&lt;p&gt;That comes in very handy when it comes to statistican analysis on very large populations, as we can grab smaller grups of random samples and infere knowledge about the whole population!&lt;/p&gt;

&lt;h4&gt;
  
  
  Some resources and references:
&lt;/h4&gt;

&lt;p&gt;&lt;a href="https://towardsdatascience.com/understanding-the-normal-distribution-with-python-e70bb855b027"&gt;https://towardsdatascience.com/understanding-the-normal-distribution-with-python-e70bb855b027&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="http://sphweb.bumc.bu.edu/otlt/MPH-Modules/PH717-QuantCore/PH717-Module6-RandomError/PH717-Module6-RandomError5.html"&gt;http://sphweb.bumc.bu.edu/otlt/MPH-Modules/PH717-QuantCore/PH717-Module6-RandomError/PH717-Module6-RandomError5.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.investopedia.com/terms/c/central_limit_theorem.asp"&gt;https://www.investopedia.com/terms/c/central_limit_theorem.asp&lt;/a&gt;&lt;/p&gt;

</description>
      <category>beginners</category>
      <category>datascience</category>
    </item>
    <item>
      <title>Covid19 tests and Thomas Bayes</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Wed, 22 Apr 2020 16:53:41 +0000</pubDate>
      <link>https://dev.to/artikblue/covid19-tests-and-thomas-bayes-3i95</link>
      <guid>https://dev.to/artikblue/covid19-tests-and-thomas-bayes-3i95</guid>
      <description>&lt;h3&gt;
  
  
  Bayes theorem
&lt;/h3&gt;

&lt;p&gt;Created by reverent Thomas Bayes, who created this famous rule to infere the existence of God... needless to say that the existence of God is still in the realm of faith but his famous rule has became the very fundamentals of inference probability&lt;/p&gt;

&lt;p&gt;The bayes theorem can be written as the following:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--06e9rdV4--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/2kpvps09llukmmjhf86b.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--06e9rdV4--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/2kpvps09llukmmjhf86b.png" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Do you remember the covid detection example we had in the last tutorial? &lt;/p&gt;

&lt;p&gt;We want to know the probabilitty for one person of having covid if he or she had tested positive&lt;/p&gt;

&lt;p&gt;We have the on average probability for a random individual of having covid or not&lt;/p&gt;

&lt;p&gt;p(covid) = 0.01&lt;br&gt;
p(¬covid) = 0.99&lt;/p&gt;

&lt;p&gt;Then we have this test we just bought that goes:&lt;/p&gt;

&lt;p&gt;90% of the time outputs POSITIVE if you have covid&lt;/p&gt;

&lt;p&gt;But also&lt;/p&gt;

&lt;p&gt;90% of the time marks NEGATIVE ig you don't have it&lt;/p&gt;

&lt;p&gt;So we can mind our problem the following way:&lt;/p&gt;

&lt;p&gt;P(C | POS) = P(C) * P(POS|C)&lt;br&gt;
P(¬C | POS) = P(¬C) * P(POS|¬C)&lt;/p&gt;

&lt;p&gt;In this case P(C | POS) will be the probability of having covid given a positive test result, and P(POS|C) the probability of having a positive test result giving that the patient actually has covid, it is important to mark those concepts as sometimes we may get confused on what do they represent.&lt;/p&gt;

&lt;p&gt;So let's compute them on our case.&lt;/p&gt;

&lt;p&gt;P(C | POS) = P(C) * P(POS|C) =  0.01 * 0.9 = 0.009&lt;/p&gt;

&lt;p&gt;And&lt;/p&gt;

&lt;p&gt;P(¬C | POS) = P(¬C) * P(POS|¬C) = 0.099&lt;/p&gt;

&lt;p&gt;But as you might have seen, those probabilities don't add up to one, they should be somehow complementary right? Given a positive test if we don't have covid, then we have it right? No other options around, so they should add to one, thats not exactly like that because we are selecting random individuals and they may or may not have the virus, we need to "focus" on our target.&lt;/p&gt;

&lt;p&gt;If we want to relativize our context to those two options, we need to normalize them, so first we add those values together:&lt;/p&gt;

&lt;p&gt;0.009 + 0.099 = 0.108 that is the total probability of having a positive test (wether the person has the virus or not) after selecting a random person on town.&lt;/p&gt;

&lt;p&gt;So finally we can obtain whats called the posterior probability by normalizing our probabilities, that means, dividing them by their sum.&lt;/p&gt;

&lt;p&gt;P(C|POS) = 0.009 / 0.108 = 0.0833&lt;/p&gt;

&lt;p&gt;P(¬C|POS) =0.099  / 0.108 = ¨0.9166&lt;/p&gt;

&lt;p&gt;So those are the probabilities, pretty impressive right? That is because we are assuming an initial positive result on the test. We can see it better if we go in the other direction, let's evaluate for negative results. &lt;/p&gt;

&lt;p&gt;P(C | NEG) = P(C) * P(NEG|C) = 0.001&lt;br&gt;
P(¬C | NEG) = P(¬C) * P(NEG|¬C) = 0.891&lt;/p&gt;

&lt;p&gt;And the normalizer is = 0.001 + 0.891 = 0.892&lt;/p&gt;

&lt;p&gt;Finally:&lt;/p&gt;

&lt;p&gt;P(C | NEG) = 0.0011&lt;br&gt;
P(¬C | NEG) = 0.9989&lt;/p&gt;

&lt;p&gt;So if we get a negative on our test, there we can be 99% sure that we are covid free, said that you can easily detect that those tests are great for "discarting", for doing the initial triage and detecting negatives, but when it comes to positives, aditional checks should be made to get a veredict.&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>beginners</category>
    </item>
    <item>
      <title>A small lesson on conditional probability</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Fri, 17 Apr 2020 19:51:10 +0000</pubDate>
      <link>https://dev.to/artikblue/a-small-lesson-on-conditional-probability-40e7</link>
      <guid>https://dev.to/artikblue/a-small-lesson-on-conditional-probability-40e7</guid>
      <description>&lt;p&gt;By continuing to study the many aspects of probability we have to look at the conditional probability, that answers the fundamental question: What is the probability of having X phenomenon if Y phenomenon already happened?&lt;/p&gt;

&lt;p&gt;When we are just flipping coins the results of the second throw won't depend on the results given at the first one, but for example if we pick a random person there is some chance that he or she will be a night person or day person depending whether the person prefeers the night or the day it will be more or less probable that he or she likes to go out running early in the morning. That is one of the fundamentals of conditional probability, events that are correlated.&lt;/p&gt;

&lt;h4&gt;
  
  
  The covid19 test
&lt;/h4&gt;

&lt;p&gt;Imagine that we are in charge of a hospital and we have this novel test that we have to use and can detect covid19 in a matter of minutes, the test works as follow:&lt;/p&gt;

&lt;p&gt;If the person has covid it will detect it 90% of the time&lt;br&gt;
If the person does not have it it will also detect that 80% of the time&lt;/p&gt;

&lt;p&gt;And with those you can easily calculate the complementaries. We can also write it in other terms:&lt;/p&gt;

&lt;p&gt;P(POSITIVE | COVID) = 0.9&lt;br&gt;
P(NEGATIVE | COVID) = 0.1&lt;/p&gt;

&lt;p&gt;P(POSITIVE | ¬COVID) = 0.2&lt;br&gt;
P(NEGATIVE | ¬COVID) = 0.8&lt;/p&gt;

&lt;p&gt;And from our statistical data we assume that if we pick a random person on our town only 1 out of 10 have covid, SO&lt;/p&gt;

&lt;p&gt;P(COVID) = 0.1&lt;br&gt;
P(¬COVID) = 0.9&lt;/p&gt;

&lt;p&gt;So using that we can generate a table and calculate some probabilities:&lt;/p&gt;

&lt;p&gt;If we want to know the probability we have that if we pick a random person on the street, this person will have covid and will be detected as positive we can just multiply the general probability of having covid by the probability of having a positive test given that he or she has covid.&lt;/p&gt;

&lt;p&gt;We can even build a table.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;COVID&lt;/th&gt;
&lt;th&gt;TEST&lt;/th&gt;
&lt;th&gt;P()&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Y&lt;/td&gt;
&lt;td&gt;POS&lt;/td&gt;
&lt;td&gt;0.09&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Y&lt;/td&gt;
&lt;td&gt;NEG&lt;/td&gt;
&lt;td&gt;0.02&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;N&lt;/td&gt;
&lt;td&gt;POS&lt;/td&gt;
&lt;td&gt;0.18&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;N&lt;/td&gt;
&lt;td&gt;NEG&lt;/td&gt;
&lt;td&gt;0.72&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Note that all of those add to one and we can even calculate probabilities such as: what is the probability that if we pick a random person on the street he or she will do positive on a test, no matter if he or she has the virus o not?&lt;/p&gt;

&lt;p&gt;We just need to add the probability of doing POS having the virus and doing POS not having it, that is 0.009 + 0.18 = 0.27 very intuitive!&lt;/p&gt;

&lt;h3&gt;
  
  
  Moar coins
&lt;/h3&gt;

&lt;p&gt;Yes... more of them. So if we have two identical coins and we flip one, and then the other the probabilities for the second throw to be a tail won't depend on the first throw at all, as all of them will have the same probability and we will pick one or the other. But let's say one of the coins is loaded so the probability of getting a head after throwing it is 0.9, the other is just a regular coin. We put them both on a box and we ask someone to pick one of them and throw it we assume that 50% of the time the person will pick one 50% will pick the other, the probabilities of getting a tail will change here as firstly the person will have to pick a coin and DEPENDING on the coin he or she picks the probabilities of getting head will be diferent!&lt;/p&gt;

&lt;p&gt;Again we can get the probability of getting a tail after picking the non loaded coin by doing 0.5 (picking the coin) * 0.5 (and getting tail) and after that we can build the table like this:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;PICK COIN&lt;/th&gt;
&lt;th&gt;FLIP COIN&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.05&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The methodology is the same. You can think about more complex things such as, starting by the coin picking game and then doing multiple throws or using fake coins that have a head on both sides, the logic is the same.&lt;/p&gt;

&lt;p&gt;See you next with some more probabiliy with bayes theorem :)&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>beginners</category>
    </item>
    <item>
      <title>More about probability: Binomial distributions</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Tue, 14 Apr 2020 12:42:18 +0000</pubDate>
      <link>https://dev.to/artikblue/more-about-probability-binomial-distributions-1a0c</link>
      <guid>https://dev.to/artikblue/more-about-probability-binomial-distributions-1a0c</guid>
      <description>&lt;p&gt;Let's go back to our previous coin throwing example! &lt;/p&gt;

&lt;p&gt;Let's suppose that we flip a regular coin two times. How many times will we get two heads or two tails?&lt;/p&gt;

&lt;p&gt;Exactly two! Good easy&lt;/p&gt;

&lt;p&gt;Now let's flip it five times. How many outcomes do have the same number of heads than tails? Weeell... it will be zero, as five is an even number, you see sometimes we can know the result easily without having to do a lot of math behind.&lt;/p&gt;

&lt;p&gt;Going back to the same example, let's now say 1 heads and then 4 tails, in how many ways can we achieve that? There are five scenarios when this is posible. We can image five slots each slot being one throw and as we are looking for 1 head only, the head can be in only one position at a time, so it can move through every slot but be in one slot at a time. But if we now want to know about scenarios with 2 heads, this gets complicated.&lt;/p&gt;

&lt;p&gt;You can think about that this way: We have 5 slots and heads can be in 2 slots at a time, so if the first head is on one spot, then the other one has 4 slots to move on so it would be 4 * 5.&lt;/p&gt;

&lt;p&gt;But... those hads can move to diferent positions but those are not independent entities, they are events so we cannot understand them as head1 and head2 being their permutation a diferent scenario. We'll have to divide by two to get rid of those permutations.&lt;/p&gt;

&lt;p&gt;The calculation will be&lt;/p&gt;

&lt;p&gt;5*4/2 = 10&lt;/p&gt;

&lt;p&gt;If instead of two heads we want three of them, the calculation is similar &lt;/p&gt;

&lt;p&gt;5*4*3 / 3*2*1&lt;/p&gt;

&lt;p&gt;Again we have five slots for the first head, four for the second and three for the third, but we need to avoid those permutation scenarios (for the first throw the coin can be in one out of three positions, for the second throw in two, one for the last throw).&lt;/p&gt;

&lt;p&gt;As you already saw, there can be a generalization related to this process. We see that we are dividing by numbers that are a result of a secuence of multiplication from the number itself all the way down to one, that is called the factorial and it looks like this:&lt;/p&gt;

&lt;p&gt;n! = n*(n-1)*(n-2)...*1&lt;/p&gt;

&lt;p&gt;So something like&lt;/p&gt;

&lt;p&gt;5*4*3 / 3*2*1&lt;/p&gt;

&lt;p&gt;Can also be represented as:&lt;/p&gt;

&lt;p&gt;5*4*3 / 3!&lt;/p&gt;

&lt;p&gt;Understanding this we can define the formula for calculating the posible situations on a particular event can happen, the binomial distribution:&lt;/p&gt;

&lt;p&gt;n! / k! * (n-k)!&lt;/p&gt;

&lt;p&gt;Where n is the number of times a phenomenon repeats (ex: 10 coin flips) and k is the times a result has to repeat (ex: 5 heads) assuming that we only have two options (heads or tails) for the phenomenon.&lt;/p&gt;

&lt;p&gt;Let's apply the rule to calculate real probabilities withouth having to draw&lt;br&gt;
 those tedious truth tables:&lt;/p&gt;

&lt;p&gt;We flip a coin 5 times and we want to get EXACTLY one heads, whats the probability?&lt;/p&gt;

&lt;p&gt;The total amount of situations we can expect to have on that phenomenon is 2^5 = 32 (32 combinations of diferent results)&lt;/p&gt;

&lt;p&gt;The about the specific situations we are interested in:&lt;/p&gt;

&lt;p&gt;n = 5&lt;br&gt;
k = 1&lt;/p&gt;

&lt;p&gt;5! / (5-1) * 1 = 5&lt;/p&gt;

&lt;p&gt;So 5/32 = 0.15625&lt;/p&gt;

&lt;p&gt;That is our probability&lt;/p&gt;

&lt;p&gt;But what if the coin is loaded and P(heads) is 0.8?&lt;/p&gt;

&lt;p&gt;Again, we can start by calculating the number of scenarios when this will happen:&lt;/p&gt;

&lt;p&gt;5! / 4! * 1! = 5; &lt;/p&gt;

&lt;p&gt;And on this situation(s) we will get heads 5 times and tails one so it will be something like&lt;/p&gt;

&lt;p&gt;1*0.8 * 1*0.8..... *1*0.2 (probability of tails * 1 tails) In other terms:&lt;/p&gt;

&lt;p&gt;0.8^4 * 0.2^1&lt;/p&gt;

&lt;p&gt;So 0.4&lt;/p&gt;

&lt;p&gt;Now you know about the binomial distribution... use this knowledge with responsability...&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Introduction to probability</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Tue, 14 Apr 2020 11:18:30 +0000</pubDate>
      <link>https://dev.to/artikblue/introduction-to-probability-2n8m</link>
      <guid>https://dev.to/artikblue/introduction-to-probability-2n8m</guid>
      <description>&lt;p&gt;Statistics and probability are very related! Probability makes predictions about future events based on models and statistics studies data (imagine, events that already happened and produced results) to gather information about them and perhaps maybe build models.&lt;/p&gt;

&lt;p&gt;Imagine we have a coin and we flip it to see if we get heads or tails. By fliping the coin we are making data, data that can be studied later on to for gaining more knowledge about the phenomenon.&lt;/p&gt;

&lt;p&gt;So we flip it once and we get heads. At that point our data will be DATA={Heads}, we flip it again and... heads another time, so then DATA={Heads,Heads} and then we go for a third time and we get tails so finally DATA={Heads, Heads, Tails}. At that point, if we stop the experiment right there we might think that one time in every three times the coin returns Tails, the rest of the times Heads or in other terms: Probability(Tails) = 1/3. And what about the probability of Heads, then as we get Heads 2 times every 3 throws it will be 2/3. If we add Probability(Tails) + Proability(Heads) we get 1, so another way of getting the probability of let's say Heads is to do 1-Probability(Tails), thats called the complementary probability!&lt;/p&gt;

&lt;p&gt;And that may be true but, what if we repeat this experiment milions of times and we keep track of every result?&lt;/p&gt;

&lt;p&gt;Well in that case we will probably get something as: P(tails) = 1/2. Think about it that way: If a coin has two sides, we can only get heads or tails one result per throw, and if the coin is perfectly balanced, it is common sense that we can either get one or another right? So our experimental data should demonstrate that.&lt;/p&gt;

&lt;p&gt;Now we will assume that our coin is perfectly balanced, we have a probability of 1/2 for every result.&lt;/p&gt;

&lt;p&gt;What if we want to know the probability of getting Heads two times, one after another by throwing the coin two consecutive times. &lt;/p&gt;

&lt;p&gt;Now that we know a little bit about probability, let's image all posible situations and try to use the rule we just presented:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;FLIP 1&lt;/th&gt;
&lt;th&gt;FLIP 2&lt;/th&gt;
&lt;th&gt;PROBABILITY&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.25&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;We can derive that the probability of getting two heads will be 0.25 knowing that the probability of each of the posible outcomes or results of this phenomenon (throwing a coin 2 consecutive times) needs to add to 1. So we have four posible situations each one of them equally probable thus 0.25 is the probability of each one to happen. Another way of thinking about that is just to multiply 0.5 (the probability of getting a head) by the probability of getting heads (0.5).&lt;/p&gt;

&lt;p&gt;What about the probability of getting two heads when the coin is loaded and we know that we can get heads with a probability of 0.6 (and thus probability of tails will be... 0.4)? If we follow that rule we just learnt, that is easy, we can multiply 0.6 by itself and we get it!.&lt;/p&gt;

&lt;p&gt;The table will look like that this time:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;FLIP 1&lt;/th&gt;
&lt;th&gt;FLIP 2&lt;/th&gt;
&lt;th&gt;PROBABILITY&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.36&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.16&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;We may think that we know all about coin throws and probabilitty right now but there are several other questions to be asked, for example...what if we want to know the probability of getting exactly one head? &lt;/p&gt;

&lt;p&gt;Regarding to our loaded coin, let's look at the table one more time&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;FLIP 1&lt;/th&gt;
&lt;th&gt;FLIP 2&lt;/th&gt;
&lt;th&gt;PROBABILITY&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.36&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.16&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;There are two posible situations when we are getting exactly one head. (H,T) and (T,H) on those others we get more than one head or no heads at all.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;FLIP 1&lt;/th&gt;
&lt;th&gt;FLIP 2&lt;/th&gt;
&lt;th&gt;PROBABILITY&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;T&lt;/td&gt;
&lt;td&gt;H&lt;/td&gt;
&lt;td&gt;0.24&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;We can calculate the probability of getting exactly one head by adding those two probabilities together. So the total probability will be 0.48.&lt;/p&gt;

&lt;p&gt;In situations such as the one we just reviewed the product is asociated with AND and the addition with OR. Our last example could be written like:&lt;/p&gt;

&lt;p&gt;The probability of getting exactly one Head is:&lt;/p&gt;

&lt;p&gt;The probability of getting Heads AND Tails OR The probability of getting Tails AND Heads.&lt;/p&gt;

&lt;p&gt;Voilà!&lt;/p&gt;

&lt;p&gt;For the extra mille, let's now imagine a standard dice. The probability of getting one of the possible results is 1/6. What's the probability of getting an even number? As ech result has a probabilitty of 1/6, the probability of an even number will be 3/6 or 1/2 as it can be represented as:&lt;/p&gt;

&lt;p&gt;The probability of getting a one OR getting a three OR getting a five.&lt;/p&gt;

&lt;p&gt;And about getting the same number twice on a two times dice throw??&lt;/p&gt;

&lt;p&gt;Well, we just need to think about the situations when we will have the same number twice, calculate their individual probability and add it all together :)&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;FLIP 1&lt;/th&gt;
&lt;th&gt;FLIP 2&lt;/th&gt;
&lt;th&gt;PROBABILITY&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;1/6 * 1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;And... that add ups to 1/6 again!!&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>beg</category>
    </item>
    <item>
      <title>Measures of spread in data</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Wed, 08 Apr 2020 12:03:55 +0000</pubDate>
      <link>https://dev.to/artikblue/measures-of-spread-in-data-1ab5</link>
      <guid>https://dev.to/artikblue/measures-of-spread-in-data-1ab5</guid>
      <description>&lt;p&gt;If we have some data and we want to give a first sight on it, we can initially think about the properties we saw on the previous post, such as the mean, the median of the mode. But sometimes those are not enough and we want to look at the bigger picture of the data, we may want to see how that data is distributed.&lt;/p&gt;

&lt;p&gt;For example, let us think about a university class. We are the professor and we have a bunch of students, we want to evaluate how the whole class is doing so we start by extracting the measures of center such as the mean. We discover that the mean of our class is something like 5.8 (we are using a 0-10 scale), but our class may be very large and with that, we don't know if we have a bunch of students that are not even passing and some very smart ones who are getting straight 10s or maybe we have a whole lot of average students who get marks close to 6 and 7s. Of course we can use other systems measures here to get more information such as the median or the mode, but if you think it well, as we have decimal results the mode will return something weird (or perhaps we can group the marks into categories like (zero, fail, barely pass, good...) but that might be considered cheating if you think it well.  &lt;/p&gt;

&lt;p&gt;One thing we could do in this case is to start by plotting an histogram of the data of our students (to make it clearer I adjusted all of the marks to integers instead of decimals):&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--hcEmwyIg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/41p78p8z9s8zgmvaeg1n.PNG" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--hcEmwyIg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/41p78p8z9s8zgmvaeg1n.PNG" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That histogram can be generated with python using pyplot in the following way:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;plt&lt;/span&gt;

&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;
&lt;span class="c1"&gt;# take your tame and change the bins param to see what it does!
&lt;/span&gt;&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;hist&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bins&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;show&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;What we see in the graph is that most of the students get marks closer to 6 and few of them perform very good or very bad eventhough there is some tendency in the class that says that students tend to perform more good than bad. &lt;/p&gt;

&lt;p&gt;In fact we can contrast that with our already known measures of central tendency such as the median or the mode!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mean&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;5.8&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"median:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;median&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;If we recall the histogram we just ploted, we now see that the median here is coincides with its center and its also easy to understand that the point associated with the highest bar is the mode and due to the fact that we have some more points closer to ten rather than points closer to zero the mean is closer to six instead of for example closer to four.&lt;/p&gt;

&lt;p&gt;Another way to analyze the spread of our data is by the boxplot.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--PzVAkJrc--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/vj6bu6t8rpsyp18tabij.PNG" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--PzVAkJrc--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/vj6bu6t8rpsyp18tabij.PNG" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That can be generated with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;plt&lt;/span&gt;

&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;

&lt;span class="c1"&gt;#plt.hist(x, bins = len(x))
&lt;/span&gt;&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;boxplot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;plt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;show&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;In this case, what the boxplot shows is that most of the data is located within the box, to be more specific the data that is inside the box is the data that falls within the interquartile range, the line marks the median.&lt;/p&gt;

&lt;p&gt;But wait wait, interquartile range? what is a quartile?&lt;/p&gt;

&lt;p&gt;For a better explaination, let's calculate them with the scipy stats package!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;plt&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;scipy.stats.mstats&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;mquantiles&lt;/span&gt;
&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mquantiles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mf"&gt;4.&lt;/span&gt;  &lt;span class="mf"&gt;6.&lt;/span&gt;  &lt;span class="mf"&gt;7.8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;So 4 is the quartile 1 (Q1), 6 is the Q2 and 7.8 the Q3&lt;/p&gt;

&lt;p&gt;That means that 25% of the students scored 4 or less, 50% of them did 6 or less and 75% did 7.8 or less (of course 100% of them scored 10 or less). &lt;/p&gt;

&lt;p&gt;The inter-quartile range or IQR can be calculated by substracting the first quartile from the third, in this case is 3.8 and tells us how far apart the first and third quartile are, so it indicates how spread out the middle 50% of our set of data is.&lt;/p&gt;

&lt;p&gt;Another interesting feature, but a bit less relevant in this particular case we can see here is the range, which is 10 herre as our data goes from 0 to 10.&lt;/p&gt;

&lt;p&gt;Sometimes we may work with multiple datasets and want to perform multiple automatic decisions, so plotting a histogram or a boxplot may be ver tedious. On that cause three numbers such as the quartiles may be useful but tedious to deal with as well as they won't be a single number! &lt;/p&gt;

&lt;p&gt;To deal with that situation we have a couple of measures that come very handy, those are the variance and the standard deviation and they well how far is the are features from the mean.&lt;/p&gt;

&lt;p&gt;To better understand it, let's do it with python!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Variance:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;var&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;5.26530612244898&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Standard deviation:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;std&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;2.294625486315573&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;So those measures tell us how far the average student is from the mean when it comes to his or her marks. The standard deviation is just the square root of the variance, thats because the variance is measured in squared units (we have it squared to avoid negative results).&lt;/p&gt;

&lt;p&gt;As I want to get practical on this I encourage you to read more about that on your own here: &lt;a href="https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch12/5214891-eng.htm"&gt;https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch12/5214891-eng.htm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So you can learn the theory.&lt;/p&gt;

&lt;p&gt;If your std is low that means that most of the data is grouped together near the mean and so the mean will be a strong statistic to understand the data, if otherwise the std is very high that would mean that our data is wide spread across axis. &lt;/p&gt;

&lt;p&gt;Let us now look at the histogram one more time:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--hcEmwyIg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/41p78p8z9s8zgmvaeg1n.PNG" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--hcEmwyIg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/41p78p8z9s8zgmvaeg1n.PNG" alt="Alt Text"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If we look closer to it... dont we see a pattern? it is like wave! Well yes, on that case it is clear that our data follows a pattern, one of the most important patterns on this case, the normal distribution (well, quasi normal distribution on this case, as it is not 100% symetric). Read more about it here! &lt;a href="https://statisticsbyjim.com/basics/normal-distribution/"&gt;https://statisticsbyjim.com/basics/normal-distribution/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The normal distribution has many interesting properies, it is symetric, the mean the median and the mode are equal (we have a quasi normal distribution here), and one of my favorite properties is that in a normal distribution, 68% of the data fall within +/- 1 standard deviation from the mean.&lt;/p&gt;

&lt;p&gt;So if we know that we are dealing with data that is following this distribution and we know both the mean and the std we can assume a lot of things very easy. &lt;/p&gt;

&lt;p&gt;Other kinds of shapes exist as well out there such as left and right skewed distributions. For example the athletic shape of an individual during life it starts relatively well and decreases rapidly as the person ages. Another example of a skewed distribution could be the amount of income you have, when you are young it is really low but it may increase year after year as you grow older!&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>beginners</category>
    </item>
    <item>
      <title>What is data?</title>
      <dc:creator>Artik Blue</dc:creator>
      <pubDate>Sun, 05 Apr 2020 18:52:35 +0000</pubDate>
      <link>https://dev.to/artikblue/a-1mnn</link>
      <guid>https://dev.to/artikblue/a-1mnn</guid>
      <description>&lt;p&gt;I would like to start this site as a small on board journey where I'll be keeping my notes in a format that can make them useful for others.&lt;/p&gt;

&lt;p&gt;So this first post series will be all about data analysis we will start by the very fundamentals of statistics and move on to more advanced topics such as supervised and unsupervised learning as well as deep learning.&lt;/p&gt;

&lt;h3&gt;
  
  
  Intro to descriptive statistics
&lt;/h3&gt;

&lt;p&gt;Data is literally everything we can keep track off and measure, it could be a list of the cars you are seeing on the street, your school records, the cost you have to pay for your favorite coffee recorded for every day of the year or all the usernames and emails of the whole netflix database among others. So everything we can measure, that is data.&lt;/p&gt;

&lt;p&gt;And data itself is useless, if we record data it is because we want to do something with it. A complex example may be a hedge fund investor, that always has an eye on the stock market, accessing huge amounts of data including: names and stock prices of many companies, currency change rates as well as a lot of previous buy/sell operations made by other investors. By using that data a hedge fund analyst may evaluate the state of the market related to its interests and perhaps try to perform some preditions. That last example is probably what comes to our minds when we try to think about a data analyst or someone who works with large amounts of data, but the fact is that we use data analysis on a daily basis, for example when we go out for buying something, we may quick scan through all of the items in the store that have the conditions we need and then we select the one with the lower price, we may do that without htinking but deep down inside, we are applying data analysis techniques!&lt;/p&gt;

&lt;p&gt;In general when we talk about data we talk about qualitative and quantitative data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Qualitative data?&lt;/strong&gt; We won't do any numerical operation with this kind of data. Think about data such as a car model, a coffee brand or a company name. It represents a quality.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quantitative data?&lt;/strong&gt; For example, number of cars. And if we look at this kind of data, we can see that it can be continuous or discrete. Continuous data may take negative, positive or even decimal values, the key point here is that it represents information that has an order and so it is continuous in some way. Discrete data is just quantitative data that does not follow any order. Again, there can be a bit of confusion regarding to this, as any set of quantitative data can be defined as discrete.&lt;/p&gt;

&lt;h3&gt;
  
  
  Measures of central tendency
&lt;/h3&gt;

&lt;p&gt;And we can perform all kinds of operations on data, some of them may make zero sense but, some others are pretty well kown and widely used! Everyone starts with the measures of central tendency, by using them we can get a pretty general overview about what kind of data we have.&lt;/p&gt;

&lt;p&gt;As we are willing to do some data science at the end, the best we can do is to start getting used to the tools, so we'll offer examples of our calculations by using python and some popular data science/math libraries such as numpy and scypy.&lt;/p&gt;

&lt;h4&gt;
  
  
  MEAN
&lt;/h4&gt;

&lt;p&gt;The mean is useful for obtaining a general overview of the data, it is obtained by adding all the values of the list (we can call that list a dataset for now) and then dividing by the total lenght of the list. It is useful when we need to obtain a result that includes the whole set, for example the mean is used when your teachers calculate your final marks, as every exam is important, perhaps some exams may be more important than others but we'll dig deeper into that further in these series.&lt;/p&gt;

&lt;p&gt;The main problem of using the mean though is that if we have a list of numbers for example such as: 1,2,3,2,3,2,99999 we can see that the mean will look close to   14287 and that number does not represent our list, that is mostly comprised of numbers between 0 and 5. We use other measures such as the mode when we have problems such as this one. By the way, that large number we just saw is called an outiler! And we must be aware of them.&lt;/p&gt;

&lt;p&gt;The following example will calculate the mean of a python array by using numpy.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;34&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; 
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean of arr : "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mean&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; 

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;12.8&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;h4&gt;
  
  
  MEDIAN
&lt;/h4&gt;

&lt;p&gt;We can obtain it by ordering the values from smallest to the greatest and then the mean will be the value that is right in the centre. If we can extract a centre (if we have an even number of items) the median will be calculated as the mean of the two central values.&lt;/p&gt;

&lt;p&gt;The median works particularly well if we have a dataset that has values that are way different than the others!&lt;/p&gt;

&lt;p&gt;And the median can be calculated as follows using numpy&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;999&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; 
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean of arr : "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;median&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; 

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;7.0&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;h4&gt;
  
  
  MODE
&lt;/h4&gt;

&lt;p&gt;The mode is the most common value of the dataset, so it is the value that occurs most often. For example, we may be searching for a new appartment to rent, the mean appartment price of a zone may be a good idea but, we can also be interested in knowing the mode as a lot of the appartments have prices in a range (ex: its easy to have 1500USD, 1800USD, 5000USD... but more rare to have 1443USD, 1221 USD and such) we can also use the mode so.&lt;/p&gt;

&lt;p&gt;On this case we see that numpy is not able to calculate the mode as easily, but scipy does so we'll use that.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;scipy&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;stats&lt;/span&gt;

&lt;span class="n"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;34&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; 
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean of arr : "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stats&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; 

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;h4&gt;
  
  
  MIN
&lt;/h4&gt;

&lt;p&gt;It is simply the item that has the minimum value on the selected dimension, ex: the item with the lowest price.&lt;/p&gt;

&lt;p&gt;The MIN can be calculated using numpy as follows:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;34&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; 
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean of arr : "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; 

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;The MAX can be calculated using numpy as follows:&lt;/p&gt;

&lt;h4&gt;
  
  
  MAX
&lt;/h4&gt;

&lt;p&gt;It is the element that has the maximum value ex: the biggest house.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="n"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;34&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; 
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"mean of arr : "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; 

&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;On the next parts we'll look at the measures of spread, and we'll start to see some charts!&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>beginners</category>
    </item>
  </channel>
</rss>
