DEV Community: Mariana Medeiros

Random Variables

Mariana Medeiros — Sat, 16 May 2020 12:04:45 +0000

Random what?

For me, randomness was about something that we could not predict that it was going to happen at all. And when I put it together with the concept of variable that I consider to be an element that can take any possible value under minimal circumstances ... I just could not understand at first what those statistics texts were referring to when mentioning random variable. Is it like any possible value to any random situation? Wait... what?

But thankfully now I do know what they mean. HOORAY (if you have ever watched StatQuest you will get this expression. This channel is like a statistical bible in the form of videos. And I am totally taking this expression throughout the rest of my life (and posts). ☺️

Random Variables

A random variable is just a variable whose value we don't know. It does not mean that we don't know the values (or the range of values) it can take (in fact we do!) And for each value (or range) we can associate a probability for it to happen.

The random variable is actually the set of outcomes of the event(situation/experiment) we are interested in. Hence, knowing the event, we can choose what outcomes we want to analyze. Here we go with some examples:

Event	Outcome we are interested in (Random Variable)
Flip a coin 5 times	number of heads
Powerlifting Championship	contestants scores
Tossing two dices	sum of the dices scores
Exercising weekly	how many days per week I exercise

A random variable is represented with a capital letter such as X and it has the following possible values: x₁, x₂, x₃, ..., xₙ. Notice that I am enumerating (counting) these possibilities but this is not always the case as we will see now. It is just for you to understand that

Under a specified random variable I have n possible values that it can randomly take

Types of Random Variables

There are two types of random variables: discrete and continuous.

Types of random variables

Be aware that the event by itself does not determine which type of variable we are dealing with, but our main interest in this event. As an example, let's say we want to know how much time it takes to get to university. If we want to know the EXACT time, we are dealing with a continuous variable as we can not have sure about it. I can take 12.31018793220 minutes or as many decimals you want to add here. It is not countable.

On the other hand, if I want to know the amount of time rounded so I can have just the minutes, it is possible for me to count the values: 12, 13, 14 minutes ... and so on (I can count !). In this case, I am dealing with a discrete variable.

Probability

When talking about random variables we are also talking about probabilities as they are directly correlated. I mean, all we want to do with random variables is finding the probabilities for certain values to be taken. Or rather, when analyzing an event, all we want to know is how likely are the outcomes to happen. The syntax for probabilities of a random variable X is P(X).

For discrete variables, we can find the probability of X taking a value x₁. So, *P(X = x₁) * can be defined! The probabilities can be taken from a frequency chart (histogram).

Histogram for discrete variable X

Conversely, we can not find the probability of a continuous variable taking a value as there is no such thing as an exact value for this type of variable. As by its definition, we can analyze instead a range or interval of values. So, if we have a continuous variable Y, knowing P(Y = y₁) is not possible.How can we know the probability of raining exactly 1.00001 mm today? Or 1.00000000000001589 mm? Or 1.06554979899984098090 mm? Ok, I think you got it 😁 But it is possible to know P(y₁ < Y < y₂)! And it actually makes more sense.

For this, we just need to calculate the area under the plotted curve of probabilities. These probabilities are also taken by a histogram in which the values are plotted based on intervals/bins (these intervals can be as small as we want). Then a line is plotted circumscribing the histogram bins.

Continuous variable and its probability curve)

The probability of P(y₁ < Y < y₂) is this hatched area

This is just the beginning of the beginning of probability but as any base knowledge, it is important to know. Hooray! o/

Thanks for reading 😊☕️

Any suggestions, questions, comments and critiques are highly welcome! Even if it is just to compliment my drawings. I know, they are awesome.

Also posted on Hashnode <3

Types of Data Analytics

Mariana Medeiros — Tue, 05 May 2020 01:44:47 +0000

A Short Thought About Data

At first, data is some unconnected information about anything that we can find anywhere as long as we are interested in what it can tell us. I promise it makes sense if you read it once more.

For data to be found, it needs to be seen and registered. Regardless, it is there.

Unconnected data

We usually think about data as isolated snippets of information. And it actually is. For instance, the coordinates of a point in the globe, your name and interests, how many times you drink water per day, how much you enjoyed a certain movie or book, how the temperature changed yesterday, how many times you buy coffee per week (and from where), what type of music you listen to the most... they are all data. But they would not be found if no one was never interested in knowing them.

With a lot of those snippets about you, for example, we can even identify who you are, what you like or what you WOULD like (a little of Black Mirror in this one). This is actually what has been being done by the huge companies for a considerable amount of time (I am pretty sure Amazon has reached to you suggesting some products for you to buy. Some of them you did not notice you needed by the way). Now, more and more companies are relying on data to develop their business.

Even you rely on data to make some of your decisions. You go to that restaurant because you know the average rating of the place is really good. You set aside 2 hours of your time to write because you know on average this is the amount of time you need (based on previous experiences). You study for your college exams because you know you will have a better chance to pass it if you do(this is fair correlation and I hope you believe it). In fact, the more you do something(or go through something), the more you get better in handling it. And this is your brain collecting data and improving your decisions 🤓

With this being said, we can imagine how powerful data can be. Identifying patterns and explaining phenomenons are some of the most valuable things data can do for us. But for data to be powerful, it has to be properly used, analyzed and connected. Or they will be just ... lost information snippets with nothing to tell us. Or worse, it can be connected in the wrong way.

Civil Engineering doctoral candidates have been eating a lot of mozzarella cheese, huh? Source: Spurious Correlations

Data needs to be translated into meaningful and useful information.

Types of Data Analysis

This understanding of data and analytics usually comes in four key categories that are built upon each other and are linked to a respective question.

Descriptive Analytics
Diagnostic Analytics
Predictive Analytics
Prescriptive Analytics

As data analysis develops and gets more complex, it gives back more value.

Types of data analytics. Adapted from Principa

Let's explore each type, shall we?

Descriptive Analytics

The main questions here are what happened? and what is happening?

Descriptive analytics is the foundation of data insights and from where data analysis starts. It summarizes past and current data, describing the current state of what we are interested in. In business applications, it usually reflects in building dashboards that include KPIs, revenue reports and sales overview.

Diagnostic Analytics

The main question here is Why is this happening?

Diagnostic analytics focuses on the underlying reasons for the outcomes from the descriptive analytics. It also looks upon past and current data.

With the "whys" answered, better decisions can be made (and understood), both for changing the current state or for maintaining the way it is.

Predictive Analytics

The main question here is What will/might happen?

Predictive analytics tries to describe the future by using the past/present. It is all about forecasting through predictive models that correlate causes to actions/states.

For instance: which leads have the best chance of converting for our current business, how much we are going to sell this month, what are the risks for this decision we are making, how will be the weather tomorrow, how many cups of coffee I am going to drink this month(I reckon that it will be many).

As you can see, many of the predictive analytics/models are linked to time, but they are not the only ones possible. In general, predicting refers to know the likelihood of a fact to happen before its occurrence. It is from here that machine learning starts to take place.

Not all the companies get to this stage of analytics due to its complexity and investments, even though it is a very important tool to make better decisions.

Prescriptive Analytics

The main questions here is What do I need to do for the desired outcomes?

Prescriptive Analytics gathers all the previous questions (what is happening? why is this happening? what is likely to happen?) to build possible action plans for desired outcomes. From here, we can have the most perfect data-driven solutions ever (or almost that).

Artificial Intelligence is built upon prescriptive analytics by consuming a large amount of data to learn from it (going through all the previous steps) and make data-based decisions.

As popular and simple examples of prescriptive analytics outcomes, we have the recommendation systems that big companies rely on for their users: Netflix (movies and tv shows), Youtube(videos), Amazon (products), and so on. However, it can be way more complex than that when talking about business variables and constraints. Moreover, it is crucial to test the result models to ensure that they are providing meaningful recommendations.

Wrapping up

We saw that each type of analytics is associated with one or more questions and the value of the data comes from the answers to them. Therefore, it is really important to make the right questions before diving into the data exploration. Otherwise, it can mean a waste of energy (and $money$) with no meaningful outcome coming from it.

And don't forget:

HIVERY trademark tagline

Thanks for reading 😊

Also posted on Hashnode <3

WT* GIT

Mariana Medeiros — Wed, 29 Apr 2020 16:02:27 +0000

How I met Git

When first getting in touch with the software development world, I stumbled upon expressions such as "push to master", "merge this feature", "create a new branch", "commit this and that" ... and I was really lost. Like ... are you talking about trees? 🤔

Then a senior developer took his time to explain to me at least the basics about git, the main commands, and what is its purpose. Then later to this day, I started studying more and I must say ... git is really awesome (despite having some commands that are a little hard to understand sometimes).

What is Git

Before getting into the subject per se, let me just say how I admire Linus Torvalds (as a professional, not much his temper) for creating Linux and also creating the basics of Git in less than 2 days. TWO DAYS.

"It took about a day to get to be 'self-hosting' so that I could start committing things into git using git itself..." - Linus Torvalds for Linux.com

And about 10 days to get it really together.

"So I’d like to stress that while it really came together in just about ten days or so (at which point I did my first kernel commit using git)" - Linus Torvalds for Linux.com

And why the name Git? Well, let him tell it himself:

“The in-joke was that I name all my projects after myself, and this one was named ‘Git’. Git is British slang for ‘stupid person’,” Torvalds tells us. “There’s a made-up acronym for it, too—Global Information Tracker—but that’s really a ‘backronym’, [something] made up after the fact.” - Linus Torvalds for Welcome to The Jungle

Now let's get to the point

Git is a version control system that allows you to keep track of your projects and all its versions along with their changes. Most of the version control systems store information as a list of file-based changes. For instance, dropbox files are handled this way: it keeps the changes history of each file available so you can come back to an older version. If we see those changes as the difference between the current state of the file and the last state of the file, we can reference it as a delta Δ (same concept from calculus <3). And this is why this type of versioning is called delta-based.

delta-based version control

On the other hand, Git handles the files changes and data as snapshots over time. Every time you commit (save the desired state of your project), Git basically takes a picture of what all your files look like at that moment (the snapshot). To be efficient, if files have not changed, Git doesn’t store the file again, just a link to the previous identical file it has already stored. So, in the end, we have a stream of snapshots.

Git version control

Git States

Before talking about Git states, it is important to note that we need to have Git inside our project, initiating a Git repository in it. To do that, just enter your project directory and type git init through the command line. It will create a git directory inside it called .git.

The most basic and fundamental Git concept is this one. Git has three main states in which a file can be found: modified, staged and committed. Bear in mind that you have to first say "hey, Git, track this file, please". If you do not add the file to be tracked, it will not be in any of these states as Git will not see it and therefore ignore it. When you add the file, it will automatically be staged.

Modified: it means that a tracked file is changed but you did not commit it yet (you did not save its current state)
Staged: when you have a modified file and then you think "Yes I am ready to save it as it is now!" you put it on the staging area. Then you mark it to go to your next commit (snapshot)
Committed: congrats, your modified file is now properly saved and stored to the database (and snapshotted 📸)

These three states are represented in three main places/areas/sections of your Git project: the working tree, the staging area, and the Git directory. Your file is in one of these sections depending on which state it resides.

Working Tree: single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.
Staging Area: The staging area is the area that stores what will go into your next commit (it is actually a file in the Git directory). Git refers to it as the index.
Git Directory: The Git directory is where Git stores the metadata and object database for your project. It is the repository! Every time you initiate a git project, it creates a directory called .git. And every time you clone a repository, this is what gets copied to your machine.

Git states

Simple example

Now, I will just walk through these concepts by showing a simple Git flow example.

Initiate a git repository inside your project as shown above.
Create a file. Change it until it is ready to be committed and ask Git to track it by using git add file. It will put this file in the staging area.

Now I will do my first commit! Each commit should have a message stating what it is about and, if necessary, more detailed information. There are some best practices for commit messages which I will not cover here as it is not the scope of this article.

So after committing, my staging area is clean, so is my working tree. And now I can start working on subsequent commits.

OK, so ... What is Github?

Simple and short: Github (and other services such as Gitlab) are just remote repositories in which you can keep your projects(public or private). Let's say an enhanced google drive for your Git projects 🤓. You can (and must) also check other people public repositories, follow them, check open source projects (and collaborate with them!). Companies also use these services to host their own source code.

I will cover more about it in later articles :)

This is it for now. I hope you enjoy it 😊 I find Git a powerful tool. I will cover more about Git in upcoming articles such as: what are branches, main commands, how to handle undesired commits/changes ... :)

As a reference, I used their recommended Git Book. It is really good to have a complete and rich official (and recommended) documentation 💙 We tend to underestimate those and look directly for online courses/content, but it is not good practice to do so.

Also posted on Hashnode <3

Standardization | Normalization

Mariana Medeiros — Mon, 20 Apr 2020 02:18:56 +0000

Preprocessing Data

First things first! When we come across some dataset (raw data), it is recommended to follow these steps to understand and, if necessary, adjust your data for further analyses:

Read Data
Checking for missing values, categorical data, and outliers (and what to do with them)
Apply preprocessing techniques as necessary

Some of the techniques are:

Mean removal
Standardization
Normalization
Binarization
One Hot Encoding
Label Encoding

In this post, I will cover the differences between Normalization and Standardization 🤓

Standardization vs Normalization

One of the things that we do is to understand what type of variables (which we call features, simplifying) our dataset have and in what scale/unit/range/magnitude they are presented.

If the features have a significant variability, we can use the concept of feature scaling as features with high magnitudes will weigh in a lot more in the distance calculations than features with low magnitudes. To address that, we need to bring them to similar magnitude levels by using methods such as standardization and normalization.

Normalization

This method scales the numeric variables in a certain range, usually from 0 to 1 or -1 to 1 when having negative values. It is also known as Min-Max scaling.

Through this method, we calculate the proportion of the distance between the value itself to the minimum value, and the distance of the maximum and minimum values (so we need to be careful regarding outliers).

not nice to have outliers right?

Standardization

This method uses the z-score formula to scale the values, resulting in a dataset of zero mean and unit variance.

For a brief explanation, the z-score represents how many standard deviations a value is from the mean.

thanks to Math is Fun

Quick notes:

- I am not a big fan of too long articles as I find it easier to write and absorb information by smaller ones. So I would rather split information along more articles about the subject.

- Writing has been challenging yet rewarding as it is a good way to learn as well.

- Any comment that can improve my writing/knowledge is more than welcome!

Thanks for reading 😊

Also posted on Hashnode

Thoughts about quarantine and how I am handling it

Mariana Medeiros — Sun, 19 Apr 2020 03:02:42 +0000

Thoughts about quarantine and how I am handling it

Being quarantined is a condition the most of us haven't gone through (not even our parents). Especially when the motivation is a pandemic situation, which can cause us anxiety, fear, uncertainty.

These feelings stack on other ones that come from the ongoing will (or pressure?) to be productive and the rush to get a minimum financial stability (in my case). And I reckon you all have other not so welcomed feelings that are already hard to handle.

For someone that relies on specific environments to go through the day and cope with those feelings stated above, quarantine has been hard.

Separating environments to each activity is something really important to me.

And those environments do not include only physical places, but also my preparation/process to be at them. I would work/study something in the office, then I would go to the gym to clear my mind (oh how I miss it!), then getting back home and studying a little bit more of different subjects (not to mention my weekends and social gatherings ☹️). Moreover, being frequently in touch with more experienced people and developers is something that I miss as well. I feel like I learn a lot by this means.

Because of that, I had to plan a different routine (I do very well with routines), create micro-environments as I must say, manage my time better and be more active online.

And that's what I am doing

Time Management

For my routine, I am planning specific times for each activity that I would normally do: waking up, working, studying, exercising, eating. When I am at home is easier to overlap or even not doing some of these properly (skipping training, not taking a rest, eating in crazy times or event skipping some meal).

For time management as I am working/studying, I am currently using Clockify. With this tool, I can keep track of my time and if needed, allocate it better.

Micro-environments

For each activity that I do, I follow the usual process of getting ready for it and organize. Hence, I do get ready to work in the morning, I do get ready to exercise and I do get ready to slow down and sleep. I don't spend all day in my pajamas.

I also organized my working/study desk. Having lots of things over my desk is visually unpleasant and annoys me haha. I just can't.

Being more active online

Hello, Hashnode! haha

Even though it is something I was willing to do since the beginning of this year, being more active in the dev community has brought me a sense of socialising and belonging. Besides Hashnode, I am writing more on Twitter and engaging more in groups and courses' communities. Sharing and reading experiences (and resonating with some), acquiring knowledge or just having some people to talk about similar subjects have been really important to me.

Lastly but maybe importanTER

Keep in touch with your loved ones.
Being productive is not about working/studying all day.
Learn a new recipe, read a new book (for leisure!), watch a new movie/series, play your guitar, listen to your favourite bands ...
Stay healthy. Take care of yourself. Try to meditate.
Sometimes just do nothing. We all need to rest. We are not robots, computers are 🤓
Drink a good coffee ☕️ (ok, that's for me)

Bear in mind that all of these are good advices regardless of quarantine.

How are you coping with these hard times?

Also posted on Hashnode

About Me

Mariana Medeiros — Fri, 17 Apr 2020 13:21:26 +0000

Yup. Thats's me :)

First off, I would like to introduce myself.

My name is Mariana and I am from Brazil 🇧🇷 . I have a bachelor's degree in Civil Engineering, currently finishing my specialization in Structures and Foundations. However, I always liked the power and magic that come from a computer yet never have given it a try. Along with this ignored desire, when I graduated there was not much room for me from where I live (professionally-wise) and so I started to explore more and more about programming. Starting with data science then right after I got in touch with web development. As expected, my passion just reflourished (pretends to be shocked) and now I am changing careers.

When I started to get back in touch with programming, I had the fortune to come across wonderful people from the tech industry which made a HUGE difference in my life. I am truly and deeply grateful for them. It led me to many good things: an internship, meet more wonderful people, getting scholarships from online courses ... It is like all started to happen (and still is ). And now I know what I want for my professional life. Like. For. Sure. I am struggling with a lot of things and concepts and I will struggle a lot more but I feel happy, rewarded and motivated with this environment.

So ... how do I came up with the idea of this blog? 🤔 Well, I have never written a blog or anything alike but I always had the will to communicate with people and to exchange knowledge. And this blog is a good approach I found to do so (and train my English and writing haha). I believe that when we share knowledge we both teach and learn in the best way possible (active learning). Besides, when we share our journey/perspectives/tips we enable people to resonate with us and help them in their own journey. I guarantee that having someone to push you forward and/or to look up to can have a great impact on our lives.

I hope I can support you somehow and please feel free to be in touch.

Oh, and don't forget to get your daily coffee dose.

Now, let's code 🤓

Also posted on Hashnode