DEV Community: Corey

Kaggle Housing Price Competition Top 1%

Corey — Thu, 11 Nov 2021 21:04:30 +0000

I updated my notebook for the Housing Prices Competition on Kaggle and was able to go from the top 7% to the top 1% after adding some model tuning! Even though it is more of an introductory competition, it still feels good to have done so well and have all the time I put into it pay off.

Please take a look at the Kaggle notebook. I still haven't gone though all the transformations but would be happy to if anyone would like an explanation.

Kaggle - https://lnkd.in/gy3hhW63

Kaggle Housing Price Competition Top 7%

Corey — Sat, 30 Oct 2021 20:36:12 +0000

Housing Prices (top 7%)

Hello everyone,

This is my notebook for the Housing Prices Competition. A major focus for me was to practice my feature engineering. As a result, I went through each column individually to see what I could do to simplify or fill null values. I am sure I will learn ways to do this in bulk faster, but I wanted to just go through one by one to get familiar with the process. As a result I ended up using many different types of transformations including aggregates, ordinal encoding, and one hot encoding.

Kaggle - https://www.kaggle.com/coreyhamren/housing-prices-top-7

GitHub - https://github.com/TheHamhams/Housing-Kaggle

Kaggle Titanic Notebook

Corey — Tue, 26 Oct 2021 22:02:14 +0000

Hello everyone! After learning Data Science for almost a year now I finally have done my first ML solo project. I tried to limit my scope since this is my first project, but even so I was able to score in the top 10%. Please take a look and feel free to leave a comment or suggestion on how to improve the code.

Link for Kaggle Notebook - https://lnkd.in/gnJ_6Jcr
Link for Github Repo - https://lnkd.in/gfNvagWf

The Need for Transparency and Clarity in Data Science

Corey — Tue, 28 Sep 2021 21:12:43 +0000

Before I started on my path to become a Data Scientist, I was a High School Special Education Teacher. At first it may be difficult to see Teaching and Data Science as very similar, but they in fact have many similarities. In this article, I would like to focus on their need for transparency and clarity.

During the end of my teaching career, the Special Education Department had a citizen’s complaint made against it. In the complaint, the family believed that the teachers were providing insufficient services to the student. Of course, as a department, we believed our methods were sound, so we continued to teach per usual during observations by the district. However, the verdict came and the district agreed with the family. This came as a disappointment to our department, but we wanted what was best for the students and wanted to work with the district to better serve the students. Unfortunately, the response we received basically boiled down to teach better and we are going to observe you again next year to make sure you fixed everything. No transparent explanation on how they came to their conclusions, and no clear goals for us to achieve. How could we fulfill these goals if we received no clear path or parameters to achieve?

Both Data Science and Teaching benefit greatly from transparency and clarity. If no one knows how your model works, how do we know if it is accurate or equitable? If you don’t have a clear research goal, how do we know what kind of data we need? Below I have some areas of focus and tips I have learned that I believe are important to keep in mind while we develop and implement our research.

A major part of Special Education is having students achieve individualized goals. Each of these would be based around which areas they qualified in (Math, English, etc.) and created by the Teacher who had them on their caseload. All goals are not created equally though. One teacher could have a goal that read, ‘Student will improve their Algebra skills from a C grade to a B grade’, and another could read, ‘Student will increase ability to multiply fractions from 2/10 attempts to 8/20 attempts’. The difference is that the first goal has a vague goal which means different things depending on their teacher, and the second contains a clear skill for them to work on.

Data Science can have similar issues with research questions. It is extremely important to have clear goals so you know what data to use, and what counts as a success. We All Count, a group focused on equity, has a article that talks about clear research questions for Data Science. In it, they highlight the following example:

Example: If we’re working for an organization that is trying to improve the school performance of students by making sure they have access to a healthy breakfast, one of our research questions might be:

“Did math test scores increase for the students who participated in our breakfast club at least once a week?” – This question is specific, measurable and can be answered with data.

Things that are not research questions:

“Is our program working?” – This question is too general, it’s more related to the overall goal or motivation of the data project. There are many specific research questions that could come out of this general question.

By having a clear research question, you can have a better understanding of what data you need, saving you from either getting unnecessary data, or finding yourself missing important data.

While I studied for my teaching certificate, one study stuck with me even till today. The exact article I read is lost to me, but I found an article that describes it well. The study wanted to find out if paying students money to get better grades would actually increase their scores. There were three schools that took part, two of them paid students up to $500 a semester for good test scores, attendance, and grades, and one offered students $2 to read an approved book. While the students could potentially earn more money with the method of the schools that paid for grades, the only one that saw a meaningful improvement was the school that gave $2 for students reading a book. The reason for this was that the way to receive the money for reading the book was clear and transparent. Students knew exactly what they needed to do to receive the cash. Just telling students they could earn money by doing better in school did not give them any skills to actually improve, just a reward for figuring it out.

In her book Weapons of Math Destruction, author Cathy O’Neil focuses on the issue of transparency in Data Science models. An example she addresses is a model made to evaluate teachers based on a number of factors. Unfortunately, these factors were opaque so teachers were unable to know how they would be graded until after the fact. Scores fluctuated wildly from year to year as well. One teacher in particular scored a 6 out of 100 one year, and 96 the next despite not changing his teaching practices at all. She argues that prevent these type of events from happening, we need to have transparency in our models for greater equity and effectiveness. If no one knows how your model works, it is very difficult to improve or critique it or point out flaws.

Simply creating a model that gives accurate predictions is not enough, we need to strive to create models that are clear and transparent in addition to being effective. Models can have a major impact on peoples’ lives, such as getting approved for a loan. We must keep this in mind and do our best to help people and companies, and not harm them with opaque and vague models that are ineffective and difficult to improve.

Housing Flask App

Corey — Sat, 18 Sep 2021 14:59:40 +0000

This project was completed for my internship at Data Glacier.

Hello everyone!

This project included several new aspects I have never done before. It is my first time using Flask, APIs, and Heroku. It includes a simple Linear Regression model to predict housing prices in King County, Washington from 2014-2015. However, the model isn't the important part, I just used it since I am from that area and I needed to create a model for the assignment. My focus was on implementing a Flask app that could take API requests.

Flask app -

I had never created a Flask app before this project. In order to to learn how, I used information from my internship and lesson from CodeCademy.com. My focus is not on HTML and CSS, so that code is taken from a previous project I completed and edited for this app.

I wanted to make my app work with an API request and using the app UI. Unfortunately, I was unable to do so due to my lack of experience with Flask. My solution was to create a page for the app and a page for the API. I would love to hear how I could have accomplished this though.

Another feature that I had to remove was using flask_wtf to use the date from the app. Due to the deadline for the project I had to ditch this after not being able to make it work after some hours.

Heroku -

This was also my first time putting an app on Heroku. The biggest hurdle for me was getting the requirements.txt file working. I now understand the benefit of using a digital environment for this purpose, but before I used the command to create the file using all the dependencies on the machine. However, this added way too many libraries and they couldn't be found to upload them. I also tried using a Heroku python package, but for some reason none of them worked. I just ended up adding them manually which is what I should have done in the beginning since there are so few of them.

Conclusion -

Overall, this was a great experience with features I was not familiar with. The model building was really the only step I had familiarity with, but I was able to do what I needed to complete the project. I have linked the app and the repository below.

https://shrouded-meadow-77059.herokuapp.com/

https://github.com/TheHamhams/DG-Housing

Taxi Case Study(Internship)

Corey — Tue, 07 Sep 2021 13:23:37 +0000

Finished my first internship project for Data Glacier. We were asked to use provided data sets for a fictional company and decide which taxi company they should invest in. Afterwards, we created a powerpoint to communicate our findings and recommendation.

I tried to answer questions I thought would be most relevant to which would be a good investment. My graphs are pretty simple, but I wanted to focus on them being clear and understandable rather than being fancy and pretty.

As for the ppt, I wanted to showcase my skills as far as making the information clear and too the point. I used color to highlight which company was being analyzed and made a separate section to clearly state my conclusion.

I definitely faced some challenges in cleaning the data and extracting the information I wanted, but it was a great experience and I hope to use the lessons in future projects.

https://github.com/TheHamhams/taxi/tree/main/Deliverables

Life Expectancy Project

Corey — Mon, 09 Aug 2021 21:12:36 +0000

Here is another project from Codecademy. In this one we were given a csv file with information on the GDP and life expectancy(LEABY) for 5 countries and asked to find correlations.

While this prompt didn't take too much skill to find the correlations, it was the first time I was just given a csv file and told to make a notebook with it while being given no specific steps to follow.

While I am sure more visualizations could have been used, I wanted to keep it straight forward and to the point since I don't think much was needed to prove the point. However, I am not familiar with best practices for these kind of documents as this is my first one. I would love to hear your thoughts on what looked good and what could use improvement!

https://github.com/TheHamhams/Life-Expectancy-GDP

Roller Coaster Project

Corey — Fri, 23 Jul 2021 21:34:46 +0000

Hello everyone,

I have been focusing on the Data Science pathway on Codecademy so I haven't been able to post any projects for a bit that I felt were different enough from an expected answer to post. This project was a bit more open ended where they wanted result, but how you got there was on your own. I would like to start doing some solo data projects soon, I think I almost have the skills I need to feel confident in doing so.

This project took me a few hours to complete. I have been focusing mostly on statistics and data visualization, so it has been a while since I have worked with functions. My personal projects are always function heavy so it was nice to get back to working with them. I do admit, I was a bit rusty though.

Please let me know if you anything you like or could use improvements, I am always looking to get better.

https://github.com/TheHamhams/cc_roller_coaster

Monster Hunter Weapon Recommendation (CC Portfolio Project)

Corey — Wed, 16 Jun 2021 20:37:01 +0000

Hello again!

I am currently finishing up the Computer Science pathway on CodeCademy. For the portfolio project I was tasked with making a recommendation program. I have been playing a lot of Monster Hunter Rise lately, so I decided to create a program that will recommend a weapon based on a series of questions.

As with anything with hard to define criteria, I am sure that other fans of the series might categorize the weapons differently, but I ask that you see this more of a programing project and less off a true definition of each weapon :P

Please take a look at the project on Github and let me know what you think!

https://github.com/TheHamhams/CC-CS-Portfolio-Project/tree/master

Recipe Book (Hash Map Practice)

Corey — Fri, 11 Jun 2021 22:02:59 +0000

Hello again!

This is my second self Python project. My focus was implementing a hash map in my program. The categories are not very large so I am sure an array would have been fine, but I wanted to use a data structure to gain experience in it.

My program allows you to make/edit/remove various recipe categories, add/edit/remove recipes, in addition to searching all recipes according to a specific ingredient or tag. Users can also add/edit/remove specific ingredients, steps, tags, or change the category that the recipe is associated with. I originally planned on allowing each recipe to be linked to multiple categories, but that created issues with editing or removing them. If this was for an actual program, I am sure I would have kept this feature, but decided to remove it for this practice project.

I have been trying to write my code on paper before typing it to help organize my thoughts and prevent needing to edit large amounts of code. Most of the script portion was written during a camping trip with my family. It is much more relaxing than staring at a computer the whole time.

As always, I would love for you to give my project a look and let me know both what looks good and what can be improved upon.

https://github.com/TheHamhams/recipie-book

Happy coding!

Project Prioritize(Heap Practice)

Corey — Sat, 29 May 2021 15:44:55 +0000

https://github.com/TheHamhams/Project-Heap

Hello,

I am a stay at home dad who is working to become a programmer. I am currently using Codeacademy to learn, but I am also starting to work on personal projects to practice my skills.

This project was meant to practice creating and utilizing a heap structure to prioritize personal projects. I know there is a heap library, but I wanted to implement my own as a way to understand it better. The program allows you to create a project list(house work, work projects, yard work, etc.) and add specific tasks with a prioritization level in order to sort them.

The max heap is standard save the ability to delete a project by name in addition to popping off the max. There is also a method that lets you print a sorted list of the desired project category. However, I am not very satisfied with how I accomplished it. I ended up creating a copy of the list, popping off the max values in order, adding them to another list, and then printing that list. I am sure there is a better way to accomplish this, but it was the solution I came up with.

One of my main focuses as I create my projects is to implement best practices with my code as much as possible. If anyone has any suggestions on how I can improve in this area I would love to hear it. I have a lot of practice getting feedback so I welcome hearing about areas I can improve.

Codecademy Python Final Project

Corey — Mon, 19 Apr 2021 22:29:32 +0000

Hello everyone!This is my Python final project for Codecademy. This is my first project where I used python and bash on my own computer. I decided to do a small trivia game where you would need to answer 10 questions and could only get one wrong before the game would end. I have experience making a couple games in C# with Unity, but I've never made one with the terminal.

I didn't want to worry about making the game too complicated, instead I wanted to make one that did a couple of interesting things in the code (for me at least) that would show a little bit of more advanced code than just simply using functions. These included having random questions chosen from an array, randomizing the answer choices, and removing the questions from the array once they were asked so there would be no repeats. I also made the code so I could easily remove or add questions and not need to change the functions at all. I also made it so the only line of code besides the functions and variables was game_start() and compartmentalized it to work from there.

While this is definitely a minimum viable product, I am happy with how it turned out. If I were to focus on content I would have added randomized responses to correct or incorrect answers as well as a bit more story, but that was not the focus of this project so I wanted to practice not over reaching and considering it complete.

Below is a link to the code if you would like to take a look, I would love to hear what you think! (positive or negative, I'm a big boi and can handle criticism)
https://github.com/TheHamhams/Python_Game