DEV Community: Jonathan Fetterolf

Collab Project

Jonathan Fetterolf — Thu, 25 Sep 2025 03:25:44 +0000

I recently had an IRL friend (a concert going, beer drinking, game playing kind of friend) ask me for a reference. He wasn't specifically clear on what type or what for, but he is job hunting, so I figured he meant professional. I told him that I'd be more than happy to give a personal reference but couldn't give him a professional one... UNLESS... we did a collab project on the side.

WHAT TO DO!?!?

Well, I've been mulling over an elaborate idea of creating some sort of content generation application that streamlines and automates most steps in the content generation pipeline. We had a quick brainstorming meeting and came up with a few want's, need's, and requirement's for what we'll be building.

WELL, WHAT WILL IT HAVE!?!?

At this point we know that a lot of of the services will be backend heavy so we started plotting our meat & potatoes API. Beyond that we think we'll have some sort of text-to-speech (TTS) service, image gathering/generation service, some sort of llm service, and all of this will be in front of PostgreSQL db.

WHAT MAKES THIS SO FUN!?

Well, my friend has ZERO, ZILCH, NADA collaborative development experience. So first things first, we came up with a list of things to familiarize himself with as well as the right ways to do it. So we're starting from the start here. IDE, Git (CLI), project documentation, ENV's, etc., etc., etc. Also, most importantly no cheating w/ "ChatGerald" or "Geminguy". Rely on documentation and S/O as much as you can. If you end up at one of those aforementioned helpers, use it to explain concepts to you, not do the work for you!

ANYHOW...

That's where we're at. If you want in, please let me know. I'm sure we'll need an abundance of help. Also, This all happened a month ago so the project's underway...

Peace, love, and shiny things...

Want to Follow Along?

GitHub | LinkedIn | Twitter

Coding: Growth, Learning, & Transformation

Jonathan Fetterolf — Thu, 17 Aug 2023 21:38:44 +0000

GitHub | LinkedIn | Twitter

Introduction

When I started my new position, I figured it would be technical, but I couldn’t have guessed that it would leave me completing coding challenges in my free time. This new role required me to dive into an array of unfamiliar languages, tools, and technologies. As I navigated this terrain, I recognized the importance of reinforcing my foundational coding skills. The constant problem-solving demands of my day-to-day coding motivated me to seek out resources that could aid my growth. I wanted to efficiently master these tools and demonstrate my ability to learn, adapt, and implement. This quest for knowledge led me to explore a trio of online resources recommended by a mentor outside the company. Each of these platforms continue to play a pivotal role in my coding journey.

LeetCode

LeetCode became my initial port of call. Its array of coding challenges and problems provided me with the perfect arena to hone my skills. As I delved into its challenges, I found myself facing complex problems that demanded strategic thinking. Over time, I noticed a significant shift in how I approached these tasks. I transitioned from stumbling through problems to adopting a more systematic and organized approach. This transition was thanks in part to an enlightening article on algorithms and problem-solving techniques, which revolutionized my perspective.

HackerRank

HackerRank, the second platform in my arsenal, complemented my LeetCode experience. What stood out to me were its diverse problem sets and the ability to solve challenges using multiple programming languages. This flexibility not only broadened my repertoire but also exposed me to different problem-solving methodologies. Each challenge I tackled felt like a mini-victory, reaffirming my progress.

Codewars

Codewars rounded out my trio of learning resources. Its unique approach of gamifying coding challenges injected an element of fun into my learning journey. The challenges on Codewars ranged in difficulty, allowing me to progressively build my skills. I found that working my way from easier problems to more complex ones was a productive strategy. Through this approach, I honed my problem-solving skills, breaking down intricate problems into manageable steps.

Personal Progress and Achievements

Before embracing these platforms, I often found myself overwhelmed by problems and attempted to tackle them all at once. However, as I continued to engage with LeetCode, HackerRank, and Codewars, my coding approach transformed. Concepts that initially seemed overwhelming became clearer as I gained insights from each problem completed.

Although I can't pinpoint a single problem that brought about a significant leap in my skills, an article on algorithms and problem-solving techniques acted as a turning point. This newfound understanding reshaped how I approached complex coding problems. As I encountered increasingly complex challenges, I found that my anxiety decreased, and my confidence soared. My thought process became more organized, enabling me to code more efficiently while taking smaller, deliberate steps in the problem-solving process.

Lessons Learned

These platforms weren't without their challenges. Some problems initially appeared insurmountable. However, I learned that starting with simpler problems and gradually working my way up was an effective strategy. Developing a robust problem-solving methodology provided me with a roadmap to tackle even the most intricate challenges.

Consistency emerged as a key learning approach that worked best for me. Solving a single problem each day, typically after a full day's work, proved to be an effective way to integrate learning into my routine without overwhelming myself.

Acknowledging that no one knows everything and that seeking assistance is perfectly acceptable was a transformative realization. Keeping helpful references saved and revisiting them when needed proved instrumental in overcoming these hurdles.

Conclusion

In retrospect, the benefits of engaging with these coding websites have been threefold: enhanced problem-solving techniques, quicker access to appropriate resources, and reduced anxiety when approaching coding challenges. The combined effect of these advantages has fortified my coding skills and enabled me to face new problems with confidence.

For those embarking on a coding improvement journey through online resources, my advice is simple: don't feel pressured to know everything from the outset. Dive into tutorials, address knowledge gaps along the way, and embrace the process of continuous learning.

Looking ahead, my plan involves continual integration of these platforms into my daily routine, leveraging them as tools to refine my skills after a productive day's work.

As I progress, I have my sights set on deepening my expertise in JavaScript, Java, and potentially exploring C#. The journey continues, and I'm excited to embrace new languages and areas of coding.

To stay updated with my future coding-related blog posts, feel free to follow my journey on LinkedIn.

Let me know which language or tool you think I should delve into next!

Want to Follow Along?

GitHub | LinkedIn | Twitter

Streamlining Water Well Maintenance in Tanzania: A Logistic Regression Approach

Jonathan Fetterolf — Mon, 07 Aug 2023 00:36:37 +0000

GitHub | LinkedIn | Twitter

Introduction

Access to clean and potable water is a fundamental necessity, yet many regions, including Tanzania, face challenges in providing this essential resource. The IHH Humanitarian Relief Foundation, an NGO dedicated to improving water access, strives to efficiently allocate their maintenance efforts by accurately predicting the functionality of water pumps. By building a classification model, they can optimize their operations, maximize maintenance resources, and ensure clean water is readily available to the people of Tanzania.

The Cost of Errors

With Tanzania's vast geographical area and limited resources, it is crucial to deploy maintenance and repair efforts judiciously. The country's infrastructure spans over 21,000 miles of roadways, making targeted repairs essential. Constructing a well in Tanzania can cost upwards of $10,000, considering factors such as labor, drilling depth, rock density, location, and fuel costs. Repairing wells, which can range from a few hundred to several thousand dollars, is an expense that should be allocated only to wells in genuine need.

Baseline Model and Simple Model Performance

The baseline model, which predicts all wells as functional, achieved an accuracy of approximately 54%. However, this approach proved insufficient for accurate predictions. Consequently, several simple models were explored, including Logistic Regression, Decision Tree Classifier, Random Forest Classifier, Gradient Booster Model, and XGBoost Model.

Among these models, Logistic Regression emerged as the optimal choice. It achieved an accuracy of approximately 79.1%, outperforming the other models in terms of speed, interpretability, and resistance to overfitting. The Logistic Regression model was further refined using GridSearchCV to identify the best hyperparameters, including the mean imputation strategy for numerical values, a C value of 1.0, penalty 'l2', and the 'liblinear' solver.

The Final Logistic Regression Model

Considering the similar performance of multiple models, Logistic Regression was selected as the final model due to its efficiency in training, interpretability, and robustness against overfitting. The final Logistic Regression model achieved an accuracy of approximately 79.6%. This model provides IHH with a reliable tool for predicting the functionality of water pumps, aiding in prioritizing maintenance efforts and optimizing resource allocation.

Further Exploration and Questions

While the binary classification of functional and non-functional wells is valuable, there is potential benefit in identifying specific wells that require repairs while still being functional. Such insight could enable targeted preventive maintenance, avoiding costly repairs in the future.

Given more time and resources, it would be worthwhile to create a model that predicts the original status groups ('functional,' 'non-functional,' and 'functional needs repair') instead of converting the target into a binary outcome. This expanded model could provide more detailed information and enhance decision-making processes.

Understanding the limiting factors in delivering resources to wells that require repairs is vital. Identifying the challenges related to maintenance professionals, time constraints, financial resources, availability of parts, and knowledge gaps will help IHH devise effective strategies for addressing these obstacles.

Conclusion

The implementation of a logistic regression model has empowered the IHH Humanitarian Relief Foundation to enhance their water well maintenance operations in Tanzania. By accurately predicting the functionality of water pumps, the NGO can efficiently allocate resources and prioritize repairs where they are most needed. As ongoing efforts continue to optimize resource utilization, explore more granular predictions, and address limiting factors, IHH moves closer to its goal of ensuring clean and potable water for all Tanzanians.

Want to Follow Along?

GitHub | LinkedIn | Twitter

Deploying a PERN Full-Stack CRUD App with Vercel: Hosting and Integration Made Easy!

Jonathan Fetterolf — Sat, 29 Jul 2023 17:58:47 +0000

GitHub | LinkedIn | Twitter

Over the past few weeks, I’ve been working to create a PERN full-stack CRUD application. I wanted to share it with others and make it available for friends, family, and colleagues to check out. So, I started looking into options to host my application.

I chose to deploy my application on Vercel with the hobby plan for a few reasons. First and foremost, Vercel is incredibly easy to use. Its user-friendly interface and streamlined deployment process made it a great fit for me, as I didn't want to deal with complex hosting setups. Additionally, the hobby plan offered a free tier, which was perfect for hosting my personal project. Lastly, Vercel's seamless integration with Git repositories, especially GitHub, allowed me to set up automatic deployments whenever I pushed changes to my repository, saving me time and effort.

Deploying my project on Vercel was a breeze since my code was already on GitHub. All I had to do was import my Git repository to Vercel. I went to the Vercel dashboard, clicked on the "New Project" button, and selected the "GitHub" option. After authorizing Vercel to access my GitHub account, I chose the repository I wanted to deploy. With a few configurations, like selecting the branch to deploy and setting up build options, my application was automatically deployed whenever I made new changes to the selected branch.

Besides the GitHub integration, Vercel offers other deployment methods too. These options include Vercel CLI which allows the user to deploy directly from their local development environment. Another option is deploy hooks which are unique URLs that trigger deployments when accessed. The final option is a deployment via a RESTful API which allows the user to create Deployments by making an HTTP POST request.

Vercel provided me with an excellent hosting solution for my PERN full-stack CRUD application. Its ease of use, free hobby plan, and seamless Git integration made the deployment process smooth and hassle-free. Whether deploying via GitHub, using the CLI, deploy hooks, or the REST API, Vercel offered flexibility and efficiency in hosting my web application.

In my upcoming post, I’ll walk you through the process of hosting my PostgreSQL database and integrating it with this application on Vercel. This follow-up post will be the last step of deploying a full-stack application with a backend database.

Want to Follow Along?

GitHub | LinkedIn | Twitter

Unlocking Digital Accessibility: Navigating the World of 508 Compliance

Jonathan Fetterolf — Sat, 22 Jul 2023 15:18:25 +0000

GitHub | LinkedIn | Twitter

There are...

285 million people with some sort of visual impairment...
275 million people with moderate-to-profound hearing impairment...
Many more with physical, speech, cognitive, and neurological disabilities...

Let's ensure ALL of our users have access to this service regardless of their capabilities.

"The moral test of government is how that government treats those who are in the dawn of life, the children; those who are in the twilight of life, the elderly; and those who are in shadows of life, the sick, the needy, and the handicapped." - Hubert H. Humphrey

So what is 508 compliance anyway…?

508 compliance refers to the standards and guidelines set forth by Section 508 of the Rehabilitation Act of 1973 in the United States. The goal of this standard is to eliminate barriers in technology that could prevent individuals with disabilities from accessing, using, and interacting with digital information and services. It requires federal agencies to ensure that their electronic and information technology (EIT) is accessible to individuals with disabilities.
Specifically, it mandates that all federal agencies must make their EIT accessible to people with disabilities, including employees and members of the public. It also serves as a best practice for non-governmental organizations and businesses to ensure inclusivity and equal access to digital information and services for all individuals, regardless of their disabilities.

508 practices:

From the start:

It is essential to incorporate accessibility considerations from the early stages of the application design and development process as certain requirements can prove challenging to retroactively implement for complex interactions present in modern user interfaces.

Providing alternative text for images.

The alt text is read by screen readers and other assistive technologies used by individuals with visual impairments or those who have difficulty viewing images. For example:

<img src="fetterollie-best-blog-ever" alt="Best blog that exists">

Using captions and transcripts for multimedia.

Captions and transcripts make multimedia content, such as videos and audio, accessible to individuals with hearing impairments.

Ensuring keyboard accessibility for all functionalities.

Keyboard accessibility makes digital content and applications usable for individuals with various disabilities, including motor impairments and some types of visual impairments.

Using sufficient color contrast for text and images.

Sufficient contrast makes applications and content usable for individuals with certain visual impairments such as color-blindness.

Structuring content with appropriate headings and labels.

The use appropriate headings and labels promotes accessibility, improves navigation, enhances user comprehension, and ensures compliance with web standards, leading to a better overall user experience on the web.

Avoiding content that could cause seizures or other physical reactions.
Providing resizable text and adaptable layouts.

Adaptable layouts create an accessible and user-friendly digital environment that caters to diverse audiences and devices.

Making sure forms and interactive elements are accessible.

Challenges with 508:

Designing applications for 508 accessibility poses implementation challenges such as complex interactions, third-party components, dynamic content, multimedia accessibility, assistive technology compatibility, responsive design, accessibility expertise, design vs. accessibility balance, regular updates, and compliance education. Early integration of accessibility, user testing, and staying informed about guidelines can help overcome these challenges.

In conclusion:

508 compliance is crucial because it ensures that electronic and information technology is accessible to individuals with disabilities, promoting inclusivity, equal access to digital content and services, better user experiences, and legal compliance. By adhering to these standards, organizations and developers can create technology that caters to a broader audience and supports the rights and needs of individuals with disabilities, fostering a more inclusive and equitable digital environment.

Sources:

U.S. Access Board - Advancing Full Access and Inclusion for All

Section508.gov - Buy. Build. Be Accessible.

U.S. Department of Health and Human Services - Enhancing the health and well-being of all Americans

Want to Follow Along?

GitHub | LinkedIn | Twitter

Resources for Learning React and the PERN Stack: From Setup to Authentication

Jonathan Fetterolf — Sat, 15 Jul 2023 21:30:18 +0000

This post highlights a variety of resources that can help you learn React and the PERN stack (PostgreSQL, Express, React, and Node.js) for building fully functional applications. From tools like Create React App for streamlined setup to comprehensive tutorials and crash courses on React, Material UI, and PostgreSQL, these resources cover essential concepts and techniques. Additionally, there are tutorials on working with APIs and implementing user authentication and login functionality in the PERN stack. By utilizing these resources, you can gain a solid foundation in building React applications and understand how to integrate various technologies to create robust full-stack solutions.

Create React App

Create React App is a popular tool that helps you set up a new React project with a pre-configured development environment. It provides a streamlined setup process and allows you to start building React applications quickly. Learning Create React App is important because it simplifies the initial setup of a React project and you can focus more on learning and building React applications, rather than dealing with complex configuration and tooling choices.

Full Modern React Tutorial - YouTube:

This YouTube tutorial offers a comprehensive guide to React, covering essential concepts, components, state management, and more. It's a valuable resource for beginners looking to learn React from scratch.

React JS Crash Course - YouTube:

This crash course on React provides a condensed introduction to React's core concepts and features. It's designed to give you a quick overview and get you up to speed with building React applications.

MUI (Material UI):

Material UI is a popular React component library that follows Google's Material Design guidelines. The resources you mentioned provide tutorials and crash courses on using Material UI with React, teaching you how to create visually appealing and responsive user interfaces.

Material UI Tutorial - The Net Ninja:

This playlist provides a comprehensive introduction to React by guiding you through building a complete project from start to finish.

Material UI Crash Course:

This video provides a comprehensive overview of React, covering essential topics such as components, state management, JSX, lifecycle methods, hooks, forms, and routing.

APIs:

These resources cover how to work with APIs in React applications. They introduce you to using popular libraries like Axios for making API requests and guide you through specific examples like fetching weather data from the OpenWeatherMap API.

How to Use Weather API for Beginners

React Axios API Requests

Axios - NPM

OpenWeatherMap

PostgreSQL:

PostgreSQL is a powerful open-source relational database management system. The resources you mentioned include a comprehensive tutorial that covers PostgreSQL from beginner to intermediate level, and a PERN stack course that introduces you to using PostgreSQL along with Express, React, and Node.js.

Comprehensive PostgreSQL

PERN Stack Course

Login:

These resources focus on implementing user authentication and login functionality in your React application using the PERN stack. They cover topics like JWT (JSON Web Tokens), user registration, and backend authentication implementation.

Add Login Authentication to React Applications

Frontend PERN authentication

Backend PERN authentication

API's, MUI, PERN, & more...

Jonathan Fetterolf — Mon, 10 Jul 2023 00:33:18 +0000

GitHub | LinkedIn | Twitter

This week I worked on several tasks and gained experience with various technologies. Here's a breakdown of what I worked on:

Postman Application:

I worked extensively with the Postman application, which is a powerful platform for building and using APIs. It simplifies the entire API lifecycle and promotes collaboration among team members, allowing us to create better APIs faster.

API Integration with Axios:

As part of my internship, I utilized Axios, a promise-based HTTP client, for integrating APIs. Axios is commonly used in both node.js and browser environments. It provided me with a convenient and efficient way to make HTTP requests and handle responses. I used Axios to integrate external APIs and update the weather information within our application based on the user's location. It was exciting to see real-time weather updates within the application.

Material UI:

I worked with Material UI, an open-source React component library. Material UI implements Google's Material Design principles and offers a wide range of prebuilt components that are production-ready out of the box. I utilized Material UI components such as Box, Button, Container, Textfield, and Typography to enhance the user interface of the application.

Custom Theming in Material UI:

In addition to using Material UI components, I implemented custom theming to improve the look of the application. It was fascinating to work on defining a custom palette for both light mode and dark mode, ensuring a consistent and visually appealing experience for our users.

Fourth of July Celebration:

As a fun break from work, I joined my friends in Alexandria, VA, to celebrate the Fourth of July. We enjoyed the festivities associated with this national holiday, which allowed me to relax and recharge.

PERN Stack & Interoperability:

In terms of the stack we used, I focused on achieving interoperability between our application and the database using the PERN stack. PERN stands for PostgreSQL, Express, React, and Node.js. I worked with PostgreSQL, a powerful and open-source object-relational database known for its performance. Express, a minimal and flexible Node.js web application framework, which provided a robust set of features for building the back-end. With React, I could created user interfaces by combining reusable components, and Node.js served as the runtime environment for executing JavaScript code.

Throughout this week, I gained valuable experience in API integration, front-end development with React and Material UI, custom theming, and back-end development with the PERN stack. These skills will undoubtedly contribute to my growth.

Challenges

One of the biggest challenges I faced this week was working with MUI theming. While implementing the custom theme, I encountered issues where certain components I used in the application would break the theming. This required me to scrap those components and start again to ensure the consistent look and feel across the application.

Upon closer examination, I realized that the problem stemmed from my approach to importing MUI components. Initially, I was relying on the automatically generated imports suggested by Visual Studio Code. However, I later discovered that I should have been grabbing and using the imports directly from the MUI documentation.

This realization emphasized the importance of closely following the official documentation when working with third-party libraries like Material UI. The documentation provides precise instructions and guidelines for using each component correctly, including the proper way to import them.

By adhering to the documentation, I ensured that I was using the appropriate imports, which played a crucial role in maintaining the integrity of the MUI theming and avoiding issues with component compatibility. This experience served as a valuable lesson in the significance of meticulous attention to detail and the benefits of relying on official documentation as a reliable resource.

Moving forward, I learned to be more cautious with automatically generated imports and to double-check the documentation for accurate import statements.

Want to Follow Along?

GitHub | LinkedIn | Twitter

My Journey into Software Development: First Week of Internship

Jonathan Fetterolf — Fri, 30 Jun 2023 22:44:54 +0000

GitHub | LinkedIn | Twitter

New internship, who dis...?

Introduction:

Starting my new internship in software development/data management has been an exciting and challenging experience. In this blog post, I will share the details of my first week, including the onboarding process, setting up the development environment, and creating a basic application using Create React App. Let's dive into my journey!

Onboarding:

During my first week, the onboarding process played a crucial role in helping me become familiar with the company's culture, policies, and expectations. Here are the key aspects of my onboarding experience:

General onboarding and paperwork:

I went through various forms and documents to officially join the company. This involved signing employment agreements, filling out tax forms, and providing necessary identification documents.

Computer setup:

Setting up my work computer was an essential step. I was guided through the process of connecting to the company's network, installing necessary software, and configuring security settings.

Logging into accounts:

I received credentials for different accounts, such as email, project management tools, and version control systems. I made sure to secure my passwords and enable two-factor authentication whenever possible.

Setting up applications:

To streamline my development workflow, I installed and configured applications specific to the company's tech stack. This included code editors, integrated development environments (IDEs), and communication tools.

Email Signature:

I created a professional email signature that includes my name, position, and contact information. I followed the guidelines provided by the company to maintain a consistent and professional image in my communications.

Trainings:

As an intern, I underwent several training sessions to familiarize myself with important aspects of the company's operations and policies. Here are the key areas I received training in:

Ethics:

Understanding and adhering to ethical standards is crucial in any professional setting. I received training on the ethical considerations related to my work, including intellectual property, data privacy, and confidentiality.

Harassment:

Promoting a safe and inclusive workplace environment is essential. I participated in harassment training to learn how to recognize and prevent any form of harassment, fostering a respectful work environment for all employees.

HIPAA (Health Insurance Portability and Accountability Act):

Since I am interning at an organization that handles healthcare data, I received training on HIPAA regulations. These regulations aim to protect the privacy and security of patient health information.

Code of Conduct:

Every organization has a code of conduct that outlines acceptable behavior and expectations from employees. I familiarized myself with the company's code of conduct to ensure I aligned my actions with its principles.

Hybrid Work

Understanding my work schedule and attendance requirements was crucial for a successful internship. In this case, the internship follows a hybrid model with mostly remote work but requires me to be in the office one day per week. I made sure to plan my tasks and assignments accordingly.

Setting up the Integrated Development Environment (IDE):

As a software development intern, setting up my development environment was essential for efficient coding. Here's a breakdown of the tools and frameworks I worked with:

Visual Studio Code:

I installed Visual Studio Code (VS Code), a popular code editor that provides a rich set of features for editing and debugging code. I configured VS Code according to the company's preferences.

Node.js:

I installed Node.js, a JavaScript runtime that allows me to run JavaScript code on the server-side. Node.js provides a powerful ecosystem and libraries that I can leverage in my projects.

Node Version Manager (NVM):

I utilized Node Version Manager (NVM), a handy tool that helps manage multiple versions of Node.js on my system. It allowed me to switch between different Node.js versions with ease.

Yarn:

Yarn helps manage dependencies in projects by providing a reliable and efficient way to install, update, and manage the packages your project depends on.

Git:

Git offers numerous benefits and features that make it an essential tool for developers including version control, easy collaboration, creating branches, and merging branches.

Learn the Create React App

I primarily used tutorials on YouTube to learn the Create React App framework. These are the two tutorials I found most useful:
Tutorial 1
Tutorial 2

Creating an Application

After learning the basics, I was asked to create a simple application. These were the initial requirements for the application:

Add a few pages to navigate between
Create a clicker widget that allows you to count both up and down
Make a separate page that has the functionality to filter through an array
Make a list of car objects (at least 10) that have the following data fields: Unique Id, Model, Year, Color, Image Source
Create a functionality that will ingest data, and filter results
Create text fields to allow for user input
Output results
Limit results so there are no duplicates

Summary:

My first week of interning has been an enlightening and invigorating experience. From seamless onboarding to immersive training and hands-on development, I have gained valuable insights and embarked on a remarkable journey. The supportive work environment, the introduction to cutting-edge development tools, and the captivating world of data management have set the stage for an exciting internship filled with growth, learning, and countless opportunities. I eagerly look forward to what lies ahead as I continue to expand my skills and contribute to the company's mission.

Want to Follow Along?

GitHub | LinkedIn | Twitter

Using Data Science to Fight Malaria: A Breakthrough in Blood Cell Classification

Jonathan Fetterolf — Sun, 04 Jun 2023 13:22:02 +0000

GitHub | LinkedIn | Twitter

Introduction:

Data science has the potential to revolutionize the medical field. I demonstrated this by developing an application to swiftly and accurately identify the presence of Malaria in blood cells. This innovative approach enhances the capabilities of doctors and technicians, allowing them to allocate their valuable resources more effectively and ultimately save more lives. In this blog post, we will explore the development of this application, the challenges encountered, and the exciting future possibilities.

Building the Application:

The project began with a dataset consisting of images of blood cells categorized as either Uninfected Blood Cells or Parasitized Blood Cells. To determine where the application could be most impactful, auxiliary data from the World Health Organization was incorporated to identify regions with high Malaria prevalence. Python, along with libraries such as pandas, matplotlib, seaborn, and geopandas, were utilized to analyze and visualize this additional data.

Using the power of Python, Tensorflow, and Keras, I constructed a neural network and trained it using over 15,000 blood cell images. Remarkably, the network achieved an impressive accuracy rate of 96% in classifying the blood cells. To leverage the advantages of GPU computing, Google Colab was employed to create a notebook that facilitated the training process. Subsequently, a live application was developed using streamlit.io, enabling users to submit their own blood cell images and receive predictions in real-time.

The Impact:

The neural network proved to be highly reliable, accurately predicting blood cell classifications in over 96% of cases. This breakthrough technology empowers doctors to make quicker and more informed decisions in treating Malaria cases, thereby halting further transmission and ultimately saving lives. By providing an accurate diagnosis and determining the parasitic burden, this application enhances the efficiency and effectiveness of medical interventions.

Challenges Encountered:

While developing the application, several challenges were encountered, demonstrating the complexity of implementing advanced technologies in real-world scenarios. One such challenge was the restriction on Google Colab's compute units, necessitating limitations on the utilization of GPU processing for model training. Given additional resources, such as a higher budget, the model could be trained with more data, leading to even greater accuracy in cell classifications. Furthermore, certain features of Keras did not seamlessly integrate with Colab, specifically in terms of image pre-processing. To circumvent this issue and prevent overtraining the neural network, alternative methods were employed for image pre-processing, omitting the brightness and contrast adjustments.

Future Possibilities:

Given more time and resources, I have plans for enhancements to the application. Expanding the dataset and retraining the model would improve its accuracy and robustness. Additionally, a new feature could be introduced, allowing users to submit images of entire blood smears, which would then be automatically split into individual cell images for input to the model. This advancement would enable the model to estimate the parasitic burden, a crucial factor used by clinicians to make informed decisions regarding the treatment of Malaria cases.

Conclusion:

The development of an application that utilizes data science to classify Malaria in blood cells represents a significant breakthrough in the medical field. By leveraging advanced technologies such as neural networks and GPU computing, this project demonstrates the potential for data-driven solutions to positively impact healthcare. Despite the challenges faced, the impressive accuracy achieved and the future possibilities outlined highlight the importance of continued exploration and innovation in the field of data science for medical applications. With ongoing advancements, we can look forward to a future where data-driven approaches play a vital role in combating diseases and improving patient outcomes.

If you want to see or check my work, you can find the project details on my GitHub here: Malaria Blood Cell Classification

Want to Follow Along?

GitHub | LinkedIn | Twitter

DO YOU YAML?

Jonathan Fetterolf — Tue, 17 Jan 2023 01:23:17 +0000

Say what?

Introduction of YAML

YAML stands for "YAML Ain’t Markup Language" - this is known as a recursive acronym. YAML is often used for writing configuration files. It’s human readable, easy to understand and can be used with other programming languages. Although YAML is commonly used in many disciplines, it has received criticism on the amoutn of whitespace .yml files have, difficulty in editing, and complexity of the standard. Despite the criticism, properly using YAML ensures that you can reproduce the results of a project and makes sure that the virtual environment packages play nicely with system packages. (If you're looking for another way to share environments there are other alternatives to YAML which include StrictYAML (a type-safe YAML parser) and NestedText)

One of the first steps in entering into an existing data science project is setting up your virtual environment. This makes sure that dependencies and packages used for this project do not interfere with each other or write over those previously used. There will often be a file with the .yml extension in the project files so you can quickly get working on the existing project. Below, I’ll quickly run through the steps I take to create a virtual environment on my M2 MacBook with Anaconda already installed.

Steps

So where is that YAML file on GitHub and what do I do with it?:

First, projects typically will have one .yml file but sometimes you’ll see special instructions in the project’s read me:

Here’s what the actual YAML file will look like (they're usually on the root level of the directory, but can sometimes be further down in the directory:

This is what the file will look like on GitHub:

To save the .yml file, simply click the Raw button here:

Then, in the newly opened tab, right click and save as:

(Make sure to save this somewhere you can easily find this as you’ll need to navigate to it!)

Open up a new terminal session and navigate to the directory where you saved the .yml file.

To create this new environment, I’ll enter:

conda env create -f geoenvironment.yml

After the virtual environment is done installing, you’ll need to activate it to use it. To do so, you’ll need to know it’s name which should be displayed with the command to activate it. If not, check a list of your environments by entering:

conda info --envs

Then activate the new environment by entering: (replace 'project-env' with the name of your virtual environment)

conda activate project-env

Now you’re ready to start chugging on that existing project.

When you’re done working in that virtual environment, don’t forget to deactivate and switch to the next environment you want!

conda deactivate

If you want to start a project from scratch, I prefer to start with a very basic virtual environment and add the packages I need as I go along. My basic framework usually consists of:
Python
NumPy
Pandas
MatplotLib
& sometimes Seaborn

Finally, once you've created your environment and you're ready to unleash it on the world you can run a simple command to export the .yml file. Once you have your file you can upload it or share it with whomever you need. Here is the command to export (feel free to replace "environment" with the desired name of your new environment):

conda env export > environment.yml

The whole process of creating and activating a new virtual environment is pretty simple when it works… However, if you run into errors such as not being able to find the right packages, it can get a little hairy. Luckily, there are great resources out that are just a quick google away. The most useful resources I found for these errors were on Stack Overflow and Apple Developer.

If you want to create a virtual environment from a .yml file, here’s a link to one of my projects (Tanzanian Water Wells: Predicting the Functionality of Water Wells in Tanzania) where you can try it out!

In Summary:

YAML isn’t scary (also ain’t markup language)
The .yml is an important feature in any Data Science workflow.
The .yml is used to ensure that packages and versions are the same
The .yml helps with reproducibility.
Including a .yml on a project allows for collaboration
YAML is altogether pretty simple.

Resources

The official YAML Web Site

If you’re looking for further resources on running TensorFlow and Keras on a newer MacBook, I recommend checking out this YouTube video: How to Install Keras GPU for Mac M1/M2 with Conda

If you’re looking for a resource on how to install Anaconda, I highly recommend that you go straight to the source, anaconda.org.

The M2 MacBook gave me some challenges when trying to work with TensorFlow and Keras due to some fancy chip architecture which you can read about here: TensorFlow with GPU support on Apple Silicon Mac with Homebrew and without Conda / Miniforge

If you want to take a dive into the YAML world, here's an in-depth tutorial: YAML: Everything You Need to Get Started in Minutes

Further, Very Serious Notes

Why did the YAML cross the road?

To get away from the package that broke the YAML's backend.

Want to Follow Along?

GitHub | LinkedIn | Twitter

7 Favorite Sites to Find Datasets

Jonathan Fetterolf — Fri, 16 Dec 2022 02:46:50 +0000

Introduction

So, you're looking for the next perfect dataset to use in your upcoming project? Look no further. Well, look a little further... I share my favorite sites below.

I'm always looking for the next dataset to use in solidifying a new concept I'm studying. This list should help cut down the time it takes to find that perfect dataset. The sites included all have access to free datasets.

Kaggle

Kaggle is an online community platform for data scientists and machine learning enthusiasts. Kaggle hosts over 186,000 data sets on topics ranging from games to death rates and everything in between.

What is nice about Kaggle?
Kaggle suggests categories of datasets such as trending, music, business, computer science, and classification so you can quickly get to a dataset you're interested in. If you know exactly what you're looking for (or what you don't want included), Kaggle also offers options to filter by file size, file type, and license type. My favorite feature that Kaggle offers is that each dataset has a "Usability" rating which takes into account the completeness, credibility, and compatibility of the dataset.

Data Is Plural

Data Is Plural is a weekly newsletter of useful datasets published by Jeremy Singer-Vine. There have been over 300 editions of this newsletter, the earliest is from October 21, 2015.

Pros and Cons for Data Is Plural:
This is a super fun site for discovering useful/curious datasets. It's not the easiest site to navigate but once you navigate to reading the full archive as a spreadsheet, it becomes searchable by keywords (using ctrl+f).

FiveThirtyEight

FiveThirtyEight shares the data behind some of their articles for you to use to create your own stories and visualizations. FiveThirtyEight offers datasets used in articles about culture, economics, politics, science & health, and sports.

Pros and Cons for FiveThirtyEight:
Their datasets are listed in order of most recently updated with those in the status of currently being updated first. The most prominent place to click will take you to the article(s) that reference the dataset. The best way to explore what the dataset offers is to click the info link and navigate to the GitHub repository that houses all its available datasets (Repository found here).

UCI Machine Learning Repository

The University of California, Irvine hosts its own Machine Learning Repository with over 600 datasets. The site is very intuitive but looks a bit dated. It has an open beta for its new site that you can access here.

Pros and Cons for UCI Machine Learning Repository:
Most of its datasets are intended for use with classification. The site is very easy to use and its beta site has an updated look and includes filtering features within the search option.

Data.gov

Data.gov Hosts over 245,000 datasets and is managed and hosted by the U.S. General Services Administraion, Technology Transformation Service. Recently, I used Data.gov to find a dataset for Meteorite Mania, my personal project that uses geopandas to plot the data of known meteorite landing sites onto a map of the world.

Pros and Cons for Data.gov:
Data.gov has a basic layout that's easy to use and intuitive. It is easy to filter by location, formats, publishers, and bureaus.

BuzzFeedNews

Who doesn't love a good BuzzFeed article? BuzzFeedNews has made its datasets available with links to their repositories and articles so you can see what is actually going on with the data behind the scenes. If you haven't looked into this source, I highly recommend it.

Just Love for BuzzFeedNews:

I love that you can see the notebooks behind the articles. This allows you to see exactly how the authors used the data to get to the later stated conclusions. The datasets have already been analyzed so you may struggle to come up with a new spin on them but it's a great source for learning new methods.

AwesomeData

AwesomeData is a topic-centric list of links to open datasets. This is a great place to explore a topic through the lens of data.

Pros & Cons for Awesome Data:

If you're looking for a quick and to-the-point link to a dataset, this isn't it. If you're looking to explore and find some unique and interesting places with datasets, this one is for you!

Conclusion

Kaggle is my go to for finding a useable dataset as quickly as possible. Data.gov is a very close second. If you're looking for a little bit of a detour that can lead you down a tangent but end up somewhere fun in the end: Data Is Plural is where it's at. BuzzFeedNews is great if you want to learn some new techniques or see the 'how' behind some number crunching. Any of these resources should get you to a useable data set, it just depends on how you want to get there...

Meteorite Mania

Jonathan Fetterolf — Fri, 02 Dec 2022 02:34:23 +0000

Author: Jonathan Fetterolf

*If you want to know how I put this together, I step through my process from start to finish after the conclusions and limitations...* or view the juicy details here: **[Meteorite Mania GitHub Repository](https://github.com/fetterollie/meteorite-mania)**

Overview

This notebook analyzes a dataset of known meteorite landing sites and the data associated with those meteorites. I derived some statistics about the known meteorites and mapped their landing sites as well as looked into where the majority of meteorites fell. In terms of mass, the largest known meteorite is 60000kg, which is contrasted by the smallest that comes in at 1e-05kg. The mean mass of the dataset is approximately 15.6kg, while the median is approximately 0.029kg. Nearly 74% of known meteorite landing sites are in the southern hemisphere while 26% are in the northern hemisphere. This overwhelming amount of known meteorite landing sites can be explained by this information compiled by the Department of Earth and Planetary Sciences at the Washington University in St. Louis.

They state:

"Nearly all meteorites are found in deserts. (Yes, Antarctica is a desert because the annual precipitation rate is very low.) Deserts are places that accumulate meteorites over thousands of years and then nothing much happens to the meteorite. Also, meteorites are easier to find in deserts than in places with topography, vegetation, and other rocks."

Understanding & Cleaning Data

This comprehensive data set from The Meteoritical Society contains information on all of the known meteorite landings up to 2013. The Fusion Table is collected by Javier de la Torre. This data consists of 34,513 meteorites. This dataset was obtained here through data.gov.

Conclusions

Mass

In terms of mass, the largest known meteorite is 60000kg which is contrasted by the smallest that comes in at 1e-05kg.
The mean mass of the dataset is approximately 15.6kg.
The median is approximately 0.029kg.

Location

Approximately 74% of known meteorite landing sites are in the southern hemisphere.

Approximately 26% of known meteorite landing sites are in the northern hemisphere.

Limitations

The dataset only contains data up to 2013.
There are many known sites exactly on the equator. I did not include these in either northern or southern hemisphere.
Many sites in the southern hemisphere, specifically Antartica are close so the visual representation of the data leads you to believe there are more landing sites in the northern hemisphere when in actuality, it is the opposite conclusion.

Walkthrough

Imports

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
import geopandas as gpd

I used a data set with the geometry of landmasses to plot the shapes of land under my scatterplots.

countries = gpd.read_file(
               gpd.datasets.get_path("naturalearth_lowres"))

Importing data from file:

df = pd.read_csv('data/Meteorite_Landings.csv', index_col=0)

Cleaning & Filtering

I cleaned up the columns, dropped null values, filtered out a data point with an incorrect year, filtered out any entries with a mass of zero (what is a meteorite without mass...?), converted mass(g) to mass(kg) in a new column, and changed values in the fall column from 'Fell' to 'Not Found'.

newcols = [col.strip().lower().replace(' ','') for col in df.columns]
df.columns = newcols

df = df.dropna()

df = df[ df['year'] <= 2013]

df = df[ df['mass(g)'] > 0]

df['mass(kg)'] = df['mass(g)']/1000

key = {'Not Found' : 'Fell'}
replace_key = {v: k for k, v in key.items()}
df['fall'].replace(replace_key, inplace=True)

The Juicy Code

I calculated the maximum, minimum, mean, and median for the mass values.

largest_mass = df['mass(kg)'].max()
smallest_mass = df['mass(kg)'].min()
mean_mass = df['mass(kg)'].mean()
median_mass = df['mass(kg)'].median()

The Juiciest Code

I wanted to plot coordinates of the following subsets of known meteorite landing sites. The following code filters for each and subsequently plots a scatterplot over a world map:

Top 500 Largest by Mass (kg)

df_top_500_largest_strikes = df.sort_values(by='mass(kg)', ascending=False).head(500)

fig5, ax5 = plt.subplots(figsize=(20, 10))

countries.plot(color="lightgrey", ax=ax5)

sns.scatterplot(data=df_top_500_largest_strikes, x='reclong', y='reclat', hue='mass(kg)', size='mass(kg)', ax=ax5, palette='flare')

ax5.set_title('Landing Sites of 500 Largest Known Meteorites')
ax5.grid(visible=True, alpha=0.5)
ax5.set_xlabel('Longitude')
ax5.set_ylabel('Latitude')
ax5.legend(title='Mass (kg)')

Top 100 Largest by Mass (kg)

df_top_100_largest_strikes = df.sort_values(by='mass(kg)', ascending=False).head(100)

fig4, ax4 = plt.subplots(figsize=(20, 10))

countries.plot(color="lightgrey", ax=ax4)

sns.scatterplot(data=df_top_100_largest_strikes, x='reclong', y='reclat', hue='mass(kg)', size='mass(kg)', ax=ax4, palette='flare')

ax4.set_title('Landing Sites of 100 Largest Known Meteorites')
ax4.grid(visible=True, alpha=0.5)
ax4.set_xlabel('Longitude')
ax4.set_ylabel('Latitude')
ax4.legend(title='Mass (kg)')

Top 10 Largest by Mass (kg)

df_top_10_largest_strikes = df.sort_values(by='mass(kg)', ascending=False).head(10)

fig6, ax6 = plt.subplots(figsize=(20, 10))

countries.plot(color="lightgrey", ax=ax6)

sns.scatterplot(data=df_top_10_largest_strikes, x='reclong', y='reclat', hue='mass(kg)', size='mass(kg)', ax=ax6, palette='flare')

ax6.set_title('Landing Sites of 10 Largest Known Meteorites')
ax6.grid(visible=True, alpha=0.5)
ax6.set_xlabel('Longitude')
ax6.set_ylabel('Latitude')
ax6.legend(title='Mass (kg)')

I'm happy I wasn't living yet to see any of these largest known meteorite strikes.

Northern Hemisphere Known Sites

df_northern_hem = df[ df['reclat'] > 0]

fig1, ax1 = plt.subplots(figsize=(20, 10))

countries.plot(color="lightgrey", ax=ax1)

sns.scatterplot(data=df_northern_hem, x='reclong', y='reclat',\
                hue='fall', ax=ax1, palette='flare', alpha=0.4)

ax1.set_title('Northern Hemisphere Known Meteorite Landings')
ax1.grid(visible=True, alpha=0.5)
ax1.set_xlabel('Longitude')
ax1.set_ylabel('Latitude')
ax1.legend(title='')

Southern Hemisphere Known Sites

df_southern_hem = df[ df['reclat'] < 0]

fig1, ax1 = plt.subplots(figsize=(20, 10))

countries.plot(color="lightgrey", ax=ax1)

sns.scatterplot(data=df_southern_hem, x='reclong', y='reclat', hue='fall', ax=ax1, palette='flare', alpha=0.4)

ax1.set_title('Southern Hemisphere Known Meteorite Landings')
ax1.grid(visible=True, alpha=0.5)
ax1.set_xlabel('Longitude')
ax1.set_ylabel('Latitude')
ax1.legend(title='')

Find Out More

If you want to find out more about how I put this together (including seeing some fun exploratory visualizations) you can view the GitHub Repository for this project. Please feel free to drop a comment or reach out via LinkedIn.