DEV Community: Chris Baum

Flatiron Phase 5 Portfolio Project - BetterDad

Chris Baum — Sun, 04 Dec 2022 16:07:54 +0000

For my final Flatiron portfolio project I have created a web application called BetterDad.

Being a dad is extremely rewarding and I absolutely love my kids. However, it can come with challenges. The job of being a dad never ends, at many times it can be frustrating or even scary. Balancing work life, home life, and personal time can be difficult. These challenges can lead to times of sadness and even depression. BetterDad was created to help combat these feelings. It was inspired by a TED talk that I watched several years ago by Jane McGonigal. She was able to fight depression through incorporating aspects of a game into her life, making her life more goal oriented and provides motivation to continue to move forward through emotionally tough times.

The main requirements of the project were to have a React front-end that utilizes Redux for state management. The back-end would be a Ruby on Rails API that would handle data manipulation that would then be fed to the front-end.

I started with the back-end Rails application and the sign-up/login of the users from the front-end React application. Bcrypt was used to encrypt the password and authenticate. I also wanted to provide a way for the application to recognize a returning authorized browser client so that the user does not have to login every time they go the the url. This was done by having the backend assign an encrypted JWT token and saving it to the local storage of the browser client. Then whenever a request was made from the front-end to the back-end the token would be sent in the Authorization:Bearer header of the POST request. This was the first time setting this up, so it took a little bit of research and time to get it right.

In the Rails application, I also wanted several items to be updated on a daily basis automatically for the user: eight daily tasks were to be assigned and a daily quote that would be viewed by all users. This involved creating some code that could look at the created_at or updated_at time and comparing to current day time and re-assigning the tasks and quote.

When building the front-end I wanted to limit the number of routes that were being used to make the user interface feel more centralized to their profile. The minimum requirement was 3 routes, which I stuck to. The welcome page that is the base URL was one route. The welcome page had a button to the About page. This is extra information that a user wouldn't view often and are not a part of their specific user profile so I did break that out into its own page/route. Once logged in you are taken to the profile page which is the third route. I wanted to utilize the framework as much as possible to so I did research into React-Bootstrap components that were available. I found the Offcanvas component to be very useful to being able to add functionality and different information to a page without changing the route. This made the experience feel more seamless. I added the Signup form, history of completed tasks, favorited tasks, and user settings as offcanvas components that extend out from the side when the navigation button is clicked.

Using the react-bootstrap components was nice but I did run into some challenges with it. It can be a little unwieldy when you want to customize certain bootstrap components. I wanted my styling to stay with a particular color pallet, which was not standard to the bootstrap buttons. I researched how to do custom coloring for the buttons. However when I would initially put in new css styling, I would see the button change to what I wanted but when the page would re-render the styling would revert back. Looking into this issue, I found that others had this issue. The consensus for the easiest route to resolve the issue seemed to be to use inline styling with the buttons as bootstrap tends to default to that. Obviously this is not preferred because each Button component has to have the styling cut and pasted into it. For a much larger application it would probably be worth doing some more research to figure out if there is a way to use a CSS file to style a custom variant and then each button can just use the same variant to style it. For sake of time and since it is a smaller application I left it with inline styling as it isn't "wrong", but recognize it is likely not the preferred method in a larger application.

Since the Flatiron curriculum was developed several years ago I did do some research on the modern React framework and current preferences/standards. I found that class components, though still around and supported, are typically not what is preferred for new React applications. The push from React devs and the community is to use Hooks. Hooks are a functional component that can be utilized in a similar way to class components but have advantages:

Hooks are more lightweight. Below is an example of a Hook (left) versus a Class component (right):
Allows you to implement state in a much more elegant/simplified way.
Better separation of concerns.
Code to connect to Redux store and dispatch is more intuitive and simplified.

    const currentState = useSelector((state) => state)
    const dispatch = useDispatch()

Hooks utilize "return" rather than "render". This allows hooks to return values rather than only render a div, increasing the flexibility and usability of the Hooks if needed in your application.
There are many more advantages as well which you can find by googling "Hooks vs Class Components" if you wish to do more research.

Due to the many advantages, I did utilize Hooks rather than Class components in this application and believe strongly in them as they are quite a bit simpler and intuitive to use. React Class components feel very specific to React and are not very intuitive to connecting to a Redux store or dispatching. The Hook components feel more like a standard function/method that is used in javascript and ruby (again I would refer to the pictures above in the bullet points to show the Hook versus the Class component).

One more challenge that I ran into was implementing the use of Thunk, which was a requirement to utilize some asynchronous fetching. When I would update a user setting I have a message that is displayed to indicate if it was successful or not. When I implemented the async fetch using Thunk, I would render my component before the fetch was completed, so my redux state would not get updated in time to display my message. This took some time to resolve. The solution, since I was using Hooks, was to utilize UseEffect which allows you to perform side effects in functional components. In simple terms, we are telling React that the component needs to do something after the render. In my case I run the userSettingsMessage function after the render which forces the state to be set again and message to render properly.

useEffect(() => {userSettingsMessage(currentState.usersReducer.user[0].message)})

I very much enjoyed developing this project and proving that I am capable of handling the front-end as well as back-end development of a web application from start to finish! I'm also proud that I did the research utilizing Hooks and react-bootstrap as these were not covered in the standard classwork, which is more simulated like how a real software engineering job would be. A class is not going to be handed to me for everything and I am going to have to find the resources and figure it out myself.

I can see why a team of full-time developers is needed at web application companies. Through the process I continuously was finding different ways to do things and refactoring and can see how a version 1 of a project would be pushed out, functional, but not optimized or perfect. Then optimized and fixed over a significant amount of time. It felt like I could spend hours and hours refactoring and finding better ways to do things. At some point is time to say it is "good-enough" and push it to the world!

How to Use Javascript to Add Review Stars

Chris Baum — Fri, 05 Aug 2022 18:35:00 +0000

In this tutorial I will show you how to easily add review stars using Javascript to your web application and make them have dynamic functionality. Keep in mind that I had specific code in a web application that I reference. You may need to modify to suit your purposes.

The first step is to add the Font Awesome link to your HTML page.

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">

Here is the full HTML file with the Head with the link in it.

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="styles/index.css">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
    <title>Fish Store</title>
    <style>
        body {
            padding: 25px;
        }
    </style>
</head>

Next is to create the Javascript code. For this example we are using a 5 star review. Note that my code has a createDiv() function. This is simply creating a Div element with an assigned id for the web application structure. For this examples my star ids are newreviewstars11, newreviewstar21, newreviewstar31, newreviewstar41, and newreviewstar51. These stars will start with the className "fa fa-star". The "fafa-star" className when it is initially generated it will not be highlighted.

//create the div star element
newDiv = createDiv("newreviewstars", reviewId)
    form.appendChild(newDiv)

// for loop to generate the number of stars, in this instance 5.
    for (let k = 1; k < 6; k++){
      let starDiv = createDiv('fa fa-star', k)
      starDiv.setAttribute("id", `newreviewstar${k}${reviewId}`)
      element = newDiv.appendChild(starDiv)
    }

To get a star to fill in with color, the div class name needs to be changed to "fa fa-star checked" and the ".checked" reference needs to be added to the CSS file.

Here is example of how to change the className. You would have to generate the JS code to change the className based on your required functionality.

starDiv.className = 'fa fa-star checked'

This is what goes in the CSS file to fill the stars with the color orange once they are "checked".

.checked {
  color: orange;
}

In order to make the stars dynamic, a listener needs to be added to each of the star elements.

// creates a listener for selecting the number of stars when creating a new review
  function reviewStarsListener(newReviewId){
    //obtain stars element, which is displayed as an array
    for (let i = 1; i <= 5; i++){
      let star = document.getElementById(`newreviewstar${i}${newReviewId}`)
      star.addEventListener("click", function(){highlightStar(star, newReviewId)});
    }
  }

There are couple of things to consider regarding the functionality:

If a user clicks on star 4, then we expect stars 1 through 3 to also become checked
If a user selects 5 stars by accident, but meant 4 stars, then they need a way to be able to click on another star and it automatically adjusts.

My listener executes the highlightStar() function which performs the clearStars() function first and then highlights the stars, that way it can dynamically be changed.

// highlights additional stars when a high star value is selected.
// e.g. if star 4 is selected then this will highlight star 1 - 4.
  function highlightStar(starDiv, newReviewId){
    clearStars(newReviewId)
    starSelected = starDiv.id.replace("newreviewstar", "")
    numOfStars = starSelected.charAt(0)
    for (let i = 1; i <= numOfStars; i++){
      let starDiv = document.getElementById(`newreviewstar${i}${newReviewId}`)
      starDiv.className = 'fa fa-star checked'
    }
  }

// allow reviewer to dynamically be able to click on the stars to
  // change their # of stars review
  function clearStars(newReviewId){
    for (let i = 1; i <= 5; i++){
      let starDiv = document.getElementById(`newreviewstar${i}${newReviewId}`)
      starDiv.className = 'fa fa-star'
    }
  }

Hope this helps give some guidance when creating your dynamic web applications with reviews!

Portfolio Project 4: The Fish Store Using Javascript

Chris Baum — Fri, 05 Aug 2022 17:41:00 +0000

I have finally reached the end of the Javascript module! Had some significant life changes that caused this module to last much longer than I expected.

Those changes involved working 60 to 80 hour weeks for several months, being on call on the weekends, and then being on call overnight. The job had already been very demanding over the last couple of years significantly ramped up this year. This left very little time and energy to work on my class and also give my family the time they deserve. After working at that company for over 6 years, I decided to resign at the end of July 2022, as I was no longer finding joy in what I was doing and it was impeding my personal goals for my life and my career. So moving forward I am throwing myself into finishing my class and pursuing the career that I want, in Software Engineering.

For my Javascript Portfolio project, I created a web application called The Fish Store. When I was in high school I had multiple saltwater fish tanks (both 90 gallons) and enjoyed the hobby. I remember back then that the websites for fish information and stores were pretty rudimentary in design and functionality. Thinking back to this time, I thought it would be a great "improvement project" to create a proof of concept that would bring a better user experience to this community/hobby.

For this project we had to create a ruby on rails API that would run on a server (back-end) and the HTML/Javascript front-end application would pull from it. Due to this design, I wanted the web application to be as dynamic as possible because as the background API database grows, the web application would need to be dynamic enough to handle the growing database.

What I came up with is a tile/card design for each fish along with a filter feature to be able to easily sort for fish in a certain category. Many of the older websites that I used to use for this hobby had the fish divided by category, so keeping with the similar structure, which I think makes sense, I created a filter feature that would only display the tiles of the fish that belong to that category.

Another improvement I made over existing websites is that I use video to show the fish, rather than just a picture. Pictures can sometimes be misleading and having a video showing how the fish moves and looks, creates a better user experience. I do admit that if you have too many tiles/cards on a single page with too many videos could slow down the website and require it to use a lot of resources. What I imagine is that as this website would evolve, if it was in use, the developers would limit the number of tiles on a single page and allow a user to load only a specific number at time. Then there would be a "next page" element which would not physically direct you to a new page, but would load the next set of tiles asynchronously.

I kept the HTML file very minimal, with only the title and the "Call for ordering" being the only HTML elements generated by the HTML file. Everything else would be handled by the Javascript code.

I definitely did run into some challenges when creating the Javascript code to dynamically handle the tiles. I'm not upset at all about the challenges, because this project definitely helped reinforce my understanding of Javascript as it took a significant amount of debugging and re-coding to get everything to work correctly.

During the process of developing the application, I did have to take a step back. Originally I was using one Fetch call to get the JSON structure when the page was initially loaded and then I was hiding/showing things based on the user navigating the page. I realized that this lead to some cumbersome resource use and also made the coding more messy. One example being that all videos were being loaded and running, but would just be hidden in the background. I decided it was best to create and remove elements as the user navigated the application, which helped me keep my code cleaner.

One challenge in particular gave me a lot of issues. This challenge was the setup of the Review page for each fish:

Loads a new review form in the tile
Loads existing reviews in the tile
Allows selecting of your rating out of 5 stars
Asynchronously posts and displays a newly submitted review

Figuring out the 5 star review process was enjoyable as this was new and not something that had been covered in the class (see my post on this particular topic here. It took some research and thought to get this working correctly. I had to look up how to add a star and how to use CSS to fill the star. Then I had to create a loop to generate not one, but five stars, and a listener for each star. It also took thought from a functionality standpoint because if I click on the fourth star, then I would want stars one through three to also automatically fill in and also save the star count to the API database. Also if a user accidentally clicked five stars, but meant to put four stars, and caught this mistake before submitting the review, they would need a way to allow the user to adjust the stars. I was quite proud when I got this to display and function correctly. To get the stars working I had to do the following:

Located in HTML file, I need to add a link to Font Awesome.

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">

I used the Javascript code to create the stars.

for (let k = 1; k < 6; k++){
          let starDiv = createDiv('fa fa-star', k)
          element = starsDiv.appendChild(starDiv)
          if (k <= review.stars){
            starDiv.className = 'fa fa-star checked'
          }
      }

Setup the following in the CSS file.

.checked {
  color: orange;
}

The most challenging thing I ran into with the development of the Review page was doing the submission of the new review and having it immediately update the page with the added review so the user can see their contribution without having to reload the webpage. The initial setup itself was not difficult, but since each tile runs asynchronously from the others, I ran into bugs when I would:

Open the Review page on a fish, which I will call Fish #1.
Open the Review page on a second fish, which I will call Fish #2.
Submit a review on Fish #1. This created some data integrity issues because the variables/arguments would carry over from opening the Fish #2 review page (last page opened) into the Fish #1 review submission. It took some serious thought and step-by-step troubleshooting to debug the code and make the code re-load the correct target Fish variables/arguments when the user has the ability to manipulate all tiles independent of each other. That is where console.log() function helps considerably as it can be inserted to see variable/argument information and to verify process flow of the code.

I kept the functionality of the page focused on the user experience with Javascript and did not include an admin page or online ordering/cart functionality. We have done that in the past in other phases of the class and I'm sure I will be having full blown functionality in my final project.

I am very excited for the final Phase of the course and my final project! That said, the Javascript portion has been my favorite so far because of the challenges you run into with displaying correct data, the design aspect of Javascript, and the ability to create a very dynamic web application with it.

Portfolio Project 3: Net Worth Tracker

Chris Baum — Wed, 17 Nov 2021 02:37:38 +0000

For my Rails Portfolio Project I decided to create a net worth tracker so that users can keep track of their finances. I have an interest in Personal Finance and thought that this would be a great application to build in rails. The application would allow a user to enter accounts, both assets and debts, and automatically calculate their net worth. It would also allow a user to categorize each account and the application would track the total value of each category. This could be useful information for the owner of the application...especially if they were a provider of financial services.

There are similar applications that automatically link to accounts such as Personal Capital and Mint. However for the sake of learning and creating an application with Rails, this application will be based on manual entries by the user.

One of the first tasks I performed was breaking out which models my application would incorporate. I knew that I needed a User model that would save user information and login credentials. I knew my users would need to enter their financial accounts, with each account being either an asset or debt, so I would need an Account model with a type attribute. I also knew that I wanted to have the accounts categorized, but I wanted pre-defined categories that a user would have to choose from. I also wanted to be able to have a value associated with the category as this information may be valuable to the owner of the application. For these reasons I created a Category model.

One of the requirements that would end up shaping my project was the requirement that we have a many-to-many relationship. I first took a look at the Accounts model as it seemed to be the lowest level model. An account belong to a single user and a single category. So there were no has-many relationships associate with it. Now thinking about Users and Categories. We know a User has many Categories, but does a Category have many users? If the perspective is just from a single standard user, then no, but if the perspective is from an Admin or company that wants to track data and provide needed services to their users based on information from each Category, then yes a Category does have many users. This is also many times the purpose of an application and how a company monetizes an application, as an application like this can help with marketing and target sales/services. So my many-to-many relationship would be: Users have many Categories and Categories have many Users (note that this association would be through a join table which is discussed later on in this post).

I believe going through this thought process with these relationships was eye opening because originally my perspective was from that of a single user, but thinking about the many-to-many relationship led me to think about the perspective of the Admin and also the perspective of the company that may develop this application. Having the correct perspective can change how the application is setup and developed.

Another beneficial mental exercise was thinking about how to setup a join table. We are required to have two has-many-through relationships, therefore I would need a join table somewhere to allow the models to access each other. I could create multiple join tables to do this, but is there an easier way? After thinking about the relationships I focus on the Account model because an account belongs to a user and also belongs to a category, so shouldn't I be able to join a user and category together with an account? After all, I want the association to be there if a user actually has an account in a particular category. I ended up adding the foreign keys for both user and category to the Account model.

create_table "accounts", force: :cascade do |t|
    t.string "name"
    t.string "account_type"
    t.integer "dollar_value"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
    t.integer "user_id"
    t.integer "category_id"
  end

Through that join table I am able to access User.Categories and Category.Users. Once again going through the thought process of how the relationships work and should be setup was eye opening because it can be the difference between creating multiple tables and extra work versus finding a simpler solution.

Much of the rest of the standard application development went fairly smoothly. I focused on the "get it to work", even if it didn't look the prettiest, as I would later add in validations, add-in features (such as OmniAuth), and refactoring the code.

After getting most of the standard user application working I moved on to the Admin user experience. This did take some thought as I had to think how much access is appropriate for an admin. Do we want an admin to have access to all of the users' information? Do we want them to be able to view accounts only or also create accounts? For that question I did settle on an admin being able to manipulate and create accounts in case a user needs help, delete a user in case they have not been active for quite some time, but not be able to manipulate their username, email, or password since changing either one of those could lock the user out of their account. One important function I wanted to include for an admin was that of Category management. A standard user would have a selection of pre-existing categories. The admins would have to manage this list. The categories would be along the lines of Savings, Retirement, Real Estate, Credit Cards, Crypto, etc. The reason that may need to managed is if you think back just 10 years ago, Crypto would not be a category on this list and would have to be added. Also for simplification or clarifications you may end up breaking a category out. Possibly the category started out as Investments but there are a lot of accounts that fall under this since the category name is so broad. Later on, to get better information and categorization, we may delete Investments and instead create a Stocks, Bonds, REITs, etc. category.

Another benefit to having a standard experience and an admin experience was that it allowed me to meet another requirement. That requirement was that we use a class level scope method. Within the User model I was able to use the scope method for a standard user and admin user to quickly reference which they were within the code of my application.

scope :admin, -> { where(admin: true) }
  scope :standard, -> {where(admin: false)}

Something that did trip me up within the Admin experience was, what happens if I delete a category? What happens to the accounts associate with category since Account has a Category foreign key? We don't want the accounts to just disappear from a user, which would happen since I display the user's accounts by category in their show page. I also calculate net worth based on these accounts and their net worth would show a different number than if they added up the accounts they see. However, I also don't want to delete the accounts. This would cause confusion and frustration with a user since they put the time and effort into putting the account in, which typically is more work than just typing some letters and numbers on the Accounts/new page. Likely they had to track down account statements or do some calculations to get the account balance. What I settled on was to create an "Uncategorized" category and make it unable to be edited or deleted by the Admin. This would act as the catch all when I delete a category as it assigns all of the accounts associate with the deleted category to the "Uncategorized" category. These accounts would then still show up for a user and would just need the user to recategorize it by editing the category. Though still slightly annoying, not nearly as annoying as disappearing accounts and inaccurate net worth value.

Another interesting problem I had to solve was later on when including the OmniAuth feature as I didn't have much experience using it, the parameters it outputs, and how I would associate a Facebook user with a new User object. After all I wanted a Facebook user to be able to save their account information and bring it up every time they logged in. Therefore upon initial login they would have to be associate with a User object. I ended up having to add another attribute "UID" to the User model much farther into the project build than I would have liked. This attribute would default to "0" for a non-Facebook user but save the UID from a Facebook user. The sign-up/login sequence would then be different based on whether that value was "0" or non-zero. For the facebook user a random password would be generated using SecureRandom.urlsafe_base64 because we do not want this left blank as someone could still try a standard login with the username.

def omniauth_new_user
    @user = User.new
    @user.username = auth[:info][:name]
    @user.email = auth[:info][:email]
    @user.uid = auth[:uid]

    #assign random password for omniauth
    @user.password = SecureRandom.urlsafe_base64
    @user.password_confirmation = @user.password
  end

This also highlights how things can be done different ways. In retrospect, I could possible have a "type" attribute and assign "standard" for normal login, "facebook" for ominauth and only allow standard login for a type "standard" user. That is the nice thing about programming is that there are many solutions to a single problem and many improvements that can be made over time.

As a final thought, one thing that I struggled with was the idea that "this is a very simple application, is this good enough?" After some thought I do believe it is due to the added admin features that make it a more complete application with different relationships and a purpose (data mining to eventually target financial services). Something that all programmers need to remember is that applications usually start out simple and small and then get built out over time. Even Facebook originally did not have many features. Only over a significant amount of time did it continue to grow, hire more programmers, and cause development to accelerate. It is easy to see why companies have many full time software engineers/programmers. Though the application I created works, it is fairly simple and there are always new and better features that could be created. For instance, many real-world applications that do something similar allow you to directly link to your account so they auto-update daily or upon sign-in. Also, with so many users and accounts, data mining can mean profits for a company and just to get the data into a useful form requires the application to continue to be developed even if the standard user experience doesn't change much. For this class, I have to be satisfied with learning and creating a "simple" startup application.

Sinatra Portfolio Project: Bass & Bait Tracker

Chris Baum — Fri, 06 Aug 2021 01:19:06 +0000

For my second portfolio project I had to create a Sinatra application that would track something important to me. One of my favorite activities is bass fishing. When fishing for bass, sometimes certain baits work at different times. A lot of times you end up changing baits during the day until you hopefully find something that works. However, over time I couldn't tell you which baits work the most consistently and would give me the best chance, on average, to catch a fish. My memory just isn't that good. An application to keep track would definitely be helpful to track this information over time.

During the process of brainstorming for these first two portfolio projects, I have tried to focus on what applications actually provide value to my life. Typically if it adds value to my life, then it may add value to someone else's. As a developer the main goal is to add value to a web application for your company, users, customers, and society. Practicing that during school can help get in the mindset of doing that. I also think people learn better when there is value behind what they are doing. If you don't believe in what you are doing then retention can be more difficult.

One thing that was difficult in the beginning was setting limits on what this project was to do. When doing object oriented programming and trying breaking things down into models of real objects, the scope of the project can grow very quickly. For instance, with the fishing application I created, you have a user who owns bait. With each bait you then have catches that belong to that bait, and each catch has a fish associated with it. That is what I built my basic application with. However, you can also have location of the catch, weather, bait manufacturer, boat type, fishing line type and manufacturer, fishing pole type and manufacturer, fishing reel type and manufacturer, etc etc etc. The list just goes on and on with the objects that you can have, relationships you can make, and data mining that you can do. All of this information can add value for the user. For instance, you could keep data on what weather is best to fish for, which bait manufacturer is the best overall, or how many fish have been caught in a particular location. But along with the information comes more and more pages to create to display that information which means more and more time to develop. It really puts in perspective how real world applications start small and then continuously need developers to add content, features, and support. Even some of the biggest websites in the world such as Amazon and Facebook still are constantly being developed because there are so many possible features, content, and data that adds value to people's lives but it takes significant time and brainpower to create it.

In the end, I decided that I would focus on a user being able to view their baits, see how many catches they have with each bait, and then view their catches. The catches show information about the fish and which bait it was caught on. That kept things simple enough to start with and would allow for a solid foundation to build on if this were a real web application that would continue to be developed.

Another thing that was interesting during this project was going through the process building it out in separate layers. I was able to identify four major layers:

Object Creation, Manipulation and Basic View Navigation - First everything had to be functioning from a data/object creation and navigation standpoint.
Editing and Deleting of Objects
Validation of Information - Modify what you did to include validations for the information that a user must enter. (This one was a little bit tricky for my application at one point because when I setup creating a catch, I wanted to be able to select an existing bait or create a new bait when creating the catch. This lead to having to verify check boxes and text inputs for too much or too little information).
Security - Ensure that information that belongs to a specific user is protected from others and that a user must be logged in to navigate to the correct areas.

I didn't really go into the application with the four layers fully defined in my head, but as I worked through the project, the four layers became apparent. I think I had some overlap between #3 and #r where I bounced back and forth between working on them. I think being more conscious about this four layer approach will help with development time in the future if I can stick to one layer at a time.

CLI Project: Hard Lesson Learned Between Lessons and Real-Life Websites

Chris Baum — Fri, 06 Aug 2021 00:40:36 +0000

Originally my CLI Portfolio Project started out rough. My initial idea for the project was to scrape the Megamillions and Powerball website to allow a user to view most recent numbers, past numbers, and if the information was available far enough back, to show most common numbers (even though you would think they would be distributed evenly, but maybe not!). I don’t play the lottery often but when I do, I do not particularly like going to multiple pages, websites to see the winning numbers, especially if they were past numbers.

I was pretty comfortable with the Open-URI and Nokogiri lessons and knew that many times it was just trial and error for finding the information, so I began down that path. As I began looking at the Powerball and Mega Millions websites I began to realize that the lessons do not always translate to reality. Inspecting the HTML code, was fairly easy, but ran into two problems:

Even though the winning numbers were there, they were buried deep within different div’s, ID’s, and classes.
The past numbers did not have specific webpages. You could change the date range and output data without the URL changing.
I decided to start with #1 and just trying to obtain the winning numbers. As I tried multiple .css searches, nothing was coming up specifically with the numbers, even though I was getting to locations that looked like they should be there. I continued to increase the scope of my .css search, copying the results into a word doc and searching for the winning numbers via Find (Command + F). No matter what I did, I could not find the numbers in the output HTML file, even though I could see them when I inspected the website.

As I dug deeper, it became apparent that this website had a much more advanced setup that I had seen up to this point. There were references to some javascript code and databases. From what I could gather it appears they were possibly injecting the number from a database into the website’s HTML, and Open-URI/Nokogiri was unable to obtain the values. I spent quite a bit of time to come to this conclusion as this was my first project using Open-URI/Nokogiri.

I did find that I could get the values if I used yahoo search engine, but I could only get the most recent as they would put the most recent at the top of the search results page. However, I did not think this warranted a good enough project to just be able to pull the most recent numbers.

After banging my head against this problem for a while, I decided to shift gears and come up with a different topic for my project. I had found an interesting Wikipedia website that shows the anniversaries of notable historical events on any particular day (https://en.wikipedia.org/wiki/Wikipedia:Selected_anniversaries). At my current job in a manufacturing facility we have TV screens that show facts or famous quotes on the screens that I always find interesting. I thought that this would be a perfect application for a CLI project in which I could see the historical events of a particular day.

Knowing that we want the project to have some complexity to it and allow the user to have at least one level of interaction I decided that it would be best to have two options. One option would be to see today’s events and another option would be to select select a month and then a day.

Wanting this to be an object-oriented program I broke this down into several different objects:

CLI Interface
Month
Scraper
Event
I debated making a Day object, but really the day is only an input coming from the user to direct them to the correct website. There weren’t really any other properties that required storing for a particular day. On the other hand the Month did have some properties that I wanted to store. I wanted to be able to reference the month by number (e.g. January = 1, February = 2, etc.) and I also wanted to have the number of days in the month. Storing the number of days served two purposes:

Logically we would not want a user to be looking for a number outside of the actual number of days in the month.
The websites I am parsing have the format of https://en.wikipedia.org/wiki/Wikipedia:Selected_anniversaries/#{@month}#{@day}. So I would reach an unavailable website.
Either way I would want to be able to verify the input, provide a message if it was out of bounds, and re-run the initial prompt so the user can try again.

After initial setup of my programs, I got to the part where I would need to obtain the HTML and search it for the needed information. Luckily Wikipedia is not as complex as the Mega Millions and Powerball websites and after some trial and error using Pry, I was able to obtain the correct information for a single page and turn that into a hash. Unfortunately after getting the first page to work, I ran into several issues due to the fact that my program is actually pulling data from 366 web pages (365 days + February 29th = 366 days):

Despite similar formatting, each page had information prior to the list of “eligible events” that had been confirmed. This information varied in size, so I could not just look in the exact same spot on every page.
Each day has a different number of events and I wanted to list all of the events that Wikipedia had.
Not all events are in chronological order even though the initial one of two I looked at did. From a user interface standpoint I wanted to always have my events in chronological order so the application looks more polished.
To handle problems 1 and 2 I had to make my code more dynamic. I was able to do this by using a While loop and also verifying the :year key value. I noticed that all of my correct events had an actual year entry that was correct. The hash entries that were not correct had a string with letters. I created a method to verify if the year has any non numerical characters in it so that I could eliminate the inaccurate entries.

def year_is_integer?(string)
string.to_i.to_s == string
end
My .css code returned an array, which I could use a While loop to search through incrementing the “I” method variable every time after verifying the data.

while @doc.css('div.mw-parser-output ul li')[i] != nil

Once all of the data was looked through the .css code would return a nil, which would get me out of the while loop. The combination of the While loop and verifying if the year key value is an integer allowed me to correctly grab all information. This just shows the importance of building dynamic code and working to improve your natural implementation of dynamic code.

To handle problem 3, which was that Wikipedia, though it seems they tried for the most part, still had entries that were not chronologically in order. I wanted my program to not just reflect Wikipedia, but improve the experience by having it in chronological order every time. In the Event class, I created a “sorted” method which re-sorts the class variable @@all after each new Event entry is initialized after it is put into the @@all class variable. This helps to always keep the events in order.

Two more decisions I made that I think helped speed up the program: Read only the page that is asked for, rather than just automatically reading all 366 wikipedia pages and then pulling the events from the @@all class variable. Reading the 366 pages would just be a waste of time/resources in this instance. After the output of the events I clear the Events.all array. There is no reason after the output to the interface to continue to store the events since the operation is always: get date from user, read webpage, output events.

Overall even though it was not the original direction I had for this project, I was very happy with the results. People like to see what events happened on certain days such as their birthday or just learn about random history. I thought I had some features such as the chronological order that directly improve the user experience over just looking at Wikipedia. I think some of the decisions I made from a programming standpoint were intentional in making it work quickly and not spend unnecessary time creating and/or storing unnecessary objects that did not add value to the program.

I look forward to the next phase/module of this class!