DEV Community: Joshua Mayhew

Script, Scrape, Reflect: Automating My Code Journey

Joshua Mayhew — Tue, 17 Oct 2023 16:41:21 +0000

In today's tech world, chronicling one's journey can be as enlightening as the code we write. If you're like me, you might be striving for consistent growth and ways to effectively track your coding progress. My solution? Scripting an automated process to archive my Codewars katas for personal reflection and evolution.

The Motivation Behind Archiving

Coding on platforms like Codewars is more than just solving problems—it's a chronicle of our growth, understanding, and the evolution of our problem-solving abilities. But with the vast number of challenges available, it can sometimes feel like our milestones are becoming mere data points lost in the vast sea of challenges.

For many developers, platforms like GitHub are not just code repositories but are resumes, reflections, and records of our daily commitment to the craft. As such, I identified a twofold objective:

Consolidation for Reflection: Having all my katas in one place for personal reference and review.
Visibility of Growth: Showcasing daily coding efforts as consistent GitHub activity, ensuring every problem solved is a step visibly recorded in my coding chronicle.

A Respectful Approach to Automation

Before diving into the details, it's crucial to emphasize my respect for platform rules. Codewars has valid reasons for keeping solutions private, and I wholly align with those principles. The primary objective of this automation is to archive solutions privately for my personal reference and reflection, not to publicize or share them. These solutions are committed to a private GitHub repo, accessible only by me, thereby preserving the integrity of the Codewars community.

How Other Developers Can Benefit

The method is straightforward, and the benefits are manifold:

Web Scraping for Data Extraction: Leveraging web scraping libraries, I programmatically navigate through my Codewars profile. Essential kata details like name, difficulty, my solution, and the date of resolution are extracted.
Clean Structuring: Post-extraction, the katas are formatted consistently. This ensures that when I or any developer looks back, the information is easily digestible.
Automated GitHub Activity: The daily efforts are reflected as GitHub commits in a private repository, emphasizing a daily coding habit. It's like a digital diary, only more technical.
Scheduled Automation: For the true spirit of automation, I've scheduled the script to run at set intervals, ensuring my GitHub activity stays updated without manual intervention.

To fellow coders and tech enthusiasts, this approach is more than just a technical exercise. It's a testament to our journey, a personal commitment to growth, and a nod to the ever-evolving landscape of tech. By archiving our milestones, we're not just storing code; we're storing memories, lessons, and hours of dedicated effort.

Whether you decide to adopt a similar approach or have your unique way of chronicling your journey, the essence lies in consistent growth and reflection. Here's to many more lines of code and milestones in our shared journey of development!

Understanding the CodewarsKataScraper Automation

This script offers a streamlined solution for automating the extraction of solved katas from Codewars, storing them, and subsequently pushing them to a private GitHub repository.

1. Initial Setup and Constants

The constants defined at the top of the class give a structure to the program:

API_ENDPOINT: The endpoint to fetch completed challenges (kata).
LOCAL_REPO_PATH: The path to your local repository where the solutions are stored.
GITHUB_REPO: The GitHub repository reference.
STORED_KATAS_PATH: The path to the JSON file which stores all completed katas.

2. Initializing and Preparing the Scraper

The initialize method sets up necessary configurations and loads environment variables required for logging into Codewars. Within it, the initialize_driver method ensures that the web driver for Selenium is properly set up, as Selenium is pivotal for the web scraping process.

def initialize
  # Configuration setup and environment variable loading
  @api_endpoint = 'API_ENDPOINT_URL'
  @local_repo_path = 'YOUR_LOCAL_PATH'
  # ... Other constants

  initialize_driver  # Setting up the Selenium web driver
end

def initialize_driver
  # Assuming you're using the Selenium Webdriver with Chrome
  options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--headless')  # For headless browsing
  @driver = Selenium::WebDriver.for :chrome, options: options
end

3. Running the Scraper

The run method serves as the main driver function:

It starts by logging into Codewars using login_to_codewars.
Fetches completed katas via fetch_completed_katas.
Compares them to existing katas to find any new ones.
For new katas, it views and scrapes solutions.
Stores the new katas to the local storage.
Finally, commits and pushes updates to the GitHub repository.

def run
  login_to_codewars

  katas = fetch_completed_katas
  new_katas = katas - fetch_existing_kata

  if new_katas.any?
    new_katas.each do |kata|
      scrape_kata_solutions(kata)
    end

    store_new_kata(new_katas)
    commit_and_push_to_git
  else
    puts "No new katas found."
  end
end

4. Web Interaction using Selenium

Selenium is an incredibly powerful tool for controlling a web browser through programs and performing browser automation. It's functional for all browsers, works on all major operating systems, and is available in various languages, including Python, Java, and Ruby. In the context of this script, here's why Selenium plays a pivotal role:

Dynamic Content Loading: Many modern websites, including Codewars, use AJAX and other techniques to load content dynamically. Selenium can interact with dynamic content, ensuring all necessary data is loaded and accessible.
Interactivity: Beyond just scraping static content, there might be a need to interact with page elements, like clicking on buttons or navigating through paginated content. Selenium provides the functionality to simulate these user interactions.
User Simulation: By mimicking a real user's interactions with a website, Selenium can navigate through login screens, pop-ups, and other elements that might be challenging for more basic scraping tools.
Consistency Across Browsers: While this script may be designed for a specific browser, Selenium's cross-browser compatibility ensures that adaptations for other browsers can be made with minimal adjustments.

For the Codewars kata scraping task, the ability to programmatically log in, navigate, and interact with the platform's interface is crucial, and Selenium offers the most reliable and efficient means to achieve this.

Several methods interact with web pages, like logging into Codewars and viewing kata solutions:

 def login_to_codewars
    @driver.get('https://www.codewars.com/users/sign_in')
    email_input = @driver.find_element(id: 'user_email')
    password_input = @driver.find_element(id: 'user_password')
    sign_in_button = @driver.find_element(class: 'btn')

    email_input.send_keys(@codewars_username)
    password_input.send_keys(@codewars_password)
    sign_in_button.click

    # Wait for page to load completely
    wait = Selenium::WebDriver::Wait.new(timeout: @timeout)
    wait.until { @driver.execute_script('return document.readyState') == 'complete' }
  end

def view_kata_solutions(kata_data)
    kata_id = kata_data['id']
    languages = kata_data['completedLanguages']

    languages.each do |language|
      solutions_url = "https://www.codewars.com/kata/#{kata_id}/solutions/#{language}/me"
      @driver.get(solutions_url)

      begin
        # Wait for a critical element to be visible, confirming page load
        wait = Selenium::WebDriver::Wait.new(timeout: @timeout)
        selector = '#shell_content'
        wait.until { @driver.find_element(css: selector).displayed? }
      rescue Selenium::WebDriver::Error::TimeoutError
        puts 'Timed out waiting for code solutions container to load. Skipping this language.'
        next
      end
    end
  end

5. Data Extraction

The scrape_kata_solutions method is responsible for extracting the solution code and description for each completed kata. It navigates to each kata's solution page and fetches the required data:

def scrape_kata_solutions(kata_data)
  solutions = []

  # Iterate through completed languages for the kata
  kata_data[:completed_languages].each do |language|
    # Construct the URL for the kata solution page
    solution_url = "https://www.codewars.com/kata/#{kata_data[:kata_id]}/solutions/#{language}/me"

    # Navigate to the solution page
    @driver.get(solution_url)

    # Wait for the page to load (you may need to implement this)
    wait_for_page_to_load

    # Scrape code content and description
    code_content = @driver.find_element(:css, 'div.code').text
    description_content = @driver.find_element(:css, 'div.description').text

    # Store the scraped data
    solutions << {
      language: language,
      code: code_content,
      description: description_content
    }
  end

  solutions
end

6. Storing and Handling Data

There are helper methods to interact with local storage, like reading existing kata data or writing new data:

def fetch_existing_kata
  # Assuming data is stored in a JSON file
  JSON.parse(File.read(@stored_katas_path))
end

def store_new_kata(katas)
  # Write katas to the JSON file
  File.open(@stored_katas_path, 'w') do |file|
    file.write(JSON.generate(katas))
  end
end

7. Git Operations

The last step of the process involves Git operations to update your local repository and then push the changes to GitHub:

def commit_and_push_to_git
  Dir.chdir(@local_repo_path) do
    `git add .`
    `git commit -m "Added new katas"`
    `git push origin master`
  end
end

8. Execution

Finally, an instance of the scraper class is created, and the run method is called, setting the entire process in motion:

scraper = CodewarsKataScraper.new
scraper.run

9. Automating Script Execution with Cron

Once the script is set up and ready, it's essential to have it run autonomously at specified intervals. This ensures that our archival process remains consistent without manual intervention. One of the tried and true methods for task automation in Unix-like systems is using Cron.

Setting Up a Cron Job

A. Access the Cron Table: Open your terminal and type:

crontab -e

B. Add the Job: At the end of the file, specify when you want the task to run and what command to execute. For instance, to run the script daily at noon, you'd add:

0 12 * * * /usr/bin/ruby /path/to/script.rb

Schedule Breakdown:

Minute: 0 (Exact minute)
Hour: 12 (Noon)
Day of Month: * (Any day)
Month: * (Any month)
Day of Week: * (Any day)

C. Save and Close: Depending on the editor you're using, the process will vary. If using nano, for instance, save with CTRL + O and exit with CTRL + X.

D. Verify the Job: To confirm that the job has been set up correctly, type:

crontab -l

Important Notes

Cron uses the system's timezone. If you're working across different time zones, make sure to adjust your cron timing accordingly.
Ensure that your script has the necessary permissions to execute. You might need to use chmod +x yourscript.rb to grant it executable permissions.
Test your Cron job to ensure it runs successfully and that there aren't any environment-specific issues that might prevent the script from running.

By setting up this Cron job, you've ensured that your Codewars kata archival process will run automatically, keeping your GitHub repository up to date with your latest achievements.

Concluding Thoughts

Integrating this kind of automation doesn't only aid in tracking personal progress but also showcases an adept use of diverse tools and libraries. Remember, every line of code you write is a testament to your dedication and growth. By archiving and reflecting, you're paving the way for future achievements. Happy coding!

If you're interested in trying out the script, visit the repo on Github: https://github.com/jmayheww/code-wars-ruby-scraping-script

Swiping with Stripe: Simple and Secure Online Payment Handling Powered By Stripe

Joshua Mayhew — Fri, 07 Jul 2023 15:51:15 +0000

Application frameworks offer developers at least one feasible avenue of safeguarding users and maintaining the integrity of their data in a digital landscape increasingly fraught with malicious actors and data breaches. Powerful and robust built-in functionalities, like the bcrypt gem in Rails, are an effective deterrent against hacks, ensuring that user account credentials are rigorously encrypted and computationally expensive to sift through.

And, while this is definitely a worthwhile line of defense, it should not and cannot be the only measure of security that a developer relies on in protecting their applications' data. Simply, a reduction in risk is not an elimination of risk. Therefore, it is deeply important to consider more broadly the implications and justifications of storing and associating sensitive user data on a database.

Where and how data is stored is an integral part of protecting it against attack. Why concentrate sensitive data in a single location when it is unnecessary to do so? Instead, sensitive data can be further dispersed and diversified across multiple secure locations, from where it can be subsequently queried and referenced by an application only when necessary and essential to do so. This "offshoring" of important data creates additional complexity and ensures that a breach in one location does not invariably translate to a breach of all data.

An obvious and very prevalent example of this delegation of data storage and security can be seen in the use of online payment processing platforms like Stripe. Serving, perhaps arguably, as the most reputable and foremost purveyor of online financial services, Stripe offers developers a very well-established and credible place to safely offload the risk and responsibility of managing and protecting what may be the most sensitive and frequently targeted types of data: user payment information.

Of course, any application that prompts a user for their payment details must grapple with the inherent risks entailed by creating, saving, and processing this information. In my own application, Infinite Eats, I immediately saw the glaring risk that housing this type of data poses to both myself and the user. Professionally and legally, the fallout of a data breach could be catastrophic. Hence, I found myself looking at Stripe as a well-trusted means of handling user payment data for me.

Notably, Stripe offers developers extraordinary degrees of access to its API, which is both highly functional and extremely well documented. The intuitive ease of working with and learning how to effectively integrate and leverage Stripe's API infrastructure in my project was a massive benefit to me. By delegating the overall task of payment processing to Stripe, developers are able to both simplify the task of handling user payments and focus their attention to the core functionality of their application.

Specifically, Stripe streamlines payment processing via its expansive suite of client libraries and SDKs (Software Development Kits), which are basically pre-built components that developers can use and plug into their applications without having to code everything on their own. Aside from providing developers with these useful building blocks, Stripe also gives applications the benefit of scalability, handling high volumes of transactions without compromising performance or security.

So, knowing all of this, and having chosen to integrate aspects of the Stripe API into my own project in fulfillment of my Phase 5 capstone at Flatiron School, how did I actually utilize the Stripe API?

Project Walkthrough

The following is a very basic example implementation of Stripe in a full stack React/Rails project. This walkthrough will cover the creation of payment methods and their being saved by Stripe for future reference.

1. Stripe API setup and customer creation

Before making any requests to Stripe or importing any of their SDKs, you must first create an account with Stripe and access your API keys from the developer dashboard. The API keys serve as the authentication mechanism by which the application can communicate with the Stripe servers. There is a public and private key. Importantly, I saved my keys in two different .env files (the public being saved in a .env file located in my client directory and both API keys also being stored in a .env file located in my Rails app root directory) and excluded these files from version control, to prevent uninvited access to this data.

Secondly, payment methods can only be saved and associated with users who also have stripe customer accounts, so I decided to automatically enroll all Infinite Eats users with Stripe to ensure that their payment information would be correctly registered and associated with a unique customer user id. To do this, I added some extra Stripe specific functionality to the Create Action in my Users Controller:


  def create
    new_user = User.create!(user_params)

    # Set your secret key. Remember to switch to your live secret key in production!
    # See your keys here: https://dashboard.stripe.com/account/apikeys
    Stripe.api_key = Rails.application.credentials.dig(:stripe, :secret_key)

    # Creates a new Stripe customer on signup
    customer = Stripe::Customer.create({ email: new_user.email })

    # Update the user with their new Stripe customer ID
    new_user.update!(stripe_customer_id: customer.id)

    session[:user_id] = new_user.id
    render json: new_user, status: :created
  end

Thus, new users will be simultaneously created in my Rails db following signup and registered as customers via the Stripe specific syntax that I found in Stripe's official documentation. Importantly, the user should have their stripe_customer_id attribute updated in the Rails db before rendering the user object as json for frontend access.

2. Integration

Next, I installed the @stripe/react-stripe-js library via npm. This library is what grants developers access to the pre-built React component and hooks that streamline user payment handling on the frontend. I relied on the Elements component, which is a container for Stripe components - the pre-built UI components that ensure the secure collection of user payment details.

In order to import and utilize Elements, I had to wrap it around any parts of my application that would be engaged in payment method collection. This meant wrapping any necessary context files in my index.js file..

(showing relevant portions only)


import { Elements } from "@stripe/react-stripe-js";
import { loadStripe } from "@stripe/stripe-js";


 <Elements stripe={stripePromise}>
                <PaymentProvider>
                  <CheckoutProvider>
                    <FridgeProvider>
                      <App />
                    </FridgeProvider>
                  </CheckoutProvider>
                </PaymentProvider>
              </Elements>

as well as wrapping my primary payment method form component in App.js..

import { Elements } from "@stripe/react-stripe-js";
import { loadStripe } from "@stripe/stripe-js";

<Route
            exact
            path="/myaccount"
            element={
              user ? (
                <Elements stripe={stripePromise}>
                  <PaymentProvider>
                    <AsyncMyAccountPage />
                  </PaymentProvider>
                </Elements>
              ) : (
                <Navigate to="/home" replace />
              )
            }
          />

As you'll likely see, I am passing props to Elements in the form of a promise, which is where the public API key comes into play. I declared this promise twice in both index.js and App.js as follows:

const stripePromise = loadStripe(process.env.REACT_APP_STRIPE_PUBLIC_KEY || "");

Doing so provides access to the CardElement component, which comprises a pre-built credit card input field. This component is essential for streamlining and safeguarding payment method collection on the frontend, as it securely captures sensitive user payment details from the frontend while also providing a smooth and professional user experience.

I chose to render this CardElement component within the larger context of a component I named PaymentMethodForm.js. To service the creation and saving of user payment methods to Stripe, I had to rely on some added functionality that I coded both in a payment.js context file and called in my PaymentMethodForm.js component's handleSubmit function. I chose to target payment method creation and saving as two separate async functions

3. Method Creation:

const createPaymentMethod = (cardElement) => {
    if (!stripe || !elements) return Promise.reject("Stripe is not available");

    setLoading(true);

    return stripe
      .createPaymentMethod({
        type: "card",
        card: cardElement,
      })
      .then((result) => {
        setLoading(false);

        if (result.error) {
          throw result.error;
        }

        if (!result.paymentMethod) {
          throw new Error("No paymentMethod returned from Stripe");
        }

        return result.paymentMethod;
      })
      .catch((error) => {
        setLoading(false);
        throw error;
      });
  };

In createPaymentMethod, I am creating a payment method object via the Stripe API. The very first thing is that this function is accepting a cardElement param, which represents all the card information entered by the user. Secondly, having already nested my relevant payment method components in the Stripe Elements container, I am given access to new Stripe provided 'stripe' and 'elements' objects. If either of these are missing, the function will return a rejected promise from Stripe. This check is important in that it confirms that the Stripe API and Elements components are properly loaded and accessible before proceeding to payment method creation.

Next, the function calls stipe.createPaymentMethod, which passes an object with payment method necessarily listed as 'card,' and the cardElement param. This method being called is what triggers the Stripe API to create a payment method based on the given user payment details represented in cardElement. The outcome of this method is a promise, which will either resolve successfully or unsuccessfully. If unsuccessful, the resulting object will be an error message, but if everything is correct, it will return a paymentMethod object that can then be saved via the following function.

4. Assigning and Saving the Payment Method:


  const handleSavePaymentMethod = async (paymentMethod) => {
    setLoading(true);
    setError(null);
    try {
      const response = await fetch("api/users/save_payment_method", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ pm_id: paymentMethod.id }),
      });

      if (!response.ok) {
        const errorData = await response.json();
        throw new Error(errorData.message);
      }
      setUser({ ...user, payment_method_id: paymentMethod.id });
      setLoading(false);
    } catch (error) {
      setLoading(false);
      throw error;
    }
  };

Taking that successfully returned paymentMethod obj, I am able to subsequently pass it to the handleSavePaymentMethod() as an argument and pass its id in the body of my POST request to a custom save_payment_method endpoint in my Users controller. This controller action appears thusly:


  def save_payment_method
    pm_id = params.require(:pm_id)
    customer_id = @current_user.stripe_customer_id

    Stripe.api_key = Rails.application.credentials.dig(:stripe, :secret_key)

    # Attach the PaymentMethod to the customer
    Stripe::PaymentMethod.attach(
      pm_id,
      { customer: customer_id }
    )

    # Set it as the default payment method
    Stripe::Customer.update(
      customer_id,
      { invoice_settings: { default_payment_method: pm_id } }
    )

    @current_user.update(payment_method_id: pm_id)

    render json: @current_user, status: :ok
  end

Here, I assign the pm_id variable to value of the passed paymentMethod.id params. I also make sure to access the current user's (who is authorized via other backend logic beforehand) stripe customer id. Again, the Stripe secret API is necessary and should be accessed from the Rails .env file in order to authorize the requests to the Stripe API that set and update the payment method. The following lines are what communicate with Stripe. Stripe::PaymentMethod.attach is self-explanatory; it attaches the payment method to the customer and identifies each by their respective ids.

*This is the beautiful delegation of payment security and management facilitated by Stripe -- Stripe queries customers and their associated payment methods by their provided ids and thereby frees the Rails db from having to store any of the actual payment details themselves. Of course, API keys could be leaked, etc etc, but being able to offload and delegate secure data storage to Stripe while only needing to store relevant corresponding ids in the application db greatly improves the overall security of an application and of sensitive user data. *

Next, the Stripe::Customer.update is responsible for designating the provided payment method as the default payment method (which is necessary to facilitate more than a single transaction). Finally, the current user so that its payment_method_id is assigned and thus associates the user with the saved payment method. The current user is then returned as a json object where the handleSavePaymentMethod() can conditionally handle errors or update the user state on the frontend so that the newly saved payment method id value is accessible in user state.

As demonstrated above, integrating the services of third-party payment platforms like Stripe into our applications is an effective and worthwhile approach to achieving secure and efficient handling of sensitive user data. Stripe's API tools significantly simplified the process of payment handling in my project, while simultaneously ensuring that my user data and payment handling is scalable, efficient, and secure. And, just as important, Stripe integration allowed me to focus my attention on the fundamental concerns and objectives of my application without suffering the distractions of securing private data. Ultimately, as the digital landscape continues to evolve, and new challenges continue to emerge, it is all the more imperative that we likewise evolve and adapt, learning to effectively utilize whichever available tools and services, such as the Stripe API, that allow us to ensure the highest possible industry standards of data security and user experience in our applications.

Image credit: https://news.crunchbase.com/startups/fintech-startups-look-to-kill-swipe-fees-on-behalf-of-small-business-owners/

Securing the Rails: User Authentication and Bcrypt in Ruby on Rails

Joshua Mayhew — Thu, 13 Apr 2023 10:43:05 +0000

Knowing how to properly safeguard valuable and sensitive data against malicious actors is an especially critical aspect of web development. In an ever-changing, perpetually evolving digital landscape, threats to security and privacy are legion. Thus, in order to more meaningfully anticipate and proactively neutralize these threats, and consequently ensure the safety and privacy of our data, it is essential that we familiarize ourselves with core security practices and tools that can be implemented in our code.

During Phase 2 at Flatiron School, I attempted to cobble together a crude semblance of login security and user authentication with state in React.js. I sought to toggle between states of showLogin (true or false) to conditionally render a login form on page load and thereby create an illusion of user authentication by allowing users to input data and seemingly persist it via form submission. However, in actuality, this data was never being persisted or stored in any manner by which it could be cross-referenced or utilized for authentication purposes. Instead, login merely resulted in a single user object being created in my JSON server, the presence of which (or lack thereof) would toggle a change in the showLogin state and prompt rendering of either a login form or a home page.

Moreover, I directed logout to delete the user object, so that only one user object ever existed on the JSON server at any time. So, even though my logged user might have a user object containing their inputted username and password details, none of this mattered because it was not being used for authentication. Rather, I was only relying on the presence of a single user object to update state in my app and mimic login, which meant that my Phase 2 project only allowed for a single logged user at a time and, even worse, accepted any and every conceivable user input for username and password fields. At that time, I was not versed in validations and had only the faintest notion of what real user authentication entailed.

Thankfully, by contrast, Phase 4 has imparted a much greater and more robust approach to implementing true user authentication in my apps. For this phase, I was tasked with coding out a full-stack app (utilizing React.js and Ruby on Rails) that satisfies a number of different criteria. And, of course, user authentication was included as a core requirement. Unlike my Phase 2 project, though, I have been able to access newfound knowledge and tools to properly implement user authentication in my Phase 4 app, Book-It.

Securing data and handling model validations, error messages, and db persistence is far easier when working with a fully featured backend framework like Ruby on Rails. Specifically, its pre-built features and conventions make building, maintaining, and scaling web applications with Rails a more streamlined and manageable process, while simultaneously ensuring data integrity and security throughout. For me, this means being able to safely and securely exercise far greater control and precision over data and how its used and accessed.

Additionally, Rails has access to gem dependencies like Bcrypt that help to create impressively strong foundations for building secure systems. The Bcrypt gem describes its purpose and functionality thusly:

bcrypt() is a hashing algorithm... [that] take[s] a chunk of data (e.g., your user's password) and create[s] a "digital fingerprint," or hash, of it.

On its own, Rails could store user passwords as attributes in a user object. Of course, this is tremendously insecure, as every user password would therefore be available in a database unprotected and unencrypted. This is where Bcrypt comes in, hashing passwords. As opposed to storing passwords in plain text, Brcypt lets us store passwords as hashed versions of the original input.

This hashed version is tied to the original plain text input in that "a hash is a fixed-length output computed by feeding a string to a hash function... and will always produce the same output given the same input." This is very important in that the same password input will always produce the same hashed output. If a user submits the same password at login as they submitted and stored to their user when the user was created, the hashes will match. Thus, Rails stores user passwords as hashed versions of the plain text values entered on signup and subsequently uses these hashes as the basis for authentication by cross-checking them against hashes of potentially valid password inputs on login.

Illustrating this hashing process, Flatiron School [Flatiron School course modules] explains that hashing is like:

making a smoothie. If I put the exact same ingredients into the blender, I'll get the exact same smoothie every time. But there's no way to reverse the operation, and get back the original ingredients from the smoothie.

Hash functions work in a similar way: given the same input, they'll always produce the same output; and there's no way to reverse the output and recreate the original input.

There is a very important caveat to this, however.

Malicious actors who are trying to crack passwords can "run lists of possible passwords through the same algorithm, store the results in a big database, and then look up the passwords by their hash." These mapped collections of strings to hash outputs are known as rainbow tables. In addition to rainbow tables, hackers can and will try to crack passwords through brute-force attacks, whereby the attacker systematically tries every possible combination of characters until they arrive at the right password. Unsurprisingly, these methods can take a significant amount of time, but are likely to decode password hashes. This is where having Bcrypt hash password data is doubly essential.

Bcrypt gives us a convenient and powerful solution to the threat of malicious action. By _salting _passwords, or prepending random strings to passwords before hashing them, the complexity and computational cost of running hash algorithms to guess and crack passwords increases astronomically. Even though the salt is stored in plain text beside a password, a malicious actor is unlikely to have constructed a rainbow table with the salt already prepended and must therefore rerun and reformulate their password combinations. In this way, Bcrypt is intentionally expensive computationally. The entire point is to make the process of running these hashes slower and therefore less susceptible to hacks.

Refer to the following Bcrypt gem documentation for a summary of salting:

The solution to this is to add a small chunk of random data -- called a salt -- to the password before it's hashed:

hash(salt + p) #=> <really unique gibberish>

The salt is then stored along with the hash in the database, and used to check potentially valid passwords:

<really unique gibberish> =? hash(salt + just_entered_password) bcrypt-ruby automatically handles the storage and generation of these salts for you.

Adding a salt means that an attacker has to have a gigantic database for each unique salt -- for a salt made of 4 letters, that's 456,976 different databases. Pretty much no one has that much storage space, so attackers try a different, slower method -- throw a list of potential passwords at each individual password:

hash(salt + "aadvark") =? <really unique gibberish> hash(salt + "abacus") =? <really unique gibberish> etc.

This is much slower than the big database approach, but most hash algorithms are pretty quick -- and therein lies the problem. Hash algorithms aren't usually designed to be slow, they're designed to turn gigabytes of data into secure fingerprints as quickly as possible. bcrypt(), though, is designed to be computationally expensive:

Ten thousand iterations:
user system total real md5 0.070000 0.000000 0.070000 ( 0.070415) bcrypt 22.230000 0.080000 22.310000 ( 22.493822)

> If an attacker was using Ruby to check each password, they could check ~140,000 passwords a second with MD5 but only ~450 passwords a second with bcrypt().

So, now the question is how do we use and integrate all of this hashing and salting and password safeguarding into our applications? Simply, Rails streamlines all of this for us. Rails provides access to a method called has_secure_password. To access this method, I had to first download and install the 'bcrypt' gem, including it to my gem file and running bundle install. Next, I used has_secure_password in my User model.

  class User
    has_secure_password
  end

Following this, I gain access to two new instance methods in my User model, password and password_confirmation. However, these methods do not correspond to similarly named database columns, but rather a single 'password_digest' column.

In my database schema, this looks like:

create_table 'users', force: :cascade do |t|
    t.string 'email'

    t.string 'password_digest'

    t.string 'first_name'
    t.string 'last_name'
    t.string 'phone_number'
    t.string 'address'
    t.string 'city'
    t.string 'state'
    t.string 'country'
    t.string 'passport_number'
    t.integer 'age'
    t.date 'date_of_birth'
    t.string 'nationality'
    t.string 'avatar_url'
    t.datetime 'created_at', precision: 6, null: false
    t.datetime 'updated_at', precision: 6, null: false
  end

With these things in place, I can set up my routes, models, and controllers to fully implement user authentication and authorization.

In quick summation of how this request-response flow cycle looks in my project:

Sign Up

A user visits the page for the first time and does not have an account
User submits an email, password, and password confirmation as form data
After clicking create account, a POST request is sent from the frontend to the signup route post '/signup', to: 'users#create' in the backend
this route is matches to the users controller, wherein the create action facilitates the request as such:

  def create
    new_user = User.create!(user_params)
    session[:user_id] = new_user.id
    render json: new_user, status: :created
  end

private 

 def user_params
    params.permit(:email, :password, :password_confirmation)
  end

the database gets accessed and manipulates data as directed in the controller. it conditionally creates a new user object pending validations I have set in the User model

  validates :email, presence: true, uniqueness: true
  validates :password, confirmation: true, presence: true, 
             length: { minimum: 8 }, on: :create
  validates :password_confirmation, presence: true, on: :create

Assuming these requirements are met, the User object is created in the database, saved to the current session, and returned as json data to the frontend.

Login

A user visits the app and is not currently logged in. The user is prompted to log on via submitting email and password.
Form data is submitted as a POST request, or post '/login', to: 'sessions#create'
This route path matches to the sessions controller wherein the create action facilitates the user authentication process as such:

 def create
    if params[:email].present? && params[:password].present?
      user = User.find_by(email: params[:email])

      if user&.authenticate(params[:password])
        session[:user_id] = user.id
        render json: user, include: ['bookings', 'bookings.room', 'bookings.room.hotel'], status: :created
      else
        render json: { error: 'Invalid email or password' }, status: :unauthorized
      end
    else
      render json: { error: 'Email and password are required' }, status: :unprocessable_entity
    end
  end

Here, I am checking first that there is a valid email and password present, and, if so, find the user that has an email matching the form data. If these values are missing, I return an error message as json informing the user that an email and password are required.
My User model additionally validates the email and password presence and length and returns the matching user object based on email.
Next, my controller facilitates authentication via user&.authenticate(params[:password]) (another method given to us via bcrypt) to compare the salted hashes and confirm that the submitted user password is the same hashed value as the stored user password digest
If either the password values do not match, or a user is not found via the email params supplied by the user, an error is returned as json stating "Invalid email or password"
If the password params and password digest hash values match, the user is "logged in" and stored in the session as seen in the code block above.
Moreover, all database associations are included with the user json (see multi-level nesting and json in rails docs for more details)

Authorizing Current User

Aside from logging in and signing up, a user may also stay logged in as a current user. This current user status is saved and confirmed via Session data.
If a user has yet to log out and chooses to navigate away from/return to the app or refresh the page, I check that the user is logged in as a current user via sending a GET request at page load via the useEffect hook in React. get '/me', to: 'users#show'

class Api::UsersController < ApplicationController
  before_action :set_current_user, only: %i[show update destroy]

def show
    if @current_user
      render json: @current_user, include: ['bookings', 'bookings.room', 'bookings.room.hotel'], status: :created
    else
      render json: { error: 'Not authorized' }, status: :unauthorized
    end
  end

private 

 def set_current_user
    @current_user ||= User.find_by(id: session[:user_id])
  end

end

Here, I have a custom private method called set_current_user running before the show action takes place. This method looks for a User object by searching for an id value taken from the user_id attribute in the current session. If there is an user object associated with that id in the database, it will be set as the current user.

If that conditional is satisfied, the Show action will return the current user as json, replete with all its database associations.

Logout

Logout is handled via a DELETE request sent from the frontend and handled by the backend route delete '/logout', to: 'sessions#destroy'
It is carried out as:

class Api::SessionsController < ApplicationController
  before_action :authorize, only: [:destroy]

 def destroy
    session.delete(:user_id)
    head :no_content
  end

  private

  def authorize
    return render json: { error: 'Not authorized' }, status: :unauthorized unless session[:user_id]
  end
end

Before action takes place, I confirm that the session has a user_id attribute value, or return an error of "Not authorized" back to the frontend
If there is a user stored in sessions, I execute the delete method on the session to remove the :user_id from session storage. This will mean that returning to the web page will show that no user is stored in session and therefore prompt login

With this setup, I have successfully implemented secure user authentication and authorization for my Phase 4 project, Book-It. This not only allows for multiple users to be registered, logged in, and maintain their individual sessions but also securely stores and verifies passwords using Bcrypt hashing and salting. In doing so, I have significantly improved upon my initial attempt at user authentication during Phase 2.

Moving forward, I will continue to prioritize security and implement best practices to ensure that my applications maintain the highest level of data protection and user privacy possible. As the digital landscape evolves and new threats emerge, I am committed to staying up-to-date with the latest tools and techniques to protect my applications and the data they store. Now, I will move onward to my graduation capstone and the ultimate, eventual goal of finding a job!

Sources:

Flatiron School. (n.d.). Ruby on Rails. Retrieved [4/13/2023], from [unlisted course modules].

bcrypt-ruby. (n.d.). bcrypt-ruby documentation. Retrieved [4/13/2023], from [https://github.com/bcrypt-ruby/bcrypt-ruby].

Put this in your {app} and {control} it, or how Sinatra serenades our data

Joshua Mayhew — Thu, 19 Jan 2023 15:52:38 +0000

As a quick preamble and summation of my Flatiron journey and thoughts on Ruby thus far, phases 1 and 2 of the software engineering bootcamp comprised a comprehensive and rigorous overview of frontend development (I.e. JavaScript and React.js), while phase 3 signaled the beginning of a paradigmatic shift toward learning the principles and practices of programming on the backend via Ruby. In striking contrast to the immediacy of being able to see code reflected in the DOM and being able to directly log code in the browser console, backend programming in Ruby presented me with a near total exclusivity of working and testing in the terminal.

At first, this lent an air of relative mystery and mystification to the idea of learning how to work in the backend. How could I build something if the data and methods and testing all take place in my terminal and, furthermore, what would I really be building if the backend deals in the realm of what is felt but not directly seen by users in the browser/client side of programming??

As I delved deeper, however, I soon realized that Ruby is a syntactically straightforward and powerful language that simplifies and streamlines code, and that the gulf between JavaScript and Ruby was not nearly as big as I had first thought when viewing lines of code scrolling down the terminal window. After familiarizing myself on how to write code in Ruby and 'puts' output to the terminal, I have rapidly embraced both the new environment and nature of backend programming.

In the more specific context of building my phase 3 project, I have since gained a far deeper and more practical grasp of MVC through my exposure to full stack development. As detailed on MDN https://developer.mozilla.org/en-US/docs/Glossary/MVC, the Model View Controller (MVC) describes a core pattern in software design that "emphasizes a separation between the software's business logic and display"(MDN, 2023). Aiming at a better division of labor for software applications, MVC delineates three distinct parts to ensure a clear separation of concerns:

Model: Defines and manages data in db and handles business logic
View: Handles layout and display
Controller: Routes commands to the model and view

To illustrate this division more practically, a simple shopping list app wants to list the name, quantity, and price of every item that a user would want to purchase for the week. In this particular use-case, the view would be everything the user sees and interacts with on the front end (e.g. user clicking an 'add to cart' button).

Following the user event, input is relayed to the controller, which acts as an intermediary between the front and back ends by handling the control logic in order to manipulate data in the model (e.g. receives the input from 'add to cart' click and notifies model to 'add item'. Finally, the model updates to reflect the item being added, which is in turn relayed to the view whereby the user can view the shopping list item as added to the cart.

Focusing on the controller part specifically, I utilized Sinatra to facilitate the requests/responses between my project's front and back ends. Sinatra is a small-scale, lightweight framework that allowed me to create a RESTful Ruby-based API. The routes that determine and direct how client-side user input is going to access and manipulate server-side data that gets returned as iterable json objects reside in the application_controller.rb file. Here, API endpoints are established and model methods can be invoked and executed on class models to return specific data.

The first and most broadly encompassing json object I wanted to return for my project was a nested collection of trip data including bookings and users. Because I created my database as having a many-to-many relationship between Trips and Users, joined by a UsersTrips table, I set up my first endpoint as follows:

get '/trips' do trips = Trip.show_recently_active_trips.limit(10) trips.to_json(only: %i[title budget start_date end_date img id], include: { users_trips: { include: :user } }) end

Here, the API is constructed and can be pinged via the /trips endpoint. When fetching data from the front end, a get request can be made to this endpoint in order to communicate with the backend and receive a response... and what facilitates this cycle is the controller, and, what facilitates this same front and back end interaction in the case of my project, is Sinatra. Sinatra sets up the API endpoints that manipulate the data and inform the db of what it needs to update. My trips endpoint is calling a .show_recently_active_trips and .limit method that I constructed in the Trip model to manipulate my Trip class and return the objects in particular structure and order.

Then, having my data organized as I like it, I can use .to_json to parse the data into an iterable json object that is compatible with JavaScript on the front end. Included in the .to_json are the only: and included: keywords that help to further filter and structure the content of the json object. I chose to only: return the title, budget, start_date, end_date, img, and id keys of my trip object, while include allows me to add additional nested object data because of the relational associations between Trips and Users on the backend.

So, following the Trip data, I can include the join table users_trips, and through users_trips, I can include each user that corresponds to a users_trip. Effectively, this single get request returns all of my individual trips in the database, each trips' userstrips bookings on record, and all of the users attached to each trip via their userstrip. The nesting will log in the console as follows:

Many Trip json objects

have many userstrips json objects that contain a single user each

UsersTrip json Objects:

User Objects

and therefore give a Trips object access to multiple userstrips and users.

Through Sinatra deployment in my project, I was able to set up endpoints in an API that facilitated user input requests that manipulated date on the backend and subsequently returned that updated data to the front end. As the controller in MVC, Sinatra performed a vital function in facilitating interactions between client and server sides and allowed me to create an app that renders trips wherein users can sign up, view details, and create new trips. In utilizing Sinatra, I was able to better experience and thereby understand the importance of MVC as a fundamental programming concept.

Sources:

https://developer.mozilla.org/en-US/docs/Glossary/MVC

I do declare(ative), React.js is good, clean fun!

Joshua Mayhew — Tue, 25 Oct 2022 05:32:53 +0000

To recap, my many months of unyielding dedication and diligence, spent methodically drilling and internalizing core concepts and implementing practical applications of JavaScript programming, successfully eventually paved the way to me starting Phase 2 at Flatiron School. Notably, Phase 2 consists of learning what is, more or less, the most popular contemporary JavaScript library, React.js. At first glance, React code may look cleaner and less sprawling that vanilla-JS. Why is that?

Simply, React programming signals a shift from imperative to declarative programming. Formally, imperative programming is defined as "a programming paradigm of software that uses statements that change a program's state," whereas declarative programming "expresses the logic of a computation without describing its control flow."

So, what does that even mean? Essentially, imperative programming means writing code that "focuses on describing how a program operates step by step," detailing each operation at a granular and often tedious level of code. In vanilla-JS, DOM manipulation might look like:

const btn = document.querySelector = document.querySelector(".btn")

const listener1 = () => {
  btn.style.backgroundColor = "#DB2777";
  btn.style.innerText = "Are you sure?"
  btn.removeEventListener("click", listener1)
  btn.addEventListener("click", listener2)
}

const listener2 = () => {
  container.innerHTML = 'unicorn'
}

btn.addEventListener("click", listener1)

credit: https://alexsidorenko.com/blog/react-is-declarative-what-does-it-mean/

By contrast, React is declarative. React programming merely "describes what a program must accomplish" rather than exhaustively detailing how a program accomplishes a task. Think about DOM manipulation. Vanilla-JS necessitates multiple levels of variable declaration: grabbing an element with .querySelector, assigning some value to the element, appending that element to a parent element, and possibly appending that parent element to yet another parent element.

In React, however, we can streamline the code needed to accomplish programming tasks. React abstracts and simplifies code, handling much of the nitty-gritty complexities of vanilla-JS under its hood. Through setting and managing component state, React lets us write and render code that resembles a mixture of Javascript and HTML (JSX) that is more succinct overall and dynamically responsive based on changes in state via user input.

Thus, DOM manipulation in React might look like:

const [scene, setScene] = useState('button')

if (scene === 'button') {
  return (
    <Button 
      blue 
      onClick={() => setScene('question')}>
      Show the Unicorn
      </Button>
  )
}

if (scene === 'question') {
  return (
    <Button 
      pink
      onClick={() => setScene('unicorn')}>
      Are you sure?
      </Button>
  )
}

if (scene === 'unicorn') {
  return (
    <span>unicorn</span>
  )
}

credit: https://alexsidorenko.com/blog/react-is-declarative-what-does-it-mean/

Here, React.js allows us to simply declare and set state, "scene," and handle button rendering, styling, and event listening all via a child component. Much of the specificity of imperative programming is thereby dealt with inside of React and our code is therefore considered declarative in its general higher-level description of what we want to do, rather than specifying exactly how to do it.

As such, my experience both learning and using React to build my Phase 2 project has been overwhelmingly more engaging and straight-forward than my first foray into JavaScript during Phase 1. Partly, this is because I better understand programming fundamentals and partly because React relies on declarative programming that makes my coding experience far less tedious and prone to minute error-making.

For anyone else taking their first steps into React.js, make sure to understand the distinction between imperative and declarative programming, since this distinction is what distinguishes and underlies the purpose of React.js altogether!

References:

go fetch()

Joshua Mayhew — Thu, 11 Aug 2022 11:50:05 +0000

During the first phase of my studies at Flatiron School, I spent considerable time learning how to make frequent and practical use of fetch. Referring to a modern, streamlined interface by which we "access and manipulate parts of the HTTP pipeline" via JavaScript, Fetch API is instrumental in facilitating the cross-flow of information between web browsers and remote servers.

In service of this information exchange, the Fetch API provides an eponymous global method, fetch(), that asynchronously fetches resources across the network. Essentially, this fetch method is a logical means of retrieving data from an external database and returning it where it can then be manipulated and injected into the DOM without negatively impacting page load, or the time it takes for the browser to render the page.

Fetch requests span a number of types (GET, POST, PUT, PATCH, DELETE), but in covering the context of my bootcamp project, I specifically relied on GET requests.

To start, a fetch request takes a URL string as a parameter, designating the location where data can be retrieved, and is followed by two .then() methods, wherein the data can be converted into JSON and therefore possible to work with.

fetch("your chosen URL input")
      .then(response => response.json())
      .then(data => console.log(data))

or, in the context of my bootcamp project:

fetch("https://api.teleport.org/api/continents")
  .then(resp => resp.json())
  .then(data => console.log(data))

Breaking this process down further, the fetch relays an implicit GET request (GET requests do not need an explicit method value given) to the url and returns a response, aka promise. A successful promise eventually resolves with a response object, or rather a representation of the entire HTTP response. To actually target and access the data within, the json() method returns a second promise that resolves by parsing the response body text as JSON, after which the data can be readily worked with.

Console logging the data, you should see that it's been returned as collections of objects. In the above screenshot, you will see an object key, entitled _links, containing other nested objects. Looking further still, under object key continent:items, there is an array of additional objects.

Now, still using the above-demonstrated fetch() method, we can code out what we want to do with the fetched data within the second .then(). For my project purposes, I sought to append the nested continent name values to the DOM using the forEach() array method. Specifically, I iterated through the returned JSON object values and assigned each continent name to a newly created DOM element and appended it to the DOM. This entire process can be handled within the second .then(). (See below code snippet for an approximate example).

fetch("https://api.teleport.org/api/continents")
  .then((resp) => resp.json())
  .then((data) => {
    console.log(data);
    const body = document.querySelector("body");
    const continentWrapper = document.createElement("div");
    const continentList = data._links["continent:items"];

    continentList.forEach((cont) => {

      const continentName = document.createElement("p");

      let continent = cont.name;

      continentName.textContent = continent;

      continentWrapper.append(continentName);
      body.append(continentWrapper);
    });
  });

In my final project code, I ended up abstracting functionality and handling many tasks via callback functions, but for ease of demonstrating how fetch can both retrieve external data and handle DOM manipulation, this code is a clear snapshot. First, I target the existing body element and assign it to a variable. Next, I create a div element and assign it. Then, I use object . and bracket notation to traverse the nested data objects and arrive at the list of continents. Finally, I use the forEach array method to handle creating new DOM elements to which I can append the individual continent name values to and ultimately append to the DOM.

Although there is so much more you can do with fetch and DOM manipulation, I hope this is a clear and helpful introduction to using the fetch method and accessing external data. Now, it's time to go fetch()!

Resources:

“Fetch API - Web Apis: MDN.” Web APIs | MDN, https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API.
"JavaScript Fetch API: JavaScript Tutorial https://www.javascripttutorial.net/javascript-fetch-api/