Project: Is Alameda Exploding?

I live on the island of Alameda in the San Francisco Bay area. We are bordered on our east side by the Oakland Coliseum and our west side by AT&T Park just directly across the water 3 miles away. When there are fireworks in either of these places we can hear them, and sometimes see them. On holidays we hear all the surrounding fireworks. We also get Blue Angels flying over and so on.

In short, we’re used to hearing noises, and I usually turn to my husband and say, “Must be sportsball” and shrug. But a week and a half ago and we were hearing all kinds of fireworks, my husband said, “I wish there was just a site I could go to in order to verify why I’m hearing fireworks.” (It was the A’s winning a game.) Which in turn made me think of the site “Is Oakland on Fire?” and thought it’d be fun to make something simple like that.

So this week I decided to quickly bang something out and got really excited to actually be working on a personal coding project for once, since I tend to not have a lot of time for my own.

Holidays

I figured Sinatra would be perfect to quickly get something up and running since it’s only a one page site, and then started out by looking into any plugins or API’s that would easily give me a list of holidays. Since I really only care about the ones that have fireworks, I could have hard coded the information but really wanted the site to be as automated and easily maintainable as possible. I ended up finding this Holidays Ruby gem which was perfect for what I needed. It makes it easy to also check if there are any fireworks holidays that week, since for Independence Day we tend to hear things in the days before and after, even though they’re technically illegal.

FIREWORK_HOLIDAYS = [
  "Independence Day",
  "New Year's Eve",
  "Lunar New Year's Day",
  "The second day of Lunar New Year",
  "The third day of Lunar New Year"
].freeze

def check_holidays
  logger.info "Checking Holidays"
  today = Date.today
  holidays = Holidays.on(today, :us, :hk)

  # Might be the week of Independence Day, etc
  if holidays.empty? && Holidays.any_holidays_during_work_week?(today)
    holidays = Holidays.next_holidays(1, [:us, :hk])
  end

  # We only care about the holidays that might have explosions
  holidays.map { |holiday| holiday[:name] }.detect { |hol| FIREWORK_HOLIDAYS.include?(hol) }
end

There might be some additional holidays to add, but I’m going to wait to expand it. Also since holidays don’t regularly change, I will be adding caching soon.

Sports

Then I figured the most heavy duty part was going to be finding the information about the sports games. I assumed I was going to need to use web scraping to get the information and started setting up Nokogiri and HTTParty. I quickly realized, however, that all the websites I wanted information from use JavaScript to load the information in after the initial page load. I haven’t done much web scraping before and did some research about how to address this and quickly came across this page that mentions just accessing the urls that are bringing in the information, since JavaScript is usually accessing JSON anyway before manipulating it on the front-end. Of course I read this and went, “Oh duh, that’s way more straightforward.”

So I found the behind the scenes URL’s on all the sports sites and ditched Nokogiri, since it was completely unnecessary, and just started looking through all the JSON being returned to find the information that I needed for my purposes.

class Scraper
  attr_accessor :check_for_games

  ...

  def grab_page_info(team)
    case team
    when "Athletics"
      HTTParty.get(baseball_url("531221"))
    when "Giants"
      HTTParty.get(baseball_url("531220"))
    when "Warriors"
      HTTParty.get(basketball_url)
    end
  end

  def winning_baseball_game?(response)
    response = response["dates"].first

    home_game = response["date"] == simple_date && VENUE_NAMES.include?(response["games"][0]["venue"]["name"])

    return false unless home_game

    night_game = response["games"].first["dayNight"] == "night"
    is_winner = response["games"].first["teams"]["home"]["isWinner"]

    home_game && night_game && is_winner
  end

  def check_for_games
    games ||= []

    SPORTS_TEAMS.each do |team|
      puts "Checking site for #{team}"

      response = grab_page_info(team)
      return unless response.code == 200

      if team == "Warriors"
        home_game = response["games"].detect do |game|
          game["home"] == true && game["date"] == simple_date
        end

        games << team unless home_game.nil?
      else
        games << team if winning_baseball_game?(response)
      end
    end

    games
  end
end

And that was basically it. I put it up on Heroku and then started fiddling with it to make it faster. I ended up adding IronCache to store the results after the JSON had been retrieved and parsed, which was the biggest bottleneck.

Summary

It’s a bit of a frivilous project, but it was fun. Here are some of the variations of what you might see.

In the future I will probably add some checks for whether it’s Fleet Week, and other such loud disturbances to expand it. Hope the Alamedans enjoy it! You can check out the live site or view the project on Github.