Setlistopedia

Not the best name but a fun project nevertheless.

A few months ago I interviewed for a job where I was assessed on my knowledge of Node.js. At the time, it was very close to nothing. To prevent this from happening again, I decided to learn it. Around the same time I started listening to the Grateful Dead a lot. While listening to the Grateful Dead, I noticed some patterns in their setlists. I enjoyed hearing how one song would flow into the next across many different shows. Curious if there were more of these patterns in their setlist sequencing, and wanting to learn Node.js, I decided to build a web application which analyzed a band’s setlists. I utilized React, Typescript, Node.js, and Prisma as an ORM for a PostgreSQL database. Computations performed on an artist’s data included the top five songs they ever played, what year they played a particular song the most, and which song is most likely to be played before or after.

While certain parts were easy to ascertain a path to implementation, certain parts were frustratingly difficult. But only some. For instance, I wrote an iterative function which creates a frequency counter for each unique song in a collection of an artist’s setlists. This was easy. But getting comfortable with typescript and the plethora of syntax that follows? That was particularly difficult.

I eventually came around to enjoy the benefits of typescript - I found I could code more efficiently, interacting with pre-defined data structures safely and efficiently. Since my application’s main utility is analyzing data structures, the benefits shone through quickly, however frustrating typescript/eslint’s incessant complaints were. As with most things, it just took some practice.

In order to create statistics on a band’s live performances I needed data to analyze. I knew there was a thorough data source of setlists on a website called SetlistFm. Luckily, they have an API available. I designed the app to allow users to search for an artist they were interested in. When a user searches for an artist, a check on whether said artist is in the database occurs. If the artist was not in the database a series of requests would be sent to SetlistFm.com until all pages of data were ingested and added to the database. The idea was that over time, with more user activity, the database would become more comprehensive and the reliance upon SetlistFm’s API would decrease.

This would have been fine, however, there is a rate limit for requests made to their API. I realized this would cause a decent amount of trouble for my application. It would only take a small number of users to make simultaneous requests to cause 429 response errors to be sent. Initially, after some Google searches and StackOverflow exploration, I thought implementing a retry function with exponential backoff might save a user from these errors. But with the possibility of more and more users making requests, this would still not suffice. I thought about it like water flowing through a pipe with holes in the top of it - at a slow flow rate, all of the water would pass through fine and make it to its destination, but with too much water flowing through the pipe, you’re going to have a mess. As an alternative, I decided to seed my database with a lot of artists’ data using SetlistFm’s API. This would significantly uncouple any reliance upon SetlistFm’s API.

I learned a lot but below I’ve listed some of the things I found to be most interesting.

I learned about utilizing local storage to persist state on a refresh.
I learned about organizing code more efficiently and how to better follow DRY principles. For example, writing basic helper functions and services.
I learned about Domain Driven Design and the importance of naming variables.
I learned how to troubleshoot, debug, and measure efficiency better through a myriad of console.logs and even inspecting memory usage in devTools.
I learned about database architecture and how to be strategic in creating only the tables you need with only the fields you need - the same goes for querying said database.
I learned about creating dynamic URLs which send pertinent information from the client to the server.