Being unemployed sucks. But the only thing that doesn't suck about being unemployed is that you have more time to spend on developing fun projects. The ones whose ideas come to you in the shower, you know?
In this post, I'm going to talk more about one of them. More specifically, about:
- How did it all start?
- What is Cartogrify?
- Who developed Cartogrify?
- Stack and hosting providers
- What I learned from all of this
How did it all start?
In the past 5 years, I have noticed, like most music junkies, the fantastic tools that analyse your music taste through Spotify. Obscurify, How Bad Is Your Streaming Music and Icebergify are some great examples.
They do an amazing work with the analysis. But something that I appreciated even more was seeing people sharing their results and just talking about the music they love. I have discovered great songs from great people this way, actually.
One day in 2021, I was thinking: why not develop something like this? It seemed like a funny thing to do. Since it should be something new and different from the ones that already exist, I was trying to find a gap that the other tools haven't covered yet. Then, this idea came to my mind:
It basically says: get a playlist, find where the artists and the playlist author come from, highlight the artists' countries on a world map and show interesting details about it.
And that's how Cartogrify was born. 2 years later, after I graduated, concluded my internship and my research, I finally found time to bring it to life.
Ok, so... what is Cartogrify?
It's a web app that fetches your top artists from Last.fm and Spotify and finds where they come from.
Then, it counts how many different countries, regions and continents are associated with your artists. A world map highlighting their countries and a Sankey diagram showing their regions and continents are both generated.
Finally, it compares the geographic data with other users to find how internationally diverse your music taste is.
It also has a ranking showing which country has more diversity and popularity.
You can see it here.
Who developed Cartogrify?
Me! I'm Leonardo, a software developer from Brazil. I recently graduated in Information Systems from the Federal University of Alagoas while also being an intern for almost 2 years and working with research for 1 year.
Here's my LinkedIn and GitHub profiles, in case you want to see it.
Which stack and hosting providers did I use?
Frontend
In the frontend, I used Angular 16 with PrimeNG components and D3.js for data visualization. The deployment is made with Google Firebase Hosting.
Why Angular?
According to some career advice I received, choosing one technology and focusing on it to increase proficiency can increase the chances of getting a job that uses this technology.
Because I have already used Angular in my internship for 1 year and 6 months, it seemed reasonable to keep using Angular for that reason since getting a job is a priority for me at this moment.
Otherwise, I would have probably chosen Svelte to learn something new since I have previously worked with React and Vue.js. Any of them should work just fine, I think.
Why PrimeNG?
In my internship, Nebular is used as the component library. Since I have also used Material UI in other projects, why not try PrimeNG and see something new?
Why D3.js?
I've already used D3.js in my other project called LogCharts. Because of this, I'm familiar with it and know how capable this tool is for data visualization. So it wasn't a difficult choice.
But the thing they say about D3's learning curve is true. Sometimes, it can be frustrating. I pondered about switching to Chart.js or Google Charts to make only the bar charts in a more straightforward way but decided to be stubborn. Turns out it was a good decision.
Why Firebase?
Because I have seen lots of people talking about it and using it in their projects. Eventually, I got curious and wanted to try it. The free tier is good for small projects and it turns out to be simple to deploy and manage.
If things get bigger, I might go to Cloudflare, though.
Database
PostgreSQL.
Why PostgreSQL?
One requirement that was on my mind since the beginning was zero cost. I don't have funds at the moment to use in the project so I'm looking for generous free tiers.
When I was searching about database hosting providers, I noticed that Supabase not only offers free PostgreSQL database hosting but also a RESTful API that handles CRUD requests.
It kills two birds with one stone. Seems like a great deal for me! The unlimited API requests and 500MB database space are nice too.
I think I have to say: I'm not sponsored by Supabase. Oh, how I wish...
Questions
When I made my choice, I must say that some questions were on my mind. They might seem silly to some people but I decided not to ignore it. Here are they:
- "Is a relational database like PostgreSQL suitable for this project?"
- "Would it be better if I used a non-relational database like MongoDB? What difference does it make?"
- "DynamoDB has 25GB of free storage and 200 million free requests per month. No way my app gets big enough to get out of this free tier! It's future-proof!!"
- "If I choose a hosting provider that doesn't offer a CRUD API, I'm gonna have to build it myself and host it. Is it gonna worth it? What about cost?"
As you can see, they are divided into three categories: paradigm, scalability and utilities. Let's tackle them.
Paradigm
Since I used MongoDB in my internship for 1 year and 6 months, I knew that this relational vs non-relational thing wouldn't make a big difference. My database is not rocket science, after all. Should I care so much about this question?
Searching about this topic, the main advice seems to be: "If you don't know the answer, use a relational database. It will probably work." So I did this and, until now, it's working.
Scalability
DynamoDB seems to be the winner here. It's tempting to just use it and forget about some fears like: "What if Supabase's 500MB is not enough?". But at the same time, Supabase is more straightforward and simple to use.
Since I didn't know if DynamoDB resources would even be necessary to handle Cartogrify presumably small scale, I just opted for Supabase. Like they say: keep it simple, stupid.
Until this day (October 18, 2023), with 900 users and 15k artists, I'm using less than 100MB of database space. But, there's a catch: Only Last.fm users can use Cartogrify.
I have submitted a request to Spotify to use their API's Extended Quota Mode which allows the app to be used by every Spotify user. If they accept my request, more users and artists will come and more space will be used. Will it be enough to justify a migration to DynamoDB then? Let's wait for the next chapters!
Utilities
It's not always that a database hosting provider offers you a RESTful API like Supabase does. In this case, you need to develop the API yourself, which isn't a problem since it's just CRUD operations.
But to use this API, you must send a request to one of its endpoints, which implies that you must set up and deploy a server that listens and responds to these requests. Until now, I haven't found a reason to follow this path instead of using Supabase API.
If I were using DynamoDB or another database without this utility, there would be a reason: Supabase isn't fit for the project anymore. But I don't know when or even if this day will come so more points to Supabase.
Backend
In the backend, I used AWS Lambda running JavaScript code in a Node.js environment. Besides that, I also used Supabase Edge Functions running JavaScript code in a Deno environment.
Serverless or not?
As you already know, zero cost is a must and a generous free tier is what I'm looking for. Using the Free For Developers website to search for hosting possibilities, I noticed some serverless options that were quite interesting regarding their free tier.
I didn't know much about this serverless thing so I started searching about it. When I noticed that I didn't needed a server running 24/7 or have to deal with cold starts like the ones from Heroku or Render, the serverless seemed like a great idea.
Since Supabase already offers free Edge Functions, it just decided to try it and it worked nicely!
Why AWS Lambda?
In the beginning, I only used Edge Functions. However, some limitations that I will address in a later post made me migrate the country detection algorithm to another place.
I searched for alternatives and found that AWS Lambda can run for up to 15 minutes and supports Server Sent Events. That was more than enough to solve my problems. Not only that but a generous free tier. So I decided to try it and it works great too.
Why Edge Functions?
As I said earlier, it's free, it's from Supabase and it works nicely for any other demand than the country detection code. It just works.
What I learned from all of this
- Sometimes, it's great to try something new. Sometimes, it's better to seek improvement in what you already know.
- You shouldn't worry so much about getting out of free tiers while developing hobby projects.
- If your app does get out of the free tier due to being viral or something like that, it's actually a good thing. People are using it.
- Sometimes, all you want is just a simple Serverless Function. No machine running 24/7 in the backend and no cold starts.
- To have an experienced mentor to discuss things can surely make a world of difference.
- There's no way you can be good at so many things, like UI/UX, frontend, backend and cloud at the same time. You will just be "enough" in all of them. And that, in my context, is not enough to get a job. So it makes sense to focus on one thing and be "enough" on the others. Contributors are key for big projects!
In the next posts...
In the next posts, I'm gonna talk more about the challenges I faced to make Cartogrify work and also share with you what I learned from them.
I think I'm gonna start with "How to find an artist's country of origin knowing only his name?". That was an awesome one! But if there are other things you would like to know about the Cartogrify development journey, please let me know!
See you there! Thanks!
Top comments (0)