Recently I have started to adopt the
'right tool for the job' mentality
when building my projects. The biggest impact it has is on the System Design aspect of the project.
Where I used to spend hours trying to juggle doing everything with one tool or language, now I spend about half the time researching the right tool for each part of the system and save myself a lot of tedious work.
Once you get into that mentality, a fundamental change happens in the project. Before, it was easy to setup the project and start coding it. But the bigger the project became, the more the drawbacks started to show, it became harder and harder to continue. Whereas, now, the project set up is harder, since you need to research the system design and architecture and set everything up, however, coding has become easier and more fun.
This shift has changed me for the better. Because I enjoy both the research and the system design, I find myself having much more passion and fun with my projects.
What is System Design?
Before we dive into how this change has affected me, and how to change for the better yourself, we should know what System Design is.
System Design is the beautiful and mysterious practice of
creating efficient and scalable systems
What does that mean?
It means creating and architecting software systems that can scale well, handling growth seamlessly - for example, scalability might mean growing an API from 1000 req/s to 100000 req/s, or growing an app to handle 1000 users instead of 100.
The meaning of scalability is open-ended, but it usually centers around performance, reliability and adaptability.
My philosophy of 'right tool for the job' drives this by breaking down a complex project into smaller parts, and then finding and researching about the best tool for that specific use case.
This also fits really well with the Micro-Service architecture, which is known for its scalability and is well taken within the software world.
This can involve dividing a project into microservices, or at least into smaller, more manageable components—even if it's not a full microservices architecture.
This approach transforms a complex project’s initial chaos into clarity.
Breaking free from the Single-Tool Trap
In my early days of building projects, I - like many beginners - Often shaped my project around whatever tool I was using. Though it is fine when learning or building small projects, but the rookies tend to bring this ideology with them whenever they make a more complex project.
Its not just a rookie mistake - even some businesses fall into this trap. But you can't entirely blame them, no one pays System Design enough attention; people only care about System Design as a way to crack interviews and jobs at various companies but its so much more than that. System Design can convert a good project into a great one.
To break free from the Single-Tool trap is too easy, all you have to do is explore. Explore other projects, other languages, other tools. Even if you don’t master every new tool, simply knowing what’s out there can make all the difference.
Case Study: My Journey in Architecting Complex Systems
In my recent projects, I’ve ventured into more complex territory, inspired by the gaps I noticed in my own developer experience. And following this, some complex project ideas have sprung in my mind. 2 in particular where I really let my inner architect come through.
I have been a backend developer at heart for quite a long time, and to no surprise, Its the field I'm the most comfortable in. Due to that, I have come to always try to split my projects into smaller parts, and oftentimes, the most variable is the Database.
Therefore, in my recent projects, I tend to split my databases across multiple tools. While I always use SQL as my main database, it comes with its downsides.
One such case I ran into was when I was building a post platform similar to reddit in some sense. There I encountered this drawback of SQL, recursive comments. When trying to replicate the recursive comment system of reddit, I found myself very confused and flustered by how such a system would work in SQL. What I thought of, with the help of the lovely people of StackOverflow was that I could create something similar with a table in the form of:
| comment_id | parent_id | comment | on_post |
|---|---|---|---|
| 1 | 1 | "Hello World" | true |
But this is a very crude implementation, in which parent_id is vague, it could either be the post_id or the comment_id of the parent comment, based on the on_post variable.
Also querying this would be complex and slow.
Thus I set out to find an alternative for this use case and I found, 'JSON' or I should say, MongoDB. The NoSQL database allowed me to store all the comments as a JSON file, with none of the on_post jargon and also have fast reads since I'm reading it all at once, no multiple queries.
Another similar use-case for MongoDB which I found is chat messages, SQL apparently is bad at handling large amount of writes, and although Cassandra, is better at that, the lack of a well known go driver made me choose Mongo.
I must mention that System Design not just contributes towards a more scalable project, but it also improves developer experience. In my chat app as mentioned previously, the fact that I can split up my backend code into parts such as WebSocket, REST and Auth etc, means that me, and other contributing developers have an easier time working together. Less merge conflicts is a gift for all.
How to Find the Right Tool
To find the right tool you must first find out what the job is. And thus the first step in designing a good system architecture is to first make a mental model of your project and to break it down into what you want to do. Only when you have carefully considered the scope of the project can you start with identifying the best tool.
Next comes research. If you’re new to the topic, start by looking for real world systems similar to what you want to build. Take a look at what tools they use, why they chose them, and whether those choices apply to your case. You will rarely find an exact copy of your use case and thus you will always have to consider the drawbacks and advantages.
AI can be invaluable for researching, you can give context to AI and also provide a overview of your knowledge, and let it provide you with the existing projects and what Architecture they use. Then for any thing that doesn't make sense, or any tool you haven't dealt with, you can get detailed information on its advantages and drawbacks.
Beware though, do not rely on the AI's System Design, always think for yourself if the tool fits you case. Have a conversation with the AI, ask it why, and why not.
How Microservices Amplify My Philosophy
My `right tool for the job' philosophy really shines in a microservices architecture. Generally a microservices architecture breaks a project into parts such as - Authentication, REST API, GraphQL etc and uses a message broker such as Kafka to send and receive messages to or from a service.
Each microservice has its own database and other components, with their personal design principles.
"But doesn't this increase overhead, or even increase complexity since there are multiple instances?"
It does, but its a fair trade. We do get more overhead, and there exists plenty of tools to manage it, but its a trade worth making for more:-
- Scalability: Each service is isolated, and on its own VM, therefore they are not influenced, and can be scaled better.
- Reliability: Breaking one part of the architecture doesn't make the whole app break.
- Development: The most important of all is that it can make development clean, specially in a multi-team environment.
If we look at the architecture as a whole, yes, it will feel a lot more complex, since there are multiple services, interacting with each other etc. But when you look into each service itself, they are a lot more simplified, and this is what we want. A developer can create or update a service with a certain level of Abstraction from the other services.
Thus, a microservice architecture allows or even expands on my 'right tool for the job' philosophy, allowing a clean isolation of each service, and therefore less constraints when finding the right tool.
The Fun in System Design
While experimenting with System Design and building one for your app the I get to do two of my favorite things, and that is research and learn more about other tools. I have fun when I sit and research for hours how other products work, or how their implementation might not be for me.
My favorite thing though, is that I get to talk to people and get their opinions and give mine. To discuss about something I am passionate about with others who are just as passionate is like a cherry on top.
Another thing to note is that its not just the designing that is fun, since I get create cleaner services, and divide my code. I have a lot more fun coding as well, to actually create what I thought in my head. Sometimes it's boring, like setting up tailwind in a project, or eslint etc. But overall throughout the project I have a much better time.
My Lessons From the 'Make-It-Fit' Trap
I have spent a lot of time being deluded by the 'Make-It-Fit' mentality, but I think of it as a learning experience. Only when I spent my time doing that did I realize its drawback. Some may think of it as time wasted, but to me, no learning experience is a waste of time. And you learn from your failures.
What actually were the lessons?
- Rigidity of Tools: Tools are often created for a use case in mind, If you try to expand on that and stretch them past their limits, you will realize it really quickly why that was a bad case. It will stop being a straight process. Yes, programming has its hurdles, but it also has its solutions. When you take a tool too far, It quickly devolves into constant problem-solving rather than productive building, and you have to replace the tool or stop relying on it.
- Readibility: Most of the times, you end up having to create a workarounds to make it work. It may work, and yes it may even be less lines of code than using a different tool, but is it really the best way?
As you start to upgrade towards the 'right tool for the job' you will start to realize the small things as well, and I guarantee you will start to have more fun when programming.
Conclusion
There exists no 'Master Tool', something that will satisfy your use case perfectly for the whole application. You will need to compromise for most of the time, and you can reduce that compromise to almost nothing adopting the 'right tool for the job' mentality.
I could have easily given a list of tools I found, and for what use case they excelled at. But the realization is not in reading someone else's thoughts, Its by experimenting yourself and creating your own opinions. You may find out that you may prefer the Single-Tool approach, but Its you who must decide this.
All I can say is to try it out and experience all System Design and Architecture has to offer, don't just treat it as a way of cracking interviews.
Ultimately, system design is about personal growth, continual learning, and discovering what works for you. Enjoy the process and make it your own.
Top comments (5)
This resonates a lot. Adopting the ‘right tool for the job’ mindset completely changes how you approach building things. I’ve noticed the same in my own work when the architecture truly fits the problem; everything feels lighter and more manageable.
We’ve taken a similar path with haveto.com, focusing on designing systems that make AI run smoothly and transparently on-chain. It’s amazing how much more enjoyable the process becomes when you get the foundation right.
Nice explanation, this has really expanded my horizons!
Well articulated post! It was a lovely read.
This is a thoughtful perspective on system design. The core idea of matching architectural decisions to specific needs rather than forcing one approach is crucial.
We've seen similar benefits when separating components by their data patterns - some parts naturally fit structured storage while others work better with flexible schemas. The key is recognizing these differences early.
One challenge we continue to balance is the research/implementation tradeoff. Spending time evaluating options often pays off later, but can slow initial progress. How do others approach this timing?
Every detail is so well thought out, brilliant job.