Greetings, fellow developers! I hope you're all doing well. Today, I'm thrilled to delve into the realm of multi-tenant architecture and share my i...
For further actions, you may consider blocking this person and/or reporting abuse
There are many problems to this and one of the most important, it will create huge issues when you needs to scale and need to do some sort of load balancing.
You are keeping these database connections in an LRU Cache (in-memory in the same machine), if I am gonna add a load balance and balance my request using 4 instances what's the chance that my users request will always go to the same instance which actually have the connection in the cache.
Most of the cases you will end-up opening a new database connections, yeah you can use sticky sessions and make sure the request does not go to a different instance based on some sort of identifier, like tenantId or something like that but in this case you won't be able to balance the request properly in all the instances.
Will points out one problem and you will surely understand that, you really can't scale with this model.
Well, thanks for your comments and insights. First, I would like to clarify that this is just a beginner's guide and should not be viewed as a complete solution. This model can easily handle 1 million requests or users; after that, you will have to start designing strategies to handle the load, as is the case with every software. But you first have to release the software and get customer feedback. You can't make the software complex from the get-go to handle 10 million requests, which you won't get.
As for the scaling solution, what I have done previously is move the connections cache to a separate machine. Now, I have made my backend truly stateless, and it can just request the connection from the other machine whenever it needs. But I repeat again, you should consider these problems after you truly hit the bottleneck, which may take time. And if you do reach the bottleneck, which means your software is earning you money, you can dedicate more effort since you will be getting an incentive for that.
Interesting article, I have worked on similar project in different language. I'm always interested in understanding how other people design the multi tenant architecture.
Can you explain about the "truly stateless backend" part ?
In the current backend architecture, a state is maintained as it keeps track of connections using an LRU (Least Recently Used) cache. If another instance of this backend is spun up, it will manage its own LRU cache. To address this issue, I separated the repository part of the code and created a separate Node.js ( later migrated to golang for speed ) project for it. This repository process now manages its own LRU cache and is responsible solely for running queries sent from the "stateless" backend, which contains the business logic.
This approach allows me to spin up multiple instances of the business logic process without concerns about managing connections. In a production environment, I only require two instances of the repository process. When I mentioned "truly stateless" I was referring to the backend handling the business logic.
I'd also appreciate hearing about your approach to this problem. What logic did you follow, and do you have any valuable insights? There's always room for improvement, and I believe there may be better solutions that I haven't yet discovered.
When you mention 1 million request/users using this way, would love to know a few things:
What's your request frequency (and how many requests/sec you are handling)?
What was the average latency for request processing?
Now, I am considering your repository process implementation, which is great but still it's hard to scale out your server for database request processing, you still can scale up but.
It all comes down to one very important thing and that's how many connections you have opened for the database and how many you are actually wasting.
We build multi-tenant system with sort of a hybrid approach, we design the system in a way where it understand 2 ways of communication so we can separate tenant based on their requirements and let other system work however it is.
I will explain it now: let's imagine we got a tenant (we are into B2B, business have their own taste for security based on the money), if tenant is interested in fully dedicated system so it's just easy now, but when it comes to shared we keep the users in the same database (protected using Row Level Security) and whenever required we can move them a few tenants to any other instance (we aren't handling every tenant directly from the same instance), based on the tenant id we decides which server to hit based on the centralized config which is managed differently (you can think on an INIT request via an API)
It's sort of an Application level sharding by tenant.
This solution helps us to scale out and scale up (horizontally or vertically respectively).
Thank you for sharing how your organization has handled multi-tenancy. The product I was building was a B2B CRM app, and the client's requirement was to have separate databases for each tenant, rather than separate code bases. Otherwise, I would have simply written code to manage a single connection. Your provided method can be a viable option for those without constraints unlike mine, as it offers easier management and scalability.
Regarding your questions:
First, it's important to establish that our backend primarily involved CRUD (Create, Read, Update, Delete) operations, with no intensive computations. All tasks were I/O operations. In its peak state, the monolithic architecture of the backend could handle up to 1000 requests per second (RPS) with an average latency of 300 milliseconds. Regarding database connections, we created five connections for each tenant, though unfortunately, I didn't measure the extent of connection wastage at that time.
Nice article on multiple database solutions! If you are interested in achieving multi-tenancy using a single database, you should check out how to do it with Prisma:
Multi-tenant implementation approaches with Prisma and ZenStack
JS for ZenStack ・ Mar 18
What do you guys think about this approach. Multi-tenant using node and mongoDB