So I have this 6 years old rails application monolith that contains all these things:
- user interface
- database
- additional tables for statistics (snowflake schema)
- API for android app
- workflow engine
- batch workers (sidekiq)
Currently I'm running 4 instances of this application, each is configured especially for the customer who's using the instance. Yes, that makes 4 customers, each has ~300 users, so we're talking about 1200 users. A virtual server with 2 cores and 8 GB RAM handles it easily. The web server is nginx and the ruby application server is passenger.
In 2020 a new UI (SPA) will be written, so I can get rid of the UI in rails. However I will have to provide an API for the SPA and it will be totally different than the API for the android app.
What a great opportunity to split the monolith into microservices!
I might split the microservices like this:
- API for new UI
- API for android app
- batch workers
- statistics
- what's left in the old rails app (business logic, workflow engine)
So far I pretty much know what I'm doing. However I don't have much experience in the microservices world, so I have a bunch of questions for you:
- What about user authentication at the API for the new UI? Are JSON web tokens the way to go?
- What about the database? Should I split it along the microservices?
- How do the microservices communicate with each other? I prefer a message bus over direct REST calls, so...?
- RabbitMQ?
- Kafka?
- Nats?
- Is it worth learning the whole Kubernetes stack? Or is docker plus docker-compose (or helm???) enough? Any other alternatives?
Actually my monolith isn't even dockerized, yet. Currently I feel overwhelmed by all the technologies I can chose from. But what's the right technology stack for rails? And for me? This rails application is only a hobby project. Any recommendations?
Top comments (13)
I think you are putting too much risk (changes) on the table, and Microservices is not the right solution for all kind of projects and teams.
Analyze your business first and identify the minimum steps that you need to continue evolving.
Yes, I agree. A lot of times there is this general pressure from management to get on with latest trends. Also writing everything as microservices does introduce more complexity in development, testing, deployment, troubleshooting and maintenance.
I have 2 major pain points in my Rails app:
So for the UI part I am glad that I can shift this responsibility to another developer who's writing the SPA. Thus I can focus completely on the 2nd pain point.
I know a bit of Ansible and concluded, that it's not the right tool for my situation. But I'm sure that containerization is the right way. That's why my question is targeting the docker/kubernetes eco system. How much of it do I need? How much of it can I handle?
Strangler pattern discusses a good migration path and I'm going through this process right now so we should compare notes sometime.
A few options suggested by Microservices authors:
In the existing monolith separate out the UI layer of your web app from the API layer. This fits in nicely with your wish to offload to another dev. Rails I'm unfamiliar with but sounds much like sails.js framework I use, therefore going api first on the server / UI split is a good move. This will allow you to discover the boundaries and potential scope of services with little disruption. The aim of the game here is to peel back the layers of onion to expose a core set of services, you would eventually end up taking this step but removing lockstep deployments early can give some early benefits to the mindset.
Create services for new features to stem the flow of legacy being built into the monolith
Consider taking any low hanging fruit features which exist today in the monolith which can be separated out into a service, to reduce complexity you can still use your existing dB as the data store. Infact on this point, to reduce stress and complexity it seems sensible to work through the tiers making changes until you hit the data storage layer last. I prefer this technique as the last thing I need to battle is data consistency and accuracy on a new service when I can use existing data models/tables to verify behaviours.
Ive seen many advocate breaking services out for things which change the most. However I have some quite critical components which I don't wish to disrupt whilst I'm still in discovery phase with Microservices so I've infact picked out 3 initial candidates which have changed maybe once a year compared to the weekly and daily services in my main app. This allows me to produce a well defined service knowing I don't have a change hitting amongst other deployments which just leads to tight deadlines and undue stress
First take a step back and consider what your thinking about this holistically. My immediate questions I am going to ask you?
Why is rails not working for you ?
Why are the Android and SPA apis so different? You can support multiple auth endpoints. Long term your adding technical debt by having two api stacks.
Your already running a copy of the stack for each customer. What are your pain points that’s require this? How could you address these?
You already have redis which will handle most use cases you have for Rabbit, Kafka or NATS. Look at he the thing you know before jumping into something new. (Also don’t use rabbitmq rpc calls please)
If this is a hobby project make sure what your doing is actually achievable or your going to feel like you failed.
Thank you, Kevin, for your questions. Here are my answers...
Rails is actually working very well for me. I don't want to get rid of rails. But the app grew bigger and bigger, because I added new features in the UI and always used a quick way (short-term) for the implementation. So I was afraid of architectural changes for too long.
The Android UI targets different users (a different business role) than the SPA. There's not much overlap in the objects these 2 APIs handle, so I think splitting the DB for the 2 APIs into separate DBs would be rather easy.
Great question! It all started with data separation between the customers (a requirement from one of the first customers), but it turned out to be invalid. So there's no need anymore to run a copy of the complete stack for each customer. However running a copy of the complete stack provides a great feature that I don't want to lose: I can upgrade the apps for some customers only (i.e. one customer is a beta tester) while the others stay on an old version.
Uhm... not quite, at least not in the way I currently use these products. Redis is currently only a (temporary) key-value store and each instance has its own namespace. There's no communication between the instances in Redis. I came up with messaging systems, because I see the opportunity to optimize some parts of the whole system with the help of a pub/sub message bus. And I think the full potential of said message bus will unfold after splitting up my monolith.
Microservices are an organization tool, not just a technical one. Will this app be managed by multiple autonomous teams? If not you will either increase N folds your workload or end up with a distributed monolith. What about a modular monolith? engineering.shopify.com/blogs/engi...
Use the Spring (Spring Boot, Spring Security, et-al) framework.
This is a bad piece of advice - it's advocating a complete rewrite from scratch. Is there a good reason to do this?
Well, since you want to break up a monolith into microservices, it's like a rewrite. And if you're open to a rewrite, I thought of recommending a solid and mature framework like Spring. So I don't think it's bad advise, but my bad that I didn't add the detail about a rewrite.
If I were to do it, I would rewrite because it allows freedom to apply new design ideas when going down microservices route.
Also your advocating a completely new language which is not exactly a great start without understanding the problem domian.
Well, now I have to say I find Spring quite a good framework based on personal experience. And the original poster asked for opinion, and I gave mine. Jeez so much analysis of my opinion? I'm flattered :-)
Please if ignore my replies if it is so wrong ....
A complete rewrite is not what I want.