I can think of two scenarios where that might be the case. Two assumptions; you control both lambdas, these lambdas are services.
The second service let's call it service B can emit an event when ever data is written to its database. Then service A can listen to that and store the formatted data in it's own database/storage. So that when service A gets hit through the API, it does not need to do a call to service B/second Lambda, it can just get the already formatted data that it stored in its own DB.
It might be 'okay' if you only have 1 Lambda calling another Lambda using an API, if you are willing to accept the consequences (at the end). At this point the 2 Lambdas are dependent upon each other anyway so you might as well remove the API and call it directly with the AWS SDK. I would only do this direct coupling with 2 Lambdas at most if you really have to, more than that indicates a serious design flaw in the system.
Which option to choose:
It depends on what type of system you have and how much complexity you want to introduce. Method 1 would be great if have a read heavy application as you now only have to invoke 1 Lambda and the total duration to fetch the data will be greatly reduced. This creates an event driven system that brings the data to the "edge"/preformatted for the API to just return.
The first method quickly falls apart if you have a write heavy application as it will continuously be firing events and service A might be updating its own data that will not be read at that update frequency. So it will be more expensive. There is also now latency introduced by the eventual consistency of the event driven architecture and will be felt much more if it is a write heavy application.
For the second method you will be paying for both Lambda execution times, roughly the whole API call duration. This will turn out to be roughly twice as expensive compared to the first solution if the application is read heavy. You will also reach your account limits, the maximum invocation count twice as fast. Solution 2 is kind of a design smell, but then you have to draw that black box around both the Lambdas, if either 1 is down the whole service is down. Solution 1 also had the unexpected benefit that Service A became a write-throught-cache to service B. So if the DB goes down, the data is still stored at the API level and will still be returned to the caller. Your system will just not be able to write anymore because service B is down.
I think there is many ways to approach this problem, that is just 2 ways I would consider.
Thanks for the clarifications, definitely shed more light on the subject! 💡
It's a complex topic indeed. I'm very interested in improving my skills to make better decisions on this type of scenario. If you have any suggestions for articles/books/etc that were helpful to you, I would be grateful!
I gained most of this experience by designing and maintaining these architectures in production. Along the way I constantly watch and read articles on microservices and lambdas, not always taking the content 100% in but at least just making a note about it somewhere back in my mind.
I will look through my bookmarks a bit later to see if I might be able to give you a few resources that will be helpful on the topic.
It's almost been a year :) but I finally put my method onto paper and blogged about it. I am sure you might have skills by now and might already know this, but I thought I'll leave the link to that blog here for those that also want to solve this type of problem.
Helpful experience, thanks for sharing! I'm also looking into uses of EventBridge for different architectural applications. The Schema Registry makes it really flexible and easy to work with
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Comprehensive compilation! 😉
What is your approach when using SQS would add too much latency (#11 Coupling)?
Let's say you have this sort of data flow:
Say an SQS implementation would add too much latency here, we need to respond in < 500 ms. What ideas do you have for decoupling in such cases?
I can think of two scenarios where that might be the case. Two assumptions; you control both lambdas, these lambdas are services.
The second service let's call it service B can emit an event when ever data is written to its database. Then service A can listen to that and store the formatted data in it's own database/storage. So that when service A gets hit through the API, it does not need to do a call to service B/second Lambda, it can just get the already formatted data that it stored in its own DB.
It might be 'okay' if you only have 1 Lambda calling another Lambda using an API, if you are willing to accept the consequences (at the end). At this point the 2 Lambdas are dependent upon each other anyway so you might as well remove the API and call it directly with the AWS SDK. I would only do this direct coupling with 2 Lambdas at most if you really have to, more than that indicates a serious design flaw in the system.
Which option to choose:
It depends on what type of system you have and how much complexity you want to introduce. Method 1 would be great if have a read heavy application as you now only have to invoke 1 Lambda and the total duration to fetch the data will be greatly reduced. This creates an event driven system that brings the data to the "edge"/preformatted for the API to just return.
The first method quickly falls apart if you have a write heavy application as it will continuously be firing events and service A might be updating its own data that will not be read at that update frequency. So it will be more expensive. There is also now latency introduced by the eventual consistency of the event driven architecture and will be felt much more if it is a write heavy application.
For the second method you will be paying for both Lambda execution times, roughly the whole API call duration. This will turn out to be roughly twice as expensive compared to the first solution if the application is read heavy. You will also reach your account limits, the maximum invocation count twice as fast. Solution 2 is kind of a design smell, but then you have to draw that black box around both the Lambdas, if either 1 is down the whole service is down. Solution 1 also had the unexpected benefit that Service A became a write-throught-cache to service B. So if the DB goes down, the data is still stored at the API level and will still be returned to the caller. Your system will just not be able to write anymore because service B is down.
I think there is many ways to approach this problem, that is just 2 ways I would consider.
Thanks for the clarifications, definitely shed more light on the subject! 💡
It's a complex topic indeed. I'm very interested in improving my skills to make better decisions on this type of scenario. If you have any suggestions for articles/books/etc that were helpful to you, I would be grateful!
Thanks again! 👍
I gained most of this experience by designing and maintaining these architectures in production. Along the way I constantly watch and read articles on microservices and lambdas, not always taking the content 100% in but at least just making a note about it somewhere back in my mind.
I will look through my bookmarks a bit later to see if I might be able to give you a few resources that will be helpful on the topic.
It's almost been a year :) but I finally put my method onto paper and blogged about it. I am sure you might have skills by now and might already know this, but I thought I'll leave the link to that blog here for those that also want to solve this type of problem.
rehanvdm.com/serverless/refactorin...
Helpful experience, thanks for sharing! I'm also looking into uses of EventBridge for different architectural applications. The Schema Registry makes it really flexible and easy to work with