DEV Community

loading...

Discussion on: An Exploration of How to Use Amazon S3 as an Event Bus

Collapse
kylegalbraith profile image
Kyle Galbraith Author

I don't know that I would go all-in on this idea either. Another comment mentioned EventBridge which I believe could be a better longer-term solution with a lot of upsides. However, not all organizations can switch to a new service just like that. Some organizations are using S3 for a lot of other things so that is where an idea like this could be plugged in.

In terms of what inspired the thought experiment, curiosity on where the idea breaks down. I think for something simple like Publish everything to everyone, this mostly works. But the concept breaks down when you have > than 100 microservices. At that point not being able to filter events to consumers via SNS Message Attributes on S3 events is a major bummer. I believe the solution there is working around a limitation of S3 events, writing the producer event to S3, and sending a custom event via SNS. At that point, we are just using S3 as event storage now instead of a notifier.

Once you reach that limitation I think you could be better off just putting the event in the SNS payload. Or better yet, using a better option for an event like EventBridge or even Kinesis could be an option.