Holding the load - Avoiding duplicated Webhook requests

What is the motivation?

Imagine the scenario where you have a Webhook integration and the application responsible to notify your application sent the Webhook request more than 1 time.

The application will take the webhook data to create a subscription, so in this scenario will create 2 subscriptions where it will change the user 2 times instead of 1 time, so the user will be angry because of it.

How to resolve it?

Short answer is idempotency key.

The solution applied is converting the data sent on Webhook request to md5 hash and use the md5 hash as id to store on the database. For example:

The webhook request data: { “message”: “test”}

Md5 hash: 42cc32636e077687972862938d538929

So everytime I send the same Webhook request data will generate the same id and I using the column id as PRIMARY KEY, important point about PRIMARY KEY constraint is: each value needs to be unique on column is PRIMARY KEY instruction, save the first Webhook request, but next Webhook request with the same data doesn’t store because already exists.

Summary

When I convert webhook request data to md5 hash to use as the id combined by column id PRIMARY KEY instructions on the database to make sure the column id has only unique values, that way I prevent duplicated Webhook requests.

Link of project: https://github.com/tiago123456789/holding-the-load