DEV Community

loading...

Timeout with a third party library or another server - How to handle?

nav_devl profile image Naveen Honest Raj ・1 min read

This example I am stating obviously a rare scenario, but I would like to understand how this would be handled? (or they should be totally avoided?)

Consider there's two application server (AS1 and AS2) having their own databases(DB1 and DB2). An HTTP call to one AS1 will internally make HTTP call to AS2. Once AS2 validated the payload from AS1, it will write to DB2 and returns an hash, which AS1 will store in DB1.

Looks very simple to construct, but how can we handle this following scenario. Say, AS1's internal HTTP call to AS2 has a specific timeout (let's assume 15seconds). In that time period, AS2 writes to DB2 but didn't return before the timeout, so AS1 don't have the hash and now it cannot make a duplicate call to do AS2 again.

How should this be handled? I know the scenario looks vague for few of you. But let's pop up some discussion further if needed. Thanks.

Discussion (7)

pic
Editor guide
Collapse
brandinchiu profile image
Brandin Chiu

For processes that you know specifically have a long processing time, you can use a combination of synchronous and asynchronous messaging and polling.

For example, when you complete a git merge in Bitbucket, the client (website) sends a request to the api server to merge the request.

The api server replies telling the client that it has received the request and is processing.

In the meantime, Bitbucket continuously polls a different API with a processID given to it in the prior request: each time asking if the job is done yet. Once it receives an affirmative, it updates the view.

This kind of scenario can be done server to server just as easily.

I will also vouch for a message queue solution as suggested by Erick as a great alternative.

Collapse
nav_devl profile image
Naveen Honest Raj Author

Thanks Brandin. This is something new that I learnt today.

Collapse
srleyva profile image
Stephen Leyva (He/Him)

I’ve seen process forking and then utilizing a Linux signal alarm to enforce a timeout on the child process. Then what ever third party function you want to run can be the forked process. This could be a bad idea depending on things like mutations and state.

Collapse
eruizdechavez profile image
Erick Ruiz de Chavez

The solution depends mostly on the workflow, which is not described here, but an async way of solving this would be:

  • AS1 sends request to AS2 including a "job id" which can be easily generated using UUIDs.
  • AS2 works its (slow) magic and sends a request to AS1 with the result and the job UUID.
  • AS1 continues its process now.

Another approach could be to completely decouple AS1 and AS2 and use a message queue for the interactions.

  • Both AS1 and AS2 use a message queue MQ1.
  • AS1 creates a message on MQ1 in the right channel to be consumed by AS2 workers.
  • AS2 workers see a new message on their channel and consume it, doing their validations, storing in DB2 and finally pushing a new message in MQ1 in the right channel for AS1 workers.
  • AS1 workers see the message in their channel and consume with whatever logic is required.
Collapse
nav_devl profile image
Naveen Honest Raj Author

Thanks Eric. I know I didn't provide a complete picture in here, but after discussing your suggestions with my teammates, we felt the second approach you suggested goes well with our current scenario! Thanks again Eric :))

Collapse
eruizdechavez profile image
Erick Ruiz de Chavez

I am glad my (very generic) comment helped!

Collapse
davelsan profile image
David Velasco

Mine is an uneducated opinion, but I like your second approach. It is very intuitive, the kind of logic I would expect from what appears to be a loosely coupled architecture.