DEV Community

hemantkhadun
hemantkhadun

Posted on • Updated on

Handling big API request with small execution time

Hello,

I am a web developer, using wordpress as a base framework to manage and handle boats that i get from an API,

The api has multiple endpoints to get all the data for each boat.
For example,
1 endpoint will give you all the data of one boat but all the data are given in IDs, which means you need to map these IDs with another endpoint from their API :
Boat :

{"boats":{"checkinDays":"monday","locationId":"2301","RegionId":"1283","CountryId":"1283","EquipmentId":"12","ServiceId":"123"}}

After parsing and trying to save all the data in the database, the server get overloaded by memory usage and the update (Cron task) crashed the server.

To handle this problem i've used "https://github.com/halaxa/json-machine" to decrease the memory usage and "https://github.com/spatie/async" to decrease the time taken for the "update process" to complete

Note :
Before updating any boats, i run a Cron task that will hit the API endpoints then save the data as json file in my server. ( it's faster and uses less memory)

Kind regards,
Hemant.

Top comments (0)