My $646 mistake with PHP

#php #laravel #programming #googlecloud

Yesterday I learned a very valuable lesson when iterating a while loop using the Google Places API - Web Service that cost me $646. That's right, I pinged the Places API 17,000 times in one hour 😳 🤦🏻‍♂️. So I decided to write an article today documenting my mistake in the hopes that this prevents someone else from running into this same issue. Let's dive in.

What Happened?
Well...I'm currently building a Web and Mobile Application that uses the Google Places API and I needed to return results using the nearby search. The Google API only returns 20 results with a max of 60 which you can get by using their version of pagination. So let's go through the code and explain what went wrong.

// This is the initial request that we send to get the initial 20 results.
        $response = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=' . $request->lat .'%2C'. $request->lng.'&radius='.$request->radius.'&type=restaurant&key=' . env('GOOGLE_MAP_KEY'))->json();

        // We push those results to our google_places collection
        $this->google_places->push(...$response['results']);

        // Here we check the status of the response, if it's ok we continue.
        if ($response['status'] == 'OK') {
            // If the response has a next page token we loop through until it doesn't, getting the next 20 results.
            while ($response['next_page_token']) {
                // The google places api has a delay from when the next_page_token is created and those results are actually available.
                // So we need to create a delay before we try and grab the next results otherwise we will get an INVALID_REQUEST response.
                sleep(2);

                // Now we get the next 20 results.
                $new_results = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?pagetoken=' . $response['next_page_token'] . '&key=' . env('GOOGLE_MAP_KEY'))->json();

                // Then we push those results to our collection.
                $this->google_places->push(...$new_results['results']);
            }
        }

Were you able to figure out where I went wrong? A while loop in PHP will run until it gets a false back, the value that I put in the while loop always returns true and therefore sends the while loop into an infinite loop. I was using the $response['next_page_token'] from the initial request which will always return true.

How did I fix this issue?

$response = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=' . $request->lat .'%2C'. $request->lng.'&radius='.$request->radius.'&type=restaurant&key=' . env('GOOGLE_MAP_KEY'))->json();

        $this->google_places->push(...$response['results']);

        // We check to see if the above response includes the key next_page_token, if it does we set the next_page_token property to true
        // This is super important, this is how we're going to ensure our loop doesn't run infinitely. 
        if (array_key_exists('next_page_token', $response)) {
            $this->next_page_token = true;
        }

        if ($response['status'] == 'OK') {
            // For the while loop we check to see if the next_page_token is true, it will end when it returns false.
            while ($this->next_page_token) {
                sleep(2);

                $new_results = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?pagetoken=' . $response['next_page_token'] . '&key=' . env('GOOGLE_MAP_KEY'))->json();

                $this->google_places->push(...$new_results['results']);

                // This is also super important, here we check to see if next_page_token does not exist or if we're getting an invalid request response.
                // If we do we want to set the next_page_token to false so that the loop ends.
                if (!array_key_exists('next_page_token', $new_results) || $new_results['status'] == 'INVALID_REQUEST') {
                    $this->next_page_token = false;
                }
            }
        }

        // Being that the Google API only returns 60 results that means the loop should only have to run a max of 2 times.
        // So you could create a $count variable and add that like so while($this->next_page_token || $count <= 2) and increment the count each time.
        // This will act as a safety in case you accidentally throw a forever truthy statement in there like I did.

Conclusion
Be cautious when working with APIs that charge and running them inside a loop. As a precaution you can always create a $count variable and stop the loop at a set count in case you make a mistake. This is what I will do for now on just to ensure I don't do this again. Hopefully this has helped you or someone else from making the same mistake. ✌🏻

Oldest comments (21)

Andrew McCombe • Jan 21 '22 • Edited

I did a similar thing but with the lookup taking place in a background job using SQS. A bug in the code meant the job always failed and got retried.

£20,000 later...

Luckily Google were cool about it.

Lesson: set decent retry counts on your Laravel worker jobs.

Ricardo Boss • Jan 21 '22

Looks like you also need to use the correct next_key_token. You are using the initial next_key_token from the first request in your URL.

Jonathon Ringeisen • Jan 21 '22

Good catch! I'm actually not using the Google Web Service anymore so I won't be using this code but, still good to know.

ColdFox • Jan 21 '22

It happened in my company as well, after a 2k€ bill we set the max limit equal to the free tier usage. This way we only pay if there's a need to increase the limit in any given month.

Kunal Rajput • Jan 21 '22

I did similar thing, causing 500$ for Google timezone api.

I just sent them a email explaining them it was a mistake, they gave me waiver off on everything lol

Peter Fox • Jan 21 '22

I think the mistake here is more, always write unit tests when you can. A test running this function could easily find that a while loop is never completing.

Equally never hurts to put a limit on a loop for an API so it never iterates more than say 50 times etc.

Anwar • Jan 22 '22

Good idea, and since I see some Laravel code up there, just mock the API return so that you never really consume credits (and you still find out infinite loops, but for free)