DEV Community

Cover image for Fail Fast and Fail Often: Handling API Errors at Scale

Fail Fast and Fail Often: Handling API Errors at Scale

Akshay Nathan on September 18, 2019

At Monolist, we’re building the software engineer’s ideal inbox. Our users depend on us to surface all relevant and actionable tasks and context ac...
Read full post
Collapse
 
pbalaban profile image
Petr Balaban •

Hello Akshay,

I'm just curious, why you can not extract the logic for comments fetching and action item creation to a separate Sidekiq job? In that case when client.get_pull_requests API call will be finished successfully the Sidekiq job also will be finished

Collapse
 
allanklaus profile image
Allan Klaus •

Hey Akshay.

I have one suggestion.One sentry you can configure what exceptions you don't want to log just add it to config of sentry.

Raven.configure do |config|
  config.dns = ''
  config.excluded_exceptions += ['Gitplace::ConnectionTimeout']
end

I would try the approach of Peter too. To use another sidekiqjobs to avoid some api calls.

Example...

class GetPullRequestsJob
  def perform(user, client)
    pull_requests = client.get_pull_requests
    GetCommentsJob.perform_async(client, user, pull_requests)
  end
end

class GetCommentsJob
  def perform(client, user, pull_requests)
    time = user.gitplace_last_sync

    pull_requests.each do |pull_request|
      next if pull_request.created_at < time  # this guy could be in a reject. pull_requests.reject {}.each

      comments = client.get_pull_request_comments(pull_request)
      CreateCommentsJob.perform_async(user, pull_request, comments)

      time = pull_request.created_at
    end
  ensure
    user.update!({ gitplace_last_sync: time })
  end
end

class CreateCommentsJob
  def perform(user, pull_request, comments)
    create_action_item(user, pull_request, comments)
  end
end
Collapse
 
darkwiiplayer profile image
𒎏Wii 🏳️‍⚧️ •

The title reminds me of thing 20 of the book 97 Things every Programmer should Know. It's a very different concept on the surface, but it also comes down to wasting less resources on things that are wrong in some sense, except it's the developers time instead of some servers processing power :)