API Integrations: Building Client Classes

Code speaks louder than words, so we'll go through an example integration with a weather service, called WhatWeather, that returns current weather conditions by latitude and longitude or city name. Let's describe that imaginary service briefly and then proceed with the integration.

WhatWeather API Overview

WhatWeather API is accessible over HTTP and offers one endpoint that can be used in two ways:

1. /weather?lat=...&lon=... returns weather conditions at the given coordinates.
2. /weather?name=... returns weather conditions in the given city.

The API responds with a 200 OK and a JSON object containing two keys: "temperature" and "humidity". Authentication is handled via a token passed in a custom header named X-API-Key. Any other status code, even in the 200-299 range, can be assumed to be an error.

Step 1: Build Client Class

First, we must define the client class interface. Even though the WhatWeather API offers one endpoint it makes sense to have two separate methods: #weather_by_coordinates and #weather_by_name. We should strive for maximum code clarity, not mirroring the structure of the underlying API.

Second, we need a namespace (a Ruby module) for all weather-related code. Its name shouldn't refer to a specific API provider to keep the code generic and avoid coupling to a specific vendor. WeatherService seems like a reasonable name.

Third, we need a class representing results returned by the weather service. In simple cases, a Struct should be enough, like the one below:

WeatherService::Weather = Struct.new(:temperature, :humidity)

In more complex cases, a custom class might be needed. It's important to make them independent from the actual API provider -- otherwise, it might become an octopus whose tentacles reach to the very depths of the app. For example, the client class should parse and convert HTTP responses into instances of WeatherService::Weather and similar classes so that if their API change there's only one place in our code base that needs to be adapted.

We're ready for the fourth and final step: implementing the actual client class. We can use a third-party client gem or interact with the API via HTTP, GraphQL or another protocol. Whatever method we use, it's important to keep the point from the previous paragraph in mind -- don't let the actual client implementation leak through the client class and flood the rest of the code.

The example client below takes an HTTP client object via the constructor. Keep in mind it's not production ready as its input validation, error handling, and response parsing is rudimentary but is enough to illustrate the idea.

class WeatherService::WhatWeather
  def initialize(http_client:, api_key:)
    @http_client = http_client
    @api_key = api_key
  end

  def weather_by_coordinates(coordinates)
    result = http_client.get(
      "/api/weather",
      params: { lat: coordinates[0], lon: coordinates[1] },
      headers: { "X-API-Key": api_key }
    )
    return [:error, reason: :invalid_response] if result.status != 200

    build_weather(result)
  end

  def weather_by_name(name)
    result = http_client.get(
      "/api/weather",
      params: { name: name },
      headers: { "X-API-Key": api_key }
    )
    return [:error, reason: :invalid_response] if result.status != 200

    build_weather(result)
  end

  private

  attr_reader :http_client, :api_key

  def build_weather(result)
    [
      :ok,
      WeatherService::Weather(
        temperature: result.json.fetch("temperature"),
        humidity: result.json.fetch("humidity")
      )
    ]
  rescue KeyError
    [:error, reason: :invalid_response]
  end
end

Much more can be said about implementing client classes but the above example is enough to proceed to the next step: integrating the client with the app.

Step 2: Integrate Client into App

For simplicity, we assume there's one client instance in the whole app. This assumption is safe when the class is stateless but might need to be revisited in more complex scenarios. It doesn't affect the gist of the method, though.

The client must be instantiated early in the boot process and be accessible from various places throughout the code. Ideally, it'd be possible to inject it as a dependency to each controller it but sadly it doesn't seem to be possible (or at least it's non-obvious). We're left with using some kind of global state.

I used to rely on module attributes (like WeatherService.client) but have changed to using actual global variables, e.g. $weather_service. I find it much cleaner than a de facto global masquerading as a method call.

The best place to instantiate the client is a Rails initializer. The weather service client can be instantiated in config/initializers/weather_service.rb:

Rails.application.config.to_prepare do
  $weather_service = WeatherService::WhatWeather.new(
    http_client: HttpClient.new,
    api_key: ENV.fetch("WHATEVER_WEATHER_API_KEY")
  )
end

We can now use $weather_service whenever we need to make an API call. That's not the end of the story, though, as there are benefits to be gained in testing and development.

Step 3: Testing

The test suite shouldn't depend on any third-party APIs as this would make it flaky and dependent on Internet access. Fortunately, with a global client instance it's trivial to mock it.

The first step is mocking the client before each test case and cleaning it up afterwards. We're using MiniTest and Minitest::Mock below but the same idea applies to other test and mocking frameworks.

class ActiveSupport::TestCase
  setup do
    @old_weather_service = $weather_service
    $weather_service = Minitest::Mock.new
  end

  teardown do
    $weather_service.verify
    $weather_service = @old_weather_service
  end
end

Now, each test case can declare weather service methods it needs along with their desired return values. For example, the code below will make the weather service return a predefined value for NYC:

$weather_service.mock(
  :weather_by_name,
  WeatherService::Result.new(temperature: 80, humidity: 63),
  ["New York City"]
)

Creating test cases for edge cases or errors is a matter of making the mock service return an appropriate value or an exception.

Let's turn our attention to the benefit this approach can yield in development.

Step 4: Development

Interacting with the app is an essential component of the development feedback loop. However, hitting a real API in development can make it difficult to explore edge cases (unlikely return values or errors). For example, we can't force WhatWeather to lie about the temperature in New York. That's another situation where our approach comes in handy.

The solution is a development-specific client class returning stubbed responses. For example:

def weather_by_name(name)
  case name
  in "New York City" then Result.new(temperature: 5, humidity: 50)
  else Result.new(temperature: 65, humidity: 47)
  end
end

Other edge cases, including errors and exceptions, can be implemented in the same way.

Switching between the real and stub API client can be a matter of configuration. For example, the client initializer file can be modified to read:

Rails.application.config.to_prepare do
  case ENV.fetch("WHATWEATHER_CLIENT", "production")
  in "production"
    $weather_service = WeatherService::WhatWeather.new(
      http_client: HttpClient.new,
      api_key: ENV.fetch("WHATEVER_WEATHER_API_KEY")
    )
  in "development"
    $weather_service = WeatherService::StubWhatWeather.new
  end
end

Now, the client can be chosen by setting WHATWEATHER_CLIENT to "production" or "development".

Further Steps

We outlined a high-level approach but have skipped many implementation details that need to be addressed in a real production system. For example:

• Establishing client configuration patterns. What variables are needed? How are we going to configure stub and production clients?
• Making it easy to switch between stub and production clients.
• Client input validation, response parsing and error reporting.
• Creating a set of specialized client tests that make use of the real API to ensure the client is working correctly.
• Maintaining multiple clients (or service objects in general) and their relationships.

These decisions can only be made after taking a specific project's circumstances into account. We also need to be aware of the drawbacks of this approach in order to manage the risks and challenges they introduce:

• Global state makes it possible to use the client from anywhere without an explicit dependency. It's easy to use the client from a model which, in general, should be discouraged.
• Complex integrations, like Stripe, may require a lot of custom result parsing and conversion code. It may make sense to build some kind of a DSL to accomplish that, increasing implementation complexity, or let it flow through the client to the rest of the app, increasing coupling. It can be a tough call.
• Mocking the API can result in a situation where the test suite is green but production is not.

Closing Thoughts

Calling third-party APIs is always going to be more difficult than ordinary method calls. However, hiding the API behind a well-defined interface that's entirely within our control makes it much easier to manage the complexity resulting from the integration. Its bulk will live in the client class and won't spill over to the rest of the code base making testing and stubbing easy.

If you enjoyed reading the article then I recommend you take a look at other articles on my website.