External Service testing in Phoenix

#phoenix #elixir

Getting Started

Before going to the detail, let's me share a little bit about our system - we're using Elixir-Phoenix framework to build a backend system and from the requirement, we need to build an API that can support our front-end client (ReactJS/React-Native) upload files to AWS_S3.

In Phoenix framework, we used an AWS client's hex package called ex_aws to upload files to S3. Basically, the controller code will be:

unique_filename = "#{UUID.uuid4(:hex)}_#{filename}"

{:ok, image_binary} = filepath |> File.read()

Application.get_env(:my_app, :image_bucket_name) |> ExAws.S3.put_object(unique_filename, image_binary) |> ExAws.request!()

ExAWS.request!() will return the status_code is 200 if uploading is successful, otherwise, it will return another status_code.

Uploading module

As usual, we moved uploading code from UploadController to a UploadService module - this will make the controller looks more readable and easy to write the test.

defmodule MyApp.UploadController do
  use MyApp, :controller

  @upload_service Application.get_env(:my_app, :upload_service)
  import UploadService

  def upload_image(conn, params) don
    case upload(params) do
      {:ok, filename} ->
        json(conn, %{url: resolve_url(filename), error: nil})

      {:error, reason} ->
        json(conn, %{url: nil, error: reason})
    end
  end
end

defmodule UploadService do
  def upload(params) do
    %{"file" => %{filename: filename, path: filepath}} = params
    unique_filename = "#{UUID.uuid4(:hex)}_#{filename}"
    {:ok, image_binary} = filepath |> File.read()

    case Application.get_env(:my_app, :image_bucket_name)
         |> ExAws.S3.put_object(unique_filename, image_binary)
         |> ExAws.request!() do
      %{status_code: 200} ->
        {:ok, unique_filename}

      _ ->
        {:error, "can't upload"}
    end
  end
end

Write test for Uploading API

When integrating with external services we want to make sure our test suite isn’t hitting any 3rd party services. Our tests should run in isolation. ThoughBot

With our UploadService module, we don’t need to test the request to S3 because the package itself already test. So, we only need to mock module to return ok or error response.

Setup the corresponding modules for different environments

The development and production environment will use the real UploadService and test environment will use the UploadService.Mock.

# dev.exs and prod.exs
config :my_app, :upload_service, UploadService

# test.exs
config :my_app, :upload_service, UploadService.Mock

And then, we changed a bit in the controller to dynamic loading the corresponding modules.

defmodule MyApp.UploadController do
  use MyApp, :controller

  @upload_service Application.get_env(:my_app, :upload_service)
  @upload_image_token Application.get_env(:my_app, :upload_image_token)

  def @upload_image_token.upload_image(conn, params) don
    case upload(params) do
      {:ok, filename} ->
        json(conn, %{url: resolve_url(filename), error: nil})

      {:error, reason} ->
        json(conn, %{url: nil, error: reason})
    end
  end
end

Next, we will create UploadService.Mock module.

Create a Mocking module

defmodule UploadService.Mock do
  def upload(%{"file" => %{
    "filename" => "success", "path" => _path
  }}) do
    {:ok, "your-file.png"}
  end

  def upload(%{"file" => %{
    "filename" => "fail", "path" => _path
  }}) do
    {:error, "can't upload"}
  end
end

We used pattern-matching with different filename to return ok or error.

Write the test

And now the test will not difficult to write.

test "uploads success", %{image_token: image_token, conn: conn} do
  conn = put_req_header(conn, "authorization", image_token)
  file = %{
    "filename" => "success",
    "path" => "/your/image/path"
  }
  response =
    post(
      conn,
      "/upload",
      %{
        "file" => file
      }
    )

  assert %{"error" => nil, "url" => _url} = json_response(response, 200)
end

test "uploads fail", %{image_token: image_token, conn: conn} do
  conn = put_req_header(conn, "authorization", image_token)
  file = %{
    "filename" => "fail",
    "path" => "/your/image/path"
  }
  response =
    post(
      conn,
      "/upload",
      %{
        "file" => file
      }
    )

  assert %{"error" => "can't upload", "url" => nil} = json_response(response, 200)
end

Conclusion

This is the way how we write test for API without hitting to external services. It could not a good way, so if you guys have any idea, fell free to comment.

Thank you!

Top comments (6)

rhymes • May 14 '18 • Edited

Hi Vincent, what about using something like exvcr ?

I have no knowledge or experience in Elixir but I often use a "vcr" library with other languages. It allows me to actually hit the server at least once (following requests are mocked) and you can also refresh all the interactions from time to time to make sure that the real server is responding with the expected data.

I obviously agree that trusting the Elixir S3 library for the response is a good ideae but this way you can also make sure the HTTP traffic is consistent

Vincent Nguyen • May 14 '18

I heard about exvcr, it's another approach - can work but technically, I don't want to hit the server and ‘replay’ it back during tests.

From the awesome thoughtbot post, there're considerations when using VCR:

Communication on how cassettes are shared with other developers.
Needs the external service to be available for the first test run
Difficult to simulate errors

I also realized the mocking module looks very simple, I should handle multiple response statuses.

rhymes • May 14 '18

I do mock errors sometimes or I just call the server with invalid data and register the possible error messages.

Some APIs have different error messages for different cases and for me it's faster to record them than to actually figure out how to mock all of them.

My issue with API mocks is that they can be difficult to maintain updated, it's not this case because S3 is not going to change at all but for APIs where there's no client, just the documentation, I am way faster if I record it once instead of using mocks.

To be able to mock those, you still need to wire up Postman, hit them once, check the response, mock such response, so why not do that using VCR?