DEV Community

Cover image for Providers On A Laptop
Anton Brilliantov
Anton Brilliantov

Posted on

Providers On A Laptop

#testing #api #architecture

Third-party APIs raised on a laptop as emulators built from their own published API, answering by scenario - a provider that returns 5xx, one with the wrong account settings, timeouts, duplicated callbacks. A proposal; nothing is built yet.

👋 Hi, I'm Anton - a software engineer. These days I work on a platform: contracts, generated code, data paths and the tooling around them. Running notes live on my GitHub: github.com/brilliant-almazov.

This whole part is a proposal. QA, the platform and DevOps are working it out; no provider emulator is built. What I know about how our services reach their providers, I know by reading code.

This is part 8 of the series on testing cross-zone, cross-datacenter and cross-provider behaviour on a laptop. Maybe you already do it better, maybe you'd do it another way - this is how it is going, and where it stops.

The calls that leave

Part 7, Clouds On A Laptop, put cloud services on a laptop. The last part is about the calls that leave a system for someone else's: third-party providers - a results provider, a data provider with callbacks, a read-only console API. I name them only by role.

These calls are the ones tested least. A provider in a test is either the real one - slow, costly, and impossible to make fail on purpose - or a mock inside a unit test that agrees with whatever the code expected. Neither shows what happens when the provider misbehaves in the way providers actually do.

The proposal: raise every provider in the local environment as an emulator. Services call it as they would call the provider; the emulator answers by scenario. That makes provider behaviour a separate class of test scenarios, next to the failure domains of the earlier parts.

Emulated or real

Each provider gets a mode: emulated or real. Real is expensive, and for now it is forbidden on the stand.

Two halves: real - expensive, forbidden for now, cannot fail on purpose; emulated - free, answers by scenario, fails when told to

The mode is per provider, so one can be switched to real for a single check while the rest stay emulated - when the owner allows it.

Built from the provider's own API

An emulator is not written from what our code expects. It is built from the provider's official API: from its OpenAPI where the provider publishes one, otherwise from its documentation. The difference matters - an emulator built from our expectations would only ever confirm them.

A flow of dashed tiles: the provider's OpenAPI or documentation, an emulator, a scenario on top; a pill reads a proposal

The catalogue of an emulator covers more than the calls a request makes. A sending service also calls the same provider in the background - polling a balance or a tariff, re-sending a callback - and those calls belong in the emulator too, or the stand will try to reach the real provider behind everyone's back.

A scenario on top

On top of the API sits a scenario. The first ones QA has in mind:

  • the provider returns 5xx for one upstream while the others answer;
  • a provider with the wrong account settings - valid API, wrong answer;
  • timeouts;
  • duplicated callbacks - the same event delivered twice.

A strip of four panels: 5xx for one upstream; wrong account settings; a timeout; a callback delivered twice

The last is the one I would bet on finding something. Only one of our providers uses callbacks - the data provider with callbacks, by the code - and a callback that arrives twice must be accepted once.

An animation: the data provider's emulator sends a callback, then the same callback again; the service accepts the first and recognises the second as a duplicate; a pill reads a proposal

Three shapes of provider

Not every provider is hard. By role:

Provider Shape What an emulator needs
a results provider request and response the API and the scenarios
a data provider with callbacks request, then callbacks into our webhook the API, the callbacks, duplicates and delays
a read-only console API reads only the API, and a stub of whatever issues its tokens

Three tiles: a results provider, request and response; a data provider with callbacks, calls back into a webhook; a read-only console API, reads only - a simple case

Three ways to point a service at it

How a service reaches an emulator depends on how it names its provider. By the code, there are three cases, cheapest first:

  1. one environment variable - the read-only console API's base URL is an ENV; set it and the service talks to the emulator;
  2. swap the config file - a sending service keeps provider base URLs as literals in a YAML config with no ENV, but the whole config file is swappable on the stand, so its stand copy points the base URL at the emulator over http; no new ENV, no code change;
  3. a network alias and a local CA - a service whose provider URLs are generated literals, with no ENV and no swappable config, reaches the emulator through stand network aliases on the provider's hostnames, plus a stand-local CA in its trust store, so TLS still verifies.

Three numbered tiles, cheapest first: 1 one ENV; 2 swap the config file; 3 a network alias on the provider host and a local CA in the trust store; a pill reads analysis, not built

The third is the form QA already wrote down as the local provider stub in Every Verdict Carries A Trace - aliases, a local CA, an internal network with no way out, and "0 outbound connections" as the acceptance line. It is the fallback that works for any service, and the most work.

What an emulator must also stub

The read-only console API looks like the simplest case, and it is - with one catch. The service never does the provider's OAuth itself: its tokens come from a separate auth service over gRPC. So the stand needs two things, the provider emulator and a stub of that auth service.

A flow: the console service asks a token stub over gRPC, then calls the provider emulator with the token; both dashed; a pill reads also stub what issues the tokens

That is the general lesson: an emulator covers a provider, not a call. Whatever the call depends on - its tokens, its callbacks, its background pollers - comes with it.

What the world uses

None of this needs a new tool. The world has several, each from its own documentation:

  • WireMock - stubs as JSON mappings, with delays (fixedDelayMilliseconds) and faults: CONNECTION_RESET_BY_PEER, EMPTY_RESPONSE, MALFORMED_RESPONSE_CHUNK, RANDOM_DATA_THEN_CLOSE (stubbing, faults);
  • Prism - mocking and contract testing from OpenAPI v2 and v3.x, Apache-2.0, prism mock <spec>; in Docker it needs -h 0.0.0.0 (Prism);
  • Mockoon - a CLI that runs mock APIs headless, MIT, mockoon-cli start --data (Mockoon CLI);
  • Hoverfly - open-source API simulation with six modes; Capture records traffic as a proxy, and the capture is exported and re-imported for Simulate (modes, capture);
  • MockServer - a self-hosted mock for HTTP, HTTPS and gRPC, with expectations as httpRequest / httpResponse, and a proxy that records for later replay (MockServer).

A scenario in WireMock's own form:

{
  "request":  { "method": "GET", "url": "/fault" },
  "response": { "fault": "CONNECTION_RESET_BY_PEER" }
}
Enter fullscreen mode Exit fullscreen mode

A dark JSON card with a WireMock mapping: a request with method and url, a response with the fault connection reset by peer, highlighted; below, a pill reads from the WireMock docs

Five tiles: WireMock, JSON mappings and faults; Prism, from OpenAPI; Mockoon, a headless CLI; Hoverfly, capture and simulate; MockServer, expectations and record

Prism fits "built from the provider's own API" when an OpenAPI exists. WireMock fits the fault scenarios. Hoverfly and MockServer fit the later step below. Which one is chosen is not decided.

Data

The emulators would answer with synthetic data for now. Later, the plan is real responses exported from production through the admin API, so the emulator's answers have production's shape - planned.

An emulator sees what tracing missed

One side effect is worth naming. Once every provider call lands on an emulator on the stand, it is easy to see which of those calls has no client span of its own. A provider call over HTTP without a client span is an observability gap - and an emulator exposes it for free.

In a zone, under chaos

An emulator is a container. Like the cloud emulators of part 7, it can be placed in a zone of the planet from part 3, and then it falls under the same chaos: take its provider's sites down, degrade the link to it, cut it off. A provider outage becomes one more failure domain.

The series in one picture

A flow of eight tiles: one login, four failure domains, a planet on a laptop, placement in every trace, how others test, every verdict carries a trace, clouds on a laptop, providers on a laptop

Eight parts, one idea each: One Login For Every Stand, Four Failure Domains, A Planet On A Laptop, Placement In Every Trace, How Others Test Cross-Zone Locally, Every Verdict Carries A Trace, Clouds On A Laptop, and this one. The built parts are the login stand, the topology emulation and the verdict with its trace. The rest is in review, planned or proposed, and each part says which.

What I do not claim

  • Not that any provider emulator exists. This part is a proposal.
  • Not which tool is chosen. That is not decided.
  • Not that production responses are exported. That is planned.
  • Not the names of the providers or what they provide. Roles only.

What of this is method

  • Emulate from the provider's API, not from your code. An emulator built from expectations only confirms them.
  • An emulator covers a provider, not a call. Tokens, callbacks and pollers come with it.
  • Cheapest pointer first. One ENV, then a config file, then an alias and a local CA.

Cross-anything on a laptop - Part 8. This closes the series.

If you emulate third-party APIs, do you build them from the provider's spec or from your own expectations? If you test duplicated callbacks, how did you make the provider send one twice? If you record and replay provider traffic, how do you keep it fresh? Which provider would you rather see misbehave on a laptop than in production?

Top comments (0)