DEV Community: Richard Hatherall

Matching HTTP requests with WebMocks in Delphi

Richard Hatherall — Wed, 17 Mar 2021 11:26:22 +0000

If you've been following along, you'll now know how to set up a DUnitX project with WebMocks and understand how to define the HTTP response for a stubbed request. You've also seen the most basic form of request matching in the previous articles, e.g. StubRequest('GET', '/'). The StubRequest method starts with the required elements of a request match, the HTTP method GET in this example, along with the document path /. Let's take a closer look at how we can match requests with WebMocks in Delphi.

Methods and Document Paths

The HTTP method is one of the most significant components of the request. Therefore, the StubRequest method puts that upfront, along with the intended document path. They are both specified as string arguments, allowing arbitrary method and path values. RESTful endpoints can be matched as follows:

// Create Widget
WebMock.StubRequest('POST', '/widgets');

// List all Widgets
WebMock.StubRequest('GET', '/widgets');

// Show Widget 1
WebMock.StubRequest('GET', '/widgets/1');

// Update Widget 1
WebMock.StubRequest('PATCH', '/widgets/1');
WebMock.StubRequest('PUT', '/widgets/1');

// Delete Widget 1
WebMock.StubRequest('DELETE', '/widgets/1');

As you can see from these examples, there are often many similar requests that you will need to distinguish correctly based upon method and path alone.

Sometimes it is useful to catch more than a single value, and you might, for example, want to stub the update requests represented by PATCH and PUT with a single call. A simple wildcard "*" string matches any value:

WebMock.StubRequest('*', '/widgets/1');

Now, the eagle-eyed amongst you will be saying, "won't that match the GET and DELETE actions too?". Yes, it will. Most test scenarios don't need all of the actions defined simultaneously. If you need to, then it is easily achieved through the order of stub definitions. All request stubs are evaluated in the order they are defined, and the first matching stub is applied.

WebMock.StubRequest('GET', '/widgets/1');
WebMock.StubRequest('DELETE', '/widgets/1');
WebMock.StubRequest('*', '/widgets/1'); // PATCH and PUT

Another potential problem here is the ID in the URL. You might not know the exact number when writing the test, or you might want to match and respond the same to multiple identical requests for different resources. Unfortunately, the * is a simple wildcard only, and you can't combine it with a partial path. Luckily, WebMocks does allow regular-expressions in place of the path argument:

// Get Widget with an integer ID
WebMock.StubRequest('GET', TRegEx.Create('/widgets/\d+'));

Be sure to include System.RegularExpressions in your uses clause.

Matching Query Parameters

The document path is only one part of the URI. It is just as essential to be able to match by query parameters. Query parameters can be matched by name and value easily. For example, to match a specific value for a parameter supplied on the URL query:

WebMock.StubRequest('*', '*')
  .WithQueryParam('Action', 'DoSomething');

If all you want is to test for the presence of a particular query parameter, you can use a simple wildcard *:

WebMock.StubRequest('*', '*')
  .WithQueryParam('Action', '*');

Using regular expressions is also supported for matching values:

WebMock.StubRequest('*', '*')
  .WithQueryParam('Action', TRegEx.Create('List.*'));

Matching Headers

Headers provide a lot of information about a request, and you are likely going to want to match a request by header values.
Let's start with a basic example:

WebMock.StubRequest('*', '*')
  .WithHeader('content-type', 'video/mp4');

This example will match any request with a content-type header of video/mp4. Like the HTTP method and document path, the header value can be a simple wildcard *. The wildcard is useful if you want to match against the presence of the header. Again, like the document path, the header value can be specified as a regular expression. The following example is useful for matching any video content:

WebMock.StubRequest('*', '*')
  .WithHeader('content-type', TRegEx.Create('video/.+'));

If you already have a TStringList populated with headers, the WithHeaders method can be used to match against all values:

var
  LHeaders: TStringList;
begin
  LHeaders := TStringList.Create;
  LHeaders.Values['content-type'] := 'application/json';
  LHeaders.Values['accept'] := 'application/json';

  WebMock.StubRequest('*', '*')
    .WithHeaders(LHeaders);
end;

Matching Body Content

Matching by body content is where it can get a little more interesting. You can probably guess that we have a method called WithBody that does pretty much exactly what you think:

WebMock.StubRequest('*', '*')
  .WithBody('Hello');

Matching a pattern can be achieved with a regular expression.

WebMock.StubRequest('*', '*')
  .WithBody(TRegEx.Create('Hello'));

Matching an exact string or pattern is all well and good, but when testing APIs, you're most likely dealing with some form of structured data. WebMocks provides several methods for dealing with two standard formats: form-data; and JSON.

Matching Form Data

The WithFormData method matches form-data values as submitted with the content-type application/x-www-form-urlencoded.

WebMock.StubRequest('*', '*')
  .WithFormData('field', 'value');

Or, to match a RegEx pattern:

WebMock.StubRequest('*', '*')
  .WithFormData('field', TRegEx.Create('pattern'));

Matching JSON

JSON data is more complicated as it has structure and typed values. To match a JSON attribute in the example content:

{
  "name": "Chester Copperpot"
}

The following example will provide a positive match:

WebMock.StubRequest('*', '*')
  .WithJSON('name', 'Chester Copperpot');

Overloaded versions of the WithJSON method are defined to handle the data types in JSON: string, number (integer and decimal), and boolean. The JSON types of array and object are structural, and comparing them is not possible but navigating them is.

If you have a deeply nested JSON structure that you want to match values against you can provide the first argument as JSONPath. For example:

WebMock.StubRequest('*', '*')
  .WithJSON('users[0].name', 'Chester Copperpot');

Putting it all together

To demonstrate a more comprehensive example:

WebMock.StubRequest('POST', '/register')
  .WithQueryParam('source', 'http://example.com')
  .WithHeader('content-type', 'text/json')
  .WithJSON('user.email', 'ccopperpot@example.com')
  .WithJSON('user.password', 'goonie4ever')
  .WithJSON('user.name', 'Chester Copperpot')
  .ToRespond
  .WithStatus(Created);

This one example demonstrates the use of matching all HTTP message parts. While WebMocks provides an elegant and flexible interface for this, remember what the goal here is: identifying a request and responding appropriately to test some behaviour. To that end, maybe matching the document and method is enough?

If you want to verify the request's format, you should look at "assertions". Luckily I have an upcoming article all about that "Asserting HTTP requests with WebMocks in Delphi". In the meantime, try WebMocks and Delphi for yourself. The demos from this article and more are available on GitHub. Let me know how you get on in the comments below or on Twitter.

All about HTTP responses with WebMocks in Delphi

Richard Hatherall — Mon, 15 Feb 2021 11:28:18 +0000

In the previous post Testing HTTP clients in Delphi with DUnitX and WebMocks we looked at a DUnitX test's basic structure using WebMocks to stub an HTTP endpoint. At this point, it appears very simplistic and not at all like many real-world HTTP interactions. Most requests don't merely respond "OK" with no content. Let's explore how and what content we can mock, giving your test subjects scenarios allowing you to write tests asserting their behaviour.

The Defaults

When stubbing a request without specifying the response, the default is empty but successful:

HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
content-length: 0

This response differs from not stubbing the request only in status code which would be 501 Not Implemented. So, how do we customize the response? Using ToRespond is the answer. Let's take a look at what we can do by chaining a ToRespond call.

Setting Status Codes

The ToRespond function accepts an optional argument AResponseStatus allowing you to specify the response status code. Returning a status of 204 No Content is achieved with:

WebMock.StubRequest('GET', '/').ToRespond(204);

Not all of us have a comprehensive knowledge of HTTP status codes, so, thankfully ToRespond will accept an enum value instead, allowing you to have descriptive symbols in your test code without the need to add comments. Add WebMock.ResponseStatus to your uses clause, and you will be able to refer to status 204 as NoContent:

WebMock.StubRequest('GET', '/').ToRespond(NoContent);

I'll pause a moment here to point out some small syntactic sugar WebMocks provides: WithStatus. By chaining WithStatus after ToRespond the test becomes more descriptive:

WebMock.StubRequest('GET', '/')
  .ToRespond
  .WithStatus(204);

// or

WebMock.StubRequest('GET', '/')
  .ToRespond
  .WithStatus(NoContent);

Using WithStatus also provides the ability to return a custom status code and text:

WebMock.StubRequest('GET', '/')
  .ToRespond
  .WithStatus(321, 'Back In The Room');

Now we know how to change the response status. Unless you are returning 204 No Content however, you'll likely want to send some data back too. Let's take a look at setting the body content next.

Setting Body Content

As we saw earlier, the default response content is a zero-length plain-text response with the UTF-8 character set. Setting body content is done using WithBody:

WebMock.StubRequest('GET', '/')
  .ToRespond
  .WithBody('Hello');

That's all well and good for plain-text, but you're quite likely to want to return something else like JSON. That's easy enough, specify a second argument with a content-type:

WebMock.StubRequest('GET', '/')
  .ToRespond
  .WithBody('{ "key": "value" }', 'application/json');

Now we know how to respond with small snippets of data, what if we need something larger — or not easily represented in text? That's where WithBodyFile comes in. You provide a path to a file, and WebMocks will attempt to return it with the correct content type.

WebMock.StubRequest('GET', '/video')
  .ToRespond
  .WithBodyFile('video.mp4');

Again, as with WithBody, WithBodyFile accepts a second argument for content-type if the file type is not commonly recognized or ambiguous.

WebMock.StubRequest('GET', '/video')
  .ToRespond
  .WithBodyFile('video.3gp', 'video/3gpp');

Adding Headers

When testing APIs, you will, at some point, need to respond with specific headers. For example in ReST API designs, it is common to return a location header when creating a resource and responding 201 Created:

WebMock.StubRequest('POST', '/resources')
  .ToRespond
  .WithStatus(Created)
  .WithHeader('location', 'http://example.com/resources/1');

You can chain as many calls to WithHeader as you like to build up a list of headers.

WebMock.StubRequest('GET', '/resources/1')
  .ToRespond
  .WithHeader('cache-control', 'no-cache')
  .WithHeader('last-modified', 'Sat, 13 Feb 2021 12:45:26 GMT');

As a convenience, seeing as it is common in Delphi to handle HTTP headers in a TStringList, WebMocks provides WithHeaders accepting a TStrings instance. This allows you to populate a response with headers from an existing list of header values:

var
  Headers: TStringList;
begin
  Headers := TStringList.Create;
  Headers.Values['cache-control'] := 'no-cache';
  Headers.Values['last-modified'] := 'Sat, 13 Feb 2021 12:45:26 GMT';

  WebMock.StubRequest('GET', '/resources/1')
    .ToRespond
    .WithHeaders(Headers);
end;

These previous two examples are identical in behaviour. While the call to WithHeaders is a convenient short-hand if you already have a collection of headers built, I believe chaining calls to WithHeader to be more descriptive when writing tests. Ultimately its a preference, you choose what works for you.

What we've learnt so far

In the previous article, we saw how easy it is to set up a DUnitX project with WebMocks. In this article, we learnt how to customize HTTP responses with WebMocks using its HTTP-oriented interface methods. Give it a try in your projects and let me know how you get on in the comments or via Twitter. You can find the demo code on GitHub. The next part of this series will be along shortly "Matching HTTP requests with WebMocks in Delphi".

Testing HTTP clients in Delphi with DUnitX and WebMocks

Richard Hatherall — Tue, 09 Feb 2021 09:54:24 +0000

Delphi makes writing HTTP client code easy. In Delphi, there are many choices to build on when writing an HTTP client, including two bundled options: Indy, and System.Net. It doesn't matter which way you go, the problems presented when testing it is the same — HTTP clients need a server with which to interact. If you don't control the server, you can't control the outcome. If you can't control the outcome, how do you know the results your test should produce?

You might solve some of these issues by testing against a sandbox or testing environment. If you don't have these already, it can be a lot of work to set this up, and maybe it's not you providing the service? You might be depending on another team for building your backend. It might not be ready yet, and all you have is their documentation?

Another solution is to run an in-process HTTP server within your test project. You get fine-grained control over the HTTP server, but the setup code can be a lot of effort hampering progress. Also, unless you carefully craft your test code, an HTTP server's setup code may obscure your tests intentions.

With these issues in mind, I set about solving them by writing WebMocks for Delphi, freely available and open-source on GitHub and available through GetIt. I designed WebMocks to integrate with DUnitX, the popular unit testing framework, also useful for running integration tests. WebMocks provides an in-process HTTP server with responses defined by an HTTP domain-specific-language interface.

WebMocks allows you to focus on what's essential to your tests.

Getting Started

You should make sure you have both DUnitX and WebMocks installed.
If you installed RAD Studio/Delphi via the web installer, installing DUnitX is as simple as checking the box DUnit Unit Testing Frameworks under Additional Options in the Platforms and Extensions Manager.

The quickest and easiest way to install WebMocks is via the GetIt Package Manager. Search for WebMocks and click Install.

Let's take a look at the most basic usage of WebMocks within a DUnitX test. We will use the THTTPClient as the "class under test" so we don't need to write any additional classes. Testing an existing class like this will allow us to focus on the behaviour of WebMocks.

Now, we can start setting up our project. Create a new DUnitX Project found under File > New > Other.. > DUnitX.

Make sure Create Test Unit and Create Setup and TearDown Methods are selected. Give the TestFixture Class a name, such as TBasicDemoTests and click "OK". Save your project and unit before continuing (Shift+Ctrl+S).

Next, we need to configure WebMocks in our TestFixture class: Add a private member WebMock: TWebMock to the TestFixture class.

Declare a member for the "class under test" too — Client: THTTPClient in our example.

Add the missing dependencies by adding System.Net.HttpClient and WebMock to your uses block. You should initialise and clean up these dependencies in your Setup and TearDown procedures.

You should now have a unit that looks like:

Now you're ready to write some tests.

Writing your first test with WebMocks

Let's add the most basic example of a WebMock stubbed request.

Start by adding a new test procedure in the public section and complete your class by pressing Shift+Ctrl+C.

[Test]
procedure Get_WhenStubbed_ReturnsOK;

We might write our test like this:

We need a local variable to keep track of our HTTP interactions — LResponse: IHTTPResponse.
Then we stub a "GET" request for the root path ("/").
Then we call the client to perform a GET request and store the response.
Finally, we check to see if the client received an "OK" response, represented by the HTTP status code 200.

procedure TBasicDemoTests.Get_WhenStubbed_ReturnsOK;
var
  LResponse: IHTTPResponse;
begin
  WebMock.StubRequest('GET', '/');

  LResponse := Client.Get(WebMock.URLFor('/'));

  Assert.AreEqual(200, LResponse.StatusCode);
end;

Let's run it without debugging (Shift+Ctrl+F9) and see what happens.

Now, if everything worked correctly, you should finally be able to see your test results:

Give WebMocks a try!

WebMocks makes the tricky task of writing tests for HTTP clients simple. In the next in this series we learn all about HTTP responses with WebMocks in Delphi. You can find the code for this demo on GitHub. You can reach out to me on Twitter @RichHatherall.