EnvelopeNetwork, or how to abstract Alamofire, and provide strongly-typed mock responses in unit-tests + RxSwift bindings.

Introduction

This article opens a series of technical posts describing Envelope framework, which aims at providing modern protocol-based abstraction layers over several existing foundational frameworks, including UIKit, Alamofire, Realm, and others, that make unit-testing a first-class citizen™. Every set of APIs and extensions is organized as a separate framework under the Envelope umbrella project. In each article, I’ll provide a rationale, discuss design choices, and give examples using the new APIs. While I will be mostly focusing on Swift implementation, the discussed ideas could be applied in wider contexts.

I'm going to demonstrate how the application of a modern Swift protocol-based programming approach, S.O.L.I.D. programming principles and dependency injection (DI) pattern allowed to solve a cumbersome task of writing perfectly (and easily) unit-testable networking code, without having to introduce test-specific code to the production code. I would like to promote the thinking of testing as a first-class citizen that helps driving design decisions during each stage of application or framework development, which is vital to building robust scalable applications.

Examples provided in the articles are based on EnvelopeNetwork framework under the Envelope project. The project also contains a playground with more examples and unit tests. The project itself has emerged as a result of my work on OkiDokiMessenger app during 2016-2017, which has benefited a lot from the careful application of the concepts mentioned above.

Defining protocols of the networking layer

What would you normally do to test a function that makes a network request? You would invoke this function from a unit test and supply a mock response to validate function’s logic.

Ability to inject mocked responses should be an integral part of the networking library in order to make writing unit tests both possible and easy. Unfortunately, not even major libraries, like Alamofire, provide this possibility naturally (see, e.g., this issue). The answer on the SO suggests implementing a variant of the Response class that has built-in support for returning mocked responses, when enabled by unit tests. There are other frameworks that are built upon this concept, for example, TRON, which has the ability to provide mock responses to the caller. This approach, however, mixes tests-specific code with the production code, instead of abstracting it away, which, in turn, makes production code less robust, and harder to maintain.

Another solution would be to mock the response at the networking stack level when running unit-tests, for example, with OHHTTPStubs library, but that only allows mocking the raw response data, which is clumsy, not type-safe, and also hard to maintain as the number of unit tests grow, and APIs and data structures tend to change.

It’s also worth mentioning here that usually, network response handlers, even in the most well-structured and granularly broken-down application architectures, where API calls are encapsulated in very thin wrapper classes, contain several repeated bits of logic nevertheless, like parameters and request body encoding logic, response validation and error-checking logic, and response deserialization logic. A good unit test should test all these pieces of logic separately, making sure that the request is properly built given the variance of inputs, the response is properly checked for error conditions, and the response body is deserialized. Writing unit-tests that check all these conditions using the approaches mentioned above, like providing raw network responses, is a time-consuming, and not very inspiring task. Imagine there are 20 API methods each requiring more or less similar testing. Maintaining unit tests written in such a way is even a bigger problem. Such approach definitely won’t scale.

A modern design strategy, allowing to solve this challenge, is to hide all implementation details behind a cleanly defined protocols layer. When the code is invoked from the production environment, a real networking layer implementation is instantiated by the application bootstrapper, and gets dependency-injected into the API wrapper. The test suite, on the other hand, instantiates a “mock” networking layer, which does not actually make any network calls, but returns mock responses that are being set up in unit-tests.

Another consideration when designing both the abstraction layer and test-side mocking mechanisms is that clients of that networking library are likely to prefer strongly-typed deserialized objects, not raw data or JSON responses. While the library could well provide means to return strongly-typed deserialized objects to its callers, the mocking layer should also provide means for unit tests to instantiate those same types, and provide them as mocked responses to the class under test, instead of dealing with raw responses or JSON data.

Writing a full-scale networking library utilizing URLSession, that would satisfy the requirements above, is not a trivial task on its own. A quicker way would be to use one of the existing libraries and add a protocol-based abstraction layer on the top of it, that would allow using all the features of the library while solving the testability issues. For my work I’ve chosen Alamofire framework as the network foundation, because it provides means to simplify and abstract away repeated cumbersome tasks, like encoding request parameters, validating and deserializing responses. We will design a protocol-based abstraction layer providing a similar interface (ideally that would allow drop-in replacement for existing client code), and also provide an implementation of the mocking layer that would allow writing easy-to-maintain unit tests while leveraging the rest of the Alamofire framework’s power.

Let’s start with reviewing Alamofire framework architecture first. Its top object is SessionManager, which instantiates a URLSession with configurable parameters. This object then allows issuing requests against the underlying URLSession instance. Each request is an instance of either DataRequest, DownloadDataRequest, or UploadDataRequest, and allows to control its lifecycle and get raw response data. On top of that resides a set of response serializers that convert raw responses to typed ones.

So, let’s start defining our own top-level networking abstraction layer protocol:

protocol Networking {
}

Whenever a class needs to use the networking layer, it can have it via dependency injection, so that the calling code has full control over which instance of the networking layer this class will be using:

final class TwitterAPIService {
    private let network: Networking
    init(network: Networking) {
        self.network = network
    }
}

Alamofire’s SessionManager exposes a few functions that create network requests. Let’s model our networking layer after it, so, ideally, we can have a simple drop-in replacement of the Alamofire code in the future. Let’s declare request() function as this:

protocol Networking {
   func request(
       _ url: URLConvertible,
       method: HTTPMethod,
       parameters: Parameters?,
       encoding: ParameterEncoding,
       headers: HTTPHeaders?)
       -> NetworkRequesting
}

Here we’re using types defined in Alamofire framework itself (e.g., URLConvertible, HTTPMethod, etc), which is OK for our purpose of creating a protocol-based abstraction over Alamofire, since these are already either protocol types or value types.

The request() function returns an instance of NetworkRequesting type. This type abstracts away the request details, including validation, progress reporting, and response serialization:

protocol NetworkRequesting {
   var request: URLRequest? { get }
   var response: HTTPURLResponse? { get }

   func cancel()

   @discardableResult
   func progress(
       queue: DispatchQueue,
       progressHandler: @escaping Request.ProgressHandler)
       -> Self

   @discardableResult
   func response<T: DataResponseSerializerProtocol>(
       queue: DispatchQueue,
       responseSerializer: T,
       completionHandler: @escaping (DataResponse<T.SerializedObject>) -> Void)
       -> Self

   @discardableResult
   func validate(validation: @escaping DataRequest.Validation)
       -> Self
}

This protocol is also modeled after Alamofire’s DataRequest class, which will allow simple drop-in replacement in the future. The most interesting function, response(), takes an instance of a generic response serializer, and reports the typed result value via the completionHandler callback.

With this in place, we can already start writing client code, for example, to search tweets:

struct TweetsResponse: Decodable {
    // ...
}

protocol TwitterAPIServicing {
    func searchTweets(q: String, responseCallback: @escaping (Result<TweetsResponse>) -> ())
}

class TwitterAPIService: TwitterAPIServicing {
    struct Configuration {
        let endpointUrl: URL
        static func defaultConfiguration() -> Configuration {
            return Configuration(
                endpointUrl: URL(string: "https://api.twitter.com/1.1/search/tweets.json")!
            )
        }
    }

    private let network: Networking
    private let configuration: Configuration

    init(network: Networking,
         configuration: Configuration = Configuration.defaultConfiguration()) {
        self.network = network
        self.configuration = configuration
    }

    // MARK: - TwitterAPIServicing

    func searchTweets(q: String, responseCallback: @escaping (Result<TweetsResponse>) -> ()) {
        network
            .request(configuration.endpointUrl, method: .get, parameters: ["q": q], encoding: URLEncoding.queryString, headers: nil)
            .response(queue: DispatchQueue.main, responseSerializer: DataRequest.dataResponseSerializer(), completionHandler: { (dataResponse: DataResponse<Data>) in
                switch dataResponse.result {
                case .success(let responseData):
                    do {
                        // read json from network response
                        let tweetsResponse: TweetsResponse = try JSONDecoder().decode(TweetsResponse.self, from: responseData)
                        // deserialize typed object from json
                        responseCallback(.success(tweetsResponse))
                    } catch {
                        responseCallback(.failure(error))
                    }
                case .failure(let error):
                    responseCallback(.failure(error))
                }
            })
    }
}

Here we’re making use of the Alamofire data response serializer, and are decoding JSON ourselves. This is going to be a repetitive task, and the client code will benefit if the library provides a generic solution for it. This can be achieved with a custom response serializer class, and a generic extension function in the NetworkRequesting protocol:

public final class CodableSerializer<T: Decodable>: DataResponseSerializerProtocol {

    // MARK: - DataResponseSerializerProtocol
    public typealias SerializedObject = T

    public var serializeResponse: (URLRequest?, HTTPURLResponse?, Data?, Error?) -> Result<SerializedObject> {
        return { (request: URLRequest?, response: HTTPURLResponse?, data: Data?, error: Error?) -> Result<SerializedObject> in
            let result = Request.serializeResponseData(response: response, data: data, error: error)
            switch result {
            case .success(let data):
                do {
                    let decodedObject = try JSONDecoder().decode(T.self, from: data)
                    return .success(decodedObject)
                } catch {
                    return .failure(AFError.responseSerializationFailed(reason: .jsonSerializationFailed(error: error)))
                }
            case .failure(let error):
                return .failure(error)
            }
        }
    }
}

public extension NetworkRequesting {

    @discardableResult
    func responseObject<T: Decodable>(
        queue: DispatchQueue = DispatchQueue.main,
        completionHandler: @escaping (DataResponse<T>) -> Void) -> Self {

        return response(queue: queue, responseSerializer: CodableSerializer<T>()) { (dataResponse: DataResponse<T>) in
            completionHandler(dataResponse)
        }
    }
}

With this extension in place, and a few other protocol extensions, the TwitterAPIService function will start to look like this:

    // …
    func searchTweets(q: String, responseCallback: @escaping (Result<TweetsResponse>) -> ()) {
        network
            .get(configuration.endpointUrl, parameters: ["q": q])
            .responseObject { (response: DataResponse<TweetsResponse>) in
                responseCallback(response.result)
        }
    }

Pretty neat, taking into account that there’s no actual implementation of the Networking and NetworkRequesting protocols in place yet, huh?

Providing default implementation using Alamofire

Providing the default implementation of the networking layer is simple enough, as the protocol layers were modeled after the public Alamofire class interface.

public final class AlamofireNetwork: Networking {
    private let alamofireSessionManager: SessionManager
    public init(alamofireSessionManager: SessionManager) {
        self.alamofireSessionManager = alamofireSessionManager
    }

    // MARK: - Networking
    public func request(
        _ url: URLConvertible,
        method: HTTPMethod,
        parameters: Parameters?,
        encoding: ParameterEncoding,
        headers: HTTPHeaders?)
        -> NetworkRequesting {

        let alamofireRequest = alamofireSessionManager.request(url, method: method, parameters: parameters, encoding: encoding, headers: headers)
        return AlamofireNetworkRequest(alamofireRequest: alamofireRequest)
    }
}

public class AlamofireNetworkRequest: NetworkRequesting {
    private let alamofireRequest: DataRequest
    public init(alamofireRequest: DataRequest) {
        self.alamofireRequest = alamofireRequest
    }

    // MARK: - NetworkRequesting
    public final var request: URLRequest? { return alamofireRequest.request }
    public final var response: HTTPURLResponse? { return alamofireRequest.response }

    public final func cancel() {
        alamofireRequest.cancel()
    }

    @discardableResult
    public final func progress(
        queue: DispatchQueue,
        progressHandler: @escaping Request.ProgressHandler)
        -> Self {

        alamofireRequest.downloadProgress(queue: queue, closure: progressHandler)

        return self
    }

    @discardableResult
    public final func response<T: DataResponseSerializerProtocol>(
        queue: DispatchQueue,
        responseSerializer: T,
        completionHandler: @escaping (DataResponse<T.SerializedObject>) -> Void)
        -> Self {

        alamofireRequest
            .response(queue: queue, responseSerializer: responseSerializer, completionHandler: completionHandler)
            .validate()

        return self
    }

    @discardableResult
    public final func validate(validation: @escaping DataRequest.Validation)
        -> Self {

        alamofireRequest.validate(validation)

        return self
    }
}

The application bootstrapper instantiates the networking layer by creating and configuring Alamofire’s SessionManager, and creating an AlamofireNetwork instance with it:

    fileprivate static func defaultNetwork() -> Networking {

        let urlSessionConfiguration: URLSessionConfiguration
        // initialize URLSessionConfiguration

        let trustPolicyManager: ServerTrustPolicyManager
        // initialize ServerTrustPolicyManager

        let alamofireSessionManager = SessionManager(configuration: urlSessionConfiguration,
                                                     serverTrustPolicyManager: trustPolicyManager)
        let network = AlamofireNetwork(alamofireSessionManager: alamofireSessionManager)
        return network
    }

The configured instance of Networking is stored at the proper level in the application’s scope, and is dependency-injected into classes that require networking, in our example case, TwitterAPIService:

protocol MainDependency {
    var network: Networking { get }
}
protocol MainScreenBuilding {
    func build() -> MainScreenRouting
}
final class MainScreenBuilder: MainScreenBuilding {
    let dependency: MainDependency
    init(dependency: MainDependency) {
        self.dependency = dependency
    }
    // MARK: - MainScreenBuilding
    func build() -> MainScreenRouting {
        let twitteApi = TwitterAPIService(network: dependency.network)
        // ...
    }
}

Testing Swift code with manual mocks

Let’s take another look at the searchTweets() function in order to list conditions we would like to test for in the unit-test suite:

    func searchTweets(q: String, responseCallback: @escaping (Result<TweetsResponse>) -> ()) {
        network
            .get(configuration.endpointUrl, parameters: ["q": q])          // 1
            .responseObject { (response: DataResponse<TweetsResponse>) in
                responseCallback(response.result)                          // 2
            }
    }

The function has only two lines of code that contain logic:

• Line #1: creating the request with the proper URL and parameters;
• Line #2: returning the result to the caller.

The important thing to note is that a properly organized test should only test the logic that is explicitly contained in the tested class/function. When writing unit tests for the searchTweets function, we only want to test the conditions listed above, that express the original intention of the developer, and nothing else. We should not go to other abstraction levels, and test for, e.g., that get() function actually creates a GET HTTP request, or test the details of how the resulting JSON data is deserialized. We should expect that these details have been tested already, in this case, in the networking library’s test suite.

Let’s write our ideal test suite for the function above:

class TwitterAPIServiceSpec: TestSpec {
    override func spec() {
        describe("TwitterAPIService") {
            var network: NetworkingMock!
            var sut: TwitterAPIService!

            let mockEndpointUrl = URL(string: "https://search.twitter.com/1")!
            let mockConfiguration = TwitterAPIService.Configuration(endpointUrl: mockEndpointUrl)

            beforeEach {
                network = NetworkingMock()
                sut = TwitterAPIService(
                    network: network,
                    configuration: mockConfiguration)
            }

            describe("searchTweets()") {
                let mockQuery = "#awesome_testing"
                let mockResult = Result.success(TweetsResponse())
                var actualUrl: URL?
                var actualQuery: String?
                var observedResult: Result<TweetsResponse>?
                beforeEach {
                    network.mockResponse(mockResult, validateRequest: { (url: URLConvertible, method: HTTPMethod, parameters: Parameters?, encoding: ParameterEncoding, headers: HTTPHeaders?) in
                        actualUrl = try? url.asURL()
                        actualQuery = parameters?["q"] as? String
                    })

                    _ = sut.searchTweets(q: mockQuery, responseCallback: { (result: Result<TweetsResponse>) in
                        observedResult = result
                    })
                }
                it("calls network.request()") {
                    expect(network.requestCallCount) == 1
                }
                it("uses correct endpoint URL") {
                    expect(actualUrl) == mockEndpointUrl
                }
                it("uses correct parameter value") {
                    expect(actualQuery) == mockQuery
                }
                it("returns expected result") {
                    expect(observedResult).to(beSuccess())
                    expect(observedResult?.value) == mockResult.value
                }
            } // describe("searchTweets()")
        } // describe("TwitterAPIService")
    }
}

What I like about this test is that it is completely isolated. It does not depend on the global state such as network connectivity, or on a shared global NSURLSession object. TwitterAPIService class declares all its dependencies explicitly, in the form of Swift protocols injected via the initializer, so that test doubles can be easily constructed and substituted by the test suite in place of real dependencies.

The NetworkingMock class conforms to Networking protocol and implements a typical testing-double helper interface, allowing to inspect and handle method invocations, and also adds a bit to allow more high-level testing by letting tests to provide strongly-typed mock responses to requests, and inspect properties of the request by specifying validation closures.

Another useful extension is beSuccess() Quick matcher function, which allows asserting whether the received Result<> value was a success. Similarly, beFailure() Quick matcher function implements a predicate that matches Result.failure values.

An icing on the cake: RxSwift bindings

Extensions in EnvelopeNetworkRx framework allow writing networking code as neat as this:

    func searchTweets(q: String) -> Single<TweetsResponse> {
        return network
            .rx.get(configuration.endpointUrl, parameters: ["q": q])
            .object()
    }

Unit-testing Rx code has some peculiarities, and I want to cover several of them in the following test case:

    describe("searchTweets()") {
        let mockQuery = "#awesome_testing"
        let mockResult = Result.success(TweetsResponse())
        var actualUrl: URL?
        var actualQuery: String?
        var observedResult: SingleEvent<TweetsResponse>?
        beforeEach {
            network.mockResponse(mockResult, validateRequest: { (url: URLConvertible, method: HTTPMethod, parameters: Parameters?, encoding: ParameterEncoding, headers: HTTPHeaders?) in
                actualUrl = try? url.asURL()
                actualQuery = parameters?["q"] as? String
            })

            sut
                .searchTweets(q: mockQuery)
                .subscribe { (result: SingleEvent<TweetsResponse>) in
                    observedResult = result
                }
                .disposed(afterEach: self)
        }
        it("network.request() is called") {
            expect(network.requestCallCount) == 1
        }
        it("correct URL was used") {
            expect(actualUrl) == mockEndpointUrl
        }
        it("cotrrect parameter was used") {
            expect(actualQuery) == mockQuery
        }
        it("observed result as expected") {
            expect(observedResult).to(beSuccess())
            expect(observedResult?.value) == mockResult.value
        }
    } // describe("searchTweets()")

First, the setup code is completely the same as in the TwitterAPIServiceSpec class above. Second, network.mockResponse() call is the same as well. I personally like it very much, that the mocking layer works below native → Rx mapping layer. An obvious change: the test code subscribes to Rx sequence, in this case, Single<TweetsResponse>, instead of providing a closure to the network handler, and the returned Disposable instance is then being taken care of by disposed(afterEach: self) extension function available to tests extending TestCase class (defined in EnvelopeTest framework). And finally, there is another pair of beSuccess()/beFaulure() matcher functions that operate on instances of SingleEvent.

So, check out the repo at https://github.com/ivanmisuno/Envelope and take a look at the EnvelopeNetwork.playground included in the repository, and test extensions. Hope you’ll find them useful.

Next steps for developing the library:

• Add macOS, tvOS, watchOS targets for EnvelopeNetwork, EnvelopeNetworkRx and EnvelopeTest frameworks → version 0.1.1 of the library;
• Add UIKit / AppKit abstraction layers;
• Add RealmSwift abstraction layers;

Would very much appreciate your input.

Thank you!

---

About me: I started as a self-taught programmer some 28 years ago; I was a lead C++ developer while doing my Ph.D. research in machine learning in 2006; I have co-founded several software startups where I was leading teams of developers, and have worked in international hi-tech companies. I was lucky enough to work in very inspiring and challenging environments while working on some of the most rapidly growing mobile applications in the world, and have experienced how a few wisely chosen architectural principles, followed thoroughly by every member of a 400+ engineering team, allowed everybody to move fast, while allowing to create very robust, stable, and scalable applications. With this series of articles, I’d like to give practical examples of how the application of those principles can lead to a better-quality software.