DEV Community: Roubert Edgar

Effective Tests

Roubert Edgar — Mon, 22 Jul 2024 00:15:50 +0000

A bunch of good things may come to our minds when we think about automated tests, such as bug detection, but in fact, the reason to have tests is the same reason to have good design; reducing costs by having changeable code.

Changeability requires three correlated capabilities:

Refactoring
Flexible design
Effective tests

But how do automated tests enable changeability? Particularly, I can see only two main reasons that turn into many others

Confidence to change
Documentation

Confidence to change

Without tests, every change is a possible bug (Martin, 2008).

No matter if the infrastructure is well convinced if the domain complexity is not handled in the design (Eric, 2003), turning our lens over tests, no matter how flexible the architecture is, how nicely partitioned the design, the reluctance of change will always be there without tests (Martin, 2008). A fearful environment stifles the ability of refactoring.

Refactoring

Refactoring consists of improving the existing code by restructuring its internal design aiming to remove tech debts, improve readability, and accommodate new features… but maintaining the current behavior.

Guarding the expected behavior with tests gives the possibility to find and fix bugs earlier in the process, that's where the refactoring confidence comes from. More the coverage, more the confidence, more the confidence, less the stress. It's safe to change constantly improving code design that's surrounded by tests; in fact, refactoring is a crucial practice for having a flexible code.

Flexible design

A good design preserves maximum flexibility at minimum cost by putting off decisions at every opportunity (Metz, 2018)

In addition to being strongly linked to refactoring, good design also depends on testing. When the design is bad, testing is hard.

A painful setup indicates that the code requires too much context, meaning the object under test needs to know too much about its dependencies.
The need for many objects suggests excessive dependencies.
The more dependencies there are, the less stable the code becomes and the greater the change amplification.
If it's hard to write tests, then the code probably it's also hard to reuse

These "indicators" help to track how flexible, maintainable and reusable is the code. A good design it provided by refactoring and test practices, and at the same time, the good design corroborates with refactoring and makes tests easier to create. They feed from each other.

The benefits of testing go beyond the codebase. Without a safety net of tests, a new requirement can cause apprehension, pressure, or even extra working hours. In addition, tests act as living documentation for the system (one of its main goals), contributing to lowering the learning curve, and reducing cognitive load.

Documentation

Cognitive load (a symptom of complexity) relates to how much a developer needs to know in order to complete a task. When the cognitive load is higher, higher is the time required to learn what needs to be changed and greater the risk of bugs due to a missing point (Ousterhout, 2021).

An elevated cognitive load comes from the obscurity caused by inadequate documentation, which stems from bad code that's not able to express its intention. Static documents, such as comments become a crutch when we fail to express ourselves in the code.

Static Documents

One of the more common motivations for writing comments is bad code (Martin, 2008)

In analogy to websites, comments and doc files are a form of static document, which means that they remain the same even if the behavior changes. They are awesome to clarify some piece of code and they are also the best ones for propagating misinformation.

That doesn't mean that we should never write static documents. They are crucial to guide outsiders on how to use an SDK, providing use cases and code examples. For other purposes though, prioritize dynamic documents.

Dynamic Documents

In contrast to comments and doc files, the code itself can and should be used to dynamically document the system design and behavior, adapting to every requirement. But that doesn't come for free, as shown in the How to test section, the code must be expressive, simple, and structured.

Keep in mind that tests are also code, first-class citizens in the codebase. They are crucial to the system documentation, offering a practical perspective on how the production code should be used; highlighting design decisions, assumptions, and constraints. The story they tell remains true long after paper documents become obsolete and human memory fails (Metz, 2018).

Focusing solely on static documents can overshadow the importance of clean and well-tested design. More obvious is the design, the greater the collaboration within the team (developers and stakeholders), the less the need for comments and documents.

Automated tests go hand in hand with agile development and are a key part of continuous delivery. Entire books could be written about their benefits and the problems they prevent. But, they must be well-written with a clear purpose regarding what and how to test. Without intentionality and clear strategies, testing can be a burden, not a benefit.

Test Strategy

Automated tests can leverage different strategies that offer valuable insights into the cost-benefit trade-off. Discussing test strategies is almost impossible without mentioning the test pyramid. The concept introduced by Mike Cohn in the Succeeding With Agile book, helps to visualize the ideal structure for an automated test suite. It emphasizes a strong foundation of unit tests at the bottom, gradually transitioning to fewer but broader integrated tests at the top.

The Pyramid remains a valuable tool, but it falls a little short if we take a closer look. It can be overly simplistic, misleading in naming and in some conceptual aspects; e.g. Service Test is something specific to the Service-oriented Architecture and UI Tests do not necessarily mean slow. But the main concept remains.

Write tests with different granularity
The higher level you get, the fewer tests you should have

Beyond these core concepts, consider changing the test layer naming using terms that align with the codebase. Here's an option of a consistent naming convention based on the integration level that matches the industry standards; Unit Test, Integration Tests (Also called component test), and End to End Test.

Unfortunately, the terminology used to describe test strategies can differ significantly between authors. Don't be attached to the test naming conventions though, concentrate on the core concepts and agree upon common terms to be used in the team.

End to End

Starting from the top of the Pyramid we have the end-to-end(E2E) tests. As the name suggests, the idea here is to test some part of the application simulating real-world scenarios, fully integrated, involving connections with database, filesystem, network calls, etc. The user interface is often involved in E2E tests, but a physical client isn't required to create or execute them.

End-to-end tests sit at the top of the testing pyramid due to their inherent trade-offs. This kind of test requires an extensive setup and configuration, which makes it more time-consuming to develop and maintain. Additionally, E2E tests are susceptible to flakiness, meaning they can sometimes fail due to external factors or environmental changes, leading to unreliable results.

Despite their drawbacks, E2E tests provide invaluable confidence asserting that a complete user journey functions as expected after a change. However, it's important to use them strategically and sparingly. Smoke tests, a type of End-to-End test, are a great choice here. They focus on testing just the core functionalities of the system with a small set of test scenarios, which minimizes the downsides of this test strategy.

Integration Tests

Also known as Component Tests, Integration tests offer flexibility to the test scope when compared with E2E. They count on Test Doubles to replace some external components, such as database and network calls in order to isolate and verify interactions between application components avoiding real-world dependencies. This controlled environment gives the possibility to focus on a specific integration at a time, promoting efficient and target testing.

Integration tests can be effectively done with a narrower scope. All the network calls can be replaced by mocked responses and Contract Tests can be created to verify that the protocol with the server remains intact instead. In a layered architecture, for example, it's possible to swap out any layer component with a test double to achieve different levels of integration.

By focusing on a smaller portion of the system, integration tests run quicker and reduce the configuration cost, making them easier to create and maintain. This enables a large amount of them in the test suite while maintaining its healthiness.

Unit Tests

Unit tests form the cornerstone of any effective test suite. They focus on testing small pieces of code at a time, isolated from external actors. Although the term Unit in the context of Unit Test is not stone carved, it can range from a function to an entire class. The isolation in Unit Tests can also vary depending on the sociability level, being them solitary or sociable.

Sociable - While integrating with real collaborators, unit tests can gain more realistic behavior. But, this approach comes with trade-offs. First, setting up the test becomes more complex as all dependencies of the object under test also need to be instantiated. Second, these tests are susceptible to side effects. Any changes within the collaborators can cause multiple tests to fail, even if the object under test itself functions correctly.
Solitary - To maximize the isolation, unit tests can rely exclusively on test doubles to simulate dependencies, eliminating the issues associated with sociable tests but losing the realistic behavior.

The sociability level is not a big deal when defining the test strategy, take the chance to learn what works best for the team using the codebase to experiment with different approaches. A mixed attitude between sociable and solitary is also valid, where just complex collaborators are replaced by test doubles, or only data objects are real collaborators.

An intentional test strategy is crucial for a robust test suite. Center on writing a focused set of End-to-End scenarios, complemented by a substantial set of Integration Tests, and a huge amount of Unit Tests as the test pyramid foundation. This balanced approach avoids the "ice cream cone" anti-pattern, where too many E2E tests slow down development and reduce test suite maintainability.

Now that we've explored test strategies and their trade-offs, let's delve into what to test and how to create them effectively.

Effective Test

It's tough to cover the nuances of each test strategy in a book, and It's impossible to do that in an article. The following sections explore some best practices that can be applied to all kinds of tests, but emphasize a solid foundation on Unit Test for the test suite from an Object-oriented Programming perspective.

What to test?

You don’t send messages because you have objects, you have objects because you send messages. (Metz, 2018)

Traditionally, class-based design focuses on defining objects and their functionalities. By putting messages at the center of the design, the application revolves around the communication between objects rather than focusing on the objects themselves. This can be understood as a shift from "what" an object is to "what" it needs (the message).

Imagine saying, "I need this to be done" instead of dictating how to do it. This "blind trust" fosters collaboration between objects. Messages act as requests, allowing objects to fulfill their responsibilities without tight coupling or knowledge of each other's internal workings. This promotes loose coupling and modularity in the overall system design.

Answering the initial question: What to test? Messages are what we need to test.

Objects deal with two main kinds of messages in a conversation, the Incoming Messages and the Outgoing ones. Testing these messages ensures both sides of the conversation are working as expected.

Incoming messages

Incoming messages define the public interface of the receiving object, establishing its communication protocol. The receiving object is responsible for testing its own interface, which is done through tests of state, asserting expected results upon incoming messages.

Outgoing messages

Outgoing messages are the ones that an object sends to another. They are naturally the incoming messages for other objects. In a conversation that goes from the object A to the object B, the outgoing message from A becomes the incoming message to B.

There are two kinds of outgoing messages:

Queries: These messages retrieve information without causing lasting changes. Since only the sender cares about the response, queries typically don't require testing.
Commands: These messages trigger actions within the system, potentially affecting other components. Commands are crucial for system functionality and should be thoroughly tested.

It's important to note that the sending object should not assert on the receiving object's public interface. Instead, it should focus on ensuring the command is correct: sent with the right data, at the appropriate frequency. These tests are focused on verifying the message's behavior, not the internal workings of the receiver.

The coffee maker

Imagine owning a fully automated coffee maker. You don't need to worry about heating water, grinding beans, or any of the intricate details. All you care about is enjoying your perfect cup. With a simple selection, you hand over the entire coffee-making process to the machine, trusting it to deliver the desired result.

The coffee maker receives a recipe as the incoming message, specifying water temperature, grind size, and other specifications.

To fulfill the recipe, it coordinates with components like the grinder, sending some outgoing messages to them. However, the initial design coupled the coffee maker too tightly with the grinder's operations, requiring it to interact with the silo directly.

To enhance modularity and reduce dependencies, the coffee maker should delegate the bean acquisition process to the grinder. By providing the grinder with a desired powder profile, the coffee maker allows the grinder to work autonomously by just sending a simple query message, abstracting the complexity of bean dispensing.

The coffee maker can also cache the last selected recipe. When a coffee drink is chosen, the coffee maker stores the selected recipe by sending a command message to the cache, making it possible to quickly select the same coffee for the next brew.

A message-centric approach offers significant advantages. By focusing on messages, systems become more flexible due to looser coupling between objects, making maintenance and expansion easier. This perspective also aids in discovering new objects, as a message inherently requires a corresponding object to handle it, leading to more modular and reusable designs.

It's important to note that messages an object sends to itself are never directly tested, as they are private methods and not part of the public communication interface. If you believe some internal messages should be tested, ensure they are being sent to the correct place. This may indicate the need to create a new object to handle these messages, allowing for proper testing.

How to test?

As we discussed previously, If not well-written, tests can increase costs instead of saving the company's money. They require thought, design, and care, and readability is everything when it comes to clean tests. But how can we make tests readable? In the same way that we create readable code, with expressiveness, simplicity, and structure.

Expressiveness
Use meaningful names for variables, methods, and classes, avoiding writing code that is overly clever or obscure. Concise code is often desirable, but it should never come at the cost of clarity.

// Throws an exception if the boiler is not ready for the recipe
fun prepare(recipe: Recipe){
    if(boiler.temperature != recipe.temperature) throw Exception()
      //continue with the process
    }
}

Initially, a comment vaguely indicated that an exception would be thrown if the boiler wasn't ready. However, the code lacked clarity as the exception type was generic and the boiler's readiness check was implicit.

fun prepare(recipe: Recipe){
    val boilerIsNotReady = boiler.temperature != recipe.temperature
    if(boilerIsNotReady) throw BoilerNotReadyException()
    //continue with the process
}

By introducing a variable to explicitly check the boiler's readiness and specifying a custom exception, we've transformed a vague comment into clear, actionable code. This approach enhances code readability and maintainability while providing valuable information about the system's behavior.

Simplicity
Code should be as simple as possible while still fulfilling its requirements. Break down complex logic into smaller and manageable units.

fun listComponentsForMaintenance() : List<Component>{
    val componentsForMaintenance = emptyList()
    components.forEach{ component ->
        val now = Clock.System.now()
        val days = components.lastMaintenance.daysUntil(now, TimeZone.UTC)
        val eligibleForMaintenanceList = emptyList<String>()
        if(days >= 30) {
            componentsForMaintenance.add(component)
        }  
    }
    return componentForMaintenance
}

In this example, a component is added to the componentsForMaintenance list if the last maintenance was 30 days ago or more. However, the code is difficult to read due to its chained logic within a for-each loop. There's no storytelling.

fun listComponentsForMaintenance() : List<Component> {
    return components.filter{ component->
        isEligibleForMaintenance(compnent)
    }
}

private fun isEligibleForMaintenance(component: Component): Boolean {
    val now = Clock.System.now()
    val lastMaintenance = component.lastMaintenance
    val daysSinceLastMaintenance = lastMaintenance.daysUntil(now, TimeZone.UTC)
    return  daysSinceLastMaintenance >= 30
}

Instead of having a long chain of logic, identify smaller steps and create separate functions for each. This makes the code easier to understand. The days since last maintenance logic could also be extracted to its own method. In addition, the isEligibleForMaintenance method could be a Kotlin's extension function of the Component class. Even better, the Component itself could determine if it is eligible for maintenance.

Structure
Separate concerns and responsibilities. This makes the code easier to navigate and understand.

fun prepare(recipe: Recipe) : Coffee {
    // other steps here
    val beans = grinder.silo.dispense(recipe.weight)
    grinder.grind(beans, recipe.granulometry)
}

The coffee maker directly interacts with the silo to obtain beans, unnecessarily coupling the two components.

fun prepare(recipe: Recipe) : Coffee {
    // other steps here
    grinder.grind(powderProfile)
}

By transferring the responsibility of bean acquisition to the grinder, we enhance encapsulation and improve code structure, avoiding the Demeters law violation.

These small examples shown above seem to cause no harm to the codebase, but remember, complexity is sedimentary, it accumulates in small chunks.

The coffee maker test

Before we start, note that the Coffee Maker's dependencies are not treated as abstractions here. While Robert Martin advocates for robust abstractions and recommends using the Impl suffix when there's just a single implementation for an interface, I prefer Sandi Metz’s perspective. She suggests that code can initially focus on functionality and evolve towards abstractions as needed. Modern IDEs make it easy to extract an object's public interface when necessary.

Remember, the language (Kotlin in this case) is just a tool. We can translate these ideas to any object-oriented programming language. MockK is also used as the mocking framework, but consider using stubs when testing against an abstract dependency. Unlike mocks, stubs focus on specific behaviors of the dependency while keeping the rest of its functionality intact. Check out this article about test doubles to learn more.

Let's "brew" some tests...

class CoffeeMakerTest {
    @MockK private lateinit var grinder: Grinder
    @MockK private lateinit var cache: Cache
    private lateinit var coffeeMaker: CoffeeMaker

    @Before
    fun setUp() {
        MockKAnnotations.init(this)
        coffeeMaker = CoffeeMaker(grinder, cache)
    }

    @Test
    fun `should return coffee when call the prepare function`() {
        val espressoRecipe = EspressoRecipe()
        every {
        grinder.grind(PowderProfile(weight = 15f, granulometry= 1.5)) 
        } returns Powder("espresso")
        val coffee = coffeeMaker.prepare(espressoRecipe)
        assertThat(coffee).isInstanceOf(Espresso::class.java)
        assertThat(coffee.ratio).isEqualTo("1:2")
        verify(exactly = 1) { cache.store(espressoRecipe) }
        //other assertions and verifications
    }
}

In this example, we ensure that a coffee is returned when an espresso recipe is prepared. We also verify that the selected recipe is stored in the cache. However, the test is only marginally readable because the example is not very complex, and we are testing just one small interaction of the coffee maker without any error cases. The code lacks expressiveness, simplicity, and structure. At the very least, it uses real-world examples for the test.

Naming

Let's start with the test naming: should return coffee when call the prepare function. This name doesn't convey anything about business rules; it focuses more on function calls, also, every piece of code "should" do something or be deleted instead.

When creating a test, try to answer the question: What behavior is expected from the code under test given a specific action? For example, brew an espresso when an espresso recipe is prepared.

Given, When, Then

"Given, when, then", "Build, Operate, Check" or "Arrange, Act, Assert" are structure patterns valid for every kind of test and consist of splitting the test function into three sections:

Given: This section sets the initial state, including data and preconditions for the test.
When: This part represents the action that triggers the behavior being tested, usually a function call.
Then: This section checks if the operations yielded the expected results.

class CoffeeMakerTest {
    // setup

    @Test
    fun `brew an espresso when an espresso recipe is prepared`() {
        val espressoRecipe = EspressoRecipe()
        every {
        grinder.grind(PowderProfile(weight = 15f, granulometry= 1.5)) 
        } returns Powder("espresso")

        val coffee = coffeeMaker.prepare(espressoRecipe)

        assertThat(coffee).isInstanceOf(Espresso::class.java)
        assertThat(coffee.ratio).isEqualTo("1:2")
        verify(exactly = 1) { cache.store(espressoRecipe) }
        //other assertions and verifications
    }
}

One assertion per test

Some testers suggest that each test case should include only one assertion. This is less about limiting the number of assert method calls and more about focusing on verifying a single behavior per test case.

In the current example, the test method is checking both the incoming mesasge to the coffee machine and outgoing messages sent to the cache. These represent two distinct scenarios and should be tested separately.

Instead, one test case should be created to to assert the state of the coffee maker when a prepare(recipe) message is sent. Another test case should be dedicated to ensuring that the cache is called to store the recipe.

class CoffeeMakerTest {
    // test setup

    @Test
    fun `brew an espresso when an espresso recipe is prepared`() {
        stubEspressoGrinding()

        val coffee = coffeeMaker.prepare(EspressoRecipe())

        assertThat(coffee).isInstanceOf(Espresso::class.java)
        assertThat(coffee.ratio).isEqualTo("1:2")
    }

    @Test
    fun `store espresso as the last chosen recipe on prepare an espresso`(){
        stubEspressoGrinding()
        val espressoRecipe = EspressoRecipe()

        val coffee = coffeeMaker.prepare(espressoRecipe)

        verify(exactly = 1) { cache.store(espressoRecipe) }
    }

    private fun stubEspressoGrinding(){
        val powderProfile = espressoPowderFixture()
        every { grinder.grind(powderProfile) } returns Powder("espresso")
    }
}

Domain-Specific language

Domain-specific languages (DSLs) are a great approach to enhancing test writing. Instead of using production and test APIs directly, a set of functions is created to abstract them, increasing readability and maintainability. DSLs are also especially useful for managing complex data sets within test scenarios, enabling dynamic test data manipulation to meet intricate requirements and variations.

For example, in the previous test, we can improve the expressiveness of the cache command verification by abstracting the verify block into verifyRecipeHasBeenCached(espressoRecipe), just as we did with the espresso grinding stub abstraction (stubEspressoGrinding).

Test fixtures are also useful for creating simple test cases with well-defined data. In addition, they can be used alongside domain-specific languages, serving as building blocks for them.

Keep it clean

Having dirty tests is equivalent to, if not worse than, having no tests (Martin, 2008)

Achieving clean and maintainable tests is a continuous effort. There's no single magic solution, but rather a combination of techniques. Clear naming conventions, proper test structure, and selecting the right test data management approach (fixtures or DSLs) depending on the scenario are all key factors.

Sometimes, improving the production code itself can significantly benefit test quality. Refactoring code for better separation of concerns or creating helper methods can lead to more expressive and maintainable tests.

There are situations where controlled code duplication actually improves test readability and maintainability, even if it violates the DRY (Don't Repeat Yourself) principle. Finding a balance between these two aspects is essential..

Unmaintained tests lose their value and can create a false sense of security. Regularly review your tests. If a test becomes difficult to maintain, consider refactoring it, improving the code it targets, or even removing it entirely if it no longer serves a purpose.

Avoid using multiple architectures and test patterns. While a specific solution may seem optimal for a single problem, it can increase cognitive load when considered in a broader context. "Perfect" is a personal thing, seek improvement instead.

TDD vs BDD

As mentioned, it's almost impossible to cover everything related to testing in a single article, but these two practices deserve special mention when discussing clean tests.

TDD (Test-Driven Development) relies on writing automated tests before writing any production code. This practice ensures code quality by forcing developers to think about the desired functionality and potential issues upfront.
BDD (Behavior Driven Development) is a development approach that focuses on describing the desired behavior of a software system. It promotes collaboration between developers, testers, and non-technical stakeholders by using a natural language (like Gherkyn Syntax) to write test cases. This shared understanding ensures the final product aligns with business requirements.

Both Behavior Driven Development (BDD) and Test-Driven Development (TDD) are not mandatory practices, but they significantly enhance the software quality assurance process.

F.I.R.S.T

Last but not least, create tests F.I.R.S.T

Fast: Tests should execute quickly to provide rapid feedback and avoid slowing down development cycles.
Independent: Tests shouldn't rely on the outcome of other tests, allowing them to be run in any order or isolation.
Repeatable: Tests should produce the same results consistently regardless of the environment or previous test runs.
Self-Validating: Tests should clearly indicate success or failure without requiring manual interpretation.
Timely: Write tests before or alongside the code they are testing. This promotes Test-Driven Development (TDD) where tests guide the development process.

Conclusion

Test automation offers a several amount of benefits and are a key for continuous delivery, with several practices impacting IT performance. Reliable automated tests are essential: when tests pass, teams can trust their software is ready for release, and test failures accurately reflect defects. Flaky or unreliable test suites can lead to false positives or negatives, so it's important to invest in making tests dependable.

Developers should primarily create and maintain acceptance tests, as they can reproduce and fix these tests on their own workstations. Tests created and maintained by QA or outsourced parties do not correlate with better IT performance. Involving developers in the creation and maintenance of tests improves testability and aligns with test-driven development (TDD), which encourages more testable designs (Forsgren, Humble, & Kim, 2018).

Ultimately, embracing a culture of testing is about more than just checking boxes or reaching a certain code coverage percentage. It's about fostering a mindset of quality throughout the entire development process. It's about building systems that are not only functional but also flexible, maintainable, and resilient to change. By adopting a thoughtful and intentional approach to testing, we can create software that truly stands the test of time.

References

A journey through end-to-end testing

Roubert Edgar — Sun, 11 Feb 2024 03:54:15 +0000

There's no consensus about test naming patterns, usually, we have different terms to represent the same test strategy, which makes it valuable to have team discussions to create a test strategy ubiquitous language.

What end-to-end (E2E) means? I know what it doesn't mean, regression tests. They are confused with each other all the time, and that's ok, as e2e's are part of the regression test suite and have the objective to check if a bug was introduced after a change, but keep in mind that regression test involves the execution of the entire test suite, which includes the ent-to-end ones.

End-to-end tests can be also referred to as Broad Stack Tests and basically imply that we are testing some part of the system entirely, fully integrated, which makes them expensive when automated. E2E can be structured in a bunch of different ways, but let's focus on two of them, Journey Tests and Smoke Tests.

Journey Test

Journey tests are also kinds of Business Facing Tests and have the intent to simulate real-world scenarios consisting of the execution of user journeys inside the app. The idea is to cover the whole application, checking how different components interact and whether the system behaves correctly in complex scenarios.

Automated journey tests can also be implemented using the Component Test strategy, offering advantages over traditional end-to-end (E2E) tests. Component tests are generally faster to execute and maintain because they leverage test doubles to isolate dependencies, particularly those requiring network calls, thereby reducing test flakiness. However, even with this approach, component tests can be expensive to create, so be assertive with them.

Smoke Test

First of all, we are not planning to stress any device until we see it smoking... The Smoke Test term comes from the idea of turning on a piece of electronic equipment for the first time to see if it starts smoking due to a major flaw, indicating that there might be a serious issue with the hardware.

In software, a smoke test is a kind of end-to-end test that consists of executing some user journeys on the application to check if the basic functionalities work, which means that smoke tests can be seen as a small version of User Journey Tests. The selected journey depends on how critical is the given feature to the system.

Keep in mind to create them using the E2E strategy, even if that means "more expensive", once the idea is to test just the happy path of a few user journeys.

How to write

Automated or not, test scenarios also serve as documentation to the system, which means that we should care about well-defined scenarios and readability, making them comprehensive not only for engineers but also for stakeholders.

Well, readability may depend on the audience, however, we can tailor the message to make it niche or broadly comprehensive by changing its structure according to the target readers.

We can achieve our readability goal by using the gerkin syntax to structure our test scenarios using the keyword Given, When, And, and Then to represent each step of the user journey in the app. While Gherkin is commonly used with Behavior-Driven Development (BDD), it's not a strict requirement. Both Gherkin syntax and BDD are valuable practices, but their applicability depends on the team context.

Here's how we can structure a test scenario using Gherkin:

Given my application is installed

And I tap on the app icon

When the app is launched

Then I see the Login screen

The scenario above can be executed manually, by automation, or both. But it's important to have these scenarios documented somehow, in a spreadsheet, a Confluence page, or any kind of document that anyone in the team can have access to reproduce the test step or search for some business rule in the system.

What about automating these kind of test scenarios? Well, the range of possibilities is huge, which also depends on the platform, however, we can establish some automation rules that you can follow, no matter the platform, framework, etc.

Automation Rules

Given my application is installed
And I tap on the app icon
When the app is launched
Then I see the Login screen

Looking into our example scenario above, It's possible to extract some rules from it and infer some others. Some of the following rules may seem to be obvious, but what's obvious for you may not be to another person, so let's write them down.

Ordered steps
Test scenarios are composed of ordered steps. A user journey changes according to the user steps in the application. A different step sequence means a different test scenario.
One step, one assertion
Asserting each step of the user journey helps to find the exact step that caused some test failure, improves test logging, and increases the step reusability, once this practice collaborates with the single responsibility principle.
Logging matters
Each step assertion should print an execution result describing the test step, e.g. And I tap on the app icon - SUCCESS.
Ordered Scenarios
As we're simulating the real world, it should not be possible to open the account screen before authenticating a user for example (If the application requires authentication, of course).

Journey test automation articles

Android journey test automation

References

User Journey Test Guide - Dominik Szahidewicz
User Journey Tests - Martin Fowler
Business Facing Test - Martin Fowler

The Mobile Service

Roubert Edgar — Tue, 13 Jun 2023 13:48:06 +0000

Mobile devices seem to be part of the human body now, with billions of people using them to access the internet, make payments, communicate with their friends (I should get some), and so on. Many opportunities have arisen with the fact that people are connected "all the time". With geolocation services and multiple sensors, devices allow many opportunities to provide custom experiences. However, extracting more from these possibilities means developing specific mobile applications, being native or hybrid.

Native mobile development has some specific challenges and we are going to cover two of the main ones here. As good advice, developing a mobile application is much more than just writing a thousand lines of code and publishing it to the Apple Store or Play Store; most importantly, mobile development begins in the back end.

Over the years, many back-end developers have gotten used to working with web clients, accumulating experience that is more than welcome in the mobile world., However, it's not the same, and if you think so, I'm sorry to say you may be making a big mistake.

Challenges

Availability

With Web Clients, we have the possibility to update the application almost simultaneously for all users, making a new version of the application available with no friction. The same doesn't happen with mobile applications; this is because, in addition to doing the entire versioning, build, and deployment process, we must patiently wait for the stores' analysis (especially when dealing with Apple). After the analysis phase, we still have to hope that users update their application, or we can force it, but this is an undesirable and unpleasant experience for users. After all, they are not obligated to do anything.

If this scenario doesn't open your eyes, here's a slightly more drastic perspective: if a bug occurs and the fix is urgent, it could take a couple of hours, or even days, for the users to have that fix in their hands. This implies putting even more emphasis on quality, avoiding forced updates and burning candles praying for the store update acceptance (Help the environment).

Work Duplication

If in web development we have only one way of distribution, which in this case is the webservers with different browsers accessing the contents that will be interpreted/rendered to the users, in mobile development, we have two main vehicles: the Android and iOS systems. These carry with them the need to develop the same functionality for two different platforms, except in cases where hybrid technologies are used (Flutter, React Native...), which are still susceptible to the timing problems mentioned above and some others that we will not deal with in this text.

This duplication introduces the following premise: you often end up paying twice for the same problem. A process of complex integrations, rework, excess logic on the client side, etc., would increase the development costs on one platform alone. Following our premise, this additional cost is often doubled, since two people (iOS and Android) would be in charge of acting in these scenarios.

Okay, now that we understand the complexities of mobile development, it's time to propose some solutions and best practices, starting from the foundation the backend.

The Mobile Service

The mobile service should be treated as an extension of the client, being highly dedicated to it, even providing abstractions of business rules. Yes, this idea can be applied to all client types, but the emphasis of this article is mobile and I needed a title for this text.

There are a lot of good practices that can be applied to backend development that help to avoid rework and prevent exposing your customers to risk. In this section, we will discuss some practices that can be used to help with services and client integration.

You don't pay for what you don't use

As Bjarne Stroustrup (creator of C++) used to say: "You don't pay for what you don't use" From the perspective of a user, this mantra means that information must be retrieved only when it's requested, and it must contain only the data that will be presented to the user.

For example: in a list, where each of the items has only a title and description, it makes no sense to bring the item details in this first search; the answer should contain only the identifier, title and description. With this, the user doesn't spend mobile data downloading information that they will not use, since most likely only some items will have their details visualized. Another point is that by bringing less data, we can reduce loading time and memory consumption, providing a better experience with greater stability.

If there's a need to subsequently load more details for a particular item, we only need to use the identifier present in the list item to retrieve its details in a later request.

Alignment between the Client, Backend and Design is a crucial point to achieve the premise of "don't pay for what you don't use". It's the best moment to define where certain data is needed and which data are not. Another point is to make it clear to people focused on the backend how and what data will be presented to users, even facilitating later interactions about some specific feature.

Confidence

I'm sorry if you have trust issues, I suggest you work this out with a professional (I'm in this process myself), but we should be able to trust the services we are accessing. If that's not possible then we should build a layer of trust over those toxic services, using the concept of Backend For Frontend, for example. As in human relationships, trust is not bought, but earned, which leads us to the following premises that help the backend gain the client's trust:

Be cohesive: The message content must match the response code. Returning some success code(200..299) with failure content is a bad practice in any system as it creates a huge inconsistency between the HTTP code and the content. If there is an error in the service, explain it clearly with the codes that best represent the problem and the corresponding content.
Do not work with null values: Tony Hoare has already apologized for inventing this disaster of the computing world with the presentation entitled The billion dollar mistake, There's no reason to make the client work with null values, this demands more data handling and increases the potential points of failure in the code, also, null values most of the time brings ambiguity to the code, some field can be null because of a failure or just because there's no value for that field (exceptions exist for every rule 🙂).
Use default values: An alternative to null values are default values, in a text field for example, we can return an empty text instead of simply not returning it and allowing the client to decide whether to present this empty field or hide it.

Cache

Cache is an important ally in improving performance and saving resources and it is interesting to use it whenever it makes sense. If poorly planned, the caching strategy can generate problems with outdated information and break the users' experience, this layer, both in the service and in the client, must be designed with utmost care.

Clients Generate Demand

It's usual to modify the client code to deal with some backend limitations or just because is "simple" to solve some problems on the client, which means that we are hiding the problem under more code instead of solving it I think that at this point in this article, it's clear how dangerous this can be, the client is the one that generates work demands for the backend since the backend is a service provider to the client and not the opposite.

Creating more code is always easier than designing a solution or finding the real cause of an issue and we can observe this tendency not only in the client-backend relationship. This is what happens when we add logic to work around problems in a certain part of the code instead of solving the issue at the source. That is, we add complexity, but we don't solve the underlying problem.

Specific to General

Sandi Metz in her book Practical Object-Oriented Design (one of the best technical books I've read), establishes that we should depend on abstractions rather than concrete classes. This statement, however, comes along with another premise; code is not born abstract, it moves to abstraction. In other words, you don't create abstractions just to have them as good practice, you design the abstraction when it's needed.

Based on the concrete-to-abstract idea, we can build specific services and move them to more general ones. When creating an API that will initially serve only one client, it doesn't make sense to build it for general purposes. We can start this service with a specific focus and as needed, shape it to a general purpose. This is relatively easy if what makes your API specific is just a layer of the architecture that prepares data for the current client. With this, we just need to extract the lower layers of the system to another service and create a generic communication interface over these extracted layers (obviously, thoughtfully and safely).

General to Specific

There are cases where a company already has a range of general-purpose services out of the box, with many of these services using null values and lacking data response refinement. We may also face scenarios where the services built will serve different types of clients, which in turn have different demands for information and may burden those clients who need fewer data than others (over fetching).

We have two options to improve the relationship between the backend and its clients for these kinds of scenarios. The first is to build a dedicated service, called BFF (Backend for Frontend) and the second is to use GraphQL.

BFF

Backend For Frontend (BFF) is the clients "Best Friend Forever", being dedicated services and not general purpose. In short, BFFs are experience-oriented they are in charge of returning exactly what a particular client needs to expose to its users.

The use of these dedicated services makes it possible to extract logic that was previously embedded in clients, such as the validation of information and even business rules. This logic extraction brings the advantage of avoiding work duplication in native mobile development (Android and iOS), as part of the job is done in the BFF.

In addition, abstracting logic to the backend can also help in solving occasional problems in the client, since much of the complexity that was present in the client is now in the service. This capability is extremely advantageous when it comes to mobile clients, as it avoids the need to go through the bureaucratic process of deploying in stores.

Benefits

Performance: The construction of the response can be done in parallel and, as it transmits less data, ends up being faster and less costly to the clients.
Makes use of REST: Given REST is commonly used already, BFF doesn't introduce a new language for those who are already used to working with REST. This is valid for the Client and the API implementation.
Dedicated to the client: As it is a specific service, it is possible to abstract client logic in the API, increasing the ease of integration and reducing the rework between Android and iOS.
Cache: It is possible to improve the caching strategy with the addition of this tool where it was not possible before, either because it is an external API or because it is not interesting to cache data at a lower level of the application.

Disadvantages

More services are being created
Low reusability: Due to its specificity, a mobile BFF will hardly be used by a WEB client or something else.
Code duplication: If you have a service dedicated to each type of client, duplication of certain parts of the code ends up being unavoidable.
Versioning: This problem can arise from the need to maintain compatibility with an older client application. An alternative, in this case, is to do the minimum version control of the application, forcing all users to update their applications in a service contract break.

Client engineers must translate the screen into models that make sense to work with, supplying all the presentation needs and variations. Of course, this requires some Object Oriented modeling skills, which makes the Android and iOS engineers' interactions even more valuable. At the end of this process, the generated models will be translated to the API contract, and that's why client engineers should have ownership of the BFF and work along with the backend team. This knowledge is extremely beneficial and it can greatly speed up feature development, in addition to disseminating more knowledge within the team, making it a little more multidisciplinary.

This proximity of the people from the client to the BFF starts with the language chosen for the construction of the service. For Android and iOS, it's valid to use Kotlin, since it's the default language on Android, can be executed in the JVM and it's supported by Spring, widely used to create backend services. Also, Kotlin syntax is closer to Swift, bringing iOS engineers closer.

More details about BFF can be found in this text written by Sam Newman and also in this article that shows how the Sound Cloud team implemented this dedicated services strategy.

GraphQL

While in Back for Front we have the service defining what data will be sent to the client, in GraphQL we have the client defining what data it needs through queries.

Benefits

Highly reusable: With each customer looking for the data they need, it is not necessary to create a service for each one.
Does not have versioning: Backward compatibility is achieved by creating schemas, which eliminates the complexity of maintaining more than one version of a given service.
No endpoints: As it works through queries, GraphQL only needs an endpoint to carry the response data of the executed searches.

Disadvantages

Performance: The complexity in the graph can greatly burden performance due to the well-known N+1 problem, where a resolver is called for each field searched. These performance issues can be worked on and require a bit more effort and attention to do so, the good news is that there's a lot of performance-related content like this article about N+1.
Complexity: Not a big deal, but the cognitive load becomes higher, since it is necessary to understand more about the framework for its use, maintenance... Boilerplate can also become a problem when new models are added.
Responsible clients: Because they are highly reusable, It's not simple to abstract client-specific logic into Graph services, making those clients more responsible for handling these rules.
No caching strategies: This problem can be solved using some tools like Apollo, however, this new tool, at least on Android clients, brings an increase of about 3MB in the application.

The responsible clients disadvantage can be reduced or solve by having dedicated resolvers for the mobile clients, which means that we don't have a BFF for free when we create a GraphQL Server, we need to specify the resolvers.

GraphQL still generates a certain kind of controversy and concern when it comes to its overwhelm (which can happen with any technology). Some people suggest hybrid models, with GraphQL being used as a BFF, or a dedicated REST BFF consuming from a GraphQL in mobile cases and web clients consuming directly from the Graph.

Here's an awesome article about mobile development with GraphQL. There are tons of content related, keep reading about it :)

Conclusion

If a service doesn't serve its customer easily, then it's not fulfilling its role and we can go back to the old Client / Database applications. As mentioned before, the service is an extension of the client, therefore, people who work on these two fronts (back, client) must interact constantly, avoiding problems and promoting the exchange of knowledge.

Android Observables

Roubert Edgar — Sat, 01 Jan 2022 18:29:43 +0000

This text only deals with Explicit Intents. Implicit Intent, Intent Filters, and Deep Links are covered in the text Android Channels.

Have you declared this activity in your AndroidManifest.xml? If you've already written a few lines of code for Android, most likely you've run into this message when opening an Activity that wasn't declared in your app's manifest file. Well, but why do we have to declare every Activity in this file? The answer would be "Because this is how it works", but, let's go into the "hard" way and try to understand a few things about the Application Components, Intents and of course, the AndroidManifest.xml file.

Android doesn't allow an Activity to be simply instantiated and attached to the activity stack. Unlike what happens on iOS, the entire hierarchy of screens is handled by the green robot. The system also allows executing Activities from other applications, that will be in distinct tasks to your App. Therefore, for an Activity to be launched, we need to send a message to the operational system, telling the context (Context) and the class of the target activity. Such messages are called Intents.

val intent = Intent(context, SomeActivity::class.java)startActivity(intent)

When receiving an Intent, whose purpose is to launch an Activity, or to start a Service, for example, the operating system checks in its records if any application can perform such action, whether your app and/or a third-party application. That same Intent passed to the operating system is delivered to the target component as soon as it is found. However, we may have dozens of apps installed, which would make this app-by-app search quite inefficient, so we have the use of the AndroidManifest.xml file.

Okay, one of the purposes of declaring Activities in the manifest is to inform Android that your application can perform such activities and thereby optimize the operating system's call to these screens. But how does this happen? Let's answer this question in four steps , taking as an example the installation of an App that contains only one Activity, the SomeActivity.

1

Assuming our App project is already set up, let's take a look at how our activity (SomeActivity) was declared in AndroidManifest.xml:

Example 1 :

<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.some.package"> 
  <application ...some parameters > 
      <activity android:name="com.some.package.SomeActivity"/>
  </application>
</manifest>

Example 2 :

<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.some.package"> 
  <application ...some parameters > 
    <activity android:name=".SomeActivity"/> 
  </application>
</manifest>

In example 1, it is possible to observe that the declaration of an Activity is given by the name of its class, which includes the package that this class belongs to.We can also make this declaration in a shortened way, omitting the base package name (com.some.package) as demonstrated in example 2. In this case, the base package will be appended to the class name during the compile phase.

2

When installing our app, the operating system reads and records all data declared in the manifest, which includes:

Services, Broadcast Receivers, and Content Providers, belonging to the application components group together with Activities.
Configuration of such components, such as how an Activity will be launched for example
Intent Filters, which we talked about here
Specifications, permission requests, and some other things that can be found in the AndroidManifest documentation

3

With the app installed, when a message, Intent, is sent to the OS in the format Intent(context, SomeActivity::class.java), the operating system will take the class name specified in the Intent and will check which apps can open com.some.package.SomeActivity.

4

After checking which apps support running SomeActivity, which in this case would just be our app, the operating system uses the Java ClassLoader to get an instance of this screen, either by creating a new one, taking an existing one or always creating a new instance for that class, which depends on the launch strategy used. Finally, this screen is placed on top of the stack and the Intent sent is delivered to it, either in the onCreate method or in the onNewIntent method in some cases, we then have the activity presented to the user.

But what would happen if the operating system didn't find any application that supports running SomeActivity? The ActivityNotFoundException exception would be thrown, and the message "Unable to find explicit activity class {com.package/com.package.SomeActivity}; have you declared this activity in your AndroidManifest.xml?" would be visible in the Logs.

Conclusion

From a systemic viewpoint, declaring an Activity in the manifest is like subscribing to a channel, in this case, an Android channel, whose name is the class of the activity declared in the manifest. Intents are messages sent to the publisher (Android) and specify which channel should be notified, featuring the Observer pattern. There are other ways to subscribe and application components in different channels, making use of Intent Filter and Implicit Intent. Despite this, the standard way of declaring (subscribing) an Activity is mandatory.