DEV Community: Shubham Sharma

The Dark Side of Don't Repeat Yourself

Shubham Sharma — Wed, 06 Nov 2024 06:11:24 +0000

Why Over-Emphasising DRY Can Hurt Code Quality

The “Don’t Repeat Yourself” (DRY) principle is a foundational tenet of software development, widely taught as a means to create more maintainable, modular, and efficient code. But despite its acclaim, DRY has a less-discussed downside, particularly when developers wield it indiscriminately. Overzealous application of DRY can introduce unnecessary abstractions, reduce readability, and ultimately create more technical debt rather than reducing it. This article will explore why DRY is sometimes dangerous, examining both practical and theoretical issues, and outline strategies for applying it in a balanced, effective manner.

1. Understanding DRY's Original Purpose

At its core, DRY was introduced to prevent "knowledge duplication," not necessarily line-for-line code duplication. It encourages developers to avoid re-implementing logic or business rules in multiple places, which would increase the cost of changes by requiring updates in multiple locations. The idea was that duplicating "knowledge" could lead to inconsistencies, where one part of the code changes but others don’t, resulting in errors.

However, DRY has evolved into a more rigid mandate in many teams, focusing on eliminating even minor similarities in code. This perspective misses the underlying purpose of DRY and leads to more rigid, brittle code structures. Instead of creating an adaptable codebase, misapplied DRY can lead to code that’s difficult to understand, maintain, and extend.

2. The Dangers of Premature Abstraction

One of the primary issues with overusing DRY is premature abstraction. Developers often create abstractions to encapsulate what they perceive as similar behaviour, but when this is done prematurely, it forces a narrow view on the code's structure before the problem is fully understood.

For example, imagine a developer is working on a module that processes different food items. Cooking an egg and cooking meat might both involve "placing an ingredient on a hot surface," but these processes differ significantly. Eggs might require a quick sear, while meat might need gradual cooking over a longer period. Prematurely abstracting this "cooking" process may initially save a few lines of code, but it also removes the flexibility to handle these processes independently in the future.

By focusing on DRY too early, developers may close doors to better abstractions, losing the opportunity to improve the design once they have a clearer understanding of the domain. This often results in "de-DRYing" later or shoehorning new requirements into an existing, ill-fitting abstraction.

Practical Consequences of Premature Abstraction:

Reduced Readability: When every function or method is reduced to share an abstraction, it can be challenging for others (or even the original developer) to understand the code's intent at a glance.
Increased Complexity: Developers often need to add configuration options or conditional logic to make a single abstraction work for all cases, making the code harder to follow and maintain.
Fragile Code: Abstractions that try to do too much become brittle, as minor changes in one part of the code can cause unintended effects elsewhere.

3. Losing Sight of the Data Structure

Overuse of DRY can also obscure the structure and flow of data in an application, particularly when developers attempt to fit diverse data manipulations into a common base structure. This often happens when they create a generic “wrapper” class or base structure to represent different entities that share only superficial similarities. While this can save lines of code in the short term, it sacrifices clarity in data handling, which is crucial for understanding and debugging.

In applications with complex data flows, preserving the "shape" of the data is essential. Distinct operations and data structures offer clear, logical boundaries that help convey the data's purpose and relationships. When developers "DRY up" structures, they may hide these boundaries, making it difficult to grasp the original data intent. This can also introduce unintended behaviours, as methods become responsible for handling diverse data forms they weren’t designed to manage.

4. The Myth of DRY and Maintainability

One of the most commonly cited reasons for enforcing DRY is maintainability. The argument is that having a single source of truth for a piece of logic will make it easier to update that logic in the future. However, in practice, the supposed benefits of DRY often don’t outweigh the drawbacks, especially for code that rarely changes.

For instance, developers might fear the need to modify similar code in multiple locations, but modern development tools such as find-replace or ripgrep make it easy to search and update patterns across a codebase. In reality, DRY should be applied with consideration for how often the code is likely to change. If the code in question is stable and unlikely to undergo frequent updates, the cost of maintaining minor duplication is likely far lower than the complexity introduced by trying to create a single abstraction.

Moreover, studies have shown that duplication is often less problematic than poorly conceived abstractions. Abstractions that are forced can lead to confusing dependencies, making the code harder to test, debug, and modify. As software engineer Sandi Metz famously said, “Duplication is far cheaper than the wrong abstraction.” DRY should be seen as a means to an end, not an end in itself.

5. DRYing with a Pinch of Salt: Practical Tips

Applying DRY effectively goes beyond simply removing duplication. It requires balancing readability, maintainability, and adaptability. Below are practical strategies:

Wait Until the Abstraction Emerges Naturally: Avoid rushing into abstractions. Allow similar code patterns to develop until a clearer, more robust abstraction becomes evident. This prevents the creation of rigid structures that can hinder adaptability.
Apply the AHA (Avoid Hasty Abstractions) Principle Alongside DRY: AHA encourages developers to avoid premature abstractions. By letting abstractions grow naturally from clear patterns, we prevent forced generalisations that could lead to maintenance difficulties down the line.
Embrace OCF (Optimise for Change First): OCF encourages designing code with adaptability in mind, making it easier to modify as requirements evolve. Before abstracting code to reduce duplication, assess whether each section is likely to change independently. OCF helps prioritise flexible, modular code over rigid DRY abstractions.
Follow the Single Responsibility Principle (SRP): SRP states that a function or module should have one and only one reason to change. When applying DRY, SRP helps ensure that abstractions are clean and cohesive, rather than attempting to handle multiple unrelated tasks. By adhering to SRP, we avoid “kitchen-sink” abstractions that can become unwieldy and confusing as they try to serve too many purposes.
Use the Small Functions Principle: Small, focused functions make code easier to understand, test, and modify. Instead of creating a large, shared abstraction to eliminate duplication, consider breaking code into smaller, single-purpose functions that communicate intent clearly. Small functions encourage simplicity and modularity, both of which help preserve the flexibility needed for future changes.
Focus on Constants and Closed Abstractions: Constants, configurations, and values that are stable are good candidates for DRY. JSON values, enums, and other fixed elements can safely be centralised without risking rigid abstractions that need frequent modification.
Use DRY for Knowledge, Not Syntax: DRY is about eliminating duplicated knowledge, not just identical lines of code. Focus on sharing logic, calculations, and rules that have a real benefit when centralised.
Assess the Frequency of Change: Before creating an abstraction, consider if the code is likely to evolve independently. If similar sections will diverge over time, keeping them separate may ultimately be more maintainable.

6. Recognising and Avoiding Zealotry in Code Reviews

One unfortunate side effect of DRY’s popularity is the zealotry that can develop around it. Many developers, especially in code reviews, become so focused on enforcing DRY that they fail to see the bigger picture, such as the code’s readability, maintainability, and adaptability. This DRY zealotry often manifests as nitpicking in code reviews, where reviewers insist on combining even loosely related pieces of code.

To mitigate this, development teams should encourage a balanced view of DRY. Code reviews should focus on the intent of the code rather than enforcing superficial principles like DRY at all costs. Here are some guidelines to foster a more balanced code review culture:

Encourage Developers to Defend Their Design Choices: Developers should be able to justify their choices when they diverge from DRY. This encourages thoughtful discussion about trade-offs and design decisions.
Focus on Readability and Domain Logic: Instead of forcing every line to adhere to DRY, consider whether the code clearly expresses the domain's logic and intent. If enforcing DRY makes it harder to understand the business rules or purpose of a section, it’s likely a poor abstraction.
Use DRY as a Discussion Point, Not a Mandate: In code reviews, DRY should be treated as a point of consideration rather than a rule. A good code review culture is one that prioritizes meaningful abstractions, flexibility, and simplicity over rigid adherence to principles.

7. The DRY Trap in Specific Communities

The issue of over-DRYing isn’t equally prevalent across all development communities. Some programming communities or frameworks encourage excessive adherence to DRY, creating cultures where minor duplication is considered an unforgivable offense. Developers working in these environments should be cautious about adopting DRY as an automatic reflex and instead strive for a pragmatic approach to duplication.

A balanced DRY approach involves understanding when to embrace some duplication, particularly when working with frameworks that heavily promote DRY practices. It’s essential to assess whether DRY truly improves the code or if it merely reflects the coding norms of the framework.

8. The Benefits of Strategic Duplication

In some cases, duplication is not just acceptable but desirable. Here’s why intentional duplication can sometimes be the best choice:

Clearer Intent: When similar operations are duplicated, each occurrence can retain its unique purpose, making the intent clearer and reducing the mental overhead of understanding an abstraction.
Independent Evolution: Duplication allows for similar pieces of code to evolve independently without impacting each other. This can be crucial when requirements change for only one instance of the duplicated code.
Reduced Complexity: A simple, duplicated solution can often be more straightforward than a complex, overly generalised one, especially for code that isn’t expected to change frequently.

DRY is a powerful concept, but like any tool, it must be used judiciously. Overzealous application of DRY can lead to brittle code, premature abstractions, and decreased readability. By taking a thoughtful approach—considering factors such as readability, flexibility, and the frequency of change—developers can avoid the pitfalls of DRY and create more resilient, maintainable code. Remember, duplication is often cheaper than a poor abstraction. DRY should be a guide, not a strict rule, and balancing it with a clear understanding of the domain and future requirements is key to sustainable software development.

The Case Against Mocking Libraries

Shubham Sharma — Wed, 21 Aug 2024 07:51:40 +0000

In the world of software testing, mocking libraries have long been a popular tool for isolating components and simulating dependencies. However, as our understanding of clean code and maintainable tests evolves, there's a growing sentiment that over-reliance on mocking libraries can lead to brittle, hard-to-maintain test suites. This article explores why you might want to reconsider your use of mocking libraries and opt for custom fakes instead.

The Problem with Mocking Libraries

Ugly Syntax: Many mocking libraries introduce their own DSL (Domain Specific Language) for setting up mocks. This can make tests harder to read and understand, especially for developers who aren't intimately familiar with the mocking framework.
Tight Coupling to Implementation: Mocks often require detailed knowledge of the internal workings of the system under test. This can lead to tests that are tightly coupled to implementation details, making them fragile and prone to breaking when refactoring.
Overuse and Abuse: While mocking libraries can be useful for verifying specific interactions, they're often overused. Developers may find themselves mocking every dependency, leading to tests that are more about the mocks than the actual behaviour being tested.
Inconsistent Assumptions: When mocks are scattered throughout a test suite, each mock may make different assumptions about how a dependency should behave. This can lead to inconsistencies and make it harder to reason about the expected behaviour of the system.

The Case for Custom Fakes

Instead of relying on mocking libraries, consider creating your own fake implementations of dependencies. Here's why:

Cleaner Syntax: Custom fakes use plain language constructs, making them easier to read and understand without knowledge of a specific mocking framework.
Reduced Coupling: Fakes can be designed to mimic the public interface of a dependency without exposing implementation details, reducing coupling between tests and production code.
Consistent Behaviour: By creating a single fake implementation of a dependency, you ensure consistent behaviour across all tests that use that dependency.
Less Code: Tests using fakes often require less setup code compared to those using mocking libraries, leading to more concise and focused tests.
Better Encapsulation: Fakes allow you to encapsulate complex behaviour in a reusable way, which can be especially useful for simulating external services or complex components.

Example: Refactoring from Mocks to Fakes

Let's look at an example of how we can refactor a test from using mocks to using a custom fake:

// Using a mocking library
import { jest } from '@jest/globals';

interface IUser {}
interface IUserStore {
  findById(id: string): IUser | undefined;
  store(user: IUser): void;
}

class User implements IUser {}

class UserService {
  constructor(private userStore: IUserStore) {}

  activate(user: IUser): void {
    this.userStore.store(user);
  }
}

test('UserService should activate user - using mocks', () => {
  // Arrange
  const user = new User();
  const userStoreMock = {
    findById: jest.fn().mockReturnValue(user),
    store: jest.fn(),
  };

  const userService = new UserService(userStoreMock);

  // Act
  userService.activate(user);

  // Assert
  expect(userStoreMock.store).toHaveBeenCalledTimes(1);
  expect(userStoreMock.store).toHaveBeenCalledWith(user);
});

// Using a custom fake
class UserStoreFake implements IUserStore {
  users: IUser[] = [];

  constructor(user?: IUser) {
    if (user) {
      this.users.push(user);
    }
  }

  findById(id: string): IUser | undefined {
    // Simplified for example, would likely use ID in a real implementation
    return this.users[0];
  }

  store(user: IUser): void {
    this.users.push(user);
  }
}

test('UserService should activate user - using a fake', () => {
  // Arrange
  const user = new User();
  const userStoreFake = new UserStoreFake(user);
  const userService = new UserService(userStoreFake);

  // Act
  userService.activate(user);

  // Assert
  const storedUser = userStoreFake.users.slice(-1)[0];
  expect(storedUser).toBeDefined();
});

In the refactored version, we've replaced the mock with a custom UserStoreFake. This fake implementation can be reused across multiple tests, ensuring consistent behaviour and reducing the amount of setup code needed in each test.

Conclusion

While mocking libraries have their place in the testing toolkit, they should be used judiciously. By favouring custom fakes over mocks, we can create cleaner, more maintainable, and more resilient test suites. This approach encourages us to think more deeply about the contracts between components and helps ensure that our tests remain valuable as our codebase evolves.

Remember, the goal of testing is not just to increase code coverage, but to provide confidence in the behaviour of our system. By writing cleaner tests with custom fakes, we can achieve this goal more effectively and with less maintenance overhead.

The Easiest Way to Identify Flaky Tests in Jest

Shubham Sharma — Sun, 05 May 2024 07:19:30 +0000

Flaky tests are the bane of every developer’s existence. They pass one minute and fail the next, making it challenging to trust your test suite and causing frustration and wasted time. Fortunately, there’s a tool called flaky-test-detector that employs the KISS (Keep It Simple, Stupid) approach to help you identify and manage flaky tests in your Jest test suite.

Introducing `flaky-test-detector`

flaky-test-detector is a command-line tool that helps you identify flaky tests in your test suite by running your tests multiple times and analysing the results. It supports various test runners, including Jest, and can be easily integrated into your project. For this example, we’ll be using yarn to install and run flaky-test-detector.

Setting up `flaky-test-detector with Jest`:

To use flaky-test-detector with Jest, you'll need to follow these steps:

1. Generate Jest test results in JUnit format:

First, you’ll need to generate Jest test results in the JUnit format (as this is a limitation with the tool). This can be done by using the jest-junit package:

yarn add --dev jest-junit

Then, configure Jest to generate JUnit reports by adding the following to your jest.config file:

{
  "reporters": [ "default", "jest-junit" ]
}

2. Install and configure flaky-test-detector

Next, install flaky-test-detector and the JUnit parser:

yarn add --dev @smartesting/flaky-test-detector

Then, configure flaky-test-detector to use the JUnit parser and run your Jest tests multiple times (e.g., 5 times):

yarn flaky-test-detector --run-tests "yarn test" --repeat=5 --test-output-file=junit.xml --test-output-format=junit

If you want, you can turn this command-line execution into a script in your package.json, like this:

{  
  "detectFlaky:unitTests": "flaky-test-detector --run-tests \"yarn test:browser\" --repeat=5 --test-output-file=junit.xml --test-output-format=junit"
}

This command will run your Jest tests 5 times and generate a JUnit report named junit.xml. The JUnit parser built into flaky-test-detector will then parse the Jest/JUnit results and detect any flaky tests based on inconsistent test results across the 5 runs.

Interpreting the Results

After running flaky-test-detector, it will output a summary of any flaky tests it encountered in the terminal. If you’re wondering how the tool works, the main error reporting happens in the BasicReporter class. When flaky tests are detected, the flakyTestsFound method is called, which logs the number of flaky tests found and their names to the logger.error method.

For example:

Found 2 flaky tests
 - myFlakyTest1
 - myFlakyTest2

Additionally, after detecting flaky tests, detectFlakyTests throws an error: throw new Error('Flaky tests found'). This ensures that the process fails, providing a clear and actionable output when flaky tests are detected.

With this information, you can investigate the root causes of the flaky tests and take appropriate action. Here are some potential mitigation strategies and examples:

Refactoring the Test: If the flakiness is caused by issues within the test itself, such as race conditions or improper setup/teardown, refactoring the test code may be necessary. For example, if a test depends on the order of asynchronous operations, you could introduce synchronisation mechanisms or use libraries like async/await to ensure proper control flow.
Fixing Underlying Issues: If the flakiness stems from issues in the production code, fixing the underlying bugs or race conditions should be the priority. This might involve adjusting concurrency control mechanisms, fixing data access race conditions, or addressing any other root causes that lead to non-deterministic behaviour.
Improving Test Isolation: Flaky tests can sometimes be caused by shared state or dependencies between tests. To mitigate this, you can leverage test doubles (mocks, stubs, or fakes) to isolate the tests from external dependencies or shared resources. This ensures that each test runs in a controlled and predictable environment, reducing the risk of flakiness.
Improving Test Data Management: If the flakiness is related to test data, consider using external data sources (e.g., JSON files or databases) or data factories to generate test data consistently. This approach separates test data from test logic, making it easier to maintain and reducing the risk of flakiness caused by hard-coded test data.
Parallelisation and Test Ordering: In some cases, flakiness can be caused by parallel test execution or test ordering issues. To address this, you can experiment with different parallelisation strategies or consider running tests in a specific order or in serial order by using --runInBand to mitigate potential race conditions or conflicts.

By leveraging these mitigation strategies and addressing the root causes of flakiness, you can gradually improve the reliability and stability of your test suite, ensuring that your tests provide consistent and trustworthy feedback throughout the development lifecycle.

Conclusion

Flaky tests can be a frustrating and time-consuming issue, but with the help of flaky-test-detector, you can easily identify and manage them in your Jest test suite. By integrating this tool into your development workflow, you can ensure that your tests are reliable and consistent, saving you time and increasing your confidence in your codebase.

DEV Community: Shubham Sharma

The Dark Side of Don't Repeat Yourself

Why Over-Emphasising DRY Can Hurt Code Quality

1. Understanding DRY's Original Purpose

2. The Dangers of Premature Abstraction

Practical Consequences of Premature Abstraction:

3. Losing Sight of the Data Structure

4. The Myth of DRY and Maintainability

5. DRYing with a Pinch of Salt: Practical Tips

6. Recognising and Avoiding Zealotry in Code Reviews

7. The DRY Trap in Specific Communities

8. The Benefits of Strategic Duplication

The Case Against Mocking Libraries

The Problem with Mocking Libraries

The Case for Custom Fakes

Example: Refactoring from Mocks to Fakes

Conclusion

The Easiest Way to Identify Flaky Tests in Jest

Introducing flaky-test-detector

Setting up flaky-test-detector with Jest:

1. Generate Jest test results in JUnit format:

2. Install and configure flaky-test-detector

Interpreting the Results

Conclusion

Introducing `flaky-test-detector`

Setting up `flaky-test-detector with Jest`: