DEV Community

Roshan Raj
Roshan Raj

Posted on

TDD Your Way to Bulletproof Integrations

#tdd #testing #eventdriven #programming

TDD

What We're Building

An event processing system that receives events, enriches them via external APIs, and publishes to an event bus. Ideal for demonstrating Test-Driven Development (TDD) in integration scenarios.

Event In → Transform → Call External API → Enrich → Publish to Event Bus
Enter fullscreen mode Exit fullscreen mode

Why TDD for Integrations?

Integration code is notoriously difficult to test. External dependencies can fail, APIs can timeout, and data transformations can be complex. TDD forces us to think about these challenges upfront.

The TDD Journey

Step 1: Start Simple - Define the Contract

Red Phase: Write a test for the basic structure

test('should create processor with required dependencies', () => {
  const processor = new EventProcessor(mockApiClient, mockEventBus);
  expect(processor).toBeDefined();
});
Enter fullscreen mode Exit fullscreen mode

Green Phase: Implement just enough to pass

class EventProcessor {
  constructor(apiClient, eventBus) {
    this.apiClient = apiClient;
    this.eventBus = eventBus;
  }
}
Enter fullscreen mode Exit fullscreen mode

Key Insight: We've defined our dependencies through the test. The processor needs an API client and event bus.

Step 2: Test the Happy Path

Red Phase: Test basic event processing with mocks

test('should process and enrich user registration event', async () => {
  // Mock external dependencies
  const mockApiClient = {
    getUserProfile: jest.fn().mockResolvedValue({ tier: 'premium' })
  };

  // Test the transformation
  await processor.process(userRegistrationEvent);

  // Verify enrichment happened
  expect(mockEventBus.publish).toHaveBeenCalledWith(
    expect.objectContaining({ enrichedData: { tier: 'premium' } })
  );
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: Mocking reveals the interface we need from external systems.

Step 3: Handle Failures (The Real Value of TDD)

Red Phase: What happens when the API fails?

test('should handle API failure gracefully', async () => {
  mockApiClient.getUserProfile.mockRejectedValue(new Error('API timeout'));

  await expect(processor.process(event))
    .rejects.toThrow('Failed to process event');

  expect(mockEventBus.publish).not.toHaveBeenCalled();
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: TDD forces us to think about error scenarios before they happen in production.

Step 4: Add Retry Logic

Red Phase: Test that we retry on temporary failures

test('should retry API calls on failure', async () => {
  mockApiClient.getUserProfile
    .mockRejectedValueOnce(new Error('Temporary failure'))
    .mockResolvedValueOnce({ tier: 'basic' });

  await processor.process(event);

  expect(mockApiClient.getUserProfile).toHaveBeenCalledTimes(2);
  expect(mockEventBus.publish).toHaveBeenCalled();
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: The test drives us to implement retry logic with configurable attempts.

Step 5: Validate Input

Red Phase: Test validation before processing

test('should validate required event fields', async () => {
  const invalidEvent = { eventType: 'USER_REGISTERED' }; // missing userId

  await expect(processor.process(invalidEvent))
    .rejects.toThrow('Invalid event: missing userId');
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: TDD helps identify validation needs early.

Step 6: Handle Different Event Types

Red Phase: Test event-specific transformations

test('should apply different transformations based on event type', async () => {
  // For ORDER_PLACED events, we need both user and order data
  mockApiClient.getOrderDetails.mockResolvedValue({ items: [...] });

  await processor.process(orderEvent);

  expect(mockApiClient.getUserProfile).toHaveBeenCalled();
  expect(mockApiClient.getOrderDetails).toHaveBeenCalled();
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: Tests reveal that different events need different enrichment strategies.

Step 7: Add Dead Letter Queue

Red Phase: Test handling of persistent failures

test('should send to DLQ after all retries fail', async () => {
  mockApiClient.getUserProfile
    .mockRejectedValue(new Error('Service unavailable'));

  await processor.process(event);

  expect(mockDLQ.send).toHaveBeenCalledWith({
    originalEvent: event,
    error: expect.stringContaining('after 3 attempts')
  });
});
Enter fullscreen mode Exit fullscreen mode

Key Insight: TDD drives us to implement proper failure handling strategies.

The Final Architecture

Through TDD, we've built a robust event processor with:

  • ✅ Input validation
  • ✅ Retry logic with configurable attempts
  • ✅ Different transformation strategies per event type
  • ✅ Dead letter queue for failed events
  • ✅ Proper error handling and logging

Key Lessons from TDD Integration Development

1. Start with the Contract

Tests define what your external dependencies should provide. This makes integration points explicit.

2. Embrace Mocking

Mocks aren't just for isolation - they're design tools that help you discover the right interfaces.

3. Error Paths First

TDD forces you to think about failures before successes. In integration code, this is invaluable.

4. Progressive Enhancement

Each test adds a new capability. You build complexity incrementally, not all at once.

5. Tests as Documentation

Your test suite becomes living documentation of how the system handles various scenarios.

TDD Benefits for Integration Code

  • Confidence in Error Handling: You've tested failures before they happen
  • Clear Dependencies: Tests make external dependencies explicit
  • Refactoring Safety: Comprehensive tests let you improve code fearlessly
  • Better Design: TDD drives you toward loosely coupled, testable code
  • Reduced Debugging: Issues are caught during development, not in production

When to Write Integration Tests vs Unit Tests

Unit Tests (with mocks):

  • Business logic validation
  • Transformation rules
  • Error handling flows
  • Retry mechanisms

Integration Tests:

  • End-to-end flow validation
  • Contract verification
  • Performance characteristics
  • Real failure scenarios

Common Pitfalls to Avoid

  1. Over-mocking: Don't mock internal implementation details
  2. Testing Implementation: Focus on behavior, not how it's coded
  3. Skipping Error Cases: These are often more important than happy paths
  4. Tight Coupling to Tests: Keep tests focused on public interfaces

Conclusion

TDD transforms integration development from a debugging nightmare into a systematic process. By writing tests first, we:

  • Design better interfaces
  • Handle errors gracefully
  • Build confidence in our code
  • Create maintainable systems

The key is to start simple, test one behavior at a time, and let the tests guide your design. Your future self (and your ops team) will thank you when those 3 AM integration failures don't happen because you've already tested for them.

Top comments (1)

Collapse
 
nikhil-amin profile image
Nikhil Amin •

Great read! Loved how you applied TDD to real-world integrations—especially the retries, validations, and DLQ handling. Clean and practical approach! 👏