Service Discovery in Microservices With .NET and Consul

Microservices have revolutionized how we build and scale applications. By breaking down larger systems into smaller, independent services, we gain flexibility, agility, and the ability to adapt to changing requirements quickly. However, microservices systems are also very dynamic. Services can come and go, scale up or down, and even move around within your infrastructure.

This dynamic nature presents a significant challenge. How do your services find and communicate with each other reliably?

Hardcoding IP addresses and ports is a recipe for fragility. If a service instance changes location or a new instance spins up, your entire system could grind to a halt.

Service discovery acts as a central directory for your microservices. It provides a mechanism for services to register themselves and discover the locations of other services.

In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.

What is Service Discovery?

Service discovery is a pattern that allows developers to use logical names to refer to external services instead of physical IP addresses and ports. It provides a centralized location for services to register themselves. Clients can query the service registry to find out the service's physical address. This is a common pattern in large-scale distributed systems, such as Netflix and Amazon.

Here's what the service discovery flow looks like:

1. The service will register itself with the service registry
2. The client must query the service registry to get the physical address
3. The client sends the request to the service using the resolved physical address

The same concept applies when we have multiple services we want to call. Each service would register itself with the service registry. The client uses a logical name to reference a service and resolves the physical address from the service registry.

The most popular solutions for service discovery are Netflix Eureka and HashiCorp Consul.

There is also a lightweight solution from Microsoft in the Microsoft.Extensions.ServiceDiscovery library. It uses application settings to resolve the physical addresses for services, so some manual work is still required. However, you can store service locations in Azure App Configuration for a centralized service registry. I will explore this service discovery library in some future articles.

But now I want to show you how to integrate Consul with .NET applications.

Setting Up the Consul Server

The simplest way to run the Consul server locally is using a Docker container. You can create a container instance of the hashicorp/consul image.

Here's an example of configuring the Consul service as part of the docker-compose file:

consul:
  image: hashicorp/consul:latest
  container_name: Consul
  ports:
    - '8500:8500'

If you navigate to localhost:8500, you will be greeted by the Consul Dashboard.

Now, let's see how to register our services with Consul.

Service Registration in .NET With Consul

We'll use the Steeltoe Discovery library to implement service discovery with Consul. The Consul client implementation lets your applications register services with a Consul server and discover services registered by other applications.

Let's install the Steeltoe.Discovery.Consul library:

Install-Package Steeltoe.Discovery.Consul

We have to configure some services by calling AddServiceDiscovery and explicitly configuring the Consul service discovery client. The alternative is calling AddDiscoveryClient which uses reflection at runtime to determine which service registry is available.

using Steeltoe.Discovery.Client;
using Steeltoe.Discovery.Consul;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddServiceDiscovery(o => o.UseConsul());

var app = builder.Build();

app.Run();

Finally, our service can register with Consul by configuring the logical service name through application settings. When the application starts, the reporting-service logical name will be added to the Consul service registry. Consul will store the respective physical address of this service.

{
  "Consul": {
    "Host": "localhost",
    "Port": 8500,
    "Discovery": {
      "ServiceName": "reporting-service",
      "Hostname": "reporting-api",
      "Port": 8080
    }
  }
}

When we start the application and open the Consul dashboard, we should be able to see the reporting-service and its respective physical address.

Using Service Discovery

We can use service discovery when making HTTP calls with an HttpClient. Service discovery allows us to use a logical name for the service we want to call. When sending a network request, the service discovery client will replace the logical name with a correct physical address.

In this example, we're configuring the base address of the ReportingServiceClient typed client to http://reporting-serviceand adding service discovery by calling AddServiceDiscovery.

Load balancing is an optional step, and we can configure it by calling AddRoundRobinLoadBalancer or AddRandomLoadBalancer. You can also configure a custom load balancing strategy by providing an ILoadBalancer implementation.

builder.Services
    .AddHttpClient<ReportingServiceClient>(client =>
    {
        client.BaseAddress = new Uri("http://reporting-service");
    })
    .AddServiceDiscovery()
    .AddRoundRobinLoadBalancer();

We can use the ReportingServiceClient typed client like a regular HttpClient to make requests. The service discovery client sends the request to the external service's IP address.

app.MapGet("articles/{id}/report",
    async (Guid id, ReportingServiceClient client) =>
    {
        var response = await client
            .GetFromJsonAsync<Response>($"api/reports/article/{id}");

        return response;
    });

Takeaway

Service discovery simplifies the management of microservices by automating service registration and discovery. This eliminates the need for manual configuration updates, reducing the risk of errors.

Services can discover each other's locations on demand, ensuring that communication channels remain open even as the service landscape evolves. By enabling services to discover alternative service instances in case of outages or failures, service discovery enhances the overall resilience of the microservices system.

Mastering service discovery gives you a powerful tool to build modern distributed applications.

You can grab the source code for this example here.

Thanks for reading, and I'll see you next week!

---

P.S. Whenever you're ready, there are 3 ways I can help you:

1. Pragmatic Clean Architecture: Join 2,900+ students in this comprehensive course that will teach you the system I use to ship production-ready applications using Clean Architecture. Learn how to apply the best practices of modern software architecture.

2. Modular Monolith Architecture: Join 750+ engineers in this in-depth course that will transform the way you build modern systems. You will learn the best practices for applying the Modular Monolith architecture in a real-world scenario.

3. Patreon Community: Join a community of 1,050+ engineers and software architects. You will also unlock access to the source code I use in my YouTube videos, early access to future videos, and exclusive discounts for my courses.