DEV Community: Mark Henke

What Is Whitebox Monitoring? Everything You Need to Know

Mark Henke — Tue, 10 Mar 2020 16:11:12 +0000

When I first heard about whitebox monitoring, I thought I knew what it was just by name. But like many software terms, it's easy to conflate one thing for another. It's also easy to jump to conclusions too quickly and think you know everything about a term. Whitebox monitoring is a very valuable tool in our DevOps toolbox. Because of this, it's prudent for us to ensure we fully understand what it is and how to use it.

We're going to cover quite a bit about whitebox monitoring: why it's valuable, how it differs from blackbox monitoring, and how such monitoring is implemented in a system.

Blackbox vs. Whitebox Monitoring

Whitebox has often come with its counterpart, blackbox, through many variations. One famous example is blackbox vs. whitebox testing. I'll quickly cover the difference between blackbox and whitebox monitoring.

Blackbox Monitoring

The name blackbox invokes a feeling of mystery. Black is opaque; we cannot see through it. In the same way, we can't see into the system we're monitoring. Take a series of houses on a street, for example. From the outside, we can know a few things. We can see if anyone is home by whether the lights are on or how many cars are in the driveway. If we read the gas meter, we know how much gas they use per month. It's the same for the water meter and water usage.

But we don't know what happens inside. We have no idea what sort of interesting celebrations, arguments, or hobbies could be happening in these houses. It's the same for monitoring. I can monitor the traffic and responses of the software. I can measure the CPU and memory utilization. But in blackbox monitoring, I have no idea what's happening inside these requests. I don't know what database calls it's making or what sort of fancy rules it applies to the data.

Whitebox Monitoring

Whitebox monitoring is, of course, the opposite of blackbox. The idea of a whitebox invokes a feeling of transparency, though maybe clearbox would be a better name. Imagine a household with a swear jar, a calendar of events, or a weekly menu. I can record and watch these things change, as long as I have a window to see inside. Whitebox monitoring is the same. It's the act of putting windows into our code and showing the outside world, usually our developers, what's happening inside.

What Do I Get out of Whitebox Monitoring?

If you know exactly what's happening inside your application, you can easily answer questions that may pop up from time to time. Back to our household, what if we wanted to know how often they eat fish? Well, we just need to peek through the window and record their menu. We keep doing this every week, and we can get an average for how much fish they eat per month.

With software, having such things gives us some key benefits in two categories: operations and product insights.

Operations

Operational whitebox monitoring helps us keep our system alive and healthy. It also helps us keep the risk low when trying new architectures and techniques. It does this by giving us quick feedback once we've released something. For example, if we see traffic is slowing down for one of our endpoints, we need a way to drill down and see what internal method is causing the bottleneck. Monitoring database queries is a fantastic example of a whitebox monitor. It gives us insight into a key part of our system that is often the cause of slowdowns. Operational whitebox monitoring helps us build a thing right.

Product Insights

While operational monitoring helps us build a thing right, product insight monitors help us build the right thing. They help us answer the question, are we making customers happy? One popular version of this monitoring is shopping cart abandonment. It's common in e-commerce to see how often customers put things inside a shopping cart, but never place an order. By looking at what they put in, or where they clicked next, we can improve our systems to encourage customers to place more orders.

How Do I Wire up Whitebox Monitoring?

Armed with an understanding of what whitebox monitoring is, the question naturally pops up on how to implement it. This is extremely sensitive to the languages and frameworks you're using, but I'll describe some important characteristics here.

Monitoring Categories

There are two main categories of whitebox monitoring that affect how we implement them: event-centric monitors and time-series data.

Event-Centric Monitors

Event-centric monitoring is where we report out interesting events from our system. For our household, this could be tallying on a chalkboard every time a family has an argument. We may record what room the argument was in, how many decibels the speaking reached, and how long it lasted. We don't know up front how the data will be used—we just know it's interesting. With software, this could be things like what web requests occurred, whether they were successful, how many database calls they made, and how long they took.

Event-centric data is great when we need to answer questions that we don't know we have ahead of time. Oftentimes production incidents catch us by surprise, and we need answers right now! Event-centric data can give us these answers.

Time-Series Data

Time-series data is information that is aggregated over a time period. For our households, it might be the number of dishes used per week or pounds of food consumed per day. In software, this can be things like the number of requests per minute or average latency per hour. The key to time-series data is that it's aggregated inside the application before being sent through our window. This makes it ready to be shown in graphs so that we can see trends over time.

Reporting

The cost of whitebox monitoring is that it must be built into the app as a first-class concept. I need the application itself to tell me what's going on inside it. In a household, this would be equivalent to the family telling me what they ate that night or giving me a key to their house. This usually consists of a reporter component that blasts out the data to somewhere else. It also involves some sort of instrumentation that plugs into our workflow life cycle. Many frameworks give us extensions to plug in such instrumentation. Sometimes we have to do it ourselves via things like aspect-oriented programming.

Key Traits

However we report, here are some key traits. Most reporters, especially event-centric ones, should be asynchronous. This is because we don't want the time it takes to report out the data to interfere with our responsiveness to customers. We should also ensure its failures are isolated. We don't want a failed log event to stop up someone trying to make a payment!

Storage

Once we have a reporter, we need something to report to. Sometimes we report to multiple places. We should have a persistent store somewhere that we can send this data to. As a note, this place should exist outside our production system. It'd be quite a pain to troubleshoot when production is down if your monitoring goes down with it!

For the record, Scalyr offers both storage and reporters as part of its product. Other offerings may focus solely on one or the other. Spring Sleuth, for example, only handles reporting and instrumentation. It leaves where to put it up to you.

Viewing Monitors

Now that we have the data stored, we likely want to see it and see it frequently. We have a few options for that.

Dashboard

The almighty dashboard is the most popular way for visualizing monitors. There's nothing quite like the feeling of confidence a trending graph gives. Dashboards excel at giving you a pulse of the system at a glance. Healthy ones let you see patterns and detect abnormalities. They should be uncluttered, focusing only on a few things. The flip side to this is that they should easily let you drill down at runtime into more specific data. This will make it easier to diagnose problems quickly.

Search

Search is the ultimate ability to answer questions you didn't know you would need to ask in advance. Searchable storage works very well with event-centric monitoring. It gives you the full flexibility to ask very specific questions about your system. No amount of dashboards can always tell you all the information about a bug, but searching can.

Alerts

Alerts are when the system tells you when something is going on, as opposed to the other way around. These are great for letting you relax, and not having to spend time every day looking through graphs or logs. They let the system do the hard work of detecting abnormalities. On the other hand, they can easily get out of hand. I have seen badly managed alerts that can ping you every five minutes for the next four hours while your system is down.

What to Monitor

Once we get whitebox monitoring up and running, we want to leverage it effectively. I'll be using a couple of ideas from Google's Site Reliability Engineering book to touch on this.

Service Level Indices

Service level indices are the baseline of what's happening in your system today. They're useful to get a pulse on how your system works. You may find some surprising results. What things you want to index are usually based on what service level objectives you have.

Service Level Objectives

This is one of the keys to all monitoring. Monitoring is by no means a purely technical endeavor. A team should have objectives on how to make its customers happy. If your system deals with payments it could be "we want payments 20% more likely to succeed without a snag." In this case, let's measure the rate of successful vs. failed payment transactions. Service level objectives are something for which you should intimately collaborate with all of your stakeholders. You all are responsible for these, and they'll help guide what monitors you plug in.

Conclusion

In a household, there are many interesting things that happen inside. We have to explicitly create systems that let us measure those things, like swear jars. In the same way, software systems are full of valuable details that let us peer into what's happening. Building whitebox monitoring into our app will let us turn these details into valuable insights. There are many tools, like Scalyr, that help us implement this quickly, leaving us time to figure out what service level objectives we want to measure.

Top 4 Ways to Make Your Microservices Not Actually Microservices

Mark Henke — Tue, 04 Feb 2020 17:07:49 +0000

Microservices have gained such a level of popularity these days that they're often touted as "the way" to build software. That's all well and good, except there are a lot of microservice anti-patterns flying around.

The vast majority of microservice systems I've seen aren't really microservices at all. They're separately deployable artifacts, sure. But they're built in a way that, for developers, causes more pain than solves problems.

So let's dive a little more into this. First, we'll talk about what a microservice was intended to be. Then we'll get into some of these anti-patterns.

What Does "Microservice" Mean?

As with many terms in the software industry, we have conflated "microservice" with plenty of things. For most people, it usually seems to mean "a deployment artifact." However, it's more than that. Members of ThoughtWorks developed the term in 2014. They were combating against the abuse of service-oriented architecture while still driving towards good software system design. As described here, microservices:

Are autonomous, relying minimally on other services through lightweight communication mechanisms.
Center around a business capability, in a similar vein to bounded contexts.
Are heavily reliant on automated infrastructure to get into production.
Require monitoring and good tooling to stitch together insights into what's happening across services.
Are designed for failure. A dependency going down should not bring down the entire service.

If we break these traits, then what we call microservices are not actually microservices.

How to Turn Your Microservice Into an Anti-Pattern

Let's explore some common ways that teams take this original idea of microservices and twist it. I'll share some of the top microservice anti-patterns that I have seen in the wild.

1. Ensure Your Services Are Around Technical Concerns

Centering your services around technical concerns is a surprisingly common microservice anti-pattern. I believe this mostly comes from combining two principles incorrectly: the single responsibility principle and clean architecture. Clean architecture says to use ports and adapters to isolate your business logic from things that need to use your business logic.

This is all well and good, but it can easily mutate into something called layering, which is not all well and good. Layering was a key factor in the failure of service-oriented architecture. When you view everything in layers, your single responsibility sense kicks in and you say, "Well, I need to separate my persistence logic from my business rules logic."

So when someone says "microservices," you decide you'll follow what you have and make all these layers into different, separately deployed services. They may end up looking like this: security service, data services, orchestration service, business service. This goes directly against the idea that microservices center around a business capability. Healthy microservices would look more like this: shipping service, inventory service, ordering service.

It's also common to slice out cross-cutting concerns, like auditing or email notifications, into a separate service. This will have the same pain points as layered services.

When you follow this anti-pattern, you ensure that your system will be:

Very slow due to the latency in network calls across services.
Susceptible to network failure since if one layer fails to get its data across the network, all downstream layers fail.
Very hard for developers to add new features as they struggle to follow the code through all the layers.

You can avoid this through diligent design and understanding your customer's needs. You need subject matter experts to help guide you through the different aspects of your business. Then, you can separate them out. It's often helpful to start with a well-modularized monolith before splitting it into separate services.

2. Ensure Low Autonomy

Let's say you've successfully avoided the first anti-pattern: your services have pristine names like inventory, purchasing, product, and customer support. However, you realize that your inventory service needs to talk to the product service in order to get some details on where in the warehouse a product belongs. And then you need to connect to the customer support service so that you can figure out which products customers really want so you can stock them appropriately.

Of course, whenever you run a request, you have to ping each individual service for information. Before you know it, you have a tangle of services all trying to talk to each other to do their jobs.

It can be even worse when two services try to talk back and forth like this:

If your services follow this anti-pattern, they're no longer microservices. That's because microservices need to be (relatively) autonomous. They should not heavily depend on other services to do their work. By following this anti-pattern, you ensure that:

Development is painful since you need to go back and forth to add or change functionality if within the same team.
Each service team incurs a coordination cost to try to get both teams to align on when to make changes.
Failure of one affects the others.
Developers will get confused about who owns what data.

You can avoid this by doing a couple things. First, ensure you that you diligently figure out the boundaries such that the correct service owns the correct data. In the diagrammed case, inventory should own stock and storage locations but should not care about price or invoice numbers. Perhaps inventory may need to know when a customer places a new order, so it can fulfill the order. Or purchasing might need to know when an item is not in stock. In these cases, you can use event-driven architecture to ensure that when something happens to one service, the other services can follow up with their own actions.

3. Have a Tightly-Coupled Orchestration Service

In large organizations, it's popular to slice development teams horizontally, similar to layering. They then try to bring it all back together with an orchestration service. Orchestration acts as a bandage over their self-inflicted wounds.

An orchestration service is not a microservice. Instead, it's a vicious anti-pattern that is hard to combat. These services don't revolve around a business capability. They're also not autonomous. These types of services are not designed for failure. You cannot easily deploy them independently, as they are often useless until their underlying services are deployed.

The pain of orchestration services is similar to the above microservice anti-patterns. They:

Slow everything down by adding network latency.
Cannot respond faster than the slowest underlying service.
Slow development across all services, as teams have to coordinate with orchestration to get their features out.
Will often be blamed for any problems the underlying services have, and will not be equipped to understand these failures.

If you find yourself in this position, there are some things you can do. You can build your orchestrator to be as schema-agnostic as possible. Try not to transform anyone's data unless absolutely necessary. If possible, push the composition of these services up to the customer-facing application.

4. Ensure That Your Service Fails When Another Service Fails

You may have avoided the pitfalls above and have only a few dependencies on other services when you make requests. But what happens when those underlying services fail? Teams may have low coupling to others, but if those dependencies fail, your service could grind to a halt. This is especially true of orchestration services. Taking our purchasing/inventory example, what happens if someone tries to place an order with purchasing, but inventory says the item is out of stock? Or even worse, what if inventory is not responding? Do you give up?

If you don't design for failure, then your service is not a microservice. If you don't design for failure, you might:

Get blamed by your consumers for failures that you do not own.
Lose customer trust with a higher-than-expected fail rate.
Spend an inordinate amount of time spent on production support.

You can avoid this through various strategies. As stated above, trying to decouple the services through events works well. You can also add retries around your service calls, avoiding intermittent failures like network timeouts or dropped packets. If you're sending commands, you can add queuing and offline processing. Then you can put the command "on pause" and try again when things are working. If you're reading data, you can show an "empty value" on failure. You can also have a fallback cache of values to use if the service is currently non-responsive.

In Conclusion: Learn More

Microservices can be useful for handling large systems. However, you can see how easy it is to mess up and not have microservices at all, even though you think you do.

Follow these anti-patterns and you're guaranteed to have a hard time. Be careful and diligent when separating your services. Learn as much as you can about your business processes and follow those seams when you design systems. Understand what microservices are meant to do and design accordingly, and you'll save yourself lots of pain in the long run.

Spring Boot Microservices—Examples to Help You Get Started

Mark Henke — Tue, 01 Oct 2019 13:38:32 +0000

It can feel daunting to build a new microservice. It feels like there are a lot of things to keep in mind. Fortunately, Spring has a variety of libraries that can get us started quickly. We can focus on the bits that matter to us and let Spring scaffold the rest. In this post, we're going to take a look at what makes microservices different from other types of applications and how Spring helps us get up and running fast.

What Do We Mean by "Microservice"?

So, what exactly do we mean when we talk about microservices? They have their origins in a very specific type of service. It's not just any deployment artifact. In Martin Fowler's article about microservices, he mentions a few key characteristics that separate microservices from just another deployed app:

Componentization via services.
Organized around business capabilities.
Products, not projects.
Decentralized data management.
Design for failure.

We're going to take a look at which Spring libraries help us achieve these characteristics. But before that, let's talk about how we can easily set up a new microservice with any set of libraries.

Spring Initializr, Our Launchpad

You're about to find out just how overwhelming the amount of Spring libraries that exist for building microservices is. Fortunately, there are two great tools to help us in our path. The first is Spring Initializr. This little site will get you up and running with a new Spring project in minutes, along with all the components you want to use.

The second tool is the website Baeldung, which is chock-full of in-depth Spring tutorials on all of the libraries we'll be using. You can use this site to dive deeper to any library for interest. They also have open-source code examples for their tutorials, from which we will be borrowing.

Now, onto the libraries.

Componentization Via Services

Componentizing into services is the idea that a microservice is independently deployable and runnable. In this vein, let's look at libraries that help us start up our application.

Spring Boot

Let's start with the foundation of everything: Spring Boot. This library is the basis of almost every other Spring library out there. Spring Boot sets up our application context, wiring up all our software components It also makes it really easy to execute our JAR—our software package—as a console application.

To include Spring Boot in your project, use Spring Initializr or add the following:

buildscript {
   ext {
      //This is the most recent version at the time of writing. 
      springBootVersion = '2.1.1.RELEASE'
   }
   dependencies {
      //This makes the jar executable.
      classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
   }
}
...
apply plugin: 'org.springframework.boot'
//This makes it easier to manage the correct versions of the Spring libraries. The bill of materials ensures the versions are compatible with each other.
apply plugin: 'io.spring.dependency-management'
...
repositories {
    mavenCentral()
        //Some of the libraries we discuss are stored in the milestones repository.
    maven { url "https://repo.spring.io/milestone" }
}
...
dependencies {
   //This let's our code sping up a Spring ApplicationContext in our Main method
   implementation('org.springframework.boot:spring-boot-starter')
   //This gives us some unit testing utilities and runners
   testImplementation('org.springframework.boot:spring-boot-starter-test')
}
...
dependencyManagement {
 imports {
    //This is the actual bill of materials for spring dependencies
    mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
 }
}

For the examples in this post, I'll be using Gradle for dependency management, but you can also use Maven. Spring Initializr supports both.

With our dependencies in place, our application main method can look like this:

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

And that's all you need to get started! Note, however, that there's a lot more functionality built into Spring Boot than this; explore it at your leisure.

Organized Around Business Capabilities

Microservices should be aligned only with business concepts like ordering, fulfillment, shipping, and customer service. They shouldn't be centered around concepts like data access, authentication, or email. Spring doesn't directly help with figuring out proper business boundaries. In fact, this is probably one of the hardest aspects of creating healthy microservices, and it's out of scope for this article. However, once you figure out your boundaries, Spring provides some libraries that will let you expose this functionality to your customers.

Web

Spring Web is a classic library that allows us to serve up both web pages and HTTP endpoints to our users. It also spins up an embedded Tomcat web server and binds to a port, so we can talk to the wider world.

The web dependency will look like this:

dependencies {
        //This can replace the "spring-boot-starter" dependency from above. 
    implementation('org.springframework.boot:spring-boot-starter-web')
    ...
}

With that in place, we can build HTTP endpoints:

@Controller
public class SimpleController {

    @GetMapping("api/hello")
    public String homePage() {
        return "hello";
    }
}

If I GET /hello with curl or Postman, I'll see a response with "hello" as the only content.

There are loads more capabilities with Spring Boot Web. What I have shown here just barely scratches the surface. You'll have the ability to add security, deal with exceptions, add request/response middleware, and much more. Try out some tutorials on it.

Alternatives

Spring has a newer variant of building web apps called Spring WebFlux. This variant aims to be more reactive and to more easily support asynchronous, scalable operations.

Products, Not Projects

Products over projects is the spirit of DevOps. You build it, you run it. You can't just deploy a microservice into the ether—you have to monitor it and maintain it. Spring has a couple of libraries that can help us not only build but also run our microservices.

Actuator

You set it up like so:

dependencies {
   implementation('org.springframework.boot:spring-boot-starter-actuator')
   ...
}

By default, only the /actuator/info and /health endpoints are enabled. You can enable all endpoints in your property file with:

management.endpoints.web.exposure.include=*

There are many endpoints in Actuator, and I recommend exploring them all. You can also see them through the /actuator endpoint. My favorite is /actuator/metrics. Its response looks something like this:

{
  "names" : [ "jvm.memory.max", "jvm.memory.used", "jvm.memory.committed", "jvm.buffer.memory.used", "jvm.buffer.count", "jvm.buffer.total.capacity" ]
}

If you drill into a specific one, such as /actuator/metrics/jvm.memory.max, you can see something like this:

{
  "name" : "jvm.memory.max",
  "description" : "The maximum amount of memory in bytes that can be used for memory management",
  "baseUnit" : "bytes",
  "measurements" : [ {
    "statistic" : "VALUE",
    "value" : 2.384986111E9
  } ],
  "availableTags" : [ {
    "tag" : "area",
    "values" : [ "heap", "nonheap" ]
  }, {
    "tag" : "id",
    "values" : [ "Compressed Class Space", "PS Survivor Space", "PS Old Gen", "Metaspace", "PS Eden Space", "Code Cache" ]
  } ]
}

This immediately gives you a level of insight into your latency, error rates, and so on. You can also customize existing or new actuator endpoints as you desire.

Sleuth

It's highly likely that our microservices aren't running in isolation. At the end of the day, they have to communicate with queues, databases, and even other microservices to do their job fully. When things go wrong, it can be hard to track all the work that has happened in a request. If I want to be able to quickly debug issues across multiple deployed services, I need some tooling. Spring Sleuth lets us trace these requests across microservice boundaries. It can even let us trace to database calls.

We add it like so:

dependencies {
   implementation('org.springframework.cloud:spring-cloud-starter-sleuth')
   ...
}

After this, we technically don't need to wire up any more code. It works with other Spring libraries to add tracing context when calling other services and databases. You can see this context when you log:

2018-01-10 22:36:38.254  INFO 
  [Microservice Starter,4e30f7340b3fb631,4e30f7340b3fb631,false] 12516 
  --- [nio-8080-exec-1] c.b.spring.session.SleuthController : Hello Sleuth

The first GUID is the trace ID, which is the same across the request. The next GUID is the span ID, which represents the current unit of work. Having this context lets us query and group our log messages in a way that lets us see the life cycle of a request. You can also report these traces to external storage, but that is outside the scope of this article. Read this article by Baeldung for more information.

Decentralized Data Management

A microservice should own its data through and through and have minimal coupling to another service's data. No one else should be able to access its data directly.

JPA With SQL Server

Spring makes it easy for a microservice to own its own data for multiple data stores. Using the Spring JPA library lets us use Hibernate and the JPA specification to interact with a relational database like SQL Server.

You can wire it up like this:

dependencies {
    implementation('org.springframework.boot:spring-boot-starter-data-jpa')
    runtimeOnly('com.microsoft.sqlserver:mssql-jdbc')
        ...
}

You then enable it in your application or an @Configuration:

@EnableJpaRepositories("org.scalyr.persistence.repo") 
@EntityScan("org.scalyr.persistence.model")
@SpringBootApplication
public class Application {
   ...
}

Then we can map our classes to database tables:

@Entity
public class Book {
  
    @id

    @GeneratedValue(strategy = GenerationType.AUTO)
    private long id;
 
    @Column(nullable = false, unique = true)
    private String title;
 
    @Column(nullable = false)
    private String author;
}

and use Spring repositories to work with the data:

public interface BookRepository extends CrudRepository<Book, Long> {
    List<Book> findByTitle(String title);
}

For more information, check out this tutorial.

Cloud Stream With Rabbit

Decentralizing data is a powerful way to keep microservices autonomous, but it's inevitable that some of this data will need to be shared across services. We don't want to share our databases, and we want to avoid runtime coupling on other services when possible. After all, we can't count on those services always being up and running. We can have our cake and eat it, too, by sharing data through event-driven messaging. Spring Cloud Stream with RabbitMQ makes this relatively easy to do.

We can add the dependencies as so:

dependencies {
    implementation('org.springframework.cloud:spring-cloud-starter-stream-rabbit')
    testImplementation('org.springframework.cloud:spring-cloud-stream-test-support')
}

Then we can publish and subscribe to messages through Rabbit queues in our application:

@SpringBootApplication
@EnableBinding(Processor.class)
public class MyLoggerServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(MyLoggerServiceApplication.class, args);
    }
 
    @StreamListener(Processor.INPUT)
    @SendTo(Processor.OUTPUT)
    public LogMessage enrichLogMessage(LogMessage log) {
        return new LogMessage(String.format("[1]: %s", log.getMessage()));
    }
}

INPUT and OUTPUT are built-in channels that let us specify from where we subscribe to messages and to where we publish them. We need to bind Rabbit to these channels:

spring:
  cloud:
    stream:
      bindings:
        input:
          destination: queue.log.messages
          binder: local_rabbit
          group: logMessageConsumers
        output:
          destination: queue.pretty.log.messages
          binder: local_rabbit
      binders:
        local_rabbit:
          type: rabbit
          environment:
            spring:
              rabbitmq:
                host: localhost
                port: 5672
                username: guest
                password: guest
                virtual-host: /
server:
  port: 0
management:
  health:
    binders:
       enabled: true

The application code remains blissfully ignorant of the specific transportation being used to publish or receive messages—we push all of that to the above configuration. You can see that we bound the INPUT and OUTPUT channels to specific exchanges in Rabbit. These exchanges and queues are automatically declared for us by the Cloud Stream Rabbit library. For more information, check out this article.

Alternatives

Spring also has support for NoSQL databases, like MongoDB. Spring supports just about any popular persistence mechanism. If using Cloud Stream support, we can use Kafka instead of RabbitMQ.

Design for Failure

When dealing with distributed, autonomous services, we can't count on them being up at all times. When communicating with other services, we should be ready to handle the inevitable.

Hystrix

Netflix built a library called Hystrix that lets us apply the circuit breaker pattern when communicating with other services. Using circuit breakers when communicating externally gives us a measure of resiliency to system outages. We can fall back to a default behavior when the service with which we want to communicate is unavailable.

The dependency is:

dependencies {
    implementation('org.springframework.cloud:spring-cloud-starter-netflix-hystrix')
}

The configuration is dead simple:

@HystrixCommand(
  commandKey = "ratingsByIdFromDB", 
  fallbackMethod = "findCachedRatingById", 
  ignoreExceptions = { RatingNotFoundException.class })
public Rating findRatingById(Long ratingId) {
    return Optional.ofNullable(ratingRepository.findOne(ratingId))
      .orElseThrow(() -> 
        new RatingNotFoundException("Rating not found. ID: " + ratingId));
}
 
public Rating findCachedRatingById(Long ratingId) {
    return cacheRepository.findCachedRatingById(ratingId);
}

You can see that if the repository call fails, we can use a cached version. Feel free to read more here.

Retry

In many cases, circuit breaking may be a bit of overkill. Often we have intermittent network failures that we can overcome with a simple retry. Enter Spring Retry.

We add it to our Gradle build with:

dependencies {
    implementation('org.springframework.retry:spring-retry'))
        ...
}

We then enable it via:

@Configuration
@EnableRetry
public class AppConfig { ... }

And we implement it with:

@Service
public interface MyService {
    @Retryable(
      value = { SQLException.class }, 
      maxAttempts = 2,
      backoff = @Backoff(delay = 5000))
    void retryService(String sql) throws SQLException;
    ...
}

This says, "Please retry 'retryService' up to two times if you see an SQLException, and wait 5,000 ms between each retry."

... And Many More

As you can see, Spring provides a myriad of libraries we can use to get a large boost into building microservices. We have much of what we need, from providing APIs to accessing data and even monitoring our application once it's in production. There are many more libraries you can use for more advanced use cases, so go out and explore using Spring for your own microservices.

Want to read more about Spring? We covered it in our "getting started quickly with logging" series, so head there next!