DEV Community

shubham goel
shubham goel

Posted on

Understanding Retries and Failures in a Kubernetes Operator

In the previous post, I added status and conditions to my Greeting custom resource.

That gave me something like:

status:
  observedGeneration: 1
  configMapName: hello-greeting

  conditions:
    - type: Ready
      status: "True"
      reason: ConfigMapReady
      message: Managed ConfigMap hello-greeting is in the desired state
Enter fullscreen mode Exit fullscreen mode

This was useful for the successful path.

But until now, everything was mostly working.

So the next question was:

What happens when reconciliation fails?

For example, what if the operator cannot create the ConfigMap?

That led me into failure handling, retry behavior, and Kubernetes Events.


Current Flow

The normal flow looks like this:

Greeting
    |
    v
Greeting Operator
    |
    v
ConfigMap
    |
    v
Ready=True
Enter fullscreen mode Exit fullscreen mode

But I also want the failed path to be visible:

Greeting
    |
    v
Greeting Operator
    |
    v
Failure
    |
    +----> Ready=False
    |
    +----> Warning Event
    |
    +----> Retry
Enter fullscreen mode Exit fullscreen mode

The goal is not only to log the exception.

The custom resource itself should explain that something went wrong.


Adding Failure Status

When reconciliation fails, I update the Greeting status to:

status:

  conditions:
    - type: Ready
      status: "False"
      reason: ReconciliationFailed
      message: ...
Enter fullscreen mode Exit fullscreen mode

Conceptually:

reconcile()
    |
    v
exception
    |
    v
update status
    |
    v
Ready=False
Enter fullscreen mode Exit fullscreen mode

The full exception still goes to the operator logs.

But the custom resource gets a shorter and more useful message.

I think about it like this:

Operator logs
=
developer debugging
Enter fullscreen mode Exit fullscreen mode

while:

status.message
=
resource-level information
Enter fullscreen mode Exit fullscreen mode

Error Status Handling

The reconciler now handles failures using the Java Operator SDK error status mechanism.

The flow is roughly:

@Override
public ErrorStatusUpdateControl<Greeting> updateErrorStatus(
        Greeting greeting,
        Context<Greeting> context,
        Exception exception) {

    var status = greeting.getStatus();

    if (status == null) {
        status = new GreetingStatus();
    }

    status.setObservedGeneration(
            greeting.getMetadata().getGeneration()
    );

    status.setConditions(
            List.of(
                    createCondition(
                            status,
                            greeting.getMetadata().getGeneration(),
                            "False",
                            "ReconciliationFailed",
                            errorMessage(exception)
                    )
            )
    );

    greeting.setStatus(status);

    return ErrorStatusUpdateControl.patchStatus(greeting);
}
Enter fullscreen mode Exit fullscreen mode

So now the resource can move between:

Ready=True
Enter fullscreen mode Exit fullscreen mode

and:

Ready=False
Enter fullscreen mode Exit fullscreen mode

depending on the reconciliation result.


Recording Kubernetes Events

I also wanted failures to be visible through normal Kubernetes tooling.

So the operator now records Kubernetes Events.

For a successful reconciliation:

Normal  Reconciled
Enter fullscreen mode Exit fullscreen mode

For a failure:

Warning  ReconciliationFailed
Enter fullscreen mode Exit fullscreen mode

This means I can run:

kubectl describe greeting hello
Enter fullscreen mode Exit fullscreen mode

and see what happened directly on the resource.

For example:

Events:
  Type     Reason                  Message
  ----     ------                  -------
  Normal   Reconciled              Managed ConfigMap hello-greeting is in the desired state
Enter fullscreen mode Exit fullscreen mode

During a failure:

Events:
  Type     Reason                  Message
  ----     ------                  -------
  Warning  ReconciliationFailed    ...
Enter fullscreen mode Exit fullscreen mode

This is much nicer than requiring someone to immediately search operator logs.


RBAC for Events

Because the operator now creates Kubernetes Events, it also needs permission to do that.

I added:

- apiGroups:
    - ""

  resources:
    - events

  verbs:
    - get
    - create
    - patch
Enter fullscreen mode Exit fullscreen mode

So the operator permissions now include:

watch Greetings
update Greeting status
manage ConfigMaps
record Kubernetes Events
Enter fullscreen mode Exit fullscreen mode

Creating a Real Failure

Instead of adding something artificial like:

spec:
  fail: true
Enter fullscreen mode Exit fullscreen mode

I wanted to create a real Kubernetes authorization failure.

The Greeting operator normally has permission to create ConfigMaps through its ClusterRole.

I opened the ClusterRole directly:

kubectl edit clusterrole greeting-operator
Enter fullscreen mode Exit fullscreen mode

The ConfigMap permissions normally look something like:

- apiGroups:
    - ""

  resources:
    - configmaps

  verbs:
    - get
    - list
    - watch
    - create
    - update
    - patch
    - delete
Enter fullscreen mode Exit fullscreen mode

For this experiment, I temporarily removed the write permissions:

- apiGroups:
    - ""

  resources:
    - configmaps

  verbs:
    - get
    - list
    - watch
Enter fullscreen mode Exit fullscreen mode

Now the operator could still read and watch ConfigMaps, but it could no longer create one.

I verified this using:

kubectl auth can-i \
  create configmaps \
  --namespace default \
  --as=system:serviceaccount:platform-system:greeting-operator
Enter fullscreen mode Exit fullscreen mode

The expected result was:

no
Enter fullscreen mode Exit fullscreen mode

Creating a New Greeting Directly

Now I wanted the operator to attempt creation of a brand new ConfigMap.

Instead of modifying my existing sample YAML, I created a temporary Greeting directly from the terminal:

kubectl apply -f - <<'EOF'
apiVersion: platform.shubforge.dev/v1alpha1
kind: Greeting

metadata:
  name: failure-test

spec:
  message: "This should fail temporarily"
EOF
Enter fullscreen mode Exit fullscreen mode

Here:

kubectl apply -f -
Enter fullscreen mode Exit fullscreen mode

means that kubectl reads the manifest from standard input instead of a file.

This is useful for small temporary experiments because I do not need to add another YAML file to the repository.

I checked the new resource:

kubectl get greeting failure-test
Enter fullscreen mode Exit fullscreen mode

and then its full YAML:

kubectl get greeting failure-test -o yaml
Enter fullscreen mode Exit fullscreen mode

The operator tried to create:

failure-test-greeting
Enter fullscreen mode Exit fullscreen mode

but Kubernetes rejected the request because the ServiceAccount no longer had permission to create ConfigMaps.

The failure was a real:

403 Forbidden
Enter fullscreen mode Exit fullscreen mode

The Greeting Moved to Ready=False

The resource status eventually showed:

status:

  conditions:
    - type: Ready
      status: "False"
      reason: ReconciliationFailed
Enter fullscreen mode Exit fullscreen mode

So the flow became:

Greeting
    |
    v
Operator
    |
    v
create ConfigMap
    |
    v
403 Forbidden
    |
    v
Ready=False
Enter fullscreen mode Exit fullscreen mode

I could also inspect the Kubernetes Events:

kubectl describe greeting failure-test
Enter fullscreen mode Exit fullscreen mode

and see something like:

Events:
  Type     Reason                  Message
  ----     ------                  -------
  Warning  ReconciliationFailed   ...
Enter fullscreen mode Exit fullscreen mode

The Operator Was Retrying

One thing I initially misunderstood was what happened after the failure.

The Events output showed:

Warning  ReconciliationFailed  5m12s (x6 over 5m31s)
Enter fullscreen mode Exit fullscreen mode

The important part was:

x6
Enter fullscreen mode Exit fullscreen mode

That meant the operator had not failed only once.

It had retried several times.

The actual flow was closer to:

Attempt 1
   |
   v
403 Forbidden
   |
   v
retry

Attempt 2
   |
   v
403 Forbidden
   |
   v
retry

Attempt 3
   |
   v
...

retry limit reached
Enter fullscreen mode Exit fullscreen mode

So automatic retry was already working.

The operator simply does not retry forever.


Retry vs Reconciliation Trigger

This experiment helped me understand an important difference.

A retry means:

The previous reconciliation failed,
so try it again.
Enter fullscreen mode Exit fullscreen mode

A reconciliation trigger means:

Something relevant changed,
so evaluate the resource again.
Enter fullscreen mode Exit fullscreen mode

These are not the same thing.

The automatic retries eventually stopped after the retry limit was reached.

At that point, the Greeting remained:

Ready=False
Enter fullscreen mode Exit fullscreen mode

until another relevant event happened.


Restoring the Permissions

After observing the failure and retries, I restored the correct RBAC permissions.

Instead of manually editing the ClusterRole back again, I reapplied the operator manifests from the repository:

task operator:deploy
Enter fullscreen mode Exit fullscreen mode

This restored the ConfigMap permissions defined in:

k8s/operator/rbac.yaml
Enter fullscreen mode Exit fullscreen mode

I verified that the operator could create ConfigMaps again:

kubectl auth can-i \
  create configmaps \
  --namespace default \
  --as=system:serviceaccount:platform-system:greeting-operator
Enter fullscreen mode Exit fullscreen mode

This time the result was:

yes
Enter fullscreen mode Exit fullscreen mode

But something interesting happened.

The ConfigMap was still not created immediately.


Why Fixing RBAC Did Not Trigger Reconciliation

The retries had already been exhausted.

Changing:

ClusterRole
Enter fullscreen mode Exit fullscreen mode

does not produce an event for the Greeting controller.

The controller is mainly watching resources relevant to its reconciliation flow, such as:

Greeting
ConfigMap
Enter fullscreen mode Exit fullscreen mode

It is not watching:

ClusterRole
Enter fullscreen mode Exit fullscreen mode

So this happened:

ConfigMap creation fails
        |
        v
retries exhausted
        |
        v
RBAC fixed
        |
        v
ClusterRole changed
        |
        v
no Greeting event
        |
        v
no immediate reconciliation
Enter fullscreen mode Exit fullscreen mode

The permission problem was fixed, but nothing told the controller to evaluate failure-test again.


Triggering Another Reconciliation

For this experiment, I changed the Greeting.spec.

I ran:

kubectl patch greeting failure-test \
  --type=merge \
  -p '{"spec":{"message":"Recovery test"}}'
Enter fullscreen mode Exit fullscreen mode

This changed the desired resource.

Kubernetes increased the resource generation, which produced a new event for the controller.

Now the flow was:

RBAC fixed
    |
    v
Greeting spec changed
    |
    v
generation increased
    |
    v
new reconciliation
    |
    v
ConfigMap creation retried
    |
    v
success
Enter fullscreen mode Exit fullscreen mode

This time the operator had permission to create the ConfigMap.


Checking the Recovery

I verified the ConfigMap:

kubectl get configmap failure-test-greeting
Enter fullscreen mode Exit fullscreen mode

and then inspected the Greeting:

kubectl get greeting failure-test -o yaml
Enter fullscreen mode Exit fullscreen mode

The status had moved back to:

conditions:
  - type: Ready
    status: "True"
    reason: ConfigMapReady
Enter fullscreen mode Exit fullscreen mode

The Events now showed both parts of the story:

kubectl describe greeting failure-test
Enter fullscreen mode Exit fullscreen mode

with output similar to:

Warning  ReconciliationFailed  5m12s (x6 over 5m31s)
Normal   Reconciled            8s     Managed ConfigMap failure-test-greeting is in the desired state
Enter fullscreen mode Exit fullscreen mode

That was probably the most useful part of this experiment.

The event history showed:

failure
    |
    v
automatic retries
    |
    v
retry limit reached
    |
    v
external problem fixed
    |
    v
new reconciliation
    |
    v
recovery
Enter fullscreen mode Exit fullscreen mode

Failure and Recovery Lifecycle

The complete experiment looked like this:

Greeting created
      |
      v
ConfigMap creation fails
      |
      v
Ready=False
      |
      v
Warning Event
      |
      v
automatic retries
      |
      v
retry limit reached
      |
      v
RBAC fixed
      |
      v
no reconciliation yet
      |
      v
Greeting spec updated
      |
      v
new reconciliation
      |
      v
ConfigMap created
      |
      v
Ready=True
      |
      v
Normal Event
Enter fullscreen mode Exit fullscreen mode

This gave me a much clearer picture of how operator failure handling works.


Status, Events and Logs

At this point, the operator exposes information in three different places.

Logs

Useful for debugging the operator itself.

stack traces
internal errors
implementation details
Enter fullscreen mode Exit fullscreen mode

Status

Useful for understanding the current state of the custom resource.

type: Ready
status: "False"
reason: ReconciliationFailed
Enter fullscreen mode Exit fullscreen mode

Events

Useful for understanding important things that happened to the resource.

Warning  ReconciliationFailed
Normal   Reconciled
Enter fullscreen mode Exit fullscreen mode

So I now think about them like this:

Logs
    |
    v
developer debugging

Status
    |
    v
current resource state

Events
    |
    v
important resource history
Enter fullscreen mode Exit fullscreen mode

Each solves a different problem.


Successful Flow

The normal path is now:

Greeting
    |
    v
Operator
    |
    v
ConfigMap
    |
    +----> Ready=True
    |
    +----> Normal Reconciled Event
Enter fullscreen mode Exit fullscreen mode

Example:

status:

  configMapName: hello-greeting

  observedGeneration: 1

  conditions:
    - type: Ready
      status: "True"
      reason: ConfigMapReady
      message: Managed ConfigMap hello-greeting is in the desired state
Enter fullscreen mode Exit fullscreen mode

Failure Flow

The failed path looks like:

Greeting
    |
    v
Operator
    |
    v
Failure
    |
    +----> Ready=False
    |
    +----> Warning Event
    |
    +----> Retry
Enter fullscreen mode Exit fullscreen mode

If retries keep failing:

failure
    |
    v
retry
    |
    v
failure
    |
    v
retry
    |
    v
retry limit reached
Enter fullscreen mode Exit fullscreen mode

The resource then waits for another reconciliation trigger.


Taskfile Commands

I added a command for checking Greeting Events:

task greeting:events
Enter fullscreen mode Exit fullscreen mode

which is basically:

kubectl describe greeting hello
Enter fullscreen mode Exit fullscreen mode

The useful Greeting commands now include:

task greeting:create
task greeting:get
task greeting:status
task greeting:events
Enter fullscreen mode Exit fullscreen mode

For the operator:

task operator:image:build
task operator:image:load
task operator:deploy
task operator:restart
task operator:status
task operator:logs
Enter fullscreen mode Exit fullscreen mode

The Taskfile continues to be the main developer interface for the project.


Current Architecture

The current operator flow now looks like:

                     Greeting
                         |
                         v
                  Greeting Operator
                         |
                  +------+------+
                  |             |
               success        failure
                  |             |
                  v             v
              ConfigMap     Ready=False
                  |             |
                  v             v
              Ready=True    Warning Event
                  |             |
                  v             v
            Normal Event       Retry
Enter fullscreen mode Exit fullscreen mode

The operator is now handling more than resource creation.

It is also exposing:

state
failures
retry behavior
events
recovery
Enter fullscreen mode Exit fullscreen mode

through Kubernetes-native mechanisms.


Source Code

The complete implementation is available in my Platform Lab repository.

Repository: Platform Lab

The changes covered in this post are available in:

Pull Request: Add Failure Handling and Kubernetes Events

The project currently includes:

  • Greeting CRD
  • Java Operator SDK controller
  • ConfigMap dependent resource
  • status and conditions
  • Docker packaging
  • Kubernetes deployment
  • ServiceAccount and RBAC
  • failure handling
  • automatic retries
  • Kubernetes Events

What I Learned

The biggest thing I learned from this step was that:

retry
Enter fullscreen mode Exit fullscreen mode

and:

reconciliation trigger
Enter fullscreen mode Exit fullscreen mode

are different things.

The operator automatically retried the failed reconciliation.

But those retries were limited.

Once they were exhausted, fixing an unrelated external dependency like RBAC did not automatically create a new Greeting event.

A new relevant event, such as changing the resource specification, caused reconciliation to run again.

That gave me a much clearer picture of how event-driven controllers actually behave.


What's Next?

There is still more to understand around reconciliation itself.

Some questions I want to explore separately are:

Can reconciliation be triggered manually?
Should there be periodic reconciliation?
When should we reschedule?
How should external dependency changes be handled?
What are good reconciliation practices?
Enter fullscreen mode Exit fullscreen mode

I want to keep that as a separate topic.

After that, I also want to explore:

owner references
resource deletion
finalizers
integration tests
GitHub Actions
Enter fullscreen mode Exit fullscreen mode

For now, the Greeting Operator can report both success and failure using Kubernetes-native mechanisms and automatically retry transient failures.

Top comments (0)