DEV Community

shubham goel
shubham goel

Posted on

Running My Java Kubernetes Operator Inside the Cluster

#kubernetes #devops #tutorial #java

In the previous post, I built a simple Kubernetes controller in Java.

The flow was:

Greeting
    |
    v
Java Controller
    |
    v
ConfigMap
Enter fullscreen mode Exit fullscreen mode

The controller was working, but there was one important limitation.

It was still running on my laptop.

My Laptop
    |
    | Java Process
    |
    v
Greeting Controller
    |
    | kubeconfig
    |
    v
Kind Kubernetes Cluster
Enter fullscreen mode Exit fullscreen mode

That was useful while developing the controller, but eventually an operator should run inside Kubernetes itself.

So the next step was to package the Java application into a container and deploy it into the cluster.

The new setup looks like:

Kind Kubernetes Cluster

Greeting
    |
    v
Greeting Operator Pod
    |
    v
ConfigMap
Enter fullscreen mode Exit fullscreen mode

This introduced a few new things for me:

  • building an executable JAR
  • creating a Docker image
  • loading the image into Kind
  • creating a ServiceAccount
  • understanding Kubernetes RBAC
  • using a ClusterRole
  • using a ClusterRoleBinding
  • deploying the operator as a Kubernetes Deployment

Building an Executable JAR

Previously, I was starting the controller locally using Maven:

task operator:run
Enter fullscreen mode Exit fullscreen mode

Internally, this was running something similar to:

./mvnw \
  -f operators/greeting-operator/pom.xml \
  compile exec:java
Enter fullscreen mode Exit fullscreen mode

For a container, I wanted something simpler:

java -jar greeting-operator.jar
Enter fullscreen mode Exit fullscreen mode

So I configured Maven to create an executable JAR containing the dependencies required by the operator.

Now:

task operator:build
Enter fullscreen mode Exit fullscreen mode

creates the application JAR under:

operators/greeting-operator/target/
Enter fullscreen mode Exit fullscreen mode

This gives me a Java application that can be started independently from Maven.

Creating the Docker Image

The operator Dockerfile is kept under:

operators/greeting-operator/Dockerfile
Enter fullscreen mode Exit fullscreen mode

The basic idea is simple:

FROM eclipse-temurin:26-jre

WORKDIR /app

COPY target/greeting-operator-0.1.0-SNAPSHOT.jar app.jar

USER 10001

ENTRYPOINT ["java", "-jar", "/app/app.jar"]
Enter fullscreen mode Exit fullscreen mode

The build flow is now:

Java Source
    |
    v
Maven Build
    |
    v
Executable JAR
    |
    v
Docker Build
    |
    v
greeting-operator:dev
Enter fullscreen mode Exit fullscreen mode

I wrapped this inside the Taskfile as well.

Build the image:

task operator:image:build
Enter fullscreen mode Exit fullscreen mode

Since I am using Kind, the image also needs to be made available to the Kind cluster.

task operator:image:load
Enter fullscreen mode Exit fullscreen mode

This loads the local image into the Kind nodes.

Deploying the Operator

I created a separate directory for the operator Kubernetes resources:

k8s/operator/
├── namespace.yaml
├── service-account.yaml
├── rbac.yaml
└── deployment.yaml
Enter fullscreen mode Exit fullscreen mode

The operator itself runs inside:

platform-system
Enter fullscreen mode Exit fullscreen mode

I created a separate namespace because I want platform components to be separate from application workloads.

apiVersion: v1
kind: Namespace

metadata:
  name: platform-system
Enter fullscreen mode Exit fullscreen mode

Why Does the Operator Need a ServiceAccount?

When I was running the operator on my laptop, the Java process could access Kubernetes using my local kubeconfig.

Conceptually:

Java Controller
      |
      v
~/.kube/config
      |
      v
Kubernetes API
Enter fullscreen mode Exit fullscreen mode

Once the application is running inside Kubernetes, I do not want to copy my personal kubeconfig into the container.

Instead, Kubernetes provides a workload identity using a ServiceAccount.

apiVersion: v1
kind: ServiceAccount

metadata:
  name: greeting-operator
  namespace: platform-system
Enter fullscreen mode Exit fullscreen mode

Now the operator Pod runs using:

ServiceAccount: greeting-operator
Enter fullscreen mode Exit fullscreen mode

The Deployment connects the Pod to that identity:

spec:
  template:
    spec:
      serviceAccountName: greeting-operator
Enter fullscreen mode Exit fullscreen mode

So now the identity chain looks like:

Greeting Operator Pod
        |
        v
ServiceAccount
greeting-operator
Enter fullscreen mode Exit fullscreen mode

But a ServiceAccount by itself does not automatically have permission to watch Greetings or create ConfigMaps.

That is where RBAC comes in.

Understanding Kubernetes RBAC

This part was initially a little confusing because Kubernetes has a few similarly named resources:

Role
ClusterRole
RoleBinding
ClusterRoleBinding
Enter fullscreen mode Exit fullscreen mode

The easiest way I found to understand them is to separate two questions.

Question 1

What operations are allowed?

That is defined by:

Role
or
ClusterRole
Enter fullscreen mode Exit fullscreen mode

Question 2

Who gets those permissions?

That is defined by:

RoleBinding
or
ClusterRoleBinding
Enter fullscreen mode Exit fullscreen mode

So:

Role / ClusterRole
        =
permissions

RoleBinding / ClusterRoleBinding
        =
attach those permissions to somebody
Enter fullscreen mode Exit fullscreen mode

The "somebody" could be:

User
Group
ServiceAccount
Enter fullscreen mode Exit fullscreen mode

In this project, it is the operator's ServiceAccount.

Role vs ClusterRole

A Role belongs to one namespace.

For example, a Role created in:

namespace: default
Enter fullscreen mode Exit fullscreen mode

can give permissions to resources in the default namespace.

Conceptually:

Role
namespace: default

can:
  get ConfigMaps
  create ConfigMaps
Enter fullscreen mode Exit fullscreen mode

But only inside:

default
Enter fullscreen mode Exit fullscreen mode

A ClusterRole, on the other hand, is not tied to one namespace.

For the Greeting operator, I created:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole

metadata:
  name: greeting-operator
Enter fullscreen mode Exit fullscreen mode

The rules include permissions for our custom resource:

- apiGroups:
    - platform.shubforge.dev
  resources:
    - greetings
  verbs:
    - get
    - list
    - watch
    - patch
    - update
Enter fullscreen mode Exit fullscreen mode

and permissions for ConfigMaps:

- apiGroups:
    - ""
  resources:
    - configmaps
  verbs:
    - get
    - list
    - watch
    - create
    - update
    - patch
    - delete
Enter fullscreen mode Exit fullscreen mode

This defines what the operator is allowed to do.

But defining a ClusterRole alone does not give anybody those permissions.

We still need to bind it.

What Does ClusterRoleBinding Do?

This is the important part.

The operator runs using this ServiceAccount:

platform-system/greeting-operator
Enter fullscreen mode Exit fullscreen mode

The ClusterRoleBinding connects that ServiceAccount to the ClusterRole.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding

metadata:
  name: greeting-operator

subjects:
  - kind: ServiceAccount
    name: greeting-operator
    namespace: platform-system

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: greeting-operator
Enter fullscreen mode Exit fullscreen mode

The relationship is:

Pod
 |
 v
ServiceAccount
platform-system/greeting-operator
 |
 v
ClusterRoleBinding
 |
 v
ClusterRole
 |
 +---- watch Greetings
 |
 +---- create ConfigMaps
 |
 +---- update ConfigMaps
 |
 +---- delete ConfigMaps
Enter fullscreen mode Exit fullscreen mode

The ClusterRoleBinding does not contain the actual permissions.

It only says:

Give this subject the permissions defined by this ClusterRole.

How Does It Get Access to All Namespaces?

This was one of the things I specifically wanted to understand.

The important combination is:

ClusterRole
      +
ClusterRoleBinding
Enter fullscreen mode Exit fullscreen mode

Our Greeting resource is namespaced.

For example, we could have:

default/hello
Enter fullscreen mode Exit fullscreen mode

and:

team-a/hello
Enter fullscreen mode Exit fullscreen mode

and:

team-b/hello
Enter fullscreen mode Exit fullscreen mode

The ClusterRole says the operator may work with:

greetings
configmaps
Enter fullscreen mode Exit fullscreen mode

And because that ClusterRole is attached using a ClusterRoleBinding, those permissions apply across namespaces.

Conceptually:

ClusterRoleBinding
        |
        v
ClusterRole
        |
        +--------------------------+
        |                          |
        v                          v
namespace: default          namespace: team-a
Greeting                    Greeting
ConfigMap                   ConfigMap
        |
        v
namespace: team-b
Greeting
ConfigMap
Enter fullscreen mode Exit fullscreen mode

The ServiceAccount still belongs to:

platform-system
Enter fullscreen mode Exit fullscreen mode

but that does not restrict where it can operate.

This is an important distinction.

The namespace of a ServiceAccount tells Kubernetes where the ServiceAccount object lives.

ServiceAccount
platform-system/greeting-operator
Enter fullscreen mode Exit fullscreen mode

It does not mean:

This ServiceAccount can only access resources inside platform-system.

Its actual permissions depend on the RBAC bindings attached to it.

ClusterRoleBinding vs RoleBinding

Here is another useful comparison.

Suppose I have the same ClusterRole:

ClusterRole: greeting-operator
Enter fullscreen mode Exit fullscreen mode

If I bind it using a ClusterRoleBinding:

ClusterRole
      |
      v
ClusterRoleBinding
Enter fullscreen mode Exit fullscreen mode

the permissions apply across namespaces.

But Kubernetes also allows a RoleBinding to reference a ClusterRole.

For example:

kind: RoleBinding

metadata:
  name: greeting-operator
  namespace: team-a
Enter fullscreen mode Exit fullscreen mode

with:

roleRef:
  kind: ClusterRole
  name: greeting-operator
Enter fullscreen mode Exit fullscreen mode

In that case, the same ClusterRole permissions are restricted to:

team-a
Enter fullscreen mode Exit fullscreen mode

So:

ClusterRole
   +
ClusterRoleBinding

= cluster-wide access
Enter fullscreen mode Exit fullscreen mode

while:

ClusterRole
   +
RoleBinding in team-a

= permissions only in team-a
Enter fullscreen mode Exit fullscreen mode

This was probably the easiest way for me to understand the difference.

A Simple RBAC Mental Model

I now think about it like this:

Role
    permissions inside one namespace

ClusterRole
    reusable permission definition
    that can work cluster-wide

RoleBinding
    give permissions inside one namespace

ClusterRoleBinding
    give permissions cluster-wide
Enter fullscreen mode Exit fullscreen mode

For our operator:

ServiceAccount
platform-system/greeting-operator

        |
        v

ClusterRoleBinding

        |
        v

ClusterRole
greeting-operator

        |
        +---- Greetings
        |
        +---- ConfigMaps
        |
        v

all namespaces
Enter fullscreen mode Exit fullscreen mode

One Important Detail: RBAC vs Watch Scope

RBAC controls what the operator is allowed to access.

It does not necessarily decide what the controller chooses to watch.

These are two separate ideas.

RBAC
=
What am I allowed to access?

Controller configuration
=
What do I actually watch?
Enter fullscreen mode Exit fullscreen mode

For example, the ServiceAccount may have cluster-wide permission, but the controller could still be configured to only watch:

team-a
Enter fullscreen mode Exit fullscreen mode

Similarly:

RBAC: all namespaces

Controller:
watch namespace = team-a
Enter fullscreen mode Exit fullscreen mode

would mean the operator has more permission than it actually uses.

Ideally, permissions and controller scope should stay as close as possible.

For this learning project, I want the Greeting operator to work across namespaces, so using a ClusterRole with a ClusterRoleBinding keeps the setup simple.

For a production platform, I would think more carefully about least privilege and whether cluster-wide access is really needed.

Deploying the Operator

The operator itself runs as a regular Kubernetes Deployment.

apiVersion: apps/v1
kind: Deployment

metadata:
  name: greeting-operator
  namespace: platform-system

spec:
  replicas: 1

  selector:
    matchLabels:
      app: greeting-operator

  template:

    metadata:
      labels:
        app: greeting-operator

    spec:

      serviceAccountName: greeting-operator

      containers:

        - name: greeting-operator

          image: greeting-operator:dev

          imagePullPolicy: IfNotPresent
Enter fullscreen mode Exit fullscreen mode

The important part from the RBAC perspective is:

serviceAccountName: greeting-operator
Enter fullscreen mode Exit fullscreen mode

That connects the Pod to the ServiceAccount.

The complete authorization flow is now:

Greeting Operator Pod
        |
        v
ServiceAccount
        |
        v
ClusterRoleBinding
        |
        v
ClusterRole
        |
        v
Kubernetes API
Enter fullscreen mode Exit fullscreen mode

Why Is There No Kubernetes Service?

I did not create a Kubernetes Service for the operator.

That is because the operator is not exposing an HTTP endpoint that other applications need to call.

The operator mainly communicates outward to the Kubernetes API.

Operator
   |
   v
Kubernetes API
Enter fullscreen mode Exit fullscreen mode

It watches resources and reacts to events.

There is currently no need for:

Service
Ingress
Gateway
Enter fullscreen mode Exit fullscreen mode

That may change later if the operator exposes metrics, health endpoints, admission webhooks, or some other network API.

But for this simple controller, a Deployment is enough.

Running Everything

The full flow now looks like this.

Create the cluster:

task cluster:create
Enter fullscreen mode Exit fullscreen mode

Install the CRD:

task crd:install
Enter fullscreen mode Exit fullscreen mode

Build the operator image:

task operator:image:build
Enter fullscreen mode Exit fullscreen mode

Load it into Kind:

task operator:image:load
Enter fullscreen mode Exit fullscreen mode

Deploy the operator:

task operator:deploy
Enter fullscreen mode Exit fullscreen mode

Check its status:

task operator:status
Enter fullscreen mode Exit fullscreen mode

The operator Pod should now be running inside:

platform-system
Enter fullscreen mode Exit fullscreen mode

For example:

kubectl get pods -n platform-system
Enter fullscreen mode Exit fullscreen mode

Now I can create the Greeting:

task greeting:create
Enter fullscreen mode Exit fullscreen mode

and check:

kubectl get configmaps
Enter fullscreen mode Exit fullscreen mode

The controller should create:

hello-greeting
Enter fullscreen mode Exit fullscreen mode

Testing the RBAC Setup

I can also verify the ServiceAccount permissions directly using kubectl auth can-i.

For example:

kubectl auth can-i \
  list greetings.platform.shubforge.dev \
  --as=system:serviceaccount:platform-system:greeting-operator \
  --all-namespaces
Enter fullscreen mode Exit fullscreen mode

Expected:

yes
Enter fullscreen mode Exit fullscreen mode

For ConfigMaps:

kubectl auth can-i \
  create configmaps \
  --as=system:serviceaccount:platform-system:greeting-operator \
  --all-namespaces
Enter fullscreen mode Exit fullscreen mode

Again:

yes
Enter fullscreen mode Exit fullscreen mode

This is a useful way to understand what RBAC is actually allowing instead of only reading the YAML.

I can also test one particular namespace:

kubectl auth can-i \
  create configmaps \
  --namespace default \
  --as=system:serviceaccount:platform-system:greeting-operator
Enter fullscreen mode Exit fullscreen mode

Testing Across Namespaces

Since the operator now has cluster-wide permissions, I can create another namespace:

kubectl create namespace demo
Enter fullscreen mode Exit fullscreen mode

Then create a Greeting inside it:

apiVersion: platform.shubforge.dev/v1alpha1
kind: Greeting

metadata:
  name: hello
  namespace: demo

spec:
  message: "Hello from demo namespace"
Enter fullscreen mode Exit fullscreen mode

After applying it, the controller should be able to create:

demo/hello-greeting
Enter fullscreen mode Exit fullscreen mode

So the operator can remain in:

platform-system
Enter fullscreen mode Exit fullscreen mode

while managing resources in:

default
demo
team-a
team-b
...
Enter fullscreen mode Exit fullscreen mode

as long as its RBAC permissions and controller watch configuration allow it.

The Architecture Now

The project has now moved from:

Laptop

Java Controller
      |
      v
Kind Cluster
Enter fullscreen mode Exit fullscreen mode

to:

Kubernetes Cluster

+------------------------------------+
|                                    |
| platform-system                    |
|                                    |
|  greeting-operator Pod             |
|        |                           |
|        v                           |
|  ServiceAccount                    |
|        |                           |
+--------|---------------------------+
         |
         v
  ClusterRoleBinding
         |
         v
    ClusterRole
         |
         v
+------------------------------------+
|                                    |
| Kubernetes API                     |
|                                    |
| Greeting ------> ConfigMap         |
|                                    |
+------------------------------------+
Enter fullscreen mode Exit fullscreen mode

The controller is now a Kubernetes workload itself.

That makes the setup feel much closer to a real operator.

Source Code

The complete implementation is available in my Platform Lab repository.

Repository: Platform Lab

The changes for this step are available in:

Pull Request: Deploy Greeting Operator to Kubernetes

The repository contains:

  • Java Greeting controller
  • Greeting CRD
  • Dockerfile
  • Kind setup
  • ServiceAccount
  • ClusterRole
  • ClusterRoleBinding
  • operator Deployment
  • Taskfile commands

What's Next?

The operator is now running inside Kubernetes and has the permissions required to manage resources.

The current flow is:

Greeting
    |
    v
Java Operator
    |
    v
ConfigMap
Enter fullscreen mode Exit fullscreen mode

and the operator itself is now running as:

Deployment
    |
    v
Pod
    |
    v
ServiceAccount
    |
    v
RBAC
Enter fullscreen mode Exit fullscreen mode

The next things I want to explore are what makes an operator more production-like:

  • status updates
  • conditions
  • reconciliation behavior
  • error handling
  • retry behavior
  • owner references
  • finalizers
  • automated operator tests
  • GitHub Actions

For now, moving the controller from my laptop into Kubernetes helped me understand an important part of operators:

The controller is just another application, but Kubernetes gives it an identity and permissions to interact with the Kubernetes API.

Top comments (0)