In the previous seven articles we've thoroughly explored the Informer framework — Reflector, DeltaFIFO, Indexer, WorkQueue, EventBroadcaster, and the internal Controller. Now we shift from framework internals to application development: building your own Kubernetes extensions with CRD and Operator.
This article covers CRD — the foundation. The next article will build the Operator (controller) on top of it.
1. The Kubernetes Declarative Model
Before defining a CRD, it's worth understanding why CRDs exist.
Kubernetes is built on a declarative model:
┌─────────────────────────────────────────────────────────────┐
│ Kubernetes Declarative Model │
│ │
│ User declares desired state: │
│ "I want 3 replicas of nginx running" │
│ │ │
│ ▼ │
│ API Server stores it in etcd │
│ │ │
│ ▼ │
│ Controller Manager watches for drift: │
│ actual state ≠ desired state → take action │
│ │ │
│ ▼ │
│ Actual state converges to desired state │
│ (user never specifies HOW — only WHAT) │
└─────────────────────────────────────────────────────────────┘
Built-in resources (Deployment, StatefulSet, DaemonSet, Service, Ingress, ConfigMap…) cover most use cases. But when they don't, Kubernetes provides two extension mechanisms:
| Mechanism | What it provides |
|---|---|
| CRD (Custom Resource Definition) | A new resource type — defines the schema |
| Operator (Custom Controller) | The reconciliation logic — watches the CRD and acts on it |
CRD = the "what". Operator = the "how".
2. Anatomy of a CRD YAML
Here's a complete CRD definition for a canary deployment resource:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
# Format: {plural}.{group}
name: canaries.canarycontroller.tech.com
annotations:
"api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/78458"
spec:
# API group — appears in REST path: /apis/{group}/{version}
group: canarycontroller.tech.com
versions:
- name: v1alpha1
served: true # this version is active and served by the API Server
storage: true # this version is used for storage in etcd (only one allowed)
# Namespaced: resource belongs to a specific namespace
# Cluster: resource is cluster-wide (like Node, PersistentVolume)
scope: Namespaced
names:
plural: canaries # kubectl get canaries
singular: canary # kubectl get canary
kind: Canary # used in YAML: kind: Canary
shortNames:
- can # kubectl get can
subresources:
status: {} # enables /status subresource (separate update endpoint)
Key fields explained
┌──────────────────────────────────────────────────────────────────┐
│ metadata.name = "canaries.canarycontroller.tech.com" │
│ ──────── ────────────────────── │
│ plural group │
│ name name │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ REST API path after registration: │
│ /apis/canarycontroller.tech.com/v1alpha1/namespaces/{ns}/canaries│
│ ──────────────────────── ──────── │
│ group version │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ subresources.status: {} │
│ → enables separate /status endpoint │
│ → controller updates status via PATCH /status │
│ → user updates spec via PATCH /spec │
│ → prevents spec/status update conflicts │
└──────────────────────────────────────────────────────────────────┘
Register the CRD with the cluster
# Apply the CRD definition
kubectl create -f crd-canary.yaml
# Verify it's registered
kubectl get crd canaries.canarycontroller.tech.com
# The new resource type is now available
kubectl get canaries --all-namespaces
kubectl get can -n tech-prod
3. A Custom Resource Instance
Once the CRD is registered, you can create instances of it — just like built-in resources:
apiVersion: canarycontroller.tech.com/v1alpha1
kind: Canary
metadata:
name: go-hello-canary
namespace: tech-prod
labels:
app: go-hello
tech.com/clusterId: prod
# Auto-populated by API Server:
# resourceVersion: "430698608"
# uid: c5bb13a2-b2be-11ea-8fef-e4434b7c7170
spec:
info:
type: CanaryDeploy
totalBatches: "2" # deploy in 2 batches
currentBatch: "2"
pauseType: First # pause after first batch — wait for human approval
newDeploymentYaml: | # full YAML of the new version deployment
...
oldDeploymentYaml: | # full YAML of the current version deployment
...
status:
info:
availableReplicas: "4"
batch2Status: Finished
What this describes:
go-hello application canary release strategy:
┌─────────────────────────────────────────────────────────┐
│ Total instances: 4 │
│ Release strategy: 2 batches │
│ Batch 1: deploy 2 new instances → PAUSE │
│ (wait for human approval via kubectl patch) │
│ Batch 2: deploy remaining 2 instances → DONE │
└─────────────────────────────────────────────────────────┘
4. Go Type Definitions for the CRD
To build a controller for this CRD, you need to define the corresponding Go types. These are what code-generator will use to generate the typed client, informer, and lister code.
Directory structure
pkg/
└── apis/
└── canarycontroller/
├── register.go ← group name constant
└── v1alpha1/
├── doc.go ← code-gen annotations
├── types.go ← CRD Go type definitions ← main file
└── register.go ← register types with scheme
register.go — group name
// pkg/apis/canarycontroller/register.go
package canarycontroller
const (
GroupName = "canarycontroller.tech.com"
)
doc.go — code generation annotations
// pkg/apis/canarycontroller/v1alpha1/doc.go
// +k8s:deepcopy-gen=package
// +groupName=canarycontroller.tech.com
package v1alpha1
types.go — the core type definitions
// pkg/apis/canarycontroller/v1alpha1/types.go
package v1alpha1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// Canary is the Schema for the canaries API
type Canary struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec CanarySpec `json:"spec,omitempty"`
Status CanaryStatus `json:"status,omitempty"`
}
// CanarySpec defines the desired state of a Canary release
type CanarySpec struct {
Info CanaryInfo `json:"info"`
}
type CanaryInfo struct {
// "CanaryDeploy" or "ABTest"
Type string `json:"type"`
// Total number of batches to roll out
TotalBatches string `json:"totalBatches"`
// Current batch being processed
CurrentBatch string `json:"currentBatch"`
// "First": pause after first batch and wait for approval
// "None": roll out all batches automatically
PauseType string `json:"pauseType"`
// Full YAML of the new version Deployment
NewDeploymentYaml string `json:"newDeploymentYaml"`
// Full YAML of the current version Deployment
OldDeploymentYaml string `json:"oldDeploymentYaml"`
}
// CanaryStatus defines the observed state (written by the controller)
type CanaryStatus struct {
Info CanaryStatusInfo `json:"info,omitempty"`
}
type CanaryStatusInfo struct {
AvailableReplicas string `json:"availableReplicas,omitempty"`
Batch1Status string `json:"batch1Status,omitempty"`
Batch2Status string `json:"batch2Status,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// CanaryList is required by the API machinery for List operations
type CanaryList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata"`
Items []Canary `json:"items"`
}
Code generation annotations explained
| Annotation | Effect |
|---|---|
// +genclient |
Generate a typed client (CanaryInterface) for this type |
// +k8s:deepcopy-gen:interfaces=... |
Generate DeepCopyObject() — required by runtime.Object
|
// +k8s:deepcopy-gen=package |
Generate DeepCopy() for all types in this package |
// +groupName=... |
Set the API group for generated code |
5. Spec vs Status: The Controller Contract
The separation of spec and status is a fundamental Kubernetes design pattern:
┌─────────────────────────────────────────────────────────────┐
│ spec — desired state │
│ Written by: the user (kubectl apply) │
│ Read by: the controller │
│ │
│ status — observed state │
│ Written by: the controller (via /status subresource) │
│ Read by: the user (kubectl get/describe) │
└─────────────────────────────────────────────────────────────┘
Controller reconciliation loop:
┌──────────────────────────────────────────────────────────────┐
│ │
│ Read spec → compare with status → take action │
│ │ │
│ ▼ │
│ update status │
│ (reflect new reality) │
└──────────────────────────────────────────────────────────────┘
Enabling subresources.status: {} in the CRD YAML is critical — it creates a separate /status API endpoint, which means:
- Users updating
specwon't accidentally overwritestatus - Controllers updating
statuswon't accidentally overwritespec - RBAC can grant different permissions for spec vs status updates
6. The CRD Lifecycle
Step 1: Define the CRD schema (YAML)
kubectl create -f crd-canary.yaml
→ API Server registers new resource type
→ REST endpoint /apis/canarycontroller.tech.com/v1alpha1/... becomes available
Step 2: Define Go types (types.go)
→ mirrors the CRD schema in Go structs
Step 3: Run code-generator
→ generates typed client (Clientset)
→ generates typed Informer + Lister
→ generates DeepCopy methods
Step 4: Create resource instances
kubectl apply -f canary-go-hello.yaml
→ Canary object stored in etcd
Step 5: Controller watches and reconciles
→ Informer watches Canary resources
→ Controller reads spec, takes action, updates status
→ Desired state converges to actual state
7. Summary
| Concept | Detail |
|---|---|
| CRD | Registers a new resource type with the API Server — no code required |
| Custom Resource | An instance of a CRD — stored in etcd, managed via kubectl |
| spec | Desired state — written by users, read by controllers |
| status | Observed state — written by controllers, read by users |
| subresources.status | Separates spec/status update paths — prevents conflicts |
| scope |
Namespaced or Cluster — determines resource visibility |
| group/version | Forms the REST API path: /apis/{group}/{version}/...
|
| code-generator | Generates typed client, Informer, Lister, and DeepCopy from Go types |
CRD is the "schema" half of the Operator pattern. It extends the Kubernetes API with your domain-specific resource types — giving you the same declarative, watch-driven, etcd-backed machinery that built-in resources enjoy. The controller (Operator) is what brings those resources to life.
Next in this series: Operator in Practice: Building a Canary Release Controller (Part 9 — Final)
Follow the series for more deep dives into Kubernetes development.
Top comments (0)