Multitenancy with Hierarchical namespaces

Hierarchical namespace is a tweak over vanilla kubernetes namespace where a namespace can optionally have a parent namespace implemented through a CRD called Hierarchical Namespace Controller(HNC). Main advantages of HNC are:

• helps establish ownership of namespaces as a group
• policy inheritance
• administer with lesser privileges (than a cluster admin)

Installation

❯ HNC_VERSION=v1.0.0
❯ kubectl apply -f https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/download/${HNC_VERSION}/default.yaml
namespace/hnc-system created
customresourcedefinition.apiextensions.k8s.io/hierarchyconfigurations.hnc.x-k8s.io created
customresourcedefinition.apiextensions.k8s.io/hncconfigurations.hnc.x-k8s.io created
customresourcedefinition.apiextensions.k8s.io/subnamespaceanchors.hnc.x-k8s.io created
role.rbac.authorization.k8s.io/hnc-leader-election-role created
clusterrole.rbac.authorization.k8s.io/hnc-admin-role created
clusterrole.rbac.authorization.k8s.io/hnc-manager-role created
clusterrole.rbac.authorization.k8s.io/hnc-proxy-role created
rolebinding.rbac.authorization.k8s.io/hnc-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/hnc-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/hnc-proxy-rolebinding created
secret/hnc-webhook-server-cert created
service/hnc-controller-manager-metrics-service created
service/hnc-webhook-service created
deployment.apps/hnc-controller-manager created
mutatingwebhookconfiguration.admissionregistration.k8s.io/hnc-mutating-webhook-configuration created
validatingwebhookconfiguration.admissionregistration.k8s.io/hnc-validating-webhook-configuration created

# Install helper plugin
❯ kubectl krew install hns

Usage

❯ kubectl create ns team-abc
namespace/team-abc created
❯ kubectl hns create team-alpha -n team-abc
Successfully created "team-alpha" subnamespace anchor in "team-abc" namespace
❯ kubectl hns create team-beta -n team-abc
Successfully created "team-beta" subnamespace anchor in "team-abc" namespace
❯ kubectl hns tree team-abc
team-abc
├── [s] team-alpha
└── [s] team-beta

[s] indicates subnamespaces

---

Policy inheritance

By default, HNC propagates RBAC Role and RoleBinding objects. If you create objects of these kinds in a parent namespace, it will automatically be copied into any descendant namespaces as well. You cannot modify these propagated copies; HNC’s admission controllers will attempt to stop you from editing them.

❯ kubectl hns config describe
Synchronized resources:
* Propagating: rolebindings (rbac.authorization.k8s.io/v1)
* Propagating: roles (rbac.authorization.k8s.io/v1)

Conditions:

Updating inheritance

Synchronization across namespace hierarchies is configurable in 3 modes:

1. Propagate: propagates objects from ancestors to descendants and deletes obsolete descendants.
2. Remove: deletes all existing propagated copies, but does not touch source objects.
3. Ignore: stops modifying this resource. New or changed objects will not be propagated, and obsolete objects will not be deleted. This is the default mode

Adding quota and limitrange propagation

❯ kubectl whoami # kubectl krew install whoami
kubernetes-admin
❯ kubectl hns config set-resource resourcequota --mode Propagate
❯ kubectl hns config set-resource limitrange --mode Propagate
❯ kubectl hns config describe
Synchronized resources:
* Propagating: limitrange (/v1)
* Propagating: resourcequota (/v1)
* Propagating: secrets (/v1)
* Propagating: rolebindings (rbac.authorization.k8s.io/v1)
* Propagating: roles (rbac.authorization.k8s.io/v1)

Conditions:

Check policy inheritance

❯ cat /tmp/cpu-quota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: cpu-quota
  namespace: team-abc
spec:
  hard:
    requests.cpu: "200m"
    limits.cpu: "1000m"
❯ kubectl apply -f -
resourcequota/cpu-quota configured
❯ kubectl get resourcequotas -n team-abc
NAME        AGE   REQUEST                LIMIT
cpu-quota   93s   requests.cpu: 0/200m   limits.cpu: 0/1
❯ k get resourcequotas -n team-abc
NAME        AGE   REQUEST                LIMIT
cpu-quota   28m   requests.cpu: 0/200m   limits.cpu: 0/1
❯ k get resourcequotas -n team-alpha
NAME        AGE     REQUEST                LIMIT
cpu-quota   6m18s   requests.cpu: 0/200m   limits.cpu: 0/1

Conclusion

✓ Installation is simple, there is no configuration per se
✓ Resource inheritance is intuitive in Propagate mode
Why is it not mainstream and the fact that it took so long to graduate to 1.0 makes one worry about what is happening

Cook book

https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/how-to.md