[k8s] Basic Resource Management

kubernetes provides 3 objects to manage the resource in container, pod and namespace level

Resource management in container by pod spec

In pod spec, we can add the fields "resource:requests" and "resource:limits" for CPU and memory in container level.

CPU time unit, m means Milli, 500m = 0.5 CPU time
memory unit, Mi means MB

apiVersion: v1                               
kind: Pod                                    
metadata:                                    
  creationTimestamp: null                    
  name: pod-a                                
spec:                                        
  containers:                                
  - image: busybox                           
    command: ["/bin/sh", "-ec","sleep 1000"] 
    name: main                               
    resources:                               
      limits:                                
        cpu: 500m                            
        memory: 50Mi                         
      requests:                              
        cpu: 200m                            
        memory: 20Mi                         

different behavior between requests and limits

Requests guarantee the assigned resources, which are never less than the setting values. By contrast, the limits allow the resource usage more than the setting values, but when the total resource usage over the capability of node, Kubernetes will do something based on the limits. By CPU usage, the limits will work to limit the CPU time shared by different containers. By memory usage, the limits will affect which pod to be OOMKilled(out of memory).

LimitRange: for each pod or container in the same namespace

LimitRange is a namespace-scope object to set limit and requests range for pod, container, and persistentVolumeClaim.

apiVersion: v1
kind: LimitRange
metadata:
  name: limit-range
spec:
  limits:
  - type: Container  
    max:
      cpu: "800m"
      memory: "1Gi"
    min:
      cpu: "100m"
      memory: "50Mi"
    default:
      cpu: "600m"
      memory: "90Mi"
    defaultRequest:
      cpu: "110m"
      memory: "60Mi"

  - type: Pod  
    max:
      cpu: "2"
      memory: "2Gi"
    min:
      cpu: "500m"
      memory: "2Gi"

  - type: PersistentVolumeClaim
    max:
      storage: 2Gi
    min:
      storage: 1Gi

With container type, LimitRange set value range for container resource spec, and apply default values to pods which don't assign any resource requests and limits in spec. The default and defaultRequest make the management of container easier, we don't have to set spec for each containers, only focus on specific containers, which need separate settings.

With pod type, the value range means the total resource usage limits and requests of containers in the same pod.

With pvc type, it provides the range of storage we can set.

Resource Quota: total resource usage of pods in the same namespace.

Administrator can use this object to plan resource configurations for different purpose namespace, for example, production namespace needs more resource, and test namespace needs less. ResouceQuota can not only apply to resouces like CPU, memory, and storage, which you have already seen in LimitRange, but also provide the capability to limit storage class and the number of different objects, like services/pod/job.

apiVersion: v1                                                                           
kind: ResourceQuota                                                                      
metadata:                                                                                
        name: resource-quota                                                             
spec:                                                                                    
      hard:                                                                              
       # requests of cpu and memory                                               
        requests.cpu: 300m                                                         
        requests.memory: 100Mi                                                     

        # limits of cpu and memory                                                 
        limits.cpu: 1                                                              
        limits.memory: 1Gi                                                         

        # storage and total number of pvc                                                                  
        requests.storage: 300Gi        
        persistentvolumeclaims: 3                             

        # storage class                                                            
        dev-storage-class.storageclass.storage.k8s.io/requests.storage: 500Gi  
        dev-storage-class.storageclass.storage.k8s.io/persistentvolumeclaims: 3

        # Object Count
        pods: 2
        configmaps: 10
        services: 1                                                                

If you change your current LimitRange and ResourceQuota, the change won't apply to running/existing objects, only affect when you create new object which consumers more resource than limit and quota. By the way, ResouceQuota need each pod has request and limit, we can use LimitRange to apply default value rather than writing value in different pods.

The following diagram shows scopes of these 3 objects