Scaling applications

You can change the number of application replicas manually or configure an autoscaler to adjust it based on metrics such as CPU usage.

Doing it manually means that you will have to identify yourself when you need to scale and perform the scaling operation yourself.

To start with, we are going to cover the manual approach. There are two ways of scaling a deployment manually. One approach would be to edit the deployment manually by running:

On a real cluster, kubectl edit deployment nginx-deployment opens an editor where you can change replicas, for example from 1 to 3. In the browser, use the scale command below; interactive kubectl edit is not supported.

To set the replica count from the command line, run:

kubectl scale deployment nginx-deployment --replicas=3

If you have done either of these steps, you can now verify that your application has more pods to handle incoming traffic to them by running:

kubectl get pods
nginx-deployment-557f78c779-bts9t   1/1     Running   0          6m42s
nginx-deployment-557f78c779-s9hqs   1/1     Running   0          5m37s
nginx-deployment-557f78c779-sprdg   1/1     Running   0          5m37s

A Horizontal Pod Autoscaler (HPA) adjusts the replica count using observed metrics and a configured target. For example, it can add replicas when average CPU utilization exceeds a threshold. New Pods still need time and cluster capacity to start, so test how the application handles a sudden increase in traffic.

There are also two ways of creating a Horizontal Pod Autoscaler for your applications. One approach would be to invoke a command to install the resource on a target deployment. The command for this would be:

kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10

By running this command you have created a HorizontalPodAutoscaler that maintains between 1 and 10 replicas of the Pods controlled by the nginx-deployment Deployment. Notice that we configured the --cpu-percent heuristic as 50. This means that the HorizontalPodAutoScaler will aim to maintain a 50% CPU utilization on average across all pods in the deployment.

If you ran the autoscale command listed above you should be able to query the HPA status by running:

kubectl get hpa
NAME               REFERENCE                     TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
nginx-deployment   Deployment/nginx-deployment   <unknown>/50%   1         10        3          3m47s

But wait, why doesn't Kubernetes know how much CPU our deployment is using? To debug this we should run:

kubectl describe hpa
Name:                                                  nginx-deployment
Namespace:                                             default
Reference:                                             Deployment/nginx-deployment
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  <unknown> / 50%
Min replicas:                                          1
Max replicas:                                          10
Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: failed to get cpu utilization: missing request for cpu in container nginx

Notice this part:

ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count:
failed to get cpu utilization: missing request for cpu in container nginx

This is telling us that we need to add a request for CPU as a part of the resource management for the deployment.

Pod resource management

Resource requests and limits are Kubernetes' way of allowing you to specify the computing needs of your pods. If a resource request is set, the kube-scheduler uses this information to make sure that this pod will be run on a node with enough of that resource available. Limits are as the name might indicate a way for us to specify a limit of that resource that will be available to our pod.

This is a good safeguard against pods going rogue and eating up all of our resources. When a pod exceeds its memory limit, Kubernetes terminates the container (OOM kill). When a pod exceeds its CPU limit, the container is throttled rather than killed.

Let's revisit our deployment manifest and add a resource specification to it:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app.kubernetes.io/name: my-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: my-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: my-nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.26.2
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 80

The sandbox has this updated manifest stored as nginx-deployment.yaml. Apply it:

kubectl apply -f ./nginx-deployment.yaml

Now try getting the status of the HorizontalPodAutoscaler. The browser models a fixed idle CPU utilization of 10% and scales toward the minimum after CPU requests exist. It does not measure real CPU or model the real controller's stabilization windows. Values on a local cluster will differ:

kubectl get hpa
NAME               REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx-deployment   Deployment/nginx-deployment   10%/50%   1         10        1          46h

A very interesting thing to notice is that before we fixed the Autoscaler the replica count was 3 pods. After we fixed it the HPA realized that we were underutilizing the requested resources for pods so the HPA scaled it down for us to only one replica. If the CPU resource load were to exceed 50% it would bring up another pod so the average utilization of the target resource would fall back under the specified criterion.