Worker Node Maintenance

GSK takes care of Kubernetes upgrades and OS maintenance of your worker nodes. In some situations, for example when a security vulnerability is published before a fix is rolled out to GSK, you may need to access a worker node directly, upgrade its packages or apply a temporary mitigation yourself.

This page describes how to do that safely.

Root access to a worker node

Worker nodes do not expose SSH. To get a root shell on a node, use kubectl debug with the sysadmin profile. This starts a privileged pod on the node that mounts the node’s root filesystem under /host.

First, list your nodes to find the name of the node you want to access:

$ kubectl get nodes

Then start a debug container on the node:

$ kubectl debug node/<node-name> -it --image=ubuntu:24.04 --profile=sysadmin

# once inside the container, run:
chroot /host

After chroot /host you are in the node’s own filesystem as root and can use the node’s tools, for example apt, systemctl and journalctl.

To leave the node, type exit twice: once to leave the chroot and once to leave the debug container.

kubectl debug leaves the debug pod behind after you exit. Delete it once you are done:

$ kubectl get pods -A -o wide | grep node-debugger
$ kubectl delete pod -n <namespace> <node-debugger-pod-name>

Note: The debug pod runs with full privileges on the node. Changes you make to the node are your responsibility and are not covered by GSK OS maintenance. Worker nodes are replaced during cluster upgrades, so any manual changes to a node are lost when the node is replaced.

Upgrading OS packages and rebooting a worker node

Use this procedure to install pending Ubuntu package updates on a worker node and reboot it. Perform the procedure on one node at a time so that your workloads keep running on the remaining nodes.

1. Cordon and drain the node

Mark the node as unschedulable and move its workloads to other nodes:

$ kubectl cordon <node-name>
$ kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Wait for the drain command to finish. If a PodDisruptionBudget blocks the eviction of a pod, drain waits until the eviction is allowed. Make sure your cluster has enough free capacity on the other nodes to host the evicted pods.

2. Get root access to the node

Follow Root access to a worker node to open a root shell on the node. kubectl debug binds the pod directly to the node, so it works on a cordoned node.

3. Upgrade the packages

Inside the chroot, update the package lists and install the upgrades:

apt update
apt upgrade -y

4. Reboot the node

Still inside the chroot, reboot the node:

reboot

Your shell is disconnected when the node goes down. The reboot takes around 5 minutes. Watch the node status until it reports Ready again:

$ kubectl get node <node-name> -w

5. Uncordon the node

Once the node is Ready, allow pods to be scheduled on it again:

$ kubectl uncordon <node-name>

Finally, delete the leftover debug pod as described in Root access to a worker node and repeat the procedure for the next node.

CVE-2026-98432

CVE-2026-98432 affects the Ubuntu operating system of GSK worker nodes. A fix for GSK is being prepared and will be rolled out to all clusters as part of regular OS maintenance. Until then, no GSK-side mitigation is available. We will inform you as soon as GSK patches for CVE-2026-98432 are available.

If you need to protect your worker nodes before the GSK fix is available, follow the Ubuntu security advisory at https://ubuntu.com/security/CVE-2026-98432 and apply the mitigation it describes temporarily on each worker node:

  1. Cordon and drain the node as described in Upgrading OS packages and rebooting a worker node.
  2. Get root access to the node.
  3. Apply the mitigation from the Ubuntu advisory. If the advisory provides a fixed package, run apt update and apt upgrade -y and reboot the node.
  4. Uncordon the node and delete the debug pod.

Keep in mind that manual changes are lost when a node is replaced, for example during a cluster upgrade or when a node pool is scaled. Re-apply the mitigation on new nodes until the GSK fix has been rolled out. We will notify you when the patched GSK version is available; the fix will also be listed in the Release Notes.