2.5. Beyond Deployments
In this lab, you run a Pod on every node with a DaemonSet, give Pods a stable identity with a StatefulSet and run tasks once with a Job or on a schedule with a CronJob, which you then pause.
Step 1: Run a Pod on Every Node
Create a directory for the files of this lab and change into it:
Create the file node-exporter.yaml in ~/labs/2.5. node-exporter is a monitoring agent that collects
the CPU, memory and disk metrics of its node:
Create the DaemonSet:
List the DaemonSets:
If READY isn’t 3 yet, the Pods are still starting. Wait a moment and repeat the command.
Show on which nodes its Pods run. -o wide adds the column NODE:
Explanation
A DaemonSet runs exactly one Pod on every node, so its manifest has no replicas. DESIRED 3
is the number of nodes in your cluster. Agents that every node needs (for monitoring, logging or networking) run
as DaemonSets.
Step 2: Give Pods a Stable Identity
Create the file redis.yaml with a StatefulSet that runs three instances of the database Redis:
Create the StatefulSet:
Watch its Pods right away:
Stop the watch with Ctrl+C and delete the Pod redis-1:
List the Pods again:
Explanation
The manifest looks like a Deployment: only the kind differs. But a StatefulSet numbers its Pods
instead of giving them random names. It starts them one after another: redis-1 only starts once redis-0 runs. A
replaced Pod gets the same name back, here a new redis-1.
Databases and other clustered applications rely on this stable identity, e.g. a replicated database can always make
redis-0 its primary. The StatefulSet only provides the identity: the three Redis instances don’t replicate any data
between each other by themselves.
Step 3: Run a Task Once
Create the file backup.yaml with a Job that simulates a backup:
Create the Job:
Wait about ten seconds and list the Jobs:
If STATUS isn’t Complete yet, wait a moment and repeat the command.
Show the output of the Job:
Explanation
A Job runs a Pod until its task has ended successfully. COMPLETIONS 1/1 means that the task ran
once and succeeded. A Job only accepts restartPolicy: Never or OnFailure. With Always, the task would never
end. Unlike a Deployment, a Job needs no selector: it labels its Pods itself, with job-name.
The finished Pod stays with the status Completed, so you can still read its output. job/backup tells
kubectl logs to find the Pod of the Job for you.
Step 4: Run a Task on a Schedule
Create the file backup-cron.yaml with a CronJob that runs the backup every minute:
Create the CronJob:
List the CronJobs:
Wait two minutes and list the Jobs:
If the newest Job isn’t Complete yet, wait a moment and repeat the command.
Show the output of the newest Job. Use a name from your output:
Explanation
A CronJob creates a new Job on every scheduled time. jobTemplate describes that Job in the same
way as the Job in Step 3. The number in the name of each Job is its scheduled time, in minutes since 1970.
The schedule uses the cron format with five fields: minute, hour, day of month, month and day of week. * * * * *
means every minute, 0 2 * * * would mean every night at 2:00. Without the field timeZone, the times are in the
time zone of the control plane (UTC in your cluster, like the time in the output of the Job).
The CronJob keeps the last three successful Jobs and deletes older ones.
Step 5: Pause the Schedule
The CronJob creates a new Job every minute. Pause it with kubectl patch, which changes single fields of a resource
in the cluster:
List the CronJobs:
Apply the manifest again:
Explanation
-p passes the fields to change as JSON, here spec.suspend. kubectl patch merges them into the CronJob and
leaves all other fields as they are. A suspended CronJob creates no new Jobs.
Like kubectl edit in 2.4, kubectl patch is imperative: it changes the cluster, but not your manifest. Your
manifest has no field suspend, and kubectl apply only sets the fields that the manifest contains. That’s why it
reported unchanged, and the CronJob stays paused, although nobody can tell from the manifest. Use kubectl patch
only for a quick fix or a test, e.g. to pause a backup during maintenance. If the change should stay, write
suspend: true into the manifest.
Optional Tasks
Finished early? Try these tasks. The following labs don’t depend on them. Each task lists the commands you need. Open the solution if you get stuck.
Task 1: New Node, New Pod
Add a fourth node to the cluster. What does the DaemonSet do? Remove the node again afterwards.
| Command | Effect |
|---|---|
k3d node create <name> --cluster labs --role agent | Adds an agent node. Its container is named k3d-<name>-0 |
k3d node delete <container> | Removes the container of a node |
kubectl delete node <name> | Removes the Node from Kubernetes |
Task 2: Last In, First Out
Scale the StatefulSet down to one replica and back up to three. In which order do the Pods stop and start?
| Command | Effect |
|---|---|
kubectl scale statefulset <name> --replicas=<number> | Changes the number of replicas |
Task 3: Three Strikes
Create a Job whose task always fails, and limit the number of retries with backoffLimit: 2 in its spec. How
often does Kubernetes try, and what is the result?
You know all the commands you need. A container fails when its command ends with an exit code other than 0, e.g.
exit 1.
Clean Up
Delete all resources of this lab, since the following labs don’t need them:
Kubernetes deletes their Pods and the Jobs of the CronJob along with them. If you did Task 3, the Job
three-strikes is deleted as well.