2.1. Kubernetes Architecture

In this lab, you create the cluster you use for all labs and find the components of the Kubernetes architecture in it.

Step 1: Create the Cluster

Create a directory for the files of this lab and change into it:

mkdir -p ~/labs/2.1
cd ~/labs/2.1

Create the file registries.yaml in ~/labs/2.1. It makes the nodes download images from Docker Hub through a mirror:

mirrors:
  docker.io:
    endpoint:
      - https://mirror.gcr.io
  registry-1.docker.io:
    endpoint:
      - https://mirror.gcr.io

Create a k3s cluster named labs with one server node and two agent nodes:

k3d cluster create labs \
  --agents 2 \
  --k3s-arg "--cluster-init@server:0" \
  --k3s-arg "--disable=traefik@server:*" \
  --k3s-arg "--disable=servicelb@server:*" \
  --port "30080-30089:30080-30089@server:0" \
  --port "31337:31337@server:0" \
  --registry-config registries.yaml

Creating the cluster takes about 25 seconds. The output ends with:

INFO[0024] Cluster 'labs' created successfully!
INFO[0024] You can now use it like this:
kubectl cluster-info

Check that kubectl now talks to the new cluster:

kubectl config current-context

The output is k3d-labs.

Explanation

k3d starts the nodes as Docker containers, installs k3s in them and connects them to one cluster. The flags define what the cluster looks like:

FlagEffect
labsName of the cluster. k3d prefixes containers and the kubectl context with k3d-
--agents 2Adds two agent nodes (worker nodes) to the server node
--cluster-initStores the cluster state in an embedded etcd
--disable=traefikStarts k3s without its built-in Ingress controller
--disable=servicelbStarts k3s without its built-in load balancer
--portMakes the ports 30080 to 30089 and 31337 reachable from your machine
--registry-configCopies registries.yaml into the nodes

--k3s-arg passes a flag on to k3s. The suffix after @ selects the nodes a flag applies to: server:0 is the first server node, server:* every server node. k3d also added the context k3d-labs to your kubeconfig (~/.kube/config) and switched kubectl to it.

Docker Hub mirror

Each node downloads the images of its containers itself. In the training room, all laptops share one internet address, and Docker Hub allows only 100 downloads per address in 6 hours. registries.yaml sends these downloads to mirror.gcr.io, Google’s copy of Docker Hub. The chart in 4.1 calls Docker Hub registry-1.docker.io, so the file lists both names.

Stop the cluster after the training day

k3d cluster stop labs stops all node containers and frees memory on your machine. k3d cluster start labs starts them again, and your resources are still there.

Port is already allocated

If k3d fails with port is already allocated, another program on your machine uses one of the ports 30080 to 30089 or 31337. Stop the program, delete the cluster with k3d cluster delete labs and create it again.


Step 2: Explore the Nodes

List the nodes of your cluster:

kubectl get nodes
NAME                STATUS   ROLES                AGE   VERSION
k3d-labs-agent-0    Ready    <none>               38s   v1.35.5+k3s1
k3d-labs-agent-1    Ready    <none>               48s   v1.35.5+k3s1
k3d-labs-server-0   Ready    control-plane,etcd   50s   v1.35.5+k3s1

If a node isn’t Ready yet, wait a moment and repeat the command.

Explanation

Every machine in a cluster is represented by a Node. Your cluster has three of them:

  • k3d-labs-server-0 carries the roles control-plane and etcd: it runs the control plane and stores the cluster state.
  • The two agent nodes have no role: they are worker nodes. In your cluster, the server node runs workloads as well.

Ready means the kubelet on the node is healthy and accepts new Pods. VERSION is the version of the kubelet.


Step 3: Find the Control Plane

In many Kubernetes clusters, each control plane component runs in a Pod in the Namespace kube-system. A namespace groups resources, and -n selects it. Check what runs there in your cluster:

kubectl get pods -n kube-system
NAME                                      READY   STATUS    RESTARTS   AGE
coredns-8db54c48d-nfxvd                   1/1     Running   0          45s
local-path-provisioner-5d9d9885bc-fr54m   1/1     Running   0          45s
metrics-server-786d997795-9w9q8           1/1     Running   0          45s

If a Pod isn’t Running with 1/1 yet, wait a moment and repeat the command. Right after the cluster is created, the add-ons need about half a minute to start.

None of the control plane components shows up: k3s runs them inside the k3s server process. Its log shows which components it starts:

docker logs k3d-labs-server-0 2>&1 | grep -o -E "Running (k[a-z-]*|containerd)|Starting etcd"
Running containerd
Starting etcd
Running kube-apiserver
Running kube-scheduler
Running kube-controller-manager
Running kube-proxy
Running kubelet

Compare it with an agent node:

docker logs k3d-labs-agent-0 2>&1 | grep -o -E "Running (k[a-z-]*|containerd)|Starting etcd"
Running containerd
Running kube-proxy
Running kubelet

Explanation

You found the components of the Kubernetes architecture in your cluster:

ComponentRuns on
kube-apiserverServer node
etcdServer node
kube-schedulerServer node
kube-controller-managerServer node
kubeletEvery node
kube-proxyEvery node
containerdEvery node

containerd is the container runtime: the kubelet tells it which containers to start.

k3s packs all of these components into a single binary. That is what makes it so lightweight.

The Pods in kube-system are add-ons that k3s installs out of the box:

Add-onPurpose
corednsDNS inside the cluster, resolves Service names
local-path-provisionerCreates persistent volumes in a directory on the node
metrics-serverCollects CPU and memory usage of Pods and nodes

Step 4: Find the Nodes in Docker

k3d runs every node as a Docker container. List the containers of your cluster:

docker ps --filter "name=k3d-labs" --format "table {{.Names}}\t{{.Image}}\t{{.Ports}}"
NAMES               IMAGE                            PORTS
k3d-labs-serverlb   ghcr.io/k3d-io/k3d-proxy:5.9.0   0.0.0.0:30080-30089->30080-30089/tcp, [::]:30080-30089->30080-30089/tcp, 80/tcp, 0.0.0.0:31337->31337/tcp, [::]:31337->31337/tcp, 0.0.0.0:65172->6443/tcp
k3d-labs-agent-1    rancher/k3s:v1.35.5-k3s1
k3d-labs-agent-0    rancher/k3s:v1.35.5-k3s1
k3d-labs-server-0   rancher/k3s:v1.35.5-k3s1

Show the address kubectl uses to reach the API server:

kubectl cluster-info
Kubernetes control plane is running at https://0.0.0.0:65172
...

The port is chosen randomly when the cluster is created, so yours differs.

Explanation

Each node is a container based on the rancher/k3s image. k3d adds one helper container:

ContainerPurpose
k3d-labs-server-0Server node: control plane, etcd and workloads
k3d-labs-agent-0Agent node: workloads
k3d-labs-agent-1Agent node: workloads
k3d-labs-serverlbProxy of k3d: forwards the ports of your machine into the cluster

Depending on your Docker setup, you may also see k3d-labs-tools, a short-lived helper that k3d uses for cluster operations.

The random port on k3d-labs-serverlb leads to the API server on port 6443. So every kubectl command takes this path:

kubectl -> https://0.0.0.0:65172 -> k3d-labs-serverlb -> k3d-labs-server-0:6443 -> kube-apiserver

Optional Tasks

Finished early? Try this task. The following labs don’t depend on it.

Task 1: Pull the Plug

Simulate a node failure: stop the container of an agent node and check what Kubernetes reports. It takes almost a minute to notice. Then start the container again: the following labs need all three nodes.

CommandEffect
docker stop <container>Stops a container
docker start <container>Starts a stopped container

Check the nodes with kubectl get nodes from Step 2.

Solution

Stop the container of an agent node:

docker stop k3d-labs-agent-1

Repeat kubectl get nodes every few seconds. After about 50 seconds, the node turns NotReady:

kubectl get nodes
NAME                STATUS     ROLES                AGE   VERSION
k3d-labs-agent-0    Ready      <none>               10m   v1.35.5+k3s1
k3d-labs-agent-1    NotReady   <none>               10m   v1.35.5+k3s1
k3d-labs-server-0   Ready      control-plane,etcd   10m   v1.35.5+k3s1

The kubelet reports regularly that its node is alive. When these reports stop, a controller in the kube-controller-manager marks the node as NotReady.

Start the container again:

docker start k3d-labs-agent-1

After a few seconds, kubectl get nodes shows the node as Ready again.