2.6. Services and Networking

In this lab, you make an application reachable with a Service: inside the cluster, from your machine and with two versions to switch between.

Step 1: Create a Service

Create a directory for the files of this lab and change into it:

mkdir -p ~/labs/2.6
cd ~/labs/2.6

Create the file web-blue.yaml in ~/labs/2.6. The application whoami replies with the name of the Pod that answers, so you can see where each request ends up:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
      version: blue
  template:
    metadata:
      labels:
        app: web
        version: blue
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.12.0

Create the file web-service.yaml with a Service for these Pods:

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
    version: blue
  ports:
    - port: 80
      targetPort: 80

Create the Deployment:

kubectl apply -f web-blue.yaml
deployment.apps/web-blue created

Create the Service:

kubectl apply -f web-service.yaml
service/web-service created

List the Services:

kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes    ClusterIP   10.43.0.1       <none>        443/TCP   4h
web-service   ClusterIP   10.43.195.110   <none>        80/TCP    4s

Show the details of the Service:

kubectl describe service web-service
Name:                     web-service
...
Selector:                 app=web,version=blue
Type:                     ClusterIP
...
IP:                       10.43.195.110
...
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
Endpoints:                10.42.1.30:80,10.42.0.46:80
...

If Endpoints is empty, the Pods are still starting. Wait a moment and repeat the command.

Explanation

A Service gives a group of Pods a stable IP address and name. It finds its Pods with the selector, like kubectl get pods -l in 2.3: all Pods with the labels app=web and version=blue. Endpoints lists the IP addresses of the two Pods it found. Pods come and go with new IP addresses, but the IP of the Service stays. The second label version lets you tell two versions apart in Step 5.

port is the port of the Service, targetPort the port of the containers. The type ClusterIP (the default, as your manifest sets no type) makes the Service reachable inside the cluster only. The Service kubernetes belongs to the API server.


Step 2: Call the Service by Name

Start a temporary Pod and call the Service six times from inside the cluster:

kubectl run client --image=busybox:1.37 --rm -it --restart=Never -- \
  sh -c 'for i in 1 2 3 4 5 6; do wget -q -O - http://web-service | grep Hostname; done'
Hostname: web-blue-6cd47fdb56-ftn47
Hostname: web-blue-6cd47fdb56-lr687
Hostname: web-blue-6cd47fdb56-lr687
Hostname: web-blue-6cd47fdb56-ftn47
Hostname: web-blue-6cd47fdb56-ftn47
Hostname: web-blue-6cd47fdb56-lr687
pod "client" deleted from default namespace

kubectl may also print a few more lines, e.g. If you don't see a command prompt, try pressing enter. or Session ended, resume using 'kubectl attach …' command. They come from connecting your terminal to the temporary Pod. You can ignore them. Now and then, kubectl can’t connect in time: it then prints couldn't attach to pod/client, falling back to streaming logs and shows the output a second time.

Explanation

kubectl run with --rm -it starts a Pod, shows you its output and deletes it afterwards. --restart=Never keeps it from starting again after the command ended (the restartPolicy from 2.3). Everything after -- is the command of the container.

The Pod reached the Service by its name: CoreDNS, the DNS server of the cluster, resolves web-service to the IP of the Service. The Service spreads the requests over both Pods.


Step 3: Forward a Port to Your Machine

Forward the port 8080 of your machine to the Service. The command keeps running:

kubectl port-forward service/web-service 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

If the port 8080 is already in use on your machine, use another one, e.g. 8081:80, and call localhost:8081.

Open http://localhost:8080 in your browser, or call it in a second terminal:

curl http://localhost:8080
Hostname: web-blue-6cd47fdb56-ftn47
...

For every request, the first terminal shows Handling connection for 8080. Stop the port forwarding with Ctrl+C.

Explanation

kubectl port-forward opens a tunnel from your machine through the API server to one Pod of the Service. All requests go to this Pod, and the Service doesn’t spread them. It’s the quickest way to look at an application while you develop or debug it, but it only works while the command runs. The port listens on your machine only, which makes it a safe way to reach a database that shouldn’t be exposed.


Step 4: Expose the Service with a NodePort

Change the type of the Service in web-service.yaml to NodePort and add a fixed nodePort:

spec:
  type: NodePort
  selector:
    app: web
    version: blue
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080

Apply the manifest:

kubectl apply -f web-service.yaml
service/web-service configured

List the Services:

kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes    ClusterIP   10.43.0.1       <none>        443/TCP        4h
web-service   NodePort    10.43.195.110   <none>        80:30080/TCP   9s

Call the application four times from your machine:

for i in 1 2 3 4; do curl -s http://localhost:30080 | grep Hostname; done
Hostname: web-blue-6cd47fdb56-lr687
Hostname: web-blue-6cd47fdb56-lr687
Hostname: web-blue-6cd47fdb56-ftn47
Hostname: web-blue-6cd47fdb56-lr687

Explanation

A Service of type NodePort opens its nodePort on every node of the cluster. 80:30080/TCP in the column PORT(S) means that port 80 of the Service is also reachable on port 30080 of every node. In 2.1, k3d forwarded the ports 30080 to 30089 of your machine to the server node. That’s why localhost:30080 reaches the Service.

A Service of type LoadBalancer would get its own external IP from a load balancer. Your cluster has none, so the EXTERNAL-IP of such a Service would stay <pending>.


Step 5: Switch Between Two Versions

Create the file web-green.yaml with a second version of the application:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-green
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
      version: green
  template:
    metadata:
      labels:
        app: web
        version: green
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.12.0

Create the Deployment:

kubectl apply -f web-green.yaml
deployment.apps/web-green created

Call the application twice:

for i in 1 2; do curl -s http://localhost:30080 | grep Hostname; done
Hostname: web-blue-6cd47fdb56-lr687
Hostname: web-blue-6cd47fdb56-lr687

Both versions run now, but the Service still sends all requests to blue. Change the selector in web-service.yaml to the green version:

  selector:
    app: web
    version: green

Apply the manifest:

kubectl apply -f web-service.yaml
service/web-service configured

Wait a moment, then call the application again:

for i in 1 2 3 4; do curl -s http://localhost:30080 | grep Hostname; done
Hostname: web-green-7d5f65c999-djcvw
Hostname: web-green-7d5f65c999-djcvw
Hostname: web-green-7d5f65c999-djcvw
Hostname: web-green-7d5f65c999-v4dgj

Explanation

This is a blue/green release: the new version runs next to the old one, and the selector of the Service decides which of them gets the requests. After a second or two, the switch applies to all requests. Switching back is just as quick, because blue keeps running until you delete it. Both versions run the same image here, so only the Pod names show the switch. In a real release, green runs the new image.


Optional Tasks

Finished early? Try these tasks. The following labs don’t depend on them. Each task lists the commands you need. Open the solution if you get stuck.

Task 1: Canary

Send most requests to blue and only a few to green. Hint: the Service sends requests to all Pods that match its selector, evenly per Pod.

CommandEffect
<command> | sort | uniq -cCounts how often each line of output occurs
Solution

Remove the line version: green from the selector in web-service.yaml, so that the Service selects the Pods of both versions:

  selector:
    app: web

Set replicas: 1 in web-green.yaml. Apply both manifests:

kubectl apply -f ~/labs/2.6/web-green.yaml
kubectl apply -f ~/labs/2.6/web-service.yaml

Wait a moment until the green Pod is ready, then call the application twelve times and count the answers:

for i in $(seq 1 12); do curl -s http://localhost:30080 | grep Hostname; done | sort | uniq -c
      4 Hostname: web-blue-6cd47fdb56-ftn47
      4 Hostname: web-blue-6cd47fdb56-lr687
      4 Hostname: web-green-7d5f65c999-djcvw

Two blue Pods and one green Pod: about a third of the requests reach the new version (the canary). Each request picks a Pod at random, so your numbers differ. If the canary works well, you scale green up and blue down.

Task 2: Long Distance Call

Call the Service from a Pod in another namespace. Does the short name web-service still work?

CommandEffect
kubectl run <name> -n <namespace> ... -- <command>Starts the temporary Pod in another namespace
http://<service>.<namespace>Address of a Service in another namespace
Solution

Create a namespace:

kubectl create namespace other
namespace/other created

Call the Service from a temporary Pod in this namespace:

kubectl run client -n other --image=busybox:1.37 --rm -it --restart=Never -- wget -q -O - http://web-service
wget: bad address 'web-service'
pod "client" deleted from other namespace
pod other/client terminated (Error)

The short name only works inside the same namespace. Add the namespace to the name:

kubectl run client -n other --image=busybox:1.37 --rm -it --restart=Never -- wget -q -O - http://web-service.default
Hostname: web-green-7d5f65c999-djcvw
...
pod "client" deleted from other namespace

The full name of the Service is web-service.default.svc.cluster.local. Delete the namespace again:

kubectl delete namespace other

Task 3: Nobody Home

Change the selector of the Service to version: purple, a version that doesn’t exist. What happens when you call the application, and how can you find the cause?

You know all the commands you need.

Solution

Change the selector in web-service.yaml and apply the manifest:

kubectl apply -f ~/labs/2.6/web-service.yaml
service/web-service configured

Call the application:

curl http://localhost:30080
curl: (52) Empty reply from server

The details of the Service show the cause:

kubectl describe service web-service
...
Selector:                 app=web,version=purple
...
Endpoints:
...

No Pod matches the selector, so the Service has no endpoints. A typo in the selector is one of the most common reasons why a Service doesn’t work, so check Endpoints first. Set the selector back and apply the manifest again.


Clean Up

Delete all resources of this lab, since the following labs don’t need them:

kubectl delete -f ~/labs/2.6/
deployment.apps "web-blue" deleted from default namespace
deployment.apps "web-green" deleted from default namespace
service "web-service" deleted from default namespace

If you did Task 2 and the namespace other still exists, delete it with kubectl delete namespace other.


Complete files

~/labs/2.6/web-blue.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
      version: blue
  template:
    metadata:
      labels:
        app: web
        version: blue
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.12.0

~/labs/2.6/web-green.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-green
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
      version: green
  template:
    metadata:
      labels:
        app: web
        version: green
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.12.0

~/labs/2.6/web-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  type: NodePort
  selector:
    app: web
    version: green
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080