Skip to content

Kubernetes

Direktiv works with Kubernetes offerings from all major cloud providers and the requirements for on-premise or local installations is Kubernetes 1.19+. The following documentation describes a small installation with k3s.

K3s

Direktiv is using k3s as one of the recommended certified Kubernetes distribution because of its low resource requirements. Although all version starting from 1.20 will most likely work, the installation has been tested with the following versions.

Versions
v1.20.5+k3s1
v1.21.4+k3s1

Server configuration

Direktiv supports Kubernetes setups with seperate server and agents nodes as well as small setups with nodes acting as server and agent in one node. The small setup can use the internal etcd instance whereas seperated installation might use external database. Both installation types achieve high availability.

The nodes communicate with each other on different ports and protocols. The following table shows the ports required to be accessible (incoming) for the nodes to enable this. On some Linux distributions firewall changes have to be applied. Please see k3s installation guide for detailed installation instructions.

Protocol Port Source Description
TCP 6443 K3s agent nodes Kubernetes API Server
UDP 8472 K3s server and agent nodes VXLAN
TCP 10250 K3s server and agent nodes Kubelet metrics
TCP 2379-2380 K3s server nodes Required for HA with embedded etcd only

Firewall changes (Centos/RedHat):

sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=8472/udp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --reload

Info: For additional Centos/RedHat instructions: https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-red-hat-centos-enterprise-linux

One of Kubernetes' requirements is to disable swap on the nodes. This change need to be applied permanently to survive reboots.

Disable swap

sudo swapoff -a
sudo sed -e '/swap/s/^/#/g' -i /etc/fstab

Node Installation

K3s provides a script to install K3s. It is recommended to use this for installation. The configuration can be done via environment variables during installation. For Direktiv the default ingress controller (Traefik) needs to be disabled because Kong will be used. For installations using the embedded etcd the first server node requires the '--cluster-init' flag.

First Server node

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --write-kubeconfig-mode=644 --cluster-init" sh -

To add nodes to the cluster the node token is required, which is saved under /var/lib/rancher/k3s/server/node-token. With this token additional nodes can be added.

Additional server nodes

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --write-kubeconfig-mode=644" K3S_TOKEN="<TOKEN FROM NODE-TOKEN FILE>" K3S_URL=https://<cluster ip>:6443 sh -

K3s will download container images during installation. For the downloads of those internet connectivity is required. If the nodes are behind a proxy server the Linux environment variables need to provided to the service, e.g.:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable traefik --write-kubeconfig-mode=644" K3S_TOKEN="<TOKEN FROM NODE-TOKEN FILE>" K3S_URL=https://<cluster ip>:6443 HTTP_PROXY="http://192.168.1.10:3128" HTTPS_PROXY="http://192.168.1.10:3128" NO_PROXY="localhost,127.0.0.1,svc,.cluster.local,192.168.1.100,192.168.1.101,192.168.1.102,10.0.0.0/8" sh -
Back to top