Deploy kube-router CNI

In this guide you will learn how to set up kube-router CNI on Talos.

kube-router is a turnkey solution for Kubernetes networking that provides pod networking, network policy enforcement, and service proxy using Linux kernel technologies (BGP, IPVS, IPTables, and eBPF).

Machine configuration preparation

When generating the machine config for a node, set the CNI to none and disable the default kube-proxy since kube-router replaces it.

cat <<EOF > patch.yaml
cluster:
  network:
    cni:
      name: none
  proxy:
    disabled: true
EOF

talosctl gen config \
    my-cluster https://mycluster.local:6443 \
    --config-patch @patch.yaml

If you want to use kube-router alongside kube-proxy instead of replacing it, omit proxy.disabled: true from the patch and set kubeRouter.run_router=--run-service-proxy=false in the kube-router DaemonSet arguments.

Installation

Omni

If you are using Omni, you can deploy kube-router using the manifest sync feature in a cluster template.

Step 1. Download the kube-router manifest:

curl -Lo kube-router.yaml https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter-all-features.yaml

Step 2. Reference the manifest in your Omni cluster template using the file field:

Step 3. Apply the cluster template:

omnictl cluster template sync --file cluster-template.yaml

Omni will wait until the Kubernetes API is available and the cluster is healthy before applying the kube-router manifests. See Sync Kubernetes Manifests for more details on manifest sync modes and status monitoring.

Manifest install

After applying the machine config and bootstrapping, Talos will appear to hang on phase 18/19 with the message: retrying error: node not ready. This happens because nodes in Kubernetes are only marked as ready once the CNI is up. As there is no CNI defined, the boot process is pending and will reboot the node to retry after 10 minutes, this is expected behavior.

Step 1. During this window, deploy kube-router with all features (pod networking, network policies, and service proxy) by applying the upstream manifest:

kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter-all-features.yaml

After kube-router is installed the boot process should continue and complete successfully.

If you only want kube-router for pod networking and network policies (keeping kube-proxy for service proxy), use the following manifest instead and do not disable kube-proxy in your machine config:

kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml

Cleanup of kube-proxy

If you deployed kube-router as a replacement for kube-proxy and kube-proxy was previously running, clean up the iptables rules left behind:

kubectl -n kube-system delete ds kube-proxy
kubectl -n kube-system delete cm kube-proxy

Refer to the kube-router user guide for more configuration options.