Local Storage

Using local storage for Kubernetes workloads.

Using local storage for Kubernetes workloads implies that the pod will be bound to the node where the local storage is available. Local storage is not replicated, so in case of a machine failure contents of the local storage will be lost.

User volumes

The simplest way to use local storage is to use user volume.

Once the user volume is created, it is automatically mounted under /var/mnt/u-<user-volume-name> path on the node.

For example, create a configuration patch for a user volume named local-storage:

local-storage.yaml

apiVersion: v1alpha1
kind: UserVolumeConfig
name: local-storage
provisioning:
  diskSelector:
    match: "!system_disk"
  minSize: 2GB
  maxSize: 2GB

Apply the patch to the machine configuration:

talosctl --nodes <worker-ip> patch mc --patch @local-storage.yaml

If there is enough space available on a non-system disk (see diskSelector), the user volume will be created and mounted under /var/mnt/u-local-storage path on the node.

talosctl -n <worker-ip> get volumestatus u-local-storage
talosctl -n <worker-ip> get mountstatus u-local-storage

Now you can use the /var/mnt/local-storage path in your Kubernetes manifests to refer to the local storage:

apiVersion: v1
kind: Pod
metadata:
  name: local-storage-pod
spec:
  containers:
  - name: local-storage-container
    # ...
    volumeMounts:
    - mountPath: /usr/share
      name: local-storage-volume
  volumes:
  - name: local-storage-volume
    hostPath:
      path: /var/mnt/local-storage
      type: DirectoryOrCreate

Local path provisioner

Local Path Provisioner can be used to dynamically provision local storage.

First, we will create a separate user volume for the Local Path Provisioner to use. Apply the following machine configuration patch:

Ensure that you have enough space available to provision the user volume.

apiVersion: v1alpha1
kind: UserVolumeConfig
name: local-path-provisioner
provisioning:
  diskSelector:
    match: disk.transport == 'nvme'
  minSize: 200GB
  maxSize: 200GB

Make sure to update Local Path Provisioner configuration to use the user volume path /var/mnt/local-path-provisioner as the root path for the local storage.

For example, Local Path Provisioner can be installed using kustomize with the following configuration:

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- github.com/rancher/local-path-provisioner/deploy?ref=v0.0.31
patches:
- patch: |-
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: local-path-config
      namespace: local-path-storage
    data:
      config.json: |-
        {
                "nodePathMap":[
                {
                        "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
                        "paths":["/var/mnt/local-path-provisioner"]
                }
                ]
        }
- patch: |-
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: local-path
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
- patch: |-
    apiVersion: v1
    kind: Namespace
    metadata:
      name: local-path-storage
      labels:
        pod-security.kubernetes.io/enforce: privileged

Put kustomization.yaml into a new directory, and run kustomize build | kubectl apply -f - to install Local Path Provisioner to a Talos Linux cluster. There are three patches applied:

• change default /opt/local-path-provisioner path to /var/mnt/local-path-provisioner
• make local-path storage class the default storage class (optional)
• label the local-path-storage namespace as privileged to allow privileged pods to be scheduled there

To test the Local Path Provisioner, you can refer to the Usage section of the official guide.

You can check that directories for PVCs are created on the node’s filesystem with the talosctl ls /var/mnt/local-path-provisioner command.