Expose the Etcd Metrics Endpoint

Learn how to expose the etcd metrics endpoint.

To allow monitoring tools to collect metrics from your etcd database, you need to explicitly expose the etcd metrics endpoint.

Choose the tab that matches how you manage your cluster.

Talos

1. Create a patch file named etcd-metrics-patch.yaml that exposes the etcd metrics endpoint on port 2381, accessible from all network interfaces:

   cat << EOF > etcd-metrics-patch.yaml
   cluster:
     etcd:
       extraArgs:
         listen-metrics-urls: http://0.0.0.0:2381
   EOF
   

2. Create a CP_IPS variable that contains the IP addresses of your control plane nodes:

   CP_IPS="<control-plane-ip-1>,<control-plane-ip-2>,<control-plane-ip-3>"
   

3. Ensure you are in your project’s directory and apply the etcd-metrics-patch.yaml patch to your control plane nodes.

   talosctl patch machineconfig \
   --patch @etcd-metrics-patch.yaml \
   --endpoints $CP_IPS \
   --nodes $CP_IPS \
   --talosconfig=./talosconfig
   

   Note: You can also export your TALOSCONFIG variable and then remove the --talosconfig=./talosconfig flag in the patch command above.

4. After the node reboots, run the following command to confirm that the etcd metrics endpoint is accessible:

   CP_IP=$(echo $CP_IPS | cut -d',' -f1)
   curl "${CP_IP}:2381/metrics"
   

5. Secure your control plane IP addresses to prevent public access. See the Ingress Firewall guide for instructions on securing your control plane.

Omni

1. This is the config patch that exposes the etcd metrics endpoint on port 2381, accessible from all network interfaces:

   cluster:
     etcd:
       extraArgs:
         listen-metrics-urls: http://0.0.0.0:2381
   

2. Create a CP_IPS variable that contains the IP addresses of your control plane nodes. You’ll need this for the verification step below:

   CP_IPS="<control-plane-ip-1>,<control-plane-ip-2>,<control-plane-ip-3>"
   

3. Apply the patch as a cluster config patch targeted at your control plane nodes:

   1. Select the Clusters tab in the left-hand menu.
   2. Open the cluster menu (⋯) and select Config Patches. (Alternatively, open the specific cluster and select Config Patches from the right-hand panel.)
   3. Click Create Patch.
   4. Select Control Planes from the Patch Target dropdown.
   5. Enter the config patch from step 1 into the patch editor.
   6. Click Save. Omni applies the configuration to the selected machines.

4. Confirm the nodes come back healthy in Omni.

5. After the nodes are back up, run the following command to confirm that the etcd metrics endpoint is accessible:

   CP_IP=$(echo $CP_IPS | cut -d',' -f1)
   curl "${CP_IP}:2381/metrics"
   

6. Secure your control plane IP addresses to prevent public access. See the Ingress Firewall guide for instructions on securing your control plane.