Welcome to the pvc-autoresizer Project!

pvc-autoresizer resizes PersistentVolumeClaims (PVCs) when the free amount of storage is below the threshold.

It queries the volume usage metrics from Prometheus that collects metrics from kubelet.

Our supported platforms are:

• Kubernetes: 1.33, 1.32, 1.31
• CSI drivers that implements the following features
  • Volume Expansion
  • NodeGetVolumeStats
• CPU Architecture: x86 (*1), arm64 (*2)

*1 Tier1 support. The official docker images are provided and all functionalities are tested by CI.
*2 Tier2 support. The official docker images are provided, but no tests run by CI.

Container images are available on ghcr.io.

Getting Started

Prepare

pvc-autoresizer needs a source of volume usage metrics. You can provide metrics in one of two ways:

• Prometheus: collect kubelet metrics with Prometheus and point pvc-autoresizer to your Prometheus HTTP endpoint.
• Kubernetes Metrics API: use the cluster's Metrics API (for example, a metrics-server or a compatible implementation) instead of Prometheus.

Note:

• When this document mentions Prometheus, any Prometheus-compatible backend (for example, VictoriaMetrics or other systems that expose a Prometheus-compatible HTTP query API) can be used as well.
• Using the Kubernetes Metrics API, the kube-apiserver might be loaded to some degree. For large clusters or high-scale environments, using a Prometheus-based solution is recommended to avoid extra pressure on the API server.

If you plan to use Prometheus, please refer to the following pages to set it up:

• Installation | Prometheus

In addition, configure scraping as follows:

• Monitoring with Prometheus | TopoLVM

Installation

You must configure how pvc-autoresizer obtains volume usage metrics. Two supported methods are described below.

• Prometheus (HTTP endpoint): set the --prometheus-url argument to point to your Prometheus server endpoint.
• Kubernetes Metrics API: enable use of the cluster Metrics API by setting the --use-k8s-metrics-api=true argument.

pvc-autoresizer can be deployed to a Kubernetes cluster via helm.

Example: install using Prometheus endpoint

helm repo add pvc-autoresizer https://topolvm.github.io/pvc-autoresizer/
helm install --create-namespace --namespace pvc-autoresizer pvc-autoresizer pvc-autoresizer/pvc-autoresizer --set "controller.args.prometheusURL=<YOUR_PROMETHEUS_ENDPOINT>"

Example: install using Kubernetes Metrics API

helm repo add pvc-autoresizer https://topolvm.github.io/pvc-autoresizer/
helm install --create-namespace --namespace pvc-autoresizer pvc-autoresizer pvc-autoresizer/pvc-autoresizer --set "controller.args.useK8sMetricsApi=true"

See charts/pvc-autoresizer/README.md for detailed Helm options and additional configuration.

How to use

To allow auto volume expansion, the StorageClass of PVC need to allow volume expansion and have resize.topolvm.io/enabled: "true" annotation. The annotation may be omitted if you give --no-annotation-check command-line flag to pvc-autoresizer executable.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: topolvm-provisioner
  annotations:
    resize.topolvm.io/enabled: "true"
provisioner: topolvm.io
allowVolumeExpansion: true

To allow auto volume expansion, the PVC to be resized needs to specify the upper limit of volume size with the annotation resize.topolvm.io/storage_limit. The value of resize.topolvm.io/storage_limit should not be zero, or the annotation will be ignored.

The PVC must have volumeMode: Filesystem, too.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: topolvm-pvc
  namespace: default
  annotations:
    resize.topolvm.io/storage_limit: 100Gi
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 30Gi
  storageClassName: topolvm-provisioner

The PVC can optionally have resize.topolvm.io/threshold, resize.topolvm.io/inodes-threshold and resize.topolvm.io/increase annotations. (If they are not given, the default value is 10%.)

When the amount of free space of the volume is below resize.topolvm.io/threshold or the number of free inodes is below resize.topolvm.io/inodes-threshold, .spec.resources.requests.storage is increased by resize.topolvm.io/increase.

If resize.topolvm.io/increase is given as a percentage, the value is calculated as the current spec.resources.requests.storage value multiplied by the annotation value.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: topolvm-pvc
  namespace: default
  annotations:
    resize.topolvm.io/storage_limit: 100Gi
    resize.topolvm.io/threshold: 20%
    resize.topolvm.io/inodes-threshold: 20%
    resize.topolvm.io/increase: 20Gi
spec:
  <snip>

Initial resize

PVC request size can also be changed at the creation time based on the largest PVC size in the same group. PVCs are grouped by labels, and the label key for grouping is specified by resize.topolvm.io/initial-resize-group-by annotation.

For example, suppose there are following three PVCs.

### existing PVCs (excerpted)
kind: PersistentVolumeClaim
metadata:
  name: pvc-x-1
  labels:
    label-foobar: group-x
  annotations:
    resize.topolvm.io/initial-resize-group-by: label-foobar
spec:
  resources:
    requests:
      storage: 20Gi

kind: PersistentVolumeClaim
metadata:
  name: pvc-x-2
  labels:
    label-foobar: group-x
  annotations:
    resize.topolvm.io/initial-resize-group-by: label-foobar
spec:
  resources:
    requests:
      storage: 16Gi

kind: PersistentVolumeClaim
metadata:
  name: pvc-y-1
  labels:
    label-foobar: group-y
  annotations:
    resize.topolvm.io/initial-resize-group-by: label-foobar
spec:
  resources:
    requests:
      storage: 30Gi

When creating the following new PVC, pvc-x-1 and pvc-x-2 with label-foobar: group-x are considered to be in the same group, and pvc-y-1 is not. Therefore, the PVC is created with 20Gi based on pvc-x-1, which has the largest capacity in the group.

kind: PersistentVolumeClaim
metadata:
  name: pvc-x-3
  labels:
    label-foobar: group-x
  annotations:
    resize.topolvm.io/initial-resize-group-by: label-foobar
spec:
  resources:
    requests:
      storage: 10Gi

When creating the following new PVC, pvc-y-1 with label-foobar: group-y is in the same group. However, since the new PVC's size(50Gi) is larger than the existing one(30Gi), the PVC is created with 50Gi.

kind: PersistentVolumeClaim
metadata:
  name: pvc-y-2
  labels:
    label-foobar: group-y
  annotations:
    resize.topolvm.io/initial-resize-group-by: label-foobar
spec:
  resources:
    requests:
      storage: 50Gi

When the size of the largest PVC in the same group is larger than the value set to resize.topolvm.io/storage_limit annotation, the PVC is resized up to this limit.

Prometheus metrics

pvcautoresizer_kubernetes_client_fail_total

pvcautoresizer_kubernetes_client_fail_total is a counter that indicates how many API requests to kube-api server are failed.

pvcautoresizer_metrics_client_fail_total

pvcautoresizer_metrics_client_fail_total is a counter that indicates how many API requests to metrics server(e.g. prometheus) are failed.

pvcautoresizer_loop_seconds_total

pvcautoresizer_loop_seconds_total is a counter that indicates the sum of seconds spent on volume expansion processing loops.

pvcautoresizer_success_resize_total

pvcautoresizer_success_resize_total is a counter that indicates how many volume expansion processing resizes succeed.

pvcautoresizer_failed_resize_total

pvcautoresizer_failed_resize_total is a counter that indicates how many volume expansion processing resizes fail.

pvcautoresizer_limit_reached_total

pvcautoresizer_limit_reached_total is a counter that indicates how many storage limit was reached.

Contributing

pvc-autoresizer project welcomes contributions from any member of our community. To get started contributing, please see our Contributor Guide.

Communications

If you have any questions or ideas, please use discussions.

Resources

docs directory contains designs, and so on.

License

This project is licensed under Apache License 2.0.