Preparing against failure

You can prepare against the possibility and consequences of some types of failures.

Parent topic: Preparing your cluster for fault tolerance

Preparing for failure of a distributed state persistent volume

About this task

You can take some actions to minimize the risk and effects of failure on a distributed state persistent volume.
Note: The figures are indicated as examples only. Adjust them to your needs and constraints.

Procedure

  1. Regularly back up the persistent volume.
  2. Enable Kafka topic retention and set it to a higher frequency than the frequency of the persistent volume backup.

    For example, you might back up the persistent volume every 24 hours and set the Kafka topic retention to 60 hours.

Preparing for Elasticsearch failure

About this task

You can take some actions to minimize the risk and effects of failure on the Elasticsearch cluster.

Procedure

  1. Regularly take snapshots of Elasticsearch.

    See also Elasticsearch aliases and indexes.

  2. Regularly back up the persistent volume.
  3. Enable Kafka topic retention and set it to a higher frequency than the frequency of the persistent volume backup.

    For example, you might back up the persistent volume every 24 hours and set the Kafka topic retention to 60 hours.