Configuring retention buckets

Configure retention policies to define how long IBM® QRadar® is required to keep event and flow data, and what to do when that data reaches a certain age.

About this task

Changes to the retention bucket filters are applied immediately to incoming data only. For example, if you configured a retention bucket to retain all data from source IP address 10.0.0.0/8 for 1 day, and you later edit the filter to retain data from source IP 192.168.0.1, the change is not retroactive. Immediately upon changing the filter, the retention bucket has 24 hours of 10.0.0.0/8 data, and all data that is collected after the filter change is 192.168.0.1 data.

The retention policy on the bucket is applied to all data in the bucket, regardless of the filters criteria. Using the previous example, if you changed the retention policy from 1 day to 7 days, both the 10.0.0.0/8 data and the 192.168.0.1 data in the bucket is retained for 7 days.

The Distribution of a retention bucket indicates the retention bucket usage as a percentage of total data retention in all your retention buckets. The distribution is calculated on a per-tenant basis.

Procedure

  1. On the navigation menu ( Navigation menu icon ), click Admin.
  2. In the Data sources section, click Event Retention or Flow Retention.
  3. If you configured tenants, in the Tenant list, select the tenant that you want the retention bucket to apply to.
    Note: To manage retention policies for shared data in a multi-tenant configuration, choose N/A in the Tenant list.
  4. To configure a new retention bucket, follow these steps:
    1. Double-click the first empty row in the table to open the Retention Properties window.
    2. Configure the retention bucket parameters.
      Learn more about retention bucket parameters:
      Properties Description
      Name Type a unique name for the retention bucket.
      Keep data placed in this bucket for The retention period that specifies how long the data is to be kept. When the retention period is reached, data is deleted according to the Delete data in this bucket parameter. QRadar does not delete data within the retention period.
      Delete data in this bucket Select Immediately after the retention period has expired to delete data immediately on matching the Keep data placed in this bucket for parameter. The data is deleted at the next scheduled disk maintenance process, regardless of disk storage requirements.

      Select When storage space is required to keep data that matches the Keep data placed in this bucket for parameter in storage until the disk monitoring system detects that storage is required.

      Deletions that are based on storage space begin when the free disk space drops to 15% or less, and the deletions continue until the free disk space is 18% or the policy time frame that is set in the Keep data placed in this bucket for field runs out. For example, if the used disk space reaches 85% for records, data is deleted until the used percentage drops to 82%. When storage is required, only data that matches the Keep data placed in this bucket for field is deleted.

      If the bucket is set to Delete data in this bucket: immediately after the retention period has expired, no disk space checks are done and the deletion task immediately removes any data that is past the retention.

      Description Type a description for the retention bucket.
      Current Filters Configure the filter criteria that each piece of data is to be compared against.
    3. Click Add Filter after you specify each set of filter criteria.
    4. Click Save.
  5. To edit an existing retention bucket, select the row from the table and click Edit.
  6. To delete a retention bucket, select the row from the table and click Delete.
  7. Click Save.

    Incoming data that matches the retention policy properties is immediately stored in the retention bucket.