Question & Answer
Question
What are the Ariel Data Retention Policies in QRadar 7.2.0 and later?
Answer
Prior to the release of QRadar Version 6.2, event and flow data was deleted as soon as it was older than the retention settings specified in the QRadar Admin tab. Many users instead asked for the ability to keep data as long as possible, and to delete based on a percentage used instead of at a set date. The use of a set date would also occasionally cause issues where at a certain data rate, a user would specify too long a time period and the system would reach the maximum capacity and shut down.
In current versions of QRadar, a new event and flow data deletion policy has been enabled. In QRadar version 7.2.0 and up to 7.2.6, data is kept as long as possible by writing to disk (uncompressed, for performance) until storage hits 85% used. This occurs regardless of the specified retention setting. Once 85% usage is reached, QRadar compresses data starting with the oldest and ending with data more than 4 hours old. Once all data is compressed and usage is still at 85%, QRadar will then delete data that is older than the retention setting. As of QRadar 7.2.7 all data is compressed. If you set your retention too high, you may still over commit your storage and increase usage to 95%, where QRadar automatically stops collecting data. The net effect is that QRadar now stores data for as long as possible and only deletes data as required.
A few customers have reported that they are required by policy to delete data after a certain period. In current versions of QRadar you can set custom retention buckets for Events and Flows. The 10 non-default retention buckets are processed sequentially from top to bottom. Any events that do not match the retention buckets are automatically placed in the default retention bucket, located at the bottom of the list. Custom retention buckets allow the ability to add a time period and filters. If you enable a retention bucket with a defined criteria it will start deleting data from the time is was created. Any data that matches the custom retention bucket before it was created is subject to the criteria of the default retention bucket setting. If you need to delete data from before the Custom retention bucket was created you can shorten the default retention bucket so data is deleted immediately.
Note: If you change this setting, this process may take a long time. It may be best to manually delete old data after backing up the data to an external drive.
Where do you find more information?
Related Information
Historical Number
1235
Was this topic helpful?
Document Information
Modified date:
21 June 2018
UID
swg21622758