Data aggregation
Data aggregation is the process where raw data is gathered and expressed in a summary form for statistical analysis.
For example, raw data can be aggregated over a given time period to provide statistics such as average, minimum, maximum, sum, and count. After the data is aggregated and written to a view or report, you can analyze the aggregated data to gain insights about particular resources or resource groups. There are two types of data aggregation:
- Time aggregation
- All data points for a single resource over a specified time period.
- Spatial aggregation
- All data points for a group of resources over a specified time period.
Time intervals for data collection and aggregation
Data is collected and presented in a view or report within the context of various time intervals:
- Reporting period
- The period over which data is collected for presentation. For example, a resource summary table might contain data collected for a particular network device over a period of one day. A reporting period can include aggregated data points or raw data (data that has not been aggregated). Supported reporting periods are Daily, Weekly, Monthly, Quarterly, and Yearly.
- Granularity
- The period over which data points for a given resource or set of resources are collected for aggregation. For example, if you want to find the average of the data points for a given resource collected over a 5 minute period, the granularity is 5 minutes. Granularity can range from 1 minute to one month, depending on the reporting period, and view or report type. DataView aggregates data dynamically for a granularity of less than a day. DataChannel aggregates data for larger granularity values.
- Polling period
- The time duration that determines how often resources are sampled for data. For example, a group of resources might be polled every 5 minutes, meaning that a data point for each resource is generated every 5 minutes. Polling period and granularity are both factors in the result of a spatial aggregation. For example, suppose you want to find the average of a set of data points collected for a group of devices over a 10 minute period (the granularity). If the polling period is also 10 minutes, the result is the average of single data points collected for each device. But if the polling period is 5 minutes, each device is sampled twice within the 10 minute granularity period. The aggregated result is the average of all collected data points, the single data points collected for each resource during the first polling period plus the single data points collected during the second polling period.
This table shows the granularity supported for a given reporting period, and view or report type.
| View or Report | Day | Week | Month | Quarter | Year | Sliding 14 Month |
|---|---|---|---|---|---|---|
| GST | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| RTT | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| RST | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| Proactive RST | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| Proactive RTT | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| TopN | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
| Line Chart | Raw 1, 5, 10, 15, 30 Minutes, 1 Hour | Raw 1, 5, 10, 15, 30 minutes, 1, 2, 4, 6, 12 Hours, 1 Day | 5, 10, 15, 30 Minutes, 1, 2, 4, 6, 12 Hours, 1 Day, 1 Week | 1 Day, 1 Week | 1 Day, 1 Week, 1 Month | 1 Day, 1 Week, 1 Month |
| Time Series Table Summary | Raw 1, 5, 10, 15, 30 Minutes, 1 Hour | Raw 1, 5, 10, 15, 30 Minutes, 1, 2, 4, 6, 12 Hours, 1 Day | 5, 10, 15, 30 Minutes, 1, 2, 4, 6, 12 Hours, 1 Day, 1 Week | 1 Day, 1 Week, 1 Day, 1 Week | 1 Month | 1 Day, 1 Week, 1 Month |
| Baseline | Raw 1, 5, 10, 15, 30 Minutes, 1 Hour | Raw 1, 5, 10, 15, 30 Minutes, 1, 2, 4, 6, 12 Hours, 1 Day | 1, 2, 4, 6, 12 Hours, 1 Day, 1 Week | Not supported | Not supported | Not supported |
| Rank Variation | 1 Day | 1 Week | 1 Month | Not supported | Not supported | Not supported |
| Pie Chart | 1 Day | 1 Week | 1 Month | 1 Month | 1 Month | 1 Month |
Daily and weekly group time series charts do not support raw data. Advanced group time series charts are restricted to a granularity of one week or less. With monthly reporting periods, Line Charts and Time Series Table Summaries can only be assigned a granularity of 5, 10, 15, or 30 minutes. For information about changing time series granularity from the portal, see Changing time series granularity.
Data storage
Data is written to the Tivoli® Netcool® Performance Manager database as follows:
- Raw (not aggregated) data is written to the database once per hour.
- Aggregated data is written to the database once per hour.
- Near real-time (NRT) data is raw data collected since the last time data was last written to the database. Since data is written to the database once an hour, NRT data can be collected for up to an hour.