IBM Support

How to process the data files that failed to be imported previously (post Data Import errors resolution)?

Troubleshooting


Problem

In situations where an IBM Security Guardium™ Aggregator appliance has faced some troubles running the "Data Import" process successfully, for example due to low disk space, corrupted database tables, or some configuration issues, it is important to run the process again once these problems are resolved, so that the data that was not processed can be prepared for querying and reporting.

Symptom

This document is intended to be informational only, and is not focused on any specific error message, condition or symptom.

Cause

Explain how to find and process the data files that were not successfully imported into an IBM Security Guardium™ Aggregator appliance after resolving one or more problem conditions that caused the "Data Import" process execution to fail.

Environment

The sample images shown in this document were taken from an IBM Security Guardium™ v10.6 system.

The same process should be similar in other versions of the product (e.g. 11.0 and above).

Diagnosing The Problem

This document is intended to be informational only, and is not focused on any specific error message, condition or symptom.

Resolving The Problem

1. Login to the appliance using the Guardium™ "cli" account.

2. Look for the pending data files of the latest failed data imports, by running the following cli command:

aggregator list failed imports
 

Example:

image-20180910124943-1

3. Restore the data file(s) you want to be processed again in the next execution of the "Data Import" job. To do this, execute the "aggregator recover failed import" cli command followed by the full name of a file obtained in the previous step. The syntax of the command is as follows:

aggregator recover failed import <filename | all >
Where:
<filename>.  File name of a failed data import job, obtained using the  "aggregator list failed imports" cli command.
Alternatively, you can use the "all" (without quotes) option of this command to process all the failed import files found.
Example:

image-20180905120103-2

Note 1. The command can process one file at a time, or all the files found, but not some at once.

In the context of this process, "restore" means to place a data file that could not be imported previously into the queue for processing and use it in the next execution of the "Data Import" process.

Tip 1: First you should try to recover a single file, then run the "Data Import" process to test if it does already work correctly.

 

 

Tip 2. If the "Data Import" process works correctly after doing a test, then only those files whose data is within the retention period should be restored, by executing the "aggregator recover failed import" cli command for each required file.
Alternatively, the "aggregator recover failed import all" command can be used to restore all the failed import files found, if considered prudent.

 

 

Tip 3. Since IBM Security Guardium™ v10 it is possible to know the data retention period by executing the "show purge_age_period" cli command.
This value can be set up from the GUI. It is represented by the value of the field "Data older than" when the "Purge" aggregation process is configured.
image 8298

 

 

Tip 4. To know if a data file should be restored, you can refer to the string "...-dYYY-MM-DD..." in the file name of the intended data import file (where, YYYY are 4 numeric characters that represent the year, MM represents the calendar month and DD represents the day).
This string refers to the date of the data contained in the file.

4. Test if the "Data Import" works. Open the GUI, navigate to "Manage > Data Management > Data Import".
Locate the "Run Once Now" button and click over it.

image-20180904155951-5


Expected outcome:

The "Manage > Reports > Data Management > Aggregation/Archive Log" report must show the "Data Import" finished with success and the files queued ("restored") were processed.

image-20180904160214-6

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMPHH","label":"IBM Security Guardium"},"Component":"Aggregation","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
10 February 2021

UID

ibm10730483