Archive, Purge and Restore

Archive and purge operations should be run on a scheduled basis. Use Data Archive and Results Archive to store captured and information for auditing. Amazon S3 Archive and Backup in Guardium also appears at the end of this topic.

Data Archive can be found by clicking Manage > Data Management.

Data Archive backs up the data that has been captured by the Guardium system, for a time period. When configuring Data Archive, a purge operation can also be configured. Typically, data is archived at the end of the day of everyday to ensure that in the event of a catastrophe, only one day of data is lost. The purging of data depends on the application and is highly variable, depending on business and auditing requirements. In most cases, data can be kept on the Guardium systems for more than six months.

Archive files can be sent using SCP or FTP protocol, or to an EMC Centera or TSM storage system (if configured). You can define a single archiving configuration for each Guardium® system.

Guardium’s archive function creates signed, encrypted files that cannot be tampered with. DO NOT change the names of the generated archive files. The archive and restore operations depend on the file names that are created during the archiving process.

Archive activities use the system shared secret to create encrypted data files. Before information encrypted on one system can be restored on another, the restoring system must have the shared secret that was used on the archiving system when the file was created.

Whenever archiving data, be sure to verify that the operation completes successfully. To do this, open the Aggregation/Archive Log by clicking Reports > Guardium Operational Reports > Aggregation/Archive Log. There should be multiple activities that are listed for each archive operation, and the status of each activity should as completed.

Perform System Backup tasks by clicking Manage > Data Management > System Backup. You can also perform backup tasks from the CLI.

Default Purging

The default value for purge is 60 days.
The default purge activity is scheduled every day at 5:00 AM.
For a new install, a default purge schedule is installed that is based on the default value and activity.
When a unit type is changed between manager that is managed or back to standalone, the default purge schedule is applied.
The purge schedule will not be affected during an upgrade.
When purging a large number of records (10 million or higher), a large batch size setting (500k to 1 million) is the most effective way to go. Using a smaller batch size or NULL causes the purge to take hours longer. Smaller purges finish quickly, so a large batch size setting is only relevant for large purges.

Note: Setting batch size is not available in the UI. Use the GuardAPI command grdapi set_purge_batch_size batchSize to set batch size.

How to determine what days are not archived

Use the Report Builder to view the list of all files with archive dates. Open the Report Builder by clicking Reports > Report Builder. From the Query menu, select Location View. Dates not on this report indicate that those dates have not been archived. Run archive for the dates not on the list, if required.

Configure Data Archive and Purge

Open the Data Archive by clicking Manage > Data Management > Data Archive.
To archive, check the Archive check box. Additional fields will appear in the Configuration panel.
For Archive data older than, enter a value and select a unit of time from the menu. To archive data starting with yesterday’s data, enter the value 1, and select Day(s) from the menu.
Use Ignore data older than to control how many days of data is archived. Any value that is specified here must be greater than the Archive data older than value. This is a mandatory field which has a default value of two.
Check the Archive Values check box to include values from SQL strings in the archived data. If this box is cleared, values are replaced with question mark characters on the archive (and hence the values will not be available following a restore operation).
Select a Protocols option, and fill in the appropriate information. Depending on how your Guardium system has been configured, one or more of these buttons might not be available. For a description of how to configure the archive and backup storage methods, see the description of the show and store storage-system commands.
Perform the appropriate procedure, depending on the storage method selected:
- Configure SCP or FTP Archive or Backup
- Configure EMC Centera Archive or Backup
- Configure TSM Archive or Backup
- Configure Amazon S3 Archive or Backup
Check the Purge check box to define a purge operation.
If purging data, use the Purge data older than field to specify a starting day for the purge operation as a number of days, weeks, or months before the current day, which is day zero. All data from the specified day and all older days are purged, except as noted. Any value that is specified for the starting purge date must be greater than the value specified for the Archive data older than value. In addition, if data exporting is active, the starting purge date that is specified here must be greater than the Export data older than value. See the IMPORTANT note.
Note:
There is no warning when you purge data that has not been archived or exported by a previous operation.

The purge operation does not purge restored data whose age is within the do not purge restored data timeframe that is specified on a restore operation.
Click Apply to save the configuration changes. The system attempts to verify the configuration by sending a test data file to that location.
- If the operation fails, an error message is displayed and the configuration will not be saved.
- If the operation succeeds, the configuration is saved.
To run or schedule the archive and purge operation, do one of the following:
- Click Run Once Now to run the operation once.
- Click Modify Schedule to schedule the operation to run on a regular basis.
Click Done when you are finished.

Configure SCP or FTP Archive or Backup

After selecting SCP or FTP in an archive or backup configuration panel, the following information must be provided:

For Host, enter the IP address or host name of the host to receive the archived data.
For Directory, identify the directory in which the data is to be stored. How you specify this depends on whether the file transfer method used is FTP or SCP.
- For FTP: Specify the directory relative to the FTP account home directory.
- For SCP: Specify the directory as an absolute path.
For Port that can be used to send files over SCP and FTP. The default port for ssh/scp/sftp is 22. The default port for FTP is 21.
Note: Seeing a zero (0) for port indicates that the default port is being used and that there is no need to change.
For Username and Password, enter the credentials for the user logging on to the SCP or FTP server. This user must have write/execute permissions for the directory that is specified in Directory.
For Windows, a domain user is accepted with the format of domain\user
Click Apply to save the configuration.

Configure EMC Centera Archive or Backup

This backup or archiving task copies files to an EMC Centera storage system off-site. A license is needed with user name and password from EMC. Four main actions are needed for this task:

Establish account with an EMC Centera on the network (IP addresses and a ClipID are needed)
Configure the data and/or configuration files from a Guardium system
Define and export a library
Confirm that your files are stored on the EMC Centera storage system.

CLI action

From the CLI, run these commands:

store storage-system centera backup ON
show storage-system

Configure TSM Backup

Open System Backup by clicking Manage > Data Management > System Backup. Select TSM. The following information must be provided:

For Retention, enter the number of days to retain the data. The maximum is 24855 (68 years). If you want to save it for longer, you can restore the data later and save it again.
For Centera Pool Address, enter the Centera Pool Connection String; for example: 10.2.3.4,10.6.7.8?/var/centera/us1_profile1_rwe.pea txt
Note: This IP address and the .PEA file comes from EMC Centera. The question mark is required when configuring the path. The .../var/centera/... path name is important as the backup might fail if the path name is not followed. The .PEA file gives permissions, username, and password authentication per Centera backup request.
Click Upload PEA File to upload a Centera PEA file to be used for the connection string. The Centera Pool Address is still needed.
Note: If the message Cannot open the pool at this address.. appears, check the size of the Guardium system host name. A timeout issue has been reported with Centera when using host names that are fewer than four characters in length.
Click Apply to save the configuration. The system attempts to verify the Centera address by opening a pool using the connection string specified. If the operation fails, you will be informed and the configuration will not be saved.
Click Run Once Now to perform the backup using the downloaded .PEA file.

Confirm that your files have been copied to the TSM location. The name of the files and a ClipID are required for this task.

Configure TSM Archive

Before archiving to a TSM server, a dsm.sys configuration file must be uploaded to the Guardium system, via the CLI. Use the import tsm config CLI command. After you select TSM in an archive or backup configuration panel, provide following information:

For Password, enter the TSM password that this Guardium system uses to request TSM services, and re-enter it in the Re-enter Password box.
Optionally, enter a Server name matching a servername entry in your dsm.sys file.
Optionally, enter an As Host name.
Click Apply to save the configuration. When you click the Apply button, the system attempts to verify the TSM destination by sending a test file to the server using the dsmc archive command. If the operation fails, you will be informed and the configuration will not be saved.
Return to the archiving or backup procedure to complete the configuration.

Restore Data

Before Restoring Data

Before restoring from TSM, a dsm.sys configuration file must be uploaded to the Guardium system, via the CLI. Use the import tsm config CLI command.
Before restoring from EMC Centera, a pea file must be uploaded to the Guardium system, via the Data Archive panel.
Before restoring or importing a file that was encrypted by a different Guardium system, make sure that the system shared secret used by the Guardium system that encrypted the file is available on this system (otherwise, it will not be able to decrypt the file). See About the System Shared Secret in System Configuration.
Before restoring on a Guardium system run the CLI command stop inspection-core to stop the inspection-core process.
Note: The data cannot be captured during the restore process.

To restore data:

Open Data Restore by clicking Manage > Data Management > Data Restore.
Enter a date in From to specify the earliest date for which you want data.
Enter a date in To to specify the latest date for which you want data.
For Host Name, optionally enter the name of the Guardium system from which the archive originated.
Click Search.
In the Search Results panel, check the Select check box for each archive you want to restore.
In the Don't purge restored data for at least field, enter the number of days that you want to retain the restored data on the system.
Click Restore.
Click Done when you are finished.

Amazon S3 Archive and Backup in Guardium

Use this feature to archive and backup data, from Guardium, to Amazon S3.

Amazon S3 (Amazon Simple Storage Service) provides a simple web service interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, secure, inexpensive infrastructure that Amazon uses to run its own web sites.

Prerequisites

An Amazon account.
Register for S3 service
Amazon S3 credentials are required in order to access Amazon S3. These credentials are:
- Access Key ID - identifies user as the party responsible for service requests. It needs to be included it in each request. It is not confidential and does not need to be encrypted. (20-character, alphanumeric sequence).
- Secret Access Key - Secret Access Key is associated with Access Key ID calculating a digital signature included in the request. Secret Access Key is a secret, and only the user and AWS should have it (40-character sequence). This key is just a long string of characters (and not a file) that is used to calculate the digital signature that needs to be included in the request.

Data Archive backs up the data that has been captured by the system, for a given time period.

When Guardium data is archived, there is a separate file for each day of data.

Archive data file name format:

 <time>-<hostname.domain>-w<run_datestamp>-d<data_date>.dbdump.enc

Guardium's archive function creates signed, encrypted files that cannot be tampered with. The names of the generated archive files should not be changed. The archive operation depends on the file names that are created during the archiving process.

System backups are used to backup and store all the necessary data and configuration values to restore a server in case of hardware corruption.

All configuration information and data is written to a single encrypted file and sent to the specified destination, using the transfer method that is configured for backups on this system.

Backup system file format:

<data_date>-<time>-<hostname.domain>-SQLGUARD_CONFIG-9.0.tgz
<data_date>-<time>-<hostname.domain>-SQLGUARD_DATA-9.0.tgz

Use the Aggregation/Archive Log report in Guardium to verify that the operation completes successfully. Open the Aggregation/Archive Log by clicking Reports > Guardium Operational Reports > Aggregation/Archive Log. There should be multiple activities that are listed for each Archive operation, and the status of each activity should be Succeeded.

Enable Amazon S3 from the Guardium CLI

Amazon S3 archive and backup option is not enabled by default in the Guardium GUI. To enable Amazon S3 via Guardium CLI, run the following CLI commands:

store storage-system amazon_s3 archive on
store storage-system amazon_s3 backup on

Amazon S3 requires that the clock time of Guardium system to be correct (within 15-minutes). Otherwise, this results in an Amazon error. If there is too large a difference between the request time and the current time, the request will not be accepted.

If the Guardium system time is not correct, set the correct time using the following CLI commands:

show system ntp server
store system ntp server (An example is ntp server: ntp.swg.usma.ibm.com)
store system ntp state on

User Interface

Use the System Backup to configure the backup. Open the System Backup by clicking Manage > Data Management > System Backup.

User input requires:

S3 Bucket Name (Every object that is stored in Amazon S3 is contained in a bucket. Buckets partition the namespace of objects that are stored in Amazon S3. Within a bucket, you can use any names for your objects, but bucket names must be unique across all of Amazon S3.
Access Key ID
Secret Access Key

If bucket name does not exist, it will get created.

Secret Access Key is encrypted when saved into the database.

Check that files got uploaded on Amazon S3

Log onto AWS Management Console using your email address and password.

http://aws.amazon.com/console/

Click S3.
Click the bucket that you specified in Guardium UI.