Introduction to Transparent Cloud Tiering
This topic provides a brief introduction to the Transparent Cloud Tiering feature available in IBM Spectrum Scale™.
Transparent Cloud Tiering is a separately installable feature of IBM Spectrum Scale that provides a native cloud storage tier. It allows data center administrators to free up on-premise storage capacity, by moving out cooler data to the cloud storage, thereby reducing capital and operational expenditures. The Transparent Cloud Tiering feature leverages the existing ILM policy query language semantics available in IBM Spectrum Scale, and administrators can define policies to tier data to a cloud storage. On an IBM Spectrum Scale cluster with multiple storage tiers configured, this external cloud storage can be used as the cooler storage tier to store infrequently accessed data from a cool storage pool. For performance reasons, it is recommended not to move any active or hot data to this external storage pool, as it drives excessive data traffic on the Transparent Cloud Tiering which in turn can cause delays, leading to problems like application timeouts.
Figure 1 illustrates this feature.
- Core Java™ library, also known as Transparent Cloud Tiering server package, that handles the connection with the cloud object storage provider. It is installed only on CES nodes running RHEL 7.1 and above. The node on which the Transparent Cloud Tiering server package is installed is hereafter referred to as the Gateway node.
- A binary or client package that enables IBM Spectrum Scale ILM (Information Lifecycle Manager) policy engine to redirect migration of cold data to the Gateway node and then to the cloud storage tier. This component is installed on all other cluster nodes running supported Linux distributions.
- Supported use cases
- Tiering of file data from IBM
Spectrum Scale file
system to a cloud Object storage, improving storage efficiency and cost effectiveness.
- Data must be cool and not actively accessed by applications.
- Specify selection of cool data by using ILM policies. You can use the policy to enable migration of files from a particular storage pool or fileset (existing ILM functions)
- File stubs that are present in the IBM Spectrum Scale file system should not be removed.
- When file stubs are accessed, files are transparently recalled (this feature can be used for occasional data access, not for high performance access as the latency can potentially cause application timeouts). It is recommended that customers must keep the transparent recall policy applied at all times on the file system that is managed by Transparent Cloud Tiering.
- For high performance applications, use the policy to perform a bulk recall operation
- For performance reasons, it is recommended that the average file size to be migrated to the cloud tier should be greater than 1 MB. Migration is supported for file size less than 1 MB, but performance will be slower due to overheads associated with small files.
- For the Transparent Cloud Tiering, data on the cloud object storage is opaque and cannot be accessed directly by applications. All I/O operations must happen through IBM Spectrum Scale system.
- Unsupported use cases
- Transparent Cloud Tiering does not support the following use cases:
- Using Transparent Cloud Tiering to migrate/recall hot (active) data
- Using Transparent Cloud Tiering as a backup mechanism
- Using Transparent Cloud Tiering in disaster recovery scenarios.