Easy Tier automatic processing

Edit online

With Easy Tier®, both IOPS and bandwidth algorithms determine when to migrate your data. This process can help you improve performance.

Use Easy Tier to relocate extents to the most appropriate storage tier in a multiple-tier pool, which is based on usage. Because workloads typically concentrate I/O operations on only a subset of the extents within a volume or LUN, Easy Tier identifies the subset of the frequently accessed extents and relocates them to the higher-performance storage tier.

Subvolume or sub-LUN data movement is an important option to consider in volume movement because not all data at the volume or LUN level becomes hot data. For any workload, there is a distribution of data that is considered either hot or cold, which can result in significant overhead that is associated with moving entire volumes between tiers. For example, if a volume is 1 TB, you do not want to move the entire 1 TB volume when the generated heat map indicates that only 10 GB is considered hot. This capability uses your higher performance tiers to reduce the number of drives that you need to optimize performance.

You can use high performance storage tiers with a much smaller cost. This means that you invest a small portion of storage in the high-performance storage tier. You can use Easy Tier for relocation and tuning without the need for your intervention, generating cost-savings while optimizing storage performance.

You also have the option of assigning specific logical volumes to a storage tier. This feature is useful to ensure that critical data is always highly available, regardless of how often the data is accessed.

Three-tier automatic processing is supported by the following Easy Tier functions:
  • Support for ESE volumes with the thin provisioning
  • Support for a matrix of device (DDM) and adapter types
  • Monitoring of both bandwidth and IOPS limitations
  • Data demotion between tiers
  • Automatic hot spot rebalancing, which applies to the following auto performance rebalance situations:
    • Redistribution within a tier after a new rank is added into a managed pool
    • Redistribution within a tier after a rank is removed from a managed pool
    • Redistribution when the workload is imbalanced on the ranks within a tier of a managed pool
  • Logical volume assignment to specific storage tiers by using Easy Tier Application
  • Heat map transfer to secondary storage by using the Heat Map Transfer Utility

To help manage and improve performance, Easy Tier is designed to identify hot data at the subvolume or sub-LUN (extent) level, which is based on ongoing performance monitoring, and then automatically relocate that data to an appropriate storage device in an extent pool that is managed by Easy Tier. Easy Tier uses an algorithm to assign heat values to each extent in a storage device. These heat values determine on what tier the data would best reside, and migration takes place automatically. Data movement is dynamic and transparent to the host server and to applications that are using the data.

Easy Tier provides capabilities to support the automatic functions of auto-rebalance, warm demotion, and cold demotion. These capabilities include support for pools with three tiers.

Easy Tier can also help you to manage the thin provisioning of your ESE volumes.

Auto-rebalance

Rebalance is a function of Easy Tier to balance the extents in the same tier that is based on usage. Auto-rebalance supports single-tier managed pools and multiple-tier pools. You can use the Storage Facility Image (SFI) control to enable or disable the auto-rebalance function on all pools of an SFI. When you enable auto-rebalance, every standard and ESE volume is placed under Easy Tier management for auto-rebalancing procedures. Using auto-rebalance gives you the advantage of these automatic functions:
  • Easy Tier operates within a tier, inside a managed storage pool.
  • Easy Tier automatically detects performance skew and rebalances extents within the same tier.
  • Easy Tier automatically rebalances extents when capacity is added to the extent pool.

In any tier, placing highly active (hot) data on the same physical rank can cause the hot rank or the associated device adapter (DA) to become a performance bottleneck. Likewise, over time skews can appear within a single tier that cannot be addressed by migrating data to a faster tier alone, and require some degree of workload rebalancing within the same tier. Auto-rebalance addresses these issues within a tier in both hybrid and homogeneoussingle-tier and multiple-tier pools. It also helps the system respond in a more timely and appropriate manner to overloading, skews, and any under-utilization that can occur from the addition or deletion of hardware, migration of extents between tiers, changes in the underlying volume configurations, and variations in the workload. Auto-rebalance adjusts the system to continuously provide optimal performance by balancing the load on the ranks and on DA pairs.

Easy Tier supports auto-rebalancing within single-tier pools. If you set Easy Tier to monitor tiered pools only, pools with a single-tier can rebalance the intra-tier ranks. If Easy Tier is turned off, then no volumes are managed. If Easy Tier is on, it manages all the volumes that it supports (Standard or ESE).

Notes:
  • Standard and ESE volumes are supported.
  • If Easy Tier is set to monitor tiered pools only, then single-tier pools are also managed to rebalance intra-tier ranks.

Warm demotion

Warm demotion operation demotes warm (or mostly sequential-accessed) extents to the next lowest tier to protect the drive performance on the system. The ranks being demoted to are selected randomly. This function is triggered when bandwidth thresholds are exceeded. Extents are warm-demoted from one rank to another rank among tiers when extents have high bandwidth but low IOPS.

It is helpful to understand that warm demotion is different from auto-rebalancing. While both warm demotion and auto-rebalancing can be event-based, rebalancing movement takes place within the same tier while warm demotion takes place among more than one tier. Auto-rebalance can initiate when the rank configuration changes. It also periodically checks for workload that is not balanced across ranks. Warm demotion initiates when an overloaded rank is detected.

Cold demotion

Cold demotion recognizes and demotes cold or semi-cold extents to an appropriate lower-cost tier. Cold extents are demoted in a storage pool to a lower tier if that storage pool is not idle.

Cold demotion occurs when Easy Tier detects any of the following scenarios:
  • Extents in a storage pool become inactive over time, while other data remains active. This situation is the most typical use for cold demotion, where inactive data is demoted to a lower performance tier. This action frees up extents on the higher performance tier before the extents on the lower performance tier become hot. This helps the system to be more responsive to the new, hot data.
  • All the extents in a storage pool become inactive simultaneously due to either a planned or unplanned outage. Disabling cold demotion assists you in scheduling extended outages or experiencing outages without effecting the extent placement.
  • All extents in a storage pool are active. In addition to cold demotion by using the capacity in the lowest tier, an extent is selected which has close to zero activity, but with high sequential bandwidth and low random IOPS for the demotion. Bandwidth available on the lowest tier is also used.
All extents in a storage pool can become inactive due to a planned non-use event, such as an application that reaches its end of life. In this situation, cold demotion is disabled and you can select one of the following options:
  • Allocate new volumes in the storage pool and plan on those volumes that become active. Over time, Easy Tier replaces the inactive extents on the higher performance tier with active extents on the lower performance tier.
  • Depopulate all of the higher performance ranks. When all higher performance ranks are depopulated, all extents in the pool are on the lower performance ranks. Store the extents on the lower performance ranks until they need to be deleted or archived to tape. After the higher performance ranks are depopulated, move them to a storage pool.
  • Leave the extents in their current locations and reactivate them later.
Figure 1 illustrates all of the migration types that are supported by the Easy Tier enhancements in a three-tier configuration. The auto-performance rebalance might also include more swap operations.
Figure 1. Three-tier migration types and their processes
Three-tier migration types and their processes

Easy Tier Application

You can assign logical volumes to specific storage tiers. This action enables applications or storage administrators to proactively influence data placement in the tiers. Applications, such as databases, can optimize access to critical data by assigning the associated logical volumes to a higher performance tier. Storage administrators, as well, can choose to assign a boot volume (for example) to a higher performance tier.

Assigning a logical volume applies to all extents that are allocated to the logical volume. Any extents added to a logical volume by dynamic extent relocation or volume expansion are also assigned to the specified tier. All assignments have an infinite lease. Assigning a volume across multiple tiers is not supported.

The completion of a logical volume assignment is a best-effort service that is based on the following Easy Tier priorities:
  1. System Health

    Easy Tier monitors devices to ensure that they are not overloaded for the current configuration and workload. Warm Demote operations and extra checks receive the highest priority processing in this regard.

  2. Performance

    Logical volume assignment requests are performed on the appropriate device types based on configuration, device capabilities, and workload characteristics.

  3. Capacity

    System capacity requirements are monitored.

Additionally, assignment requests can originate at multiple sources and can be delivered to Easy Tier through channels that do not guarantee ordering of messages. For this reason, the order of servicing volume assignment requests cannot be guaranteed.

Because system health is the highest priority, a logical volume assignment can be overridden by a migration operation (such as, Warm Demote), or by DS8000® microcode. As a result, although Easy Tier Application is designed to achieve eventual consistency of operations, there is no system state guarantee for an assignment, even for completed requests. The status of a logical volume assignment request can be:
  • Failure

    The request command is invalid and cannot be complete. A failure response is returned to the calling function.

  • Transient State

    The request cannot currently be completed, but is awaiting processing. A request that completed can revert to a pending state if any of its actions are undone by a higher priority request (such as a Warm Demote operation).

    Additionally, the threshold (maximum capacity) for assigning logical volumes to a specified tier can be reached. The threshold is 80% of the total capacity available on that tier. In this case, all assignment requests for that tier remain pending until the assignments fall below the threshold.

  • Assignment Failure
    In some situations, a volume assignment request is acknowledged by the Easy Tier Application, but subsequent system state changes require that the Easy Tier Application return the request as a volume assignment failure. Possible scenarios are:
    • A tier definition change due to rank addition, deletion, depopulation, or merging the extent pool.
    • Easy Tier is disabled for the volume.

    The assignment failure remains until you unassign the volume. However, even while in assignment failure status, the volume is still managed by EasyTier auto functions based on its heat map.

    If a logical volume is deleted, Easy Tier Application unassigns the volume, but does not identify the status as an assignment failure.

    Note: For Version 7 Release 4, assignment failure works differently with the introduction of Easy Tier Application for IBM® Z . A new status indicator, "assign pending hardware condition," is used to describe the following conditions. If a condition is later resolved, the assignment continues to be processed.
    Easy Tier becomes disabled:
    The assignment remains in a pending state, and you receive a status of "assign pending hardware condition" instead of an "assign fail." If you later activate Easy Tier, the committed assignment automatically proceeds.
    Target tier becomes unavailable:
    You receive a status of "assign pending hardware condition," and the assignment remains in a pending state. If you later add ranks to the target tier, the committed assignment automatically proceeds.
    Tier definition changes:
    The physical tier is remembered, and a tier definition change does not impact the assignment.
    Before Version 7 Release 4, for all the assignment failures described above, even though the condition is later resolved, the affected volumes still stay in the "assign failure" state. You need to send an unassign request to fix it. In Version 7 Release 4, you can still expect assignment failures caused by various conditions (the target tier does not exist; Easy Tier management functions are turned off; the 80% capacity limitation is exceeded; and so on) that cause the assign command to be rejected. However, after the conditions are fixed and an assign command is accepted, any changes that affect assignment activities produce only an "assign pending hardware condition," rather than an assignment-request failure.

Logical volume assignment state and request information are regularly saved to local storage or storage on the peer server. If interruptions or error conditions occur on the storage system, this data is automatically restored from the persistent storage.

Easy Tier Application for IBM Z

Easy Tier Application for IBM Z provides comprehensive data-placement management policy support between an application and storage. With this feature, you need to program the policy only once, and it is then enforced automatically. With hints about the data usage and performance expectations, storage is automatically optimized towards higher performance and efficiency. At the same time, the hint semantics relieve the application from the burden of storage resource management.

Easy Tier Application Control at pool and volume levels

Easy Tier Application Control at the pool and volume levels provides a more granular and flexible control of workload learning and data movement, as well as providing volume-level tier restriction where a volume can be excluded from either the lower performance tier or both the higher performance and lower performance tiers..

Before this feature, Easy Tier provided control at the system level. To prevent or control the placement of data, you had to disable and enable Easy Tier for the entire DS8000. Flexibility was limited. For example, if there was a performance problem within a single pool or volume, Easy Tier for the entire DS8000 needed be stopped until the problem was corrected. This stoppage resulted in a loss of performance benefits in other pools or volumes.
Note: System-level control always has higher priority than the pool-level and volume-level control settings. If any of the system-level control settings (Easy Tier monitor; Easy Tier management) are changed, the pool and volume level control settings are reset. Changes to the system-level control settings are detected by Easy Tier every five minutes.
Several scenarios of how you can use Easy Tier customer control at the pool level and volume level are described in Table 1.
Table 1. Scenarios for Easy Tier Customer control at pool and volume levels
Function Scenario
Suspend/resume Easy Tier learning At the pool level
  • A bank has a monthly and quarterly temporary batch workload, during which the workload differs from normal workloads. During the temporary batch workload, Easy Tier moves data to get good performance. However, the data configuration might not be optimal for normal workloads, so when the normal workload starts again, the performance is not as good as before. In this case, you can suspend pool learning with a duration setting when you start the temporary batch workload. After the duration expires, the pool learning resumes automatically, which makes the control easier. Alternately, you can resume the pool learning manually.
  • Similarly, you could use Easy Tier control at the pool level for other tasks that have workloads that differ from normal workloads. Examples of such one-off tasks are restoring a database from a backup, database loading, and database reorganization.
At the volume level
  • One application is running the Monday-through-Friday workload, and another application is running the Saturday-through-Sunday workload. During the first workload, the application gets good performance because Easy Tier recognizes that it is hot and promotes it to the highest tier. But during the weekend, the first workload is no longer hot, and Easy Tier might swap another application into the highest tier. On the next Monday morning, the application that depends on the Monday-through-Friday workload might encounter a performance impact because Easy Tier needs time to readjust the data placement for it. In this case, you can suspend the volume learning (keep the heat) of that application at the end of the Monday-through-Friday period with an additional 48 hours of lease time. On the next Monday morning, the learning resumes automatically and the performance should be stable.
  • During the application maintenance, such as code upgrade or backup, the performance statistics are not reflecting the real workload. To avoid the population of the learning data, in this case you can suspend the learning when you are doing an upgrade and resume it after the upgrade is done.
Reset Easy Tier learning At the pool level
  • You want to redefine the use of all the volumes within a storage pool. The original learning data of the volumes in the pool are no longer relevant, but you can reset the Easy Tier learning in the pool, so that Easy Tier learning reacts to the new workload quickly.
    Note: In many environments (especially open systems), a pool-level reset of learning is less typical as there is likely to be a mix of applications and workloads. However, this is effectively a volume-level reset of learning for all volumes in the pool.
  • Another scenario is when you transport a workload. You select a target pool, and the target pool’s learning data is no longer relevant. But you can reset the pool learning to react to the new workload quickly.
At the volume level
  • When an application with a large amount of hot data is no longer used, the heat of the volumes that are associated with the application might take time to cool and stop other applications from leveraging the flash drive quickly. In this case, you can reset the learning history of the specific volumes, so other data can take advantage of the flash drive quickly.
  • During a database reorganization, the hot-data indexes are moved to another location by the database, so the learning history of the original data location is no longer relevant. In this case, you can reset the learning.
  • When you deploy a new application, you might define the file system, migrate the application data, and complete some testing before you put the new application online. The learning data during the deployment might create data-storage “noise” for the normal production workload. To alleviate the noise, you can reset the learning before the application goes online so that Easy Tier reacts quickly to the presence of the application.
Suspend/resume extent relocation At the pool level
  • In one scenario, there might be some response time-sensitive period during which you want to prevent any data-movement impact to the performance. In this case, you can suspend the pool migration with a duration setting. After the duration is expired, the pool migration resumes automatically, which makes the control easier. You can also resume it manually.
  • In another scenario, there is a performance issue in a pool, and you want to analyze the problem. You can prevent an impact to storage during your analysis by suspending the pool’s migration to stabilize the performance of the storage.
At the volume level
Not applicable.
Query pool-level and volume-level Easy Tier control state You can query the Easy Tier control state of a pool or volume.
Exclude from lower performance tier control At the pool level
Not applicable.
At the volume level
  • If there is an application for which you do not want the data of the volume to be moved to the lower performance tier, you can exclude the volume from the lower performance tier.
  • During the deployment of an application, before the workload starts, the volumes that are allocated for the application might be idle. You can exclude the idle volumes from being demoted to the lower performance tier to avoid the performance issues when application starts.
  • To more efficiently prevent a volume from ever being demoted to the lower performance drives, you can exclude the volume from the lower performance tier so that it is only assigned to the non-lower performance tiers.