Metadata collection with multiple data collectors

You can deploy multiple data collectors to help ensure that metadata collection is not interrupted by unexpected data collector failures or to optimize the metadata collection from data centers in different geographical regions or locations.

You add storage systems, switches, and fabrics to get valuable insights. To get those insights, you rely on the data collector, which collects the asset, configuration, capacity, and performance metadata from your data center for analysis and presentation in IBM® Storage Insights Pro.

Balancing the metadata collection load

Deploying multiple data collectors to balance the processing load for collecting metadata can be an effective strategy for maintaining and sustaining your monitoring operations. It can help you avoid bandwidth, connectivity, network, or other potential performance issues when metadata is collected. A key method for implementing this strategy is the ability to manually assign your devices to specific data collectors.

Keep in mind the following information about load balancing when you deploy and assign multiple data collectors for monitoring:

Assignments are disabled	Assignments are enabled
If data collectors are available and operating normally, IBM Storage Insights does not balance the metadata collection load across multiple data collectors. You must enable data collector assignments to balance the load effectively across multiple data collectors.	You can balance the metadata collection load by explicitly assigning individual monitored devices to specific data collectors. When you change the data collectors that are assigned to a device, IBM Storage Insights begins implementing the changes immediately. The time that is required depends on the number of devices that you reassign and might take up to an hour to complete. To view assignments and assignment changes, go to Configuration > Data Collectors and click Assignments.
If the primary data collector for a monitored device fails to collect metadata, IBM Storage Insights attempts to reassign the device to another available data collector. The data collector with the fastest response time becomes the new primary data collector and resumes collecting metadata for the device. When the original primary data collector for the device becomes available again, it's not reassigned as the primary data collector.	If the primary data collector for a monitored device fails to collect metadata, IBM Storage Insights attempts to reassign the device to another data collector that is allowed to monitor the device. The data collector with the fastest response time that is allowed to monitor the device becomes the new primary data collector and resumes collecting metadata for the device. When the original primary data collector for the device becomes available again, it's not reassigned as the primary data collector.

Assignments are disabled

Assignments are enabled

If data collectors are available and operating normally, IBM Storage Insights does not balance the metadata collection load across multiple data collectors. You must enable data collector assignments to balance the load effectively across multiple data collectors.

You can balance the metadata collection load by explicitly assigning individual monitored devices to specific data collectors. When you change the data collectors that are assigned to a device, IBM Storage Insights begins implementing the changes immediately. The time that is required depends on the number of devices that you reassign and might take up to an hour to complete. To view assignments and assignment changes, go to Configuration > Data Collectors and click Assignments.

If the primary data collector for a monitored device fails to collect metadata, IBM Storage Insights attempts to reassign the device to another available data collector.

The data collector with the fastest response time becomes the new primary data collector and resumes collecting metadata for the device. When the original primary data collector for the device becomes available again, it's not reassigned as the primary data collector.

If the primary data collector for a monitored device fails to collect metadata, IBM Storage Insights attempts to reassign the device to another data collector that is allowed to monitor the device.

The data collector with the fastest response time that is allowed to monitor the device becomes the new primary data collector and resumes collecting metadata for the device. When the original primary data collector for the device becomes available again, it's not reassigned as the primary data collector.

Pro tip: To ensure the availability of metadata collection and to help balance workload, deploy two or more data collectors on separate servers in each of your data centers.

Deployment scenarios

In following scenario, data collectors are deployed on two different servers in the same data center. If one of the data collectors fails, the remaining data collectors are tested. The data collector with the fastest response time that is also allowed to monitor the device resumes collecting the metadata for the device.

In following scenario, two data collectors are deployed in two data centers that aren't connected over IP.

Tip: To help manage and troubleshoot the collection of metadata, specify the location of the resources that you add for monitoring. If you don't specify the location when you add storage resources, you can specify it later. From the Resources menu, select the type of resource, such as Block Storage Systems. Then, right-click the resources that you selected, and click Edit Propertices.