5.3.4 Limitations and Known Issues for IBM Spectrum Control

Upgrading IBM Spectrum Control in a Windows domain from V5.2.13 or older directly to V5.3.4 causes the authentication of domain users to fail.
Upgrading an eWAS-based version of IBM Spectrum Control server (5.2.13  or older) directly to IBM Spectrum Control 5.3.4 in a Windows domain environment causes the authentication of domain users to fail.

IBM Spectrum Control V5.3.4 installation might appear to hang at Database component upgrade while database tables are cleaned up and migrated 
When upgrading IBM Spectrum Control 5.2.11 or earlier to 5.2.16 or later, some customers may experience a long duration of the upgrade where it appears to hang while database tables are cleaned up and migrated.  This is a known limitation, and is a required migration step that (depending on your database size) may take a long time to complete.  It is a one-time only clean up process, and will not be experienced at the next upgrade of IBM Spectrum Control.

You can review the log files to determine if your upgrade is undergoing this database clean and migration process:  In traceTPCInstall.log look near the end of the log file, there would be an executing SQL statement similar to one of these:
"SET INTEGRITY FOR T_RPT_HOST_KEYS ..."  or  "REFRESH TABLE T_RPT_HOST_KEYS ..."
And the last lines in the log file during the process would be similar to this:
"... INFO com.ibm.tpc.install.frontend.ia.actions.ProgressBarUpdater setPercentageComplete Seconds complete = ..."

Context Root Not Found in 5.3.4 on RedHat 7

On RedHat 7, Db2 and IBM Spectrum Control's web server are configured to start at the same time on boot up.  If Db2, for whatever reason, takes significantly longer to start than the IBM Spectrum Control web server, the user will see a 'Context Root Not Found' page when trying to load the IBM Spectrum Control GUI.  Restarting the webserver after DB2 has fully started will resolve the issue until the next reboot. 

To fix the issue across reboots, complete the following steps:

1.) Edit /etc/systemd/system/webWLP.service by adding the lines in bold:

[Unit]
Description=IBM Spectrum Control Web Server
After=db2fmcd.service

[Service]
ExecStartPre=/usr/bin/sleep 45
ExecStart=/usr/bin/bash -c "/opt/IBM/TPC/wlp/bin/server start webServer --clean > /opt/IBM/TPC/wlp/bin/../../web/log/startServer.log"
ExecStop=/usr/bin/bash -c "/opt/IBM/TPC/wlp/bin/server stop webServer  > /opt/IBM/TPC/wlp/bin/../../web/log/stopServer.log"
TimeoutStopSec=360
Type=forking
LimitNOFILE=4096
LimitNPROC=131072
LimitFSIZE=infinity
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

2.) Execute the following command as root:
# systemctl daemon-reload

Ensure that you've also implemented this fix for Db2 autostart on RedHat 7:
https://www.ibm.com/support/docview.wss?uid=swg21964393

Verifying that the Netlogon service is running when the Spectrum Control server is in a Windows Domain

General IBM Spectrum Control issues

macOS Catalina (10.15) increased security policies impacts IBM Spectrum Control 

APAR IT24763: Adding or modifying the connection information for an IBM FlashSystem V9000

When adding or modifying the connection information, be sure to use the correct credentials for the FlashSystem V9000: the connection information for the V9000 control nodes.

Probe of a FlashSystem V9000 will fail if connection credentials were for the internal FlashSystem 900 module rather than providing connection information for the V9000 control nodes.

The reported IBM Spectrum Control probe error is:  CMMVC7205E The command failed because it is not supported

IBM Spectrum Control can only manage the control nodes of a V9000 cluster, it cannot manage the backend Flashsystem module.

APAR IT23975: Port down/offline alerts generated for FlashSystem 900 when the port is simply unused/disconnected.

IBM Spectrum Control is generating "error" alerts for an IBM FlashSystem 900 storage device, for ports that are not physically connected to a SAN.  IBM Spectrum Control is receiving port alerts for ports that are known to be disconnected, and no matter the suppression settings selected, the alerts come back.  This differs from the way that all other IBM storage operates, including SVC, Storwize, and other IBM storage where unused ports do not trigger alerts and do not report errors about unused ports.  Typically port alerts should only trigger if a port that was connected, becomes disconnected.  That is how the rest of IBM storage environment operates.  At this time there is no way to make this behavior consistent with the rest of their storage.

The alert for inactive and configured FlashSystem port is working as designed because the condition is an "offline" status according FlashSystem GUI also.  Inactive and unconfigured FlashSystem port does not represent an error condition to generate alerts in IBM Spectrum Control.  The different port status inactive configured / inactive unconfigured reported by FlashSystem device are due to the installed small form-factor pluggable (SFP) transceivers onto the interface card in the system canister.  When SFP is installed and the port is not connected, the status reported by FlashSystem is inactive and configured port.

The workaround for changing the port status is to remove the unused SFP and reset the interface card in the canister.  More information how to add/remove SFPs to the FlashSystem storage enclosure can be found in the FlashSystem knowledge center:  https://www.ibm.com/support/knowledgecenter/en/STKMQB_1.5.1/com.ibm.storage.flashsystem900.1.5.1.doc/tms_add_sfps.html

Provisioning fails when requesting compressed volumes from a data reduction pool

When you provision compressed volumes from storage systems that run IBM Spectrum Virtualize 8.1.1 or later, you might see the following error message:

HWNCA0005E  IBM Spectrum Control cannot provide the requested <n> GiB of capacity that also satisfies the requirements of the service class.

This issue occurs under the following conditions:
- The service class specifies that the storage resources must be able to compress volumes.
- IBM Spectrum Control attempts to provision the storage from a data reduction pool.

Workaround:
Use a service class that does not specify that the storage resources must be able to compress volumes. Alternatively, ensure that the available IBM Storage Virtualizers contain storage pools that are not configured as data reduction pools.

HDS back-end storage devices behind a SAN Volume  shows as "not monitored"

IBM support worked with Hitachi support and this is the answer from Hitachi:

We have identified the root cause and are working on new SVP SMI-S Provider code to fix the issue.

Details: When IBM SMI-S Spectrum Control version 5.2.15.0 sends query “HITACHI_StorageProcessorSystem”, the
Controller ID on DKC1 side that the SVP SMI-S provider calculates was incorrect; so IBM SMI-S Spectrum Control
version 5.2.15.0 does not receive the port information for DKC1 side correctly from the SVP.
 
Unfortunately there is no workaround at present. The permanent fix will be to upgrade to the new code.
You can see the status of this APAR here: http://www.ibm.com/support/docview.wss?uid=swg1IT25032

Submitting two tiering tasks to run at the same time or overlap might cause one of them to not complete 

When submitting two tiering tasks to run at the same time or overlap their running times, they might stall and fail to complete.  There are no errors displayed, and the logs indicate that the tasks are still running, however if they stall no further tasks are updated or completed.  This situation can occur for the first tiering task or for both.

To avoid this problem, submit one tiering task at a time and waitr for it to complete before submitting another tiering task.

In Metro Mirror or Global Copy relationship, if a Peer to Peer Remote Copy (PPRC) is terminated and then deleted on one of the storage systems, the other storage system retains the relationship  
If a storage system is not aware of the termination due to the loss of the PPRC path, volumes on that system might be counted as both unprotected and protected.

To correct the reporting, take one of the following actions:

• Issue a terminate to the storage system, which retains the relationships.
• Reestablish the PPRC paths and the relationship by using the dscli command or another copy services management tool (such as the CSM product).

Provisioning fails with ADD_PORTS_TO_HOST message on XIV 11.5.0 when modifying host   

When trying to provision storage from an IBM XIV® 11.5.0 system, the provisioning task fails with an "ADD_PORTS_TO_HOST task failed" message when trying to provision storage to a host that is already defined on the XIV subsystem.  This can happen when the host definition does not contain all the Fibre Channel ports that IBM Spectrum Control is trying to assign the volume to.

For example, if you try to provision storage to a host that has 2 Fibre Channel ports and the host is
already defined on the XIV 11.5.0 system but with only one of the Fibre Channel ports, the assign job will fail and the second port will not be added to the host.

This condition does not occur when IBM Spectrum Control is creating the host definition, or when an existing host definition already has all ports defined. This issue does not occur with XIV 11.4 systems, and 11.5.1 or later systems.

Workaround: modify the host definition on the XIV 11.5.0 system to contain all the Fibre Channel ports, then run a probe of the XIV 11.5.0 system in IBM Spectrum Control or upgrade to XIV 11.5.1 or later.

Enabled ethernet port that is not connected to a switch can cause the GUI to hang  

On some IBM servers running Windows 2012, an enabled ethernet port can cause the IBM Spectrum Control web-based GUI to stop responding if the port is not connected to a switch.  This condition causes a Db2 SQL -911 error and also causes the IBM Spectrum Control GUI to stop responding.

To prevent this, disable the ethernet port or configure the port by using the instructions here:  https://publib.boulder.ibm.com/infocenter/bladectr/documentation/index.jsp?topic=/com.ibm.bladecenter.hs22.doc/dw1iu_t_lan_over_usb_install_windows_driver_install.html

Disable Windows User Account Control to edit configuration files

On Windows installations, if you installed IBM Spectrum Control by using a domain user account, you must disable User Account Control to edit the IBM Spectrum Control configuration files.

IBM Spectrum Control cannot detect the mount status of file systems on Scale Out Network Attached Storage and Storwize V7000 Unified storage systems   

If a file system is manually unmounted from an IBM Scale Out Network Attached Storage (SONAS) or Storwize® V7000 Unified storage system, IBM Spectrum Control cannot detect that the file system was unmounted. Because of this limitation, information that is displayed in the IBM Spectrum Control GUIs is not accurate and some storage-related actions on the file system might fail.

For example, when you provision storage, a file system on a Storwize V7000 Unified storage system might be recommended for provisioning. However, if that file system was manually unmounted after it was first detected by IBM Spectrum Control, the provisioning action will fail. An unmounted file system cannot be used for provisioning. To avoid this issue, ensure that the monitored file systems on the Storwize V7000 Unified storage system are mounted before you run the provisioning action.

IBM Spectrum Control cannot monitor SAS ports on Storwize devices   

After you add a Storwize device for monitoring in IBM Spectrum Control, you can view information about its health, status, capacity, and performance. However, you can't monitor the SAS ports on a Storwize device. To monitor those ports, use the management GUI of the device.

A new IBM SONAS fileset might be mistakenly deleted

The fileset information was deleted in the other system before the new fileset with the existing ID was created.

If IBM Spectrum Control discovers a new IBM SONAS fileset, the ID of a deleted file set might be reused for the new file set. If you delete the older fileset in IBM Spectrum Control, the new file set is deleted instead. This issue occurs in the following circumstances:

A file set that is known to IBM Spectrum Control is deleted in a system other than IBM Spectrum Control.
IBM Spectrum Control retained the deleted file set information, either because the probe was not run after the file set was deleted or because the retention period of the file set has not expired.

The file set information was deleted in the other system before the new fileset with the existing ID was created.

IBM SAN Volume Controller and IBM Storwize devices might not accurately report used space of data reduction pools and deduplicated volumes

When an SAN Volume Controller or Storwize device is configured with data reduction pools, it (on certain firmware levels, including 8.1.3.3) no longer reports the used space for a specific volume, and the tier distribution is no longer reported.  This is a limitation from the firmware with data reduction pools/deduplicated volumes.

Running multiple optimization recommendations at the same time results in a Pool Exhaustion Exception error  

In some cases, running more than 15 optimization recommendations at the same time can cause one or more of the individual recommendation executions to result in a Pool Exhaustion Exception error. This can be caused by a lack of available resources in the operating system thread pool. You can manually complete the failed recommendations.

Changing an existing NetApp filer that uses a Storage Resource agent to use NAPI 

You cannot change an existing NetApp filer that is managed by using a Storage Resource agent to be directly managed by using NetApp NAPI-based management. If you want to change from using a Storage Resource agent to NAPI for your existing NetApp filers, you must add the filers to IBM Spectrum Control again. Specify the NetApp credentials when you add the filer.

IBM Spectrum Control determines that the NetApp resource is already monitored. IBM Spectrum Control modifies the data source information so that the NetApp filer data is retrieved directly from the NetApp itself rather than from the Storage Resource agent.

Setting the rate at which data is copied between volumes that run IBM Spectrum Virtualize

IBM Spectrum Control can do tasks that significantly alter the structure of storage. For example, IBM Spectrum Control can add a mirrored volume copy, or migrate volumes from one pool to another by using storage optimization functions. When IBM Spectrum Control performs these tasks, a default synchronization rate of 100% is set.

For more information about how to change the synchronization rate, see the "Changing the synchronization rate for volumes on SAN Volume Controller or Storwize family systems"  http://www.ibm.com/support/knowledgecenter/SS5R93_5.3.4/com.ibm.spectrum.sc.doc/tbs_svc_setting_sync_rate.html .

Probes with a Storage Resource agent (SRA) on Linux servers hang and remain "In Progress" forever

SRA probes run Logical Volume Manager (LVM) commands and these can hang which causes the SRA to hang and appear to be "In Progress" indefinitely.  

To verify that this is the situation causing the hang, view the SRA agent trace log on the SRA Deployment and verify that the last command was an LVM one.

To recover from this situation:

1. Kill the hung SRA process
2. Kill the hung LVM command
3. From a command line, retry the LVM command and verify that it runs successfully. If it does not, use the LVM documentation to repair the LVM so that it does.  
4. Restart the SRA

You can follow the progress of the APAR related to this issue here:  http://www.ibm.com/support/docview.wss?uid=swg1IT15105

RHEL known issues with fprintd-pam module (fingerprint authentication module)

Application Groups and Aggregated Metrics 

For application groups, the performance data is aggregated from the volumes added to the group, either directly or using a filter.  The values displayed for the aggregated Read Response Time, Write Response Time, and Overall Response Time are incorrect.  All other metrics currently supported for application aggregation are rendered correctly.

IBM Spectrum Control might report incorrect vmdk size and incorrect capacity of VM

IBM Spectrum Control persists the relation between a vmdk and vmdisk, restricting the relation to vmdisk <-> current vmdk file. The current vmdk file has an aggregate size which includes the sizes of snapshots up to the current vmdk file. IBM Spectrum Control associates the latest running delta file with vmdisk, hence IBM Spectrum Control might report the size smaller than actual size of the vmdisk if there are newer snapshots (which are not included in the current vmdk size).

Export Performance Data Tool  

The output format from the exportPerformanceData Script has changed in IBM Spectrum Control 5.3.4. To minimize the impact of this and all future output format changes,  users of the CSV files are encouraged to parse the column headers they are interested in.

 

EMC Support Issues

In 5.2.14 and later, you can view more information about the capacity, space usage, and performance of EMC™ storage systems. However, keep in mind the following issues for EMC support:

On the details page for an EMC storage system, duplicate NAS nodes might be displayed if the EMC's NAS Servers support both NFS and CIFS protocols. This issue occurs because each protocol represents a node. For example, on the same NAS server, if nas1451 supports NFS and NAS1451 supports CIFS protocols, two node entries are displayed on the details page for the storage system.

Information about both FC and Serial Attached SCSI (SAS) ports are returned when FC ports are probed.

VNXe:
Thin-provisioned volumes: The value for allocated space is incorrect.
Total or available capacity values of each storage tier in a storage pool are incorrect.

VNX:
Thin pools: The value for physical allocated space is incorrect.
File system pools are not displayed.
Total or available capacity values of each storage tier in a storage pool are incorrect.
The VNX regular thin pool's "pool capacity" value, returned by EMC SMI-S provider, is slightly lower than reported by the element manager.  This applies only to VNX thin pool, referred to as "Pools" in Unisphere.
Disk and volume response times and utilization percentages: The displayed values for these metrics are higher than their actual value.
 
VMAX3:
Available pool space or any allocated capacity values are always 0.
All aggregated volume capacity values are 0 because of the missing relationship between pools and volumes.
Mappings of volume to pool and volume to disk are not displayed.

Reporting: In Cognos Analytics, pre-defined reports might not be available for EMC storage systems.

Cognos Analytics

GPFS/GSS/Spectrum Scale is not supported in Cognos Analytics

GPFS/GSS/Spectrum Scale™ file systems are not correctly represented in Cognos reports and are not supported in Cognos.

Information is not collected about NFS and CIFS exports that are defined on a GPFS/GSS/Spectrum Scale clusters.

Links to Help in IBM Spectrum Control 5.3.4 predefined reports open the public IBM Spectrum Control Knowledge Center

When using the Cognos Analytics User Interface to generate IBM Spectrum Control predefined reports, the Help link will open the IBM Spectrum Control Knowledge Center on ibm.com.  The system running Cognos Analytics will need access to the internet to open to the knowledge center.  Alternatively, you can access the online help in the IBM Spectrum Control GUI and navigate to the help information for reports.

 

Documentation Updates and Corrections

Scripts to start and stop IBM Spectrum Control services on Windows operating system need Administrator authority

Browser Issues

Browser displays a security certificate error 

Some browsers will display: "There is a problem with this website's security certificate."  when you log in to IBM Spectrum Control. To continue to the GUI, click "Continue to this website" and you will be returned to the previous page.  This behavior does not occur on all browsers.