APAR status
Closed as program error.
Error description
Backup to AZURE using a bandwidth of less than 5 MB/s have frequent copy to cloud failures. JOBLOG shows this error: ERROR,..,<<date>>,2,CTGGA0309,Copy failed for snapshot (ID: 1419) from source [server: xxx volume: spp_1028_xxx snapshot: spp_1028_2365_xxx] to target [server: https://core.windows.net:443 volume: xxx]. Error: TransferError: Transfer failed: The data transfer was cancelled because the upload of data to the target cloud server or repository server did not make progress. Ensure that the vSnap server has adequate connectivity to the cloud server or repository server. Currently, the default concurrency preferences for cloud offload start to break down with low throttle values (5 MB/s or less). A cloud offload job with a low throttle value will fail. IBM Spectrum Protect Plus Versions Affected: IBM Spectrum Protect Plus 10.1.3.0 and later Additional Keywords: SPP, SPPlus, TS012548138, Storage Protect Plus, AZURE, throttle
Local fix
For vSnaps with a low bandwidth throttle (5 MB/s) where cloud offload jobs are timing out, the following commands can be run on the vSnap to reduce the concurrency of the vSnap to ensure that the timeouts no longer occur: vsnap system pref set --name cloudOffloadThreads --value 1 vsnap system pref set --name cloudMaxStreams --value 1 vsnap system pref set --name cloudStallCount --value 60 vsnap system pref set --name cloudStallInterval --value 60 Note that setting these preferences may cause a job to take longer, but the job will no longer timeout.
Problem summary
**************************************************************** * USERS AFFECTED: * * IBM Spectrum Protect Plus level 10.1.3 till 10.1.16 * **************************************************************** * PROBLEM DESCRIPTION: * * See Error Description * **************************************************************** * RECOMMENDATION: * * Apply the fixing level when available. This problem is * * projected to be fixed in IBM Spectrum Protect Plus level * * 10.1.17. This is subject to change at the discretion of IBM. * ****************************************************************
Problem conclusion
Before IBM Spectrum Protect Plus 10.1.17, cloud offloads with a source vSnap configured with a low bandwidth throttle (such as 1 MB/s or 2 MB/s) time out (causing the job to fail). For low bandwidth throttles, the default level of concurrency on the vSnap would cause requests from the vSnap to the cloud endpoint to starve one another, possibly resulting in a timeout. In IBM Spectrum Protect Plus 10.1.17 and later, the default concurrency for cloud offload jobs on the vSnap is scaled when there is a bandwidth throttle configured for the vSnap. This results in the appropriate level of concurrency for throttled cloud offload jobs (especially with low throttles), ensuring that the offload job does not time out.
Temporary fix
Comments
APAR Information
APAR number
IT45517
Reported component name
SP PLUS
Reported component ID
5737SPLUS
Reported release
A1B
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2024-02-19
Closed date
2024-05-20
Last modified date
2024-05-20
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
SP PLUS
Fixed component ID
5737SPLUS
Applicable component levels
[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSNQFQ","label":"IBM Spectrum Protect Plus"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"A1B","Line of Business":{"code":"LOB69","label":"Storage TPS"}}]
Document Information
Modified date:
20 May 2024