APAR status
Closed as program error.
Error description
Error 502 bad gateway when navigating the API Manager UI The problem was caused by some cloud webhooks with extremely large state_change_history - up to 200,000 state change entries - containing many duplicate events (same id and timestamp). The webhooks filter field was also very large with many duplicate values. Performance improved dramatically after cleaning up these tables and other database tables. The following changes were made in v10 to prevent this happening again: 1. Make webhook.state_change_history read-only 2. Remove duplicates from filter and events fields in requests to create or update webhooks. 3. Before adding a new event to webhook state_change_history, remove the oldest events from the state_change_history, keeping only the most recent 20 events. This is instead of removing the 1 oldest event and it is a safeguard to clean up the history size if it has exceeded the limit especially in existing deployments where the state_change_history could be updated directly via APIM API. The changes made in v2018 are slightly different, to reduce the risk of breaking tools that update webhook state_change_history: 1. Truncate the state_change_history to the last 21 events in requests to update a webhook 2. Remove duplicates from filter and events fields in requests to create or update webhooks. 3. Before adding a new event to webhook state_change_history, remove the oldest events from the state_change_history, keeping only the most recent 20 events. This is instead of removing the 1 oldest event and it is a safeguard to clean up the history size if it has exceeded the limit especially in existing deployments where the state_change_history could be updated directly via APIM API. Changes (2) and (3) are the same in v2018 and v10; only change (1) is different. RESOLUTION | Release Stream | Date | Pull Request | | master | 14 July 2021 | velox/apim#12298 | | v2018.4.1.18 | 16 Aug 2021 | <a href="https://github.ibm.com/velox/apim/pull/12312">velox/apim#1 2312</a> | | v10.0.1.5 | 19 Aug 2021 | <a href="https://github.ibm.com/velox/apim/pull/12399">velox/apim#1 2399</a> |
Local fix
Problem summary
Error 502 bad gateway when navigating the API Manager UI
Problem conclusion
The problem was caused by some cloud webhooks with extremely large state_change_history - up to 200,000 state change entries - containing many duplicate events (same id and timestamp). The webhooks filter field was also very large with many duplicate values. Performance improved dramatically after cleaning up these and other database tables. We made <span style="background-color:rgb(255, 255, 255)">state_change_history field read-only and made sure it can't grow beyond 20. This helped us fix the issue where we had unbounded state change entries.</span> The fix was made in v10.0.4, v10.0.1.5 and v2018 fp18.
Temporary fix
Comments
APAR Information
APAR number
LI82318
Reported component name
API CONNECT ENT
Reported component ID
5725Z2201
Reported release
18X
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2021-09-14
Closed date
2022-01-10
Last modified date
2022-01-10
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
API CONNECT ENT
Fixed component ID
5725Z2201
Applicable component levels
[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMNED","label":"IBM API Connect"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"18X"}]
Document Information
Modified date:
11 January 2022