Troubleshooting
Problem
When adding a gateway server to a gateway cluster, it goes into "errored" state and the following message can be seen in the API Connect cmc.out log file:
- GATEWAY ADDITION: Error while adding gateway server to gateway cluster in a background thread. Gateway server ID: <gw server ID>; gateway cluster ID: <gw cluster ID>
Diagnosing The Problem
Problem - Scenario 1:
API Connect cmc.out log shows the following errors when trying to add a gateway server to a cluster:
WARNING [T-134584] [com.ibm.datapower.gwapi.soma.Response.checkForOK] SOMA checkForOK failed
==Response:500==
...
SEVERE [T-134584] [com.ibm.datapower.gwapi.impl.GatewaysImpl.deploy] Deploy: APIMgmt_0123456789F/gw1: failed: SomaException:set-system-var failed
Throwable occurred: com.ibm.datapower.gwapi.soma.SomaException: set-system-var failed
...
[T-134584] [com.ibm.apimgmt.gateway.DatapowerConfigurationHelper.handleGwApiException] Error deploying domain for <domain.com>
[T-134584] [com.ibm.apimgmt.resources.topology.runnables.GatewayAdditionThreadRunnable.run] **** GATEWAY ADDITION: Error while adding gateway server to gateway cluster in a background thread. Gateway server ID: <gw server ID>; gateway cluster ID: <gw cluster ID>
2018-08-04 00:01:19.837 SEVERE [T-134584] [com.ibm.apimgmt.exception.APIGenericException$Serializer.serialize] Error(5b64eccfe4b0d5b038bc2742), Http-Code(500), Message(An internal error occured when deploying the gateway named <gateway name> and identified by <endpoint url> on the server APIMgmt_0123456789/gw1. The exception message is: set-system-var failed. Please see the server log, or contact administrator for more details.), User(null/SYSTEM), Path(get:null)
Throwable occurred: com.ibm.apimgmt.exception.APIGenericException
...
T-134584] [com.ibm.apimgmt.exception.APIGenericException$Serializer.serialize] Error(5b64eccfe4b0d5b038bc2745), Http-Code(500), Message(Server is not added to cluster <gw cluster ID>. Reason null.), User(null/SYSTEM), Path(get:null)
Throwable occurred: com.ibm.apimgmt.exception.APIGenericException
Problem - Scenario 2:
A gateway sever was previously removed from a gateway cluster. Upon trying to add the gateway server back into the cluster, the following errors are seen because the domain for the original server was never deleted in DataPower:
API Connect cmc.out:
2018-08-20 18:32:58.639 SEVERE [T-2440453] [com.ibm.apimgmt.gateway.DatapowerConfigurationHelper.deployDomain] Error deploying domain for <client gateway server mame>
2018-08-20 18:32:58.640 SEVERE [T-2440453] [com.ibm.apimgmt.resources.topology.runnables.GatewayAdditionThreadRunnable.run] **** GATEWAY ADDITION: Error while adding gateway server to gateway cluster in a background thread. Gateway server ID: <gw server ID>; gateway cluster ID: <gw cluster ID>
2018-08-20 18:32:58.642 SEVERE [T-2440453] [com.ibm.apimgmt.exception.APIGenericException$Serializer.serialize] Error(5b7b095ae4b0c380ac28b962), Http-Code(500), Message(Cannot add Gateway Server <client gateway server mame> as its information already matches a server that belongs to this same Gateway Cluster. Domain APIMgmt_0123456789 already exists in DataPower.), User(null/SYSTEM), Path(get:null)
Throwable occurred: com.ibm.apimgmt.exception.APIGenericException
...
DataPower gateway error-report:
20180820T182922.823Z [0x81000013][cli][error] : tid(4721): Cannot be deleted. In use by: snmp(SNMP-Settings)
20180820T182922.823Z [0x81000013][mgmt][error] domain(APIMgmt_APIMgmt_0123456789): tid(4721): Cannot be deleted. In use by: snmp(SNMP-Settings)
Resolving The Problem
Resolution - Scenario 1:
This problem is caused by SOAP not being enabled for the DataPower XML Management Interface. It can be resolved by:
- Within the default domain of DataPower webgui, navigate to Objects->Device Management-> XML Management Interface and enable the SOAP services by placing a check in each box.
- Apply the change and save the configuration.
- Within the API Connect Cloud Management Console, add the gateway server to the gateway cluster.
Resolution - Scenario 2:
This problem can be caused by manual configuration performed after the gateway has been deployed and which references the APIMgmt_xxxxxxxxx domain. In order to resolve the issue:
1. Use the DataPower cli or webgui to remove the any references to the domain as highlighted by the information in the error-report message that includes the string "Cannot be deleted. In use by:"
2. Manually delete the domain in DataPower
3. Within the API Connect Cloud Management Console, add the gateway server to the gateway cluster.
Was this topic helpful?
Document Information
Modified date:
15 November 2018
UID
ibm10740697