Fixes are available
APAR status
Closed as program error.
Error description
There is no special error in the logs, however, the polling method assumes uniqueness of its operation and might have strange applications results if run twice simulatneously. For example, a user may receive duplicate push notifications should that occur. Polling is used widely in Worklight for push notifications. For example, a user may write an adapter with the following JS code: WL.Server.createEventSource({ name: 'Source2', poll: { interval: 40, // seconds onPoll: 'getNotificationFromBackend' } }); function getNotificationFromBackend() { WL.Logger.info('sending notifications now'); // will iterate here over a list of users and will send a notification to each user... }
Local fix
Reduce the number of active Worklight servers to one.
Problem summary
**************************************************************** * USERS AFFECTED: * * Worklight servers which are configured as a cluster or farm * * such that there are several Worklight servers sharing * * identical configuration (database,Worklight WAR etc). If * * only one server is used for Worklight this problem will not * * occur. * **************************************************************** * PROBLEM DESCRIPTION: * * A race condition may occur when two or more Worklight * * servers are processing a Worklight event source. This race * * condition was not handled well, and, as a result, the event * * source may be fired simultaneously in more than one server. * * This can have many side effects when using EventSource for * * mass push notifications, including high CPU utilization and * * significant Worklight database activity. * * The other side effect with push and event sources is that * * many mobile users may receive multiple notifications * * (depending upon the cluster size and the race condition) on * * their mobile devices. Each time the user opens the push * * notification, it will trigger a login process which will * * probably add extra traffic to the Worklight servers (which * * already may be in a high CPU utilization state). * * An example for a Worklight adapter that may trigger this * * race condition is the following adapter Java script code: * * * * WL.Server.createEventSource({ * * name: 'Source2', * * poll: { * * interval: 40, // seconds * * onPoll: 'getUsersFromBackend' * * } * * }); * * * * * * function getUsersFromBackend() { * * WL.Logger.info(' getUsersFromBackend'); * * // here the code will probably iterate through a list of * * users and will send each one a notification * * // or some other long operation. * * // if you notice the INFO log line in more than one * * server : then you have hit the race condition. * * } * **************************************************************** * RECOMMENDATION: * * - * ****************************************************************
Problem conclusion
A cluster syncronization issue causing an event source method to be run simultaneously on multiple servers has been corrected. By design, an event source's poll methods are meant to be run only in one server within a cluster.
Temporary fix
A temporary fix can be to shutdown all Worklight servers in the cluster except for one. However, a single server may not hold the standard traffic (depending on the total mobile network throughput.) In some cases the way to recover from the race condition is to reboot the Worklight servers.
Comments
APAR Information
APAR number
PI44032
Reported component name
MFPF/WORKLIGHT
Reported component ID
5725I4301
Reported release
506
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2015-06-29
Closed date
2015-06-30
Last modified date
2015-07-02
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
MFPF/WORKLIGHT
Fixed component ID
5725I4301
Applicable component levels
R506 PSY
UP
[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSZH4A","label":"IBM Worklight"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"506","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
15 October 2021