Troubleshooting
Problem
Scheduled email activities accumulate in 'Waiting' status over time, impacting scheduled reports, jobs, and event agent tasks.
Running the NC drop script and recycling services temporarily clears out the items in 'Waiting' status, but the number of items grows as more emails are sent out.
Symptom
1. In the run history, error codes such as "CNC-SDS-0408 There are email messages waiting to be sent" are found.
2. In a generated thread dump or javacore file, SMTP threads are locked, as indicated by "JMS.Session.QueueSession.SMTP" messages.
Cause
Due to various network connectivity issues an SMTP thread can go into a locked status.
This behavior can occur in environments with a high volume of emails with limitations set on the SMTP mail server side.
By default, 10 SMTP connection threads are generated by the delivery service to send emails.
In the scenario that all 10 threads were to go into a locked status, all further email attempts would fail, simply going into a 'Waiting' status.
Environment
N/A
Diagnosing The Problem
Review the run history status of all activities stuck in 'Waiting' status for "CNC-SDS-0408 There are email messages waiting to be sent" messages.
Next, generate a thread dump or javacore.txt file to be generated during the time when items are stuck in 'Waiting' status.
Refer to the following documentation for generating the thread dump / javacore.txt file:
- https://www.ibm.com/support/pages/creating-system-dump-core-and-javacore-file-using-websphere-libery-profile-server-command
Review the javacore files, searching for "JMS.Session.QueueSession.SMTP", and identify locked threads by confirming the number of threads "waiting to be notified"
The output in the javacore file can look similar to the following output:
Waiting to be notified:
"JMS.Session.QueueSession.SMTP-1"...
"JMS.Session.QueueSession.SMTP-2"...
"JMS.Session.QueueSession.SMTP-3"...
"JMS.Session.QueueSession.SMTP-4"...
"JMS.Session.QueueSession.SMTP-5"...
"JMS.Session.QueueSession.SMTP-6"...
"JMS.Session.QueueSession.SMTP-7"...
"JMS.Session.QueueSession.SMTP-9"...
"JMS.Session.QueueSession.SMTP-10"...
Notice that SMTP-8 is not waiting to be notified.
These log messages indicate that the thread is locked.
Resolving The Problem
In order to ensure that any locked threads do not remain in the locked status, configure the following two advanced settings on the global settings level:
- https://www.ibm.com/docs/en/cognos-analytics/12.0.0?topic=configuration-configuring-advanced-settings-globally
Review the JavaMail API documentation for reference:
- https://javaee.github.io/javamail/docs/api/index.html?com/sun/mail/smtp/package-summary.html
- emf.mail.smtp.connectiontimeout=120000
- mail.smtp.connectiontimeout is the socket connection timeout value in milliseconds.
- This timeout is implemented by java.net.Socket. The default is infinite timeout.
- emf.mail.smtp.timeout=120000
- mail.smtp.timeout is the socket read timeout value in milliseconds.
- This timeout is implemented by java.net.Socket. The default is infinite timeout.
These settings ensure that any locked threads are timed out within 120000 milliseconds or 2 minutes.
Restart the environment for the settings to be applied.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTSF6","label":"IBM Cognos Analytics"},"ARM Category":[{"code":"a8m50000000Cl6sAAC","label":"Install-\u003EConfigure"}],"ARM Case Number":"TS013414777","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
26 October 2023
UID
ibm17054049