LO90491: CLUSTER REPLICATOR HANGS

APAR status

• Closed as program error.

Error description

• Problem:
  cluster replicator hangs
  
  Environment:
  9.0.1FP7 August  17, 2016 (64-bit) on
  Red Hat Enterprise Linux Server release 7.2 (Maipo)
  
  Troubleshooting:
  
  (1)
  nsd_Linux_Servername_2016_10_06@09_45_36.log:
  
  Script started at: Thu Oct  6 09:45:37 CEST 2016
  Script ended   at: Thu Oct  6 09:56:30 CEST 2016
  
  semdebug.txt is filled with cluster replicator (SEM 0x1619 )
  semaphoring from zero pid owners :
  06/10/2016 09:45:38 CEDT sq="000D2625" THREAD
  [55753:00002-00007FFA979E5740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007FFA89459B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:45:38 CEDT sq="000D2626" THREAD
  [55752:00002-00007F3DA3888740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007F3D953D9B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:45:39 CEDT sq="000D2627" THREAD
  [55754:00002-00007FF722D50740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007FF713459B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:45:39 CEDT sq="000D262A" THREAD
  [55751:00002-00007FA5486FD740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007FA539459B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:45:40 CEDT sq="000D262B" THREAD
  [55729:00002-00007F3B8E031740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007F3B7F459B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:46:08 CEDT sq="000D2644" THREAD
  [55753:00002-00007FFA979E5740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007FFA89459B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  06/10/2016 09:46:08 CEDT sq="000D2645" THREAD
  [55752:00002-00007F3DA3888740] WAITING FOR SEM 0x1619 CLREPL
  Task Static data semaphore (@00007F3D953D9B88)
  (OWNER=00000:0000000000000000) FOR 30000 ms
  
  (2) same nsd_Linux_Servername_2016_10_06@09_45_36.log shows
  however
  cluster replicator idle at the same time semaphoring occurs:
  
  306546: Server.Task = Cluster Replicator: Idle: [10/04/2016
  20:19:05 CEDT]
  306547: Server.Task = Cluster Replicator: Idle: [10/04/2016
  20:19:05 CEDT]
  306548: Server.Task = Cluster Replicator: Idle: [10/04/2016
  20:19:05 CEDT]
  306549: Server.Task = Cluster Replicator: Idle: [10/04/2016
  20:19:05 CEDT]
  306550: Server.Task = Cluster Replicator: Idle: [10/04/2016
  20:19:05 CEDT]
  
  (3) console.log doesn't have  long held locks
  (4) customer has  RTR_Logging=63 in notes.ini ,
  however cannot see replication events on console.log
  during time when nsd above was taken, although
  cluster replicator semaphoring was going on at the same time
  per above.
  
  
  (5)
  
  According to customer this issue occurs only on
  901FP7 servers.
  "
  Not seen with 901FP6IF3
  
  When restarting server everything seems to be fine. After a
  while
  RTR_Logging=63 produces
  
  no output in console.log and in this situation the task clrepl
  do not
  accept any command
  
  e.g. tell clrepl dump
  
   "
  

Local fix

Problem summary

• A programming error was found and will be corrected in a future
   release.
  

Problem conclusion

• A programming error was found and will be corrected in a future
   release.
  

Temporary fix

Comments

• This APAR is associated with SPR# KBRNAEMPX2.
  A programming error was found and will be corrected in a future
   release.
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DOMINO SERVER

• Fixed component ID

  5724E6200

Applicable component levels

• R901 PSN

     UP