IT41460: 8.1.15 DATA MOVER CRASHES DURING STARTUP WITH "DOUBLE FREE OR CORRUPTION" OR "FREE(): DOUBLE FREE DETECTED IN TCACHE 2"

APAR status

• Closed as program error.

Error description

• IBM Spectrum Protect Backup-Archive Client version 8.1.15
  running on Linux operating systems crashes during startup if
  the client is configured for automatic failover to a
  replication server.
  
  One of the following messages might accompany the crash:
  
   - free(): double free detected in tcache 2
     Aborted (core dumped)
  
   - *** Error in `xxxx': double free or corruption (!prev):
     nxnnnnnnnnnnnnnnnn ***
     ======= Backtrace: =========
     ...
     ======= Memory map: ========
     ...
  
     where 'xxxx' is the module that crashed, such as dsmc, and
     nxnnnnnnnnnnnnnnnn is a hexadecimal number
  
  The dsmerror.log file might include this message:
  
  <date> <time> ANS0361I DIAG: main thread, fatal error, signal 6
  
  The crash can occur in the command line client, the remote
  agent, or the API. This means that IBM Spectrum Protect Data
  Protection products and other API applications can also be
  affected by this problem.
  
  Additional diagostics:
  Use the GNU Debugger (gdb) to obtain a backtrace from the core
  file:
  
   gdb /opt/tivoli/tsm/client/ba/bin/dsmc core.nnnnn
  
   where core.nnnn is the core file.
  
   At the "(gdb)" prompt type this command to see the backtrace:
  
    bt
  
   Then type this command to exit from gdb:
  
    q
  
  Example backtrace (the hexadecimal numbers will differ).
  
   #0  0x00007f04e2c06387 in raise () from /lib64/libc.so.6
   #1  0x00007f04e2c07a78 in abort () from /lib64/libc.so.6
   #2  0x0000000000712a74 in psTrapHandler(int) ()
   #3  <signal handler called>
   #4  0x00007f04e2c06387 in raise () from /lib64/libc.so.6
   #5  0x00007f04e2c07a78 in abort () from /lib64/libc.so.6
   #6  0x00007f04e2c48ed7 in __libc_message () from
                          /lib64/libc.so.6
   #7  0x00007f04e2c51299 in _int_free () from /lib64/libc.so.6
   #8  0x00007f04e2c3e1b7 in fclose@@GLIBC_2.2.5 () from
                          /lib64/libc.so.6
   #9  0x00000000006efe2f in clientOptions::optSaveReplConnInfo()
                          ()
   #10 0x000000000068103f in cuSignOnEResp(Sess_o*) ()
   #11 0x0000000000652b75 in scSignOnTheSession(Sess_o*) ()
   #12 0x0000000000658ce3 in NegotiateSession(Sess_o*) ()
   #13 0x0000000000653648 in OpenSess(Sess_o*, bool) ()
   #14 0x0000000000654ad0 in Logon(Sess_o*) ()
   #15 0x0000000000656c36 in CheckSession(Sess_o*,
                          sessLoadPolicy_t) ()
   #16 0x000000000043e5fe in dscInit(int, char**, cliType_t) ()
   #17 0x000000000043eb5b in dscmain(int, char**) ()
   #18 0x000000000043b3fa in main ()
  
  If a core file is not created, check the ulimit setting for
  core files (ulimit -c). If the ulimit is set to a low value or
  0, then temporarily change it to unlimited, reproduce the
  problem to obtain the core file, then change the ulimit to it's
  original value.
  
  Alternatively, see these instructions to collect core file
  information:
  
  https://www.ibm.com/support/pages/collecting-data-spectrum-prote
  ct-client-crash-linux
  
  
  IBM Spectrum Protect Versions Affected:
  IBM Spectrum Protect Data Protection for VMware 8.1.15 on Linux
  
  | MDVREGR 8.1.15.0-TIV_5698ISMCL | IT40861
  
  Additional Keywords:
  TS009849950 crash tcache free IT40861
  

Local fix

• Set option
  USEREPLICATIONFailover NO
  under server stanzas in dsm.sys file
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Data Protection for VMware version 8.1.15 running on Linux   *
  * x86                                                          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is       *
  * projected to be fixed in level 8.1.15.1 and 8.1.16. Note     *
  * that until the fixing level is actually available, this      *
  * information is subject to change at the discretion of IBM.   *
  ****************************************************************
  

Problem conclusion

• The double close error is fixed so that the crash no longer
  occurs.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IT41452

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM FOR VE DP V

• Fixed component ID

  5725TVEVM

Applicable component levels