Preventive Service Planning
Abstract
Important information regarding a potential memory leak issue when running applications compiled with the XL C/C++ for AIX 16.1 - xlclang++ compiler on systems with the XL C++ Runtime 17.1.1 or up.
Content
Problem
Applications using C++ built with the xlclang++ compiler from the XL C/C++ for AIX 16.1 product may encounter memory leaks when running on systems that have the 17.1.1 (or higher) libc++.a library installed. The problem occurs in applications which utilize std::string and std::wstring whose lengths are longer than the small string threshold (thus require heap allocation), during several string operations such as copy assignment.
The root cause of the problem is due to a change in the C++ standard’s deallocation requirements on string::reserve, which is used in the implementation of these operations.
Testing with 16.1 (both compiler and runtime) will show no issues as the problem only occurs when the application is run using the 17.1.1 (or greater) runtime. The 17.1.1 C++ runtime was shipped as part of AIX 7.3 TL2 base OS, as well as part of the 17.1.1 and 17.1.2 Open XL compilers, and may be installed as a dependency by applications built using these compilers.
Checking to see if you are affected
The version of the C++ runtime installed on an AIX system can be found either by using `lslpp` to query the installed fileset level and/or using the `what` command directly on `libc++.a`:
$ lslpp -l libc++.rte
Fileset Level State Description
----------------------------------------------------------------------------
Path: /usr/lib/objrepos
libc++.rte 17.1.1.3 COMMITTED IBM XL C++ Runtime for AIX 7.1
and later
$ what /usr/lib/libc++.a
/usr/lib/libc++.a:
Licensed Materials - Property of IBM.
IBM Open XL C/C++ for AIX 17.1.1 (5725-C72, 5765-J18), version 17.1.1.3,
Copyright IBM Corp. 2018, 2023 and by others.
US Government Users Restricted Rights -
Use, duplication or disclosure restricted by
GSA ADP Schedule Contract with IBM Corp.
-----------------------------------------
Version: 17.01.0001.0003
230419 ID:7252368700672fccc98004fad34683295c9b21f8
230419 ID:ac9978bed1f3bb6d07327c07b067598787677eee
Date: Wed Apr 19 14:31:27 EDT 2023
-----------------------------------------
Potentially affected user applications and shared libraries compiled with affected versions of xlclang++ (16.1.0.0-16.1.0.15) can be found by references to the std::basic_string<charType>::reserve :
$ dump -X32_64 -Tv application|grep -E 'basic_string.*reserve'
[14] 0x00000000 undef IMP DS EXTref libc++.a(shr_64.o) _ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7reserveEm
These references may be included indirectly via C++ libraries loaded by applications, so the absence of these references directly in the main application does not definitively rule out the application being affected. Dependent libraries can be scanned in a similar fashion. Note that fixed applications may still have such references, as not all such references are necessarily problematic.
Fix and Workarounds
The C++ string header of the XL C/C++ for AIX 16.1.0.16 xlclang++ compiler has been fixed to inline the required deallocations in the affected operations, avoiding the problematic library call. The fix requires recompiling the affected application using the new header, or at least all files using C++ std::string and std::wstring.
Fix list for XL C/C++ for AIX
https://www.ibm.com/support/pages/fix-list-xl-cc-aix
--------------------------------------------------------------------
Applications using C++ exclusively built with xlclang++ version 16.1.0.15 or lower which cannot be recompiled may optionally work around the issue by downloading a standalone copy of the libc++.a 16.1.0.10 runtime library and loading it during startup using the LIBPATH environment variable.
Fix list for XL C/C++ Runtime for AIX
https://www.ibm.com/support/pages/fix-list-xl-cc-runtime-aix
To extract the 16.1.0.10 libc++.a from the standalone XL C++ Runtime package:
$ uncompress 16.1.0.10-IBM-xlCrte-AIX-FP010.tar.Z
$ tar xf 16.1.0.10-IBM-xlCrte-AIX-FP010.tar
$ restore -qvf libc++.rte.16.1.0.10.bff
...
x 7440196 ./usr/lpp/xlC/lib/aix61/libc++.a
...
$
To run the affected application if the 16.1.0.10 libc++.a runtime was placed in the directory `/opt/mylib`, one could execute:
$ env LIBPATH=/opt/mylib:$LIBPATH /opt/app/bin/my_application
Such a workaround can be included in shell script wrappers for affected executables:
$ cat /opt/app/bin/my_application
#!/usr/bin/ksh
export LIBPATH=/opt/mylib:$LIBPATH
exec /opt/app/bin/my_application.real
$
Further Technical Details
The problem is due to a change in requirements in the C++20 standard. The 16.1 compiler, which is based on clang 4, followed the C++ standard at the time which specified when the memory for std::string could optionally be freed by std::basic_string::reserve(), which the implementation of operations such as copy assignment relied upon. The 17.1.1 runtime, which is based on clang 15, follows the current C++ standard which specifies that under the same conditions the memory for std::string must not be freed by std::basic_string::reserve(), specifically to not free the memory for the string when requesting a reduced size.
The end result is that if object files with long std::strings generated by xlclang++ 16.1 are mixed with the 17.1.1 libc++ runtime, the memory for the string will not be freed during certain operations such as copy assignment. Applications built using the Open XL C/C++ for AIX 17.1.1 or newer compilers may be reliant on the new allocation behavior, so the original std::basic_string::reserve() cannot be restored.
Applications using C++ built with the xlclang++ compiler from the XL C/C++ for AIX 16.1 product may encounter memory leaks when running on systems that have the 17.1.1 (or higher) libc++.a library installed. The problem occurs in applications which utilize std::string and std::wstring whose lengths are longer than the small string threshold (thus require heap allocation), during several string operations such as copy assignment.
The root cause of the problem is due to a change in the C++ standard’s deallocation requirements on string::reserve, which is used in the implementation of these operations.
Testing with 16.1 (both compiler and runtime) will show no issues as the problem only occurs when the application is run using the 17.1.1 (or greater) runtime. The 17.1.1 C++ runtime was shipped as part of AIX 7.3 TL2 base OS, as well as part of the 17.1.1 and 17.1.2 Open XL compilers, and may be installed as a dependency by applications built using these compilers.
Checking to see if you are affected
The version of the C++ runtime installed on an AIX system can be found either by using `lslpp` to query the installed fileset level and/or using the `what` command directly on `libc++.a`:
$ lslpp -l libc++.rte
Fileset Level State Description
----------------------------------------------------------------------------
Path: /usr/lib/objrepos
libc++.rte 17.1.1.3 COMMITTED IBM XL C++ Runtime for AIX 7.1
and later
$ what /usr/lib/libc++.a
/usr/lib/libc++.a:
Licensed Materials - Property of IBM.
IBM Open XL C/C++ for AIX 17.1.1 (5725-C72, 5765-J18), version 17.1.1.3,
Copyright IBM Corp. 2018, 2023 and by others.
US Government Users Restricted Rights -
Use, duplication or disclosure restricted by
GSA ADP Schedule Contract with IBM Corp.
-----------------------------------------
Version: 17.01.0001.0003
230419 ID:7252368700672fccc98004fad34683295c9b21f8
230419 ID:ac9978bed1f3bb6d07327c07b067598787677eee
Date: Wed Apr 19 14:31:27 EDT 2023
-----------------------------------------
Potentially affected user applications and shared libraries compiled with affected versions of xlclang++ (16.1.0.0-16.1.0.15) can be found by references to the std::basic_string<charType>::reserve :
$ dump -X32_64 -Tv application|grep -E 'basic_string.*reserve'
[14] 0x00000000 undef IMP DS EXTref libc++.a(shr_64.o) _ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7reserveEm
These references may be included indirectly via C++ libraries loaded by applications, so the absence of these references directly in the main application does not definitively rule out the application being affected. Dependent libraries can be scanned in a similar fashion. Note that fixed applications may still have such references, as not all such references are necessarily problematic.
Fix and Workarounds
The C++ string header of the XL C/C++ for AIX 16.1.0.16 xlclang++ compiler has been fixed to inline the required deallocations in the affected operations, avoiding the problematic library call. The fix requires recompiling the affected application using the new header, or at least all files using C++ std::string and std::wstring.
Fix list for XL C/C++ for AIX
https://www.ibm.com/support/pages/fix-list-xl-cc-aix
--------------------------------------------------------------------
Applications using C++ exclusively built with xlclang++ version 16.1.0.15 or lower which cannot be recompiled may optionally work around the issue by downloading a standalone copy of the libc++.a 16.1.0.10 runtime library and loading it during startup using the LIBPATH environment variable.
Fix list for XL C/C++ Runtime for AIX
https://www.ibm.com/support/pages/fix-list-xl-cc-runtime-aix
To extract the 16.1.0.10 libc++.a from the standalone XL C++ Runtime package:
$ uncompress 16.1.0.10-IBM-xlCrte-AIX-FP010.tar.Z
$ tar xf 16.1.0.10-IBM-xlCrte-AIX-FP010.tar
$ restore -qvf libc++.rte.16.1.0.10.bff
...
x 7440196 ./usr/lpp/xlC/lib/aix61/libc++.a
...
$
To run the affected application if the 16.1.0.10 libc++.a runtime was placed in the directory `/opt/mylib`, one could execute:
$ env LIBPATH=/opt/mylib:$LIBPATH /opt/app/bin/my_application
Such a workaround can be included in shell script wrappers for affected executables:
$ cat /opt/app/bin/my_application
#!/usr/bin/ksh
export LIBPATH=/opt/mylib:$LIBPATH
exec /opt/app/bin/my_application.real
$
Further Technical Details
The problem is due to a change in requirements in the C++20 standard. The 16.1 compiler, which is based on clang 4, followed the C++ standard at the time which specified when the memory for std::string could optionally be freed by std::basic_string::reserve(), which the implementation of operations such as copy assignment relied upon. The 17.1.1 runtime, which is based on clang 15, follows the current C++ standard which specifies that under the same conditions the memory for std::string must not be freed by std::basic_string::reserve(), specifically to not free the memory for the string when requesting a reduced size.
The end result is that if object files with long std::strings generated by xlclang++ 16.1 are mixed with the 17.1.1 libc++ runtime, the memory for the string will not be freed during certain operations such as copy assignment. Applications built using the Open XL C/C++ for AIX 17.1.1 or newer compilers may be reliant on the new allocation behavior, so the original std::basic_string::reserve() cannot be restored.
Related Information
[{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGH3R","label":"XL C\/C++ for AIX"},"ARM Category":[{"code":"a8m3p000000hBTtAAM","label":"C\/C++-\u003E#Planning"},{"code":"a8m3p000000hBTwAAM","label":"C\/C++-\u003E#Runtime errors"}],"Platform":[{"code":"PF002","label":"AIX"}],"Version":"16.1.0"}]
Was this topic helpful?
Document Information
Modified date:
13 February 2024
UID
ibm17116815