IBM Support

Out of virtual memory space due to inefficiency of default memory allocation policy on Windows XP and 2003

Troubleshooting


Problem

For some models, 32-bit version of CPLEX may report out-of-memory errors on Windows XP and 2003, although task manager reports low memory usage. This is because the default memory allocation policy of Windows XP and 2003 can waste a lot of virtual memory space.

Environment

32-bit Windows XP and 2003 only.

Diagnosing The Problem

To confirm this symptom, please follow these steps:

1. download Process Explorer from


http://technet.microsoft.com/en-us/sysinternals/bb896653
2. Export the model from your application and solve it in interactive optimizer. Interactive optimizer has the minimal overhead, so its memory metrics are most accurate. If you try to read the memory metrics of your applications (especially those with another memory manager, such as JVM or .NET CLR), the readings could be misleading.
3. When interactive optimizer runs out of memory, please read the metrics of "Peak Working Set" and "Virtual Size" for this process.

If the value of "Virtual Size" is close to 2GB but the "Peak Working Set" is less than 500MB, it is very likely that you are being impacted by the inefficiency of the default memory allocation policy of Windows XP and 2003.

Resolving The Problem

Low-fragmentation Heap (LFH) policy can improve the situation. This policy is already available in Windows XP and 2003, but not activated by default. You can find more information here:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366750%28v=vs.85%29.aspx

You can activate LFH policy manually. The attached is a sample project for Visual Studio 2008. The EnableLFH() routine in the LFHHelper.cpp file shows how to enable LFH policy in C/C++ programs. The project compiles the code into a .dll file, which can be invoked by other programming languages. In the archive, there are examples showing how to activate LFH policy in C# and Java. You can find more information in the file read.me in the archive.

Note that LFH policy cannot be enabled on some computers, although its failure should not crash your application. Specifically, LFH policy cannot be enabled for heaps created with HEAP_NO_SERIALIZE or for heaps created with a fixed size. The LFH cannot be enabled if you are using the heap debugging tools, either. To enable LFH policy when running under a debugger, set the _NO_DEBUG_HEAP environment variable to 1. You can find more information at the links below:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366705%28v=vs.85%29.aspx
http://support.microsoft.com/kb/929136

Please send anonymous feedback if you find bugs or request samples on how to invoke it from other programming languages. Entitled customers should open a PMR to request further assistance.

LFHHelper.zipLFHHelper.zip

[{"Product":{"code":"SSSA5P","label":"IBM ILOG CPLEX Optimization Studio"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"General","Platform":[{"code":"PF033","label":"Windows"}],"Version":"12.4;12.3;12.2.0.1;12.2","Edition":"Edition Independent","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
16 June 2018

UID

swg21594086