KB15008: ‘Could not spawn process. Failed in fork System Error (ENOMEM): Not enough space’ error message is seen in the DSSErrors.log with MicroStrategy Intelligence Server Universal becoming unresponsive at the same time

SYMPTOM:
Connecting to Strategy Intelligence Server Universal on UNIX/Linux from Strategy Developer fails with an error indicating that the server is unreachable. Users may also face problems logging into or performing any action from Strategy Web.
The DSSErrors.log records the following errors:

2006-11-20 07:14:27.903-05:00 Process Server returns error message: Cannot spawn process.
2006-11-20 07:14:28.022-05:00 Could not spawn process.Cannot spawn process System Error (ESRCH): No such process
2006-11-20 07:15:31.426-05:00 Connection::Connect failed. Unable to connect. Error type: System call error. Could not spawn process.Failed in fork System Error (ENOMEM): Not enough space. Connection String: DSN=EM_STATS;UID=em_stats;.

It has been seen that at this point, Strategy Intelligence Server may become unresponsive to the stop command and has to be terminated. Strategy Developer may also display an error message that the version of Strategy Intelligence Server is incorrect or that the server is unreachable.
CAUSE:
The fork subroutine is used to create new processes. In the above error, the ENOMEM error in the fork subroutine call by the Database connection process is due to not enough memory / space available for the process.
This behavior may most often be seen in machines with small memory configurations.
ACTION:
Ensure that enough paging / swap space is available. Increasing the amount of paging / swap space available should provide a near term solution. Additionally on AIX systems, ensuring that 'early allocation' is not turned on, as detailed in the following document, may also help:
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.prftungd/doc/prftungd/prftungd.pdf
Monitoring memory usage of Strategy Intelligence Server to reduce the memory consumption is one way to fix the problem. The single most effective means of addressing this issue is by setting an appropriate level for MCM to start regulating the Intelligence Server process before the system memory is exhausted. This can be done by setting the Memory Contract Manager to reserve at least 10% of the available memory for other processes (system kernel, M8MulPrc etc). For more information about MCM please see the following Strategy Knowledge base Documents:
KB7295: What is the Memory Contract Manager in MicroStrategy Intelligence Server 9.x and later? PART 1
KB7295: What is the Memory Contract Manager in MicroStrategy Intelligence Server 9.x and later? PART 2
Follow the steps listed below and contact Strategy Technical Support with the information requested if and when issue occurs again:
1.If available, enable NMON logging be run on a regular basis to collect system performance data points, at a frequency of every five minutes as described in the following Strategy Knowledge base document:
 KB14769: How to use NMON and NMON Analyser to obtain detailed system performance data from AIX and Linux operating systems
2. Note the time frame when the issue happens and also the exact symptoms (with screenshots of errors if any)
3.Send the DSSErrors.log and the DSSErrors.log.bak from Intelligence Server machine. Refer to the following Strategy Knowledge base document for the location of the log files:
KB11140: What is the DSSErrors.log, how does it work and where is it located for MicroStrategy Intelligence Server 9.x?
4.Check if Smart heap is enabled or disabled with the steps in the following Strategy Knowledge base document:
KB14662: How to check whether the Smart Heap libraries are enabled for MicroStrategy Intelligence Server Universal 9.x
5.Enable the counters (in parenthesis) for the following categories in the Strategy Performance Monitoring tool.

• Memory (Free paging Space, Free Real Memory, Pinned Memory, reserved paging space, used real memory, Page in, Page out, Total Paging Space, Total Real Memory)
• Strategy Server Jobs (Open Reports, Executing Reports)
• Strategy Server Users (Executing Reports, Open Reports)
• Process (RSS, Size, %CPU Time)
• Processor (% Idle Time, % System Time, % User Time, % Wait Time, Number of Active CPUs, Processor Speed)
• System (RAM Used, Swap Used, Total CPU, Virtual Memory Used)

Refer to the following Strategy Knowledge base document for the information on the Performance Monitoring Tool:
KB13313: How to enable logging of performance counters using the Diagnostics and Performance Logging tool for MicroStrategy Intelligence Server 9.x in Microsoft Windows and Unix
6.The number of files in the /tmp directory that are owned by the user running the Intelligence Server.
Third Party Software Installation:
WARNING:
The third-party product(s) discussed in this technical note is manufactured by vendors independent of Strategy. Strategy makes no warranty, express, implied or otherwise, regarding this product, including its performance or reliability.
KB15008