KB13610: System settings to check to ensure that core files for the MicroStrategy Intelligence Server process are correctly created by Linux/Unix operating systems

SUMMARY:
In some cases when a process crashes on Linux or Unix operating systems, the core files needed for further investigation may not be automatically generated. This article provides suggested system configuration settings to check or change to ensure core file generation.  The settings provided have been tested to enable core file generation on standard out-of-the-box systems but should be reviewed by a system administrator to make any changes required by the specific environment before applying them.  
In most Linux Distributions core file creation is disabled by default for an average user. Core file generation is controlled by the Operating System, and not by Strategy processes, so difficulties in generating or obtaining core files must be followed up with the system administrator.
Strategy Technical Support recommends that in cases where the size of the dump that will be created is unpredictable, that the limits should be set to unlimited.
 
Linux:
1.-Create a directory where the core files are going to be stored:

# mkdir /path_to_the_location/core

 
2.-To check the current environment configuration for core dump in any shell configuration files for this current user (Zero means core dump was disabled), in a terminal type:

# $ ulimit -c
#  

 
3.-If the value returned in step 2 is "0", please edit /etc/security/limits.conf and add the following line entry at the bottom of the file (before # End of file) which will allow the generation of core files of unlimited size:

*               -   core   unlimited

 
4.-Add this line to /etc/sysconfig/init (if not already there):

DAEMON_COREFILE_LIMIT='unlimited'

 
4.1.-Then to set some sysctl parameters to enable the core file (This must be done as user root) add the following two lines to /etc/sysctl.conf:

kernel.core_pattern = /path_to_the_location/core.%e.%p.%h.%t
fs.suid_dumpable = 2

 Filename variables: 
%e is the filename 
%g is the gid the process was running under 
%p is the PID of the process 
%s is the signal that caused the dump 
%t is the time the dump occurred 
%u is the uid the process was running under
4.2.-To check if changes are there:

# cat /etc/sysctl.conf | grep kernel.core_pattern

 
4.3.-Load the new sysctl settings:

# sysctl -p /etc/sysctl.conf

 
Notes:
-If Strategy Intelligence Server is already running, it may be necessary for the server to be shut down and the user account starting it to be logged off and on again for the new settings to be applied.
-Also, if Strategy Intelligence Server has been started as a service, the entire machine may need to be restarted for these changes to take effect.
-Before starting Strategy Intelligence Server application, verify that the settings have taken effect by running the following command:

# ulimit -a

 
-It is critical to ensure that the disk space available is sufficient for the system dump to be written to it. The system dump is written to the directory specified in step 1. The expected size is in the order of the amount of memory used by the Strategy Intelligence Server process at the time it hits the exception. If the Strategy Intelligence Server folder is loaded using NFS, ensure that large files can be created on the filesystem.
4.4.-To check if the core can be created correctly by forcing a core file to be created with, it is critical to execute this from /IntelligenceServer/bin folder (the location of the MSTRSvr executable).

# gcore -o <file> <pid>

If the core file still does not get generated and you are using RHEL 6.x or superior version, then Operating System requirements to configure the core file generation might change, please proceed with STEP 5.
5.-On particular RHEL 6.x versions and forward, Red Hat Enterprise Linux 6 has a service called abrt, which automatically collects the core dump files on the server and stores them inside the /var/spool/abrt.
5.1.-Assuming you have tried the configuration "sysconfig", please proceed to comment or remove the following parameters from sysctl.conf:

kernel.core_pattern = /path_to_the_location/core.%e.%p.%h.%t
fs.suid_dumpable = 2

5.2.-Then update the /etc/abrt/abrt-action-save-package-data.conf file and set following variables;

OpenGPGCheck = no
ProcessUnpackaged = yes

5.3.-Also add the following parameter to /etc/abrt/abrt.conf:

DumpLocation =/Your_desired/Path_to_generate/the_core_file

DumpLocation = /var/spool/abrt
-This directive is commented out by default.  
-The default location is set to the /var/spool/abrt directory. 
-Whichever directory you specify in this directive, you must ensure that it exists and it is writable for abrtd. 
-If you change the default value of this option, be aware that in order to ensure proper functionality of ABRT, this directory must not be the same as the directory specified for the WatchCrashdumpArchiveDir option.
Then restart the services - as root - for the new settings to take effect;

# service abrtd restart
# service abrt-ccpp start

The following document has more details about like the issues/scenarios addressed;
Why does the abrtd daemon delete recently created application core dumps?
 
5.4.-To check if the core can be created correctly by forcing a core file to be created with (it is critical to execute this from /IntelligenceServer folder and note the location of the core file created):

# gcore -o <file> <pid>

 
6.-Step for when the Intelligence Server is registered as a service
The default user profile may be set to disable the creation of core files especially for processes launched in the init phase. 
To address this (which requires root access), the user should modify the /etc/init.d/mstr-InstallName-iserver-CastorServer init script (InstallName is of the form user@timestampan and can be found in the MSIReg.reg file), and under the line that reads:

start ()
{

Add the following line to allow the Intelligence Server to start as a service with the core files enabled:

ulimit -c unlimited

Important Note: 
If the above settings have been confirmed and core-files are being truncated, users should work with their system administrator in order to review the system message log(/var/log/messages) for details on why the core file generation fails.
REFERENCES:
How to enable core file dumps when an application crashes or segmentation faults
How to collect core dump file of a crashing program that is shipped in Red Hat Enterprise Linux 6/7
Core Dump Management on the Solaris OS
 
Third Party Software Installation:
WARNING:
The third-party product(s) discussed in this technical note is manufactured by vendors independent of Strategy. Strategy makes no warranty, express, implied or otherwise, regarding this product, including its performance or reliability. 

KB13610