KB34084: Troubleshooting MicroStrategy Intelligence Server Crashes

What to do when the Strategy Intelligence Server on Unix Crashes

When the Strategy Intelligence Server Universal crashes, the operating system detects the crash and begins to create a core file. The core file size will typically match the amount of memory being used by the Intelligence Server process at the time of the crash. In  most situations, a core file is automatically generated provided there is enough space for the core file to be created. If there is not enough disk space, the core file will still be written, but will be truncated or incomplete.
 
A core may also be truncated if the operating system is not able to fully write the memory to disk, or if an error occurs during the writing process, which is outside Strategy's control. If a core is truncated, Strategy cannot read it to tell the cause of the Intelligence Server crash.
 
To ensure a complete and valid core, proper procedures must be followed as outlined in the following Strategy Knowledge Base article: KB13610: System settings to check to ensure that core files for the MicroStrategy Intelligence Server Universal process are correctly created by the Unix/Linux operating system platform
To find the cause of the Intelligence Server crash, the following must be collected and submitted to Strategy Technical Support for further troubleshooting:

• Readcore output (see section below).
• Core file (if excessively large, this step can be postponed until readcore has been reviewed. To save space, the file should be compressed).
• DSSErrors.log and DSSErrors.log.bak (from all machines if in a cluster).
• mstr.hist file (located in the log directory) for the exact Strategy Intelligence Server version.
• File information about the Strategy installation:
  
  
  Linux
  ls  -Rlta  /opt/MicroStrategy/  >  ~/mstr_snapshot.txt     #“/opt/Strategy” is where Strategy is installed and can be changed
   
  Windows
  dir  "C:\Program Files (x86)\Common Files\Strategy"  "C:\Program Files (x86)\Strategy"  /s /a:-d /o:-d  >  "C:\Users\Administrator\Desktop\mstr_snapshot.txt”
   
  The ATTACHED are examples. 
   
  
• The output of the following commands get the exact operating system version:
  
• RedHat Enterprise Linux
  
  
  

  
  uname -a
  cat /etc/redhat-release

  
  
  
  
• SuSE Linux Enterprise Server
  
  
  
  

  
  uname -a
  cat /etc/SuSE-release

  
  
  
  
  
• AIX
  
  
  

  
  uname -a
  oslevel -g

  
  
  
  
• SunOS
  
  

  
  uname -a

  
  
  
• HP-UX
  
  

  
  uname -a

  
  
   
  

Readcore

It can take a long time to bring an extremely large core file from the customer site to in-house for troubleshooting. If this core is truncated, this effort is wasted. To save time it is useful to check whether the core generated is valid to be sent in to Strategy Technical Support for further investigation.
These scripts are designed to perform the following two tasks:

• Check if a core is truncated
• If the core is not truncated, then the script obtains debug information using a native debugger. This can be used to determine if crash is due to a known issue without requiring the full core file to be sent in house.

Notes:

1. You do not need to download the file below if you are using Strategy 2019 or later. The scripts in the file are shipped with Strategy 2019 and are installed in the bin folder.
2. The download link for these scripts is at the bottom of this page. These scripts are only for AIX and Linux platforms and are included in the zip files Readcore_AIX and Readcore_Linux. Use the one appropriate for your operating system.
3. To run the script, the customer needs to have the appropriate debugger installed on the Intelligence Server machine (gdb on Linux; dbx on AIX).
4. Copy (readcore and core_analyzer) to a specific location and give the files execute permissions using command "chmod +x readcore core_analyzer".
5. The script can be executed using the command below.  Take note that the MSTRSvr.x.x.xxx.xxx file in the <MSTR_INSTALL>/IntelligenceServer/bin folder is the actual executable binary, while the mstrsvr.x.xxx.xxx file in the <MSTR_HOME>/bin is the launcher script used to start Intelligence Server.  The readcore script needs to be pointed at the executable binary, not the launcher script.
6. readcore MSTRSvr.X.X.XXX.XXX <core_file> where X.X.XXX.XXX corresponds to the Strategy build number.

For example (based on default install paths and default core generation path): 

./readcore /opt/MicroStrategy/IntelligenceServer/bin/MSTRSvr.9.0.307.174 /var/opt/MicroStrategy/IntelligenceServer/core.5902

Readcore can also attempt to determine if the crash was triggered during execution of a specific report or document.  To activate this option, add "-o" when running the script, i.e.:

./readcore -o /opt/MicroStrategy/IntelligenceServer/bin/MSTRSvr.9.0.307.174 /var/opt/MicroStrategy/IntelligenceServer/core.5902

If readcore can identify a specific report or document, it will do so and print the info to the screen as well as adding it to the coreinfo output file.  If no such frame is found, a message will be printed accordingly.  Note that even if a specific report or document was running that the time of the crash, it is not necessarily the case that the report or document caused the crash.
 

What to do when Strategy Intelligence Server on Windows crashes?

When Strategy Intelligence Server crashes on a Windows platform, users can configure the collection of crash dumps using Windows Error Reporting (WER).  For details on WER, refer to the following Strategy Knowledge Base article KB33429: How to use the Windows Server 2008 Windows Error Reporting (WER) feature with MicroStrategy Intelligence Server
In Windows, there is no such utility such as the readcore script. However, in the case of a huge core file the following steps can be taken to product a "minidump". This minidump will not contain heap data but should contain the crashing callstack which can be used similar to readcore to determine if the crash is due to a previously known issue without requiring the full core file. For more information, see KB297913: How to create a minidump from a MicroStrategy Intelligence Server crash dump file on Windows operating systems
To find the cause of the Intelligence Server crash, the following must be collected and submitted to Strategy Technical Support for further troubleshooting:

1. The core file itself - full WER dump or Minidump.
2. DSSErrors.log and DSSErrors.log.bak (from all machines if in a cluster)
3. mstr.hist file (C:\Program Files (x86)\Common Files\Strategy) for the exact Strategy Intelligence Server version.
4. Operating System version.

 

What to expect once you have submitted all the above files to Strategy Technical Support?

Once Strategy Technical Support receives the above files, the core is extracted (if it is zipped) and loaded in the debugger to be analyzed and cause of the crash determined. 
Depending on the complexity of the issue, this process can take 1-2 business days.