KB19843: How is the execution time calculated when selecting performance cycles in MicroStrategy Integrity Manager?

In Strategy Integrity Manager the user can run performance cycles to determine how long it takes Strategy Intelligence Server to execute a given set of reports or documents.
When using performance cycles to compare execution times, the same comparison formats (SQL, Data, Graph and Excel) must be selected for the reports and documents in the test in order to get more accurate results. When a test is executed with more than one format selected, the execution time is not recorded for each format; instead it shows the aggregation of the selected formats of the test.
The following example shows the differences between execution times depending on the formats selected for testing. The test was executed with the following considerations:
 

• Single project test.
• Caches were invalidated for the report.
• Two performance cycles were selected.

SQL comparison
 

• Only SQL generation is being considered for the execution time.
• Avg = 00:00:00.437 sec.

Data and SQL comparison
 

• Both SQL generation and report execution are considered to calculate the execution time.
• Avg = 00:00:01.062 sec.

Data, SQL and Excel comparisons
 

• All SQL generation, report execution and report export times are considered to calculate the execution time.
• Avg = 00:00:01.445 sec.

From these results, it is evident that comparing a SQL comparison test with a Data plus SQL comparison test (or any other combination between different formats) is likely to give inaccurate results.