KB441998: How to configure data file retention policy for MicroStrategy Messaging Services (Kafka) 10.5 and higher versions

Note: Starting in Strategy 11.0, the behavior has been changed and the Intelligence Server does not Messaging Services by default. For more details, see KB442160.
Beginning in Strategy 10.5 (on Linux) and 10.6 (on Windows), Strategy Messaging Services uses the third-party Kafka framework to handle the DSSErrors logging output from the Intelligence Server process and other processes, such as the PDF Export Engine.  Platform Analytics, a new product in Strategy 11.0, also utilizes the Kafka framework to handle logging of statistical data.  While the data handled by Kafka will typically be consumed and output into various logfiles and the Platform Analytics warehouse, it is also saved onto the filesystem for temporary storage in an OS-dependent location.  For example, in Strategy 11.0 installed on Linux the default path is:

/opt/MicroStrategy/MessagingServices/Kafka/tmp/kafka-logs

On Windows, the default storage path is:

C:\Program Files (x86)\Strategy\Messaging Services\tmp\kafka-logs

This article discusses how to configure the retention policy for this temporary storage.
The Kafka service has a configuration file called server.properties where the retention settings (and other settings) are stored.  This file is located in the MessagingServices/Kafka/kafka_<version_number>/config/ subdirectory of the Strategy install path.  For example, in Strategy 11.0 installed on Linux the default path is:

/opt/MicroStrategy/MessagingServices/Kafka/kafka_2.11-1.1.0/config/server.properties

On Windows, the default path is:

C:\Program Files (x86)\Strategy\Messaging Services\Kafka\kafka_2.11-1.1.0\config\server.properties

By default, the log retention policy section of the server.properties file will contain the following:

############################# Log Retention Policy #############################

# The following configurations control the disposal of log segments. The policy can
# be set to delete segments after a period of time, or after a given size has accumulated.
# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
# from the end of the log.

# The minimum age of a log file to be eligible for deletion
log.retention.hours=168

# A size-based retention policy for logs. Segments are pruned from the log as long as the remaining
# segments don't drop below log.retention.bytes.
#log.retention.bytes=1073741824

The default setting (log.retention.hours=168) will automatically delete all data files older than 168 hours (one week).  This time can be adjusted up or down as needed.  Strategy recommends a value no lower than 24 hours.
The data files can also be deleted according to size, though this setting is not enabled by default.  To enable the setting, remove the # character from the beginning of the (log.retention.bytes) parameter, and set an appropriate value.  The default value is approximately 1GB.  When enabled, this setting will automatically delete data files when the total size of all data files exceeds 1073741824 bytes, beginning with the oldest files first.
It is also possible to configure the storage location for the Kafka data files, using the log.dirs parameter of the server.properties file:

############################# Log Basics #############################

# A comma separated list of directories under which to store log files
log.dirs=/opt/MicroStrategy/MessagingServices/Kafka/tmp/kafka-logs

Note that when configuring the storage location on Windows, directories in the path must be separated by two backslashes instead of one:

############################# Log Basics #############################

# A comma seperated list of directories under which to store log files
log.dirs=C:\\Program Files (x86)\\Strategy\\Messaging Services\\tmp\\kafka-logs

After making any changes to the server.properties file, the Kafka service will need to be restarted for the changes to take effect.  It is not recommended to make changes to any other settings in the server.properties file unless specifically directed to by Strategy Technical Support.