KB442072: Text columns in the parquet file created by Impala are loaded as binaries

SYMPTOM:
 
In Strategy 10.x and 11.x, text columns in the parquet file created by Impala are loaded as binaries.
 
Refer to Technical Document below on Hadoop Gateway advanced logging with log4j.properties.
KB39553: How to enable the advanced logging to troubleshoot Hadoop Gateway issues in MicroStrategy 10.6 and later?
 
Refer to Technical Document below on Strategy Hadoop Gateway deployment:
KB248914: Overview and installation of the MicroStrategy 10.0 - 10.5 Secure Enterprise Platform Hadoop Gateway (Big Data Engine)
 
Refer to Technical Document below on tuning suggestion for Impala:
KB259058: Tuning Impala for optimization in MicroStrategy Analytics Enterprise 10.x
 
 
CAUSE:
 
The Impala Parquet writer always creates an unannotated binary field for string columns.
 
 
WORKAROUND:
 
 
It is possible to specify that the field include the UTF-8 annotation to indicate UTF-8 string data. It was resolved in Impala 2.6 and CDH 5.11 has 2.8.
 
If all these columns need to be converted to UTF-8 strings, add 

spark.sql.parquet.binaryAsString=true

 into configuration file 

<Strategy Hadoop Gateway installation path>

/conf/hgos-sar

 
Article Reference Number: KB442072