Un-Oh, Error fetching the list of files for the given network path: com.Strategy.webapi.MSTRWebAPIException:(Error in loading file from URL. Check the URL.[Folder Browser from HDFS Server failed. Error in importing files. Please check the file to import. Error type: Data Import Error. Couldn’t connect to the server. Connection String: HTTP://10.242.109.10:4020/RESOURCE/HDFSCATALOG?HDFSURL=hdfs://10.242.109.2:8020/;.])
Refer to Technical Document below on Hadoop Gateway advanced logging with log4j.properties.
KB39553: How to enable the advanced logging to troubleshoot Hadoop Gateway issues in Strategy 10.6 and later?
Refer to Technical Document below on Strategy Hadoop Gateway deployment:
KB248914: Overview and installation of the MicroStrategy 10.0 - 10.5 Secure Enterprise Platform Hadoop Gateway (Big Data Engine)
CAUSE:
There could be different factors make Strategy could not connect to HDFS server.
ACTION:
Below is a list of check points to diagnose the issue:
- Make sure Strategy Hadoop Gateway service has been started successfully and port number 4020 is open on Strategy Hadoop Gateway host machine. In Strategy10.11, the port 4020 is configurable in Hadoop. You can change the value (port number) of parameter hgos.restful.port in <Strategy Hadoop Gateway installation path>/conf/hgos-spark.properties.
- Check if the HDFS service is running and NameNode is accessible from the Strategy Hadoop Gateway host machine.
- Check if an appropriate connection string is configured and the values are correct. All the connection string attributes are case sensitive.
Connection string template:
hadoopName=<Active Hadoop NameNode>;hdfsPort=<NameNode port>;BDEIP=<Strategy Hadoop Gateway host machine>;BDEPORT=<Strategy Hadoop Gateway port>;
If Hadoop NameNode High Availability Is Configured
- Check with cluster administrators about the HDFS nameservice.
- Replace the attribute hadoopName with hadoopNameService in the connection string, and remove the attribute hdfsPort.
hadoopNameService=<HDFS nameservice>;BDEIP=<Strategy Hadoop Gateway host machine>;BDEPORT=<Strategy Hadoop Gateway port>;
If Kerberos Authentication Has Been Enabled
1. Make sure the principal ticket is valid. You can refresh the ticket by running the following command:
kinit -k -t <Principal keytab file path> <Principal> -c <Cache name>
2. Make sure that the present principal has sufficient permissions to browse HDFS by running the following command:
hadoop fs -ls /
The expected result is like the output below:
[root@ash-109-32r sbin]# hadoop fs -ls /
Found 11 items
-rw-r--r-- 3 hdfs supergroup 1996573 2017-01-10 03:09 /0_0_0_parquet
drwxr-xr-x - hdfs supergroup 0 2017-05-23 08:43 /Blitz
-rw-r--r-- 3 root supergroup 21 2017-06-26 03:26 /DE65820.txt
drwxrwxrwx - root supergroup 0 2017-02-09 06:16 /DataSets
-rw-r--r-- 3 root supergroup 189 2017-11-13 06:49 /Date_Format.csv
-rw-r--r-- 3 root supergroup 308 2017-11-08 07:31 /Date_Format_b.csv
-rw-r--r-- 3 root supergroup 476 2017-11-01 10:26 /Date_Format_c.csv
drwxr-xr-x - root supergroup 0 2017-08-31 12:53 /Demo_Files
drwxrwxrwx - root supergroup 0 2017-07-13 14:56 /EPA
drwxr-xr-x - root supergroup 0 2017-09-07 02:55 /EPA_3C
drwxr-xr-x - root supergroup 0 2017-09-07 03:36 /EPA_3C_Part
NOTE:
In-memory cubes cannot be published or generate dashboards in the Live Connect mode. By using the Strategy Hadoop Gateway log, you can determine if data is transmitted back to Intelligence Server successfully from the Hadoop cluster.
Make sure that the port number used to receive data from Strategy Hadoop Gateway is open on the Intelligence Server. By default, the port number is 30241, and it is configurable in the registry.
If the intelligence Server and Hadoop cluster are installed on separate networks, the Intelligence Server communicates with the cluster via a gateway. Make sure to specify the Intelligence Server host name and IP. Otherwise, the Strategy Hadoop Gateway will submit data to the gateway machine instead, because the Strategy Hadoop Gateway detects the source automatically from the network request.
To specify the Intelligence Server host name and IP, modify the registry key and manually set DSHostName with the Intelligence Server host name and IP.
The ability to specify the Intelligence Server hostname and IP is only available for Strategy 10.10 and later.
Registry key:
HKEY_LOCAL_MACHINE/SOFTWARE/Wow6432Node/MicroStrategy/DSS Server/Castor/DSHostName
Registry file on the Intelligence Server:
MSIReg.reg
3. Restart the Intelligence Server and the Strategy Hadoop Gateway.
Article Reference Number: KB441881
