From Data import interface, navigate to hadoop. From this portal, you can navigate to the Gateway Manager. This is where you will add your cluster configuration.
You will need to provide the following parameters in Web:
Name: Avoid special characters
Hadoop Properties:
Hadoop NameNode: FQDN or IP
HDFS Port: browse files, def. 8020
WebHDFS: preview file, def. 50070
Gateway Properties:
Host: machine to install Gateway
Port: I-Server to HG, def. 30004
Spark Properties:
YARN: Jar: path of spark assembly
Standalone: Master: Spark service; ex. spark://SparkMasterNode:7077
Local: Threads: simulated threads
Advanced:
Memory of executor:
~RAM/executors per node
spark.executor.instance:
# executors on cluster
spark.executor.cores:
# cores per executor
Once we do this, we can deploy the gateway. Once it is deployed (installed); we can start the gateway from the gateway manager. We can go back to the file browsing interface to ensure the WebHDFS, HDFS ports have been defined properly. And then we can navigate to the yarn application to ensure the application has properly started. Even though you might not be Big Data cluster admin, we have seen that most people in the network will typically have access to this interface to monitor the application.
