EducationSoftwareStrategy.com
StrategyCommunity

Knowledge Base

Product

Community

Knowledge Base

TopicsBrowse ArticlesDeveloper Zone

Product

Download SoftwareProduct DocumentationSecurity Hub

Education

Tutorial VideosSolution GalleryEducation courses

Community

GuidelinesGrandmastersEvents
x_social-icon_white.svglinkedin_social-icon_white.svg
Strategy logoCommunity

© Strategy Inc. All Rights Reserved.

LegalTerms of UsePrivacy Policy
  1. Home
  2. Topics

How to publish a cube with the Hadoop Gateway


David Harsh

Software Engineer in Test II • Strategy


This video provides an example of when to use the Hadoop Gateway

SCENARIO: Put yourself in the shoes of an analyst building a report. You are a store manager and you are looking at store profit in Florida for books. You only have profit stored at the Month level. However you want to drill down to the day level because you want to see what could be going on in the summer months that have led to a profit. Were there specific weeks that were in particularly high, was there specific weekends that performed well. AND THEN YOU REMEMBER, Saumit down the hall mentioned that he was loading daily transactional data into HDFS BUT John your Big Data cluster admin does not have the capacity to be able to load this data into Hive tables. AND THEN YOU REMEMBER, David at Strategy World a few months ago was talking about this new gateway that you can connect to to publish cubes directly from HDFS. 
DEMO
First, select external data from Hadoop  For the purposes of this demo, let’s assume that the Gateway has already been configured
A few things that I want to highlight during file browsing:

  1. Security is maintained: if you don’t have access, you can see the top folder but won’t be able to select it.. 
  2. Multiple files can be accessed and published (you can see avro, orc, json, and flat files like txt and csv files) 
  3. The final thing  is that when the folder is saved with the file extension, it can be dragged over to the data import screen. In this case the dailytransaction folder is the parent of all the transaction level data. 

As long as all of the underlying files have the same file structure, the cube will be published. At cube publication, the partitions will be unioned together into one table. 
So we get into the file and we see it is not exactly in the format that we need it to be in. However, we still do not need to reach out to the Big Data cluster admin to apply any changes. We can apply data wrangling on this file on-the-fly. In this interface, we can manipulate columns, add columns. We can then go back into the dataset and ensure that the different columns will be imported properly. Here we can change the data type and convert attributes to metrics.  And now we can begin the cube publication process. We can go to Ambari or cloudera manager, and navigate to YARN application to ensure jobs are kicked off. 
Once the table is imported, we link Month to the top dataset and then we now have the top dataset acting as a filter for the daily transaction data that we have just loaded from HDFS.


Comment

0 comments

Details

Knowledge Article

Published:

May 3, 2017

Last Updated:

May 3, 2017