SCENARIO: Put yourself in the shoes of an analyst building a report. You are a store manager and you are looking at store profit in Florida for books. You only have profit stored at the Month level. However you want to drill down to the day level because you want to see what could be going on in the summer months that have led to a profit. Were there specific weeks that were in particularly high, was there specific weekends that performed well. AND THEN YOU REMEMBER, Saumit down the hall mentioned that he was loading daily transactional data into HDFS BUT John your Big Data cluster admin does not have the capacity to be able to load this data into Hive tables. AND THEN YOU REMEMBER, David at Strategy World a few months ago was talking about this new gateway that you can connect to to publish cubes directly from HDFS.
DEMO
First, select external data from Hadoop For the purposes of this demo, let’s assume that the Gateway has already been configured
A few things that I want to highlight during file browsing:
As long as all of the underlying files have the same file structure, the cube will be published. At cube publication, the partitions will be unioned together into one table.
So we get into the file and we see it is not exactly in the format that we need it to be in. However, we still do not need to reach out to the Big Data cluster admin to apply any changes. We can apply data wrangling on this file on-the-fly. In this interface, we can manipulate columns, add columns. We can then go back into the dataset and ensure that the different columns will be imported properly. Here we can change the data type and convert attributes to metrics. And now we can begin the cube publication process. We can go to Ambari or cloudera manager, and navigate to YARN application to ensure jobs are kicked off.
Once the table is imported, we link Month to the top dataset and then we now have the top dataset acting as a filter for the daily transaction data that we have just loaded from HDFS.