EducationSoftwareStrategy.com
StrategyCommunity

Knowledge Base

Product

Community

Knowledge Base

TopicsBrowse ArticlesDeveloper Zone

Product

Download SoftwareProduct DocumentationSecurity Hub

Education

Tutorial VideosSolution GalleryEducation courses

Community

GuidelinesGrandmastersEvents
x_social-icon_white.svglinkedin_social-icon_white.svg
Strategy logoCommunity

© Strategy Inc. All Rights Reserved.

LegalTerms of UsePrivacy Policy
  1. Home
  3. Topics

Performance comparison between Druid data store and MicroStrategy Multi-table Data Import (MTDI) Cubes

Community Admin

• Strategy

Purpose of the test conducted was to compare a normalized warehouse with single-table warehouse over simple queries: a join with lookup tables, aggregation and sorting.

The dataset comes in two shapes: a 478 GB data set and a subset of 100 GB. They both have a flattened version and a normalized version. The initial warehouse has 6 recursive lookup tables and a fact table. When flattening the model, the 6 recursive tables got flattened in 25 different levels each, then joined with the fact table. 
 
Flattened table example:

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

32
DIM101...DIM125... ...DIM501...DIM525Fact        NameComponentHardwareDruid single node

 Druid 0.10.1x1.16xlarge (64 vCPU, 976 RAM)Strategy 10.8 GAm4.xlarge (4 vCPU, 16 RAM)Hadoop cluster (deep storage and cube publishing)

Normalization example:

ka04W00000148uAQAQ_0EM44000000Q05o.png
Comment

0 comments

Details

Knowledge Article

Published:

September 27, 2017

Last Updated:

September 27, 2017