IBM 大数据解决方案 Enterprise DataNode Using X3690X5 and EXP2512 Smarter Systems for a Smarter Planet 2011 IBM Corporation
Reasons for using X3690X5 Data Node IBM System and Technology Group Advantage Description Comments Higher Reliability Compute power Denser DataNode Business Continuance Internal Needs NL SAS HDD instead of SATA HDDs X3690X5 higher RAS over X3630M4 Customer may consider less copies of Data Node (default=3) 2 * 8C E7-2820 should provide sufficient compute resources when compared against x3630m4 2 * 6C E5-2420 Denser Data Node means fewer Data Node, that means less node to manage More expansion capacity for C requirement such as SNAPSHOT or CDP (Continuous Data Protection) IBM Client Rep can claim credit E7 server is generally recognized with higher reliability In term of DB throughput. Tivoli CDP or GPFS, future concerns This is reality Page 2
IBM BigData HW and SW stack IBM System and Technology Group 大数据加速器 文本统计 金融 地理空间 声音 图像 / 视频 挖掘 时间序列 数学 连接器应用蓝图 大数据企业引擎 InfoSphere Streams 生产力工具 & 优化 工作负载管理 & 优化 管理 配置 管理工具 工作流 配置管理器 InfoSphere BigInsights IBM Platform Symphony 低延迟 Hadoop 兼容具有动态资源调配的 MapReduce 运行时间库 作业调度 活动监视器 作业追踪 认证 & 访问管理 数据导入 数据保护 Platform Computing provide efficient scheduling and multiple interfaces GPFS File System Management Expansion, ILM, Replication 2 - IBM System X Server Data Nodes hold massive raw information Page 3
4 2012 IBM Corporation
5 2012 IBM Corporation
IBM System and Technology Group IBM 大数据基于 x3630m4 and 3690X5 数据节点的配置方案 基本配置高性能配置企业级配置 CPU 2 x E5-2420 1.9 GHz 6-core 2 x E5-2450 2.1 GHz 8-core 2 x E7-2820 2.0 GHz 8-core 内存 48GB 6 x 8GB (1333 Hz) 72GB 6 x 8GB + 6 x 4GB 96GB 12 x 8GB (1600 M Hz) 64 GB 8 x 8GB (1066 M Hz) 96 GB 12 x 8GB 硬盘 (OS) 1 x 2TB 3.5 1 x 2TB 3.5 1 x 300GB SAS 2.5 硬盘 (data) 24TB, 12 x 2TB NL SATA 3.5 36TB, 12 x 3TB NL SATA 3.5 硬盘阵列控制器 24TB, 12 x 2TB NL SATA 3.5 36TB, 12 x 3TB NL SATA 3.5 Up to 2 EXP2512, each with 24TB, 12 x 2TB NL SAS 3.5 36TB, 12 x 3TB NL SAS 3.5 6Gb JBOD Controller 6Gb JBOD Controller ServeRAID M1015 ServeRAID M5025 存储保护 None (JBOD) None (JBOD) Single HDD Ra 用户可用空间 6TB w/2tb drives 9TB w/3tb drives 6TB w/2tb drives 9TB w/3tb drives 6TB w/ 2TB drives 9TB w/ 3TB drives 网络 1GbE switch w/4 10GbE uplinks (IBM G8052) 10GbE switch w/4x 40GbE uplinks (IBM G8264) 10GbE switch w/4x 40GbE uplinks (IBM G8264) Page 6 6 假设三份数据拷贝, 非压缩, 为 Map/Reduce 中间过渡文件保留 25% 扩展能力 默认数据节点部署为三的倍数 (3,6,9 )
大数据解决方案硬件配置 :System x3690 X5 IBM System and Technology Group 3690 X5 是 2U 机架式 x86 架构服务器, 属于企业级服务器 支持低成本, 高容量 2.5 HDD 磁盘 配置灵活, 有多种型号可供选择, 具有很高的适用性 当前最高可达到 16 2.5 本地存储插槽 ; 利用 M5025 加上两个 EXP2512 扩展存储 24ge 3.5 硬盘 CPU 可以选择 6-core, 8-core, 10-core; 最大内存达到 1TB; 具有极高性价比 12 + 3.5 HDDs M5025 12 + 3.5 HDDs X3690X5 Page 7 7
IBM System and Technology Group IBM 大数据解决方案 System x M4 预先定义配置 管理节点 x3550 M4 with Two E5-2650 2GHz 8-core CPU 128GB RAM, 16x 8GB 1600MHz RDIMM Four 600GB 2.5 HDD (OS) Two Dual-port 10GbE (data) Dual-port 1GbE (mgmt) 数据节点 x3630 M4 with Two E5-2420 2.2GHz 6-core CPU 48GB RAM, 6x 8GB 1333MHz RDIMM Two 2TB 3.5 HDD (OS/app) Twelve 2TB 3.5 HDD (data) Optional 3TB/4TB HDD upgrade Dual-port 10GbE (data) Dual-port 1GbE (mgmt) 配置 Starter Half Rack Full Rack w/mgmt Node* Full Data Node Rack* 存储 (2TB/3TB/4TB) 72TB / 108TB / 144TB 216TB / 324TB / 432TB 456TB / 684TB / 912TB 480TB / 720TB / 960TB 裸数据空间 (2TB/3TB/4TB) 18TB / 27TB / 36TB 54TB / 81TB / 108TB 114TB / 171TB / 228TB 120TB / 180TB / 240TB 管理节点 / 数据节点 1 Mgmt / 3 Data 1 Mgmt / 9 Data 1 Mgmt / 19 Data 0 Mgmt / 20 Data 交换机 1 x 10GbE / 1 x 1GbE 1 x 10GbE / 1 x 1GbE 1 x 10GbE / 1 x 1GbE 1 x 10GbE / 1 x 1GbE Page 8 8
IBM System and Technology Group IBM 大数据解决方案 System x X5 预先定义配置 管理节点 x3550 M4 with Two E5-2650 2GHz 8-core CPU 128GB RAM, 16x 8GB 1600MHz RDIMM Four 600GB 2.5 HDD (OS) Two Dual-port 10GbE (data) Dual-port 1GbE (mgmt) 数据节点 x3690x5 with Two E7-2820 2.0GHz 8-core CPU 64GB RAM, 96 GB optional 8x 8GB 1066MHz RDIMM Two 300GB 2.5 HDD (OS/app) Twelve 2TB 3.5 HDD (data) Optional 3TB / 4TB HDD upgrade Dual-port 10GbE (data) Dual-port 1GbE (mgmt) 配置 Starter Half Rack Full Rack w/mgmt Node* 存储 (2TB/3TB/4TB) 72TB / 108TB / 144TB 144TB / 216TB / 288 TB 288TB / 432 / 576 TB 裸数据空间 (2TB/3TB/4TB) 18TB / 27TB / 36TB 36TB / 54TB/ 72TB 72TB / 108TB / 144TB 管理节点 / 数据节点 1 Mgmt / 3 Data 1 Mgmt / 3 Data 1 Mgmt / 6 Data 交换机 1 x 10GbE / 1 x 1GbE 1 x 10GbE / 1 x 1GbE 1 x 10GbE / 1 x 1GbE Page 9 9
PureFlex x86 和 x3630m4 推荐配置 Rack of 20 x3630m4 and switches 20 pre-defined x3630m4 servers x3630m4 : 2 E5-2430, 48 GB memory, 14x 3.5 3TB HDDs 1 ToR G8264 for 10Gb data network 1 ToR G8052 for 1Gb HW management network Rack of 28 (PureFlex x220 + Side-Car) + switches 4 Flex chassis, each chassis has 2 Pass-Thru modules 28 (x220 + side-car), each chassis has 7 full-wide (x220 + side-car) x220: 2 E5-2430, 48 GB memory, 2x 1TB HDDs Side-Car: 12x 1TB HDDs 1 ToR G8264 for 10Gb data network 1 ToR G8052 for 1Gb HW management network Rack of 28 (PureFlex x240 + Side-Car) + switches 4 Flex chassis, each chassis has 2 Pass-Thru modules 28 (x240 + side-car), each chassis has 7 full-wide (x240 + side-car) x240: 2 E5-2630, 64 GB memory, 2x 1TB HDDs Side-Car: 12x 1TB HDDs 1 ToR G8264 for 10Gb data network 1 ToR G8052 for 1Gb hardware management network Rack of 6 x3690x5 + 12 EXP2512 + switches 6 pre-defined x3690x5 servers x3690x5 : 2 E7-2820, 48 GB memory,, 2* 2.5 300G HDD 1-2 EXP2512 depends on data density Expansion capability based on requirement 12 EXP2512 each with 12x 3.5 2TB/3TB/4TB HDDs 1 ToR G8264 for 10Gb data network 1 ToR G8052 for 1Gb hardware management network 10 2012 IBM Corporation
IBM System and Technology Group BigInsights Hardware Foundation - Key Components Value Data Node IBM System x3630 M3 Two Intel Xeon E5-2420 CPUs Data: 12 x 2TB NL SATA HDDs, JBOD OS: 1 x 2TB NL SATA HDD 48GB DDR3 RDIMMs Enterprise Data Node IBM System x3690 X5 + EXP2512 Two Intel Xeon E7-2820 CPUs Data: 12 x 2TB NL SAS HDDs 24 HDDs OS: 1 x 300GB NL SAS HDD 64 GB DDR3 RDIMMs 96GB 1GbE Switch IBM RackSwitch G8052 48 1 GbE RJ45 ports and four standard 10 GbE SFP+ ports Low 130 W power rating and variable speed fans to reduce power consumption Value Management Node (JobTracker, NameNode, Console) IBM System x3550 M4 Two Intel Xeon E5-X5675 CPUs OS: 4 x 900GB NL SAS HDD, mirrored DVD drive 96GB DDR3 RDIMMs A storage-dense server is the right choice for Hadoop clients who need an ultra-dense, 10GbE Switch (optional) storage-rich server that offers a IBM RackSwitch G8264 low cost per terabyte, a wide range of processors, memory, networking and management options. Optimized for applications requiring high bandwidth and low latency Up to 64 1 Gb/10 Gb SFP+ ports, four 40 Gb QSFP+ports 1.28 Tbps non-blocking throughput Energy efficient cost effective design Page 11 11
IBM System and Technology Group Additional Information Page 12
IBM System and Technology Group Relative CPU comparison in a server G Hz Max Cache Turbo MB QPI GT/s Cores Watt SPECint_ rate base2006 SPECfp_ rate base2006 Spec JBB2005 bops Spec JBB2005 bops/jv M TPC-C E5-2420 1.90 2.40 15 7.20 6 95 368 293 E5-2450 2.10 2.90 20 8.00 8 95 552 401 E5-2470 2.30 3.10 20 8.00 8 95 572 414 1366787 170848 E7-2820 2.00 2.26 18 5.86 8 105 367 273 E7-2870 2.40 2.80 30 6.40 10 130 581 384 1305745 130575 E7-8870 2.40 2.80 30 6.40 10 130 2020 1100 E5-2620 2.00 2.50 15 7.20 6 95 394 336 E5-2650 2.00 2.80 20 8.00 8 95 539 430 E5-2670 2.60 3.30 20 8.00 8 115 646 484 E5-2690 2.90 3.80 20 8.00 8 135 697 507 1578448 197306 X3850X5 3,014,684 X240 1,503544 Personal Conclusion: Based on Big Data workload, E5-24xx is the best cost effective solution. However, coupled with NFR, X3690 should be an efficient alternative when 24 HDDs are attached Page 13