HP’s First Xeon Superdome Aimed At SAP HANA
The first of several systems that will bring technologies from Hewlett-Packard's Superdome Itanium-based machines to big memory ProLiant servers based on Xeon processors is making its debut this week at SAP's annual customer shindig.
Code-named "Project Kraken," the system is commercialized as the ConvergedSystem 900 for SAP HANA and as such has been tuned and certified to run the latest HANA in-memory database and runtime environment. The machine, part of a series of high-end shared memory systems collected known as "DragonHawk," is part of a broader effort by HP to create Superdome-class machines out of Intel's Xeon processors.
In fact, if you drill down into the spec sheets, the ConvergedSystem 900 is also known as a ProLiant Superdome system. HP is keeping some of the engineering secrets behind the Kraken box to itself for now, but Paul Miller, vice president of marketing for ConvergedSystems within HP's Enterprise Group, did give EnterpriseTech some of the details.
The Kraken machine is based on a very precise "Ivy Bridge-EX" Xeon E7-2800 v2 series processor from Intel. Specifically, it is using the fifteen-core E7-2890 v2 with the C0 stepping; this runs at 2.8 GHz and has 37.5 MB of cache and a thermal design point of 155 watts. This version of the Xeon E7 chip is designed for two-socket NUMA nodes and is a bit less expensive than the E7-4800s designed for four-socket machines and even less pricey than the E7-8800s aimed at eight-socket machines. Rather than use Intel's own "Patsburg" C602J chipset and the E7-8800s to build a system that can scale up to 12 TB of main memory, HP is gluing together a bunch of two-socket nodes and for a number of different reasons.
Each blade in the Kraken server has four physical sockets and employs a variant of HP's own "Orion 2" chipset to lash them together into two-way nodes. The Orion 2 chipset is part of what HP calls its PREMA architecture, which is short for Performance, Resiliency, Efficiency, Manageability, and Availability. This is the second generation of Orion chipsets, the first being used in the ProLiant DL980 G7 server, which scaled up to eight sockets with the "Sandy Bridge-EX" Xeon E7 v1 processors. (HP has not done an eight-socket follow-on Gen8 machine to this box as yet, and it may never do so, focusing on the more scalable DragonHawk designs instead.) The Orion 2 chipset hooks into the QuickPath Interconnect (QPI) point-to-point interconnect ports on the Xeon E7 chips.
To glue up to eight of these two-socket nodes together, HP is using a variant of the sx3000 chipset that is employed in the Superdome 2 machines based on Itanium 9300 and 9500 processors. This chipset is what is often called a node controller or a crossbar switch and it links the multiple nodes together so they can present a single memory space to the operating system. (If you want to go the all-Intel route, the C602J chipset does the same thing for up to eight sockets if the system is using the E7-8800 variant of the processor.)
SAP allows for HANA nodes to be clustered so they can be scaled out for performance but this requires for data to be partitioned across those nodes. The whole point about deploying HANA on a single big NUMA node like a Kraken machine is that it has a very high-speed crossbar linking the nodes that is orders of magnitude faster than an Ethernet or InfiniBand network. (This is how the shared memory across the nodes is even possible.) HP is not saying how much bandwidth there is across the nodes in the Kraken machine or how low the latency is but on a 32-socket Superdome 2 with Itanium 9500 processors the sx3000 chipset provided 1.22 TB/sec of system fabric bandwidth and 816 GB/sec of I/O bandwidth across those 32 sockets in the system. (The Superdome 2 machine had L4 cache memory for each socket, and it is not clear if the Kraken machine employs L4 cache.) Miller would not give out fabric interconnect bandwidth or latency figures for the Kraken machine but did say that the sixteen-socket machine had a maximum aggregate I/O slot bandwidth of 640 GB/sec and an additional aggregate LAN bandwidth of 160 Gb/sec from LAN-on-motherboard ports.
All SAP HANA machines must be built using Xeon E7 processors. This is something that the German software giant insists on because it makes use of the extra memory bandwidth and resiliency features of the Xeon E7 chip. When all of the corporate data is residing in main memory you can't make any mistakes. At the moment, SAP has capped the largest HANA instance size at 12 TB, but make no mistake about it: The underlying DragonHawk machine can address far more than 12 TB. Miller will not say, but theoretically, a sixteen socket machine should be able to do 24 TB using 64 GB memory sticks. HANA memory compression boosts the effective memory capacity of a system by anywhere from 3X to 10X, says Miller, so the effective maximum size of a database on the Kraken machine is probably somewhere between 36 TB and 120 TB. These would be some pretty large databases, and indeed, there is no reason this could not have been done with a standard, all-Intel eight-socket box.
By going its own way with chipsets, HP has done two things. First, it can deploy twice as much compute underneath a given memory footprint. This will be valuable for certain customers. Moreover, with an upgrade coming after the initial Kraken shipments, HP will be supporting its nPar physical partitioning on the Kraken machine, which will allow for customers to have electrically isolated HANA partitions on the same physical box for redundancy. In other words, companies will be able to replicate HANA databases over that Superdome crossbar within a single system.
The obvious question, with SAP allowing for HANA nodes to be clustered, is: Why bother with a big NUMA setup instead of a cluster?
"If you look at HANA, it is really targeting three different workloads," explains Miller. "You need low latency for transactions, and in fact, you can't get that over a cluster. In fact, SAP does not recommend for transactions with a 12 TB system over a cluster as a certified configuration for running apps on top of HANA. The clustered architecture is OK for data warehouses and analytics, but because of the performance criteria for transactions, SAP recommends putting HANA applications atop a single memory pool."
As mentioned above, the nodes in a NUMA machine have a much faster interconnect than in possible with Ethernet or InfiniBand. And with the future nPar partitioning support, HP will have a disaster recovery angle to play as well.
The Kraken machine will come in two flavors: one with eight sockets and another with sixteen sockets, as you can see in the table above. HP is preconfiguring SUSE Linux Enterprise Server 11 SP3 on the machine, which is a requirement for SAP HANA databases; Miller was not at liberty to say when HP would support Red Hat Enterprise Linux on the Kraken machine underneath HANA, which SAP and Red Hat are also talking about this week at the SAPPHIRE event in Orlando. The Kraken machine is configured with internal storage and 3PAR storage arrays for operating systems, systems software, and snapshots out to disk. And with the nPar capability coming a little further down the pike, HP will be able to carve a bunch of Kraken machines up into slices to run workloads that scale up to 80 TB and beyond in physical memory.
Interestingly, HP is a big SAP shop and it has been running its own business atop Superdome 2 machines with HP-UX Unix and Oracle databases, and it is moving over to HANA on Kraken. Miller says that on preliminary testing on HP workloads, complex queries that used to take two hours are now taking two minutes and other queries that took seven minutes now take about seven seconds. That's a factor of 60X performance increase.
HP will start taking orders on the Kraken machines on June 16. They will ship to customers sometime in the third quarter.
Miller is mum on what other possible ConvergedSystem 900 machines will be available, or when. But the sx3000 chipset can scale to 32 sockets and each socket can currently address 768 GB of physical memory. The current Xeon architecture has 46-bit physical addressing, which means the most you can cram into a single NUMA instance is 64 TB. So, in theory, HP could forge a 32 socket ConvergedSystem machine with 32 TB or even 64 TB if it can get enough wires in the box. Incidentally, you cannot load anything but SLES and HANA on this machine – if you try anything else, it won't work. So don't get clever.
Pricing for the ConvergedSystem 900 is still being set but Miller gave us the preliminaries. The "not to exceed" list price for the based eight-socket Kraken machine with 6 TB of main memory, including the hardware outlined above and all of the software except the HANA SP7 stack (which you have to license separately but which HP preconfigures on the machine for you) is $870,000. HP Technology Services on top of that cost $170,000, and that includes 40 hours of consulting, deployment acceleration and engineering services plus three years of proactive care for the whole shebang. So it is just a tad over $1 million list price. The sixteen-socket, 12 TB configuration has a little discounting built into the package and costs around $1.79 million, with $1.56 million of that coming from the system and $230,000 coming from the services. These prices do not include fees for HP's ServiceGuard high availability replication software, which will be used to replicate data over the nPar partitions.