As the introductory article mentioned, PetaFLOPS may be here faster than you may realize. You can construct PetaFLOPS systems today as discussed Part 1 of this series because Jaguar at Oak Ridge and Roadrunner at Sandia have already hit the PetaFLOPS level. However, it’s definitely not cheap or easy to reach this level. This article examines what PetaFLOPS systems might look like today and in the near future using CPUs.

Measuring Performance:
To determine when we reach 1 PFLOPS we need an objective measure. While there are arguments for and against the Top500 benchmark, this article will assume that the performance of systems is measured by the HPL (Top500) application. An idealized model of estimating the performance using the number of double precision operations per clock cycle, the total number of cores, and the “efficiency” of the system on HPL. Basically:

Performance = Number of operations per clock * number of cores * efficiency

These three simple numbers, operations per clock (abbreviated “ops per clock”), number of cores, and efficiency, will allow us to estimate performance. However, making educated estimates of these three numbers is not easy and always leaves room for debate. In addition, these three numbers have implications for configurations, particularly the network configuration.

Let’s take a look at current CPUs from Intel and AMD to help us develop these three numbers.


Intel Systems:
Intel is using what they call a “tick-tock” model of processor development. The “tick” is a die shrink and the “tock” is a change to the architecture. The current generation of Nehalem (Xeon 5500) processors is in the “tock” phase of the process, and the upcoming processors, code-named Westmere, are a “tick”, shrinking the die to 32 nm. So the Nehalem and Westmere processors have the following key parameters (key to our estimates of the performance):

· Nehalem:
o Clock speed is limited to 2.93 GHz at 95W
o Capable of four operations per clock (double precision)
o Four cores per socket
o Restricted to 2 sockets per node

· Westmere:
o It is assumed that clock speeds will remain about the same for the same power. So the top speed is likely to be approximately 2.93 GHz or 3.0 GHz (Note: this is an assumption since Intel has not yet announced the processor)
o 6 cores per socket (http://www.theregister.co.uk/2010/02/03/intel_westmere_ep_preview/)
o Capable of four operations per clock (again, this is an assumption since this processor is a “tick” and is only a die shrink).
o Restricted to 2 sockets per node

· Nehalem-EX:
o Maximum speed looks to be 2.26 GHz (http://en.wikipedia.org/wiki/Xeon#Beckton)
o 8-cores per socket (http://en.wikipedia.org/wiki/Xeon#Beckton)
o Four operations per clock (again, this is an assumption since this processor is a “tick” and is only a die shrink).
o 4 sockets and larger are allowed (http://en.wikipedia.org/wiki/Xeon#Beckton)


The next generation of processors after Westmere involves an architecture change. This generation, called Sandy Bridge, has some of the following characteristics:

· Next generation (Sandy Bridge):
o 2.8 GHz to 3.4 GHz without Turbo (http://en.wikipedia.org/wiki/Intel_Sandy_Bridge_%28microarchitecture%29)
o 8 flops per clock (http://en.wikipedia.org/wiki/Intel_Sandy_Bridge_%28microarchitecture%29)
o Sandy Bridge B2: 8 cores and 6 core variants (http://en.wikipedia.org/wiki/Intel_Sandy_Bridge_%28microarchitecture%29)
o I have kept the number of sockets per system at 2 (just conjecture)

AMD:
AMD is taking a slightly different approach compared to Intel by putting more cores in their CPUs with lower clock speeds. The list of processors starts with the current processor, the 6-core “Istanbul” processor. Then it continues with the next-generation processors code-named Magny-Cours (named after a Formula 1 race track).

· Istanbul
o This is AMD’s current generation CPU
o 2.8 GHz is the maximum clock speed at this time (that I know of)
o Four operations per clock

· Magny-Cours (G34):
o Next generation AMD processor
o 8-core and 12-core per socket versions (http://blogs.amd.com/work/tag/magny-cours/)
o 2.4 GHz appears to be the faster clock speed (http://www.techspot.com/news/37972-amd-begins-shipping-magnycours-opteron.html)
o Four operations per clock (This is an assumption since the processor hasn’t been announced).
o Up to 4 sockets per system

Future Intel and AMD Processors:
Intel and AMD have both hinted at processors beyond the next generation with AMD providing more detail on a relative near-term processor code-named Interlagos. However, for these processors I have taken some “poetic” license and played something of a “what-if” game to project performance. While these projections are fun to a certain extent, they also help in determining what processors will have to deliver to reach 1 PetaFLOPS for certain system configurations. I have no knowledge of these processors beyond what I read in the open news as everyone else.

Interlagos uses the same basic socket as the upcoming Magny-Cours processor but increases the number of cores per socket to 16 (http://www.cio.com/article/490450/Inside_AMD_s_16_Core_Interlagos_Server_Chip). For the purposes of this article I lowered the clock speed to 2.3 GHz. However, I have left the number of operations per clock at 4 compared to Intel’s Sandy Bridge processor that has 8 operations per clock (they are both to be released about the same time).

Beyond Interlagos, I have taken guesses at what configurations might look like. These configurations help in determining what processor performance has to be to achieve 1 PetaFLOPS for a given system configuration. These processors are purely conjecture at this point.

Future Intel Processor:
· Increased the number of cores per socket to 16 (doubling the number of cores from Nehalem-EX).
· Kept clock speed constant at 2.93 GHz
· Kept number of operations per clock at 8
· 2-socket systems

Future AMD Processor:
· Increased the number of cores per socket to 24 (doubling the number of cores of Magny-Cours)
· Kept clock speed at 2.4 GHz (conjecture)
· Increased number of operations per clock to 8
· Allowed for 4-socket systems

Below is a table of results for some real systems and some conjectured systems. I have left the decimals in just so you can reproduce the results if you like. We all know that you can’t have 0.2 of a node or 0.16 of an IB port. J