Accelerators.html

Graphical Processor Units (GPUs) are considered by many to be the next frontier in High Performance Computing (HPC). With CPU performance stalling, vendors have been forced to increase the number of cores per chip at the expense of increased power and cooling requirements. CPU systems requiring over 200,000 cores are being built (DOE: Jaguar, NSF: Blue Waters) toward achieving over a PetaFlop of computing necessary to run climate models and global weather models at cloud resolving scales (3-4 KM horizontal resolution) within 3 years. However, systems of this size are impractical for weather forecasting for many reasons including power and cooling requirements, infrastructure costs, system reliability, and power costs. Power bills alone account for over $1M in moderate sized systems today; future CPU based systems being proposed could require $100M annually if conventional CPUs are used. GPUs are poised to provide the computing horsepower necessary for HPC systems in the future.

Using Video Game Technology for Science: Until recently, GPUs were used primarily by the gaming community; over 189 million GPUs were sold by NVIDIA since 2007. Interest by the HPC community was perked when NVIDIA released a high-level language, called CUDA, to the community which allows programmers to write code for GPU processors. As a result, there is growing interest in using GPUs for HPC applications as a low-cost, high performance alternative to CPUs.

Recent GPU Activities

The Advanced Computing Section of ESRL’s Global Systems Division began exploring GPUs in 2008 and is leading research and development of this technology at NOAA. We summarize our major activities to date:

Purchased a 16 node NVIDIA GTX280 system

Developed a Fortran-to-CUDA compiler. The compiler, called F2C-ACC, was released in June 2009 and has been updated several times since. Depending on the capabilities the recently released PGI GPU compiler, we will continue to develop our compiler as necessary.

Demonstrated 21x performance improvement on six representative subroutines from the Flow-following finite-volume Icosahedral Model (FIM) to assess the hardware and appropriateness of using this technology for next generation models. This work demonstrated that GPUs are feasible for our weather and climate codes. The Non-hydrostatic Icosahedral Model (NIM), developed at ESRL, slated to run at cloud resolving scales (3-4km) in 2012, is now being developed to run on the GPU.

Parallelized NIM model dynamics and achieved 25x performance improvement. These initial results are for a single GPU node; we plan to have the code running on multiple GPU nodes (using MPI for communication) by the end of 2009.

Recent Presentations

September 2009: M.Govett, Using Graphical Processing Units (GPUs) for Next Generation Weather and Climate Models, Climate and Atmospheric Sciences (CAS) workshop, Annecy, France.
June 2009: M.Govett, Development and Use of a Fortran-to-CUDA translator to run NOAA Weather Models, Path to PetaScale Accelerator workshop, NCSA, Illinois.
December 2008: C.Tierney, Using GPUs to Meet Next Generation Computational Requirements, Special Session on GPUs, American Geophysical Union Fall Meeting, San Francisco, CA.