Like a modern-day version of the 20th-century space race, companies and governments worldwide are scurrying to build an what’s called an exascale computer, which could do a billion billion calculations per second. With every hardware upgrade in the meantime, scientists have to modernize their software. Many now utilize parallel programming models such as OpenACC, which helps programmers migrate legacy code to new platforms faster.

Last summer, scientists at Oak Ridge National Lab (ORNL) sought out scientists who could adapt a miniapp, a proxy of a larger application, for use on multiple high-performance hardware systems using OpenACC. The miniapp, called Minisweep, uses radiation transport codes to model neutron flow in a nuclear reactor.

Sunita Chandrasekaran, an assistant professor in computer and information sciences at the University of Delaware, and Robert Searles, a doctoral student in computer science, answered the call. This work in collaboration with ORNL scientists Oscar Hernandez and Wayne Joubert uses OpenACC to adapt Minisweep to run accurately and quickly on Titan, the world’s fifth largest supercomputer, which is housed at ORNL. Their work is also being used in benchmarking for the Oak Ridge Leadership Computing Facility’s new Summit supercomputer.

Chandrasekaran is revealing her team’s code at the GPU Technology Conference, HPC & Supercomputing Conference Sessions in Silicon Valley from March 26 to 29, 2018. She is one of the nine featured speakers at the conference.

Adapting Minisweep

The Minisweep miniapp models scenarios within a nuclear reactor by “sweeping” across a grid that is akin to a giant Rubik’s Cube. The squares on the grid represent points in space and are used to calculate the positions, energies, and flows of neutrons.  

This parent application to Minisweep is used to reduce the odds of a meltdown and to safeguard engineers who work around the nuclear reactor from radiation exposure.

Chandrasekaran and Searles set out to modify the miniapp to match the performance of CUDA, a low-level parallel programming language developed specifically for graphics processing units (GPUs) made by NVIDIA, the computing technology company. Using CUDA, Minisweep runs 83.72 times faster than code that is not parallelized on NVIDIA’s new Volta GPU computing platform.

The OpenACC implementation of Minisweep developed by Chandrasekaran and Searles ran 85.06 times faster than code that was not parallelized.

“If you are running OpenACC code and you are in the ballpark of CUDA in terms of performance, you’re doing really well,” said Searles. “We’re neck and neck with it with Minisweep.”

The Minisweep app will be used in acceptance testing for the Summit computer, which is expected to be significantly faster than Titan.

“It is huge to be involved in this capacity, and that a doctoral student’s work is making so much of an impact is even cooler,” Chandrasekaran said of Searles.

This success could translate to successes with other applications.

“On the domain science side, we can model reactors and make sure everything is safe,” said Searles. “On the computer science side, we’re accelerating a particular type of code structure that appears in many other different types of scientific applications, so it could help accelerate other codes in other domains as well.”

The team is prepared for plenty of questions about how they obtained such impressive, unusual results. One answer is that they studied the algorithms closely. Their expertise spans hardware, programming model, algorithms and code optimization.

“Once you know how the data flows and what element is dependent on the other, then you can think about computationally transferring that to a hardware model,” said Chandrasekaran.

Chandrasekaran, the Director of OpenACC user adoption, is an early adopter of the programming model and has also co-edited a book with Guido Juckeland from Helmholtz-Zentrum Dresden-Rossendorf (HZDR) in Germany, released last year, on OpenACC concepts and strategies.

She also credits tenacity for the team’s success. In close to nine months of work on this project, they have had plenty of stumbles to work through. Her advice?

“Don’t give up the first time,” she said. “Profile the code, work through the code, and find out what are the issues. Do one run, a second run, a third run, and iteratively improve the code.”