ALBUQUERQUE, N.M. — The full complement of 84 cabinets that compose the teraflops high-performance ultracomputer – the fastest computer in the world – is now operating at Sandia National Laboratories.
The Intel massively parallel computer was developed under direction of the Department of Energy for the Accelerated Strategic Computing Initiative (ASCI), a 10-year program designed to develop the higher-resolution, three-dimensional physics modeling needed to evaluate the aging nuclear stockpile without actual testing.
The teraflops (which stands for one trillion floating point operations per second), is made up of 76 actual computer cabinets with 9,072 Pentium Pro processors, and has nearly six hundred billion bytes of memory. The remaining eight cabinets are called disconnect cabinets and separate the machine so that classified and unclassified calculations can be run at the same time.
The entire computer consists of four rows with 21 cabinets in each row. It covers about 1,600 square feet, enough to fill a moderate-sized home.
The computer is capable of performing up to 1.8 teraflops, or floating point operations per second. It would take someone operating a hand-held calculator about 57,000 years to calculate a problem the teraflops computer could compute in one second.
This is the same computer that achieved the one-trillion math-operations per second computing milestone last December in a test demonstration at Intel’s Beaverton, Ore., plant. That demonstration, however, was achieved using 7,264 Pentium Pro processors in 57 cabinets, or three-fourths of the full machine.
“Teraflops computing and ASCI provide an extraordinary opportunity for the three weapons laboratories in DOE to work together on behalf of the science-based stockpile stewardship program,” says Sandia Director C. Paul Robinson. “It is a very important step in shifting from a test-centered program to a computational-centered program.”
The teraflops computer ushers in a new era in which high-fidelity 3-D simulation will enable scientists to reach the eventual goal of preserving a safe, secure and reliable nuclear deterrent without underground testing. Sandia scientists and engineers already have been using the computer to calculate stockpile-related problems.
“The outstanding applications software development skills of Sandia and our DOE partners complement Intel Corporation’s superb computer hardware capabilities to create a dynamic combination that promises to revolutionize computational science in many disciplines,” says Bill Camp, director of Computational Sciences, Computer Sciences and Mathematics at Sandia.
The $55 million teraflops computer and its more powerful successors under the ASCI program are needed to simulate the complex 3-D physics involved in nuclear-weapon performance, and to accurately predict the degradation of nuclear weapons components as they age in the stockpile. Powerful multi-teraflops computers also will permit analysts to quickly run full-system 3-D simulations of complex accident environments, such as an airplane crash followed by a fuel fire.
The fastest computer prior to the completion of the teraflops was a special-purpose Japanese Hitachi computer that reached the 368 gigaflops (billion floating point operations per second) mark, a record which stood until last December’s achievement of 1.06 teraflops. Sandia scientists have since achieved 1.28 teraflops on the new ultracomputer and expect that to go higher in the near future.
Sandia scientists and engineers have achieved the following calculations so far on three-fourths of the full machine:
100 million-cell calculation models performance of ballistic weapon system
Sandia scientists have completed an unprecedented 100 million-cell CTH code calculation that modeled the performance of a ballistic weapon system employed in the contact fuzing mode. Never before has the computing capability (memory size) been available to model, in 3-D and with greatly enhanced resolution, the entire region of interest in the weapon for this event. These recent calculations with the teraflops machine have demonstrated that Sandia scientists are now beginning to be able to address the various issues involved in certifying fuze performance in a single, full-system simulation of the event. This is an important aspect of the Department of Energy’s Accelerated Strategic Computing Initiative, designed to provide the higher-resolution, three-dimensional physics modeling needed to evaluate the aging nuclear weapons stockpile without actual testing.
The challenge was to simulate the several “time races” involved in the fuzing and firing of the weapon, evaluating the vulnerability of critical fireset components to impact and primary explosive detonation shocks. Certain components, such as neutron generators, must be isolated from shock damage long enough after impact (fractions of a millisecond) to perform their respective function in the firing sequence before being destroyed by the impinging shock waves. The time margin for component survival will depend strongly on such things as the impact velocity, angle, and target materials. Numerical simulations, tied to limited test data, can provide detailed performance evaluations for system impact conditions that cannot be tested.
Computer model of comet striking the ocean shows teraflops capabilities to DOE
At the request of the Department of Energy for the dual purpose of generating unclassified data to test visualization techniques and to assist in installation testing of the new teraflops computer, Sandia scientists performed a computational simulation using the CTH shock physics hydrocode. The calculation, consisting of 54 million zones ran for 48 hours on 1,500 processors of the teraflops. The problem modeled a one-kilometer comet, weighing about 1 billion tons, traveling 60 kilometers a second, and striking the Earth’s atmosphere at a 45 degree angle. The calculation showed large quantities of ocean water would be vaporized by the tremendous energy of the impact and ejected into suborbital ballistic trajectories that reenter worldwide. The result would be devastating tidal waves and a cloud of water and debris enveloping the globe that would affect the Earth’s climate.
Can-Crushing Problem Shows Power of “Measured Scalability”
Sandia scientists recently developed and demonstrated software that is the first to enable large finite-element models to be run efficiently on hundreds or thousands of processors in distributed-memory parallel computers such as the teraflops. Called the Parallel Material Contact Software, it enables scientists and engineers to perform computer analyses of larger, more complex systems than ever before, faster than ever before, and with greater accuracy. The ability to distribute a problem to great numbers of processors on a parallel computer is called scalability.
The software has been used to simulate the crushing of a thin-walled cylinder by an inclined block. As the can crumples, the buckling and consequent material contacts are computed dynamically. Running on 512 processors, with each processor handling 1875 hexahedral elements, scientists are simulating a model with almost one million elements. The Parallel Material Contact Software is currently used at Sandia in the PRONTO3D, JAS3D, and ALEGRA computation mechanics codes, and is enabling scientists and engineers to conduct finite-element simulations of unprecedented resolution in such areas as nuclear stockpile stewardship problems, reservoir modeling, and structural dynamics problems.
Sandia is a multiprogram Department of Energy laboratory, operated by a subsidiary of Lockheed Martin Corp. With main facilities in Albuquerque and Livermore, Calif., Sandia has broad-based research and development programs contributing to national defense, energy and environmental technologies, and economic competitiveness.
Technical contacts:
Bill Camp, wjcamp@sandia.gov (505) 845-7655,
Art Hale (Sandia teraflop program manager), alhale@sandia.gov (505) 845-7802