ALBUQUERQUE, N.M. — Sandia National Laboratories researcher Eric C. Cyr has received a 2018 Early Career Research Program award of $500,000 every year for five years to improve deep neural networks so they more efficiently combine experimental results with the most complex computer models.
The national award from the Department of Energy’s Office of Science is meant to “identify and provide support to those researchers early in their careers who have the potential to develop new scientific ideas, promote them and convince their peers to pursue them as new directions.”
The title of Cyr’s proposal is “Parallel-in-Layer Methods for Extreme-Scale Machine Learning.” Machine learning in the form of deep neural networks has seen unprecedented success in the past few years, leading to innovations in self-driving cars, pattern recognition (including facial and speech recognition) and natural language processing.
A deep neural network is a model whose architecture is loosely inspired by the brain. It uses a sequence of mathematical operations collected in layers to automatically embed knowledge of previously observed data. These models can make predictions and associations not previously observed; thus, machines “learn.”
Cyr attributes recent successes of these mainly commercial innovations to the advent of high-density computing devices and the existence of large data sets used to train the computer-generated models. For his project, he proposes to advance this technology further into the realm of more complex scientific devices. In particular, he wrote in his proposal, the goal is to develop “an algorithmic toolset for scientific machine learning that will transform how experts integrate experimental data and computer simulations through the use of deep neural networks.”
One problem has been that despite the success of deep neural networks, the cost of these approaches remains high, with training times measured in days on relatively small computer clusters. Compared to the commercial applications of deep neural networks, scientific data sets are massive, measuring in terabytes to petabytes. To Cyr, algorithmic advances are required to robustly apply machine learning technologies to these demanding data sets.
Another problem: current approaches used in commercial deep-neural-network architectures don’t generalize well to scientific data sets.
Both of these issues limit the applicability of deep neural networks as a general tool for use in scientific machine learning.
Cyr aims to reduce or eliminate these limitations.
“Supporting talented researchers early in their career is key to building and maintaining a skilled and effective scientific workforce for the nation. By investing in the next generation of scientific researchers, we are supporting lifelong discovery science to fuel the nation’s innovation system,” said Secretary of Energy Rick Perry in a DOE news release. “We are proud of the accomplishments these young scientists have already made, and look forward to following their achievements in years to come.”
Cyr was a summer 2002 intern at Sandia’s Computer Science Research Institute, before finishing his bachelor’s degree summa cum laude at Clemson University in 2003 and earning a doctorate in computer science from the University of Illinois at Urbana-Champaign in December 2008. He joined Sandia in January 2009 as a postdoctoral researcher, became a senior member of the technical staff in 2010 and a principal member of the technical staff in 2015. He is a joint author on 25 published papers.
DOE Early Career grants are available in the program areas of advanced scientific computing research, biological and environmental research, basic energy sciences, fusion energy sciences, high energy physics and nuclear physics. To be eligible for the DOE award, a researcher must be an untenured, tenure-track assistant or associate professor at a U.S. academic institution or a full-time employee at a DOE national laboratory, who received a doctorate within the past 10 years.
Thirty researchers from DOE’s national laboratories and 54 from U.S. universities were selected for the prestigious award this year.