Measuring the Limits of Data Parallel Training for Neural Networks

1 · Google AI Research · March 19, 2019, 4 p.m.
Posted by Chris Shallue, Senior Software Engineer and George Dahl, Senior Research Scientist, Google AIOver the past decade, neural networks have achieved state-of-the-art results in a wide variety of prediction tasks, including image classification, machine translation, and speech recognition. These successes have been driven, at least in part, by hardware and software improvements that have significantly accelerated neural network training. Faster training has directly resulted in dramatic imp...