2020-04-30 Kim Martineau
They showed that a deep neural network could perform with only one-tenth the number of connections if the right subnetwork was found early in training.
Train the model, prune its weakest connections, retrain the model at its fast, early training rate, and repeat, until the model is as tiny as you want.
https://news.mit.edu/2020/foolproof-way-shrink-deep-learning-models-0430