History

1998 LeCun - digits

2012 AlexNet - Krizhevsky et al.

ZFNet

2013

2014 - more efficient

2015 Residual Networks vs Plain net architectures

you can do more dropout

2016 - scale down, go wide

ILSVRC

2016 - Transformers

https://arxiv.org/pdf/1706.03762.pdf