See https://github.com/tensorflow/models/tree/master/official/mnist/mnist_eager.py