Self-labelling via simultaneous clustering and representation learning

Combining <PRE_TAG>clustering</POST_TAG> and <PRE_TAG>representation learning</POST_TAG> is one of the most promising approaches for <PRE_TAG>unsupervised learning</POST_TAG> of <PRE_TAG>deep neural networks</POST_TAG>. However, doing so naively leads to ill posed learning problems with degenerate solutions. In this paper, we propose a novel and principled learning formulation that addresses these issues. The method is obtained by maximizing the information between <PRE_TAG>labels</POST_TAG> and <PRE_TAG>input data indices</POST_TAG>. We show that this criterion extends standard crossentropy minimization to an <PRE_TAG>optimal transport problem</POST_TAG>, which we solve efficiently for millions of input images and thousands of <PRE_TAG>labels</POST_TAG> using a fast variant of the <PRE_TAG>Sinkhorn-Knopp algorithm</POST_TAG>. The resulting method is able to self-label visual data so as to train highly competitive <PRE_TAG>image representations</POST_TAG> without manual <PRE_TAG>labels</POST_TAG>. Our method achieves state of the art representation learning performance for AlexNet and ResNet-50 on SVHN, CIFAR-10, <PRE_TAG>CIFAR-100</POST_TAG> and ImageNet and yields the first self-supervised AlexNet that outperforms the supervised Pascal VOC detection baseline. Code and models are available.