Discriminative clustering of high-dimensional data using generative modeling

Abstract

We approach unsupervised clustering from a generative perspective. We hybridize Variational Autoencoder (VAE) and Generative Adversarial Network (GAN) in a novel way to obtain a vigorous clustering model that can effectively be applied to challenging high-dimensional datasets. The powerful inference of the VAE is used along with a categorical discriminator that aims to obtain a cluster assignment of the data, by maximizing the mutual information between the observations and their predicted class distribution. The discriminator is regularized with examples produced by an adversarial generator, whose task is to trick the discriminator into accepting them as real data. We demonstrate that using a shared latent representation greatly helps with discriminative power of our model and leads to a powerful unsupervised clustering model. The method can be applied to raw data in a high-dimensional space. Training can be performed end-to-end from randomly-initialized weights by alternating stochastic gradient descent on the parameters of the model. Experiments on two datasets including the challenging MNIST dataset show that the proposed method performs better than the existing models. Additionally, our method yields an efficient generative model.