Optimal Transport and Deep Generative Models

Abstract: In this talk, I will review some recent advances on deep generative models through the prism of Optimal Transport (OT). OT provides a way to define robust loss functions to perform high dimensional density fitting using generative models. This defines so called Minimum Kantorovitch Estimators (MKE) [1]. This approach is especially useful to recast several unsupervised deep learning methods in a unifying framework. Most notably, as shown respectively in [2,3] (and reviewed in [4]) Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN) can be interpreted as (respectively primal and and dual) approximate MKE. This is a joint work with Aude Genevay and Marco Cuturi.

[1] Federico Bassetti, Antonella Bodini, and Eugenio Regazzini. On minimum Kantorovich distance estimators. Statistics & probability letters, 76(12):1298–1302, 2006.
[2] Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl-Johann Simon-Gabriel, and Bernhard Schoelkopf. From optimal transport to generative modeling: the VEGAN cookbook. Arxiv:1705.07642, 2017.
[3] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein GAN. Arxiv:1701.07875, 2017.
[4] Aude Genevay, Gabriel Peyré, Marco Cuturi, GAN and VAE from an Optimal Transport Point of View, Arxiv:1706.01807, 2017