Out-of-sample extrapolation of learned manifolds

Abstract

We investigate the problem of extrapolating the embedding of a manifold learned from finite samples to novel out-ofsample data. We concentrate on the manifold learning method called Maximum Variance Unfolding (MVU), for which the extrapolation problem is still largely unsolved. Taking the perspective of MVU learning being equivalent to Kernel Principal Component Analysis (KPCA), our problem reduces to extending a kernel matrix generated from an unknown kernel function to novel points. Leveraging on previous developments, we propose a novel solution, which involves approximating the kernel eigenfunction by using Gaussian basis functions. We also show how the width of the Gaussian can be tuned to achieve extrapolation. Experimental results, which demonstrate the effectiveness of the proposed approach, are also included.