Asap: A Stochastic Adaptive PCA Method For Increasing Block Size Setting
Abstract We propose Asap, an adaptive stochastic optimization algorithm for principal component analysis (PCA), in the increasing block size setting. Asap is a novel generalized variant of the classical Oja’s algorithm (Oja, 1982), but can compute top-k principal components without the necessity of tuning the step size. Asap performs PCA by first-order gradient-based optimization based on adaptive estimates of lower-order moments as with Adagrad and Adam. We provide a theoretical guarantee ...