Abstract We propose Asap, an adaptive stochastic optimization algorithm for principal component analysis (PCA), in the increasing block size setting. Asap is a novel generalized variant of the classical Oja’s algorithm (Oja, 1982), but can compute top-k principal components without the necessity of tuning the step size. Asap performs PCA by first-order gradient-based optimization based on adaptive estimates of lower-order moments as with Adagrad and Adam. We provide a theoretical guarantee ...