An optimal scheduled learning rate for a randomized Kaczmarz algorithm
NF Marshall, O Mickelin�- SIAM Journal on Matrix Analysis and Applications, 2023 - SIAM
NF Marshall, O Mickelin
SIAM Journal on Matrix Analysis and Applications, 2023•SIAMWe study how the learning rate affects the performance of a relaxed randomized Kaczmarz
algorithm for solving, where is a consistent linear system and has independent mean zero
random entries. We derive a learning rate schedule which optimizes a bound on the
expected error that is sharp in certain cases; in contrast to the exponential convergence of
the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal
of the Lambert-function of an exponential.
algorithm for solving, where is a consistent linear system and has independent mean zero
random entries. We derive a learning rate schedule which optimizes a bound on the
expected error that is sharp in certain cases; in contrast to the exponential convergence of
the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal
of the Lambert-function of an exponential.
Abstract
We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving , where is a consistent linear system and has independent mean zero random entries. We derive a learning rate schedule which optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert- function of an exponential.
Society for Industrial and Applied Mathematics