Stability Conditions for the Leaky LMS Algorithm Based on Control Theory Analysis - BISMOR, Dariusz; PAWELCZYK, Marek (Archives of Acoustics, 2016)
The Least Mean Squares (LMS) algorithm and its variants are currently the most frequently used adaptation algorithms; therefore, it is desirable to understand them thoroughly from both theoretical and practical points of view. One of the main aspects studied in the literature is the influence of the step size on stability or convergence of LMS-based algorithms. Different publications provide different stability upper bounds, but a lower bound is always set to zero. However, they are mostly based on statistical analysis. In this paper we show by means of control theoretic analysis confirmed by simulations that for the leaky LMS algorithm, a small negative step size is allowed. Moreover, the control theoretic approach alows to minimize the number of assumptions necessary to prove the new condition. Thus, although a positive step size is fully justified for practical applications since it reduces the mean-square error, knowledge about an allowed small negative step size is important from a cognitive point of view, and can be used on purpose in sophisticated tuning scenarios, e.g., in multiband processing.