Kernel Recursive Least-Squares Tracker for Time-Varying Regression

TitleKernel Recursive Least-Squares Tracker for Time-Varying Regression
Publication TypeJournal Article
Year of Publication2012
AuthorsS. Van Vaerenbergh, M. Lázaro-Gredilla, and I. Santamaría
JournalIEEE Transactions on Neural Networks and Learning Systems
Month PublishedAugust
Keywordskernel adaptive filtering, KRLS
AbstractIn this paper, we introduce a kernel recursive least-squares (KRLS) algorithm that is able to track nonlinear, time-varying relationships in data. To this purpose, we first derive the standard KRLS equations from a Bayesian perspective (including a sensible approach to pruning) and then take advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in nonstationary scenarios. The resulting method is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner. In addition to its tracking ability, it has a number of appealing properties. It is online, requires a fixed amount of memory and computation per time step, incorporates regularization in a natural manner and provides confidence intervals along with each prediction. We include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm.
Supplementary material or Matlab code: