On-Line Learning in Neural Networks
On-Line Learning in Neural Networks
On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.
- The only neural networks book dedicated to the important area of on-line learning
- Provides a comprehensive overview of recent developments in the field as well as of traditional methods
- The chapters were designed to contain sufficient detailed material to enable the non-specialist reader to follow most of it with minimal background reading
Reviews & endorsements
Review of the hardback: 'I recommend this book to readers with a theoretical, analytical, or mathematical interest in neural networks, especially online learning.' Computing Reviews
Product details
March 2011Adobe eBook Reader
9780511836589
0 pages
0kg
40 b/w illus.
This ISBN is for an eBook version which is distributed on our behalf by a third party.
Table of Contents
- Foreword C. Bishop
- 1. Introduction D. Saad
- 2. On-line learning and stochastic approximations Léon Bottou
- 3. Exact and perturbative solutions for the ensemble dynamics Todd Leen
- 4. A statistical study of on-line learning Noboru Murata
- 5. On-line learning in switching and drifting environments Klaus-Robert Mueller, Andreas Ziehe, Noboru Murata and Shun-ichi Amari
- 6. Parameter adaptation in stochastic optimization Luis B. Almeida, Thibault Langlois, José D. Amaral and Alexander Plakhov
- 7. Optimal on-line learning for multilayer neural networks David Saad and Magnus Rattray
- 8. Universal asymptotics in committee machines with tree architecture Mauro Copelli and Nestor Caticha
- 9. Incorporating curvature information in on-line learning Magnus Rattray and David Saad
- 10. Annealed on-line learning in multilayer networks Siegfried Bös and Shun-ichi Amari
- 11. On-line learning of prototypes and principal components Michael Biehl, Ansgar Freking, Matthias Hölzer, Georg Reents and Enno Schlösser
- 12. On-line learning with time-correlated patterns Tom Heskes and Wim Wiegerinck
- 13. On-line learning from finite training sets David Barber and Peter Sollich
- 14. Dynamics of supervised learning with restricted training sets Anthony C. C. Coolen and David Saad
- 15. On-line learning of a decision boundary with and without queries Yoshiyuki Kabashima and Shigeru Shinomoto
- 16. A Bayesian approach to on-line learning Manfred Opper
- 17. Optimal perception learning: an on-line Bayesian approach Sara A. Solla and Ole Winther.
