Numerical Methods for Unconstrained Optimization and Nonlinear Equations

Table of Contents

• Preface
• 1. Introduction. Problems to be considered
• Characteristics of 'real-world' problems
• Finite-precision arithmetic and measurement of error
• Exercises
• 2. Nonlinear Problems in One Variable. What is not possible
• Newton's method for solving one equation in one unknown
• Convergence of sequences of real numbers
• Convergence of Newton's method
• Globally convergent methods for solving one equation in one uknown
• Methods when derivatives are unavailable
• Minimization of a function of one variable
• Exercises
• 3. Numerical Linear Algebra Background. Vector and matrix norms and orthogonality
• Solving systems of linear equations—matrix factorizations
• Errors in solving linear systems
• Updating matrix factorizations
• Eigenvalues and positive definiteness
• Linear least squares
• Exercises
• 4. Multivariable Calculus Background
• Derivatives and multivariable models
• Multivariable finite-difference derivatives
• Necessary and sufficient conditions for unconstrained minimization
• Exercises
• 5. Newton's Method for Nonlinear Equations and Unconstrained Minimization. Newton's method for systems of nonlinear equations
• Local convergence of Newton's method
• The Kantorovich and contractive mapping theorems
• Finite-difference derivative methods for systems of nonlinear equations
• Newton's method for unconstrained minimization
• Finite difference derivative methods for unconstrained minimization
• Exercises
• 6. Globally Convergent Modifications of Newton's Method. The quasi-Newton framework
• Descent directions
• Line searches
• The model-trust region approach
• Global methods for systems of nonlinear equations
• Exercises
• 7. Stopping, Scaling, and Testing. Scaling
• Stopping criteria
• Testing
• Exercises
• 8. Secant Methods for Systems of Nonlinear Equations. Broyden's method
• Local convergence analysis of Broyden's method
• Implementation of quasi-Newton algorithms using Broyden's update
• Other secant updates for nonlinear equations
• Exercises
• 9. Secant Methods for Unconstrained Minimization. The symmetric secant update of Powell
• Symmetric positive definite secant updates
• Local convergence of positive definite secant methods
• Implementation of quasi-Newton algorithms using the positive definite secant update
• Another convergence result for the positive definite secant method
• Other secant updates for unconstrained minimization
• Exercises
• 10. Nonlinear Least Squares. The nonlinear least-squares problem
• Gauss-Newton-type methods
• Full Newton-type methods
• Other considerations in solving nonlinear least-squares problems
• Exercises
• 11. Methods for Problems with Special Structure. The sparse finite-difference Newton method
• Sparse secant methods
• Deriving least-change secant updates
• Analyzing least-change secant methods
• Exercises
• Appendix A. A Modular System of Algorithms for Unconstrained Minimization and Nonlinear Equations (by Robert Schnabel)
• Appendix B. Test Problems (by Robert Schnabel)
• References
• Author Index
• Subject Index.
Show more