

















Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
An overview of numerical analysis, focusing on leading journals and resources for theoretical and computational aspects. It covers the importance of numerical analysis in science and engineering, the role of functional analysis, and the development of computer software. The document also mentions some introductory texts and online resources.
Typology: Study Guides, Projects, Research
1 / 25
This page cannot be seen from the preview
Don't miss anything!
Kendall E. Atkinson University of Iowa
Introduction I. General Numerical Analysis A. Introductory Sources B. Advanced Introductory Texts with Broad Coverage C. Books With a Sampling of Introductory Topics D. Major Journals and Serial Publications
levels of the subject. It covers the theoretical framework for a variety of linear algebra problems, and it also considers the effects of computer arithmetic and serial vs. parallel computers.
A Iserles. A First Course in the Numerical Analysis of Differential Equations , Cambridge University Press, 1996. This is a very nice introductory presentation of the theoretical framework for the numerical analysis for both ordinary and partial differential equations.
I. General Numerical Analysis
A. Introductory Sources
The best general introductions to numerical analysis are beginning graduate-level textbooks that cover most of the commonly recognized topics within numerical analysis. A few of my current favorites, listed alphabetically, are the following. Each of them gives an introduction to most of what are considered basic topics in numerical analysis. I include both current texts and classical texts that are still important references.
M Allen III and E Issacson. Numerical Analysis for Applied Science , John Wiley, New York, 1998. A very nice introduction.
K Atkinson. An Introduction to Numerical Analysis , 2nd^ ed., John Wiley, New York,
W Gander and J Hrebicek. Solving Problems in Scientific Computing Using Maple and Matlab , 3 rd^ ed., Springer, 1997.
W Gautschi. Numerical Analysis : An Introduction , Birkhäuser, Boston, 1997.
P Henrici. Elements of Numerical Analysis , John Wiley, New York, 1964. This is a classic text by a master of the subject. It contains well-written discussions of a broad set of topics. It is dated in some respects, but still contains much that is useful and interesting.
E Isaacson and H Keller. Analysis of Numerical Methods , corrected reprint of the 1966 original, Dover Pub., New York, 1994. This is a classic text covering many topics not covered elsewhere. For example, this text contains a very good introduction to finite difference methods for approximating partial differential equations.
R Kress. Numerical Analysis , Springer-Verlag, New York, 1998.
A Quarteroni, R Sacco, and F Saleri. Numerical Mathematics , Springer-Verlag, New York, 2000. This was cited earlier in the introduction as giving a very good introduction to the current state of numerical analysis.
J Stoer and R Bulirsch. Introduction to Numerical Analysis , 2nd^ ed., Springer-Verlag, New York, 1993. This is a translation of a popular German text.
C Uberhuber. Numerical Computation : Methods, Software, and Analysis , Vols. 1 and 2, Springer-Verlag, New York, 1997. This text pays much attention to software and to machine aspects of numerical computing.
B. Advanced Introductory Texts with Broad Coverage
In most cases, such texts create a general framework within which to view the numerical analysis of a wide variety of problems and numerical methods. This is usually done using the language of functional analysis; and often it is directed towards solving ordinary differential, partial differential, and integral equations.
K Atkinson and W Han. Theoretical Numerical Analysis , Springer-Verlag, New York,
L Kantorovich and G Akilov. Functional Analysis , 2 nd^ ed., Pergamon Press, New York,
L Collatz. Functional Analysis and Numerical Mathematics , Academic Press, New York, 1966. This contains a functional analysis framework for a variety of problems. Especially notable is the use of partially ordered functions spaces and norms whose values belong to the positive cone of a partially order space.
J-P Aubin. Applied Functional Analysis , 2nd^ ed., John Wiley, New York, 2000. This contains developments of tools for studying numerical methods for partial differential equations and also for optimization problems and convex analysis.
C. Books With a Sampling of Introductory Topics
For a classic look at numerical analysis, one that also give some “flavor” of the subject, see the following collection.
G Golub, ed. Studies in Numerical Analysis , Vol. 24, MAA Studies in Mathematics,
D. Major Journals and Serial Publications
numerical analysis appear regularly in SIAM Review , as mentioned earlier. Other SIAM journals also contain a number of articles related to numerical analysis.
Numerische Mathematik (ISSN 0029-599X) is the flagship journal in numerical analysis from Springer-Verlag. For a number of years, virtually all of its papers have been written in English. It has excellent articles in a wide variety of areas.
Mathematics of Computation (ISSN 0025-5718) is published by the American Mathematical Society, and it dates back to 1940; it is the oldest journal devoted to numerical computation. In addition to articles on numerical analysis, it also contains articles on computational number theory.
IMA Journal of Numerical Analysis (ISSN 0272-4979) is published by the British counterpart to SIAM, the Institute of Mathematics and Its Applications.
The following is only a partial list of the journals with a general coverage of numerical analysis. The list is alphabetical.
ACM Transactions on Mathematical Software (ISSN 0098-3500) is published by the ACM (Association for Computing Machinery), one of the major professional associations for computer science, especially within academia and research institutes. This journal deals principally with issues of implementation of numerical algorithms as computer software. It includes the Collected Algorithms of the ACM , a large collection of refereed numerical analysis software.
Advances in Computational Mathematics (ISSN1019-7168), Kluwer Academic Pub. This is of more recent origin. It considers all types of computational mathematics, although most papers are from numerical analysis.
BIT : Numerical Mathematics (ISSN 0006-3835) dates from 1961, and it is sponsored by numerical analysts in The Netherlands and the various Scandinavian countries. This is published by Swets & Zeitlinger of The Netherlands.
Computing (ISSN0010-485X), Springer-Verlag. This contains numerical analysis papers that are more computationally oriented or deal with computer science issues linked to numerical computation.
Electronic Transactions on Numerical Analysis (ISSN 1068-9613) began publication in 1993. It is a fully refereed journal that appears only in electronic form. Its URL is http://etna.mcs.kent.edu/ The journal is published by the Kent State University Library in conjunction with the Institute of Computational Mathematics at Kent State University, Kent, Ohio.
Foundations of Computational Mathematics (ISSN 1615-3375), Springer-Verlag. This is a new journal focusing on the interface of computer science with mathematics, especially numerical analysis and forms of computational mathematics.
Journal of Computational and Applied Mathematics (ISSN 0377-0427), Elsevier Science Pub. This is a popular journal, publishing a wide variety of papers, conference proceedings, and survey articles.
Journal of Computational Physics (ISSN 0021-9991), Academic press. This journal treats the computational aspects of physical problems, presenting techniques for the numerical solution of mathematical equations arising in all areas of physics. Many experimental numerical procedures are discussed here.
Numerical Algorithms (ISSN 1017-1398), Kluwer Academic Pub. The journal is devoted to all aspects of the study of numerical algorithms.
SIAM Journal on Scientific Computing (ISSN 1064-8275) , SIAM Pub. This is quite similar to the SIAM Journal on Numerical Analysis, but articles in it are to be tied more closely to actual numerical computation, with possibly more consideration given to questions of implementation.
E. Other Printed Resources
Handbook of Numerical Analysis , ed. by P. Ciarlet & J. Lions. This a multi-volume work, giving advanced level and extensive introductions to major topics in numerical analysis. Six volumes have been written to date, and most have connections to partial differential equations, with several main articles devoted to numerical analysis aspects of some problems from continuum mechanics. The publishers are Elsevier Science Pub.
Numerical Recipes, authored by W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Cambridge University Press. This is published in several versions based on different computer languages, including Fortran, C, and Pascal. This work is extremely wide- ranging in its coverage, including a sampling of essentially every area of numerical analysis. The book is popular among users of numerical analysis in the sciences and engineering, as it gives a quick and useful introduction to a topic, accompanied with computer codes. I recommend it as one possible introduction to a topic of interest, but I also recommend that it be followed by a more extensive introduction to examine additional nuances of the subject.
F. Online Resources
Numerical analysis was among the earliest of areas in mathematics to make extensive use of computers, and much of the early history of computing is linked to the intended application of the computers to solving problems involving numerical analysis. For that reason, it is not surprising that numerical analysts are at the leading edge as regards using computers to make resources available online, both reference material and computer
of variables that students first encounter in elementary algebra. The QR -method is another direct method, often used with ill-conditioned problems.
For larger linear systems, there are a variety of approaches, depending on the structure of the coefficient matrix A. Direct methods lead to a theoretically exact solution x in a finite number of steps, with Gaussian elimination the best-known example. There are errors, however, in the computed value of x due to rounding errors in the computation, arising from the finite length of numbers in standard computer arithmetic. Iterative methods are approximate methods that create a sequence of approximating solutions of increasing accuracy. Linear systems are categorized according to many properties (e.g. A may be symmetric about its main diagonal), and specialized methods have been developed for problems with these special properties.
Linear Algebra and its Applications (ISSN 0024-3795), North-Holland, Elsevier Science. Many articles involve questions of numerical linear algebra.
SIAM Journal on Matrix Analysis (ISSN 0895-4798), SIAM Pub. As the title implies, this specializes in numerical linear algebra.
G Golub and C Van Loan. Matrix Computations , 3rd^ ed., John Hopkins University Press, 1996. This was cited earlier in the introduction as giving a very good overview of the current state of numerical linear algebra.
N Higham. Accuracy and Stability of Numerical Algorithms , SIAM Pub., 1996. This contains an excellent discussion of error and stability analyses in numerical analysis, with numerical linear algebra a favored topic.
The following two texts furnish an excellent introduction to numerical linear algebra, suitable for use in teaching a first year graduate course.
J Demmel. Applied Numerical Linear Algebra, SIAM Pub., 1997.
L Trefethen and D Bau. Numerical Linear Algebra , SIAM Pub., 1997.
B Parlett. The Symmetric Eigenvalue Problem , Classics in Applied Mathematics, SIAM Pub., 1998. This is a reprint, with corrections, of a classic text that appeared in
J Wilkinson. The Algebraic Eigenvalue Problem , Oxford University Press, 1965. This is a classic text for numerical linear algebra, especially the eigenvalue problem, written by the Dean of researchers in this area.
There are many approaches to developing iterative methods for solving linear systems, and they usually depend heavily on the "structure" of the matrix in the system under consideration. Most linear systems solved using iteration are "sparse systems" in which most of the elements in the coefficient matrix are zero. Such systems arise commonly when discretizing partial differential equations in order to solve them numerically. As one consequence, there are many texts on iterative methods for linear systems, and we list only a few of them here. Work of the past two decades has been along two principal lines. First, there has been a generalization of the "Conjugate Gradient method", and this has led to what are called "Krylov subspace iterative methods". Second, work on solving discretizations of partial differential equations has led to what is called multigrid iteration. We list a few recent important texts on iterative methods.
A Greenbaum. Iterative Methods for Solving Linear Systems , SIAM Pub., 1997. This work discusses Krylov subspace methods, especially their application to solving discretizations of partial differential equations.
O Axelsson. Iterative Solution Methods , Cambridge University Press, 1994. This is an extensive treatment of most iterative methods, both classical methods that are still popular and more recently developed methods.
R Barrett, M Berry, T Chan, et al. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , SIAM Pub., 1994.
W Hackbusch. Iterative Solution of Large Sparse Systems of Equations , Springer- Verlag, 1994.
Y Saad. Iterative Methods for Sparse Linear Systems , PWS Pub., 1996. This contains a comprehensive introduction to iterative methods, including the use of parallel computing.
J Demmel, M Heath, and H van der Vorst. Parallel Numerical Linear Algebra , Acta Numerica (1993), pp. 111-198.
J Dongarra, I Duff, D Sorensen, and H van der Vorst. Numerical Linear Algebra for High-Performance Computers , SIAM Pub., 1998. Solving linear systems on high- performance computers requires use of special procedures to make optimal use of the computer. That is the focus of this book.
For introductions to the numerical solution of nonlinear systems, see the following texts.
J Ortega & W Rheinboldt. Iterative Solution of Nonlinear Equations in Several Variables , Academic Press, 1970. This is a classic text on the subject. It covers many types of methods in great detail, and it also presents the mathematical tools needed to work in this area.
C Kelley, Iterative Methods for Linear and Nonlinear Equations , SIAM Pub., 1995. This is an excellent introduction to the general area of solving nonlinear systems.
In addition, the solution of nonlinear systems for particular types of problems is often discussed in the intended area of application. For example, discretizations of nonlinear partial differential equations leads to special types of nonlinear systems, and special types of methods have been developed for solving these systems. The solution of such nonlinear systems is discussed at length in the literature for solving partial differential equations.
C. Optimization
An important related class of problems occurs under the heading of optimization, sometimes considered as a sub-area of "operations research". Given a real-valued function f ( x ) with x a vector of unknowns, we wish to find a value of x which minimizes f ( x ). In some cases x is allowed to vary freely, and in other cases there are constraints on the values of x that can be considered. Such problems occur frequently in business and engineering applications. This is an enormously popular area of research, with many new methods having been developed recently, both for classic problems such as that of linear programming and for previously unsolvable problems. In many ways, this area needs a chapter of its own; and in the classification scheme for Mathematical Reviews , it has a category (MR90) separate from that of numerical analysis (MR65).
For introductions with a strongly mathematical flavor, see the following, listed alphabetically. Most of these books also address the practical problems of implementation.
We begin our list of resources and references with a specialist journal; otherwise, the list is alphabetical.
SIAM Journal on Optimization (ISSN 1052-6234), SIAM Pub. An important journal for articles on optimization theory, especially from a numerical analysis perspective.
D Bertsekas. Nonlinear Programming , Athena Scientific Pub., 1995.
J Dennis and R Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, 1983. A classic text in the numerical analysis aspects of optimization theory.
R Fletcher. Practical Methods of Optimization , 2nd^ ed., John Wiley, 1987. A classic text.
P Gill, W Murray, and M Wright. Numerical Linear Algebra and Optimization , Vol. I, Addison-Wesley, 1991.
C Kelley, Iterative Methods for Optimization , SIAM Pub., 1999. A very useful introduction. It includes optimization of functions which are "noisy", meaning they are known subject to data noise of some kind, and it includes derivative-free methods, e.g. the Nelder-Mead algorithm.
D Luenberger. Linear and Nonlinear Programming , 2 nd^ ed., Addison-Wesley, 1984. A very nice introduction to the classical theory for optimization.
J Nocedal and S Wright, Numerical Optimization , Springer-Verlag, 1999. This is an up- to-date introduction that covers a wide variety of methods, including the important interior point methods. It also discusses the use of automatic differentiation, and it allows an easier implementation of methods that use derivatives of the function being optimized.
Y Ye. Interior Point Algorithms : Theory and Analysis , John Wiley, 1997. This is an advanced level text that covers all aspects of the use and analysis of interior point methods.
III. Approximation Theory
This category covers the approximation of functions and methods based on using such approximations. When evaluating a function f ( x ) with x a real or complex number, a computer or calculator can only do a finite number of numerical operations. Moreover, these operations are the basic arithmetic operations of addition, subtraction, multiplication, and division, together with comparison operations such as determining whether x > y is true or false. With the four basic arithmetic operations, we can evaluate polynomials and rational functions, polynomials divided by polynomials. Including the comparison operations, we can evaluate different polynomials or rational functions on different sets of real or complex numbers x. The evaluation of all other functions, e.g. f(x)=√ x or cos x , must be reduced to the evaluation of a polynomial or rational function that approximates the given function with sufficient accuracy. All function evaluations on calculators and computers are accomplished in this manner. This topic is known as approximation theory, and it is a well-developed area of mathematics.
A. General Approximation Theory
G Baker and P Graves-Morris. Pade Approximants , 2nd^ ed., Cambridge University Press, 1996.
T Rivlin. Chebyshev Polynomials , John Wiley, 1974.
G Szego. Orthogonal Polynomials , 3 rd^ ed., American Mathematical Society, 1967.
A Zygmund. Trigonometric Series , Vols. I and II, Cambridge University Press, 1959.
In the not too distant past, most approximation theory dealt with functions of one variable, whereas now there is a greater interest in functions of several variables. For example, see the following references to the current literature on multivariate approximation theory.
C de Boor. Multivariate Piecewise Polynomials , Acta Numerica (1993), pp. 65-110.
C Chui. Multivariate Splines , SIAM Pub., 1988.
M Sabin. Numerical Geometry of Surfaces , Acta Numerica (1994), pp. 411-466.
"Wavelets" furnish a means to combine the separate advantages of Fourier analysis and piecewise polynomial approximation. Although the first example of wavelets, the Haar function, goes back to 1910, much of the research on wavelets and their application is from 1980 onwards. Connected to wavelets is the idea of "multiresolution analysis", a decomposition of a function or a process into different levels of precision. Wavelets and multiresolution analysis is a very active area at present, and we give only some of the better known references on it.
C Chui. An Introduction to Wavelets , Academic Press, 1992.
I Daubechies. Ten Lectures on Wavelets , SIAM Pub., 1992.
R DeVore and B Lucier. Wavelets , Acta Numerica (1992), pp. 1-56. This gives an approximation theoretic perspective of wavelets and multiresolution. It is a tightly written article, and it has a good bibliography.
S Mallat. Multiresolution Approximation and Wavelet Orthonormal Bases of L^2 ( R ) , Transactions of the Amer. Math. Soc. 315 (1989), pp. 69-87.
Y Meyer. Wavelets and Operators , Cambridge University Press, 1992.
P Wojtaszczyk. A Mathematical Introduction to Wavelets , Cambridge University Press, 1997.
B. Interpolation Theory
One method of approximation is called interpolation. Consider being given a finite set of points ( xi , yi ) in the xy -plane; then find a polynomial p ( x ) whose graph passes through the given points, p ( xi )= yi. The polynomial p ( x ) is said to interpolate the given data points. Interpolation can be performed with functions other than polynomials (although these are the most popular category of interpolating functions), with important cases being rational functions, trigonometric polynomials, and spline functions. Interpolation has a number of applications. If a function f ( x ) is known only at a discrete set of n data points, then interpolation can be used to extend the definition to nearby points x. If n is even moderately large, then spline functions are preferable to polynomials for this purpose. Spline functions are smooth piecewise polynomial functions with minimal oscillation as regards interpolation, and they are used commonly in computer graphics, statistics, and other applications.
Good introductions to polynomial interpolation theory for functions of one variable are given in most introductory textbooks on numerical analysis, e.g. the texts cited earlier in the Introduction and under I-A.
For multivariable polynomial interpolation, most presentations are given in association with the intended area of application. This includes applications to the finite element discretization of partial differential equations, the numerical solution of integral equations, and the construction of surfaces in computer graphics. We give only a few examples.
S Brenner and R Scott. The Mathematical Theory of Finite Element Methods , Springer-Verlag, 1994. This contains many results on the theory of multivariable approximation using polynomials.
G Strang and G Fix. An Analysis of the Finite Element Method , Prentice-Hall, 1973. This is a classic text for the finite element method, and it contains useful information on multivariable polynomial interpolation and approximation.
P Lancaster and K Salkaukas. Curve and Surface Fitting : An Introduction , Academic Press, 1986. This gives multivariate interpolation over varied regions in the context of computer graphics.
D Laurie and R Cools (eds). Numerical Evaluation of Integrals , Journal of Computational and Applied Mathematics 112 , 1999, no. 1-2. A collection of papers on the current state of the art in numerical integration.
For some of the current literature on multivariate numerical integration, see the following.
A Stroud. Approximate Calculation of Multiple Integrals , Prentice-Hall, 1971. This is an old book, but is still the main reference for multivariate numerical integration.
R Cools. Constructing Cubature Formulae : The Science Behind the Art , Acta Numerica (1997), pp. 1-54. This gives a current perspective on constructing multivariate numerical integration formulas.
H Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods , SIAM Pub., 1992.
I Sloan and S Joe. Lattice Methods for Multiple Integration , Oxford University Press,
IV. Solving Differential and Integral Equations
Most mathematical models used in the natural sciences and engineering are based on ordinary differential equations, partial differential equations, and integral equations. The reader should also refer to the chapter on differential equations for information on some of these topics.
The numerical methods for these equations are primarily of two types. The first type approximates the unknown function in the equation by a simpler function, often a polynomial or piecewise polynomial function, choosing it so as to satisfy the original equation approximately. Among the best known of such methods is the "finite element method" for solving partial differential equations. Such methods are often called "projection methods" because of the tools used in the underlying mathematical theory. The second type of numerical method approximates the derivatives or integrals in the equation of interest, generally solving approximately for the solution function at a discrete set of points. Most initial value problems for ordinary differential equations and partial differential equations are solved in this way, and the numerical procedures are often called "finite difference methods", primarily for historical reasons.
The same subdivision of methods also applies to the numerical solution of integral equations, although the names differ somewhat. Most numerical methods for solving differential and integral equations also involve both approximation theory and the solution of quite large linear and/or nonlinear systems.
A. Ordinary Differential Equations
There are two principal types of problems associated with ordinary differential equations (ODE): the initial value problem and the boundary value problem. The numerical methods are quite different for these two types of problems, although many of the same tools from approximation theory are used in designing numerical methods for them. In recent years, there has also been much work on specialized forms of ODE. There are many books available for these problems, and many of them cover more than one problem.
U Ascher, R Mattheij, and R Russell. Numerical Solution of Boundary Value Problems for Ordinary Differential Equations , Prentice-Hall, 1988. This is a classic text on this topic.
U Ascher and L Petzold. Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations , SIAM Pub., 1998.
K Brenan, S Campbell, and L Petzold. Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations , 2nd^ SIAM Pub., 1996. Differential-algebraic equations have become an important form of mathematical model for many problems in mechanical engineering. One solves differential equations subject to algebraic constraints on the unknowns.
K Burrage. Parallel and Sequential Methods for Ordinary Differential Equations , Oxford University Press, 1995. An introductory presentation of solving ODE on parallel and serial computers.
J Butcher. The Numerical Analysis of Ordinary Differential Equations , John Wiley,
C Gear. Numerical Initial Value Problems in Ordinary Differential Equations , Prentice- Hall, 1971. This is the classic work which initiated the study of variable stepsize, variable order methods.
E Hairer, S Norsett, and G Wanner. Solving Ordinary Differential Equations - I: Nonstiff Problems , 2 nd^ ed., Springer-Verlag, 1993. This and the following volume give a complete coverage of the modern theory of the numerical analysis of initial value problems for ordinary differential equations.
E Hairer and G Wanner. Solving Ordinary Differential Equations - II: Stiff and Differential-Algebraic Problems , 2nd^ ed., Springer-Verlag, 1996.
A Iserles. A First Course in the Numerical Analysis of Differential Equations , Cambridge University Press, 1996. This was cited earlier in the introduction as giving a