Fundamentals of Numerical Mathematics for Physicists and Engineers. Alvaro Meseguer

, usually called the residuals. Since this quantity should approach zero when converges to a root, the reader may wonder why should not be used as a measure for stopping the iterations. The reason is that the residual is not always a reliable measure (although it is often used in practice to decide convergence). We will address this question later in this chapter, when we introduce the concept of condition number of a root.

1.3 Newton's Method

Newton's method, also known as Newton–Raphson method,⁷ is probably the most important root‐finding algorithm of numerical mathematics. Its formulation naturally leads to a wide variety of other root‐finding algorithms and is easily adaptable to solving systems of nonlinear equations.⁸ To formulate this method we will assume that is differentiable and that the root is close to some initial guess . To get a better estimation of the root we assume that, close to , the function can be approximated by its tangent line at the point plotted in gray below. In other words, we locally approximate by its first order Taylor expansion at :

      (1.6)

where is the value of the derivative of at . The motivation for approximating by is simple: if is a reasonably good approximation of , then the solution of should be a reasonable good estimation of the solution of . Let be the solution of satisfying

      (1.7)

      The resulting abscissa where intercepts the ‐axis is the new (hopefully more accurate) estimation of the root. The process can be repeated approximating by its Taylor expansion at (straight gray line below the curve) in order to obtain an even better estimation satisfying

      (1.8)

      and so on, leading to the iterative formula:

      Newton's Iteration:

      (1.9)

      A simple implementation of Newton's method can be found in the next code, whose structure essentially differs from the one seen in the bisection in two main aspects. First, the code needs an initial guess as starting point, instead of an interval. Second, in addition to the string name corresponding to the M‐file function fun, the code also needs the one associated with in the argument dfun. Providing the M‐file for the exact derivative entails both analytical differentiation and its subsequent programming, both being error‐prone tasks. While this is the correct procedure, it is sometimes advisable to let the computer do the job by approximating the value of by the ratio

(1.10)

where is a small increment ( in our code). This approximation of the derivative, along with the suitability of the chosen , will be properly justified later in the chapter devoted to numerical differentiation.⁹ The reader may check that there are no noticeable differences when using either the exact or the approximate derivative (1.10).

      % Code 2: Newton's method for solving f(x) = 0 % Input: 1. a: initial guess % 2. tol: tolerance so that abs(x_k+1 - x_k) < tol % 3. itmax: maximum number of iterations allowed % 4. fun: function's name % 5. dfun: derivative's name % (If dfun = ‘0’, then f'(x) is approximated) % Output: 1. xk: resulting sequence % 2. res: resulting residuals % 3. it: number of required iterations function [xk,res,it] = newton(a,tol,itmax,fun,dfun) xk = [a]; fk = feval(fun,xk); res = abs(fk); it = 0; tolk = res(1); dx = 1e-8 ; while it < itmax & tolk > tol if dfun == ‘0’ dfk = (feval(fun,xk(end)+dx)-fk)/dx; else dfk = feval(dfun,xk(end)); end xk = [xk, xk(end) - fk/dfk]; fk = feval(fun,xk(end)); res = [res abs(fk)]; tolk = abs(xk(end)-xk(end-1)); it = it + 1; end

Let us compare the performance of Newton's method with the bisection by applying both algorithms to solve the