Newton's method

Newton's method, also known as the Newton-Raphson method, is a powerful root-finding algorithm. In mathematics, a root is a value that makes a function equal to zero. These roots are also called zeroes. Finding these points is essential for solving complex equations in science and engineering. The method works by creating a sequence of increasingly accurate approximations. It turns a difficult curve into a series of simple, straight lines.

The mechanism relies on a process called iteration. To begin, you start with an initial guess, which we call $x_0$. This guess should be a real number close to the actual root. Next, you find the derivative of the function at that point. The derivative tells you the slope of the tangent line. A tangent line is a straight line that just touches the curve at your guess.

Once you have the tangent line, you find where it crosses the x-axis. This intersection point becomes your next, improved guess, $x_1$. You then repeat this entire cycle using the new value. You calculate a new tangent line at $x_1$ and find its x-intercept. This process creates a chain of values that move closer to the true root.

If the initial guess is close enough, the method converges very quickly.

Newton's method belongs to a larger class of algorithms called Householder's methods. It is often succeeded by Halley's method, which can be even faster. There are different types of convergence based on how the error decreases. For a root with a multiplicity of one, the method shows quadratic convergence. This means the number of correct decimal digits roughly doubles with each step. However, if the root has a multiplicity greater than one, the convergence becomes linear. This means the error only reduces by a constant factor each time.

The history of this idea spans thousands of years. During the Old Babylonian period, between the 19th and 16th centuries BCE, people used a similar technique. They used it to approximate the sides of squares. This is often called Heron's method after Hero of Alexandria. In 1427, Jamshīd al-Kāshī published a method in his book *Miftāḥ al-Ḥisāb*. His work was based on earlier scholars like al-Bīrūnī and Sharaf al-Dīn al-Ṭūsī.

Modern versions of the method emerged much later in Europe. Isaac Newton wrote about his ideas between 1669 and 1671. Joseph Raphson later provided a simplified description in 1690. Raphson's version was helpful because it avoided tedious rewriting processes. He used a reusable iterative expression for polynomials. In 1740, Thomas Simpson described the method for general nonlinear equations. He also showed how it could be used for systems of equations.

Despite its power, the method has practical limitations. One difficulty is calculating the derivative. Sometimes an analytical expression for the derivative is too expensive to evaluate. In these cases, mathematicians might use the secant method instead. The secant method is slower but easier to compute. Another problem is the failure to converge. If the derivative is not well-behaved near the root, the method might oscillate.

This can cause the guesses to jump back and forth without ever hitting the target.

Newton's method is connected to many broader fields of study. It can be extended to complex functions and systems of equations. Arthur Cayley first studied the difficulties of using the method with complex roots in 1879. This research helped start the study of the iteration of rational functions. Today, the method remains a fundamental tool in numerical analysis. It helps us solve the optimization problems that define much of modern mathematics.