11 - Taylor series
3.28M subscribers
SUBSCRIBED
Taylor polynomials are incredibly powerful for approximations, and Taylor series can give new ways to express functions. Brought to you by you: http://3b1b.co/eoc1-thanksHome page: https://www.3blue1brown.com/Full series: http://3b1b.co/calculusSeries like this one are funded largely by the community, through Patreon, where supporters get early access as the series is being produced.http://3b1b.co/support------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted about new videos, subscribe, and click the bell to receive notifications (if you're into that). If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommendedVarious social media stuffs: Website: https://www.3blue1brown.comTwitter: https://twitter.com/3Blue1BrownPatreon: https://patreon.com/3blue1brownFacebook: https://www.facebook.com/3blue1brownReddit: https://www.reddit.com/r/3Blue1Brown
00:15
When I first learned about Taylor series, I definitely didn’t appreciate how important
00:19
they are. But time and time again they come up in math,
00:22
physics, and many fields of engineering because they’re one of the most powerful tools that
00:26
math has to offer for approximating functions.
00:29
One of the first times this clicked for me as a student was not in a calculus class,
00:34
but in a physics class. We were studying some problem that had to
00:37
do with the potential energy of a pendulum, and for that you need an expression for how
00:42
high the weight of the pendulum is above its lowest point, which works out to be proportional
00:48
to one minus the cosine of the angle between the pendulum and the vertical.
00:54
The specifics of the problem we were trying to solve are beyond the point here, but I’ll
00:57
just say that this cosine function made the problem awkward and unwieldy.
01:07
But by approximating cos(theta) as 1 - theta2/2, of all things, everything fell into place
01:15
much more easily. If you’ve never seen anything like this
01:18
before, an approximation like that might seem completely out of left field.
01:23
If you graph cos(theta) along with this function 1 - theta2/2, they do seem rather close to
01:30
each other for small angles near 0, but how would you even think to make this approximation?
01:36
And how would you find this particular quadratic? The study of Taylor series is largely about
01:43
taking non-polynomial functions, and finding polynomials that approximate them near some
01:48
input. The motive is that polynomials tend to be
01:51
much easier to deal with than other functions: They’re easier to compute, easier to take
01:56
derivatives, easier to integrate...they’re just all around friendly.
02:00
So let’s look at the function cos(x), and take a moment to think about how you might
02:06
find a quadratic approximation near x = 0. That is, among all the polynomials that look
02:14
c0 + c1x + c2x2 for some choice of the constants c0, c1 and c2, find the one that most resembles
02:25
cos(x) near x=0; whose graph kind of spoons with the graph of cos(x) at that point.
02:33
Well, first of all, at the input 0 the value of cos(x) is 1, so if our approximation is
02:40
going to be any good at all, it should also equal 1 when you plug in 0. Plugging in 0
02:46
just results in whatever c0 is, so we can set that equal to 1.
02:53
This leaves us free to choose constant c1 and c2 to make this approximation as good
02:57
as we can, but nothing we do to them will change the fact that the polynomial equals
03:02
1 at x=0. It would also be good if our approximation
03:07
had the same tangent slope as as cos(x) at this point of interest. Otherwise, the approximation
03:13
drifts away from the cosine graph even fro value of x very close to 0.
03:18
The derivative of cos(x) is -sin(x), and at x=0 that equals 0, meaning its tangent line
03:24
is flat. Working out the derivative of our quadratic,
03:30
you get c1 + 2c2x. At x=0 that equals whatever we choose for c1. So this constant c1 controls
03:42
the derivative of our approximation around x=0. Setting it equal to 0 ensures that our
03:48
approximation has the same derivative as cos(x), and hence the same tangent slope.
03:53
This leaves us free to change c2, but the value and slope of our polynomial at x=0 are
03:59
locked in place to match that of cos(x).
04:06
The cosine graph curves downward above x=0, it has a negative second derivative. Or in
04:13
other words, even though the rate of change is 0 at that point, the rate of change itself
04:18
is decreasing around that point. Specifically, since its derivative is -sin(x)
04:25
its second derivative is -cos(x), so at x=0 its second derivative is -1.
04:32
In the same way that we wanted the derivative of our approximation to match that of cosine,
04:37
so that their values wouldn’t drift apart needlessly quickly, making sure that their
04:42
second derivatives match will ensure that they curve at the same rate; that the slope
04:47
of our polynomial doesn’t drift away from the slope of cos(x) any more quickly than
04:52
it needs to. Pulling out that same derivative we had before,
04:56
then taking its derivative, we see that the second derivative of this polynomial is exactly
05:02
2c2, so to make sure this second derivative also equals -1 at x=0, 2c2 must equal -1,
05:13
meaning c2 itself has to be -½. This gives us the approximation 1 + 0x - ½
05:21
x2.
05:22
To get a feel for how good this is, if you estimated cos(0.1) with this polynomial, you’d
05:30
get 0.995. And this is the true value of cos(0.1). It’s a really good approximation.
05:40
Take a moment to reflect on what just happened. You had three degrees of freedom with a quadratic
05:45
approximation, the constants c0, c1, and c2. c0 was responsible for making sure that the
05:51
output of the approximation matches that of cos(x) at x=0, c1 was in charge of making
05:58
sure the derivatives match at that point, and c2 was responsible for making sure the
06:06
second derivatives match up. This ensures that the way your approximation
06:11
changes as you move away from x=0, and the way that the rate of change itself changes,
06:17
is as similar as possible to behavior of cos(x), given the amount of control you have.
06:24
You could give yourself more control by allowing more terms in your polynomial, and matching
06:28
higher order derivatives of cos(x). For example, add on the term c3x3 for some
06:35
constant c3. If you take the third derivative of a cubic
06:41
polynomial, anything quadratic or smaller goes to 0.
06:45
As for that last term, after three iterations of the power rule it looks like 123*c3.
06:54
On the other hand, the third derivative of cos(x) is sin(x), which equals 0 at x=0, so
07:04
to make the third derivatives match, the constant c3 should be 0.
07:09
In other words, not only is 1 - ½ x2 the best possible quadratic approximation of cos(x)
07:16
around x=0, it’s also the best possible cubic approximation.
07:21
You can actually make an improvement by adding a fourth order term, c4x4. The fourth derivative
07:28
of cos(x) is itself, which equals 1 at x=0. And what’s the fourth derivative of our
07:35
polynomial with this new term? Well, when you keep applying the power rule over and
07:41
over, with those exponents all hopping down front, you end up with 1234c4, which is
07:49
24c4 So if we want this to match the fourth derivative
07:53
of cos(x), which is 1, c4 must be 1/24. And indeed, the polynomial 1 - ½ x2 + 1/24
08:05
x4, which looks like this, is a very close approximation for cos(x) around x = 0.
08:13
In any physics problem involving the cosine of some small angle, for example, predictions
08:18
would be almost unnoticeably different if you substituted this polynomial for cos(x).
08:24
Now, step back and notice a few things about this process.
08:30
First, factorial terms naturally come up in this process.
08:35
When you take n derivatives of xn, letting the power rule just keep cascading, what you’re
08:43
left with is 123 and on up to n. So you don’t simply set the coefficients
08:51
of the polynomial equal to whatever derivative value you want, you have to divide by the
08:55
appropriate factorial to cancel out this effect. For example, that x4 coefficient is the fourth
09:02
derivative of cosine, 1, divided by 4 factorial, 24.
09:09
The second thing to notice is that adding new terms, like this c4x4, doesn’t mess
09:16
up what old terms should be, and that’s important.
09:20
For example, the second derivative of this polynomial at x = 0 is still equal to 2 times
09:26
the second coefficient, even after introducing higher order terms to the polynomial.
09:31
And it’s because we’re plugging in x=0, so the second derivative of any higher order
09:36
terms, which all include an x, will wash away. The same goes for any other derivative, which
09:43
is why each derivative of a polynomial at x=0 is controlled by one and only one coefficient.
09:52
If instead you were approximating near an input other than 0, like x=pi, in order to
09:58
get the same effect you would have to write your polynomial in terms of powers of (x - pi),
10:03
or whatever input you’re looking at. This makes it look notably more complicated,
10:09
but all it’s doing is making the point pi look like 0, so that plugging in x = pi will
10:17
result in a lot of nice cancelation that leaves only one constant.
10:22
And finally, on a more philosophical level, notice how what we’re doing here is essentially
10:27
taking information about the higher order derivatives of a function at a single point,
10:33
and translating it into information about the value of that function near that point.
10:39
We can take as many derivatives of cos(x) as we want, it follows this nice cyclic pattern
10:46
cos(x), -sin(x), -cos(x), sin(x), and repeat. So the value of these derivative of x=0 have
10:56
the cyclic pattern 1, 0, -1, 0, and repeat. And knowing the values of all those higher-order
11:04
derivatives is a lot of information about cos(x), even though it only involved plugging
11:09
in a single input, x=0. That information is leveraged to get an approximation
11:18
around this input by creating a polynomial whose higher order derivatives, match up with
11:25
those of cos(x), following this same 1, 0, -1, 0 cyclic pattern.
11:31
To do that, make each coefficient of this polynomial follow this same pattern, but divide
11:37
each one by the appropriate factorial, like I mentioned before, so as to cancel out the
11:42
cascading effects of many power rule applications. The polynomials you get by stopping this process
11:49
at any point are called “Taylor polynomials” for cos(x) around the input x=0.
11:53
More generally, and hence more abstractly, if we were dealing with some function other
11:58
than cosine, you would compute its derivative, second derivative, and so on, getting as many
12:04
terms as you’d like, and you’d evaluate each one at x=0.
12:09
Then for your polynomial approximation, the coefficient of each xn term should be the
12:16
value of the nth derivative of the function at 0, divided by (n!).
12:23
This rather abstract formula is something you’ll likely see in any text or course
12:29
touching on Taylor polynomials. And when you see it, think to yourself that
12:34
the constant term ensures that the value of the polynomial matches that of f(x) at x=0,
12:38
the next term ensures that the slope of the polynomial matches that of the function, the
12:46
next term ensure the rate at which that slope changes is the same, and so on, depending
12:52
on how many terms you want. The more terms you choose, the closer the
12:57
approximation, but the tradeoff is that your polynomial is more complicated.
13:02
And if you want to approximate near some input a other than 0, you write the polynomial in
13:11
terms of (x-a) instead, and evaluate all the derivatives of f at that input a.
13:19
This is what Taylor series look like in their fullest generality. Changing the value of
13:24
a changes where the approximation is hugging the original function; where its higher order
13:30
derivatives will be equal to those of the original function.
13:34
One of the simplest meaningful examples is ex, around the input x=0. Computing its derivatives
13:44
is nice, since the derivative of ex is itself, so its second derivative is also ex, as is
13:52
its third, and so on. So at the point x=0, these are all 1. This
13:59
means our polynomial approximation looks like 1 + x + ½ x2 + 1/(3!) x3 + 1/(4!) x4, and
14:16
so on, depending on how many terms you want. These are the Taylor polynomials
14:28
for ex.
14:29
In the spirit of showing you just how connected the topics of calculus are, let me turn to
14:34
a completely different way to understand this second order term geometrically. It’s related
14:41
to the fundamental theorem of calculus, which I talked about in chapters 1 and 8.
14:48
Like we did in those videos, consider a function that gives the area under some graph between
14:53
a fixed left point and a variable right point. What we’re going to do is think about how
14:58
to approximate this area function, not the function for the graph like we were doing
15:03
before. Focusing on that area is what will make the second order term pop out.
15:10
Remember, the fundamental theorem of calculus is that this graph itself represents the derivative
15:16
of the area function, and as a reminder it’s because a slight nudge dx to the right bound
15:21
on the area gives a new bit of area approximately equal to the height of the graph times dx,
15:28
in a way that’s increasingly accurate for smaller choice of dx.
15:33
So df over dx, the change in area divided by that nudge dx, approaches the height of
15:34
the graph as dx approaches 0. But if you wanted to be more accurate about
15:37
the change to the area given some change to x that isn’t mean to approach 0, you would
15:42
take into account this portion right here, which is approximately a triangle.
15:48
Let’s call the starting input a, and the nudged input above it x, so that this change
15:56
is (x-a). The base of that little triangle is that change
16:01
(x-a), and its height is the slope of the graph times (x-a). Since this graph is the
16:10
derivative of the area function, that slope is the second derivative of the area function,
16:16
evaluated at the input a. So the area of that triangle, ½ base times
16:20
height, is one half times the second derivative of the area function, evaluated at a, multiplied
16:27
by (x-a)2. And this is exactly what you see with Taylor
16:34
polynomials. If you knew the various derivative information about the area function at the
16:39
point a, you would approximate this area at x to be the area up to a, f(a), plus the area
16:50
of this rectangle, which is the first derivative times (x-a), plus the area of this triangle,
16:58
which is ½ (the second derivative) * (x - a)2. I like this, because even though it looks
17:05
a bit messy all written out, each term has a clear meaning you can point to on the diagram.
17:14
We could call it an end here, and you’d have you’d have a phenomenally useful tool
17:17
for approximations with these Taylor polynomials. But if you’re thinking like a mathematician,
17:24
one question you might ask is if it makes sense to never stop, and add up infinitely
17:29
many terms. In math, an infinite sum is called a “series”,
17:35
so even though one of the approximations with finitely many terms is called a “Taylor
17:39
polynomial” for your function, adding all infinitely many terms gives what’s called
17:43
a “Taylor series”. Now you have to be careful with the idea of
17:47
an infinite series, because it doesn’t actually make sense to add infinitely many things;
17:53
you can only hit the plus button on the calculator so many times.
17:57
But if you have a series where adding more and more terms gets you increasingly close
18:04
to some specific value, you say the series converges to that value. Or, if you’re comfortable
18:11
extending the definition of equality to include this kind of series convergence, you’d say
18:16
the series as a whole, this infinite sum, equals the value it converges to.
18:23
For example, look at the Taylor polynomials for ex, and plug in some input like x = 1.
18:31
As you add more and more polynomial terms, the total sum gets closer and closer to the
18:37
value e, so we say that the infinite series converges to the number e. Or, what’s saying
18:44
the same thing, that it equals the number e.
18:46
In fact, it turns out that if you plug in any other value of x, like x=2, and look at
18:54
the value of higher and higher order Taylor polynomials at this value, they will converge
19:00
towards ex, in this case e2. This is true for any input, no matter how
19:07
far away from 0 it is, even though these Taylor polynomials are constructed only from derivative
19:13
information gathered at the input 0. In a case like this, we say ex equals its
19:22
Taylor series at all inputs x, which is kind of a magical thing to have happen.
19:28
Although this is also true for some other important functions, like sine and cosine,
19:34
sometimes these series only converge within a certain range around the input whose derivative
19:39
information you’re using. If you work out the Taylor series for the
19:43
natural log of x around the input x = 1, which is built from evaluating the higher order
19:49
derivatives of ln(x) at x=1, this is what it looks like.
19:56
When you plug in an input between 0 and 2, adding more and more terms of this series
20:01
will indeed get you closer and closer to the natural log of that input.
20:06
But outside that range, even by just a bit, the series fails to approach anything.
20:12
As you add more and more terms the sum bounces back and forth wildly, it does not approaching
20:20
the natural log of that value, even though the natural log of x is perfectly well defined
20:26
for inputs above 2. In some sense, the derivative information
20:30
of ln(x) at x=1 doesn’t propagate out that far.
20:36
In a case like this, where adding more terms of the series doesn’t approach anything,
20:41
you say the series diverges. And that maximum distance between the input
20:46
you’re approximating near, and points where the outputs of these polynomials actually
20:50
do converge, is called the “radius of convergence” for the Taylor series.
20:57
There remains more to learn about Taylor series, their many use cases, tactics for placing
21:01
bounds on the error of these approximations, tests for understanding when these series
21:06
do and don’t converge. For that matter there remains more to learn
21:10
about calculus as a whole, and the countless topics not touched by this series.
21:15
The goal with these videos is to give you the fundamental intuitions that make you feel
21:19
confident and efficient learning more on your own, and potentially even rediscovering more
21:25
of the topic for yourself. In the case of Taylor series, the fundamental
21:31
intuition to keep in mind as you explore more is that they translate derivative information
21:36
at a single point to approximation information around that point.
21:47
The next series like this will be on probability, and if you want early access as those videos
21:51
are made, you know where to go.