04. Eigenvalues and Eigenvectors
2.71M subscribers
SUBSCRIBED
MIT 18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Spring 2018 Instructor: Gilbert Strang View the complete course: https://ocw.mit.edu/18-065S18YouTube Playlist: https://www.youtube.com/playlist?list...Professor Strang begins this lecture talking about eigenvectors and eigenvalues and why they are useful. Then he moves to a discussion of symmetric matrices, in particular, positive definite matrices. License: Creative Commons BY-NC-SA More information at https://ocw.mit.edu/termsMore courses at https://ocw.mit.edu
00:00
00:01
The following content is provided under a Creative
00:03
Commons license.
00:05
Your support will help MIT OpenCourseWare
00:07
continue to offer high quality educational resources for free.
00:11
To make a donation, or to view additional materials
00:14
from hundreds of MIT courses, visit MIT OpenCourseWare
00:18
at ocw.mit.edu.
00:19
00:22
GILBERT STRANG: So last time was orthogonal matrices--
00:27
Q. And this time is symmetric matrices, S.
00:32
So we're really talking about the best matrices of all.
00:37
Well, I'll start with any square matrix and about eigenvectors.
00:43
But you've heard of eigenvectors more than once--
00:47
more than twice--
00:48
more than 10 times, probably.
00:50
OK.
00:51
So eigenvectors.
00:54
And then, let's be sure we know why they're useful,
00:58
and maybe compute one or two.
01:02
But then we'll move to symmetric matrices
01:05
and what is special about those.
01:08
And then, even more special and more important
01:12
will be positive definite symmetric matrices--
01:16
so that when I say, positive definite, I mean symmetric.
01:20
So start with A. Next comes S. Then come the special S--
01:26
special symmetric matrices that have
01:29
this extra positive definite property.
01:31
OK.
01:33
So start with A. So an eigenvector--
01:39
if I multiply A by x, I get some vector.
01:44
And sometimes, if x is especially chosen well,
01:51
Ax comes out in the same direction as x.
01:55
Ax comes out some number times x.
02:01
So there are-- normally, there would be,
02:04
for an n by n matrix--
02:06
so let's say A is n by n today.
02:13
Normally, if we live right, there
02:16
will be n different independent vectors--
02:21
x eigenvectors-- that have this special property.
02:26
And we can compute them by hand if n is 2 or 3--
02:32
2, mostly.
02:35
But the computation of the x's and the lambdas--
02:38
so this is for i equal 1 up to n,
02:46
if I use this sort of math shorthand--
02:49
that I have n of these almost always.
02:53
And my first question is, what are they good for?
02:59
Why does course after course introduce eigenvectors?
03:05
And to me the key property is seen by looking at A squared.
03:12
So let me look at A squared.
03:13
03:17
So it's another n by n matrix.
03:20
And we would ask, suppose we know these guys?
03:24
Suppose we've found those somehow.
03:27
What about A squared?
03:30
Is x an eigenvector of A squared also?
03:35
Well, the way to find out is to multiply A squared by x,
03:39
and see what happens.
03:41
Do you see what's going to happen here?
03:43
This is A times Ax, which is A times--
03:50
Ax is lambda x--
03:54
and now what do I do now?
03:58
Because I'm shooting for the answer yes.
04:01
X is an eigenvector of A squared also.
04:04
So what do I do?
04:05
That number-- that lambda is just a number.
04:09
I can put it anywhere I like.
04:10
So I can put it out front.
04:13
And then I have Ax, which is?
04:16
AUDIENCE: Lambda x.
04:17
GILBERT STRANG: Lambda x.
04:18
Thanks.
04:18
So I have another lambda x.
04:19
So there's lambda squared x.
04:21
So I learned the crucial thing here--
04:24
that x is also an eigenvector of A squared,
04:28
and the eigenvalue is lambda squared.
04:33
And of course, I can keep going.
04:36
So A to the nth--
04:39
x is lambda to the nth x.
04:42
We have found the right vectors for that particular matrix A.
04:46
What about A inverse x?
04:49
That will be-- if everything is good--
04:53
1 over lambda x.
04:57
Well, yeah.
04:58
So anytime I write 1 over lambda,
05:01
my mind says, you gotta make some comment
05:07
on the special case where it doesn't work, which is?
05:11
AUDIENCE: Lambda is not equal to 0.
05:13
GILBERT STRANG: Yeah.
05:14
If lambda is not 0, I'm golden.
05:17
If lambda is 0, it doesn't look good.
05:22
And what's happening if lambda is 0?
05:26
AUDIENCE: A inverse [INAUDIBLE].
05:27
GILBERT STRANG: A doesn't even have an inverse.
05:29
If lambda was 0--
05:32
which it could be--
05:33
no rule against it.
05:36
If lambda was 0, this would say, A times the eigenvector
05:40
is 0 times the eigenvector.
05:42
So that would tell me that the eigenvector
05:44
is in the null space.
05:47
It would tell me the matrix A isn't invertible.
05:51
It's taking some vector x to 0.
05:55
And so everything clicks.
05:58
This works when it should work.
06:01
And if we have other fun-- any function of the matrix,
06:05
we could define the exponential of a matrix.
06:08
18.03 would do that.
06:10
Let's just write it down, as if we know what it means.
06:15
Does it have the same eigenvector?
06:17
Well, sure.
06:18
Because e to the At--
06:20
the exponential of a matrix--
06:22
if I see e to the something--
06:25
I think of that long, infinite series
06:27
that gives the exponential.
06:29
Those-- all the terms in that series have powers of A.
06:34
So everything is working.
06:36
Every term in that series--
06:38
x is an eigenvector.
06:39
And when I put it all together, I
06:41
learn that the eigenvalue is e to the lambda t.
06:47
That's just a typical and successful work use.
06:54
OK.
06:55
So that's eigenvectors and eigenvalues,
06:58
and we'll find some in a minute.
07:02
07:05
Now, so I'm claiming that this--
07:07
07:11
that from this first thing-- which was just about certain
07:16
vectors are special--
07:17
now we're beginning to see why they're useful.
07:21
So special is good.
07:22
Useful is even better.
07:25
So let me take any vector, say v. And OK,
07:37
what do I want to do?
07:39
I want to use eigenvectors.
07:42
This v is probably not an eigenvector.
07:45
But I'm supposing that I've got n of them.
07:49
You and I are agreed that there are
07:51
some matrices for which there are not
07:54
a full set of eigenvectors.
07:57
That's really the main sort of annoying point
08:02
in the whole subject of linear algebra,
08:05
is some matrices don't have enough eigenvectors.
08:08
But almost all do, and let's go forward
08:14
assuming our matrix has.
08:16
OK.
08:17
So if I've got n independent eigenvectors, that's a basis.
08:22
I can write any vector v as a combination
08:26
of those eigenvectors.
08:30
Right.
08:33
And then I can find out what A to any power.
08:40
So that's the point.
08:42
This is going to be the simple and reason why
08:49
we like to have--
08:50
we like to know the eigenvectors.
08:52
Because if I choose those as my basis vectors,
08:56
v is a combination of them.
08:58
Now if I multiply by A, or A squared, or A to the k power,
09:04
then it's linear.
09:05
So I can multiply each one by A to the k.
09:08
And what do I get if I multiply that guy by A to the kth power?
09:15
OK.
09:16
Well, I'm just going to use-- or, here
09:18
I said n, but let me say k.
09:21
Because n-- I'm sorry.
09:23
I'm using n for the size of the matrix.
09:26
So I better use k for the typical case here.
09:31
So what do I get?
09:34
Just help me through this and we're happy.
09:39
So what happens when I multiply that by A to the k?
09:44
It's an eigenvector, remember, so when I
09:47
multiply by A to the k, I get?
09:49
AUDIENCE: C1.
09:50
GILBERT STRANG: C1.
09:51
That's just a number.
09:53
And A to the k times that eigenvector gives?
09:56
AUDIENCE: Lambda 1.
09:57
GILBERT STRANG: Lambda 1 to the k times the eigenvector.
10:00
10:03
Right?
10:05
That's the whole point.
10:07
And linearity says keep going.
10:11
Cn, lambda n to the kth power, Xn.
10:18
In other words, I can take--
10:22
I can apply any power of a matrix.
10:24
I can apply the exponential of a matrix.
10:27
I can do anything quickly, because I've
10:33
got the eigenvector.
10:35
So really, I'm saying the first use
10:38
for eigenvectors-- maybe the principle use for which they
10:41
were invented-- is to be able to solve difference equations.
10:47
So if I call that Vk--
10:51
the kth power-- then the equation I'm solving here
10:55
is a one step difference equation.
11:01
This is my difference equation.
11:03
And if I wanted to use exponentials,
11:06
the equation I would be solving would be dv, dt equal Av.
11:12
11:17
Solution to discrete steps, or continuous time evolution comes
11:29
is trivial, if I know the eigenvectors.
11:32
Because here is the solution to this one.
11:35
And the solution to this one is the same thing, C1, e
11:40
to the lambda, 1, t, x1.
11:43
Is that what you were expecting for the solution here?
11:49
Because if I takes the derivative,
11:50
it brings down a lambda.
11:52
If I multiply by A, it brings down a lambda--
11:56
so, plus the other guys.
11:59
12:03
OK.
12:04
12:09
Not news, but important to remember what eigenvectors
12:15
are for in the first place.
12:17
Good.
12:17
12:21
Yeah.
12:22
Let me move ahead.
12:24
Oh-- one matrix fact is about something
12:34
called similar matrices.
12:35
So I have on my matrix A. Then I have
12:38
the idea of what it means to be similar to A,
12:41
so B is similar to A. What does that mean?
12:56
So here's what it means, first of all.
12:59
It means that B can be found from A, by--
13:04
this is the key operation here--
13:07
multiplying by a matrix M, and its inverse--
13:10
M inverse AM.
13:12
When I see two matrices, B and A,
13:18
that are connected by that kind of a change,
13:24
M could be any invertible matrix.
13:28
Then I would say B was similar to A. And that changed--
13:34
that appearance of AM is pretty natural.
13:38
If I change variables here by M, then I get--
13:43
that similar matrix will show up.
13:47
So what's the key factor?
13:49
Do you remember the key fact about similar matrices?
13:53
If B and A are connected like that--
13:56
AUDIENCE: They have the same eigenvalues.
13:58
GILBERT STRANG: They have the same eigenvalues.
14:01
So this is just a useful point to remember.
14:04
So I'll-- this is like one fact in the discussion
14:13
of eigenvalues and eigenvectors.
14:15
So similar matrices, same eigenvalues.
14:28
14:37
Yeah.
14:38
So in some way in the eigenvalue, eigenvector world,
14:42
they're in this-- they belong together.
14:45
14:49
They're connected by this relation that just turns out
14:54
to be the right thing.
14:55
14:58
Actually, that is-- it gives us a clue of how eigenvalues
15:03
are actually computed.
15:05
Well, they're actually computed by typing eig of A,
15:10
with parentheses around A. That's how they're--
15:14
in real life.
15:17
But what happens when you type eig of A?
15:22
Well, you could say the eigenvalue shows up
15:24
on the screen.
15:26
But something had to happen in there.
15:28
And what happened was that MATLAB--
15:33
or whoever-- took that matrix A, started using good choices
15:40
of m--
15:41
15:43
better and better.
15:44
15:47
Took a bunch of steps with different m's.
15:50
Because if I do another m, I still have a similar matrix,
15:53
right?
15:54
If I take B and do a different m2 to B--
16:00
so I get something similar to B, then
16:02
that's also similar to A. I've got
16:04
a whole family of similar things there.
16:07
And what does MATLAB do with all these m's, m1 and m2 and m3
16:13
and so on?
16:14
It brings the matrix to a triangular matrix.
16:22
It gets the eigenvalues showing up on the diagonal.
16:24
16:27
It's just tremendously-- it was an inspiration when that--
16:33
when the good choice of m appeared.
16:35
And let me just say--
16:38
because I'm going on to symmetric matrices--
16:41
that for a symmetric matrices, everything is sort of clean.
16:47
You not only go to a triangular matrix,
16:51
you go toward a diagonal matrix.
16:54
They off-- you choose m's that make
16:57
the off diagonal stuff smaller and smaller and smaller.
17:00
And the eigenvalues are not changing.
17:03
So there, shooting up on the diagonal, are the eigenvalues.
17:08
So I guess I should verify that fact,
17:13
that similar matrices have the same eigenvalues.
17:16
Can we-- there can't be much to show.
17:19
There can't be much in the proof because that's all I know.
17:24
And I want to know its eigenvalues and eigenvectors.
17:27
So let me say, suppose m inverse Am has the eigenvector
17:32
y and the eigenvalue of lambda.
17:35
17:41
And I want to show--
17:44
do I want to show that y is an eigenvector also, of A itself?
17:48
No.
17:50
Eigenvectors are changing.
17:52
Do I want to show that lambda is an eigenvalue of A itself?
17:56
Yes.
17:57
That's my point.
17:58
So can we see that?
18:00
Ha.
18:01
Can I see that lambda is an eigenvector?
18:05
There's not a lot to do here.
18:07
I mean, if I can't do it soon, I'm never going to do it,
18:10
because--
18:12
so what am I going to do?
18:13
AUDIENCE: Define the vector x equals my--
18:15
GILBERT STRANG: Yeah, I could.
18:17
Yeah.
18:19
X is-- m-y is going to be a key, and I can see m-y coming.
18:24
Just-- when I see m inverse over there,
18:26
what am I going to do with the darn thing?
18:28
AUDIENCE: [INAUDIBLE]
18:29
GILBERT STRANG: I'm going to put it on the other side.
18:31
I'm going to multiply that equation by m.
18:34
So I'll have-- that will put the m over here.
18:37
And I'll have A-M-y equals lambda My, right?
18:44
18:48
And is that telling me what I want to know?
18:50
Yes.
18:51
That's saying that My--
18:54
that you wisely suggested to give a name x to--
18:58
is lambda times My.
19:01
Do you see that?
19:02
That the eigenvalue lambda didn't change.
19:06
The eigenvector did change.
19:08
It changed from y to My.
19:11
That's the x.
19:13
The eigenvector of x.
19:14
This is lambda x.
19:18
Yeah.
19:19
So that's the role of M. It just gives you a different basis
19:24
for eigenvectors.
19:25
But it does not change eigenvalues.
19:28
Right.
19:28
Yeah.
19:30
OK.
19:31
So those are similar matrices.
19:35
Yeah, some other good things happen.
19:37
A lot of people don't know-- in fact,
19:39
I wasn't very conscious of the fact
19:42
that A times B has the same eigenvalues as B times A. Well,
19:48
I should maybe write that down.
19:51
AB has the same eigenvalues--
19:58
the same non-zero ones--
20:00
you'll see.
20:01
I have to-- as BA.
20:07
This is any A and B same size.
20:10
I'm not talking similar matrices here.
20:13
I'm talking any two A and B. Yeah.
20:19
So that's a good thing that happens.
20:23
Now could we see y?
20:29
And then I'm going to be really pretty happy with basic fact
20:35
about eigenvalues.
20:37
So if I want to show that two things have
20:41
the same eigenvalues, what do you propose?
20:45
Show that they are similar.
20:49
I already said, if they are similar.
20:51
So is there an m?
20:53
Is there an m that will connect this matrix?
20:58
So is there an m that will multiply this matrix that way?
21:05
So that would be similar to AB.
21:07
And can I produce BA then?
21:09
21:16
So I'll just put the word want up here.
21:19
21:22
I want-- if I have that, then I'm
21:28
done, because that's saying that those two matrices, AB and BA,
21:32
are similar.
21:33
And I know that then they have the same eigenvalues.
21:36
So what should m be?
21:41
M should be-- so what is M here?
21:49
I want that to be true.
21:50
21:54
Should M be B?
21:57
Yeah.
21:58
M equal B. Boy.
22:01
Not the most hidden fact here.
22:05
Take M equal B.
22:11
So then I have B times A, times BB inverse--
22:14
which is the identity.
22:16
So I have B times A. Yes.
22:18
OK.
22:19
So AB and BA are fine.
22:23
Now, what do you think about this question?
22:28
Are the eigenvalues-- I now know that AB and BA
22:31
have the same eigenvalues.
22:33
And the reason I had to be careful about non-zero is that
22:40
if I had zero eigenvalues, then--
22:45
AUDIENCE: [INAUDIBLE]
22:46
GILBERT STRANG: Yeah.
22:46
I can't count on those inverses.
22:49
Right.
22:50
Right.
22:51
So that's why I put it in that little qualifier.
22:56
But now I want to ask this question.
22:59
If I know the eigenvalues of A--
23:02
separately, by itself, A-- and of B--
23:05
now I'm talking about any two matrices, A and B.
23:09
If I have two matrices, A--
23:12
I have a matrix A and a matrix B.
23:15
And I know their eigenvalues and their eigenvalues.
23:19
What about AB?
23:21
A times B. Can I multiply the eigenvalues of A times
23:25
the eigenvalues of B?
23:27
Don't do it.
23:28
Right.
23:29
Yes.
23:29
Right.
23:30
The eigenvalues of A times the eigenvalues of B
23:33
could be damn near anything.
23:35
Right.
23:36
They're not connected to the eigenvalues of AB specially.
23:40
And maybe something could be discovered, but not much.
23:45
And similarly, for A plus B. So yeah.
23:50
So let me just write down this point.
23:54
Eigenvalues of A plus B are generally not
24:00
eigenvalues of A plus eigenvalues of B.
24:08
Generally not.
24:09
Just-- there is no reason.
24:12
And the reason that that's--
24:15
I get that no answer is, that the eigenvectors can
24:20
be all different.
24:21
If the eigenvectors for A are totally
24:23
different from the eigenvectors for B,
24:26
then A plus B will have probably some other, totally different
24:30
eigenvectors, and there's nothing happening there.
24:34
24:38
That's sort of thoughts about eigenvalues in general.
24:43
And I could-- there'd be a whole section on eigenvectors,
24:50
but I'm really interested in eigenvectors
24:53
of symmetric matrices.
24:56
So I'm going to move on to that topic.
25:02
So now, having talked about any matrix A,
25:06
I'm going to specialize to symmetric matrices,
25:09
see what's special about the eigenvalues
25:12
there, what's special about eigenvectors there.
25:14
And I think we've already said it in class.
25:17
So let me-- let me ask you to tell me
25:20
about it-- tell me again.
25:21
So I'll call that matrix S now, as a reminder
25:26
always that I'm talking here about symmetric matrices.
25:29
So what do I-- what are the key facts to know?
25:33
Eigenvalues are real numbers, if the matrix is.
25:44
I'm thinking of real symmetric matrices.
25:48
Of course, other real matrices could
25:51
have imaginary eigenvalues.
25:55
Other real matrices-- so just--
25:57
let's just think for a moment.
26:00
Yeah.
26:01
Maybe I'll just put it here.
26:02
Can I back up, before I keep going with symmetric matrices?
26:09
So you take a matrix like that.
26:15
26:20
Q, yeah.
26:20
That would be a Q. But it's not specially a Q. Maybe
26:25
the most remarkable thing about that matrix
26:28
is that it's anti-symmetric.
26:31
So I'll call it A. Right.
26:33
If I transpose that matrix, what do I get?
26:37
AUDIENCE: The negative.
26:38
GILBERT STRANG: The negative.
26:40
So that's like anti-symmetric.
26:42
And I claim that an anti-symmetric matrix
26:45
has imaginary eigenvalues.
26:47
So that's a 90 degree rotation.
26:51
26:54
And you might say, what could be simpler than that?
26:57
A 90 degree rotation-- that's not a weird matrix.
27:01
But from the point of view of eigenvectors,
27:03
something a little odd has to happen, right?
27:07
Because if I have a 90 degree rotation--
27:11
if I take a vector x--
27:12
any vector x-- could it possibly be an eigenvector?
27:18
Well, apply A to it.
27:20
You'd be off in this direction, Ax.
27:24
And there is no way that Ax can be a multiple of x.
27:30
So there's no real eigenvector for that anti-symmetric matrix,
27:34
or any anti-symmetric matrix.
27:38
So you see that when we say that the eigenvalues
27:43
of a symmetric matrix are real, we're
27:46
saying that this couldn't happen--
27:48
that this couldn't happen if A were symmetric.
27:51
And here, it's the very opposite, it's anti-symmetric.
27:54
27:56
Well, while that's on the board, you might say, wait a minute.
27:59
How could that have any eigenvector whatsoever?
28:02
28:05
So what is an eigenvector of that matrix A?
28:09
How do you find the eigenvectors of A?
28:13
When they're 2 by 2, that's a calculation we know how to do.
28:19
You remember the steps there?
28:21
I'm looking for Ax equal lambda x.
28:26
So right now I'm looking for both lambda and x.
28:30
I've got 2.
28:31
It's not linear, but I'm going to bring this over to this side
28:35
and write it as A minus lambda I, x equals 0.
28:39
28:43
And then I'm going to look at that
28:44
and say, wow, A minus lambda I must be not invertible,
28:48
b because it's got this x in its null space.
28:52
So the determinant of this matrix must be 0.
28:56
28:59
I couldn't have a null space unless the determinant is 0.
29:06
And then when I look at A minus lambda I, for this A,
29:12
I've got minus lambdas, minus A--
29:20
oh, A is just the 1.
29:21
And that's minus 1.
29:23
I'm going to take the determinant.
29:26
And what am I going to get for the determinant?
29:29
Lambda squared--
29:30
AUDIENCE: Plus 1.
29:31
GILBERT STRANG: Plus 1.
29:32
29:36
And I set that to 0.
29:38
29:44
So I'm just following all the rules,
29:48
but it's showing me that the lambda--
29:51
the two lambdas-- there are two lambdas here--
29:55
but they're not real, because that equation, the roots
29:58
are i and minus i.
29:59
30:02
So those are the eigenvalues.
30:04
30:07
And they have the nice--
30:08
they have all the--
30:10
well, they are the eigenvalues.
30:11
No doubt about it.
30:12
30:15
With 2 by 2 there are two quick checks that tell you, yeah,
30:20
you did a calculation right.
30:22
If I add up the two eigenvalues in this--
30:31
if I add up the two eigenvalues for any matrix,
30:34
and I'm going to do it for this one--
30:37
I get what answer?
30:38
AUDIENCE: The trace?
30:39
GILBERT STRANG: I get the same answer from the adding--
30:43
add the lambdas gives me the same answer
30:48
as add the diagonal of the matrix--
30:56
which I'm calling A. So if I add the diagonal I get 0 and 0.
31:03
So it's 0 plus 0.
31:03
31:07
And this number adding the diagonal is called the trace.
31:11
31:13
And we'll see it again because it's so simple.
31:19
Just adding the diagonal entries gives you
31:22
a key bit of information.
31:25
When you add down the diagonal it
31:27
tells you the sum of the eigenvalue-- some
31:29
of the lambdas.
31:32
Doesn't tell you each lambda separately,
31:35
but it tells you the sum.
31:37
So it tells you one fact by doing one thing.
31:41
Yeah.
31:42
That's pretty handy.
31:45
Gives you a quick check if you've--
31:49
when you compute this determinant
31:50
and solve for lambda--
31:54
the thing you-- this is a way to compute eigenvalues by hand.
32:03
You could make a mistake, because it's
32:05
a quadratic formula for 2 by 2, but you can
32:10
check by adding the two roots.
32:13
Do you get the same as the trace 0 plus 0?
32:17
32:20
Well, there's one other check, equally quick, for 2 by 2,
32:25
so 2 by 2s--
32:26
you really get them right.
32:28
What's the other check to--
32:31
we add the eigenvalues, we get the trace.
32:33
AUDIENCE: [INAUDIBLE]
32:34
GILBERT STRANG: We multiply the eigenvalues.
32:37
So we take-- so now multiply the lambdas.
32:45
So then I get i times minus i.
32:50
And that should equal-- let's-- don't look yet.
32:54
What should it equal if I multiply the eigenvalues
32:58
I should get the?
32:59
AUDIENCE: Determinant.
33:00
GILBERT STRANG: Determinant, right.
33:02
Of A. So that's two handy checks.
33:12
Add the eigenvalues-- for any size--
33:14
3 by 3, 4 by 4-- but it's only two checks.
33:18
So for 2 by 2, it's kind of, you've got it.
33:21
3 by 3, 4 by 4-- you could still have
33:23
made an error and the two checks could potentially still work.
33:29
Let's just check it out here.
33:30
What's i times minus i?
33:32
33:35
AUDIENCE: 1.
33:36
GILBERT STRANG: 1.
33:37
Because it's minus i squared, and that's plus 1.
33:40
And the determinant of that matrix is 0 minus--
33:44
is 1.
33:45
Yeah.
33:47
OK.
33:47
So we got 1.
33:49
Good.
33:49
33:53
Those are really the key fact about eigenvalues.
33:56
33:59
But of course they're not-- it's not
34:02
as simple as solving Ax equal B to find them,
34:06
but if you follow through on this idea of similar matrices,
34:13
and sort of chop down the off diagonal part, then
34:18
sure enough, the eigenvalue's gotta show up.
34:22
OK.
34:24
Symmetric.
34:25
34:27
Symmetric matrices.
34:28
34:33
So now we're going to have symmetric,
34:38
and then we'll have the special, even better than symmetric,
34:42
is symmetric positive definite.
34:45
OK.
34:46
Symmetric-- you told me the main facts are the eigenvalues real,
34:56
the eigenvectors orthogonal.
35:02
35:08
And I guess, actually--
35:10
yeah.
35:11
So I want to put those into math symbols instead of words.
35:17
35:22
So yeah.
35:26
I guess-- shall I just jump in?
35:31
And the other thing hidden there-- but very important is--
35:36
there's a full set of eigenvectors,
35:39
even if some eigenvalues happen to be repeated,
35:42
like the identity matrix.
35:44
It's still got plenty of eigenvectors.
35:47
So that's a added point that I've not made there.
35:51
And I could prove those two statements,
35:54
but why don't I ask you to accept them and go onward?
35:59
36:02
What are we going to do with them?
36:05
OK.
36:05
36:11
Can you just-- let's have an example.
36:13
36:16
Let me put an example here.
36:18
Suppose S-- now I'm calling it S--
36:22
is 0s, 1 and 1.
36:26
So that's symmetric.
36:29
What are its eigenvalues?
36:33
What are the eigenvalues of that symmetric matrix, S?
36:37
AUDIENCE: Plus and minus 1.
36:38
GILBERT STRANG: Plus and minus 1.
36:40
Well, if you propose two eigenvalues,
36:43
I'll write them down, 1 and minus 1.
36:46
And then what will I do to check them?
36:49
AUDIENCE: Trace and determinant.
36:51
GILBERT STRANG: Trace and determinant.
36:53
OK.
36:54
So are they-- is it true that the eigenvalues
36:57
are 1 and minus 1?
37:01
OK.
37:02
How do I check the trace?
37:04
What is the trace of that matrix?
37:07
0.
37:08
And what's the sum of the eigenvalues--
37:10
0.
37:11
Good.
37:12
What about determinant?
37:13
What's the determinant of S?
37:15
AUDIENCE: Minus 1.
37:16
GILBERT STRANG: Minus 1.
37:17
The product of the eigenvalues-- minus 1.
37:19
So we've got it.
37:20
OK.
37:21
What are the eigenvectors?
37:24
What vector can you multiply by and it
37:29
doesn't change direction-- in fact, doesn't change at all?
37:32
I'm looking for the eigenvector that's a steady state?
37:35
AUDIENCE: 0, 1?
37:37
GILBERT STRANG: 0, 1?
37:38
AUDIENCE: 1, 1.
37:41
GILBERT STRANG: I think it's 1, 1.
37:42
Yeah.
37:43
So here is the lambdas.
37:45
And then the eigenvectors are--
37:47
I think 1, 1.
37:48
37:51
Is that right?
37:52
Yeah.
37:53
Sure.
37:54
S is just a permutation here.
37:57
It's just exchanging the two entries.
37:59
So 1 and 1 won't change.
38:01
And what's the other eigenvector?
38:04
AUDIENCE: Minus 1?
38:05
GILBERT STRANG: 1 and minus 1.
38:07
38:15
And then, I'm thinking-- remembering about this similar
38:19
stuff--
38:20
I'm thinking that S is similar to a matrix that
38:27
just shows the eigenvalues.
38:29
So S is similar to--
38:31
I'm going to put in an M--
38:34
well, I'm going to connect S-- that matrix--
38:37
with the eigenvalue matrix, which has the eigenvalues.
38:45
So here is my--
38:48
38:50
everybody calls that matrix capital lambda,
38:53
because everybody calls the eigenvalues little lambda.
38:57
So the matrix that has them is called capital lambda.
39:02
And I-- my claim is that these guys are similar--
39:06
that this matrix, S, that you're seeing up there--
39:09
I believe there is an M I believe
39:12
there is an M. So that S--
39:15
what did I put in here?
39:17
So I'm following this pattern.
39:19
I believe that there would be an M and an M inverse,
39:24
so that this would mean that.
39:27
And that's nice.
39:29
First of all, it would confirm that the eigenvalues
39:33
stay the same, which was certain to happen.
39:38
And then it would also mean that I had got a diagonal matrix.
39:43
And of course, that's a natural goal--
39:45
to get a diagonal matrix.
39:46
39:49
So we might hope that the M that gets us there
39:52
is like an important matrix.
39:57
So do you see what I'm doing here?
39:59
It comes under the heading of diagonalizing a matrix.
40:04
I start with a matrix, S. I find it's eigenvalues.
40:08
They go on into lambda.
40:11
And I believe I can find an M, so that I see they're similar.
40:19
They have the same eigenvalues, 1 and minus 1, both sides.
40:23
So only remaining question is, what's M?
40:27
What's the matrix that diagonalizes S?
40:33
The-- what have we got left to use?
40:35
AUDIENCE: The eigenvectors.
40:36
GILBERT STRANG: The eigenvectors.
40:39
The matrix that-- so, can I put the M over there?
40:43
40:45
Yeah.
40:46
I'll put-- that M inverse is going
40:48
to go over to the other side.
40:52
Oh.
40:52
It goes here, doesn't it?
40:54
I was worried there.
40:56
It didn't look good, but yeah.
40:58
So this is all going to be right, if--
41:02
41:07
this is what I'd like to have--
41:09
SM equal M lambda.
41:11
SM equal M lambda.
41:14
That's diagonalizing a matrix.
41:17
That's finding the M using the eigenvectors.
41:22
That produces a similar matrix lambda,
41:25
which has the eigenvalues.
41:27
That's the great fact about diagonalizing.
41:35
That's how you use-- that's another way to say,
41:38
this is how the eigenvectors pay off.
41:40
You put them into M. You take the similar matrix
41:43
and it's nice and diagonal.
41:45
And do you see that this will happen?
41:47
S times-- so M has the first eigenvector
41:51
and the second eigenvector.
41:53
And I believe that first eigenvector times the second--
41:59
and the second eigenvector-- that's M again, on this side.
42:03
Let me just write in 1, 0, 0, minus 1.
42:08
42:13
I believe is has got to be confirming that we've
42:17
done the thing right--
42:19
confirming that the eigenvectors work here.
42:22
42:24
Please make sense out of that last line.
42:27
42:30
When you see that last line, what do I
42:33
mean to make sense out of it?
42:35
I want to see that that's true.
42:37
How do I see that--
42:39
how do I do this--
42:40
so what's the left side and what's the right side?
42:43
42:47
So what-- if I multiply S by a couple
42:50
of columns, what's the answer?
42:54
AUDIENCE: Sx1 and Sx2.
42:55
GILBERT STRANG: Sx1 and Sx2.
42:56
That's the beauty of matrix multiplication.
42:59
If I multiply a matrix by another matrix,
43:02
I can do it a column at a time.
43:05
There are four great ways to multiply matrices,
43:07
so this is another one--
43:10
a column at a time.
43:12
So this left hand side is Sx1, Sx2.
43:16
I just do each column.
43:19
And what about the right hand side?
43:23
I can do that multiplication.
43:25
AUDIENCE: X1 minus x2.
43:26
GILBERT STRANG: X1 minus x2 did somebody say?
43:29
Death.
43:31
No.
43:32
I don't want-- Oh, x1--
43:33
sorry.
43:34
You said it right.
43:35
OK.
43:36
When you said x1 minus x2, I was subtracting.
43:39
But you meant that that's-- the first column is x1,
43:42
and the second column is minus x2.
43:44
Correct.
43:45
Sorry about that.
43:46
43:50
And did we come out right?
43:53
Yes.
43:54
Of course, now I compare.
43:56
Sx1 is lambda one x1.
43:59
Sx2 is lambda two x2.
44:02
And I'm golden.
44:03
44:08
So what was the point of this board?
44:12
What did we learn?
44:15
We learned-- well, we kind of expected that the original S
44:20
would be similar to the lambdas, because the eigenvalues match.
44:26
S has eigenvalues lambda.
44:28
And this diagonal matrix certainly
44:30
has eigenvalues 1n minus 1.
44:33
A diagonal matrix-- the eigenvalues
44:35
are right in front of you.
44:37
So they're similar.
44:38
S is similar to the lambda.
44:40
And there should be an M. And then somebody suggested, maybe
44:44
the M is the eigenvectors.
44:46
And that's the right answer.
44:48
So finally, let me write that conclusion here--
44:53
44:55
which isn't just for symmetric matrices.
44:59
So maybe I should put it for matrix A.
45:04
So if it has lambdas and eigenvectors,
45:12
and the claim is that A times the eigenvector matrix
45:22
is the eigenvector matrix times the eigenvalues.
45:28
45:34
And I would shorten that to Ax equals x lambda.
45:39
45:42
And I could rewrite that, and then I'll
45:44
slow down, as A equal x lambda x inverse.
45:49
45:58
Really, this is bringing it all together
46:00
in a simple, small formula.
46:02
It's telling us that A is similar to lambda.
46:07
It's telling us the matrix M, that does the job--
46:10
it's a matrix of eigenvectors.
46:13
And so it's like a shorthand way to write the main fact
46:21
about eigenvalues and eigenvectors.
46:24
What about A squared?
46:26
Can I go back to the very first--
46:28
I see time is close to the end here.
46:31
What about A squared?
46:33
What are the eigenvectors of A squared?
46:36
What are the eigenvalues of A squared?
46:39
That's like the whole point of eigenvalues.
46:42
Well, or I could just square that stupid thing.
46:45
X lambda, x inverse, x lambda, x inverse.
46:50
And what have I got?
46:52
X inverse, x in the middle is--
46:54
AUDIENCE: Identity.
46:55
GILBERT STRANG: Identity.
46:58
So I have x, lambda squared, x inverse.
47:03
And to me and to you that says, the eigenvalues
47:06
have been squared.
47:07
The eigenvectors didn't change.
47:10
Yeah.
47:11
OK.
47:12
And now finally, last breath is, what
47:15
if the matrix is symmetric?
47:18
Then we have different letters.
47:19
That's the only-- that's the significant change.
47:23
The eigenvector matrix is now an orthogonal matrix.
47:30
I'm coming back to the key fact of what makes symmetric--
47:34
how do I read--
47:35
how do I see symmetric helping me in the eigenvector
47:39
and eigenvalue world?
47:41
Well, it tells me that the eigenvectors are orthogonal.
47:46
So the x is Q. The eigenvalues are real.
47:53
And the eigenvectors is x inverse.
47:56
But now I'm going to make those eigenvectors unit vectors.
48:00
I'm going to normalize it.
48:02
So I'm really allowing--
48:04
I have an orthogonal matrix Q. So I have a different way
48:08
to write this, and this is the end of the--
48:10
today's class.
48:12
Q lambda.
48:15
And what can you tell me about Q inverse?
48:18
AUDIENCE: It's Q transpose.
48:19
GILBERT STRANG: It's Q transpose.
48:20
Thanks.
48:21
So that was the last lecture.
48:23
So now the orthogonal lecture is coming up
48:27
at the last second of the symmetric matrices lecture.
48:31
And this has the name spectral theorem,
48:35
which I'll just put there.
48:36
48:40
And the whole point is that it tells you
48:46
what every symmetric matrix looks like--
48:50
orthogonal eigenvectors, real eigenvalues.
48:54
English - CC (English)
###