04 - Eigenvalues and Eigenvectors
2.7M subscribers
SUBSCRIBED
MIT A 2020 Vision of Linear Algebra, Spring 2020 Instructor: Gilbert Strang View the complete course: https://ocw.mit.edu/2020-visionYouTube Playlist: https://www.youtube.com/playlist?list...Eigenvalues and eigenvectors are a way to look deeper into the matrix. They have applications across all engineering and science disciplines including graphs and networks. License: Creative Commons BY-NC-SA More information at https://ocw.mit.edu/termsMore courses at https://ocw.mit.edu
00:00
00:13
GILBERT STRANG: Moving now to the second half
00:15
of linear algebra.
00:17
It's about eigenvalues and eigenvectors.
00:20
The first half, I just had a matrix.
00:22
I solved equations.
00:24
The second half, you'll see the point
00:27
of eigenvalues and eigenvectors as a new way
00:29
to look deeper into the matrix to see what's important there.
00:34
OK, so what are they?
00:36
This is a big equation, S time x.
00:40
So S is our matrix.
00:42
And I've called it S because I'm taking
00:45
it to be a symmetric matrix.
00:47
What's on one side of the diagonal
00:49
is also on the other side of the diagonal.
00:52
So those have the beautiful properties.
00:54
Those are the kings of linear algebra.
00:57
Now, about eigenvectors x and eigenvalues lambda.
01:02
So what does that equation, Sx equal lambda x, tell me?
01:06
That says that I have a special vector x.
01:10
When I multiply it by S, my matrix,
01:14
I stay in the same direction as the original x.
01:17
It might get multiplied by 2.
01:19
Lambda could be 2.
01:21
It might get multiplied by 0.
01:23
Lambda there could even be 0.
01:25
It might get multiplied by minus 2, whatever.
01:28
But it's along the same line.
01:30
So that's like taking a matrix and discovering inside it
01:35
something that stays on a line.
01:39
That means that it's really a sort of one dimensional problem
01:43
if we're looking along that eigenvector.
01:47
And that makes computations infinitely easier.
01:51
The hard part of a matrix is all the connections
01:55
between different rows and columns.
01:57
So eigenvectors are the guys that
02:00
stay in that same direction.
02:02
02:04
And y is another eigenvector.
02:08
It has its own eigenvalue.
02:10
It got multiplied by alpha where Sx multiplied the x
02:14
by some other number lambda.
02:16
So there's our couple of eigenvectors.
02:18
And the beautiful fact is that because S is symmetric,
02:23
those two eigenvectors are perpendicular.
02:26
They are orthogonal, as it says up there.
02:29
So symmetric matrices are really the best
02:32
because their eigenvectors are perpendicular.
02:35
And we have a bunch of one dimensional problems.
02:38
And here, I've included a proof.
02:41
You want a proof that the eigenvectors are perpendicular?
02:47
So what does perpendicular mean?
02:48
It means that x transpose times y, the dot product is 0.
02:55
The angle is 90 degrees.
02:58
The cosine is 1.
03:01
OK.
03:03
How to show the cosine might be there.
03:07
How to show that?
03:09
Yeah, proof.
03:10
This is just you can tune out for two minutes
03:13
if you hate proofs.
03:15
OK, I start with what I know.
03:17
What I know is in that box.
03:20
Sx is lambda x.
03:21
That's one eigenvector.
03:23
That tells me the eigenvector y.
03:25
This tells me the eigenvalues are different.
03:27
And that tells me the matrix is symmetric.
03:30
I'm just going to juggle those four facts.
03:33
And I'll end up with x transpose y equals 0.
03:39
That's orthogonality.
03:41
OK.
03:42
So I'll just do it quickly, too quickly.
03:46
So I take this first thing, and I transpose
03:49
it, turn it into row vectors.
03:52
And then when I transpose it, that transpose
03:57
means I flip rows and columns.
03:59
But for as symmetric matrix, no different.
04:02
So S transpose is the same as S.
04:05
And then I look at this one, and I multiply that
04:08
by x transpose, both sides by x transpose.
04:13
And what I end up with is recognizing
04:16
that lambda times that dot product
04:19
equals alpha times that dot product.
04:22
But lambda is different from alpha.
04:24
So the only way lambda times that number
04:26
could equal alpha times that number
04:28
is that number has to be 0.
04:31
And that's the answer.
04:32
OK, so that's the proof that used
04:34
exactly every fact we knew.
04:37
End of proof.
04:39
Main point to remember, eigenvectors
04:42
are perpendicular when the matrix is symmetric.
04:47
OK.
04:49
In that case, now, you always want to express these facts
04:54
as from multiplying matrices.
04:59
That says everything in a few symbols
05:02
where I had to use all those words on the previous slide.
05:06
So that's the result that I'm shooting for,
05:10
that a symmetric matrix--
05:14
just focus on that box.
05:19
A symmetric matrix can be broken up into its eigenvectors.
05:24
Those are in Q. Its eigenvalues.
05:27
Those are the lambdas.
05:28
Those are the numbers lambda 1 to lambda n
05:32
on the diagonal of lambda.
05:34
And then the transpose, so the eigenvectors are now
05:36
rows in Q transpose.
05:39
That's just perfect.
05:42
Perfect.
05:43
Every symmetric matrix is an orthogonal matrix
05:47
times a diagonal matrix times the transpose
05:51
of the orthogonal matrix.
05:53
Yeah, that's called the spectral theorem.
05:55
And you could say it's up there with the most important facts
06:00
in linear algebra and in wider mathematics.
06:04
Yeah, so that's the fact that controls what we do here.
06:12
Oh, now I have to say what's the situation if the matrix is not
06:18
symmetric.
06:19
Now I am not going to get perpendicular eigenvectors.
06:24
That was a symmetric thing mostly.
06:27
But I'll get eigenvectors.
06:30
So I'll get Ax equal lambda x.
06:35
The first one won't be perpendicular
06:37
to the second one.
06:37
The matrix A, it has to be square,
06:40
or this doesn't make sense.
06:41
So eigenvalues and eigenvectors are the way
06:45
to break up a square matrix and find this diagonal matrix
06:50
lambda with the eigenvalues, lambda 1, lambda 2, to lambda
06:54
n.
06:55
That's the purpose.
06:58
And eigenvectors are perpendicular when
07:01
it's a symmetric matrix.
07:03
Otherwise, I just have x and its inverse matrix but no symmetry.
07:08
OK.
07:09
So that's the quick expression, another factorization
07:14
of eigenvalues in lambda.
07:17
Diagonal, just numbers.
07:19
And eigenvectors in the columns of x.
07:23
And now I'm not going to-- oh, I was
07:27
going to say I'm not going to solve all
07:29
the problems of applied math.
07:31
But that's what these are for.
07:33
Let's just see what's special here about these eigenvectors.
07:38
Suppose I multiply again by A. I Start with Ax equal lambda x.
07:46
Now I'm going to multiply both sides by A.
07:49
That'll tell me something about eigenvalues of A squared.
07:53
Because when I multiply by A--
07:55
so let me start with A squared now
07:58
times x, which means A times Ax.
08:02
A times Ax.
08:04
But Ax is lambda x.
08:06
So I have A times lambda x.
08:09
And I pull out that number lambda.
08:12
And I still have a 1Ax.
08:15
And that's also still lambda x.
08:17
You see I'm just talking around in a little circle
08:20
here, just using Ax equal lambda x a couple of times.
08:24
And the result is--
08:25
do you see what that means, that result?
08:28
That means that the eigenvalue for A squared, same eigenvector
08:33
x.
08:33
The eigenvalue is lambda squared.
08:37
And if I add A cubed, the eigenvalue
08:39
would come out lambda cubed.
08:41
And if I have a to the-- yeah, yeah.
08:44
So if I had A to the n times, n multiplies-- so when
08:49
would you have A to a high power?
08:53
That's a interesting matrix.
08:56
Take a matrix and square it, cube it,
08:58
take high powers of it.
09:02
The eigenvectors don't change.
09:04
That's the great thing.
09:05
That's the whole point of eigenvectors.
09:07
They don't change.
09:09
And the eigenvalues just get taken to the high power.
09:12
So for example, we could ask the question, when,
09:16
if I multiply a matrix by itself over and over and over again,
09:19
when do I approach 0?
09:20
Well, if these numbers are below 1.
09:24
So eigenvectors, eigenvalues gives you something
09:27
that you just could not see by those column operations or L
09:33
times U. This is looking deeper.
09:36
OK.
09:38
And OK, and then you'll see we have almost already seen
09:45
with least squares, this combination A transpose A. So
09:49
remember A is a rectangular matrix, m by n.
09:53
I multiply it by its transpose.
09:56
When I transpose it, I have n by m.
10:02
And when I multiply them together, I get n by n.
10:05
So A transpose A is, for theory, is a great matrix,
10:10
A transpose times A. It's symmetric.
10:14
Yeah, let's just see what we have about A.
10:16
It's square for sure.
10:19
Oh, yeah.
10:20
This tells me that it's symmetric.
10:22
And you remember why.
10:23
I'm always looking for symmetric matrices
10:26
because they have those orthogonal eigenvectors.
10:29
They're the beautiful ones for eigenvectors.
10:32
And A transpose A, automatically symmetric.
10:36
You just you're multiplying something
10:40
by its adjoint, its transpose, and the result
10:45
is that this matrix is symmetric.
10:51
And maybe there's even more about A transpose A. Yes.
10:55
What is that?
10:56
11:00
Here is a final--
11:03
I always say certain matrices are important,
11:07
but these are the winners.
11:10
They are symmetric matrices.
11:12
If I want beautiful matrices, make them symmetric
11:16
and make the eigenvalues positive.
11:18
11:21
Or non-negative allows 0.
11:26
So I can either say positive definite
11:28
when the eigenvalues are positive,
11:31
or I can say non-negative, which allows 0.
11:34
And so I have greater than or equal to 0.
11:38
I just want to say that bringing all
11:41
the pieces of linear algebra come together
11:44
in these matrices.
11:46
And we're seeing the eigenvalue part of it.
11:49
And here, I've mentioned something called the energy.
11:53
So that's a physical quantity that
11:55
also is greater or equal to 0.
11:58
So that's A transpose A is the matrix
12:02
that I'm going to use in the final part of this video
12:10
to achieve the greatest factorization.
12:15
Q lambda, Q transpose was fantastic.
12:18
But for a non-square matrix, it's not.
12:22
For a non-square matrix, they don't even
12:25
have eigenvalues and eigenvectors.
12:27
But data comes in non-square matrices.
12:31
Data is about like we have a bunch of diseases
12:34
and a bunch of patients or a bunch of medicines.
12:37
And the number of medicines is not
12:39
equal the number of patients or diseases.
12:42
Those are different numbers.
12:43
So the matrices that we see in data are rectangular.
12:48
And eigenvalues don't make sense for those.
12:52
And singular values take the place of eigenvalues.
12:56
So singular values, and my hope is
12:59
that linear algebra courses, 18.06 for sure,
13:04
will always reach, after you explain
13:08
eigenvalues that everybody agrees is important,
13:12
get singular values into the course
13:14
because they really have come on as the big things
13:18
to do in data.
13:20
So that would be the last part of this summary video
13:27
for 2020 vision of linear algebra
13:30
is to get singular values in there.
13:33
OK, that's coming next.
13:36
English - CC (English)
###