Matrix arithmetic fundamentals
3.01K subscribers
SUBSCRIBED
An introduction to matrix arithmetic for a first course in Linear Algebra. The key operation is that of matrix multiplication, which is somewhat magical!
00:00
hello everyone I'm Norman wahlburger and
00:02
today we'll be talking about matrix
00:04
arithmetic fundamentals so this is very
00:07
classical subject but there is in fact a
00:10
more modern take on the whole story
00:13
which is so they called Maxwell theory
00:16
which I've put out in the last few years
00:18
you can check that out on the math
00:20
foundations series of videos math
00:23
foundations 166 to about 173 all of
00:26
those videos are on Mac soles and deck
00:28
soles which give a sort of a bigger
00:30
picture in which matrix theory so it has
00:32
a more consistent and general arithmetic
00:35
but we're not gonna mention that anymore
00:37
I'm just gonna show you the classical
00:39
stuff but just let you know there is a
00:41
higher level okay so with matrices we
00:45
look at arrays of numbers okay so the
00:49
first operations are just addition and
00:52
subtraction of matrices and the crucial
00:58
thing there is that the matrices have to
01:00
be the same size so only for matrices of
01:06
the same size
01:11
okay so we've already look at this in
01:15
some special cases but for example let's
01:18
say you have 1 2 3 4 5 6 there's a
01:21
matrix and another one maybe 2 1 4 7 - 3
01:27
5 ok to add those you just add them
01:30
component wise so you get 3 3 7 11 2 and
01:36
11 ok and to subtract them well again
01:39
you just do everything component wise
01:47
you would get 1 minus 2 is minus 1 2
01:50
minus 1 is 1 3 minus 4 is minus 1 4
01:52
minus 7 is minus 3 5 minus minus 3 is 8
01:55
and 6 minus 5 is 1 ok so they're just
02:00
these are component wise operations
02:10
okay so the operation addition and
02:12
operation of subtraction there's another
02:15
operation of multiplying by a scalar so
02:20
scalar multiplication so for example if
02:29
we take three times the matrix one two
02:31
three four or five six then we get the
02:34
matrix three six nine twelve fifteen and
02:38
eighteen okay so we just multiply all
02:40
the components so that's these three
02:45
operations which are all component wise
02:47
now what am i describing that
02:50
efficiently is in terms of some notation
02:55
so if a is M by n matrix so that I
03:06
remind you that means that it has M rows
03:10
and n columns then it will have the form
03:20
like this we've seen this kind of
03:24
notation before where we index the
03:26
elements with a pair of numbers so a 1 1
03:29
a 1 2 up to 8 1 n a 2 1 a 2 2 up to a 2
03:34
n to am-1 am-2 over to M a mmm ok so
03:44
this is kind of a general expression for
03:46
an M by n matrix and then we say or
03:51
we'll agree that that if we want to
03:56
focus on the IJ entry so we could do
04:01
that by taking the matrix its name
04:03
putting into square brackets and putting
04:05
a little subscript IJ that that means
04:08
the the entry in the I row and J column
04:18
okay which in this case would be little
04:21
a
04:27
all right so in then in terms of that
04:30
notation then we could rewrite the
04:34
operations that if if a and B are both M
04:42
by n matrices then the sum a plus B is
04:52
described by the following rule that the
04:54
IJ entry of the sum is the IJ entry of a
05:00
plus the IJ entry look I J entry of B
05:10
similarly that difference a minus B the
05:15
IJ entry of a minus B is the IJ entry of
05:18
a minus the IJ entry of B and if we
05:24
wanted to just do a scalar
05:25
multiplication if we take lambda times a
05:27
then the IJ entry of that is just lambda
05:31
times D IJ entry type a so those
05:37
equations describe the operations at the
05:42
level of at the level of entries okay so
05:49
then we have some properties of these
05:51
operations for example if a B and C are
05:58
all M by n matrices then we have various
06:08
laws such as a plus B equals B plus a
06:13
that's a commutative rule for addition
06:17
we have that a plus B plus C in that
06:23
order is the same as a plus B plus C
06:25
let's see associative law they don't
06:29
write down the name so the first one is
06:31
the commutative law and the second what
06:34
is the associative law
06:37
and there's also distributive laws with
06:40
respect to scalar multiplication so for
06:43
example if you have lambda times mu
06:47
times a so lambda and mu are two numbers
06:50
then that's the same as lambda mu times
06:53
a that's a kind of a distributive law
06:59
and another kind of distributive law is
07:02
that if you have lambda times a plus B
07:05
that's the same as lambda times a plus
07:09
lambda times big the second kind of
07:11
distributive law okay so this is all
07:16
well and good and and this actually
07:21
makes the space of M by n matrices into
07:25
what we took that's called a linear
07:26
space so then sometimes we introduce
07:33
this kind of notation that capital M MN
07:36
is the space of all M by n matrices and
07:48
then this is this is what we sought
07:52
that's called a linear space so a lot
07:57
like like the the vector space of our n
08:00
that we've been talking about in the
08:02
context of vectors except now the
08:04
objects are matrices but you still do
08:06
the same kind of thing you can perform
08:08
linear combinations so you can form you
08:14
can form linear combinations of matrices
08:23
in this space there's a special element
08:35
in this space which is worth mentioning
08:36
okay the the matrix in M m n with all
08:46
components or entries
08:50
zero is well after we just you know
08:54
write a sort of a bigger a zero so it's
08:57
the matrix with all zeros everywhere
09:00
okay
09:01
there's M rows and there's n columns
09:09
okay so that's all kind of well good but
09:12
what really makes matrix arithmetic so
09:16
interesting is that there's also a
09:18
matrix multiplication and the matrix
09:21
multiplication is not component wise
09:25
okay and it's the matrix multiplication
09:28
that really imbues the subject with its
09:31
richness so let's talk about matrix
09:34
multiplication and again to be honest no
09:42
this story is made a lot more
09:44
intelligible really and a lot more
09:46
general and natural if you go to maxilla
09:48
theory where things simplified to quite
09:51
an extent but anyway here's the
09:53
classical story so the classical story
09:55
is that you can multiply matrices if
09:59
their sizes are related in a very
10:02
particular way so if a is M by n matrix
10:09
a and B is a n by say P matrix then the
10:21
product a times B is defined it is
10:27
defined so we do not multiply any old
10:30
pair of matrices they have to be
10:33
especially oriented so that they the
10:39
number of columns of a equals the number
10:47
equals the number of rows of B that's
10:52
the kind of critical condition contained
10:56
in the fact that this n here and this
10:59
end here are the same okay so this end
11:02
here is telling you
11:03
number of columns in a while this IDI
11:06
here is telling you the number of rows
11:07
in B and we're assuming that those are
11:09
equal and the way I want you to
11:12
visualize this is like this so here's a
11:14
matrix a and I'm going to just draw it
11:17
like this so here's like that is the
11:19
number of rows that say m and here is
11:23
the number of columns that's say
11:25
happening so here's our matrix a and
11:29
then we have a matrix B that we're going
11:30
to multiply it by but the B is going on
11:33
the right hand side so the the crucial
11:37
condition is that the size of B in this
11:40
direction the number of rows of B is the
11:43
same as the number of columns of a yes
11:49
matrix B and the number of columns of B
11:52
well that's some people alright so here
11:54
we have an M by n matrix M rows and
11:57
columns and here we have an N by P
12:00
matrix n rows and P columns and as
12:04
before this number and this number agree
12:09
okay and geometrically that means that
12:11
this width here is the same as this
12:14
height so that's sort of at the level of
12:18
what they actually look like on the page
12:20
okay so if that condition is satisfied
12:23
then you can take your product and
12:27
what's their product going to look like
12:28
the product is going to look like well
12:31
it's good to be something about like
12:34
this and like this
12:35
so here's the product a B and it's size
12:40
in this direction is M and its size in
12:42
this direction is P so the result of an
12:46
M by n times and n by P is an M by P all
12:51
right so notice that we're taking the
12:53
the two unused dimensions here the M and
12:57
the P and they are just being copied
13:01
over here and people
13:08
okay so that's when you can multiply
13:12
matrices and that's sort of what it
13:15
looks like but what actually is the
13:17
definition of the product of matrices so
13:22
the way I want you to think about this
13:24
is the following the simplest case is
13:27
when we have a row vector times a column
13:30
vector okay so let's say a 1 a 2 up to a
13:35
n so that's going to be a matrix with
13:40
one row and n columns and we're going to
13:44
multiply it by another vector b1 b2 to
13:49
be in which is something with n rows and
13:54
one column okay so this is allowed
13:58
because that in and that n are the same
14:00
and the result of this is as we've seen
14:03
before it's just essentially like a dot
14:05
product dot product of this row vector
14:08
with this column vector so it's a 1
14:10
times B 1 plus a 2 times B 2 sum of all
14:15
such products all the way down to a n BN
14:19
and that's a 1 by 1 matrix
14:24
now sometimes people don't actually you
14:27
know put the 1 by 1 matrix in in
14:29
brackets but probably officially we
14:32
should because we're agreeing that
14:33
matrices are going to be put in brackets
14:35
so even though one by one matrix still
14:37
deserves to be in brackets so this is
14:40
the simplest case and once you've
14:42
understood this one well then the
14:44
general story is kind of just repeating
14:48
this as many times as you can so now
14:51
let's look at the general case okay I'll
14:58
try to use the same dimensions as I had
15:00
up here okay so here's our first matrix
15:03
it's a matrix a ok and it's an M by N
15:08
and then over here is the matrix we're
15:11
calling B they don't make it to this B
15:16
and it's an N by P matrix
15:21
and over here what we should get is
15:24
another matrix called a times B which
15:27
should be an M by P matrix so something
15:31
like that so what are we doing here so
15:36
what we're gonna be doing is looking at
15:39
the matrix a and looking at the rows of
15:43
a so a has altogether M rows so there's
15:50
the rows of the matrix a and the other
15:55
side B has columns and the condition on
16:05
a and B guarantees that the rows in a
16:10
have length N and the columns in B have
16:14
length n so this dimension here is the
16:19
same as this dimension here and that
16:23
means that we can multiply any one of
16:25
the rows in a with any one of the
16:28
columns in B so for the general case
16:32
what we do is we we multiply all
16:39
possible rows of a with all possible
16:50
columns of B so how many possibilities
16:59
are there
17:00
well there's m rows of a and there's P
17:04
columns of B so altogether there are M
17:07
times P different possible products
17:11
between the rows of a and the columns of
17:13
B and what we're doing when we write
17:15
down a times B is we're just organizing
17:18
those various products in a systematic
17:21
way there's M times P different entries
17:25
in a B and each one of those entries is
17:27
obtained by taking a product of a row an
17:32
A and a column in B
17:34
and I'm gonna do you note that like this
17:37
so here is the IJ entry okay of the
17:40
product a B and what is in here is the
17:44
product of the product of I throw of a
17:52
and the J row of B so that means week
18:00
like this is like the I thing here this
18:03
is the J thing here that means we have
18:05
to go over to the I physician here in
18:08
the matrix a and pick out that that I
18:12
row there it is right there okay and
18:16
then we have to go over to the matrix B
18:20
and pick out the J column okay maybe
18:25
it's right there yes the Jade column
18:28
will be and we're gonna multiply this I
18:30
throw of a with the jth column of B and
18:34
we're gonna put the result of that which
18:36
is a one by one matrix or just a single
18:38
number we're gonna put it right in here
18:42
okay and we can do that for all possible
18:44
rows of a and all possible columns are
18:47
doing okay well let's illustrate this
18:50
with some examples so suppose that we
18:53
have say 1 2 3 go back to this 1 2 3 4 5
18:56
6 and I'm going to multiply it by some
18:59
other matrix I'm going to multiply it by
19:02
a 3x3 matrix this happens to be 2 by 3
19:05
matrix I'm going to multiply it but
19:06
actually my I might multiplied by a 3 by
19:08
4 matrix okay to give us lots of
19:11
practice okay so what's the 3 by 4
19:14
matrix maybe 1 0 minus 1 4 okay that's
19:19
300 to 1 minus 1 2 4 2 0 5 so there's a
19:30
2 by 3 matrix on the left by 3 by 4
19:33
matrix on the right we can check that
19:37
this dimension and this dimension agree
19:40
so these two matrices do have a
19:42
legitimate well-defined product
19:46
but this dimension here equals this
19:49
dimension here so we are okay we can
19:51
make this product alright so now what we
19:54
have to do is we have to take all
19:56
possible products between these two rows
19:58
here two rows there and there are four
20:02
columns here now the way I would suggest
20:10
you to do this is the following so we
20:12
know what the the final picture is going
20:14
to be it's going to be a 2 by 4 so 2 in
20:18
this direction and 4 across so we get a
20:21
2 by 4 as a result of it and to compute
20:25
this what I suggest you do is that you
20:27
start with the first column compute the
20:31
first column here ok so there's going to
20:35
be two entries that we're going to
20:36
compute and what that amounts to is by
20:40
starting with this column here this
20:43
first column of B and multiplying it by
20:46
each of the rows of a the results of
20:51
those two products will end up in this
20:54
column in the product let's do that all
20:58
right so this first entry here which I'm
21:01
going to compute this is going to be
21:03
maybe it's a number which is 1 times 1
21:07
plus 2 times 2 plus 3 times 4 which is 1
21:13
plus 4 plus 12 1 plus 4 is 5 plus 12 is
21:17
17 so this 17 that goes right there the
21:22
next entry this one down here ok that's
21:25
going to be here it's going to be the
21:29
product of the vector 4 5 6 was 1 2 4 so
21:33
it's 4 times 1 plus 5 times 2 plus 6
21:37
times 4 which is 4 plus 10 that's 14
21:42
plus 24 14 plus 24 is 38 so I've done
21:50
the first column of the product now I'm
21:52
going to go to the second column okay so
21:55
the second column over here
21:57
to compute that second column I'm going
22:00
to go to the second column in the matrix
22:04
on the right-hand side and multiply it
22:06
by each one of the rows of the left-hand
22:10
matrix pain
22:11
okay so what's that I'll just write down
22:13
the answer so the vector 1 2 3 times the
22:16
vector 0 1 2 so we get 1 times 0 plus 2
22:19
times 1 plus 3 times 2 and it's best if
22:21
we do this in their heads so you get to
22:23
practice some arithmetic ok so 1 times 0
22:25
is 0 0 plus 2 plus 6 is 8 and then the
22:31
next entry we look at the second row of
22:35
a the 4 or 5 6 vector and still multiply
22:38
it by that same column 0 1 2 so 4 times
22:40
0 is 0 5 times 1 is 5 6 times 2 is 12 5
22:45
plus 12 is 17 so there's a 17-year old
22:48
now we work on the third column that
22:53
means we have to now take the third
22:54
column over here and multiply it by all
22:57
of the rows of a so 1 times minus 1 is
23:00
minus 1 2 times minus 1 is minus 2 3
23:03
times 0 is 0 so minus 1 minus 2 for a
23:06
total of minus 3 and then the next one 4
23:12
times minus 1 is minus 4 5 times minus 1
23:16
that's minus 5 for 6 times 0 so it's a
23:18
total of minus 9 and then finally our
23:23
fourth column over here is obtained by
23:26
going to this fourth column over here
23:28
and multiplying it by the two vectors so
23:32
1 2 3 times 4 2 5 4 plus 4 plus 15
23:36
that's 8 plus 15 is 23 and the next one
23:41
4 5 6 times 4 to 5 so that's 16 plus 10
23:46
that's 26 plus 30 26 plus 30 56 I think
23:53
ok so I this is how I want you to think
23:57
about things okay when we multiply
23:59
matrices you're multiplying the rows of
24:01
the left one with the columns of the
24:04
right one okay and you're going to get a
24:07
number for each possible product
24:11
that's what you do so you're just
24:13
organizing all the possible products now
24:17
you might think well this is a kind of a
24:19
crazy thing to do you know it's rather
24:21
complicated but it's not that bad but
24:24
we'll see that this is really a
24:27
remarkable operation okay and there's a
24:32
main reason why this is remarkable this
24:37
remarkable operation satisfies some what
24:46
not obvious but key property well we're
24:56
gonna say this is a theorem the theorem
24:58
is that matrix multiplication is
25:08
associative that means okay what does
25:14
that mean that if say a is an M by n B
25:20
is n by P and C is say P by Q then we're
25:31
allowed to multiply a B because this end
25:35
and this end of equal and then we'll get
25:38
an M by P matrix which we are allowed to
25:40
multiply by C so this product makes
25:43
sense a B times C makes sense on the
25:47
other hand B times C also makes sense
25:51
because if we go to B and C this P and
25:54
this Q are the same so that means this
25:57
product here BC is defined and the
26:01
result of that will be an N by Q and we
26:04
can multiply that by a because the last
26:07
index is n in a so both the left and
26:12
right hand side are well-defined and the
26:15
absolutely remarkable and beautiful fact
26:17
is that this thing is associative that a
26:22
B times C equals a times BC
26:25
okay this is a great and at this level
26:29
it should be a surprising because you
26:31
know why should this be true actually
26:34
what's one of these Vantage's of the
26:36
Maxell point of view if you look at the
26:38
maxilla so if orientation you see that
26:40
this is sort of true for much more
26:42
obvious reasons but okay
26:43
now there's a few more things we're
26:46
going to go back to this here so we have
26:47
to understand this theorem but I just
26:49
want to mention a few things so key
26:54
point is that matrix multiplication is
27:01
not commutative okay so I'm gonna
27:11
illustrate that with an example so let's
27:14
have a look at and I'll stick with two
27:16
by two because in the situation's not so
27:20
onerous to do my hand so suppose we have
27:22
the matrix one three four minus one and
27:26
that's usually how well read thing so
27:27
I'll read out the first row and then the
27:29
second row one three four minus one and
27:33
here's another matrix let's say two one
27:38
one maybe minus one five okay so this is
27:43
two by two this is two by two
27:46
let's see what the product is the
27:49
product is going to be 5 because this 2
27:51
and this 2 agree so what is that product
27:54
so remember what I told you you start
27:57
with the first column here and you
27:59
multiply it by each of the rows over
28:01
here the first entry will be 2 times 1
28:04
plus 3 times minus 1 that's minus 1 and
28:07
the next entry below it will be 4 times
28:11
2 plus minus 1 times minus 1 that's 9
28:14
and then for the next column we go to
28:17
the second column over here and still
28:19
multiply it by the 2 rows over here so
28:23
the first entry will be 1 times 1 plus 3
28:26
times 5 which is 16 and the next entry
28:29
is 4 times 1 plus minus 1 times 5 so
28:34
it's 4 minus 5 that's minus 1
28:38
okay but let's look at the same two
28:44
matrices but multiplied in the different
28:47
order so - 1 - 1 5 times 1 3 4 - 1 let's
28:52
do that product ok so just be consistent
28:58
so we start with this column here you
29:00
don't have to do this
29:03
underlining or circling I'm just doing
29:06
this so you can see what's happening but
29:08
shortly we're not going to be doing this
29:10
anymore ok but just to start with it's
29:13
good practice
29:14
ok so let's multiply these 2 rows by
29:18
this first column 1 4 so what do we get
29:21
- 1 times 1 4 is 2 plus 4 that's 6 and
29:24
then below it - 1 plus 20 is 19 and now
29:29
the second column we get by multiplying
29:31
the two rows here by the second column 3
29:34
minus 1 and that gives us 6 minus 1
29:38
which is 5 and minus 3 minus 5 which is
29:42
minus 8 ok so this is an example that a
29:47
times B does not equal the times a in
29:51
general this is very important this is
30:00
distinguishing this new arithmetic from
30:03
the familiar arithmetic that we're used
30:04
to with ordinary numbers with ordinary
30:07
numbers multiplication is commutative
30:09
three times five is the same as five
30:11
times three but here with matrices
30:13
crucially it's not but the associativity
30:18
of multiplication that is satisfied so
30:22
in our next lecture we'll start by
30:24
looking more carefully at this matrix
30:27
multiplication and seeing that it really
30:30
is associative