This lecture introduces matrix multiplication, one of the basic algebraic operations that can be performed on matrices.
Before defining matrix multiplication, we need to introduce the concept of dot product of two vectors.
Definition
Let
be a
row vector and
a
column vector. Denote their entries by
and by
,
respectively. Then, their dot product
is
Note that in the above definition the order of the product matters, that is
is not the same as
,
because the first vector
(
)
needs to be a row vector, and the second one
(
)
needs to be a column vector. Furthermore, the dot product is defined only if
and
have the same number of entries
(
).
Example
Let
be a
vector defined
by
and
a
vector defined
by
Their
dot product
is
We are now ready to define matrix product.
Definition
Let
be a
matrix and
a
matrix. Then, their product
is a
matrix whose
-th
entry is equal to the dot product between the
-th
row of
and the
-th
column of
,
for
and
.
In other words, the
-th
entry of
is
Note that the order of the product matters, that is
is not the same as
.
Furthermore, the number of columns of
needs to be equal to the number of rows of
(in which case the two matrices are said to be conformable
for the multiplication
).
The next diagram summarizes the dimensions involved in matrix
multiplication:
Example
Define the
matrix
and
the
matrix
They
are conformable for the multiplication
because the number of columns of
is equal to the number of rows of
.
The dimension of the matrix
is
.
The product
is
where,
for example, the
-th
entry of
has been obtained from the dot product of the second row of
with the first column of
:
Why is matrix multiplication defined in this way? There are many possible answers to this question, but the simplest one has to do with the need of obtaining a simple matrix representation for systems of linear equations. The next example shows how.
Example
Consider the following system of two equations in two
unknowns:This
can be represented in matrix form
as
where
the matrix of coefficients
is
the
vector of unknowns
is
and
the vector of constants
is
You
can easily check that the two ways of writing the system of equations are
equivalent by performing the matrix
multiplication
Another reason why matrix multiplication is defined in the manner shown above is that it allows us to easily deal with input-output systems in which given outputs can be obtained from fixed combinations of inputs.
Example
A factory can produce two goods, denoted by
and
,
using different combinations of two inputs,
and
.
In particular,
units of
and
unit of
are needed to produce a unit of
,
and
unit of
and
units of
are needed to produce a unit of
.
This information can be summarized by the input-output
matrix
where
the two rows correspond to the two outputs and the two columns correspond to
the two inputs. Each unit of
costs
dollars, and each unit of
costs
dollar. This information can be summarized by the vector of
prices
In
order to find the costs of producing the two outputs, it suffices to perform
the following matrix
multiplication
So,
both outputs have a production cost of
dollars.
As we have already said, unlike multiplication of real numbers, matrix
multiplication does not enjoy the commutative property, that is,
is not the same as
.
However, some of the properties enjoyed by multiplication of real numbers are
also enjoyed by matrix multiplication.
Proposition (distributive
property)
Matrix multiplication is distributive with respect to matrix addition, that
is,for
any matrices
,
and
such that the above multiplications and additions are meaningfully defined.
Let us start with the
productLet
and
be
matrices, and
an
matrix. Denote a generic
-th
element of the matrix
by
,
and a generic
-th
element of the product between
and
by
.
By the definitions of matrix addition and
matrix multiplication, we have
that
where:
in steps
and
we have used the definition of matrix multiplication; in step
we have used the definition of matrix addition. This holds for any
-th
element of the matrix. Therefore, we have
that
With an almost identical argument it is possible to prove
that
Proposition (associative
property)
Matrix multiplication is associative, that
is,for
any matrices
,
and
such that the above multiplications are meaningfully defined.
Suppose
has dimension
,
has dimension
,
and
has dimension
.
Associativity holds because a generic
-th
element of the matrix
is
where
we have used the definition of product between
and
(step
),
between
and
(step
),
between
and
(step
),
between
and
(step
).
Other properties of matrix products are listed here.
Proposition
Let
be a
matrix and
a
matrix. Let
and
be their transposes.
Then,
The
-th
entry of
is the dot product of the
-th
row of
and the
-th
column of
:
By
the definition of matrix transpose, the latter is equal to the
-th
entry of
:
The
-th
entry
is the dot product of the
-th
row of
and the
-th
column of
:
Since
the
-th
row of
is equal to the
-th
column of
,
and the
-th
column of
is equal to the
-th
row of
,
we
have
Thus,
for
any
and
.
Therefore,
Below you can find some exercises with explained solutions.
Define a
matrix
and
a
matrix
Compute
the product
.
The dimensions involved in this
multiplication are summarized in the following
diagram:Thus,
is a
matrix such that for each
and
,
the
-th
element of
is equal to the dot product between the
-th
row of
and the
-th
row of
:
Given the matrices
and
defined above, compute the product
.
The matrices
and
are not conformable for the multiplication
because the number of columns of
is not equal to the number of rows of
.
Therefore, multiplication cannot be carried out.
Define a
column
vector
and
a
row
vector
Compute
the product
.
The dimensions involved in this
multiplication are summarized in the following
diagram:Thus,
is a
matrix. It is computed as follows:
Note
that each element of
is the product of a row of
with a column of
.
But the rows of
are scalars, because
is a column vector, and the columns of
are also scalars, because
is a row vector. As a consequence, each entry of
is obtained as the product of two scalars.
Please cite as:
Taboga, Marco (2021). "Matrix multiplication", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/matrix-multiplication.
Most of the learning materials found on this website are now available in a traditional textbook format.