Search for probability and statistics terms on Statlect
StatLect
Index > Matrix algebra

Eigenvalues and eigenvectors

by , PhD

This lecture introduces the concepts of eigenvalues and eigenvectors of a square matrix. These are amongst the most useful concepts in linear algebra: studying the eigenvalues and eigenvectors of a square matrix is very frequent in applied work.

Table of Contents

Intuition

Let us first develop some intuition about eigenvalues and eigenvectors. To do so, we start from some concepts we explained in the lecture on the Determinant of a matrix.

Consider the linear space $S$ of all $2 imes 1$ real vectors, which can be represented as a Cartesian plane. A vector $sin S$ is a point in the plane, and the first and second entries of $s$ are the coordinates of the point.

Now, consider a $2 imes 2$ matrix A. The matrix transforms any set of points $Tsubset S$ (e.g., a rectangle, a circle) into another set of points $T_{A}$: [eq1]

If the points of $T$ form a region whose area is equal to $lpha $, and the area of the transformed region $T_{A}$ is $lpha _{A}$, then[eq2]

In other words, the determinant tells us by how much the linear transformation associated with the matrix A scales up or down the area of shapes. Eigenvalues and eigenvectors provide us with another useful piece of information. They tell us by how much the linear transformation scales up or down the sides of certain parallelograms.

Consider the parallelograms that have one vertex at the origin of the Cartesian plane. The four vertices are[eq3]where $x_{1}$ and $x_{2}$ are $2 imes 1$ vectors and 0 is the zero vector.

There are two vectors $x_{1}$ and $x_{2}$, called the eigenvectors of A, such that the associated parallelogram is transformed by A into a new parallelogram having vertices[eq4]where $lambda _{1}$ and $lambda _{2}$ are two scalars called the eigenvalues of A. In other words, the linear transformation multiplies the length of one pair of parallel sides by $lambda _{1}$ and the length of the other pair by $lambda _{2}$, but it keeps the angles of the parallelogram unchanged. The next figure provides an illustration of this kind of transformation: the original parallelogram (in blue) is transformed into another parallelogram (in red) by a matrix A whose eigenvalues are equal to $lambda _{1}=3/2$ and $lambda _{2}=2$.

Plot of a parallelogram formed by the eigenvectors of a matrix

Since a pair of parallel sides is scaled by $lambda _{1}$ and the other pair by $lambda _{2}$, the area of the parallelogram is scaled by a factor of [eq5]. But we also know that the area of the parallelogram is scaled by [eq6]. As a consequence,[eq7]that is, the determinant of a matrix is equal to the product of its eigenvalues, a fact that holds in general.

The definition of eigenvalues and eigenvectors we are going to provide below generalizes these concepts to linear spaces that can have more than two dimensions.

Definition

We are now ready to define eigenvalues and eigenvectors.

Definition Let A be a $K imes K$ matrix. If there exist a Kx1 vector $x eq 0$ and a scalar $lambda $ such that[eq8]then $lambda $ is called an eigenvalue of A and x an eigenvector corresponding to $lambda $.

This definition fits with the example above about the vertices of the parallelogram. The two vertices $x_{1}$ and $x_{2}$ are eigenvectors corresponding to the eigenvalues $lambda _{1}$ and $lambda _{2}$ because[eq9]Furthermore, these two equations can be added so as to obtain the transformation of the vertex $x_{1}+x_{2}$:[eq10]

Characteristic equation

Note that the eigenvalue equation[eq11]can be written as[eq12]where I is the $K imes K$ identity matrix. The latter equation has a non-zero solution only if the columns of the matrix $lambda I-A$ are linearly dependent, that is, if the matrix is singular. But a matrix is singular if and only if its determinant is zero. As a consequence, any eigenvalue of A must satisfy the equation[eq13]which is called characteristic equation.

The expression [eq14] is a monic polynomial of degree K in $lambda $, known as characteristic polynomial.

By using the fundamental theorem of algebra, it is possible to write the characteristic equation as[eq15]where [eq16] are the K solutions of the equation (i.e., the roots of the characteristic polynomial).

The fundamental theorem of algebra guarantees that exactly K solutions exist, but these solutions are not guaranteed to be real (i.e., they can be complex numbers), even when the entries of A are all real. They are also not guaranteed to be distinct, that is, two solutions could be equal.

Spectrum

In the previous section we have explained that a $K imes K$ matrix A has K not necessarily distinct and possibly complex eigenvalues. The set of all eigenvalues of A is called the spectrum of A.

Eigenspace

Note that if[eq11]then you can multiply both sides of the equation by a non-zero scalar $lpha $ and get[eq18]

In other words, if $lambda $ is an eigenvalue of A and x is an eigenvector corresponding to $lambda $, then any multiple of x is an eigenvector corresponding to $lambda $. Thus, the eigenvector corresponding to a given eigenvalue is not unique. In this section we prove that the set of all eigenvectors corresponding to a given eigenvalue is a linear space.

Definition Let A be a $K imes K$ matrix and $lambda $ one of its eigenvalues. The union of the zero vector and the set of all the eigenvectors corresponding to the eigenvalue $lambda $ is called the eigenspace of $lambda $.

Note that we include the zero vector in the eigenspace because eigenvectors are required to be non-zero.

The next proposition shows that an eigenspace is closed with respect to linear combinations, that is, it is a linear space.

Proposition The eigenspace corresponding to an eigenvalue is a linear space.

Proof

Suppose that $lambda $ is an eigenvalue of a square matrix A and take any two vectors $x_{1}$ and $x_{2}$ belonging to the eigenspace of $lambda $. Then,[eq19]Now take a linear combination $y$ of the two eigenvectors[eq20]where $lpha _{1}$ and $lpha _{2}$ are two scalars. Then,[eq21]Thus, $y$ is an eigenvector corresponding to $lambda $. In other words, any linear combination of the vectors of the eigenspace belongs to the eigenspace.

Clearly, once an eigenvalue $lambda ~$has been found (e.g., by solving the characteristic equation), the eigenspace of $lambda $ can be found by solving the linear system[eq12]

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Consider the matrix [eq23]Show that [eq24]is an eigenvector of A and find its corresponding eigenvalue.

Solution

We have that[eq25]Thus, x is an eigenvector of A corresponding to the eigenvalue $lambda =1$.

Exercise 2

Define[eq26]Find the eigenvalues of A by solving the characteristic equation.

Solution

The characteristic equation is[eq27]Therefore, the eigenvalues of A are $lambda _{1}=2$ and $lambda _{2}=1$.

How to cite

Please cite as:

Taboga, Marco (2021). "Eigenvalues and eigenvectors", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/eigenvalues-and-eigenvectors.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.