ENGR 1405 Engineering Mathematics 1

2000 Fall

Course Notes

Chapter 1:  Systems of Linear Equations and Matrices

Contents:

§1.1     Systems of Linear Equations

§1.2     Introduction to Matrices

§1.3     Special Matrices

§1.4     Matrix Row Reduction

§1.5     The Inverse (by Row Reduction)

§1.6     Determinants

§1.7     The Matrix Inverse and Cramer’s Rule

§1.8     Vector Spaces

§1.9     Eigenvalues and Eigenvectors

 

§1.1 Systems of Linear Equations

Linear equations occur in many Engineering applications. The use of Newton’s Laws to find the algebraic sum of all the forces acting on a particle in equilibrium, and the application of Kirchoff’s Laws to determine the various currents in a simple series electrical circuit are two examples of systems of linear equations.

 

A linear equation involving the variables x1 , x2 , x3 , ... , xn and the constants a1 , a2 , a3 , ... , an is a relationship of the form

a1 x1 + a2 x2 + a3 x3+ ... + an xn = b

which can be abbreviated, using the “sigma” notation, as

A linear system of equations involving the same variables  x1 , x2 , x3 , ... , xn  is of the form

a11 x1 + a12 x2 + a13 x3+ ... + a1n xn = b1

a21 x1 + a22 x2 + a23 x3+ ... + a2n xn = b2

a31 x1 + a32 x2 + a33 x3+ ... + a3n xn = b3

ap1 x1 + ap2 x2 + ap3 x3+ ... + apn xn = bp

which can be abbreviated as

This is called a system of p equations in n unknowns.

Note:

The number of equations is given first and the number of unknowns is indicated second.

The constants  aij  are the coefficients and the constants  bi  are the right side constants.

 

A specific example of such a system is

3 x1 + 2 x2 + 4 x3   =  1

x1 - x2 + 2 x3  =  4

2 x1 + 5 x2 - 3 x3   =  6

x3   =  4

This is a system of 4 equations in 3 unknowns.

 

Solution:

A solution of a system of  p equations in n unknowns is a set of values, one value for each of the unknowns, which satisfy each of the p equations at the same time.

 

The first objective in this chapter will be to develop a systematic approach for determining if a solution exists, and then finding the solution of the system of p equations in n unknowns.

Since in many actual situations the number of unknowns in a system of linear equations is very large, subscripted variables will be used in the development of our systematic approach. Hence, we will be using the variables  x1 , x2 , x3 , ... , xn   instead of the variables  x, y, z, ...   Also the constant coefficients of the variables will, in general, be represented by doubly subscripted variables of the form aij ,where the first subscript refers to the equation number, and the second  indicates the variable number. The constant values to the right of the equal sign will also have a single subscript which will refer to the equation number.

 

For the specific example

3x - 5y = 2

12x + y = 4

the general references are

x1

=

x

 

a11

=

3

x2

=

y

 

a12

=

-5

b1

=

2

 

a11

=

12

b2

=

4

 

a12

=

1

 

As a first step in developing a systematic approach to solving a system of linear equations the following general principles are to be used:

 

1.      Whenever possible, equivalent systems of equations will be retained. Hence, if the initial system has 4 equations, then, in subsequent equivalent systems that we write down, 4 equations should be retained as long as possible.

 

2.      Elimination of variables will proceed from left to right and from top to bottom, until it is possible to use back substitution.

 

Elementary Operations:

There are only three permissible elementary operations that can be used for solving a system of linear equations. These elementary operations are:

         (a) Interchange equation “i” with equation “j”.

                     This is represented symbolically by   Ei « Ej

         (b) Replacement of equation “j” by equation “j” multiplied by the non-zero constant k.

                     This is represented symbolically by   Ej¬ kEj

               (c) Replacement of equation “i”  by equation “i” plus k  times equation “j”.

                     This is represented symbolically by   Ei¬ Ei  + kEj

 

Theorem:

Two systems of linear equations are equivalent iff one system can be (or has been) obtained from another system of linear equations by means of one or more of the above set of elementary operations.

 

Theorem:

Equivalent systems of linear equations have the same solution set.

 

Sample Problem 1:

Determine, if possible, the solution for the following system of linear equations

Equation Set

 

Operation

2x1 + 6x2 + 4x3 = 18

5x1 + 6x2 + 4x3 = 24

-2x1 + x2 + 3x3 = 3

 

 

E1 ¬ 1/2 E1

 

x1 + 3x2 + 2x3 = 9

5x1 + 6x2 + 4x3 = 24

-2x1 + x2 + 3x3 = 3

 

 

E2 ¬ E2  -5 E1

 

x1 + 3x2 + 2x3 = 9

-9x2 - 6x3 =  -21

-2x1 + x2 + 3x3 = 3

 

 

E3 ¬ E3  + 2E1

 

x1 + 3x2 + 2x3 = 9

-9x2 - 6x3 =  -21

7x2 + 7x3 = 21

 

 

E2 « E3

 

x1 + 3x2 + 2x3 = 9

7x2 + 7x3 = 21

-9x2 - 6x3 =  -21

 

 

E2 « 1/7 E2

 

x1 + 3x2 + 2x3 = 9

x2 + x3 = 3

-9x2 - 6x3 =  -21

 

 

E3 ¬ E3  + 9 E2

 

x1 + 3x2 + 2x3 = 9

x2 + x3 = 3

3x3 = 6

 

 

E3 ¬ 1/3 E3

 

x1 + 3x2 + 2x3 = 9

x2 + x3 = 3

x3 = 2

 

 

Since a value has now been found for the variable x3 , it is possible to use back substitution to determine the values of the remaining variables, as follows:

 

x2 = 3 - x3 ,    x3 = 2

ή

x2 = 1

 

 

 

x1 = 9 - 3x2 - 2x3 ,    x2 = 1 ,    x3 = 2

ή

x1 = 2

 

Hence, the solution set for this problem is { x1 , x2 , x3 } = {2 , 1, 2}.

 

Note:

It has been determined that finding the solution for a system of linear equations, by using a procedure similar to that used above is the most efficient method.

 

A program that carries out the arithmetic of your chosen elementary operations for you is available from the course Web site at "www.engr.mun.ca/~ggeorge/1405/".   Software for the solution of linear systems (including the choice of which elementary operations to perform) is available commercially, (for example, in "Matlab").

Consider the system

ax + by

=

c

lx + my

=

p

Geometrically, in ϊ2 , these equations represent two lines.   Given our knowledge of geometry, we know that only three possible situations can occur:

(1) the lines intersect in a single point

(2) the lines are parallel and coincident and thus have infinitely many common points

(3) the lines are parallel but not coincident and thus have no common points

        

By analogy only three possibilities can occur for any system of linear equations:

(a) the system has a unique solution

(b) the system has infinitely many solutions

(c) the system has no solution

 

Sample problem 1 is an example of the first case. Two more sample problems will be given, one for each of the remaining cases.

 

Sample Problem 2:

Determine, if possible, the solution for the following system of linear equations

Equation Set

 

Operation

x1 + 3x2 - x3 = 1

x1 + 17x2 - 19x3 = 13

3x1 + 2x2 + 6x3 = -3

 

 

E2 ¬ E2 - E1

 

x1 + 3x2 - x3 = 1

14x2 - 18x3 = 12

3x1 + 2x2 + 6x3 = -3

 

 

E3 ¬ E3  - 3 E1

 

x1 + 3x2 - x3 = 1

14x2 - 18x3 = 12

-7x2 + 9x3 = -6

 

 

E2 ¬ 1/2 E2

 

x1 + 3x2  - x3 = 1

7x2 - 9x3 = 6

-7x2 + 9x3 = -6

 

 

E3 ¬ E3 + E2

 

x1 + 3x2 - x3 = 1

7x2  - 9x3 = 6

0x3 = 0

 

 

 

 

It is now possible to use back substitution to determine the values of the remaining variables, as follows:

7x2 = 6 + 9x3

ή

x2 = 6/7 + 9/7x3

 

 

 

x1 = 1 - 3x2 + x3 ,    x2 = 6/7  + 9/7x3

ή

x1 = -11/7 - 20/7 x3

 Hence, the solution set for this problem is

x1

=

-11/7 - 20/7 x3

x2

=

  6/7 + 9/7 x3

x3

Ξ

ϊ

Sample Problem 3: Determine, if possible, the solution for the following system of linear equations

Equation Set

 

Operation

x1 + x2 + 2x3 = 4

x2 + 2x3 = 11

3x2 + 7x3 = 2

x3 = 4

 

 

E3 ¬ E3 - 3E2

 

x1 + x2 + 2x3 = 4

x2 + 2x3 = 11

x3 = -31

x3 = 4

 

 

 

 

 

Clearly, there is no possible solution for this problem.

 

§1.2 Introduction to Matrices

As mentioned previously, in many practical situations the number of equations to be solved in a system of linear equations is frequently very large. In an effort to reduce the amount of writing that must be done when solving such a system, Mathematicians have introduced a shorthand notation for representing a system of linear equations. This shorthand notation uses matrices. Hence, we now introduce a number of definitions and properties for matrices.

 

Matrix:       

A matrix is a rectangular array of characters (numbers and/or letters) arranged in rows and columns.   Each row must have the same number of characters, and each column must have the same number of characters which does not have to be equal to the number of characters in each of the rows.

 

Note:          

A matrix is usually represented in a general fashion by an uppercase letter, for example  A.

        

Entry:                    

In a matrix the element located in row “i” of column “j” is represented in a general manner by using a doubly subscripted lower case letter corresponding to the uppercase letter used to identify the matrix being considered.   This element is called an entry of the matrix.   For example in the matrix  A  mentioned above a general entry would be aij .   The entry in row 43 and column 7 of the matrix is a43 7.

 

Dimension or Size of a Matrix:    

If a matrix  A  consists of “p” rows with each row containing “n” elements or entries, then the dimension or size of the matrix  A  is indicated by stating first the number of rows and then the number of elements in a row.   These two values are separated by the symbol “΄”. Hence, for the matrix  A  above we write


 

 

dim [A]

=

p ΄ n

or

dim A

=

p ΄ n

or

A

is

p ΄ n

or

[A]p ΄ n

 

 

Equality:     

Two matrices  A and B  are said to be equal iff the following conditions are both satisfied:

(1)

the matrices are the same size

 

dim [A] = dim [B]

(2)

corresponding entries are equal

 

aij = bij for every ordered pair {i , j}

 

If both of these conditions are satisfied then we may write  A = B.

 

Addition:    

Two matrices  A and B  with corresponding entries  aij and bij  can be added together to generate a new matrix  C  with entries  cij  iff dim [A] = dim [B].   The entry  cij  is equal to the sum of the entries  aij and bij.   Symbolically we would write

 

C = A + B

where

cij  =  aij + bij

 

As with vectors two different types of multiplication involving matrices are possible.

They are called scalar multiplication and matrix multiplication.

 

Scalar Multiplication:        

If “k” is any non-zero scalar (constant) and if  A  is any matrix with entries  aij  and

dim [A] = p΄n, then there exists a matrix  B = kA  with dim [B] = dim [A] = p΄n, and entries

bij = kaij.   The matrix  B  is called the scalar multiple of A by k.

 

Matrix Multiplication: 

If  A and B  are any two matrices with dim [A] =  p΄n and dim [B] = m΄q, then:

(1) the matrix product  C = AB  exists iff  n = m, and then dim [C] = p΄q

(2) the matrix product  D = BA  exists iff  p = q, and then dim [D] = m΄n.

 

For the matrix  C  the entry in row “i” and column “j” represented by  cij  is determined by the formula  where  aik  and   bkj  are the entries in the ith row of  A  and the  jth column of  B respectively.

 

For the matrix  D  the entry in row “i” and column “j” represented by dij  is determined by the formula  where  bik  and   akj  are the entries in the ith row of  B  and the jth column of  A respectively.

 

Properties of Matrix Multiplication:

Assuming that each of the matrix products listed below exists, then each of the following can be shown.

(1)

AB Ή BA   in general

(not commutative)

(2)

A(B + C) = AB + AC

(distributive over addition)

(3)

A(kC) = (kA)C = k(AC)

 

(4)

A(B + kC) = AB + kAC

 

(5)

A(BC) = (AB)C

(associative)

 

Sample Problem 4:   Evaluate  C = A + B  for the matrices

 

Sample Problem 5:   Determine, if possible,  C = AB  and  D = BA  for the matrices

Here   dim [A] = 2΄2,  and   dim [B] = 2΄3

 

Because matrix  A  has “2” columns and matrix  B has “2” rows, the matrix product  C = AB  is defined and  dim [C] = 2΄3.

 

But  D = BA  is not defined, because matrix  B  has “3” columns while matrix  A  has only “2” rows.

 

The entries of the matrix  C  are:

 

 

 

§1.3 Special Matrices

In this section a number of matrices that are given special names are introduced.

 

Column Matrix:

A column matrix is a matrix that has all of its entries in only one column.

 

Row Matrix:

A row matrix is a matrix that has all of its entries in a single row.

 

Square Matrix:

A matrix  A  with  dim[A] = p΄n  is a square matrix iff  p = n.

 

Triangular Matrix:

A matrix is called an upper triangular matrix iff:

(a)    the first non-zero entry in each row is to the right of the first non-zero entry in the previous row and,

(b)    all rows whose entries consist of zeros only are found at the bottom of the matrix.

 

Diagonal Matrix:

A matrix  D  is a diagonal matrix iff:

(a)    D  is a square matrix and,

(b)    the entries  dik = 0  for  i Ή k.

 

Identity Matrix:

The identity matrix, represented by  I, is a diagonal matrix with entries  ipq = 1 for  p = q.

 

Transposed Matrix:

Given any matrix  A  with  dim[A] = p΄n  and entries  aik , there exists a matrix represented by  AT or A*  with  dim[A*] = n΄p  and entries   aki  .   The matrix  AT or A*  is called the transpose of  A.

 

Properties of Transposition:

Given any matrices  A, B  and any real non-zero scalar  k  the following are always true:

(1)

(A*)*

=

A

(2)

(kA)*

=

k(A)*

(3)

(A + B)*

=

A* + B*

(4)

(AB)*

=

B*A*

 

Orthogonal Matrix:

A matrix  B  is an orthogonal matrix iff:

(a)    B  is a square matrix and,

(b)    BB* = BBT = B*B = BTB = I  

 

Invertible Matrix:

A matrix  C  is an invertible matrix iff:

(a)    C is a square matrix and,

(b)    there exists another matrix D, of the same dimension as C, such that CD = DC = I.

If this is true the matrix  D  is usually represented by  C-1, and is called the inverse of  C.

 

Matrices and Systems of Linear Equations:

Every system of “p” linear equations in “n” unknowns of the form:

a11 x1 + a12 x2 + a13 x3+ ... + a1n xn = b1

a21 x1 + a22 x2 + a23 x3+ ... + a2n xn = b2

a31 x1 + a32 x2 + a33 x3+ ... + a3n xn = b3

ap1 x1 + ap2 x2 + ap3 x3+ ... + apn xn = bp

can always be represented by a matrix equation of the form  AX = B  where

        

(a)

is called the coefficient matrix

 

 

(b)

is called the matrix of unknowns

(c)

is called the matrix of constants

 

(d)

is called the augmented matrix

 

 

§1.4 Matrix Row Reduction

In §1.1 a set of three elementary operations for a system of linear equations was introduced.   For a matrix or for a matrix equation the same set of three elementary operations are permitted.   These three operations can be used on either the rows or columns of a matrix.   However, since we will be working with matrices that are considered to have been generated from a system of linear equations, the operations will only be used on the rows.   For convenience these three elementary row operations are reproduced below, along with the necessary minor modifications to indicate row operations instead of equation operations.

 

Elementary Row Operations:

There are only three permissible elementary row operations that can be used for generating an equivalent matrix. These elementary operations are:

  (a)   Interchange row “i” with row “j”.

         This is represented symbolically by   Ri « Rj

  (b)   Replacement of row “j”  by row  “j”  multiplied by the non-zero constant  k.

         This is represented symbolically by   Rj ¬ k Rj

  (c)   Replacement of row “i” by row “i” plus  k times row “j”.

         This is represented symbolically by   Ri ¬ Ri + k Rj

 

Row Echelon Form:           

A matrix  A  is said to be in row echelon form iff:

(a)    the value of the first non-zero entry in each row has a value of unity (one),

         (this is called a leading one), and

(b)    the value of the first non-zero entry in each row is to the right of the first non-zero entry in the previous row, and

(c)    all rows whose entries consist entirely of zeros are grouped together at the bottom of the matrix.

 

Reduced Row Echelon Form:       

A matrix  A  is said to be in reduced row echelon form iff:

(a)    the matrix is in row echelon form, and

(b)    each column containing a leading one has all other entries in that column equal to zero.

 

Theorem: 

For a square linear system of “p” equations in “p” unknowns with associated matrix equation

AX = B  (A is a square p΄p matrix), if any one of the following statements is true, then all are true (or if any one is false, then all are false):

(a)    A-1  exists

(b)    AX = B  has a unique solution for every column matrix B

(c)    AX = 0  has only the trivial solution X = 0

(d)    Rank (A) = Rank (A½B) = p

(e)    the matrix  A  has the equivalent row echelon form  I

 

Rank:

The rank of a matrix is numerically equal to the number of rows containing non-zero entries in the equivalent row echelon matrix or reduced row echelon matrix.

 

Examples:

Matrix

Row Echelon Form

Reduced Row Echelon Form

Rank

no

no

unknown

yes

no

3

yes

yes

3

 

Consistent Systems:

A system of linear equations is said to be consistent iff the rank of the associated augmented matrix is equal to the rank of the associated coefficient matrix (this means that the system of equations has at least one solution).   The system of linear equations is inconsistent otherwise.

 

Solution Forms:

For a system of “p” equations in “n” unknowns with associated coefficient matrix  A, and augmented matrix  A|B  the following can be shown to be true:

(1)    the system has a unique solution iff  Rank (A) = Rank (A|B) = n

(2)    the system has infinitely many solutions iff  Rank (A) = Rank (A|B) < n

(3)    the system has no solutions iff  Rank (A) < Rank (A|B)

 

Note:

(1) For a system of “p” equations in “n” unknowns with associated augmented matrix A|B it is always true that   Rank (A|B) £ min { p , n}

 

(2) For a system of “p” equations in “n” unknowns with associated augmented matrix A|B, dim (A) = p΄n.   If  Rank (A) = r < n  and  Rank (A|B) = r,  then the associated system has a family of solutions that contains (n - r) parameters (free variables).

 

(3)    If numbers are not all stored exactly as integers or fractions, then round-off errors can cause a computer program to return an incorrect value for the rank of a matrix.

 

Sample Problem 6: Use matrices to find the solution(s), if any, for the following

 

Since  Rank (A) = Rank (A|B) = 2 < 3,  infinitely many solutions exist.

This is a one parameter family of solutions given by  x1 = -7x2,   x2 Ξ ϊ,   x3 = 3x2.

 

An equivalent solution set is generated by the reduced row echelon form

from which    x1 = -(7/3) x3,   x2 = (1/3) x3,   x3 Ξ ϊ.

 

§1.5 The Inverse (by Row Reduction)

The same technique that was used in the previous section for solving a system of linear equations can be used to determine the inverse, if it exists, for a given matrix  A.

 

The matrix A must be a square matrix with  dim(A) = p΄p .  

The following algorithm gives the essential steps of the procedure.

 

Inversion Algorithm:

(1)    Augment the given matrix  A, with  dim(A) = p΄p, by an identity matrix  I  of the same dimension as  A  to generate an augmented matrix of the form  A|I.

(2)    Apply the elementary row operations to the matrix  A|I  until the initial  p΄p block of  A|I  is in reduced row echelon form.

(3)    If the initial p΄p block of this new matrix equivalent to  A|I  is a  p΄p identity matrix, then the inverse exists, and the final p΄p block of this new matrix equivalent to  A|I  is the inverse of the matrix  A  and is represented by  A-1.

(4)    In all other cases the inverse does not exist.

 

Properties of the Inverse:

(1)    The identity matrix  I  is invertible and  I-1 = I.

(2)    If  A  is an invertible matrix then so also is  A-1, and  (A-1)-1 = A.

(3)    If  A, B  are invertible matrices, then so also is  AB, and  (AB)-1 = B-1A-1.

(4)    If  A  is an invertible matrix, then so also is  Ak, and  (Ak)-1 = (A-1)k.

(5)    If  A  is an invertible matrix and if  c  is any non-zero constant, then .

(6)    If  A  is an invertible matrix, then so also is  AT, and  (AT)-1 = (A-1)T.

(7)    If  A  is an invertible matrix, and if  AB = AC  or if  BA = CA, then  B = C.

(8)    If  A  is an invertible matrix, then  A-1  is unique.

(9)    If  A  is an invertible  n΄n matrix, then  Rank (A) = n.

(10) If  A  is an invertible  n΄n matrix, then  A can be reduced to an  n΄n identity matrix by means of the elementary row operations.

(11) If  A  is an invertible matrix, then  A is a product of elementary matrices.   An elementary matrix is a matrix obtained from an identity matrix by means of exactly one elementary row operation.

(12) If  A  is an invertible matrix, then the matrix equation

(a)    AX = 0  means that  X = 0  is the unique solution

(b)    AX = B  means that  X = A-1B  is the unique solution.

 

Note:            If  B = 0, then the linear system  AX = 0  is a homogeneous linear system, for which X = 0  (the trivial solution) is guaranteed to be a solution.   If  A  is a square singular matrix, then the homogeneous linear system has infinitely many solutions.

 

         If a homogeneous linear system has fewer equations than unknowns, then it has infinitely many solutions.

 

Sample Problem 7: 

         (a)       Determine the inverse for A, where:

         (b)       Use the inverse determined above to solve the system of linear equations:

x + 6y

=

31

12x + 10y

=

62

        

         Here, according to the theory, it is necessary to determine the matrix X such that X = A-1B

 

 

§1.6 Determinants

        

Minor:

If  A  is a  p΄p matrix with entries aij, then the minor  Mij  is the (p-1)΄(p-1) matrix obtained from  A  by deleting row “i” and column “j”, the row and column in which  aij  appears.

 

Cofactor: 

If  A is a  p΄p  matrix with entries  aij, then  Cij is the determinant of the minor  Mij  times (-1) i+j. Symbolically we write  Cij = (-1) i+j | Mij | = (-1) i+j det (Mij).

 

Determinant:

For any  p΄p  matrix A with entries  aij, there is associated with it a scalar value called the determinant, represented by either |A| or det(A).   This scalar value is calculated according to the following recursion relationship:

(1)    If  dim(A) = 1x1, then  |A| = the single entry of  A.

(2)    If  dim(A) = 2x2 with entries  aij, then

(3)    If  dim(A) =  p΄p  with entries  aij, then

Note:           The determinant can be determined (expanded) along any row or column.

 

Sample Problem 8: 

Find the minors  M1 3, M2 3, M3 3, the cofactors  C1 3, C2 3, C3 3, and the determinant of the matrix

 

 

Properties of the Determinant:

If A is a  p΄p  matrix, then:

(1)    det (A) = the product of the diagonal entries if A is a triangular matrix

(2)    det (A) = 0 if

(a)    A  has a row or column whose entries are all zeros

(b)    A  has two identical rows or columns

(c)    A  has  Rj = kRi  or  Cj = kCi  for  k Ή 0

Theorem:   

If A is a   p΄p  matrix, then the following statements are equivalent

(either all are true or all are false):

(1)    |A| Ή 0

(2)    Rank A = p

(3)    A-1 exists

(4)    the rows (columns) of A are linearly independent [i.e. they form a basis for the real space ϊp]

(5)    the only solution for the matrix equation  AX = 0  is the trivial solution X = 0

(6)    the matrix equation AX = B has the unique solution X = A-1B

 

Theorem:   

If  B is a   p΄p  matrix obtained from the   p΄p  matrix  A  by means of :

(a)    an interchange of row “i” with row “j”( Ri « Rj), then  |B| = - |A|

(b)    Multiplication of row “j” by the non-zero constant k (Rj ¬ kRj) , then  |B| = k |A|

(c)    Replacement of row “i” by row “i” plus  k times row “j” ( Ri ¬ Ri  + kRj) ,

         then  |B| = |A|

 

Sample Problem 9:  Find the determinant  |A| or |B| or |C| or |D| as appropriate:

 

                                                                                 [Expanding along row 1:]

 

Inverse:      

If A is a   p΄p  matrix with  |A| Ή 0, then A-1 exists and

where  adj A  is the adjoint of the matrix  A.

 

Adjoint:      

The adjoint matrix for the  p΄p  matrix  A  is the transpose of the  p΄p  matrix of cofactors of

the matrix  A.

 

Sample Problem 10: 

By using the theorems above determine whether or not the vectors listed below are linearly independent.    If they are, then they are said to form a basis for ϊ3.

 

 

First we form the matrix  A = (v1 v2 v3), where  vi  is the column matrix representation of the vector

Next, the determinant is evaluated.   Expanding down the first column of  A:

Since the determinant is non-zero, the vectors are linearly independent and hence form

a basis for ϊ3.

 

Sample Problem 11:             Find |A|, adj A, and A-1 when

 

 

 

§1.7 The Matrix Inverse and Cramer’s Rule

We have already stated a number of times that if  AX = B  is the matrix equation associated with a linear system of “p” equations in “p” unknowns, then the unique solution, if it exists, is given in matrix form by  X = A-1B, where  A-1  is the inverse of the matrix  A.   We also found two ways of determining  A-1.

 

Sample Problem 12:             By using both methods for determining  A-1 solve

4x1 - 7x2

=

4

2x1 - 3x2

=

8

 

Method 1:

Method 2:

 

Also note the simple form for the inverse of an invertible 2΄2 matrix:

 

Cramer’s Rule:

For the matrix equation  AX = B  if   |A| Ή 0, then the unique solution has the variables xj defined according to the following formula:

where Dj is the determinant of the  p΄p  matrix obtained by replacing column  j  of the matrix  A by the entries of the column matrix  B.

 

Sample Problem 13:                         By using Cramer’s Rule solve

4x1 - 7x2

=

4

2x1 - 3x2

=

8

 

 

 

Sample Problem 14:                         By using Cramer’s Rule solve

x1 + 2x2 - 4x3

=

3

2x1 - 5x2 + 2x3

=

6

3x1 - 4x2 - 6x3

=

5

 

Expanding each determinant along the top row:

 

§1.8 Vector Spaces

        

Vector Space

A vector space is a non-empty structure whose elements satisfy a specific set of relationships.

 

Note:           Even though these structures are called vector spaces, the elements in the space do not need to be vectors, as will be seen from a number of the examples to be considered.  The name “vector space” has been applied to any structure that satisfies the properties listed below because, originally, the first such set that satisfied the relationships was the set of vectors.   It was then found that a number of other structures also satisfied the same set of relationships.

 

Theorem:      

If     (i)           are any elements of  V [i.e. ]

 (ii)   c and d are any real scalars (constants), and

(iii)   there are operations called addition  and multiplication by a scalar  defined on the structure V,

then V is a vector space iff:

(a)                                                            [closure under addition]

(b)                                                            [closure under scalar multiplication]

(c)    and   [addition commutative]

(d)    and  [addition associative]

(e)    and    [distributive properties]

(f)     and

(g)    and

(h)    for the real scalar 1 if , then            [existence of identities]

(i)     there exists an element of  V represented by  such that

(j)     there exists an element of  V represented by  such  that                             [existence of inverse under addition]

 

Note:           In order to prove that a given structure (with addition  and multiplication by scalars  defined in some way) is a vector space it is necessary to show that all of the items (a) to (j) are satisfied.

 

Some of the standard examples of vector spaces with addition and multiplication by scalars defined in the usual way are:

(1)    the set of all real n-dimensional vectors (ϊn), with an element in ϊn represented

        

(2)    the set of all  p΄n matrices represented in general by  Mp,n

(3)    the set of all real n-dimensional polynomials (-n) where a general member is represented by

Subspace:

If  W  is a non-empty subset of the vector space  V  with the same operations of addition and multiplication by scalars defined on it as on  V, then  W  is a subspace of  V iff:

(a)    W is closed under the operation of addition  (i.e.  ή )

(b)    W is closed under the operation of multiplication by scalars (i.e.  and

         c any scalar ή )

If  W is a subspace of V, then this fact can be represented symbolically by writing  W Ν V.

 

Sample Problem 15:

Show that  W = {M2,2 | the entries sum to zero }, with the normal operations of addition and multiplication by a scalar, is a subspace.

Here it is necessary to prove that:

(1)    W  is non-empty

(2)    W  is closed with respect to the operation of addition

(3)    W  is closed under the operation of multiplication by a scalar

(1)    Let .    Since the sum of the elements of  E is zero, it follows that W is non-empty.

(2)    Let  be elements of W

\

a + b + c + d

=

0

and

u + v + w + x

=

0

         Set C = A + B

        

         and (a+u) + (b+w) + (c+v) + (d+x) = (a+b+c+d) + (u+v+w+x) = 0+0 = 0

         \C is an element of W, and hence W is closed with respect to the operation of addition.

(3)    Let  be an element of W and let “k” be any scalar.

\

u + v + w + x

=

0

         Set D = kB

            and    (ku) + (kw) + (kv) + (kx) = k(u+v+w+x) = k(0) = 0

         \D is an element of W, and hence W is closed under the operation of

         multiplication by a scalar.

         Since all of the requirements are satisfied, W is a subspace of  M2,2.

 

Sample Problem 16:                         Determine whether

,

with the normal operations of addition and multiplication by a scalar, is a subspace.

(1)    Let .   Since this is an element of W, it follows that W is non-empty.

(2)    Let  be elements of W

then

a + 2b - c = 0

,

a - d = 0

and

w + 2x - y = 0

,

w - z = 0

         Set              

where

(a+w) + 2(b+x) - (c+y) = (a + 2b - c ) + ( w + 2x - y ) = 0 + 0 = 0

and

(a+w) - (d+z) = (a-d) +  (w-z) = 0 + 0 = 0

         \ is an element of W, and hence W is closed with respect to the operation of addition.

(3)    Let  be an element of W and let “k” be any scalar.

\

w + 2x - y = 0

,

w - z = 0

         Set

        

        

where

(kw) + 2(kx) - (ky) = k (w + 2x - y ) = k (0) = 0

and

(kw) - (kz) = k(w-z) = k(0) = 0

         \ is an element of W, and hence W is closed under the operation of multiplication by a scalar.

         Since all of the requirements are satisfied, W is a subspace.

 

Sample Problem 17:                        

Determine whether  (with addition defined by

(a, b) Ε (c, d) = (a+d , b+c) and scalar multiplication defined as k Δ (a, b) = (ka, 0) whenever

(a, b) and (c, d) Ξ ϊ2) is a vector space.

 

Since W contains all vectors (or ordered pairs) in ϊ2 it is non-empty.   Also by the definitions of addition and multiplication by a scalar, W is closed under both operations.   However W cannot be a subspace because W fails to satisfy at least one of the properties for a vector space.    Of the five conditions in the theorem for vector spaces that are not satisfied, the easiest to demonstrate is part (h):     1 Δ (a, b) = (1a, 0) Ή (a, b).       [The other properties that fail are (c), (d), (f), and (g).]

Note that under these new definitions of addition and scalar multiplication, it then follows that ϊ2 itself is not a vector space.

 

Linear Combination:         

The vector  is a linear combination of the set of vectors  iff there can be found constants a1, a2, a3, Ό, an such that .

 

Linear Independence:       

The set of vectors  is called a linearly independent set of vectors iff the only way to satisfy    is by choosing  a1 = a2 = a3 = Ό = an = 0   (the trivial solution).    If a set of vectors is not linearly independent, then it is a linearly dependent set of vectors.

 

Span:                      

Every set of vectors   will span some vector space or subspace.    This means that every vector in the spanned space or subspace can be written as a linear combination of the vectors in the spanning set.

 

Usually, however, it is of interest to determine whether or not a given set of vectors  will span a given vector space or subspace V.    If  S  is a spanning set for V, then every vector in  V  can be written as a linear combination of the vectors in  S.

The number of elements in the spanning set  S  is represented symbolically as  n(S).

Note:            The elements of the spanning set  S  do not, themselves, need to be linearly independent.

 

Basis:

A basis, B, for a vector space  V  or for a subspace  W Ν V  is the smallest set of linearly independent vectors contained in the spanning set   which will still span the vector space  V  or the subspace  W Ν V.   Hence,  B is a subset of  S  which  can be written symbolically as  B Μ S.   The number of elements in the basis  B  is represented by  n(B).

 

Note:            The number of elements in the basis B,  n(B), is numerically equal to the dimension of the vector space  V  or the subspace  W Ν V.   Hence, n(B) = dim(V) or n(B) = dim(W) as appropriate.

 

Examples:

(1)    the set of all real n-dimensional vectors (ϊn), with an element in ϊn represented

         The dimension of this vector space is “n”

 

(2)    the set of all  p΄n  matrices represented in general by  Mp,n

         The dimension of this vector space is “p times n”

 

(3)    the set of all real n-dimensional polynomials (-n) where a general member is represented by

         The dimension of this vector space is “n+1”

 

Note:            (1) If  T  is a non-empty set of elements from the vector space V or the subspace W Ν V, and if n(T) < dim(V) or n(T) < dim(W), then T cannot span V or W Ν V as appropriate.

         (2) If  T  is a non-empty set of elements from the vector space V or the subspace W Ν V, and if  n(T) > dim(V)  or  n(T) > dim(W),  then the elements of T cannot be linearly independent.

 

Sample Problem 18:             Show that the subspace  W = { (0, y, z) } Μ ϊ3  is spanned by the set

S  = { (0, 1, 2), (0, 2, 3), (0, 3, 1) }

Also find a basis B Μ S  for  W.

 

If  S  spans  W  then it must be possible to find constants  a1, a2, a3  such that

(0, y, z)  =  a1 (0, 1, 2) + a2 (0, 2, 3) + a3 (0, 3, 1)

ή

a1 + 2a2 + 3a3  =  y

2a 1 - 3a2 + a3  =  z

\

a1,

=

7a3 - 3y + 2z

 

a2

=

-5a3 + 2y - z

 

a3

Ξ

ϊ

The rank of the matrix is 2, therefore dim (W) = 2.   But n(S) = 3.   Hence, the elements (vectors) of  S  are not linearly independent, since

 (0, 3, 1)  =  -7 (0, 1, 2) + 5 (0, 2, 3).

Only two of the vectors in  S  are needed to form a basis for W.   Any two of the vectors of  S  can be used.   One basis for  W  is  B = { (0, 1, 2), (0, 2, 3) }.

 

Sample Problem 19:                         Express  P(x) = x2+4x-3  as a linear combination of the set

S = {P1(x), P2(x), P3(x)} = {x2-2x+5, 2x2-3x, x+3}

 

Here we wish to find constants  a1, a2, a3  such that P(x) = a1 P1(x) + a2 P2(x) + a3 P3(x).

This requires   x2+4x-3  =  a1(x2-2x+5) + a2 (2x2-3x) + a3 (x+3)

ή

a1 + 2a2 =   1

-2a1 - 3a2 + a3 =   4

5a1 + 3a3 = -3

 

 

\

a1 = -3,

a2 = 2,

a3 = 4

 

and    x2+4x-3  =  -3 (x2-2x+5) + 2 (2x2-3x) + 4 (x+3)

 

Sample Problem 20:            

Show that    is spanned by the set

and find a basis for  W.

 

Here it is necessary to show that there exist constants   a1, a2, a3 and a4   such that

3a1 + 2a2

=

a

a1 + 3a3

=

b

-2a1 + a4

=

c

-2a1 - 2a2 - 3a3 - a4

=

-(a + b + c)

\

a1,

=

b - 3a3

 

2a2

=

a - 3b + 9a3

 

a3

Ξ

ϊ

 

a4

=

2b + c - 6a3

An equivalent solution (from the reduced row echelon form of the matrix above) is

 

a1,

=

(-c + a4) / 2

 

a2

=

(2a + 3c - 3a4) / 4

 

a3

Ξ

(2b + c - a4) / 6

 

a4

=

ϊ

 

The rank of the matrix is 3, therefore dim (W) = 3.   But  n(S) = 4.   Hence, the elements (vectors) of S are not linearly independent, since any one can be expressed in terms of the other three. Hence, only three of the vectors in  S  are needed to form a basis for  W.   Any three of the vectors of  S  can be used.

 

NOTE: The following section is an optional section that will not be covered in this course, but will be helpful in future.

 

§1.9             Eigenvalues and Eigenvectors

The matrix equation  AX = lX  can also be expressed as  AX = (lI)X   or as   (A-lI)X = 0.

 

Eigenvalues:

The values of l which satisfy the equation (A-lI)X = 0 are called the eigenvalues of the

matrix A.

Eigenvectors:

The vectors equivalent to the associated column matrix X are called the eigenvectors of the matrix A corresponding to the eigenvalues l.

 

The eigenvalues are determined by finding values for l such that det[A-lI] = 0.   From this it should be obvious that  A  must be a  p΄p matrix so that the determinant is defined.   Once the eigenvalues have been found it is then possible to find the corresponding eigenvectors by solving the matrix equation  (A-lI)X = 0  for each eigenvalue l.

 

Note:

The  p΄p  matrix  A  can have from  1 to p  distinct eigenvalues.

 

Sample Problem 21:             Determine the eigenvalues and associated eigenvectors for the matrix

 

Expanding down column 2:

Hence the eigenvalues are  l = 1, l = 2, and l = 9.

The column matrix representation of the eigenvector associated with the eigenvalue l1 = 1 is determined by solving the matrix equation (A-l1I)X1 = 0.   To find X1 it is necessary to reduce the augmented matrix  A-l1I | 0 .

The column matrix representation of the eigenvector  has a parametric form given by

 

 

 

The column matrix representation of the eigenvector associated with the eigenvalue l2 = 2 is determined by solving the matrix equation  (A-l2I)X2 = 0.   To find X2 it is necessary to reduce the augmented matrix  A-l2I | 0 .

        

The column matrix representation of the eigenvector  has a parametric form given by

 

 

 

The column matrix representation of the eigenvector associated with the eigenvalue l3 = 9 is determined by solving the matrix equation (A-l3I)X3 = 0.   To find  X3  it is necessary to reduce the augmented matrix  A-l3I | 0 .

The column matrix representation of the eigenvector  has a parametric form given by

 

 

 

Note:

The determinant  det[A-lI] = 0   generates a polynomial   p(l) = | A-lI | = 0  called the characteristic polynomial which is of degree “p” if  A  is a  p΄p matrix.

Properties of Eigenvalues:

(1)    l  is an eigenvalue of the matrix  A  iff   |A-lI| = 0  (i.e. [A-lI] is singular).

(2)    det[A] = |A|  = 0  iff  l = 0 is an eigenvalue for A.

(3)    The matrices  A  and  AT  have the same characteristic polynomial and hence the same eigenvalues.

(4)    If  B  is a nonsingular matrix ( |B| Ή 0 ), and if  C = B-1AB, then  C and A  have the same eigenvalues.

(5)    The matrices  AB  and  BA  have the same distinct eigenvalues provided that the matrix products exist and that the matrix products are both square matrices.

(6)    The matrices  AAT  and  ATA  have the same characteristic polynomial and the same eigenvalues.

(7)    If the characteristic polynomial for the  p΄p matrix  A  is  p(l) =0, then it can be shown that  p(A) =0 also.   This means that any square matrix  A  automatically satisfies its characteristic polynomial.   This result is called the Cayley-Hamilton Theorem.

(8)    If  A  is a nonsingular matrix ( |A| Ή 0 ), and if  l is an eigenvalue for A, then  is an eigenvalue for  A-1.

 

Properties of Eigenvectors:

(1)    If   is an eigenvector for the matrix  A, then so also is   for every  c Ή 0.

(2)    If  A  is a nonsingular matrix ( |A| Ή 0 ), then  A and A-1 have the same eigenvectors.

(3)    If    are eigenvectors of  A  corresponding to the same eigenvalue  l, then every linear combination   (where not both “a” and “b” are zero) is also an eigenvector of  A.

(4)    If    are eigenvectors corresponding to distinct eigenvalues  l and m  of the matrix  A, then the eigenvectors  are linearly independent.

(5)    If  l1, l2, l3, ... , ln  are distinct eigenvalues for the matrix  A  then the eigenvectors  are linearly independent.

(6)    If the  p΄p matrix  A  has “p” distinct eigenvalues, then the  p΄p matrix  X, whose columns are the column matrix representations of the “p” distinct linearly independent eigenvectors, is an invertible matrix, and  X-1AX  is a diagonal matrix whose diagonal entries are the eigenvalues of  A.

(7)    If    are eigenvectors corresponding to distinct eigenvalues  l and m  of a real symmetric matrix  A (i.e. A = AT), then it can be shown that . Hence the eigenvectors are orthogonal.

(8)    Every real symmetric  p΄p matrix  A, except the zero matrix, has a set of “p” mutually orthogonal eigenvectors.

(9)    Every set of “p” mutually orthogonal non-zero eigenvectors is linearly independent.

(10) Every real symmetric  p΄p matrix  A, except the zero matrix, is diagonalizable.

(11) The eigenvalues of a real symmetric matrix  A are real and the eigenvectors can be chosen to be real.

(12) If  l is an eigenvalue of a matrix  A, and if the matrix  adj(A-lI)  is not the zero matrix, then any non-zero column of the matrix adj(A-lI) is a corresponding eigenvector.

[Note:          This result is useful computationally only for “small” matrices  A  because of the large number of calculations involved in determining  adj(A-lI)].

 

Sample Problem 22:  (a) Determine the eigenvalues and associated eigenvectors for the matrix

 

Hence the eigenvalues are  l = -1 and l = 3.

The column matrix representation of the eigenvector associated with the eigenvalue  l1 = -1  is determined by solving the matrix equation  (A-l1I)X1 = 0.   To find X1 it is necessary to reduce the augmented matrix  A-l1I | 0 .

 

 

The column matrix representation of the eigenvector  has a parametric form given by

 

 

 

The column matrix representation of the eigenvector associated with the eigenvalue  l2 = 3 is determined by solving the matrix equation  (A-l2I)X2 = 0.   To find X2 it is necessary to reduce the augmented matrix  A-l2I | 0 .

 

 

The column matrix representation of the eigenvector  has a parametric form given by

 

 

 

(b) Show that  X-1AX  is a diagonal matrix whose diagonal entries are the eigenvalues of  A.

Back to 1405 home page  [Return to the ENGR 1405 home page]
Back to your previous page  [Return to your previous page]

Created 2000 09 13 and modified 2000 09 13 by Dr. G.H. George.