Book – Linear Algebra - Joshua, Slides of Linear Algebra

Download Book – Linear Algebra - Joshua and more Slides Linear Algebra in PDF only on Docsity!

LINEAR ALGEBRA

Jim Hefferon

Fourth edition

http://joshua.smcvt.edu/linearalgebra

Notation

R, R+, Rn^ real numbers, positive reals, n-tuples of reals N, C natural numbers {0, 1, 2,.. .}, complex numbers (a .. b), [a .. b] open interval, closed interval 〈.. .〉 sequence (a list in which order matters) hi,j row i and column j entry of matrix H V, W, U vector spaces ~v, ~ 0 , ~ (^0) V vector, zero vector, zero vector of a space V Pn, Mn×m space of degree n polynomials, n×m matrices [S] span of a set 〈B, D〉, ~β,~δ basis, basis vectors En = 〈~e 1 ,... , ~en〉 standard basis for Rn V (^) =∼ W isomorphic spaces M ⊕ N direct sum of subspaces h, g homomorphisms (linear maps) t, s transformations (linear maps from a space to itself) RepB(~v), RepB,D(h) representation of a vector, a map Zn×m or Z, In×n or I zero matrix, identity matrix |T | determinant of the matrix R(h), N (h) range space, null space of the map R∞(h), N∞(h) generalized range space and null space

Greek letters with pronounciation character name character name α alpha AL-fuh ν nu NEW β beta BAY-tuh ξ, Ξ xi KSIGH γ, Γ gamma GAM-muh o omicron OM-uh-CRON δ, ∆ delta DEL-tuh π, Π pi PIE  epsilon EP-suh-lon ρ rho ROW ζ zeta ZAY-tuh σ, Σ sigma SIG-muh η eta AY-tuh τ tau TOW (as in cow) θ, Θ theta THAY-tuh υ, Υ upsilon OOP-suh-LON ι iota eye-OH-tuh φ, Φ phi FEE, or FI (as in hi) κ kappa KAP-uh χ chi KI (as in hi) λ, Λ lambda LAM-duh ψ, Ψ psi SIGH, or PSIGH μ mu MEW ω, Ω omega oh-MAY-guh Capitals shown are the ones that differ from Roman capitals.

students to an abrupt stop. While this book begins with linear reduction, from the start we do more than compute. The first chapter includes proofs, such as the proof that linear reduction gives a correct and complete solution set. With that as motivation the second chapter does vector spaces over the reals. In the schedule below this happens at the start of the third week.

A student progresses most in mathematics by doing exercises. The problem sets start with routine checks and range up to reasonably involved proofs. I have aimed to typically put two dozen in each set, thereby giving a selection. In particular there is a good number of the medium-difficult problems that stretch a learner, but not too far. At the high end, there are a few that are puzzles taken from various journals, competitions, or problems collections, which are marked with a ‘?’ (as part of the fun I have worked to keep the original wording).

That is, as with the rest of the book, the exercises are aimed to both build an ability at, and help students experience the pleasure of, doing mathematics. Students should see how the ideas arise and should be able to picture themselves doing the same type of work.

Applications. Applications and computing are interesting and vital aspects of the subject. Consequently, each chapter closes with a selection of topics in those areas. These give a reader a taste of the subject, discuss how Linear Algebra comes in, point to some further reading, and give a few exercises. They are brief enough that an instructor can do one in a day’s class or can assign them as projects for individuals or small groups. Whether they figure formally in a course or not, they help readers see for themselves that Linear Algebra is a tool that a professional must have.

Availability. This book is Free. See http://joshua.smcvt.edu/linearalgebra for the license details. That page also has the latest version, exercise answers, beamer slides, lab manual, additional material, and LATEX source. This book is also available in hard copy from standard publishing sources, for very little cost. See the web page.

Acknowledgments. A lesson of software development is that complex projects have bugs and need a process to fix them. I am grateful for reports from both instructors and students. I periodically issue revisions and acknowledge in the book’s repository all of the reports that I use. My current contact information is on the web page.

I am grateful to Saint Michael’s College for supporting this project over many years, even before the idea of open educational resources became familiar.

And, I cannot thank my wife Lynne enough for her unflagging encouragement.

Advice. This book’s emphasis on motivation and development, and its availability, make it widely used for self-study. If you are an independent student then good for you, I admire your industry. However, you may find some advice useful.

While an experienced instructor knows what subjects and pace suit their class, this semester’s timetable (graciously shared by G Ashline) may help you plan a sensible rate. It presumes that you have already studied the material of Section One.II, the elements of vectors.

week Monday Wednesday Friday 1 One.I.1 One.I.1, 2 One.I.2, 3 2 One.I.3 One.III.1 One.III. 3 Two.I.1 Two.I.1, 2 Two.I. 4 Two.II.1 Two.III.1 Two.III. 5 Two.III.2 Two.III.2, 3 Two.III. 6 exam Three.I.1 Three.I. 7 Three.I.2 Three.I.2 Three.II. 8 Three.II.1 Three.II.2 Three.II. 9 Three.III.1 Three.III.2 Three.IV.1, 2 10 Three.IV.2, 3 Three.IV.4 Three.V. 11 Three.V.1 Three.V.2 Four.I. 12 exam Four.I.2 Four.III. 13 Five.II.1 –Thanksgiving break– 14 Five.II.1, 2 Five.II.2 Five.II.

As enrichment, you could pick one or two extra things that appeal to you, from the lab manual or from the Topics from the end of each chapter. I like the Topics on Voting Paradoxes, Geometry of Linear Maps, and Coupled Oscillators. You’ll get more from these if you have access to software for calculations such as Sage, freely available from http://sagemath.org.

In the table of contents I have marked a few subsections as optional if some instructors will pass over them in favor of spending more time elsewhere.

Note that in addition to the in-class exams, students in the above course do take-home problem sets that include proofs, such as a verification that a set is a vector space. Computations are important but so are the arguments.

My main advice is: do many exercises. I have marked a good sample with X’s in the margin. Do not simply read the answers — you must try the problems and possibly struggle with them. For all of the exercises, you must justify your answer either with a computation or with a proof. Be aware that few people can write correct proofs without training; try to find a knowledgeable person to work with you.

Contents

Chapter One

Linear Systems

I Solving Linear Systems

Systems of linear equations are common in science and mathematics. These two examples from high school science [Onan] give a sense of how they arise. The first example is from Statics. Suppose that we have three objects, we know that one has a mass of 2 kg, and we want to find the two unknown masses. Experimentation with a meter stick produces these two balances.

h c^2 15

40 50 c (^2) h

25 50

25

For the masses to balance we must have that the sum of moments on the left equals the sum of moments on the right, where the moment of an object is its mass times its distance from the balance point. That gives a system of two linear equations.

40h + 15c = 100 25c = 50 + 50h

The second example is from Chemistry. We can mix, under controlled conditions, toluene C 7 H 8 and nitric acid HNO 3 to produce trinitrotoluene C 7 H 5 O 6 N 3 along with the byproduct water (conditions have to be very well controlled — trinitrotoluene is better known as TNT). In what proportion should we mix them? The number of atoms of each element present before the reaction

x C 7 H 8 + y HNO 3 −→ z C 7 H 5 O 6 N 3 + w H 2 O

Section I. Solving Linear Systems 3

1.3 Example The ordered pair (−1, 5) is a solution of this system.

3x 1 + 2x 2 = 7 −x 1 + x 2 = 6

In contrast, (5, − 1 ) is not a solution.

Finding the set of all solutions is solving the system. We don’t need guesswork or good luck; there is an algorithm that always works. This algorithm is Gauss’s Method (or Gaussian elimination or linear elimination).

1.4 Example To solve this system

3x 3 = 9 x 1 + 5x 2 − 2x 3 = 2 1 3 x^1 +^ 2x^2 =^3

we transform it, step by step, until it is in a form that we can easily solve. The first transformation rewrites the system by interchanging the first and third row.

swap row 1 with row 3 −→

1 3 x^1 +^ 2x^2 =^3 x 1 + 5x 2 − 2x 3 = 2 3x 3 = 9

The second transformation rescales the first row by a factor of 3.

multiply row 1 by 3 −→

x 1 + 6x 2 = 9 x 1 + 5x 2 − 2x 3 = 2 3x 3 = 9

The third transformation is the only nontrivial one in this example. We mentally multiply both sides of the first row by − 1 , mentally add that to the second row, and write the result in as the new second row.

add − 1 times row 1 to row 2 −→

x 1 + 6x 2 = 9 −x 2 − 2x 3 = − 7 3x 3 = 9

These steps have brought the system to a form where we can easily find the value of each variable. The bottom equation shows that x 3 = 3. Substituting 3 for x 3 in the middle equation shows that x 2 = 1. Substituting those two into the top equation gives that x 1 = 3. Thus the system has a unique solution; the solution set is {(3, 1, 3)}.

We will use Gauss’s Method throughout the book. It is fast and easy. We will now show that it is also safe: Gauss’s Method never loses solutions nor does it ever pick up extraneous solutions, so that a tuple is a solution to the system before we apply the method if and only if it is a solution after.

4 Chapter One. Linear Systems

1.5 Theorem (Gauss’s Method) If a linear system is changed to another by one of these operations

(1) an equation is swapped with another (2) an equation has both sides multiplied by a nonzero constant (3) an equation is replaced by the sum of itself and a multiple of another

then the two systems have the same set of solutions.

Each of the three operations has a restriction. Multiplying a row by 0 is not allowed because obviously that can change the solution set. Similarly, adding a multiple of a row to itself is not allowed because adding − 1 times the row to itself has the effect of multiplying the row by 0. And we disallow swapping a row with itself, to make some results in the fourth chapter easier. Besides, it’s pointless.

Proof We will cover the equation swap operation here. The other two cases are similar and are Exercise 33. Consider a linear system.

a1,1x 1 + a1,2x 2 + · · · + a1,nxn = d 1 .. . ai,1x 1 + ai,2x 2 + · · · + ai,nxn = di .. . aj,1x 1 + aj,2x 2 + · · · + aj,nxn = dj .. . am,1x 1 + am,2x 2 + · · · + am,nxn = dm

The tuple (s 1 ,... , sn) satisfies this system if and only if substituting the values for the variables, the s’s for the x’s, gives a conjunction of true statements: a1,1s 1 +a1,2s 2 +· · ·+a1,nsn = d 1 and... ai,1s 1 +ai,2s 2 +· · ·+ai,nsn = di and

... aj,1s 1 + aj,2s 2 + · · · + aj,nsn = dj and... am,1s 1 + am,2s 2 + · · · + am,nsn = dm. In a list of statements joined with ‘and’ we can rearrange the order of the statements. Thus this requirement is met if and only if a1,1s 1 + a1,2s 2 + · · · + a1,nsn = d 1 and... aj,1s 1 + aj,2s 2 + · · · + aj,nsn = dj and... ai,1s 1 + ai,2s 2 + · · · + ai,nsn = di and... am,1s 1 + am,2s 2 + · · · + am,nsn = dm. This is exactly the requirement that (s 1 ,... , sn) solves the system after the row swap. QED

6 Chapter One. Linear Systems

1.9 Example The reduction

x + y + z = 9 2x + 4y − 3z = 1 3x + 6y − 5z = 0

−2ρ −→ 1 +ρ 2 −3ρ 1 +ρ 3

x + y + z = 9 2y − 5z = − 17 3y − 8z = − 27

−(3/2)ρ 2 +ρ 3 −→

x + y + z = 9 2y − 5z = − 17 −(1/2)z = −(3/2)

shows that z = 3 , y = − 1 , and x = 7.

As illustrated above, the point of Gauss’s Method is to use the elementary reduction operations to set up back-substitution.

1.10 Definition In each row of a system, the first variable with a nonzero coefficient is the row’s leading variable. A system is in echelon form if each leading variable is to the right of the leading variable in the row above it, except for the leading variable in the first row, and any rows with all-zero coefficients are at the bottom.

1.11 Example The prior three examples only used the operation of row combina- tion. This linear system requires the swap operation to get it into echelon form because after the first combination

x − y = 0 2x − 2y + z + 2w = 4 y + w = 0 2z + w = 5

−2ρ 1 +ρ 2 −→

x − y = 0 z + 2w = 4 y + w = 0 2z + w = 5

the second equation has no leading y. We exchange it for a lower-down row that has a leading y.

ρ (^2) −→↔ρ 3

x − y = 0 y + w = 0 z + 2w = 4 2z + w = 5

(Had there been more than one suitable row below the second then we could have used any one.) With that, Gauss’s Method proceeds as before.

−2ρ −→ 3 +ρ 4

x − y = 0 y + w = 0 z + 2w = 4 −3w = − 3

Back-substitution gives w = 1 , z = 2 , y = − 1 , and x = − 1.

Section I. Solving Linear Systems 7

Strictly speaking, to solve linear systems we don’t need the row rescaling operation. We have introduced it here because it is convenient and because we will use it later in this chapter as part of a variation of Gauss’s Method, the Gauss-Jordan Method. All of the systems so far have the same number of equations as unknowns. All of them have a solution and for all of them there is only one solution. We finish this subsection by seeing other things that can happen.

1.12 Example This system has more equations than variables.

x + 3y = 1 2x + y = − 3 2x + 2y = − 2

Gauss’s Method helps us understand this system also, since this

−2ρ −→ 1 +ρ 2 −2ρ 1 +ρ 3

x + 3y = 1 −5y = − 5 −4y = − 4

shows that one of the equations is redundant. Echelon form

−(4/5 −→)ρ 2 +ρ 3

x + 3y = 1 −5y = − 5 0 = 0

gives that y = 1 and x = − 2. The ‘ 0 = 0 ’ reflects the redundancy.

Gauss’s Method is also useful on systems with more variables than equations. The next subsection has many examples. Another way that linear systems can differ from the examples shown above is that some linear systems do not have a unique solution. This can happen in two ways. The first is that a system can fail to have any solution at all.

1.13 Example Contrast the system in the last example with this one.

x + 3y = 1 2x + y = − 3 2x + 2y = 0

−2ρ −→ 1 +ρ 2 −2ρ 1 +ρ 3

x + 3y = 1 −5y = − 5 −4y = − 2

Here the system is inconsistent: no pair of numbers (s 1 , s 2 ) satisfies all three equations simultaneously. Echelon form makes the inconsistency obvious.

−(4/5)ρ 2 +ρ 3 −→

x + 3y = 1 −5y = − 5 0 = 2

The solution set is empty.

Section I. Solving Linear Systems 9

any solutions at all despite that in echelon form it has a 0 = 0 row.

2x − 2z = 6 y + z = 1 2x + y − z = 7 3y + 3z = 0

−ρ 1 +ρ 3 −→

2x − 2z = 6 y + z = 1 y + z = 1 3y + 3z = 0

−ρ 2 +ρ 3 −3ρ^ −→ 2 +ρ 4

2x − 2z = 6 y + z = 1 0 = 0 0 = − 3

In summary, Gauss’s Method uses the row operations to set a system up for back substitution. If any step shows a contradictory equation then we can stop with the conclusion that the system has no solutions. If we reach echelon form without a contradictory equation, and each variable is a leading variable in its row, then the system has a unique solution and we find it by back substitution. Finally, if we reach echelon form without a contradictory equation, and there is not a unique solution — that is, at least one variable is not a leading variable — then the system has many solutions. The next subsection explores the third case. We will see that such a system must have infinitely many solutions and we will describe the solution set.

Note. In the exercises here, and in the rest of the book, you must justify all of your answers. For instance, if a question asks whether a system has a solution then you must justify a yes response by producing the solution and must justify a no response by showing that no solution exists.

Exercises X 1.17 Use Gauss’s Method to find the unique solution for each system. (a) 2x + 3y = 13 x − y = − 1

(b) x − z = 0 3x + y = 1 −x + y + z = 4 1.18 Each system is in echelon form. For each, say whether the system has a unique solution, no solution, or infinitely many solutions. (a) −3x + 2y = 0 −2y = 0

(b) x + y = 4 y − z = 0

(c) x + y = 4 y −z = 0 0 = 0

(d) x + y = 4 0 = 4

(e) 3x + 6y + z = −0. −z = 2.

(f) x − 3y = 2 0 = 0

(g) 2x + 2y = 4 y = 1 0 = 4

(h) 2x + y = 0

(i) x − y = − 1 0 = 0 0 = 4

(j) x + y − 3z = − 1 y − z = 2 z = 0 0 = 0

10 Chapter One. Linear Systems

X 1.19 Use Gauss’s Method to solve each system or conclude ‘many solutions’ or ‘no solutions’. (a) 2x + 2y = 5 x − 4y = 0

(b) −x + y = 1 x + y = 2

(c) x − 3y + z = 1 x + y + 2z = 14

(d) −x − y = 1 −3x − 3y = 2 (e) 4y + z = 20 2x − 2y + z = 0 x + z = 5 x + y − z = 10

(f) 2x + z + w = 5 y − w = − 1 3x − z − w = 0 4x + y + 2z + w = 9 1.20 Solve each system or conclude ‘many solutions’ or ‘no solutions’. Use Gauss’s Method. (a) x + y + z = 5 x − y = 0 y + 2z = 7

(b) 3x + z = 7 x − y + 3z = 4 x + 2y − 5z = − 1

(c) x + 3y + z = 0 −x − y = 2 −x + y + 2z = 8 X 1.21 We can solve linear systems by methods other than Gauss’s. One often taught in high school is to solve one of the equations for a variable, then substitute the resulting expression into other equations. Then we repeat that step until there is an equation with only one variable. From that we get the first number in the solution and then we get the rest with back-substitution. This method takes longer than Gauss’s Method, since it involves more arithmetic operations, and is also more likely to lead to errors. To illustrate how it can lead to wrong conclusions, we will use the system x + 3y = 1 2x + y = − 3 2x + 2y = 0 from Example 1.13. (a) Solve the first equation for x and substitute that expression into the second equation. Find the resulting y. (b) Again solve the first equation for x, but this time substitute that expression into the third equation. Find this y. What extra step must a user of this method take to avoid erroneously concluding a system has a solution?

X 1.22 For which values of k are there no solutions, many solutions, or a unique solution to this system? x − y = 1 3x − 3y = k 1.23 This system is not linear in that it says sin α instead of α 2 sin α − cos β + 3 tan γ = 3 4 sin α + 2 cos β − 2 tan γ = 10 6 sin α − 3 cos β + tan γ = 9 and yet we can apply Gauss’s Method. Do so. Does the system have a solution?

X 1.24 What conditions must the constants, the b’s, satisfy so that each of these systems has a solution? Hint. Apply Gauss’s Method and see what happens to the right side.