Download Matrix Algebra and more Exams Algebra in PDF only on Docsity!
LECTURE 1
Matrix Algebra
Simultaneous Equations
Consider a system of m linear equations in n unknowns:
y 1 = a 11 x 1 + a 12 x 2 + · · · + a 1 nxn,
y 2 = a 21 x 1 + a 22 x 2 + · · · + a 2 nxn, .. . ym = am 1 x 1 + am 2 x 2 + · · · + amnxn.
There are three sorts of elements here:
The constants: {yi; i = 1,... , m}, The unknowns: {xj ; j = 1,... , n},
The coefficients: {aij ; i = 1,... , m, j = 1,... , n};
and they can be gathered into three arrays:
(2) y =
y 1 y 2 .. . ym
,^ A^ =
a 11 a 12 · · · a 1 n a 21 a 22 · · · a 2 n .. .
am 1 am 2 · · · amn
,^ x^ =
x 1 x 2 .. . xn
The arrays y and x are column vectors of order m and n respectively whilst the array A is a matrix of order m × n, which is to say that it has m rows and n columns. A summary notation for the equations under (1) is then
(3) y = Ax.
There are two objects on our initial agenda. The first is to show, in detail, how the summary matrix representation corresponds to the explicit form of the
D.S.G. POLLOCK: INTRODUCTORY ECONOMETRICS
equation under (1). For this purpose we need to define, at least, the operation of matrix multiplication. The second object is to describe a method for finding the values of the unknown elements. Each of the m equations is a statement about a linear relationship amongst the n unknowns. The unknowns can be determined if and only if there can be found, amongst the m equations, a subset of n equations which are mutually independent in the sense that none of the corresponding statements can be deduced from the others.
Example. Consider the system
9 = x 1 + 3x 2 + 2x 3 ,
8 = 4x 1 + 5x 2 − 6 x 3 ,
8 = 3x 1 + 2x 2 + x 3.
The corresponding arrays are
(5) y =
, A =
(^) , x =
Here we have placed the solution for x 1 , x 2 and x 3 within the vector x. The correctness of these values may be confirmed by substituting them into the equations of (4).
Elementary Operations with Matrices
It is often useful to display the generic element of a matrix together with the symbol for the matrix in the summary notation. Thus, to denote the m × n matrix of (2), we write A = [aij ]. Likewise, we can write y = [yi] and x = [xj ] for the vectors. In fact, the vectors y and x may be regarded as degenerate matrices of orders m × 1 and n × 1 respectively. The purpose of this is to avoid having to enunciate rules of vector algebra alongside those of matrix algebra.
Matrix Addition. If A = [aij ] and B = [bij ] are two matrices of order m × n, then their sum is the matrix C = [cij ] whose generic element is cij = aij + bij.
The sum of A and B is defined only if the two matrices have the same order which is m × n; in which case they are said to be conformable with respect to addition. Notice that the notation reveals that the matrices are conformable by giving the same indices i = 1,... , m and j = 1,... , n to their generic elements. The operation of matrix addition is commutative such that A + B = B + A and associative such that A+(B+C) = (A+B)+C. These results are, of course, trivial since they amount to nothing but the assertions that aij + bij = bij + aij and that aij + (bij + cij ) = (bij + aij ) + cij for all i, j.
D.S.G. POLLOCK: INTRODUCTORY ECONOMETRICS
Thus the element of A from the ith row and jth column becomes the element of the jth row and ith column of A′. The symbol {′} is called a prime, and we refer to A′^ equally as A–prime or A–transpose.
Example. Let D be the sub-matrix formed by taking the first two columns of the matrix A of (5). Then
(9) D =
(^) and D′^ =
[
]
The basic rules of transposition are as follows
(i) The transpose of A′^ is A, that is (A′)′^ = A,
(ii) If C = A + B then C′^ = A′^ + B′,
(iii) If C = AB then C′^ = B′A′.
Of these, only (iii), which is called the reversal rule, requires explanation. For a start, when the product is written as
(11) (C′^ : p × m) = (B′^ : p × n)(A′^ : n × m)
and when the latter is compared with the expression under (6), it becomes clear that the reversal rule ensures the correct orders for the product. More explicitly,
if A = [aij ], B = [bjk] and AB = C = [cik],
where cik =
∑^ n
j=
aij bjk,
then A′^ = [aji], B′^ = [bkj ] and B′A′^ = C′^ = [cki]
where cki =
∑^ n
j=
bkj aji.
Matrix Inversion. If A = [aij ] is a square matrix of order n × n, then its inverse, if it exists, is a uniquely defined matrix A−^1 of order n × n which satisfies the condition AA−^1 = A−^1 A = I, where I = [δij ] is the identity matrix of order n which has units on its principal diagonal and zeros elsewhere.
In this notation, δij is Kronecker’s delta defined by
(13) δij =
0 , if i 6 = j; 1 , if i = j.
1: MATRIX ALGEBRA
Usually, we may rely upon the computer to perform the inversion of a numerical matrix of order 3 or more. Also, for orders of three or more, the symbolic expressions for the individual elements of the inverse matrix become intractable. In order to derive the explicit expression for the inverse of a 2 × 2 matrix A, we may consider the following equation BA = I:
[
b 11 b 12 b 21 b 22
] [
a 11 a 12 a 21 a 22
]
[
]
The elements of the inverse matrix A−^1 = B are obtained by solving the fol- lowing equations in pairs:
(i) b 11 a 11 + b 12 a 21 = 1,
(iii) b 21 a 11 + b 22 a 21 = 0,
(ii) b 11 a 12 + b 12 a 22 = 0,
(iv) b 21 a 12 + b 22 a 22 = 1.
From (i) and (ii) we get
(16) b 11 =
a 22 a 11 a 22 − a 12 a 21
and b 12 =
−a 12 a 11 a 22 − a 12 a 21
whereas, from (iii) and (iv), we get
(17) b 21 =
−a 21 a 11 a 22 − a 12 a 21
and b 22 =
a 11 a 11 a 22 − a 12 a 21
By gathering the elements of the equation B = A−^1 under (16) and (17), we derive the following expression for the inverse of a 2 × 2 matrix:
[
a 11 a 12 a 21 a 22
]− 1
a 11 a 22 − a 12 a 21
[
a 22 −a 12 −a 21 a 11
]
Here the quantity a 11 a 22 − a 12 a 21 in the denominator of the scalar factor which multiplies the matrix on the RHS is the so-called determinant of the original matrix A denoted by det(A) or |A|. The inverse of the matrix A exists if and only if the determinant has a nonzero value. The formula above can be generalised to accommodate square matrices of an arbitrary order n. However, the general formula rapidly becomes intractable as the value of n increases. In the case of a matrix of order 3 × 3, the inverse is given by
(19) A−^1 =
|A|
c 11 −c 21 c 31 −c 12 c 22 −c 32 c 13 −c 23 c 33
1: MATRIX ALGEBRA
Example. Consider a pair of simultaneous equations written in matrix form: [ 1 5 2 3
] [
x 1 x 2
]
[
]
The solution using the formula for the inverse of 2 × 2 matrix found under (18) is
[
x 1 x 2
]
[
] [
]
[
]
We have not used the formula under (19) to find the solution of the system of three equations under (4) because there is an easier way which depends upon the method of Gaussian elimination.
Elementary Operations
There are three elementary row operations which serve, in a process de- scribed as Gaussian elimination, to reduce an arbitrary matrix to one which has units and zeros on its principal diagonal and zeros everywhere else. We shall illustrate these elementary operations in the context of a 2 × 2 matrix; but there should be no difficulty in seeing how they may be applied to square matrices of any order. The first operation is the multiplication of a row of the matrix by a scalar factor:
[
λ 0 0 1
] [
a 11 a 12 a 21 a 22
]
[
λa 11 λa 12 a 21 a 22
]
The second operation is the addition of one row to another, or the subtraction of one row from another:
[
] [
a 11 a 12 a 21 a 22
]
[
a 11 a 12 a 21 − a 11 a 22 − a 12
]
The third operation is the interchange of one row with another:
[
] [
a 11 a 12 a 21 a 22
]
[
a 21 a 22 a 11 a 12
]
If the matrix A possesses an inverse, then, by the application of a succession of such operations, one may reduce it to the identity matrix. Thus, if the elementary operations are denoted by Ej ; j = 1,... , q, then we have
{Eq · · · E 2 E 1 }A = I or, equivalently, BA = I,
where B = Eq · · · E 2 E 1 = A−^1.
D.S.G. POLLOCK: INTRODUCTORY ECONOMETRICS
Example. A 2 × 2 matrix which possesses an inverse may be reduced to the identity matrix by applying four elementary operations:
[
1 /α 11 0 0 1
] [
α 11 α 12 α 21 α 22
]
[
1 α 12 /α 11 α 21 α 22
]
[
−β 21 1
] [
1 β 12 β 21 β 22
]
[
1 β 12 0 β 22 − β 21 β 12
]
[
0 1 /γ 22
] [
1 γ 12 0 γ 22
]
[
1 γ 12 0 1
]
[
1 −γ 12 0 1
] [
1 γ 12 0 1
]
[
]
Here, we have expressed the results of the first two opertations in new symbols in the interests of notational simplicity. Were we to retain the original notation throughout, then we could obtain the expression for the inverse matrix by forming the product of the four elementary matrices. It is notable that, in this example, we have used only the first two of the three elementary operations. The third would be called for if we were confronted by a matrix in the form of
[
α β γ 0
]
for then we should wish to reorder the rows before embarking on the process of elimination.
Example. Consider the equations under (4) which can be written in matrix format as
x 1 x 2 x 3
Taking four times the first equation from the second equation and three times the first equation from the third gives
x 1 x 2 x 3
D.S.G. POLLOCK: INTRODUCTORY ECONOMETRICS
a 1
a 2 a = ( a 1 , a 2 )
e 2 = (0, 1)
e 1 = (1, 0)
Figure 1. The coordinates of a vector a relative to two perpendicular axes
point with its coordinates, then we may write a = (a 1 , a 2 ). According to this convention, we may also write e 1 = (1, 0) and e 2 = (0, 1). The directed line segment running from the origin 0 to the point a is de- scribed as a geometric vector which is bound to the origin. The ordered pair (a 1 , a 2 ) = a may be described as an algebraic vector. In fact, it serves little purpose to make a distinction between these two entities—the algebraic vector and the geometric vector—which may be regarded hereafter as alternative rep- resentations of the same object a. The unit vectors e 1 = (1, 0) and e 2 = (0, 1), which serve, in fact, to define the coordinate system, are described as the basis vectors. The sum of two vectors a = (a 1 , a 2 ) and b = (b 1 , b 2 ) is defined by
a + b = (a 1 , a 2 ) + (b 1 , b 2 ) = (a 1 + b 1 , a 2 + b 2 ).
The geometric representation of vector addition corresponds to a parallelogram of forces. Forces, which have both magnitude and direction, may be represented by directed line segments whose lengths correspond to the magnitudes. Hence forces may be described as vectors; and, as such, they obey the law of addition given above. If a = (a 1 , a 2 ) is a vector and λ is a real number, which is also described as a scalar, then the product of a and λ is defined by
λa = λ(a 1 , a 2 ) = (λa 1 , λa 2 ).
1: MATRIX ALGEBRA
b
a
a + b
Figure 2. The parallelogram law of vector addition
The geometric counterpart of multiplication by a scalar is a stretching or a contraction of the vector which affects its length but not its direction. The axes of the coordinate system are provided by the lines E 1 = {λe 1 } and E 2 = {λe 2 } which are defined by letting λ take every possible value. In terms of the basis vectors e 1 = (1, 0) and e 2 = (0, 1), the point a = (a 1 , a 2 ) can be represented by
a = (a 1 , a 2 ) = a 1 e 1 + a 2 e 2.
Norms and Inner Products.
The length or norm of a vector a = (a 1 , a 2 ) is
(42) ‖a‖ =
a^21 + a^22 ;
and this may be regarded either as an algebraic definition or as a consequence of the geometric theorem of Pythagoras. The inner product of the vectors a = (a 1 , a 2 ) and b = (b 1 , b 2 ) is the scalar quantity
(43) a′b = [ a 1 a 2 ]
[
b 1 b 2
]
= a 1 b 1 + a 2 b 2.
This is nothing but the product of the matrix a′^ of order 1 × 2 and the matrix b of order 2 × 1. The vectors a, b are said to be mutually orthogonal if a′b = 0. It can be shown by simple trigonometry that a, b fulfil the condition of orthogonality if and only if the line segments are at right angles. Indeed, this condition is
1: MATRIX ALGEBRA
b 1 a 1
a 2
b 2
b
a
Figure 4. The determinant corresponds to the area enclosed by the parallelogram of forces.
which describe two lines in the plane. The coordinates (x, y) of the point of intersection of the lines is the algebraic solution of the simultaneous equations. The equations may be written in matrix form as
[
a b c d
] [
x y
]
[
e f
]
The necessary and sufficient condition for the existence of a unique solution is that
(48) Det
[
a b c d
]
= ad − bc 6 = 0.
Then the solution is given by
[
x y
]
[
a b c d
]− 1 [
e f
]
ad − bc
[
d −b −c a
] [
e f
]
We may prove that
Det
[
a b c d
]
= 0 if and only if
a = λc and b = λd for some scalar λ.
D.S.G. POLLOCK: INTRODUCTORY ECONOMETRICS
Proof. From a = λc and b = λd we derive, by cross multiplication, the identity λad = λbc, whence ad−bc = 0 and the determinant is zero. Conversely, if ad − bc = 0, then we can deduce that a = (b/d)c and b = (a/c)d together with the identity (b/d) = (a/c) = λ, which implies that a = λc and b = λd.
When the determinant is zero-valued one of two possibilities ensues. The first in when e = λf. Then the two equations describe the same line and there is infinite number of solutions, with each solution corresponding to a point on the line. The second possibility is when e 6 = λf. Then the equations describe parallel lines and there are no solutions. Therefore, we say that the equations are inconsistent. It is appropriate to end this section by giving a geometric interpretation of
(51) Det
[
a 1 a 2 b 1 b 2
]
= a 1 b 2 − a 2 b 1.
This is simply the area enclosed by the parallelogram of forces which is formed by adding the vectors a = (a 1 , a 2 ) and b = (b 1 , b 2 ). The result can be estab- lished by subtracting triangles from the rectangle in the accompanying figure to show that the area of the shaded region is 12 (a 1 b 2 − a 2 b 1 ). The shaded region comprises half of the area which is enclosed by the parallelogram of forces.
