
Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Here are some basics that you should know about Ordinary Least Squares (OLS)
Typology: Cheat Sheet
1 / 1
This page cannot be seen from the preview
Don't miss anything!
Economics 215, 2015 Allin Cottrell
Here are some basics that you should know about Ordinary Least Squares. Note that several of the points that
are simply asserted here are proved and/or explained more fully in the notes titled “Regression Basics in Matrix
Terms”. Key assumptions are marked as, for example, “[A1]”.
y = Xβ + u (1)
where the dependent variable y is n × 1; the regressor matrix X is n × k; the parameter vector β is k × 1; and
the error term u is n × 1.
β, is given by
β = (X
′
X)
− 1
X
′
y (2)
This exists provided that X
′
X is non-singular, which requires that the X matrix is of full column rank (no exact
collinearity among the columns of X, [A1]).
Assuming
β exists, two useful additional vectors may be formed: fitted values, yˆ = X
β, and residuals, uˆ =
y − ˆy = y − X
β.
β is given by
β) = β + E
′
− 1
′
u
On condition [A3] that E(u|X) = 0 the second term above disappears and we have E(
β) = β, or in other
words the OLS estimator is unbiased.
β (a k × k matrix) is, from first principles,
Var(
β) = E
β − E(
β)
β − E(
β)
′
If the condition for the estimator to be unbiased is met, then
Var(
β) = (X
′
X)
− 1
X
′
E(uu
′
)X(X
′
X)
− 1
(4)
If the error term has a constant variance, σ
2
u
[A4], and the drawings from the error distribution are independent,
such that E(u i u j ) = 0 for all i 6 = j [A5], then E(uu
′
) = σ
2
u
n and the OLS variance simplifies to the
“classical” formula,
Var(
β) = σ
2
u
′
X)
− 1
(5)
which can be estimated by using
s
2
u
n
i= 1
uˆ
2
i
n − k
in place of the unknown σ
2
u
β to exist; in addition, [A2] and [A3] are
required for OLS to be unbiased; and in addition [A4] and [A5] are needed for “classical” standard errors to be
valid. (The standard errors routinely reported alongside OLS estimates are just the square roots of the diagonal
elements of Var(
β)).