Least Squares

Published on March 2017 | Categories: Documents | Downloads: 56 | Comments: 0 | Views: 391
of 3
Download PDF   Embed   Report

Comments

Content

Least Squares Estimators When trying to determine the mathematical(deterministic) relationship between parameters in a dataset, a commonly encountered problem is ”fitting” a curve that best satisfies the data. One method is employing the use of least squares. Consider Yi − (β0 − β1 Xi ) the deviation of Yi from its expected value(i.e. the model error term). The least squares criterion Q is then defined as the sum of n squared deviations. That is
n

Q=
i=1

(Yi − (β0 − β1 Xi ))2

The idea of the method of least squares is to find the values of the esˆ ˆ timators β0 and β1 that minimizes the criterion Q for a given sample of observations. The normal equations can be simultaneously solved by finding the first order partial derivatives of Q with respect to either β0 or β1 . Thus, we obtain δQ = −2 δβ0 δQ = −2 δβ1
n

(Yi − (β0 − β1 Xi ))
i=1 n

Xi (Yi − (β0 − β1 Xi ))
i=1

ˆ ˆ Setting these partial derivatives to 0,and denoting β0 and β1 as the particular values that minimizes Q, we obtain
n

−2
i=1 n

(Yi − (β0 − β1 Xi )) = 0 Xi (Yi − (β0 − β1 Xi )) = 0

−2
i=1

This simplifies to 1

n

(Yi − (β0 − β1 Xi )) = 0
i=1 n

Xi (Yi − (β0 − β1 Xi )) = 0
i=1

ˆ ˆ Expressions for β0 and β1 can then be derived by further simplifying these equations. Homework 2 Q.1 I’ll omit some of the details since it’s simply a case of applying the method of least squares which is explained above. That’s not to say that you should also omit details in your own attempts. Define Q as
n

Q=
i=1

(Yi − µ)2

Then δQ = −2 δµ
n

(Yi − µ)
i=1

Equating the first order derivative to 0 and letting µ be the value that ˆ minimizes Q, we get
n

−2
i=1 n

(Yi − µ) = 0 ˆ (Yi − µ) = 0 ˆ

i=1 n

Yi − nˆ = 0 µ
i=1

µ= ˆ

n i=1

Yi

n µ=Y ˆ ¯ 2

EXTRA The least squares solution can also be obtained using the associated normal system of matrices M tM v = M ty Then the unique solution v ∗ is given by v ∗ = (M t M )−1 M t y

3

Sponsor Documents

Or use your account on DocShare.tips

Hide

Forgot your password?

Or register your new account on DocShare.tips

Hide

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close