Hypothesis Testing: Understanding Type I and Type II Errors and Power, Exercises of Statistics

Download Hypothesis Testing: Understanding Type I and Type II Errors and Power and more Exercises Statistics in PDF only on Docsity!

ERRORS AND POWER

Unit 4A - Statistical Inference Part 1

We are not guaranteed to make the correct decision by this process of hypothesis testing. Maybe you are beginning to see that there is always some level of uncertainty in statistics.

When we conduct a hypothesis test, we choose one of two possible conclusions based upon our data. When the p‐value is less than alpha we reject the null hypothesis and either: You have made the correct decision when the null hypothesis is false OR You have • made REJECTED an error Ho (^) (called when in a (^) Typefact Ho I error is TRUE ) and

• In this case, your data happened to be a RARE EVENT under Ho We do control this probability by our significance level, alpha and thus in general this will happen no more than 5% of the time using the common significance level of 0.05.

P-VALUE > ALPHA (FAIL TO REJECT Ho)

 You have made the correct decision when the null hypothesis is true

 OR

 You have made an error ( Type II ) and

• FAILED TO REJECT Ho when in fact Ho is FALSE

 This probability is NOT controlled

 How large it is will be determined by the sample size and how much the sampling distributions overlap between the null value and the true value

Possible Results of Hypothesis Tests

Here is a visual representation of the four possible choices in hypothesis testing. No possibility matter ofwhich making decision either we a correctmake (reject decision the or null an or error. fail to reject the null) there is the

A Type I error occurs when we falsely reject a true null hypothesis and A Type II error occurs when we fail to reject a false null hypothesis. Be sure to keep these two errors straight. This image can help.

Both of these probabilities are conditional probabilities. The probability the null hypothesis of a istype true. I error is the probability that we reject the null hypothesis given

• This is easy to calculate as the null distribution will be known. The probability given the null hypothesis of a type IIis error false. is the probability that we fail to reject the null hypothesis
• This is impossible to calculate as we do not know the true value if the null hypothesis is
• false.We can only make guesses at the truth and determine this probability in those cases.

Null is TRUE with Correct Decision

Null is TRUE with TYPE I ERROR

Here again the null hypothesis is that the population mean mu is 100 and the true value is indeed 100. So the null hypothesis is in fact true. It took a while to find one but here is a sample where the value falls in the shaded region and error. we reject the null hypothesis when in fact the null hypothesis is true which is a Type I

This should happen only about 5% of the time if we use an alpha of 0.05.

Here is a situation where the null hypothesis is that the population mean mu is 100 and the true value is INSTEAD 110. So the null hypothesis is in fact false. In this case, the data observed did fall in the shaded region and thus we reject the null. Since the truth about the population is that the mean is 110, we have made a correct decision. The chance of this happening varies and is based upon the sample size taken and how far the underlying true value sampling is from distributions. the null value In – which this case determines it didn’t take how long much to findoverlap a value there that is in the rejected the null hypothesis. We will define this probability as the POWER of the test to detect the specified difference, in this case 10 units.

Null is FALSE with TYPE II ERROR

Here is an illustration of the Type II error probability. On • Thethe bluetop, weline have represents the distribution the cutoff assuming for significance. the null value of 100 is true.

• Values above the blue line will result in rejecting the null hypothesis. On • Thethe bluebottom, line wecontinues have the through. distribution using the true value of 110.
• If we want to find the probability of failing to reject the null hypothesis – which would result bottom in distribution. a Type II error – then we need to find the area to the left of the blue line on this

You 37.4%. can hopefully believe from the picture that the shaded area in the bottom picture is

You don’t need to worry about how to find this value. We simply wanted to help illustrate the idea of the probability of a Type II error.