Research Rundowns >Quantitative Methods > Significance Testing (t-tests)
In this review, we’ll look at significance testing, using mostly the t-test as a guide. As you read educational
research, you’ll encounter t-test and ANOVA statistics frequently. Part I reviews the basics of significance
testing as related to the null hypothesis and p values. Part II shows you how to conduct a t-test, using an
online calculator. Part III deal s with interpreting t-test results. Part IV is about reporting t-test results in
both text and table formats and concludes with a guide to interpreting confidence intervals.
What is Statistical Significance?
The terms “significance level” or “level of significance” refer to the likelihood that the random sample you
choose (for example, test scores) is not representative of the population. The lower the significance level,
the more confident you can be in replicating your results. Significance levels most commonly used in
educational research are the .05 and .01 levels. If it helps, think of .05 as another way of saying 95/100
times that you sample from the population, you will get this result. Similarly, .01 suggests that 99/100
times that you sample from the population, you will get the same result. These numbers and signs (more
on that later) come from Significance Testing, which begins with the Null Hypothesis.
Part I: The Null Hypothesis
We start by revisiting familiar territory, the scientific method. We’ll start with a basic research question:
How does variable A affect variable B? The traditional way to test this question involves:
Step 1. Develop a research question.
Step 2. Find previous research to support, refute, or suggest ways of testing the question.
Step 3. Construct a hypothesis by revising your research question:
Hypothesis
Summary
Type
H1: A = B
There is no relationship between A and B
Null
H2: A ≠ B
There is a relationship between A and B. Here, there is a relationship, but
we don’t know if it is positive or negative.
Alternate
H3: A < B
There is a negative relationship between A and B. Here, the < suggests that
the less A is involved, the better B.
Alternate
H4: A > B
There is a positive relationship between A and B. Here, the > suggests that
the more B is involved, the better A.
Alternate
Step 4. Test the null hypothesis. To test the null hypothesis, A = B, we use a significance test. The italicized
lowercase p you often see, followed by > or < sign and a decimal (p ≤ .05) indicate significance. In most
cases, the researcher tests the null hypothesis, A = B, because is it easier to show there is some sort of
effect of A on B, than to have to determine a positive or negative effect prior to conducting the research.
This way, you leave yourself room without having the burden of proof on your study from the beginning.
Step 5. Analyze data and draw a conclusion. Testing the null hypothesis leaves two possibilities:
Outcome
Wording
Type
A = B
Fail to reject the null. We find no relationship between A and B.
Null
A =, <, or > B
Reject the null. We find a relationship between A and B.
Alternate
Step 6. Communicate results. See Wording results, below.
