A t-test checks whether a difference in means is bigger than chance would explain, given the spread of your data and your sample size. It comes in three versions. A one-sample t-test compares one mean with a fixed value. An independent t-test compares two separate groups, and a paired t-test compares two measurements on the same people.
Each version divides a difference by its standard error to give the t statistic. You then read t against the t distribution at the right degrees of freedom. For a one-sample test that’s the number of observations minus one; for a paired test, the number of pairs minus one. Choosing the wrong version can change your conclusion, as the worked example shows.
That example uses the sleep data WS Gosset published in 1908, writing as ‘Student’. Worked by hand, it reproduces Student’s own result. Analysing the same numbers as two separate groups reverses the conclusion.
When do you use a t-test?
Use a t-test when your outcome is a measurement, such as a score, a time or a concentration, and you want to compare means. Hours of sleep, reaction times and blood pressure all fit. If your data are counts of people in categories, this is the wrong tool.
The t-test is built for one or two means. For three or more groups, a one-way analysis of variance (ANOVA) is the right tool.
Which t-test fits your data?
| Your design | Test |
|---|---|
| One group, compared with a fixed or expected value | One-sample t-test |
| Two separate groups of different people | Independent (two-sample) t-test |
| Two separate groups with clearly different spreads | Independent t-test, unequal-variance (Welch) version |
| The same people measured twice, or matched pairs | Paired t-test |
| Three or more groups | One-way ANOVA |
Source: BMJ, Statistics at Square One, chapter 7.
Our guide to the statistical treatment of data introduces the difference between descriptive and inferential statistics. The t-test is inferential: it asks whether a difference in your sample is likely to reflect more than chance.
The three types of t-test
The t-test formula is the same idea for all three: t is a difference divided by its standard error. What changes is which difference, and how its standard error is worked out. For a one-sample test, t = (sample mean − set value) ÷ (SD ÷ √n).
| One-sample | Independent | Paired | |
|---|---|---|---|
| Question | Does the mean differ from a set value? | Do two groups’ means differ? | Does the mean difference within pairs differ from zero? |
| Difference used | Sample mean minus the set value | Mean of group 1 minus mean of group 2 | Mean of the within-pair differences |
| Standard error | SD ÷ √n | From both groups’ SDs and sizes | SD of the differences ÷ √(number of pairs) |
| Degrees of freedom | n − 1 | n₁ + n₂ − 2 (equal-variance version) | Number of pairs − 1 |
Source: BMJ, Statistics at Square One, chapter 7.
The paired t-test is a one-sample test in disguise. You work out each pair’s difference, then test whether the mean of those differences is zero. Because the standard error comes from the spread of the differences, not of the raw scores, pairing can shrink it a lot. The worked example below shows by how much.
How to calculate a t-test, step by step
You can calculate a t-test by hand in five steps.
- State the null hypothesis. For a paired test, it’s that the mean difference is zero. The alternative is that it isn’t.
- Work out the difference. For a paired test, take each pair’s difference first, then their mean.
- Work out the standard error. For one sample or pairs, divide the standard deviation by the square root of the number of observations or pairs.
- Divide the difference by its standard error. That’s your t statistic.
- Compare t with the t distribution at your degrees of freedom, using a table of critical values or the p-value from software.
T-test example: Student’s sleep data from 1908
In 1908 WS Gosset published The probable error of a mean in Biometrika, under the pseudonym ‘Student’. His work laid the foundations of the t-test, which is why it’s also called Student’s t-test. To show his method, he used a table by Cushny and Peebles from the Journal of Physiology.
They recorded the average extra hours of sleep 10 patients gained on two drugs, which Student described as the two optical isomers of hyoscyamine hydrobromide. We’re using this table because Student printed his own answer beside it. You can check each step against his.
Step 1: the data
| Patient | Drug 1 (hours gained) | Drug 2 (hours gained) | Difference (2 − 1) |
|---|---|---|---|
| 1 | +0.7 | +1.9 | +1.2 |
| 2 | −1.6 | +0.8 | +2.4 |
| 3 | −0.2 | +1.1 | +1.3 |
| 4 | −1.2 | +0.1 | +1.3 |
| 5 | −0.1 | −0.1 | 0 |
| 6 | +3.4 | +4.4 | +1.0 |
| 7 | +3.7 | +5.5 | +1.8 |
| 8 | +0.8 | +1.6 | +0.8 |
| 9 | 0 | +4.6 | +4.6 |
| 10 | +2.0 | +3.4 | +1.4 |
| Mean | +0.75 | +2.33 | +1.58 |
Source: Student (1908), ‘The probable error of a mean’, Biometrika 6 (University of York copy, page 20).
Every patient took both drugs, so this is paired data. The question is whether drug 2 gives more sleep than drug 1.
Steps 2 to 4: the paired t-test
The mean difference is 1.58 hours. The standard deviation of the 10 differences is 1.230. So the standard error is 1.230 ÷ √10 = 0.389.
That gives t = 1.58 ÷ 0.389 = 4.06, with 10 − 1 = 9 degrees of freedom.
Student’s own figures look different at first. His table gives a standard deviation of 1.17, not 1.23, because he divided by the number of patients (10). Today’s sample standard deviation divides by 9.
He wrote the mean as ‘+1.35 times the s.d.’. To turn his figure into t, multiply by √(n − 1), here √9: 1.35 × 3 = 4.05. Without rounding, it’s the same t of 4.06.
Step 5: the decision
At 9 degrees of freedom, the two-sided (two-tailed) 5% critical value is 2.262 and the 1% value is 3.250. Our t of 4.06 is well past both, so p is below 0.01. Software gives p ≈ 0.003.
The 95% confidence interval for the mean difference is 1.58 ± 2.262 × 0.389. That’s 0.70 to 2.46 extra hours of sleep on drug 2. The whole interval sits above zero, so it agrees with the test.
Student reached the same conclusion with his own tables. He put ‘the odds are about 666 to one that 2 is the better soporific’. He also suggested why the spread of the differences was so small: ‘the different drugs reacting similarly on the same patient’.
What if you’d ignored the pairing?
Suppose you treated the same numbers as two separate groups of 10 people. The group standard deviations are 1.789 and 2.002 (Student’s 1.70 and 1.90, recalculated by dividing by 9). The standard error of the difference becomes √(1.789²/10 + 2.002²/10) = 0.849.
Now t = 1.58 ÷ 0.849 = 1.86. The unequal-variance (Welch) version has about 17.8 degrees of freedom, which rounds to 18. With equal group sizes, the equal-variance version gives the same t, with 18 degrees of freedom.
At 18 degrees of freedom the 5% critical value is 2.101. Our t of 1.86 falls short of it, so the result isn’t significant: p ≈ 0.08.
Same data, same mean difference, opposite conclusion. Pairing removes the differences between patients, because each person acts as their own control. That’s why a paired test is more powerful when your design really is paired, and why the wrong choice can hide a real effect.
What this means for you: decide whether your data are paired from your design, before you look at the results. If the same people gave both measurements, use the paired test.
The one-sample version
You can also test each drug on its own against zero extra sleep. For drug 2, t = 2.33 ÷ (2.002 ÷ √10) = 3.68, with 9 degrees of freedom and p ≈ 0.005. For drug 1, t = 1.33 and p ≈ 0.22.
That fits the original study’s conclusion. In the usual doses, drug 2 was of value as a sleeping drug and drug 1 was not. But a non-significant result for drug 1 doesn’t show it had no effect. Student himself put the odds at about 8 to 1 that it gave some extra sleep.
How to read the result: t, degrees of freedom, p and the confidence interval
A larger t means the difference is large compared with its uncertainty. Whether it’s large enough depends on the degrees of freedom: small samples need a bigger t.
| Degrees of freedom | Two-sided 5% critical value | Two-sided 1% critical value |
|---|---|---|
| 5 | 2.571 | 4.032 |
| 9 | 2.262 | 3.250 |
| 10 | 2.228 | 3.169 |
| 15 | 2.131 | 2.947 |
| 18 | 2.101 | 2.878 |
| 20 | 2.086 | 2.845 |
| 30 | 2.042 | 2.750 |
Source: NIST/SEMATECH e-Handbook of Statistical Methods, critical values of the t distribution (columns 0.975 and 0.995).
If your t is beyond the critical value, or your p-value is below 0.05, you reject the null hypothesis of no difference. A p-value of 0.05 or more means you haven’t shown a difference. It doesn’t show that there isn’t one.
The confidence interval often tells your reader more than p does. It gives the size of the difference and how precisely you’ve measured it, in the units of your outcome.
T-test assumptions, and what to do when they fail
The two-sample t-test assumes three things:
- Your data are measurements and plausibly Normal (roughly bell-shaped).
- The two groups have similar standard deviations.
- The observations are independent of each other.
The third assumption is the most important. Repeated measurements on the same person aren’t independent. If you measured 20 leg ulcers on 15 patients, you have 15 independent observations, not 20.
For a paired test, what matters is that the differences are plausibly Normal and independent of each other.
Your measurements may come from people within groups, such as patients within hospitals. Our guide to the unit of analysis explains how to decide whether your cases are individuals or groups.
Normality: look at your data
You might expect to run a normality test first. But a normality test depends on sample size. With a small sample, passing it doesn’t show your data are Normal; with a large sample, failing it doesn’t mean you can’t use a t-test. The t-test itself is robust to moderate departures from Normality, so plot your data and look for strong skew or outliers instead.
Unequal spreads: use the Welch version
If one group’s standard deviation is more than about twice the other’s, use Welch’s unequal-variance t-test. Many statistical packages now run it by default. If your data are strongly skewed, a transformation such as taking logarithms can sometimes help more.
When a t-test isn’t suitable
If your data are clearly not Normal and your sample is small, use a rank test instead. The Wilcoxon signed rank test is the usual choice for paired data, and the Mann-Whitney U test for two independent groups. The Wilcoxon test assumes the differences are roughly symmetrical.
T-test or ANOVA?
Use a t-test for one or two means and ANOVA for three or more. Comparing every pair of groups with separate t-tests raises the overall chance of a false positive above the level you set for each test. One-way ANOVA tests all the groups together instead.
How to report a t-test in your dissertation or thesis
Report the type of t-test, t with its degrees of freedom, the p-value, and the mean difference with its confidence interval. A common format puts the statistics at the end of a plain sentence:
[Outcome] was higher with [condition A] than [condition B], paired t([df]) = [t], p = [p], mean difference [value] (95% CI [lower] to [upper]).
For Student’s sleep data, that becomes:
Patients slept longer on drug 2 than on drug 1, paired t(9) = 4.06, p = .003, mean difference 1.58 hours (95% CI 0.70 to 2.46).
A few habits make your write-up stronger:
- Say which t-test you used, and for an independent test, whether it was the equal or unequal-variance version.
- Give the means and standard deviations for each group or time point.
- Say how you checked the assumptions.
- Follow your department’s style guide on details such as italics, decimal places and leading zeros.
If your scores came from a questionnaire, your methods chapter should also describe the research instrument you used to collect them.
Running a t-test in SPSS, R and Excel
Software does the arithmetic, but each package has defaults worth knowing.
- SPSS: for paired data, go to Analyze > Compare Means > Paired Samples T Test.
- R: the t.test() function runs the unequal-variance (Welch) test by default (var.equal = FALSE). Add paired = TRUE for a paired test. R’s own help pages use Student’s sleep data as their example of a paired t-test.
- Excel: the function T.TEST(array1, array2, tails, type) returns the p-value, not t. Set tails to 2 for a two-sided test, and type to 1 for paired, 2 for equal variances or 3 for unequal variances.
Common mistake: running an independent test in software when your data are paired. As the sleep example shows, it can turn p ≈ 0.003 into p ≈ 0.08.
Using the t-test in your own research
Before you run a t-test, write down whether your design is one-sample, independent or paired, and plot your data. Report the mean difference and its confidence interval alongside t and p. That lets your reader judge the size of the effect, not just whether it passed a threshold.
Interested in whether two measurements rise together, rather than whether their means differ? A monotonic relationship and a rank correlation may answer that better.
Enjoying the statistics? Our guide to doing a PhD in statistics covers research areas, entry requirements, funding and careers.
If you’re thinking of a research degree that uses methods like these, you can browse current PhD opportunities.
Sources
