Evaluating a statistical report: the full answer
What this lesson covers
- An Excellence answer ties every statistical point to the report's context.
- Sample size alone is a weak criticism; representativeness and scope of inference matter more.
Judging a claim in a report
- A booklet of reports, a claim to judge.
- One full answer, built step by step.
- The same skills for any report.
Read the question first
- Underline the claim, the group, the command.
- Find the part of the report that matters.
- Never copy out the method.
An invented report
- 250 students volunteered online.
- Breakfast daily: 108 of 150 passed.
- Claim: breakfast improves maths results.
| The report | |
|---|---|
| Design | online survey |
| Sample | 250 volunteers |
| Where | one school |
| Breakfast daily | 108 of 150 passed |
| Not daily | 58 of 100 passed |
Use the numbers
- 72% against 58%.
- A difference of 14 percentage points.
- Check what each percentage is out of.
| Working | Value | |
|---|---|---|
| Daily | 108 ÷ 150 | 72% |
| Not daily | 58 ÷ 100 | 58% |
| Difference | 72 − 58 | 14 points |
| Passers who ate | 108 ÷ 166 | about 65% |
Read the axis before the bars
- Axis from 50%: nearly three times as tall.
- Axis from zero: much closer.
- Quote the values, not the heights.
Critiquing the causal claim
- "Improves" claims a cause.
- Observational: the students chose.
- Cause needs a randomised experiment.
| This report | |
|---|---|
| Study type | observational |
| Who chose | the students |
| Other variables | sleep, routine |
| Shows cause? | no |
Explain it in this context
- Breakfast eaters may also sleep more.
- Sleep could lift results too.
- Random allocation tends to balance it.
Who do the results describe?
- 250 volunteers from one school.
- Volunteers may be more engaged.
- Say why it does not extend.
| This report | |
|---|---|
| Sampled | 250 volunteers |
| From | one school |
| Claim is about | all students |
| Can describe | these volunteers |
"Too small" is the weak criticism
- A named 2025 error.
- Size affects precision, not bias.
- How it was chosen matters more.
| Weak | Strong | |
|---|---|---|
| Sample | too small | volunteers, not random |
| Cause | might be wrong | observational |
| Extend | not everyone | one school only |
Answer in the report's context
- "There may be bias" fits any study.
- Say who, what and why.
- Volunteers may raise the pass rates.
The full answer
- Claim, evidence, design, scope.
- Then a judgement.
- A difference here; the claim not supported.
| Step | Say |
|---|---|
| Claim | improves: causal |
| Evidence | 72% v 58% |
| Design | observational |
| Scope | these volunteers |
| Judgement | not supported |
Words that cost marks
- Bias is reduced, never eliminated.
- Data supports a claim; it does not prove it.
- Attempt every part; use spare pages.
Where the marks are
- Achieved: the features that matter.
- Merit: evidence from the report.
- Excellence: statistics and context, judged.
What to take away
- Numbers, design, scope, then a judgement, all in context.
| Question | underline key words |
| Numbers | quote them, check base |
| Cause | randomised experiment |
| Scope | who, and why not wider |
| Size | weaker than how chosen |
| Context | every sentence |
Your turn: four questions
- Pause here and answer each one.
- Warm-up: 8 of 80 injured; others 24 of 120.
- Claim: it halves injuries for all players.
| Given | Find | |
|---|---|---|
| 1 | 8 of 80 v 24 of 120 | compare |
| 2 | players chose it | cause? |
| 3 | one club only | all players? |
| 4 | "sample too small" | better point |
Show answer
1. 8 ÷ 80 = 10% of warm-up users injured, 24 ÷ 120 = 20% of others: 10 percentage points lower, half the rate, in this sample. 2. No: the players chose, so it is observational. Careful players may choose the warm-up and also play more safely. 3. No: one club, not a random sample of all netball players, so it describes this club at most. 4. The players chose the warm-up for themselves, and they come from one club: a design and representativeness point, not size.
The answers
- 10% against 20%: half, in this sample.
- No: observational, players chose.
- No: one club, not a random sample.
- Self-chosen, and one club only.
Is your child on track?
Take the free placement test