Evaluating reports: 7 exam questions solved step by step

← Previous10 of 10Next →

What this lesson covers

  • An Excellence answer ties every statistical point to the report's context.
  • Sample size alone is a weak criticism; representativeness and scope of inference matter more.
Key idea

7 reports, one method

  • 1. Compute: rate, margin, interval.
  • 2. Say it: population and variable.
  • 3. Design: how chosen? cause?
  • 4. Judge the claim, in context.
Your turn

Question 1 · Achieved

  • Invented poll of 1600 random NZ adults:
  • 46% grow some of their own vegetables.
  • Claim: "Under half of NZ adults do."
  • Form the interval. Judge the claim.
Worked example

Q1: one result, one interval

  • One result: 1/√n.
  • Then judge the claim.
  1. n = 1600, result 46%
  2. MOE ≈ 1 ÷ √16001/√n rule
  3. = 1 ÷ 40 = 2.5%
  4. 46 − 2.5 = 43.5%
  5. 46 + 2.5 = 48.5%
  6. CI: 43.5% to 48.5%
  7. 48.5% < 50%all of it under half
  8. claim supported
Your turn

Question 2 · Achieved

  • Invented survey of 250 cyclists.
  • Claim: "Riders in hi-vis had under
  • half the near-miss rate of others."
RiderNear missNoneTotal
Hi-vis127890
No hi-vis32128160
Total44206250
Worked example

Q2: rates, not counts

  • Groups differ in size.
  • Divide by each row total.
  1. 12 against 32groups differ in size
  2. hi-vis: 12 ÷ 90 = 0.1333
  3. others: 32 ÷ 160 = 0.2
  4. 13.3% against 20%
  5. (12 ÷ 90) ÷ 0.2 = 0.6667about two thirds
  6. two thirds, not under half
  7. claim not supported
Your turn

Question 3 · Merit

  • Invented poll of 625 random Tauranga
  • voters on a new pool: site A 44%,
  • site B 38%. Headline: "Voters prefer A."
  • Is the headline supported?
Worked example

Q3: a difference in one poll

  • Same poll: 2/√n.
  • Is zero inside?
  1. difference = 44 − 38 = 6 points
  2. MOE ≈ 2 ÷ √625within one group
  3. = 2 ÷ 25 = 8%
  4. 1 ÷ √625 = 4%one result only
  5. 6 − 8 = −2, 6 + 8 = 14
  6. CI: −2 to 14 points
  7. zero is insidecould be no difference
  8. headline not supported
Your turn

Question 4 · Merit

  • Two invented random surveys of teens:
  • Nelson, n = 400: 61% sleep under 8 h.
  • Hamilton, n = 1600: 55% do.
  • Is Nelson's rate really higher?
Worked example

Q4: two separate groups

  • Two groups: 1.5 × average.
  • Not overlap.
  1. Nelson: 1 ÷ √400 = 5%
  2. Hamilton: 1 ÷ √1600 = 2.5%
  3. average = (5 + 2.5) ÷ 2 = 3.75
  4. MOE ≈ 1.5 × 3.75 = 5.625
  5. difference = 61 − 55 = 6
  6. 6 ± 5.625: 0.375 to 11.625
  7. intervals overlapnot the test
  8. all above zero
  9. Nelson's rate is higher
Your turn

Question 5 · Merit, in context

  • Invented study: 400 adults chose
  • whether to take daily cold showers.
  • Claim: "Cold showers cut sick days."
GroupSick dayTotal
Cold showers30160
Others72240
Worked example

Q5: a causal claim

  • "Cut" claims a cause.
  • Who chose the groups?
  1. showers: 30 ÷ 160 = 0.1875
  2. others: 72 ÷ 240 = 0.3
  3. 18.75% against 30%
  4. study: observationalthe adults chose
  5. fitter people may choosepossible confounder
  6. cause needs random allocation
  7. causal claim not supported
  8. an association, these adults
Your turn

Question 6 · Excellence

  • Invented report: 5000 NZ teens answered
  • a pop-up survey on a gaming website.
  • 72% game for over 2 hours a day.
  • Claim: "Most NZ teens game over 2 h."
Worked example

Q6: big, but who answered?

  • Size is not the problem.
  • Who could answer?
  1. MOE ≈ 1 ÷ √5000 = 0.0141
  2. 72 ± 1.4: 70.6% to 73.4%
  3. MOE: sampling error onlynot bias
  4. frame: gaming-site visitors
  5. self-selected: keen gamersvolunteer bias
  6. 72% likely too high
  7. bigger n does not fix bias
  8. claim not supported
Your turn

Question 7 · Excellence

  • Invented experiment: 200 volunteers
  • at one Dunedin gym, randomly split.
  • Claim: "Reminder apps keep
  • New Zealanders going to the gym."
GroupStill goingTotal
Reminders64100
No reminders48100
Worked example

Q7: the difference

  • Two groups: 1.5 × average.
  • Is zero inside?
  1. reminders: 64 ÷ 100 = 64%
  2. none: 48 ÷ 100 = 48%
  3. difference = 64 − 48 = 16 points
  4. 1 ÷ √100 = 10%each group
  5. MOE ≈ 1.5 × 10 = 151.5 × average
  6. 16 ± 15: 1 to 31 points
  7. all above zero
Worked example

Q7: cause, and for whom?

  • Random allocation: cause.
  • Volunteers: not all NZ.
  1. 1 to 31 points, all above zero
  2. randomly allocated: experiment
  3. causal link: supportedfor these volunteers
  4. one gym, volunteers onlynot random selection
  5. cannot extend to all NZers
  6. claim, as worded: not supported
Summary

The method that judged all seven

  • Numbers first, then design and scope, then a judgement.
StepHow
One result± 1/√n, then say it
Same polldifference ± 2/√n
Two groups± 1.5 × average margin
Zero insideno clear difference
Causeonly a randomised experiment
Who it fitshow they were chosen

More in AS 91584

Margin of error: the three rules of thumb6 minConfidence intervals, claims, and why overlap fails7 minExperiments vs observational studies: what you can conclude7 minSurveys and bias: sampling frames, non-response and wording6 minEvaluating a statistical report: the full answer7 minMargin of error: 7 exam questions solved step by step10 minConfidence intervals: 7 exam questions solved step by step10 minStudy design: 7 exam questions solved step by step10 minSurveys and bias: 7 exam questions solved step by step10 minEvaluating reports: 7 exam questions solved step by step10 min

Is your child on track?

Take the free placement test