Surveys and bias: 7 exam questions solved step by step

← Previous9 of 10Next →

What this lesson covers

  • A sample can only speak for the population its sampling frame reaches.
  • Good design reduces bias; it never eliminates it.
Key idea

7 questions, one method

  • 1. Who could be chosen? Who answered?
  • 2. How was it asked?
  • 3. Why it matters here, and which way.
  • 4. Bias is reduced, never eliminated.
Your turn

Question 1 · Achieved

  • An invented school wants the views of all
  • 1200 students on a new uniform. It emails
  • 200 chosen at random from the 450 on its
  • newsletter list. Name the frame and the
  • problem. Pause the video and try it.
Worked example

Q1: the uniform survey

  • Target: who it is about.
  • Frame: who could be chosen.
  1. target: all 1200 studentsthe population
  2. frame: newsletter list, 450who could be chosen
  3. 450 ÷ 1200 = 0.375 = 37.5%
  4. 1200 − 450 = 750 never chosen
  5. subscribers may differkeener, more involved
  6. speaks for subscribers only
Your turn

Question 2 · Achieved

  • An invented community board asked 400
  • residents: "Do you agree the dangerous,
  • run-down skate park should be replaced?"
  • 312 said yes. Fix the question.
Worked example

Q2: the skate park question

  • Which words push?
  • Which way?
  1. 312 ÷ 400 = 0.78 = 78%
  2. "Do you agree..."leading
  3. "dangerous, run-down"loaded words
  4. 78% is likely too highpushed towards yes
  5. Neutral: "Do you support or
  6. oppose replacing the skate park?"
Your turn

Question 3 · Merit

  • An invented council posted a survey to
  • 2000 randomly chosen homes. 460 came
  • back; 64% of them oppose a new rubbish
  • charge. Evaluate the 64%.
Worked example

Q3: who sent it back?

  • Random start. Who replied?
  • Which way could it push?
  1. 460 ÷ 2000 = 0.23 = 23%response rate
  2. 77% did not replynon-response
  3. strong views reply moreoften opponents
  4. 64% may overstate opposition
  5. MOE ≈ 1/√460 ≈ 4.7%sampling error only
  6. reminders, follow-up callsreduce the bias
  7. non-response may bias the 64%
Your turn

Question 4 · Merit

  • An invented gym offered a free month to
  • members who did its satisfaction survey.
  • 340 of the 400 who replied were satisfied.
  • What did the incentive do, and why?
Worked example

Q4: a free month

  • What it did.
  • Why, in this survey.
  1. 340 ÷ 400 = 0.85 = 85%
  2. incentive: more repliesless non-response
  3. the reward comes from the gym
  4. members may answer to pleaseresponse bias
  5. ex-members not askedthe frame
  6. 85% likely overstates it
Your turn

Question 5 · Merit, in context

  • An invented council ran an online survey
  • on bus timetables: 900 replied, and 64%
  • want more evening buses. Evaluate the
  • method, in context.
Worked example

Q5: an online survey

  • Plus and minus, in context.
  • Not a learned list.
  1. cheap and quickthe upside
  2. online, chose to replyself-selected
  3. misses people rarely online
  4. often older, lower income
  5. may rely on buses mostviews may differ
  6. 64% may not speak for all
  7. face to face, random homeshigher response
  8. slower and costlierthe trade-off
Your turn

Question 6 · Excellence

  • An invented region is 45% city, 35% towns
  • and 20% rural. A survey will ask 800
  • residents. Plan a stratified sample and
  • say what it does and does not fix.
Worked example

Q6: split the 800

  • Each area in proportion.
  • Random within each area.
  1. city: 0.45 × 800 = 360
  2. towns: 0.35 × 800 = 280
  3. rural: 0.20 × 800 = 160
  4. 360 + 280 + 160 = 800
  5. random within each areanot volunteers
Worked example

Q6: what it fixes

  • Why stratify here?
  • And its limits.
  1. each area in proportionmore representative
  2. views differ by areawhy it matters
  3. whole region: 1/√800 ≈ 3.5%
  4. rural only: 1/√160 ≈ 7.9%
  5. rural estimate less precise
  6. non-response, wording remain
  7. bias reduced, not eliminated
Your turn

Question 7 · Excellence

  • An invented report: "Our random sample of
  • 3000 from a list of gym members eliminates
  • bias. 62% of adults exercise three times a
  • week (margin of error 1.8%)." Evaluate.
Worked example

Q7: the numbers, then the frame

  • The sum may be right.
  • Who could be chosen?
  1. 1/√3000 = 0.0183 ≈ 1.8%arithmetic right
  2. 62 ± 1.8: 60.2% to 63.8%
  3. target: all adultsthe population
  4. frame: gym memberswho could be chosen
  5. members exercise morebiased upwards
Worked example

Q7: does it eliminate bias?

  • Random within a wrong frame.
  • Never write eliminates.
  1. random within the frame only
  2. "eliminates bias"reduced, never removed
  3. bigger n: smaller MOE onlynot less bias
  4. MOE: sampling error only
  5. 62% overstates all adults
Summary

The method that solved all seven

  • Who could be chosen, who answered, how, and which way.
StepHow
Framewho could be chosen?
Wordingneutral, one idea
Replieswho answered, and why?
Directionwhich way it pushes
Stratifyrandom within each group
Biasreduced, never eliminated

More in AS 91584

Margin of error: the three rules of thumb6 minConfidence intervals, claims, and why overlap fails7 minExperiments vs observational studies: what you can conclude7 minSurveys and bias: sampling frames, non-response and wording6 minEvaluating a statistical report: the full answer7 minMargin of error: 7 exam questions solved step by step10 minConfidence intervals: 7 exam questions solved step by step10 minStudy design: 7 exam questions solved step by step10 minSurveys and bias: 7 exam questions solved step by step10 minEvaluating reports: 7 exam questions solved step by step10 min

Is your child on track?

Take the free placement test