Reading a chart critically
What this lesson covers
- Elements of chance affect the certainty of results from observational studies and experiments.
- Uncertainty should be taken into account when making claims.
Somebody drew that chart on purpose
- Last lesson you drew the graphs. Now someone else has.
- They had a conclusion in mind before they started.
- Every choice they made was a choice they could have made differently.
Before anyone misleads you: chance
- Toss a hundred coins. You expect fifty heads and you get 47.
- Do it again and you get 53. Nothing changed in between.
- So a small difference between two groups may be nothing at all.
| Trial | Expected | Got |
|---|---|---|
| Toss 100 coins | expect 50 heads | get 47 |
| Do it again | 53 | |
| Nothing changed | that is chance | |
| So a small difference | may be nothing at all |
Which changes how a claim should be written
- "B is larger than A" says you are certain.
- "B tends to be larger" allows for the wobble.
- The second is usually what the data actually supports.
| Claim | What it assumes |
|---|---|
| "B is larger than A" | a claim about certainty |
| "B tends to be larger" | a claim that allows for chance |
| The second one | is what data usually supports |
| Hedged language | is precision, not weakness |
Hedged language is not weak
- It sounds less confident and it is more precise.
- A claim that ignores uncertainty is overstating what is known.
- In statistics, saying exactly how sure you are is the skill.
Four things to check on any chart
- Where does the axis start, and are the intervals even?
- Who was asked, and how many of them?
- Does the picture match what the numbers say?
| Check | |
|---|---|
| Where does the axis start? | |
| Are the intervals even? | |
| Who was asked, and how many? | |
| Does the picture match the numbers? |
Where the axis starts
- Ninety-six and ninety-eight are about two percent apart.
- On an axis starting at ninety they are drawn as six and eight.
- One bar looks a third taller. Nothing on the chart is false.
| Data | Drawn as | Looks like |
|---|---|---|
| Values 96 and 98 | about 2% apart | |
| Axis from 90 | drawn as 6 and 8 | |
| One bar looks | a third taller | |
| Check the bottom | of every axis |
Whether the intervals are even
- Years marked 2016, 2018, 2019, 2024, spaced equally apart.
- The gaps are two years, one year, and five.
- So the steepness of that line means nothing whatsoever.
| Problem | Detail |
|---|---|
| Years marked | 2016, 2018, 2019, 2024 |
| Spaced evenly | but they are not even |
| So the line’s steepness | means nothing |
| Read the labels | not the shape |
When the picture is the graph
- Sales doubled, so the icon is drawn twice as wide.
- And twice as tall, or it would look stretched.
- Twice by twice is four times the area. It looks like a quadrupling.
| Step | Working | Effect |
|---|---|---|
| Sales doubled | so draw the icon | twice as wide |
| But also twice as tall | to keep the shape | |
| Area is now | 2 × 2 | four times |
| Looks like | a quadrupling |
You already know why that happens
- Scale every length by k and the area goes up by k squared.
- It is the rule from the measurement lessons, used as a trick.
- Three times as wide covers nine times the space.
Where the graph begins
- Start at an unusually low year and any rise looks enormous.
- Start at an unusually high one and the same rise disappears.
- Same data both times. Ask why that start date.
| Choice | Effect | Note |
|---|---|---|
| Start the graph | at an unusually low year | the rise looks huge |
| Start it | at an unusually high one | the rise vanishes |
| Same data | both times | |
| Ask why | that start date |
A percentage with nothing behind it
- Seventy percent prefer it. Seventy percent of how many?
- Of ten people, that is seven people.
- Of a thousand, it is seven hundred. Only one of those is evidence.
| Claim | Really |
|---|---|
| "70% prefer it" | of how many people? |
| 70% of 10 | is seven people |
| 70% of 1 000 | is seven hundred |
| One is evidence | the other is a coincidence |
And who was asked
- A survey of its own customers will find people who like it.
- That is the sampling bias from the first statistics lesson.
- A chart cannot fix a sample that was wrong before it was drawn.
Two things moving together
- Ice cream sales go up, and so do drownings. Every year.
- Neither one causes the other. Both happen in summer.
- Moving together is not causing. That is all you can say here.
| Observation | Explanation | Conclusion |
|---|---|---|
| Ice cream sales rise | and so do drownings | |
| Both rise in summer | that is the link | |
| Neither causes | the other | |
| Moving together | is not causing |
This is not about trusting nothing
- Everything in this lesson is a specific thing you can look at.
- Check them and you can decide about a chart on its merits.
- Believing nothing is as lazy as believing everything.
What to take away
- Seven lines, and the first one applies to honest charts too.
| Results vary by chance | so a small gap may be nothing |
| Claims should be hedged | "tends to" is more precise |
| Check where the axis starts | always |
| Check the intervals are even | |
| A doubled picture | covers four times the area |
| A percentage | needs a number behind it |
| Moving together | is not causing |
Your turn: four questions
- Pause here and answer all four in sentences.
- Question 2 wants the arithmetic, not just the word misleading.
- Question 4 has two things missing, not one.
| Question | |
|---|---|
| 1 | A chart’s axis runs 90 to 100. Check what? |
| 2 | "Doubled", with an icon twice as wide? |
| 3 | Ice creams and drownings both rise. Cause? |
| 4 | "70% prefer X." What is missing? |
Show answer
1. How big the differences really are. With the bottom of the axis cut off, a two percent gap can be drawn as a bar a third taller, so work out the actual difference from the numbers rather than from the picture. 2. If the icon was scaled in both directions it now covers four times the area, not twice, so the picture overstates the change. Doubling the width alone would be honest and would look wrong, which is why nobody does it. 3. No. Both rise in summer, so the weather is behind each of them separately. Two things moving together does not make one the cause of the other. 4. How many people were asked, and who they were. Seventy percent of ten people is seven people, which is not evidence of anything.
The answers
- How big the differences really are, from the numbers.
- Twice as wide and tall is four times the area.
- No. Summer is behind both of them separately.
- How many people, and who they were.
Is your child on track?
Take the free placement test