Resources
Spotting a misleading analysis
Four mechanisms explain most false conclusions drawn from real data: sample selection, choice of denominator, confusing correlation with causation, and the truncated axis. Spotting them takes seconds.
Four questions to ask any chart
Who was counted, and who is missing? A satisfaction rate computed on customers who replied describes the respondents, not the customer base. What is the percentage computed on? Changing the denominator changes the conclusion without changing a single data point. Do things that move together actually cause one another? A third variable often explains both. Does the axis start at zero? A compressed scale manufactures a dramatic slope from an ordinary variation.
- The scope counted, stated before the result
- The denominator, displayed next to the percentage
- The comparison period, chosen before looking at the figures
- The axis origin, visible on the chart
- The data's source and date, carried on the figure
The mechanism, its telltale sign, the counter
This table reflects the block devoted to the subject in the decision-makers' certificate. Each row is checked with one question, asked out loud in a meeting.
| Mechanism | Telltale sign | The check |
|---|---|---|
| Selection effect | A rate computed on volunteers or respondents | Ask for the full scope and the number of non-responses |
| Chosen denominator | A percentage with no reference set | Rebuild the raw count and recompute |
| Correlation read as cause | Two lines rising together | Look for a common variable, test a separate period |
| Truncated axis | A steep slope on a small variation | Redraw the axis from zero and re-read the conclusion |
| Chosen period | A comparison starting at a trough | Compare across several periods of equal length |
Where this skill is taught
- taught hours on the subject
- 30
- hours of applied statistics
- 240
- hours on presentation
- 160
Dedicated block of the decision-makers' certificate.
Estimation, testing, regression, experimental design.
Graphical grammar, narrative, communicating uncertainty.
Frequent questions
Do these checks require statistics?
The four main checks are a matter of careful reading and need no formalism. Statistical methods take over to quantify uncertainty or establish an effect, and they have dedicated blocks in the data programmes.
How do I present uncertainty without losing my audience?
By giving a range and the decision it supports. « Between this and that level, we add staff » is remembered and applied, where an isolated interval stays abstract. That is what the presentation block covers.
How do I ask these questions without challenging the author?
By asking them of the chart rather than of the person: asking for the scope counted and the denominator is a matter of reading, not of trust. Phrased that way, they become a team practice everyone applies to their own figures.
Explore next
- Machine learningRegression, classification, cross-validation, model selection and interpretability: what a model actually does and how it is validated.
- Models in productionInference service, drift monitoring, retraining, running cost and the trade-off between local and cloud processing.
- Data governanceData ownership, life cycle, hosting, retention, reversibility and vendor dependence: framing governance.
- Personal dataPurpose, legal basis, minimisation, retention, security and individual rights: the processing register in practice.
- Industrial site safetyHSE management system, work permits, co-activity, internal audit, leading indicators and safety culture: a site's complete arrangement.
- Assessing a riskHazard, exposure, criticality, risk register and the hierarchy of prevention measures: risk assessment applied to a real workstation.