The book describes Galton’s habit of recording observations and the author’s father keeping detailed sporting records, including successful shots and errors. The point is systematic observation rather than remembering only the dramatic result.
Make the sample visible
“Eight successful cases” is incomplete without knowing whether there were ten, fifty or a thousand eligible cases. In our exercise, an observation is eligible because it meets the predetermined condition, not because its result is interesting.
Worked example
Eight favorable outcomes out of ten observations is 80%; eight out of forty is 20%. The remembered success count is identical, but the evidence is very different.
Case connection
Reporting the attempted replications alongside the successful ones makes the denominator visible.
Psychology’s replication project
Source-grounded facts
Researchers attempted to replicate 100 psychology findings; 36% reached statistical significance in the same direction.
Context
A published finding answers a question in one study. The Reproducibility Project: Psychology asked whether independent repetitions would support a selected group of published findings.
Outcome
The project exposed uncertainty about reproducibility. A failed significance test is not automatically proof that the original claim was false or fraudulent; interpretation requires examining methods, precision and conditions.
- Researchers attempted to replicate 100 psychology findings.
- They compared the new evidence with the original results, including direction, significance and effect sizes.
- The Center for Open Science reports that 36% reached statistical significance in the same direction; replication effect sizes were about half the originals.
Case analysis
Record the number attempted as carefully as the number that met your criterion. Then explain the criterion itself: significance in the same direction is not identical to proof of an underlying theory. A denominator makes the summary intelligible; a classification rule tells the reader what was counted. Both are needed before interpreting the reported success rate.
Try it
For a small everyday claim, record every eligible case for a week. Include the cases that disappoint your expectation.