What Would Change Your Mind?
What you will be able to do: Naming a specific, obtainable observation that would lower your confidence in a position you hold
Before you start
A position you cannot imagine losing is not a conclusion; it is a commitment. The test is simple to state and uncomfortable to apply: what would you have to see to believe something else?
The two items below give you a position and ask which observation would move you. In each case, one option discriminates between explanations and the others merely add or subtract weight.
Every scenario on this page is fictional and was written for this exercise. None of them describes a real organization, person, or event.
Practice
Choose a response for each item, then read why it is stronger or weaker. There is no score, no account, and nothing is sent anywhere. Your answers stay in this browser tab and disappear when you close it.
1. Name the disconfirming evidence
You believe a local after-school programme has reduced neighbourhood vandalism, based on a drop in reported incidents since it opened.
Strongest reading
Strongest. This is the observation that discriminates. A similar decline elsewhere would indicate a common cause, such as a change in reporting practice or a wider trend, and would leave your explanation with nothing to explain.
Partly right
Relevant, and it would reduce the plausible size of the effect, but it is compatible with a real effect among those who do attend. It weakens the claim without discriminating between explanations.
Weaker reading
An assertion from an interested party, without stated evidence, does not bear much on the question. If the council member has data, the data is what matters.
Show the full answer key for this item
What remains uncertain: Reported incidents measure reporting as well as incidents. A programme that increases trust in local institutions could raise reporting while reducing events, which would obscure a real effect.
- Reported incidents fell by a similar amount in comparable neighbourhoods without the programme. Strongest reading
- Strongest. This is the observation that discriminates. A similar decline elsewhere would indicate a common cause, such as a change in reporting practice or a wider trend, and would leave your explanation with nothing to explain.
- Attendance at the programme is lower than organizers hoped. Partly right
- Relevant, and it would reduce the plausible size of the effect, but it is compatible with a real effect among those who do attend. It weakens the claim without discriminating between explanations.
- A council member says the programme is not working. Weaker reading
- An assertion from an interested party, without stated evidence, does not bear much on the question. If the council member has data, the data is what matters.
2. Name the disconfirming evidence
You believe a widely shared claim is false because the account that posted it has shared false claims before.
Strongest reading
Strongest. Your conclusion is based on the source rather than the claim. Independent primary evidence addresses the claim directly and would override a source-level judgment, which is exactly what it should do.
Weaker reading
Weak, because repetition is not independence. If the other accounts derived the claim from the same origin, their number adds no evidential weight at all.
Partly right
Mildly relevant to how you weigh the source in general, since correction practices are a real quality signal. It says nothing about whether this particular claim is true.
Show the full answer key for this item
What remains uncertain: Judging a claim by its source is efficient and sometimes necessary, but it produces exactly this failure mode: an unreliable source occasionally reports something true, and source-level reasoning cannot detect it.
- A primary document or recording, independent of that account, that supports the specific claim. Strongest reading
- Strongest. Your conclusion is based on the source rather than the claim. Independent primary evidence addresses the claim directly and would override a source-level judgment, which is exactly what it should do.
- Several other accounts repeating the same claim. Weaker reading
- Weak, because repetition is not independence. If the other accounts derived the claim from the same origin, their number adds no evidential weight at all.
- The account issuing a correction on an unrelated earlier claim. Partly right
- Mildly relevant to how you weigh the source in general, since correction practices are a real quality signal. It says nothing about whether this particular claim is true.
What to take away
- Disconfirmation beats support.
- Almost everything is consistent with almost everything. The observation worth seeking is the one that is consistent with one explanation and not the others.
- Repetition is not corroboration.
- Additional sources add evidence only if they are independent of the original, and independence has to be checked rather than assumed.
- Source-level judgments have a known failure mode.
- Judging a claim by who said it is efficient, and it will occasionally be wrong in exactly the way it cannot detect. Primary evidence overrides it.
If you found the items straightforward, apply the same question to a position you hold that matters to you, and write the answer down before you look for evidence. Writing it in advance is the whole exercise.
Nothing you select on this page is transmitted anywhere. There is no account, no score, and no record. See the privacy page for details.
Topics covered: Information ResilienceCritical Thinking
Related material
Article
How Confident Should You Be?
Confidence language, what it should be based on, and how to say what you believe in a way that can be checked later.
Article
What Else Could Explain This?
A structured way to generate alternative explanations before settling on one, adapted from analysis of competing hypotheses for everyday use.
Lesson
Understanding Confirmation Bias
What confirmation bias is, the three distinct behaviours it covers, and the small number of countermeasures that survive contact with real evidence.
How this page is produced
Published by ARAC International under theTHINK editorial standards. If you find an error, please tell us through the corrections page.