Thinking Critically By Living In Other Worlds
“Thus I saw that most [people] only care for science so far as they get a living by it, and that they worship even error when it affords them a subsistence.”
– Johann Wolfgang von Goethe
People are bad at critical thinking. Like, really bad. More specifically they are bad at interpreting evidence. Bad interpretation leads to, obviously, bad science application, but it also leads to poorly designed studies. Before we get to how that applies to study design, let’s think a little bit more about how we should consider evidence.
Evidence comes in many forms. The quality of a piece of evidence has to do solely with how strongly it allows you to update your knowledge and understanding of the world you live in. Although the findings of rigidly controlled studies tend to be higher quality evidence, that is not as important. Trust me, you can have a really well controlled study that tells you nothing useful.
Let’s think about how we critically consider an idea. Typically, we would ask, “Is this idea true or false?” Then we take some evidence and try to see if it supports the idea or not. Does that look familiar?
Now, it’s not that this is “wrong” per se, but there is a lot of room for confirmation bias. Humans like to look for evidence in support of ideas that we prefer. You know, like the idea that the device you’re reading this on keeps tigers away. I mean, you don’t see any tigers around here do you?
The best way to think about evidence is from a “many-worlds interpretation”
“THE HELL? Many-worlds interpretation??? What kind of complicated mess are you getting me into???”
Relax. This actually is not at all complicated and makes understanding evidence much simpler.
“Whatever, nerd…”
So “many-worlds” is a position that posits that there are an almost infinite number of realities. For example, there could be a reality where Hitler cured cancer or Jonas Salk didn’t invent the polio vaccine. We just happen to be in the reality that we are in. Don’t worry about whether or not this is “true” – that’s not relevant.
Got it? Great. Now, let’s come back to our idea that we were considering. Instead of asking, “Is this idea true or false?” as we did before we now would say, “There is a reality where this idea is true and a reality where the competing idea is true.” Instead of thinking about it as a true/false dichotomy, we are now thinking about it as a true/true dichotomy.
Now this true/true dichotomy is not exactly better, it just fits the way humans like to think. We like to confirm and support our ideas. It also allows you to use your imagination in a constructive way.
Considering Evidence
So let’s take some evidence and apply it to our two realities. First, your preferred reality:
“Is there any way this evidence could exist in a reality where my preferred idea is true?”
Almost always the answer would be “Yes” mainly because your reasoning is often motivated. This could very well be poor evidence with a dose of confirmation bias. Many people stop right here and consider their preferred idea to be supported. This is wrong. Very wrong. We cannot judge the quality of the evidence yet.
We need to change worlds:
“Is there any way this evidence could exist in a reality where the competing idea is true?”
Now, right here is where you can judge your evidence. If the answer is also “Yes”, then you have some crappy evidence on your hands.
Do you see how that worked? If the answer to the questions are in opposition (Yes/No) the evidence is useful. It moved your understanding forward. You now know which world is more like the one you live in.
On the other hand, if the answer to the questions are the same (Yes/Yes) then both realities are still possible. You have essentially learned NOTHING. At least, not yet…
Moving past low quality evidence
If you ended in a Yes/Yes situation, then you need to ask yourself the question, “Why?” Usually this has to do with the way the evidence was acquired. For example, a small sample size or failure to randomize or researcher bias could make both realities still be possible. This is where study design is so important. It makes the evidence useful.
Now, a good skeptic will always seek out other possible realities. In other words, just because you found a Yes/No situation, that doesn’t mean you are done. Try to imagine another, as of yet unconsidered reality where the answer, based on the current evidence, is still Yes/Yes. That is where the next studies should be directed.
Don’t be stupid
One way this gets abused is when people use really bad ad hoc reasoning to hold their preferred position anyway. For example, when you ended with a No/Yes (your preferred idea was refuted), you start really trying to force it into a Yes/Yes by making weak justifications that sound like “complexity/individuality” or “in my experience” or “some kind of quantum energy thingy”, etc. That’s called motivated reasoning. Stop that dammit.
If you do believe that you have forced a true Yes/Yes, the onus is on YOU to design a new study to arrive at a Yes/No with your new explanation. Otherwise you are flying blind but, you know what? Whatever. Do whatever you want. You’re probably going to anyway…
In summary…
- Humans are bad at critical thinking
- Confirmation bias and motivated reasoning are really bad practices and really hard to avoid
- Your blog reading device keeps tigers away
- Instead of “True/False” consider alternative realities
- When successful in getting a Yes/No, try to imagine an alternate Yes/Yes that is still possible
- Don’t be stupid
The featured image on this post is by geralt via pixabay.