When A Valid Argument Can Be False
Photo Credit: North Carolina Digital Heritage Center
“That’s valid,” you say, but what do you mean by that? A single statement can be valid by itself if it is a previously proven “truth”, but what about an argument? You remember arguments, right? Premise, premise, therefore conclusion? Funny thing about valid arguments, they have nothing to do directly with truth. Arguments can be valid and false at the same time, just as they can be invalid and true at the same time. What?
Since deductive arguments are the basis of all research, you need to understand this concept. I have quoted before on a podcast, “A flawed study is still a flawed study regardless of p-value or level of evidence.” Here’s how!
So let’s unpack this concept. The easiest example would be to use what is known as modus ponens (“the way to affirm”). It is the simplest argument structure and goes like this:
If P, then Q.
P.
Therefore Q.
First let’s define the components. Lines 1 and 2 are the premises and line 3 is the conclusion. Notice the simplicity. Even without knowing P or Q, you can see that the argument is valid. An invalid argument escapes this. For example:
If P, then Q.
Q.
Therefore P.
Even without knowing P or Q, you can see that the argument is invalid. “Validity” only has to do with the structure of the argument. So let’s fill in some blanks to understand this better.
If a rock, then it is made of minerals.
This is a rock.
Therefore it is made of minerals.
I’m playing off “All rocks are made of minerals” since this is a well known truth. This fits our structure of a valid argument above. We also know that it is likely true, but we’ll talk about that later. Let’s move things around.
If a rock, then it is made of minerals.
This is made of minerals.
Therefore it is a rock.
This argument is invalid since it fits the structure of the invalid example above. The conclusion is also false but not because it is invalid. Again, we will get to that. Let’s move things around again.
If a mineral, then it is made of rocks.
This is made of rocks.
Therefore it is a mineral.
This argument is also invalid since its format is exactly like the previous one. But is the conclusion true? Hmmm. Let’s move things around again.
If a mineral, then it is made of rocks.
This is a mineral.
Therefore it is made of rocks.
Now, is this a valid argument? Absolutely! Look at the structure. It looks just like this:
If P, then Q.
P.
Therefore Q.
But the conclusion is false. Why? What should be obvious is that one of the premises is false – “If a mineral, then it is made of rocks.” This gets to whether or not an argument is sound. In order for an argument to be sound, it must be valid and ALL the premises must be true. Let’s look at the first example of rocks and minerals again.
If a rock, then it is made of minerals.
This is a rock.
Therefore it is made of minerals.
Is this sound? Remember I said before that this is likely true. We actually don’t know if it is sound because we don’t know about the veracity of line 2. That person may be holding a tennis ball. That would also make one of the premises false and therefore the argument would be unsound.
Now what does this have to do with science? An individual research paper is nothing more than a deductive argument – Premise, premise, conclusion. I’ve talked before about Tooth Fairy Science, when valid research is conducted on a false premise. When you are looking at a research article, so many people get lost in the minutiae of statistical validity. Was it powered properly? Was there proper blinding? Was the statistical analysis performed properly? This is all very important. As I said, for an argument to be sound it must be valid. But all the premises must also be true.
Now how many of you get in over your head when people start debating validity – statistical details and all that whatnot? The great physicist Richard Feynman used to famously get exasperated over this, specifically in the aftermath of the Challenger disaster. Don’t get lost in the statistics. Before you even go there, assess the veracity of the premises. Are they actually exploring what they claim they are exploring? Think of this again:
If P, then Q.
P.
Therefore Q.
Is the initial setup premise (If P, then Q) truly been proven already? Are they sure that they are actually observing the second premise (P) through their selection? This is so much easier to assess than validity and sadly so prevalent! Then look at basic validity as I have described above. Then you can dive into the statistics, etc. Remember, the conclusion may still be true, but that study may not actually tell you anything.
“A flawed study is still a flawed study regardless of p-value or level of evidence.”