Science Series – Part 5: Seppuku and The Tooth Fairy

This is the final post of a 5-part series that I am writing on science and PT practice. If you have not read Parts 1 – 4, do that first before reading any further…

Uh oh. Here comes the guru with 20+ years of experience.

“You are just some armchair philosopher! Who are you to say what is and isn’t science?”
“Me? I’m nobody. But I do have this…” I then slam the flaming laser sword on the table knocking over glasses and dinnerware.
“But I base all of my ideas on science,” says the guru. “Logically that makes my reasoning scientific.”
“You sure about that? Have you actually tested your conclusions step by step?” I nod to the bright shiny sword on the table, blazing hot with the glow of a laser.

I rather like this metaphor, but you may prefer, “SAY HELLO TO MY LITTLE FRIEND!!!” To each his own I guess.

So now we are holding a razor sharp flaming laser sword, beautiful in its precision. I enjoy swords. I practice the Japanese martial art of iaido, which involves a connection with the sword. It’s great exercise, it’s mind clearing, it’s badass (or really nerdy?), and it helps me get out aggression in an appropriate way as opposed to writing angry blog posts with links to scenes from Scarface…

Anyway, in iaido there is a kata (series of movements) known as junto where you perform the role of assistant in the ritual of seppuku. Seppuku involves turning your blade on yourself. This is what we as a profession need to do with the flaming laser sword. I’m not talking about that PT down the street practicing craniosacral therapy. Let’s face it, that guy stopped reading this a long time ago. I’m talking about you.

This is something that I have heard recently from many physical therapists that I consider scientifically minded: “Most IF NOT ALL of our success is due to non-specific effects so that is all we need to have.” Excuse me? What did you just say? If you believe that then go ahead and pack your bags and head over to the complementary and alternative medicine department – because that is their defense.

Now I will be the first to admit that all treatments, from surgery to Reiki, have non-specific effects. I also agree that we should do everything in our power to ENHANCE those non-specific effects whenever we can – it’s like low hanging fruit. The science, however, is in the specific effects. They MUST be there for experimentation to occur and science demands experimental attempts to falsify. Subjective changes are great, but you had better show me something physiological. I can’t test objectively against perception.

Craniosacral therapy combined with acupuncture can demonstrate changes in subjective and perceived functional outcomes against controls. Does that make it science? Show me the objective physiological change and then show me EXACTLY how well you can predict that change via reliable and reproducible experimentation – THAT is science.

In my opinion, pain can rarely be directly treated – at least not scientifically. As I keep hearing these days, pain is all perception (that sounds like the hopeless situation of Descartes’ “Evil Demon” to me). Now, given a specific pain pattern, mechanism of injury, and clinical exam (which has survived specific attempts to falsify in the literature), I can determine with a high and certain level of confidence that someone has likely torn their meniscus. With the help of a surgeon (after seeing first if time will heal it), the tear can be repaired and we can restore range of motion and the subsequent strength/coordination deficit after surgery, but we directly treated the torn meniscus and objective deficits, not the pain.

Similarly, I can evaluate someone with patellofemoral pain syndrome (or anterior knee pain or whatever you want to call it this week – I don’t have time for semantics) and could objectively determine that they are weaker in their quads and their hips on the involved side. I can also objectively treat that weakness resulting in a physiological change (increased strength through specific experimentally tested exercise prescription). Is it biologically plausible or even possible that this is causing their pain? Sure, but being biologically plausible and possible doesn’t mean shit. That is induction, not science. Until then, I’ll treat what I can objectively measure and change which in this case, is strength not pain. The reason that I provide treatment is because of pain, but that is not what I am actually treating – at least not scientifically.

What about chronic pain without any specific findings? There is evidence that treating the correlating depression is more effective than anything we could do. I don’t know if you are actually trained to treat depression, but I’m not. I don’t even know what CPT code I would use for that. Therapeutic exercise (97110) to get those endorphins going?

Now what about manual therapy? We all point and laugh at the lack of science behind things like acupuncture, craniosacral therapy, etc but then hold up manual therapy as being somehow more plausible (you already know what I think about mere plausibility). Maybe it is because we have all spent a TON of time and money on manual therapy continuing education and certifications. I smell a sacred cow…

When I graduated from PT school, we all knew that manual therapy worked, but we just didn’t know HOW it worked. We simply could not find the specific physiological effect but we knew it worked because of the subjective and functional outcomes (sound familiar?). Remember, not just plausible ideas that have be inductively reasoned from scientific premises, but actual objective, experimental evidence.

It is now 15 years later and we still have no specific physiological effect. All we have now is, “Move it and move on.” Seriously? No physiological finding specific to the injured population that is then changed through manual therapy? Am I the only one uncomfortable with people calling that position “scientific”?

Actually I’m not. Read this editorial from the special issue of Physical Therapy Journal devoted to manual therapy. Now realize that it was written 20 years ago. Many of these concerns remain and many of the questions are still unanswered. Are we reasoning in circles? I really miss Dr. Rothstein…

But what about the Clinical Prediction Rule (CPR) for spinal manipulations? Isn’t that science? Let’s go back to my example of Sir Bedevere:

That silly scene ends in an experiment where they try to see if the woman weighs more or less than a duck (I preferred building a bridge out of her). They answered their question but who cares? Their entire premise for doing the experiment was based on several steps of induction that were ultimately wrong. This is known as “tooth fairy science”.

Coined by Harriet Hall of Skeptic Magazine and Science-Based Medicine fame, tooth fairy science refers to a perfectly executed experiment based on a false premise. If you assume that the tooth fairy exists, you can produce all kinds of research on tooth value, room orientation, where the child sleeps, etc. No matter what your findings, if the tooth fairy doesn’t exist, your research is moot. Also, just because the research was conducted properly and may even be very reproducible and statistically significant, it does not retroactively verify the premise – it doesn’t make the tooth fairy real (sorry to disappoint).

Remember when I wrote that induction is useful to create testable hypotheses? It is but you must test EACH STEP. One, just ONE, untested premise that is assumed to be accurate can bring down all of the following data. Remember the tooth fairy is not always so obvious as Sir Bedevere’s witch.

Sure, you can cluster findings to see if you can predict who will respond the best to spinal manipulations but you still don’t have any specific effects. How do you know whether or not you are merely enhancing the non-specific effects? Limited hip rotation means they will respond well to a spinal manipulation. Ok. Maybe earlobe length is also a predictor. Is that actually useful information or are we simply looking for a way to confirm our hypothesis that manual therapy is effective beyond placebo?

Like tooth fairy science, it skips a step. What if I created a similar CPR to find out who will respond the best to acupuncture or craniosacral therapy before I found a specific effect in which to test against? Would that make those scientific?

Don’t get me wrong, I like the intent and the authors have done a great job trying not to overstate things, I just question the sequence. Besides, when it was expanded to a larger and more diverse population, it didn’t do so well.

What about manual therapy and the neuroscience of pain? Ok, that might also be plausible. Now show me the experimental data that shows that manual therapy has a specific, consistent, measurable, and physiologic effect on that. All you have is pain and perceived function? Hmmm. Makes sense, could be right, but where is that experimental data? Can’t yet think of a way to falsify it? Well then – “No comment.”

Thank you for indulging me this series of posts. I hope that this has made you think about your application of Evidence Based Practice with a little more critical perspective.

Oh, and one other thing: Don’t just bitch about the APTA. Roll up your sleeves and volunteer or shut up. Your section, your chapter, and your national association all are in desperate need of your time and input and have plenty of opportunities for you to get involved. Make a difference in the game, not a complaint from the sidelines…