Learning From Experience

“Much learning does not teach understanding.”
-Heraclitus

The mind works like a human Bayesian reasoning system.

“Dammit! I’m not reading this! BAYES?! DON”T MAKE ME LEARN SOME KIND OF MATH!”

Easy, dear internet user. This isn’t all that complicated. All that Bayes’ Theorem really says is that you have previous beliefs and you interpret new information from that previous position. In other words:

What you already understand + new information = New position of understanding

Usually people explain Bayesianism with equations regarding prior probabilities and all that (I did that already) but here is a simple example:

• Most dogs do not bite therefore it is unlikely that this dog that is foaming at the mouth will bite me
• [Dog bites me]
• I now believe that dogs foaming at the mouth are more likely to bite me than dogs in general

Yeah, not rocket science here. I’m always confused when people act like Bayes is some sort of new magical thinking. It isn’t new and when you talk to a statistician they will point out that Bayes is merely one statistical tool, of which there are many others. Kind of like a hammer.

First, you wouldn’t use a hammer to drive a screw. Even when you want to drive a nail, you have to use the hammer properly. If you start swinging it sideways you’ll just bend the nail all to shit. You also shouldn’t start viciously swinging that hammer to force something to look like whatever it is you WANT it to look like (METAPHOR!!!!).

That said, you really should use the hammer

So how does Bayesian reasoning apply to learning? It’s simple. You start with what you already understand (current knowledge) then weigh how new data (experiences) may affect that. The problem here is that many forget that Bayesian reasoning requires that you weigh the quality of those two components accurately. Or as the statistician Charles Wheelen says:

“Most statistics books assume that you are using good data, just as a cookbook assumes that you are not buying rancid meat and rotten vegetables.”

If your prior understanding is based on little more than your experiences and your beliefs as opposed to hard calculated data, that by no means makes it unusable, but it does make the position weaker. MUCH weaker.

The one thing that Bayes does really well is help us approach new information. Ironically, it is also the one way in which I see Bayes being abused.

Using experience to gain knowledge

So how exactly do people learn through experience? On its face it really is as simple as the Bayesian inference above:

Old beliefs (or no beliefs) + new information = new position of belief

So let’s try this out and learn from experience!

You start seeing a bunch of patients with neck pain. You are a manual therapist so you start mobilizing and manipulating things. Feedback begins coming in from patients. From this feedback, you start to recognize that certain patients respond better to certain techniques. As a few years go by, you become pretty good identifying responders to all your different techniques, knowing when to poke here and when to press there.

You look at your outcomes and patient satisfaction and, well I’ll be damned, you are extremely successful! Patients are significantly better at the end of care than they were at the beginning of care and they are extremely satisfied. You have learned from you experience that what you are doing is working.

But is that actually what you are learning?

In this very common example, it is possible that we have an actual long term treatment effect. But we have the strong potential for two things to play off of each other to completely mislead our interpretation of the results: Short term anticipation and natural history/regression to the mean.

Short term anticipation

This is actually the true learning that occurred. As noted earlier, in order to learn we need “new information”. This would be the feedback in our scenario above. You did something to the patient, then you assessed the response. You then modified your process until you became really good at forecasting who would respond and in what way.

The problem here is that you are only getting feedback relating to the IMMEDIATE effects of your intervention. That means that the only learning that is happening is regarding those short-term effect. You are becoming really good at short term anticipation of what may be nothing more than clinical magic.

Natural history/regression to the mean

But what about those outcomes? Your patients were getting better were they not? Yeah, but that is the funny thing about non-specific pain: It either just runs its course and resolves or goes back to its average baseline status (regresses to the mean).

As you track these patients “in your experience”, you see short-term effects and long-term overall improvements as you do your “system” over and over, year after year. But those long-term improvements may have little to do with your interventions. Voltaire understood how this worked centuries ago:

“The art of medicine consists of amusing the patient while nature cures the disease.”
– Voltaire

What’s the harm?

Oh, I didn’t say there was any real harm.Some people feel that these treatments will foster permanent dependence of the patient on the therapist but I’m not sure that effect is very robust or common. But you may start ignoring population-based data and start focusing on your “experience” of these short term results, believing that you have unique knowledge of how this individual, this N=1 patient, will respond to your interventions.

I can say that it drives up healthcare costs and may give the patient (and society as a whole) a sense that these interventions are necessary. It may also mean that you are wasting people’s time and money on things that will provide little benefit while distracting them from the reality of their situation. A reality that may just need a little reassurance and some education.

But that’s no fun now is it?

Is it wrong to make patients better in the short term?

No, but is that what you spent all of those years in school for? Is that the reason you spent all of that money on continuing education courses and additional certifications? To get really good at identifying short term responders? Is that the skilled part of what you do?

And do you REALLY believe that is all you are doing or have you convinced yourself otherwise?

Seems like a waste of brain power. There are lots of other “less professional” providers that already do that sort of thing. And they cost a lot less. Let them get good at those skills through their own experience. Hell, who doesn’t feel better after a backrub or a visit to the spa? But does making something temporarily feel better make it medical?

In my opinion let non-medical services be provided by non-medical providers at non-medical prices

And that is assuming that your patient actually cares about you providing those short-term effects. “This will get better over time no matter what you do in the short-term to make it temporarily feel good,” can be a very powerful statement.

Appropriate reassurance is more powerful than we give it credit…

In summary…

• Stay away from dogs that are foaming at the mouth
• When you learn of a new tool, make sure you use it appropriately
• Learning from experience takes a lot of work
• Experience at short term anticipation and natural history/regression to the mean can make you think you know something that you actually don’t know
• Are you skilled finding short term responders or do you make a long term difference?

The featured image on this post is “She has a message!” by smerikal via Flicker.