Success of Evidence-Based Programmes Depends on Fidelity

If you want a successful EBP, don’t improvise

Congratulations. You have made the choice to implement an evidence-based programme (EBP) in your community. It’s a wise and fiscally responsible choice since only EBPs deliver proven and predictable outcomes.

But here’s the rub: To get these results, it’s critical to implement the program with fidelity. Too often, “fidelity” might be seen an overrated and undervalued expense. Sure, sticking to rules and guidelines developed somewhere far away, typically in some university petri dish, is hard. 

“What do they know about kids in our jurisdiction?” “It’s our money, we can spend it as we please,” and “We got the idea of Multisystemic Therapy (MST), now we can do it our own way” are just some of the sentiments I’ve heard expressed since Connecticut began adopting of EBPs 13 years ago. And it isn’t just funders who sometimes forget the value of adherence, it’s the providers, too. How many bids have I reviewed from well-intentioned prospective vendors who offer “XYZ-light” as a cost-savings alternative to those costly protocols imposed by model architects from afar?

Customize a car, not an EBP

Of course, the pull to “customize” an evidence-based program is strong. But when we don’t follow the rules of the model, we are no longer buying the empirical data that gains a model like MST its reputation for success. In that case, it is no longer evidence-based at all. A few examples come to mind.

• Hiring less credentialed (and thus less costly) therapists
• Minimizing putting parents and caregivers at the center of the treatment
• Shortening the duration of treatment or lengthening services by adding additional and formal requirements post-discharge
• Referring clients who don’t quite match the eligibility criteria.

In real-life application, these may seem like insignificant adaptations and ones that make the square plugs better fit the round holes that are the juvenile-court culture, judges’ perceptions, probations’ perspectives, families’ expectations or funding realities. And the presumption that something is better than nothing prevails. But let’s imagine the service need is medical instead of behavioral.

Each of the real-life examples of suggested changes to the MST model can be likened to a medical situation, providing a powerful message. For example, as for credentialing, no one would chose a healthcare provider who intends to study medicine, but hadn’t quite finished the coursework. And treating a juvenile away from his caregivers would be like adding exercise, but not addressing an overweight heart patient’s diet. It would be absurd to address one and not the other, and we’d think the doctor who suggested otherwise was a quack. Nor would anyone knowingly over or undertreat a problem. One would no more try to eradicate cancer with chemotherapy-light than they would choose that invasive treatment for the common cold. And we don’t keep taking antibiotics long after our infection has gone, any more than we prematurely or abruptly stop taking our medicine before the condition is ameliorated. That these are givens with our individual physical health, but not assumed to be just as true when treating family or societal ills is baffling to me. 

So, my advice to systems—if you are going to invest in an evidence-based program, don’t squander your investment by veering from the proven track. Stay the course, even when fidelity is a tough pill to swallow. That’s what gets results.