Why Program Fidelity is Critical
What is program fidelity?
At Adelphoi Innovative Solutions, we spend a lot of time thinking about program design and fidelity. We are not alone. Over the past decade, as the human service field has matured, research has advanced. And the capacity to collect and analyze data has improved. Almost everyone is thinking about fidelity more. Fidelity is not a difficult concept to understand. It is faithfulness to a cause or routine with a precise degree of exactness in its reproduction. Fidelity essentially tells you how, when, and why a program works. Preserving the components that make a practice effective can directly impact the success of desired outcomes.
Fidelity is not unique to our field.
Our field was not even the first to identify the importance of fidelity and incorporate it into its practices. Engineering, manufacturing, and farming are just a few of the disciplines that rely heavily on their ability to reproduce outcomes. The structure that fidelity provides encourages innovation and continuous quality improvement.
Our field has come to recognize the importance of fidelity’s role in helping us reproduce outcomes, but adoption presents challenges. Researchers note that observing, recording, and evaluating intervention encounters requires time and money. Further, many social and behavioral interventions are multifaceted. This creates barriers to evaluate the timing, quantity, and quality of an intervention as a whole. In addition, when interventions are complex and are made up of multiple components, it may be unclear as to which parts are critical to measure. And if they are, how to operationalize them.
I recently traveled to visit a program in another state (pre-COVID-19, of course). I was there to present to a group of executives about our FirstMatch model.
FirstMatch is a predictive analytics tool that uses a child’s clinical information and a provider’s historical outcomes to determine a match between a child in need of treatment and the most appropriate program for them.
During our conversation about their readiness to adopt such a model within their organization, they asked what the most important prerequisite was for a successful implementation. Ultimately, the answer is program and intervention fidelity. I explained that when programs operate with a high degree of fidelity for a sustained period of time, there are a number of benefits.
How and why is fidelity important?
One such benefit, and the most important to FirstMatch, is that fidelity allows the provider to learn which segment of their population has the most favorable response to their program. This is one of the guiding principles of the FirstMatch model. If you do not have fidelity, those that benefit from your program do so by accident, and without knowing why, you will not be able to reproduce it, unless again by accident. Without fidelity, you are unable to isolate any of the variables to understand their impact on outcomes.
When you have fidelity and are able to replicate the desired outcome, not only will you know why you can be poised to identify that opportunity in the future. FirstMatch operationalizes this logic and gives the provider a tool. This tool compares a referral to the population that has responded favorably to their program. Artificial intelligence, predictive analytics, and machine learning give us instruments to identify these variables and optimize the effectiveness of our programs.
Program fidelity is not only important to replicating outcomes, but it’s critical to understanding why we’ve succeeded and been able to build upon that success. Someone at the presentation joked, how did we know what our target population was before?
In our opinion
Too often, providers do not know what population is best served by their programs. This means they cannot understand why interventions work or don’t work with particular youth. This is why we see reentry rates and recidivism rates that haven’t improved despite significant investment in evidence-based programs. Too often, when programs outline their inclusionary and exclusionary criteria, they are based on perception, regulation, or preference. And not data or research.
Our system should be proud of the work it has done to improve the quality and mix of programming available to those in need of our help. There is more on the menu than ever before. Unfortunately, the system hasn’t spent the time, energy, effort, or money that’s necessary to maximize the benefits these programs have to offer. Arguably when the system has more programs and services to choose from, the likelihood of choosing the wrong service or program increases.
In conclusion
There are more programs available than ever before. It’s critical that we do a better job of matching programs with the populations of kids that they excel at working with. System improvement requires both quality programming and matching a child with a program that data demonstrates is the right program for them. FirstMatch leverages consistent outcomes through program fidelity to help us solve the complicated success equation that the kids and families involved in this system need us to solve.
