How can providers make evidence-based practices work for them, once brought into their own communities? Professional researchers give advice on what constitutes good research and how implementation challenges can be overcome.
There is little doubt that evidence-based practices are becoming essential to the work LeadingAge members do. These are practices with scientific data to back up their claims to success.
“Evidence-based practices need to become part of the fabric of an operation, just like continuous quality improvement,” says Robyn Stone, senior vice president for research at LeadingAge and co-director of the LeadingAge LTSS Center @UMass Boston. “But it’s not always clear, especially among the floods of information from the internet, what is evidence-based.”
The Advantages of Evidence-Based Practices
Quality improvement and cost reductions, patient satisfaction, and a desire for science-based outcomes are all factors in the increasing interest in evidence-based practices. The federal government is pushing providers to think more about the use of rigorous, scientific analysis-based programs for their clients and residents. “Members are progressing from anecdotes and stories to an understanding of outcomes measurements, and how to adopt practices with solid data behind them,” says Stone.
Verena Cimarolli, senior health services research associate at the LeadingAge LTSS Center, believes evidence-based standards should be part of the code of conduct for providers. “These are programs that help our clients and patients the most. It’s unethical to try out a new psychology or social work program if studies have shown it doesn’t work.”
Finding Evidence-Based Practices: a Wealth of Sources
Many websites are specific to evidence-based care and are likely to be more effective than a general Google search. (Scroll down for the sidebar, Sources for Evidence-Based Programs.)
“Health and Human Services is worth a search for sure,” says Marc Cohen, co-director of the LeadingAge LTSS Center. “The Administration for Community Living offers special programs and success stories.” He also thinks the Home and Community Based Services (HCBS) conference offered by the National Association of States United for Aging and Disabilities would be especially useful for LeadingAge members.
Vincent Mor, professor of health services, policy and practice in the Brown University School of Public Health, suggested the Center for Long-Term Care Quality & Innovation (Q&I Center) at Brown. “It’s designed,” says Mor, a principal investigator at the Center, “to link researchers with providers seeking evidence-based interventions, particularly in post-acute and long-term care, with a focus on controlled and experimental settings.”
Identifying Worthwhile Evidence-Based Programs
What steps can providers take to be confident that the research and interventions they choose are worth pursuing?
“Evaluation of a program is worth all the time it takes,” says Len Fishman, director of the Gerontology Institute and clinical professor at the McCormack Graduate School, UMass Boston. “But random controlled trials, while fundamental to medical and clinical research, are less likely to be a part of the evidence base for improving nursing home culture, employee satisfaction, or better resident outcomes. Instead, members should look for the best evidence available. This includes evaluations like the reputation of those who developed the program and recommendations by other adopters.”
“Make sure that you are not just reading some great story,” says Stone. “When considering a new evidence-based practice, look for the original research articles, which should be listed on the program site.”
“Seek programs with a population similar to yours,” says Cimarolli. “Where have they already been replicated? Can you find 2, 3 or 4 studies with similar outcomes? Do the costs and feasibility match your own conditions?”
Cimarolli and others said that conversations with other providers can be very helpful. She thinks relationships with medical centers could be wonderful partnerships for finding programs and for monitoring and evaluating outcomes.
Similarly, Mor suggests going online to find researchers who have published actively about their interventions and reach out to them for information and guidance. “I guarantee you will be embraced with open arms,” he says.
“Even if you are not a trained researcher,” says Cimarolli, “a literature search can be helpful. Google Scholar, for example, can point you to review articles, in which someone has synthesized numerous interventions.
Cimarolli also recommends a visit to Cochrane Reviews, a nonprofit organization that organizes research findings to facilitate evidence-based choices.
Sources for Evidence-Based Programs
Websites for health care research and programs:
- LeadingAge LTSS Center@UMassBoston
- Administration for Community Living
- Agency for Healthcare Research and Quality
- Centers for Disease Control
- Center for Long-Term Care Quality & Innovation (Q&I Center) at Brown University
- Center for Workforce Solutions at LeadingAge
- Center for Housing Plus Services at LeadingAge
- Centers for Medicare and Medicaid
- Health and Human Services
Other internet sources:
- Google and Google Scholar
- Cochrane Reviews
Conferences:
Journals for evaluating evidence-based programs:
- The Gerontologist
- Health Affairs
- The Journal of Aging and Social Policy
- The Journal of Aging and Health
- The Journal of Gerontology
- Research on Aging
Implementing Evidence-Based Practices
“LeadingAge members are part of a sector where it’s important to invest resources with considerable care,” says Stone, and “implementing practices in the real world is inevitably a challenge.” Sometimes this is referred to as scaling up, in which a successful small-scale experiment is applied to a larger population in a somewhat different setting.
The new field of implementation science is working toward an understanding of how to successfully apply practices to new situations. But it’s a very new field, and LeadingAge members and other providers will need to accept that implementation may not be precisely scientific.
“It seems so simple,” says Fishman. “If a new practice is working in one place, why would it not work in another? But there are many confounding factors, like level of turnover, how jobs are organized, and even how conditions change from one building to the next. The human factor is always there. And the implementation process is critical. You have to think many steps ahead. You want everyone on board and given a chance to be heard, both before and after the intervention. It’s hard to appreciate how complex systems can be even in a small organization.”
Even if you don’t have a large enough population to assign control groups for evaluation, you can learn from careful assessment, according to Mor: “Figure out what outcomes you want. Make sure you collect your data very carefully, and review it every 3 months.” He also suggested that a pilot program can be a good way to work up to a larger scale. It’s a lower-cost way to discover how changing the basic elements of a practice, such as patient population or staff training, will affect outcomes.
Successful implementation should include setting specific goals, according to Cohen. “Especially when you are investing staff resources and money you should understand what return on investment you are hoping for. This may not even be dollars, but simply a positive result in the form of family or patient or staff satisfaction.”
What if your new program seems not to be working? Fishman suggests taking a look at the foundations you’ve laid. Start with your team.
“Have they had time to study the program in advance and raise their questions? Do they think your setting is really comparable? There are times when knowledge within a team is simply not surfacing,” Fishman notes. “Make sure that staff are comfortable sharing their knowledge about why something isn’t working.”
Or, consult with someone who has replicated the same program successfully. Fishman says people in the nonprofit sphere are happy to help and to compare notes about similar difficulties.
Where to Go for Support
When a new program involves significant costs, says Cimarolli, a foundation is often a good source of funding. She noted that most foundations want a program that can be sustained once the funding ends.
Cohen agrees that a grant application is a helpful way to begin the process of investment. “Foundations that may be interested in funding new models of care include the Commonwealth Fund, the Robert Wood Johnson Foundation and the Colorado Health Foundation, to name but a few.”
“Graduate programs in gerontology and related fields may also have students interested in helping out,” says Fishman.
“A lot of universities, especially state universities, are interested in partnering with providers. And even business schools with implementation science programs may be interested in some form of partnership.”
When making the case for funding, according to Cohen, there are various metrics to focus on, and not all of them need be measured in hard dollars. For example, a reputation for strong, evidence-based programs such as fall prevention may help to attract new members as well as increasing patient satisfaction. Or, a reduction in hospital visits may not only reduce costs but improve the quality of care, something particularly important to mission-driven organizations.
LeadingAge Members’ Experience With Evidence-Based Programs
Michael Slauter, director of Marguerite’s House Assisted Living in Lawrence, MA, says “To find good programs, we contact other LeadingAge members, or attend LeadingAge roundtables, or use the internet. And we respond to corporate guidance from our parent organization, Covenant Health.”
Marguerite’s House’s current “Gentle Exercise” program started in 2014, building on a new quality improvement metric from Covenant Health.
“The evidence of several studies showed that a minimum of twice-weekly exercise and targeted fall reduction programs can be an effective way to improve strength, mobility and balance and reduce injuries sustained by falls,” says Slauter. Marguerite’s House is currently tracking participation against rates of falls, compared to nonparticipants, and is pleased with the results to date.
The organization’s earlier experience with a fall prevention program illustrates some of the possible challenges of adopting even an established evidence-based program.
“One challenge was to keep residents engaged and healthy enough for long-term participation,” says Slauter. He thinks that the varied age and frailty of residents had much to do with the loss of interest in the program.
“The use of evidence-based research is in our DNA,” says Tena Alonzo, director of education and research at Beatitudes Campus. “Improving quality of life is a matter of justice for those who may not be able to tell us what they need.”
Beatitudes Campus is a life plan community in Phoenix, AZ, serving about 700 people. Its Comfort Matters dementia program started 20 years ago. “During our journey, we developed a strong commitment to evidence-based practice,” says Alonzo.
She says that finding evidence-based programs calls for commitment and a lot of homework: “When you find what looks good, contact the program and see how they answer your questions. What articles have they published, what are the outcomes, are they meeting their needs? Is the program continuing to grow?”
Alonzo and Barbara Wood, Beatitudes’ director of development, agree that implementation can be full of challenges.
“Estimate the time it may take to implement the program, and be aware that any changes will take longer than you expect,” says Alonzo. “And remember that change is hard and prepare your team—have everything in place. At the same time, get everyone on board, making sure they understand why the program is needed.”
Beatitudes campus has received over $1 million in grants and philanthropic support for Comfort Matters, according to Wood, who emphasizes the availability of support for evidence-based practices. Like Cohen and Fishman, she thinks many local universities are looking for ways to support innovation in long-term care.
The Replication Crisis: What Is It?
For any research conclusion to be scientifically valid, it must be reproducible. The idea is that researchers should be able to reproduce the experiment and get the same results. But reproduction of results, or replicability, as it is also called, is not a given. For a wide variety of reasons an experiment may simply fail when tried again. A 2015 article in the journal Nature reported survey results showing that 90% of respondents perceived either a slight or significant crisis in replication, especially in the fields of medicine and social psychology.
Should LeadingAge members be worried about whether they can replicate an evidence-based intervention, no matter how impressive its reported results?
Marc Cohen, co-director of the LeadingAge LTSS Center @UMass Boston, does not see a crisis in replication in the field of long-term services and supports, although he says that careful consideration needs to go into all attempts to take a model in one setting and replicate it elsewhere. Len Fishman, director of the Gerontology Institute and clinical professor at the McCormack Graduate School, UMass Boston, emphasized that LeadingAge providers are at work in the real world, not in a laboratory. Fishman and Cohen agree that goal-setting and careful planning are essential.
“Providers need to ask foundational questions about whether a program will yield similar, useful results across multiple sites and settings,” says Cohen.
See also these articles (all downloaded August 31, 2018):
- Baker, Monya, “Is There a Reproducibility Crisis?” Nature, Vol. 533, May 26, 2016.
- Marcus, Gary, “Science and Its Sceptics,” The New Yorker, November 6, 2013.
- Van Bavel, Jay, “Why Do So Many Studies Fail to Replicate?” The New York Times, May 27, 2016.
- Wikipedia article on the replication crisis.