Pay for performance has captured the fancy of policymakers in the U.S., U.K. and Australia, which have integrated financial incentives or disincentives into national healthcare programs. But these efforts may be based on weak evidence, according to an analysis published online Aug. 14 in BMJ. The authors provided a checklist to guide policy makers past pitfalls while an editorial dissected the assumptions behind the pay-for-performance concept.
The U.K. launched its Quality and Outcomes Framework in 2004, which set up financial rewards for primary care physicians who achieve up to 147 performance targets. Australia offers a Medicare Practices Incentives Program that includes 11 incentives. In the U.S., the Centers for Medicare & Medicaid Services has taken a stick rather than a carrot approach with a program that penalizes hospitals with higher-than-expected readmission rates for heart failure, acute MI and pneumonia beginning in fiscal year 2013.
Paul P. Glasziou, PhD, of the Centre for Research in Evidence-Based Practice at Bond University in Queensland, Australia, and colleagues reviewed evidence on the effects of financial incentives for physicians for their analysis. Based on the review, they created a checklist for policymakers to determine if a pay-for-performance program was feasible.
“Policymakers have recognised the uncertainties and downsides of financial incentives, reflected by the large scale evaluations of the major UK and US programmes that have contributed to the evidence behind our checklist,” Glasziou et al wrote. “However, a decision to implement an incentive should include a critical assessment beforehand. Our checklist could help in that assessment.”
Evidence of the effectiveness of pay for performance on quality and outcomes is spotty, they summarized, highlighted by an article in the Cochrane Reviews that concluded what was known was insufficient to support or refute its use in healthcare systems. Studies have failed to adequately address the complexity of health behaviors, size of incentives, methods of payment, group rewards, organizational environments and cost-effectiveness. “Finally, and most crucially, most studies gathered few data on potential unintended consequences, such as attention shift, gaming and loss of motivation,” they wrote.
They devised a checklist based on six questions to determine if financial incentives were appropriate and three questions to consider the design of a program. Answering no to any of the first six suggested a program would be premature.
The nine questions were:
- Does the desired clinical action improve patient outcomes?
- Will undesirable clinical behaviors persist without intervention?
- Are there valid, reliable, and practical measures of the desired clinical behavior?
- Have the barriers and enablers to improving clinical behavior been assessed?
- Will financial incentives work, and better than other interventions to change behavior, and why?
- Will benefits clearly outweigh any unintended harmful effects, and at an acceptable cost?
- Are systems and structures needed for the change in place?
- How much should be paid, to whom, and for how long?
- How will the financial incentives be delivered?
Questions 1 through 3 should have a clear “yes,” they advised. Answering “no” or “unclear” indicates the need for pilot testing and other checks. Those who proceed anyhow should monitor and evaluate their programs. “[N]ew incentive programmes should include research to examine the impact, downsides, and cost effectiveness of incentives, and this should include evaluation of the comparative effectiveness of different strategies in different contexts,” Glasziou and colleagues wrote. “Such research should also include long term follow-up, since behaviour may revert when incentives are withdrawn.”
Steffie Woolhandler, MD, MPH, of the City University of New York School of Public Health in New York City, and colleagues took aim at pay for performance, arguing that it is based on flawed assumptions.
“One questionable assumption underlying pay for performance is that measurements of doctors’ performance reflect their overall performance and not—for example—their patients’ characteristics or their ability to ‘game’ the system,” they proposed.
Risk adjustments based on patient characteristic may be manipulated through coding and diagnostics, they argued. Patient characteristics also may confound some process-based measures.
They challenged the assumption