Is it Possible to Measure Physical Therapist Performance in Workers' Compensation?

How do you know if the physical therapists in your network are providing high-quality, cost-effective care? The simple answer is that it can be very difficult to determine.

The physical therapist (PT) plays a unique and vital role in the recovery of the injured worker. As an ancillary provider, they work with referrals from primary care doctors, orthopedic surgeons, or other specialists. In many systems that track outcomes, the overall responsibility for the claim is attributed to an entry-point provider or surgeon. This can make it challenging to separate the physical therapist’s clinical impact on a patient from the efforts of the primary care doctor.  

Further, the number of PT visits within a referral are usually pre-authorized. The referral only reaches the therapist after the physician requests care, and clinical management reviews the care for medical necessity and approves the request. Based on this, you could determine that the therapist is following a set of work orders. So how can we gauge the impact of their treatments on the injured workers’ recovery?

A Starting Point

Before beginning down this path, developing clear answers for why you want to evaluate PT performance is essential. Will it help your company’s product strategy, improve revenue, decrease medical expenses, or improve client retention? If you do decide it’s important, do you invest in developing assessment tools or look to access the information from another company? Either way, how will you apply the results?  

Which Dimensions of Performance Are Most Important?

Different stakeholders may have different priorities and expectations when evaluating care, and measuring the wrong dimensions can lead to the misalignment of goals. Identifying which dimensions to focus on is a critical step. These performance dimensions could include:

• How well the therapist retains the patient for the entire course of prescribed therapy. If the treatment cycle ends early, is it because of faster recovery or because a patient drops out of care?
• If a course of treatment needs an extension, is it clinically appropriate? Is there a pattern of requesting extensions across a wide range of injury types?
• How well do the therapist’s patients demonstrate functional improvement and/or pain reduction?
• Does the therapist utilize a logical balance of active treatment versus passive modalities appropriate for the nature and age of the injury?
• How well does the therapist accommodate patients with comorbid conditions or issues related to social determinants of health?
• Is the therapist easy to schedule (or reschedule) appointments with?
• Are their billing patterns appropriate?
• What is unique about your organization and how it interacts with the physical therapist community?

Capturing and Consuming Relevant Data

Asking questions is easy; operationally defining their answers is an entirely different matter. Ultimately, any form of provider performance assessment is directly dependent upon and limited to the variety, quantity, speed, and quality of data that can be gathered and organized.

In the early days of my career, my staff and I would spend a lot of time in “hunter-gatherer” mode trying to obtain, scrub, and conform business and clinically important data to answer critical business questions. It would take so long that there would often be little remaining time to conduct the actual analysis and create the deliverables. 

Although the development of data warehousing, on-demand analytic tools, data lakes, and data streaming has improved the situation, you must still ensure that any recurrent analytic system can access a steady flow of diverse, high-quality data. If underlying IT infrastructures are not in place, there will be additional resource expenditures that may not be immediately apparent. Last, to analyze any provider performance, the machine learning processes need to consume a lot of data (more on this point later).

Which Forms of Data Should Be Used?

One important source of information comes from the medical bill review vendor. Medical bill review data, properly groomed and conformed, can provide a wealth of insight into practice patterns, the detection of comorbid complications, the determination of pre/post-operative status, and a host of other valuable factors. Medical bill review data also furnishes a reliable window into billing patterns, enabling review of various “advanced bill review” edits that the provider regularly incurs or (among other things) the detection of potential fraud, waste, or abuse.

Clinical case management data can also be a gold mine of information about patient demographics, social determinants of health, comorbidities, attitude towards returning to work, and the injured worker’s history of interaction with case management.

Claim-level data is valuable for high-level details such as important dates, attorney involvement, and the type and duration of disability payments. Depending on the nature of your organization, however, claim-level data may be challenging to access routinely, and it can be surprisingly unstructured. In these instances, the clinical management data may be more valid and reliable. Further, accessing claim data across multiple employers, insurance carriers, and third-party claims administrators can be daunting since no two are alike. A great deal of cleaning and conforming may be necessary to yield consistent data that can be supplied to machine learning and other analytic processes.

Keep it Simple and Speak Clearly

The point may seem obvious but it needs to be said: clear, unambiguous terms must be used. This can be difficult because we are at the crossroads of medical nomenclatures such as ICD-10-CM diagnoses, procedural coding systems, and pharmaceutical coding (NDCs can be notoriously messy). We are also dealing with machine learning, IT terminology, and nebulous terms such as “quality,” “severity,” or “outcomes.”  

Vague terms can be confusing, lead to misguided expectations, and even circular logic. Here is an example. As a graduate student, I spent a great deal of time studying human intelligence. There was a tremendous body of research literature on the topic, but at the time there was no clear, universally accepted definition. The running joke was that intelligence was what intelligence tests measured. Beware of encountering similar situations in the world of workers’ compensation healthcare.  

Everyone has their own ideas about what a given term entails, but chances are good that no two experienced leaders will have identical definitions.

At Opyn, we work hard to avoid using vague terminology. Instead, we draw upon a wide range of analytic techniques to develop and hone a core set of metrics that address particular business and customer needs. In the case of measuring physical therapist performance, each metric is easily accessible, conceptually valid, and when used in combination with a matrix of additional measures, yields a robust and consistent depiction of how well a physical therapist performs with their patient load. 

We also know how easy they are to work with on tasks such as scheduling, billing, and other practicalities of effective ancillary care delivery. Transparency trumps buzzwords.

Make it Reliable, Valid, and Fair

Armed with clear operational definitions for key metrics and dependable sources of data capture, true analytic work can begin. Much has been written about machine learning tools such as k-means clustering, decision trees, naïve Bayes, and deep learning. No matter which techniques and methodologies are employed, any resulting provider assessments, whether peer comparisons, expected-to-actual metrics, or the like, must be statistically reliable, valid and, above all, fair.   

How will you adjust a provider’s scoring, for example, if a series of a provider’s cases involved comorbid type 2 diabetes? How do you compare two providers where one is in an affluent suburban setting and the other in a small farming town? How do you account for differing proportions of therapies between clinics, such as occupational, physical, hand, or aquatic?

I mentioned earlier that you need a lot of data. There are many reasons for this, but here’s one I’ve had to explain many times throughout my career. Take any large data set and divide it across numerous jurisdictions. Now, divide it further into regions, standard metropolitan (or, in many cases, micropolitan) statistical areas. Now, aggregate the patient data by the actual physical therapist. In many instances, a given PT clinic will not have enough patient volume to assess them.   

With any systematic form of provider assessment, a large number can quickly degenerate into thousands of minuscule ones. It is as if you took a hammer and cracked a giant geode, only to see it fracture into hundreds of tiny, shimmering crystals. In every outcome, fraud, prescribing pattern, or provider performance system I have developed throughout my career, this is the point where an engineering compromise has had to be made.  

One needs to have a certain minimum volume of patients per provider to generate statistically reliable measures. Set the threshold too high, and you end up excluding too many providers and wasting a lot of valuable, hard-earned data. Set it too low, and you can kiss reliability and validity goodbye. There are no easy answers here. You must drive a hard bargain with reality, and every system must be fine-tuned to strike a reasonable,  iterative balance between the number of providers assessed, good stewardship of all available data, and producing meaningful, fair, and trustworthy results.   

Fielding Questions from Providers

A thoughtfully constructed, well-engineered system can be a benefit to both the payor and the provider. The ability to confidently direct injured workers to a set of physical therapists whose past performance indicates high-quality, cost-effective care cannot be understated. So far, so good.  

Now, put yourself in the provider’s shoes. Any form of assessment shared externally will draw questions, particularly from those who were not rated very highly or whose ratings have decreased over time. Think before you rank. Determine in advance who in your organization will field these inquiries and what kind of on-demand tools they will need. How the results will be explained? Can the metrics be appealed, and if so, how? What kinds of detailed data is your organization willing to share? Without bogging down into an extended review, ensure that any related legal issues are appropriately addressed. Properly managed, a provider’s inquiry can be didactic, giving them clear, meaningful, and actionable feedback on their performance.

A Favorable Perspective

Throughout my career, I have been involved with numerous systems that assessed providers, tracked prescribing patterns, looked for potential fraud, or tried to find “the best of the best.” In the real world, well over 90% of providers -- no matter which specialty – are good, hard-working, competent people who care deeply for their patients. This statement cannot be overemphasized. Any experienced analyst, whether in property and casualty, commercial, or other forms of healthcare finance, will tell you this. The value of an assessment system, then, is to tip you off to either those who are truly excellent or, conversely, those who may require attention.

Therefore, you would be wise to question any system that results in a high number of non-optimal provider ratings. Something is wrong. Make the necessary course corrections and take a better approach. Given our unique market-driven processes at Opyn, we view our providers as valued customers. In borderline situations, our provider assessment system is designed to default favorably, which is a recommendation we would make to others.

‍

Now it is Your Turn

Given the varying perspectives on physical medicine-related metrics, it’s safe to say that quality is in the eye of the beholder. We know that what can be measured depends on the variety, volume, and utility of data resources. It also depends upon clinical expertise and no small degree of analytic panache. It must withstand the scrutiny of your clients and your provider community. Easy, right? For your organization, the “correct” solution must also support your objectives, strategies, and practical applications. All these factors will influence the direction you need to take.

Working with an ancillary partner who gives you the needed data to use in your own systems and strategies, tailored to your own definition of quality can be the easiest path forward. For example, at Opyn, our clients customize a marketplace that schedules the right ancillary care and delivers robust on-demand reporting and analytics tools that detail program and provider performance. With dashboards and key performance metrics, end users can drill down into the provider details with full transparency. Simply put, they’re in the driver’s seat to make their own actionable decisions and customize their provider panel to help them achieve their desired outcomes.   

If you have the data and analytics expertise and advanced IT resources needed, developing your own performance assessment system may be most beneficial. When building, it will be imperative that you have clear goals, adequate funding, and strong project management to ensure your project’s success. You’ll also need to remember that assessment and data capture will be an ongoing, iterative task once built. If you can commit to this, having your own tool will allow you to adapt and refine it over time as your organization’s needs evolve.

Either way, with PT playing a growing role in workers’ compensation medical care, determining how best to measure performance will remain an ongoing quest. The various dynamics of today’s post-covid workplace and the curious multi-generation composition of the American workforce may tend to push for more chronic injuries with more patients with more complications and comorbidities. It is, therefore, essential to learn all you can about the providers who play a role in making your organization thrive, if you view them collectively as a prized asset (as you should), and how to manage your organization’s relationship with them.

Greg Mast is the Senior Vice President of Data and Analytics at Opyn Market^®. With over 30 years of experience in group health and workers’ compensation, he is passionate about using his deep expertise to pioneer new uses for existing data, develop cutting-edge tools, and convert data into actionable insights. 

‍