Reporting and utilization of Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures in orthopedic research and practice: a systematic review

Background The Patient-Reported Outcomes Measurement Information SystemⓇ (PROMISⓇ) is a dynamic system of psychometrically sound patient-reported outcome (PRO) measures. There has been a recent increase in the use of PROMIS measures, yet little has been written about the reporting of these measures in the field of orthopedics. The purpose of this study was to conduct a systematic review to determine the uptake of PROMIS measures across orthopedics and to identify the type of PROMIS measures and domains that are most commonly used in orthopedic research and practice. Methods We searched PubMed, Embase, and Scopus using keywords and database-specific subject headings to capture orthopedic studies reporting PROMIS measures through November 2018. Our inclusion criteria were use of PROMIS measures as an outcome or used to describe a population of patients in an orthopedic setting in patients ≥ 18 years of age. We excluded non-quantitative studies, reviews, and case reports. Results Our final search yielded 88 studies published from 2013 through 2018, with 57% (50 studies) published in 2018 alone. By body region, 28% (25 studies) reported PROMIS measures in the upper extremity (shoulder, elbow, hand), 36% (32 studies) reported PROMIS measures in the lower extremity (hip, knee, ankle, foot), 19% (17 studies) reported PROMIS measures in the spine, 10% (9 studies) reported PROMIS measures in trauma patients, and 6% (5 studies) reported PROMIS measures in general orthopedic patients. The majority of studies reported between one and three PROMIS domains (82%, 73 studies). The PROMIS Computerized Adaptive Test (CAT) approach was most commonly used (81%, 72 studies). The most frequently reported PROMIS domains were physical function (81%, 71 studies) and pain interference (61%, 54 studies). Conclusion Our review found an increase in the reporting of PROMIS measures over the recent years. Utilization of PROMIS measures in orthopedic populations is clinically appropriate and can facilitate communication of outcomes across different provider types and with reduced respondent burden. Registration The protocol for this systematic review was designed in accordance with the PRISMA guidelines and is registered with the PROSPERO database (CRD42018088260). Supplementary Information The online version contains supplementary material available at 10.1186/s13018-020-02068-9.


Introduction
In order to determine if a patient has achieved a meaningful outcome, it is insufficient to evaluate treatment results solely on medical history, physical findings, laboratory tests, or imaging findings [1]. Patient-reported outcome (PRO) measures are a useful tool to quantify and communicate a patient's health status to healthcare providers that directly incorporates the patient's voice. Change in PROs can be one of the measures of "success" from a patient's perspective after an orthopedic procedure [2]. PROs are increasingly being used as part of the clinical encounter to guide treatment decisions and determine the effectiveness of interventions [3], but PROs have presented challenges with implementation and measure selection.
In orthopedic practice and research, there is great variability in the number of PRO measures available. As a result, there is confusion among orthopedic providers about which PRO measure is most appropriate given a patient population and how to appropriately interpret a patient's score to enhance treatment recommendations. Subsequently, in orthopedics, there has been a recent increase in the adoption of a universally accepted set of PRO measures: the Patient-Reported Outcomes Measurement Information System® (PROMIS®). PROMIS has been compared against conventional general health and disease-specific PRO measures and regularly has been found to improve coverage of the relevant health domain, increase reliability, and reduce respondent burden [4].
PROMIS measures were developed with support by the National Institutes of Health (NIH) as an effort to address the need for more valid, reliable, and generalizable measures of clinical outcomes that are important to patients [5]. PROMIS is a set of psychometrically sound measures to assess a patient's physical, mental, and social health across multiple conditions or diseases, including orthopedic conditions. PROMIS measures overcome the limitations of traditional PRO measures used in orthopedic research and practice by scoring all PROMIS domains using a common metric of a T-score that is normalized to the U.S. general population. PROMIS provides access to both fixed-length measures (e.g., 6-item measure of fatigue) and computerized adaptive testing (CAT) that tailors the measure for each individual to allow for efficient assessment when response burden is of concern [6].
In recent years, a proliferation of studies have reported the association of PROMIS measures with traditional measures and have demonstrated the reliability and performance of PROMIS measures in orthopedic populations. While there have been a few systematic reviews about the use of PROMIS measures in certain disciplines within orthopedics [7][8][9][10], these reviews do not describe how the measures have been reported neither in the literature nor the general uptake of PROMIS measures within orthopedic research and practice. Thus, we sought to evaluate the adoption of PROMIS measures in orthopedics by describing how the measures are used and reported on, including the PROMIS domains evaluated, the type of PROMIS instrument used, and other traditional measures that were reported along with PRO-MIS measures.

Review design
The protocol for this systematic review was designed in accordance with the PRISMA guidelines [11] and is registered with the PROSPERO database (CRD42018088260) [12]. We collaborated with a research librarian (LL) to develop an appropriate search strategy and management of the literature review.

Data sources and search strategy
We performed a literature search of PubMed, Embase, and Scopus from inception to November 4, 2018, using a combination of keywords and database-specific subject headings to capture studies done in an orthopedic setting and/or procedures that reported a PROMIS measure as an outcome (Additional file 1). We added search filters to exclude case studies or reports, editorials, letters to the editor, and studies not written in English.

Inclusion and exclusion criteria
Inclusion criteria included the use of PROMIS measures in studies conducted in orthopedic settings for clinical care purposes or studies that used PROMIS measures to assess an outcome from an orthopedic intervention. Our exclusion criteria were study population < 18 years of age; non-orthopedic interventions, settings, or providers performing the intervention; and qualitative studies, commentaries, or systematic reviews. All included studies were peer-reviewed, reported at least one PROMIS measure, and used an experimental, quasi-experimental, or observational design. Two authors screened articles (MH and SZG) and a third author (ER) resolved any conflicts.

Study selection and data extraction
After databases were searched, titles and abstracts of studies were uploaded into Covidence, a systematic review management software [13]. The article selection process was done in two phases. In the first phase, two authors (MH and SZG) performed independent reviews of titles and abstracts in Covidence using the predefined inclusion and exclusion criteria. Articles were moved to full-text review if one or both authors found the article potentially relevant. In the second phase, the same two authors independently reviewed full-text articles for eligibility. Any conflicts were resolved by the third author.

Data analysis
Included studies were evaluated from November 2018 to June 2020. The primary purpose of this review was to describe the uptake of PROMIS measures in orthopedic research and practice through qualitative synthesis, and then rate the quality of included studies. Therefore, we did not perform a meta-analysis of data. For the qualitative synthesis, we described the studies by publication year, clinical population, study type, and sample size. We evaluated the reporting of PROMIS measures by recording the PRO domains reported in each study and the type of PROMIS measures used (i.e., domain-specific fixed short forms, multiple domain profile short forms, or CAT). Last, we described the frequency in which PROMIS measures were reported alongside traditional measures by the clinical population. Traditional measures are non-PROMIS established measures used in orthopedics.

Quality assessment
We used the Newcastle-Ottawa Scale (NOS) to assess the quality of included studies (Additional file 2). Because this review included a heterogeneous group of studies with a wide variety of methodologies, there is likely no single risk of bias tool to perfectly evaluate study quality across such a diverse group. The NOS was developed to assess the quality of nonrandomized studies, and evaluates studies within three domains: the selection of study groups, the comparability between these groups, and the determination of the outcome of interest. We used a version of the NOS specifically adapted for cross-sectional studies [14] and for case control and cohort studies [15]. The NOS scoring of seven or more stars is generally considered high quality, though no ranges have been officially reported in the literature [16].

Results
Our preliminary search yielded 1046 citations, and after duplicates were removed, 513 citations were reviewed by their titles and abstracts. Of those, 376 were moved forward to the full-text review stage, and 88 articles remained for inclusion in the systematic review [3, (Fig. 1). After conflicts were resolved by the third author, we calculated an 81.6% agreement between the authors performing full-text review. Table 1 shows the characteristics of included studies by year, clinical population, study type, and sample size.

Study type and sample size
The studies in this review varied in the study design used to assess outcomes. The largest percentage of studies were cohort studies (59%, 52 studies). Most of these were prospective observational designs (38%, 33 studies), and 22% (19 studies) were retrospective observational designs. Many studies (41%, 36 studies) used a crosssectional study design to analyze the psychometric properties of PROMIS or to validate in a patient population. No randomized controlled trials were reported using PROMIS measures as an outcome measure. Sample sizes in the studies ranged from 11 patients to 14,679 patients, with 133 patients as the median number reported. Five studies included patients from registries including the American Orthopedic Foot and Ankle Society's National Orthopedic Foot and Ankle Research Outcomes Network and the Maryland Orthopedic Registry.

PROMIS and traditional PROs
Fourteen studies in this review reported PROMIS as the sole outcome measure. Of those 14 studies, 9 were published in 2018 alone. Widely reported traditional measures were reported alongside PROMIS measures in all studies. Traditional measures included measuring the constructs of pain, disability, psychosocial comorbidity, and quality of life. Table 3 describes the reporting of traditional measures alongside PROMIS measures by body region.

Quality of studies and risk of bias
A majority of studies assessed had a low risk of bias. All cohort and cross-sectional studies scored seven or above in their respective versions of the NOS quality assessment tool, and, with one exception, all case-control studies scored eight or above. Table 4 describes the risk of bias summary for individual studies included in this review, and Additional file 2 contains detailed results of the quality assessment.

Discussion
In this review, we evaluated the uptake of PROMIS measures in orthopedic research and practice by describing   Our review also found that the use of PROMIS measures across clinical populations varied, with 37% of studies examining lower extremity conditions, followed by upper extremity (28%) and spine conditions (19%). This finding is consistent with the supporting literature where the use of PROMIS measures in lower, upper, and spine is increasing as a primary measure across clinical populations [1,4,108]. Last, most studies in our review reported the use of CAT-based assessments as the PRO-MIS assessment type. This finding is not surprising, as the primary benefits of the PROMIS CAT measures are the decrease in patient burden and the precision of the estimate. The majority of studies reported between one and three PROMIS domains. Unsurprisingly, the most commonly reported PROMIS domains were physical function and pain interference, which are validated and compared to many traditional measures. Of the psychological domains, depression was reported more frequently than anxiety. While the field of orthopedics is focused on improved functioning and reduced pain, we would encourage a more holistic view of the patient by incorporating more psychological constructs that may affect patient prognosis. This review provides evidence that the prevalence and support for use of PROMIS measures is growing in orthopedics and that PROMIS is being recognized as a PRO measure of choice for clinical trials [109].

Limitations
Our systematic review has some limitations. First, we aimed to describe the prevalence and use of PROMIS measures within orthopedic practice and research rather than to compare outcomes or exposures in the studies. Our review had broad inclusion criteria, and thus there was high variability, with study designs often considered less rigorous. The majority of studies were retrospective and prospective cohort studies. No studies in our review were randomized clinical trials; however, this is likely because of the relative unavailability of PROMIS measures until recently. It will take some time before clinical trials that use PROMIS measures as endpoints are published.
Second, we reported on the PROMIS domains but did not perform meta-analyses to examine the effects of treatment or compare the performance of PROMIS measures with other reported measures. Last, many studies    Given the evidence for the validity and reliability of PROMIS in orthopedics, we expect a decrease in the use of other established PRO measures in order to reduce respondent burden. The implications for future research and practice in orthopedics support that PROMIS measures are versatile, reliable, and valid for orthopedic research and practice. Further, PROMIS measures provide distinct advantages over traditional measures, particularly, when the study population is heterogeneous. Multiple recent studies indicate that widespread variability exists in the particular PROs used in studies of the same diagnosis, thereby significantly limiting the translatability of many of these high-impact studies [6,8,111,112]. Future research on the use of PROMIS measures in orthopedics should focus on the use of PROMIS measures as the primary outcome measure, particularly in studies that examine heterogeneous patient populations. Last, PRO-MIS measures hold immense potential for improving patient and provider communication, particularly across specialties.
Additional file 1: Search Strategies.
Additional file 2: Quality Assessment.