Design

We conducted a type 3 hybrid study [14] combining evaluations of intervention effectiveness (e.g., impact on patient outcomes) and implementation. We used the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) planning and evaluation framework [15, 16]. Reporting follows the Standards for Reporting Implementation Studies (StaRi) [17].

Detailed intervention, implementation strategy, and evaluation methods are reported in a published protocol [18]. The Option Grid patient decision aid was rolled out one setting at a time according to a randomized stepped-wedge schedule. Relevant outcome data were collected throughout each setting’s pre-implementation and active implementation phases (Fig. 1).

Fig. 1Ethics approval and registration

The study was registered at ClinicalTrials.gov (NCT03985449) and approved by the Dartmouth College Committee for the Protection of Human Subjects (study #31464) and the Washington University Human Research Protection Office.

Intervention

The Option Grid patient decision aid for symptomatic uterine fibroids presents answers to frequently asked questions about available medical and procedural treatment options in a tabular format, available in English or Spanish, using text at a sixth-grade reading level. Adaptations made during the study led to two versions of the decision aid, covering medication and procedure-based options. Options included watch and wait, medicine with hormones, medicine without hormones, uterine artery embolization, endometrial ablation, radiofrequency ablation, myomectomy, and hysterectomy. We also created Picture Option Grid versions of the medication and procedural Option Grids, which include the same text supplemented by images that illustrate key information. An online, interactive version allowed users to select various options for comparison in a tailored Option Grid file downloaded to their computer.

Implementation strategy

The multi-component implementation strategy was guided by the Consolidated Framework for Implementation Research (CFIR) [19, 20]. The strategy comprised assessing organizational readiness for SDM at each site; providing access to the Option Grid in multiple modalities and formats, including access for clinicians through the electronic health record; conducting brief training for clinicians on SDM and how to use the decision aids; presenting monthly audit and feedback to each site’s clinical lead on reach of the decision aid; and planning for sustainability at each site (see Table 1) [18, 21]. Individual settings developed different methods to deliver the decision aids to patients (see Appendix 1 process maps). Throughout the project period, core project staff held weekly check-in meetings with site-based project team members to plan implementation processes and monitor progress at each site. Beyond the initial iterative planning discussions, these meetings provided regular opportunities for troubleshooting implementation-related issues over the course of the project. Clinician training sessions (synchronous, 1 h at each setting) were tailored to strengths, resources, and opportunities identified at each site through the Measure of Organizational Readiness for Patient Engagement [21].

Table 1 Multi-component implementation strategy bundleSettings

The five implementation settings were situated in academic medical centers. They included gynecology practices in: (1) a rural area of the Northeast United States (US); (2) an urban area of the Midwest; (3) an urban area of the Northeast; (4) an urban area of the Northeast focused on reproductive endocrinology and minimally invasive gynecologic surgery; and (5) a small city in the Midwest focused on minimally invasive gynecologic surgery.

Participants

Clinician participants included attending physicians, residents and fellows, nurse practitioners/midwives, and physician assistants who cared for patients with fibroids at the five gynecologic settings. We included patients of reproductive age (i.e. premenopausal) with new or recurrent symptoms of uterine fibroids (e.g., heavy menstrual bleeding, pelvic pressure, or pain) who were seeking treatment and met the following inclusion criteria: (1) assigned female sex at birth, (2) 18 + years of age, (3) spoke English or Spanish, and (4) could complete short surveys online independent of assistance. We did not exclude pregnant patients. We excluded postmenopausal patients because they are less likely to experience fibroid-related symptoms.

Outcomes

Table 2 summarizes study outcomes.

Table 2 RE-AIM Outcomes and analysisReach Measure

We evaluated reach as the total number of patients who received the Option Grid or Picture Option Grid. While reach is typically presented as a proportion, the non-specific symptoms associated with uterine fibroids (e.g., heavy bleeding, pain) posed challenges for systematic identification of eligible patients presenting for uterine fibroid evaluation and decision-making. Unable to fully rely on diagnosis codes or other reporting mechanisms within clinic scheduling software and electronic health records to estimate the total number of eligible patients, participating sites enlisted clinical, administrative, and/or research staff to conduct manual monthly tracking of eligible patients. These tracking methods had limited reliability due to the introduction of telehealth visits (in response to COVID-19 onset) early in the implementation phase and the limited involvement of clinical and administrative support staff in clinicians' telehealth workflows. We therefore chose not to present proportions as the likely undercounting of eligible patients inflates these reach estimates.

Data collection

Throughout each setting’s active implementation phase, we conducted monthly email and/or teleconference outreach to research staff employed at each setting to tally the total number of paper decision aids distributed to patients. Research staff determined usage by monitoring their setting’s visit schedule to identify eligible patients and tracking their inventory of decision aids. Uses of the online decision aids, accessed through the Epic electronic health record at each setting [18], were automatically tracked electronically by the publisher and reported to the study team each month. No honoraria or incentives were offered to clinicians, patients, or research staff to evaluate decision aid reach.

Analysis

We summed the total number of paper decision aids distributed and the number of times the electronic health record-based decision aids were accessed at each of the five settings.

Effectiveness Measures

We measured effectiveness using the collaboRATE patient-reported measure of SDM, adopting top-box scoring (i.e. a binary indicator of whether a respondent marked the highest possible score on all three items) [23,24,25].

Data collection

We collected electronic surveys from a convenience sample of eligible patients throughout pre-implementation and active implementation phases at two time points: (T1) immediately post-visit and (T2) three months after the visit. Surveys were hosted in Qualtrics software. Participants received a $20 gift card after completion of each survey. Before the onset of the COVID-19 pandemic in March 2020, research staff at each setting collected T1 surveys on-site by offering eligible patients the survey on a tablet computer. After the pandemic’s onset, survey data collection was paused for five months. Upon re-initiation in August 2020, surveys were collected through a combination of tablet computers in the settings and survey links sent to eligible patients via email or secure electronic message.

Analysis

Using mixed effects logistic regression, adjusting for patient characteristics as fixed effects and setting as a random effect, we conducted an as-treated analysis [26] of the association between patient-reported decision aid use and collaboRATE shared decision-making scores (i.e. comparing collaboRATE scores among those who reported receiving the decision aid vs. those who reported they did not receive the decision aid). Descriptive statistics compared intended versus selected treatments.

Adoption Measure

We defined adoption as the total number and proportion of eligible clinicians who agreed to use the decision aid with patients.

Data collection

Prior to the study’s pre-implementation phase, we tallied the number of eligible clinicians at each setting. We confirmed this number at the end of the study. At the beginning of each setting’s active implementation period, we conducted initiation visits to present the decision aids to the participating clinicians and provide brief training on their effective use. At these visits, we obtained written informed consent from clinicians who agreed to use the intervention. Throughout the study, as new clinicians were hired at the participating settings, we obtained their agreement to use the intervention. Initiation visits were conducted in person at setting 1 and, given COVID-19 travel restrictions, virtually at the other four settings.

Analysis

We calculated the proportion of eligible clinicians at each setting who agreed to use the intervention [15].

Implementation fidelity Measures

We collected patient-reported information on when and how the decision aid was delivered and used by the clinical team; we accepted multiple responses on these survey items, allowing for the possibility that a patient received the decision aid more than once. We also assessed the collaboRATE patient-reported measure of SDM top-box scores [23,24,25] and the Observer OPTION-5 measure of SDM [27].

Data collection

For patient-reported data, we collected Qualtrics electronic surveys from a convenience sample of eligible patients at two time points: (T1) immediately post-visit and (T2) three months after the visit. Full survey administration details are described under “Effectiveness” above.

Audio recordings were collected from a convenience sample of visits between eligible patients and clinicians, who had previously provided written informed consent. Research staff provided each participating patient with a handheld digital audio recorder and started the audio recorder as the visit was initiated. Two independent raters used the Observer OPTION-5 measure to assess the level of SDM observed during each audio recorded visit.

Analysis

We used intention-to-treat analysis [26] to evaluate setting-level scores on measures of SDM during the pre-implementation and implementation phases. We used mixed effects logistic regression adjusting for patient characteristics as fixed effects and setting as a random effect to evaluate the association between setting-level intervention implementation and collaboRATE SDM scores.

Using the mean of the two raters’ Observer OPTION-5 scores, we calculated descriptive statistics comparing scores in pre-implementation vs. implementation phases.

Factors contributing to maintenance Measures

We evaluated clinician attitudes about patient decision aids using the ADOPT measure, which asks participants to select from a list of ten adjectives that clinicians might use to describe patient decision aids – five positive and five negative [28]. ADOPT sum scores range from -5 to 5, with higher scores representing more positive attitudes toward patient decision aids. Survey measures of patients’ preferred methods and timing of decision aid delivery were also collected.

We planned for sustained decision aid use at each site using Schalock and colleagues’ systematic approach to sustainability, identifying organizational drivers focused on accountability, leadership, and process [22]. The Schalock et al. framework addresses factors that promote sustained quality improvement within human service organizations, including “maintaining sound outcomes, generating knowledge, building capacity, experiencing stable funding and staffing patterns, and providing value-based services and supports in an effective and efficient manner” [22].

Within the post-visit patient survey, we collected bespoke measures of patients’ preferred decision aid delivery modality (i.e., by mail, through the online patient portal, via a paper copy in the clinic, and/or viewed on a clinic computer) and timing (i.e., before, during, and/or after a visit).

Data collection

The principal investigators distributed Qualtrics survey invitations by email to all eligible clinicians at two time points: (1) the beginning of their employer’s active implementation phase and (2) the end of their employer’s active implementation phase. The Qualtrics-hosted survey included a consent form and the ADOPT measure.

During weekly site check-in meetings between core project staff and site-based project team members in year 3 of the 4-year project, we discussed key drivers and barriers of sustained decision aid use, brainstormed strategies to support sustainability, and took detailed field notes.

Patient survey measures: Full patient survey details are reported under “Effectiveness” above.

Analysis

We used descriptive statistics to compare mean ADOPT sum scores across all settings between the beginning vs. end of the implementation phase. Core project team members formalized sustainability plans in written reports according to Schalock and colleagues’ framework [22] and shared those reports with site-based project team members for member checking. We calculated frequencies and proportions for patient reports of preferred methods and timing of decision aid delivery.