We included 178 clinical and economic reviews, with a decision made between 1 January 2018 and 31 December 2022, on the CADTH website. Of these, 31 (17%) reports received a ‘do not reimburse’ recommendation and 147 (83%) received a ‘recommended with clinical criteria and/or conditions’ recommendation, a phrase used by CADTH to indicate acceptance conditional on either specific clinical condition, a price reduction, or both. Of the conditional recommendations, 128 (87%) reports were recommended with a clinical condition and 138 (95%) were recommended with a price reduction. CADTH’s required conditions included being prescribed by a specialist or an experienced physician (n = 91), or reimbursement for a limited time period, conditional on demonstrating a treatment response or limited to a hospital setting (n = 119). Electronic supplementary (ESM) Table S1 provides the reasons given by CADTH for rejecting the reimbursement requests.

Randomized controlled trials, along with observational studies, were used in model development in 63% of reports (n = 112), while 37% used only randomized controlled trials (n = 66). Most of the reports were CUAs (n = 155, 87%), followed by cost comparison analyses (n = 13, 7%), and CMAs (n = 10, 6%). Markov or semi-Markov models were used in 75% of the models, followed by a combination of decision tree and Markov models (13%), decision tree only (7%), and others (5%). Among pivotal trials, six were single-arm trials and six were phase II trials. ESM Table S2 provides general characteristics of the included reports. Results of the Chi-square test showed that study type (p = 0.79), trial characteristics (p = 0.16), model type (p = 0.43), and utility measurement tools (p = 0.18) were independent of the final CADTH recommendation and that these variables do not have any relationship with the final recommendations.

ESM Table S3 presents the type of reimbursement recommendations in the reports. There was no association between the therapeutic category and the type of CADTH recommendation made in the report (p = 0.32).

3.1 Canadian Agency for Drugs and Technologies in Health (CADTH) Critiques on Manufacturers’ Submissions

Almost all the submissions (99%) had at least one effectiveness critique. The most common was ‘the uncertainty of evidence’, which pertained to 96% of the reports, followed by ‘not enough clinical evidence available’ (76%), ‘clinical data uncertainty beyond clinical evidence’ (74%) and ‘not a proper population captured’ (59%). The results of the Chi-square test analyzing the effectiveness critiques indicate that ‘other effectiveness critiques’, representing critiques we could not categorize, were more prevalent in the ‘do not reimburse’ group compared with the ‘recommended with clinical criteria/conditions’ group (p < 0.05). Although critiques on the uncertainty of evidence were mentioned repeatedly in the CADTH critical appraisals, only 7 (4%) reports mentioned a standardized tool such as Grading of Recommendations Assessment, Development and Evaluation (GRADE) for measuring uncertainty of evidence.

Eighty-nine percent of reports had at least one structure critique, with the most common structure critique being ‘the model does not reflect current practice’, which was responsible for 59% of all structure critiques. Lack of transparency and programming errors were more frequent in the ‘do not reimburse’ group compared with the ‘recommended with clinical criteria/conditions’ group (p < 0.05).

Of the total reports, 79% and 69% had at least one cost and utility score critique, respectively. Using critiques on non-Canadian utility scores and also using unaccepted extra costs were more frequent in the ‘recommended with clinical criteria/conditions’ group compared with the ‘do not reimburse’ group (p < 0.05). Data on frequencies of critiques based on CADTH recommendations are summarized in Table 1.

Among all the reports, 78 (44%) were assessed as highly uncertain, 91 (52%) were assessed as uncertain and only 4 (2%) reports could adequately address the uncertainty levels. Among 31 reports with a ‘do not reimburse’ recommendation, 10 (32%) were rejected merely due to the uncertainty of evidence; however, 64 (44%) reports with a ‘reimburse with clinical criteria/conditions’ recommendation had a high uncertainty level. The Chi-square test results show no association between uncertainty level and final recommendation (p = 0.31). Table 2 shows the uncertainty level specified in CADTH recommendations based on the final recommendations.

All reports had at least one critique on the 3-year BIA analysis, with the two most common critiques of 3-year BIA related to ‘population of patients’ and ‘percentage of market share’, responsible for 80.2% and 73.2% of all 3-year BIA critiques, respectively. Frequency of CADTH critiques were not significantly different based on different final recommendations. The CADTH critiques on 3-year BIA are shown in Table 3.

3.2 Drugs for Rare Disease and Drugs that Address an Unmet Need

All 68 CADTH reports of medications that addressed an unmet need were recommended for ‘reimbursement with clinical criteria/conditions’. For DRD, 47 (92%) of 51 reports recommended medication reimbursement with clinical criteria/conditions. In contrast, only 100 (79%) of 127 non-DRD medication reports received recommendations with clinical criteria/conditions. Addressing an unmet need and DRD were associated with a recommendation supporting reimbursement (p < 0.001 and p < 0.05, respectively) (Table 4). With some submissions, manufacturers claimed their medication would address an unmet need, but CADTH refused to accept that notion. One reason for rejecting these claims would be that the claim was only based on a surrogate outcome.

Table 4 Final reimbursement recommendations in different therapeutic categories3.3 CADTH Reanalysis

The median incremental cost submitted by the manufacturers was $6562, with an interquartile interval of −$246 to $127,607, indicating a wide variability in the cost of interventions produced by different manufacturers. The incremental cost in the CADTH reanalysis was higher, with a median cost of $10,777 and an interquartile interval of $378–$144,505. The median difference between manufacturers and CADTH was $1837, with an interquartile interval of −$679 to $25,295, an increase of 19.6% (interquartile interval −10.1% to 110.5%) [p < 0.001].

Of the 155 CUA reports, CADTH assessed 60 medications as clinically non-inferior, indicating the new drug does not provide a superior efficacy compared with existing interventions, and therefore assumed zero incremental utility for these medications. The median incremental QALY in manufacturers’ reports was 0.29 (interquartile interval 0.04–1.36) and the median incremental QALY in the CADTH reanalysis was 0.11 (interquartile interval 0.01–0.76). The difference between the manufacturers’ analysis and the CADTH reanalysis was −0.11 (interquartile interval −0.94 to 0.00) and the percentage difference was −50.0%, with an interquartile interval of −83.7% to −1.0% (p < 0.001). The data for incremental costs and QALYs are presented in Table 5.

Table 5 Difference between manufacturers’ submissions and the CADTH reanalysis regarding incremental cost, incremental QALY, and ICUR of non-oncology medications

Of the medications with an improved QALY, 28 (29%) reported dominance. For the remaining 67 reports (71%), the ICUR from the manufacturer and the CDR’s reanalysis were assessed. The median ICUR was significantly higher in the CADTH reanalysis compared with the manufacturers’ reports—$138,658/QALY (interquartile interval $43,203/QALY–$394,076/QALY) for manufacturers and $380,251/QALY (interquartile interval $149,197–$1,347,825) for the CADTH reanalysis (p < 0.001). The difference in the median ICURs was $169,299/QALY (interquartile interval $56,040–$574,073), i.e. 151% (interquartile interval 33–353%) higher in the CADTH reanalysis than the manufacturers’ reports (Table 5).

The distributions of the manufacturers’ and CADTH’s ICURs are presented in Fig. 1; in 5 (7%) of 67, the manufacturers’ ICUR was less than the CADTH reanalysis.

Fig. 1

Distribution of manufacturers’ and CADTH reanalysis ICUR values for 67 CADTH reports from 2018 to 2022 for non-oncology medications

A total of 81 medications reported a 3-year BIA, with 44 (54.3%) assessed in 2022, 30 (37%) in 2021, and 7 (9%) in 2020. The CADTH reanalyzed 76 of these reports (94%). The median 3-year BIA for manufacturers was $13,666,621 (interquartile interval $551,393–$72,640,799) and was significantly lower than the median value reported in the CADTH reanalysis, i.e. $19,104,299 (interquartile interval $3,125,747–$98,198,139) [p < 0.001]. The difference in the median values between manufacturers and CADTH was $4,575,102 (interquartile interval $9170–$26,330,512), with a percentage reduction of 27.0% (interquartile interval 0–182%) (Table 6). The distributions of the manufacturers’ and CADTH’s 3-year BIAs are summarized in Fig. 2. The 3-year BIA in the CADTH reanalysis was higher in 66 (87%) of the 76 reports.

Table 6 Three-year BIA from the manufacturers’ report and CADTH reanalysisFig. 2

Distribution of 76 3-year budget impact analysis results of manufacturer’s reports compared to the CADTH reanalysis reports from 2018 to 2022 for non-oncology medications

The results of the univariate logistic regression model are shown in ESM Table S4. The results showed that DRD is a predictor of receiving a recommendation with criteria/conditions from CADTH. To test the robustness of the results, we ran a multiple variable logistic regression that showed that DRD and addressing an unmet need are not predictors of receiving a recommendation with clinical criteria/conditions (ESM Table S5).

3.4 Price Reduction Requested and Willingness to Pay Threshold Mentioned in CADTH Reports

CADTH requested a median price reduction of 64% (interquartile interval 35–88%) for non-oncology medications. The price reduction requested was lowest for drugs in the ‘certain infectious or parasitic diseases’ category, ‘diseases of the skin’, and ‘diseases of the musculoskeletal system or connective tissue’. The highest median CADTH requested price reduction was for ‘diseases of the blood or blood-forming organs’, ‘diseases of the respiratory system’, and ‘endocrine, nutritional or metabolic diseases’ (Fig. 3).

Fig. 3

Price reduction requested by CADTH based on therapeutic categories for non-oncology medications

The CADTH reports for non-oncology medications from 2018 through 2020 indicated that the most requested price reduction was a consequence of the ICUR being greater than the WTP of $50,000/QALY. Among the WTP thresholds mentioned in 2018, 2019, and 2020, the most frequent were $50,000 and $100,000. In 2018 and 2019, other WTP values were also mentioned by CADTH, including $25,000, $200,000, $400,000, and $500,000. In 2021 and 2022, $50,000 was the only WTP threshold mentioned in CADTH reports. Figure 4 shows the frequency of WTP thresholds mentioned in CADTH reports from 2018 to 2022. In 39 (22%) CADTH reports, no specific WTP threshold was specified. These reports were mostly those in which CADTH failed to find the medications to be superior to their comparators. For these reports, CADTH stated that the price of the new intervention should not exceed that of the currently available options.

Fig. 4

Willingness to pay threshold mentioned in CADTH price reduction requests from 2018 to 2022 for non-oncology medications