The systematic search resulted in 193 records. We screened 107 unique records of titles and abstracts after the removal of 86 duplications. The full-texts and supplementary materials of 16 potentially eligible literature were assessed. Finally, 11 studies were included for systematic review (Fig. 1) [19,20,21,22,23,24,25,26,27,28,29]. No extra potentially eligible literature was identified by browsing the reference lists of included studies or relevant reviews. The included studies and excluded full texts with justifications are recorded (Supplementary Note S2).

Flowchart of study search and selection. ccLS, clear cell likelihood score; CNKI, China National Knowledge Infrastructure

The characteristics of the 11 included studies are summarized (Table 1) [19,20,21,22,23,24,25,26,27,28,29]. The studies were more frequently published in radiological journals (8/11, 73%) [22,23,24, 26,27,28,29, 40]. Most of the studies were conducted retrospectively (9/11, 82%) [19, 22,23,24,25,26,27,28,29] within a single center (8/11, 73%) [19, 20, 22, 24, 26,27,28,29] with the modality of MRI (8/11, 73%) [19,20,21,22,23,24,25,26]. The mean ± standard deviation, median (range) of number of patients and percentage of ccRCC were 176.5 ± 191.5, 110.5 (51.0–691.0), and 53.3 ± 11.2, 51.0 (41.0–73.9) %, respectively. The rating was performed by 2.2 ± 1.2, 2 (1.0–5.0) raters per study, applying MRI ccLS algorithm version 2.0 (5/12, 42%) [22,23,24,25,26], MRI ccLS algorithm version 1.0 (4/12, 33%) [19, 20, 25, 40], and CT ccLS algorithm (3/12, 25%) [27,28,29]. The appropriate blindness method was applied in most of the studies (10/11, 91%), including blindness to histological diagnosis (8/11, 73%) [19, 22,23,24,25,26,27,28], or prospectively assigned ccLS rating in a structured radiological report (2/11, 18%) [20, 21]. All the included studies assessed the diagnostic performance of ccLS algorithms. One study compared and suggested that MRI ccLS algorithm version 2.0 potentially had better diagnostic performance than version 1.0 [25]. Another study indicated that both T2-weighted imaging with or without fat suppression were suitable for MRI ccLS algorithm version 2.0 [24]. The details of each included study are summarized (Supplementary Tables S4 to S8).

The results of the risk of bias and applicability concerns assessments are summarized (Fig. 2). The risk of bias and application concerns were mainly related to the index test and flow and timing. For the index test, the MRI protocol has not been fully reported [22], the training process of ccLS was not introduced [20], and the blindness method was not declared [29]. For the flow and timing, the intervals between the imaging and surgery were not reported in five studies [19, 20, 22, 26, 40], while the intervals were considered to be inappropriate in four studies [23, 27,28,29]. Only one study was rated as unclear risk of bias in reference standard because the histology of biopsy was used as the reference standard [28]. The QUADAS-2 assessment of each study by two reviewers was recorded (Supplementary Table S9).

Risk of bias and applicability concerns assessment. A The risk of bias and applicability concerns assessment per study; B The percentage of the risk of bias and applicability concerns assessment

The meta-analysis included nine studies [19, 21,22,23,24, 26,27,28,29] after the exclusion of two overlapping studies [20, 24] (Supplementary Note S4). The two-by-two data were directly extracted or reconstrued for meta-analysis (Supplementary Table S10). The diagnostic odds ratio of six MRI ccLS studies [