Introduction

Psychiatric disorders represent a formidable challenge to public health. According to a recent report from WHO, psychiatric disorders account for approximately 10% of the total burden of non-communicable diseases globally, and 25% of all years lived with disability.1 While patient-specific circumstances, familial factors and genetic predisposition can all influence disease outcomes, adherence to prescribed medication regimens remains a key determinant of the health of psychiatric patients.2 The results of a meta-analysis suggested that treatment non-adherence is influenced by a variety of factors, including the patient’s negative attitude towards the disease, shame associated with the disease, a lack of societal/family support and adverse drug reactions, among others.3 In addition, adherence to medication is a major factor influencing treatment outcomes and relapse in patients with psychiatric disorders,4 highlighting the importance of the timely identification and assessment of medication compliance in the treatment of psychiatric patients.

Various methods are employed for predicting treatment adherence, including direct patient observation,5 self-reporting,6 drug concentration determination7 and electronic monitoring.8 However, these methods all have their limitations. For instance, direct observation may be challenging to implement in practice, requiring substantial investment in time and other resources. Patient self-reporting may be influenced by recall bias and social desirability, while drug concentration determination and electronic monitoring necessitate the use of additional equipment and resources.9 Consequently, more effective and convenient methods are required for predicting treatment compliance.

Nomograms are intuitive and easy-to-use statistical tools for risk quantification widely used in clinical practice across a variety of conditions.10 Several nomogram-based predictive models have been developed over recent years, notably for the prediction of postoperative recurrence in non-functioning pituitary adenoma11 and the risk of postpartum stress urinary incontinence in primiparas.12 However, no established nomogram predictive model is currently available for forecasting medication adherence in patients with psychotic disorders.

The aim of this study was to develop a novel predictive nomogram to address this gap in mental disorder treatment. The model, based on the acquisition of publicly available data, incorporated demographic data, disease-dependent characteristics and therapeutic regimen information. We anticipate that our research will provide physicians with critical insights for devising effective treatment plans. Ultimately, this could improve patient adherence to treatment protocols, thereby enhancing their quality of life.

Materials and methodsSubjects

The data used in this study were collected from observational studies from 35 community mental health centres in Italy,13 a dataset from the Dryad Digital Repository (https://doi.org/10.5061/dryad.q49p6d8). The study protocol was approved by the local ethics committee of the Ethics Committee of the Azienda Ospedaliera Universitaria Integrata of Verona (Comitato Etico per la Sperimentazione Clinica of the Provinces of Verona and Rovigo, protocol n. 57 622 of 9 December 2015), and was made publicly available at the Open Science Framework online repository (https://osf.io/wt8kx/). A total of 451 patients were enrolled for whom data were collected from 1 December 2015 to 31 May 2017. The inclusion criteria were (1) aged 18 years or older; (2) diagnosed with a schizophrenia spectrum disorder, bipolar disorder, personality disorder or other organic mental disorders; (3) signed an informed consent form and (4) undergoing or preparing for long-term antipsychotic treatment, oral or otherwise. The overall protocol used for patient screening is presented in figure 1.

Figure 1

Flow chart of the study design.

Figure 3

Nomogram for predicting medication adherence among patients with psychotic disorders. BPRS, Brief Psychiatric Rating Scale; DAI-10, Drug Attitude Inventory-10.

Figure 2

Selection of the tuning parameter (λ) for the Least Absolute Shrinkage and Selection Operator (LASSO) model.

Data processing

A total of 432 patients were included after the exclusion of those with missing data. The outcome variable (patient compliance) was redefined using the clinician-rated Kemp’s 7-point scale for the assessment of medication adherence (KEMP scale), with scores of 5 or higher indicating good compliance and scores of <5 indicating poor compliance. Other clinically relevant covariates were also included.

Implementation of decision curve analysis (DCA)

We performed a decision curve analysis (DCA) to evaluate the clinical utility of our predictive model (nomogram). For each patient, predicted probabilities of the outcome were calculated. A range of threshold probabilities was defined, representing the probability above which a patient would opt for treatment. Net benefit was calculated for each threshold probability using the formula:

(1)

where N is the total number of patients. In this study, the decision curve plots the net benefit against the threshold probabilities for the nomogram, compared with the strategies of treating all patients and treating none.

Statistical analysis

The independent risk factors related to medication compliance in patients with mental disorders were determined using Least Absolute Shrinkage and Selection Operator (LASSO) regression and reference clinical significance, and a line chart was drawn using the ‘DynNom’ function in R software. The consistency index (C-index) and Brier index were calculated using relevant functions in R software such as ‘rms’, ‘glmnet’, ‘rsconnect’ and ‘MASS’. Receiver operating characteristic (ROC) curves and calibration curves were plotted. Internal validation was performed using the bootstrap resampling method with 1000 iterations. The significance level was set to α=0.05, with a p value of <0.05, indicating a statistically significant difference.

Patient and public involvement

This is a cross-sectional non-interventional study. As a result, no patients were directly involved in the creation of the study, the formation of research goals or questions or the implementation of the study. Furthermore, patients did not participate in interpreting the results or in writing the manuscript.

ResultsBaseline characteristics and redefined variables

The baseline characteristics of the eligible cohort are listed in table 1 and online supplemental S1. Among the 432 patients, 167 (38.66%) were classified as having poor adherence and 265 (61.34%) as displaying good adherence. A total of 263 (60.88%) were males and 169 (39.12%) were females. Most of the patients in both sets were unmarried and unemployed.

Table 1

Patient demographics and clinical characteristics

Risk factors affecting patient compliance

13 variables that could affect patient compliance were included in the study. The variables were processed by LASSO regression for dimension reduction. The optimal lambda parameter was selected based on the criterion of minimum cross-validation error. The variables with non-zero regression coefficients at this lambda value were considered the most representative risk factors. LASSO regression analysis results showed that the optimal lambda value was 0.0587. The variables found to be significantly associated with patient compliance included the number of hospitalisations in one year, history of long-acting injectable medication use, DAI-10 score and BPRS score (figure 2). These variables were deemed the most significant predictors of medication adherence in our study (p value <0.05).

Regression analysis

The four variables selected by LASSO regression (the number of hospitalizations in 1 year, history of long-acting injectable medication use, DAI-10 score and BPRS score) were used as independent variables and patient compliance (good compliance=1, poor compliance=0) served as the dependent variable. A binary logistic regression model was constructed, with variables being classified as follows: number of hospitalisations in 1 year: ≥3 times=high, <3 times=low; DAI-10 score: poor compliance <8, good compliance ≥8; long-acting injectable medication history: no=0, yes=1 and BPRS score: mild <40, moderate 40–52, severe >52. The results showed that the model fit the data well (p<0.05) and the four variables were independent risk factors for medication non-compliance among patients with mental disorders. The relevant statistics are shown in table 2.

Table 2

Multivariate analysis of the risk factors

Construction of the prediction model

Using the four variables selected from the multivariate regression analysis, a nomogram was plotted using R software and its related packages to predict the medication compliance of patients with mental disorders (figure 3). From the nomograph, it can be seen that patients with higher DAI-10 scores, lower BPRS scores and fewer hospitalizations, as well as those without a history of long-acting injectable medication use, are more likely to have good compliance in the future.

Evaluation of the prediction model

The nomogram chart was validated using the bootstrap resampling method with 1000 iterations. To identify and correct for bias, we used bootstrapping. Specifically, we performed 1000 bootstrap resamples to estimate the average calibration error of the model’s predictions. This estimate was then used to adjust the apparent calibration curve, resulting in the bias-corrected curve. The calibration curve (figure 4) showed good consistency between the predicted and actual probabilities after this adjustment. The C-index and the Brier score were 0.709 and 0.215, respectively, indicating that the nomogram model had good discriminatory ability. The Brier score measures the mean squared difference between predicted probabilities and actual outcomes, with lower scores indicating better accuracy. Our model’s Brier score demonstrates a reasonably good prediction accuracy, further validating its utility in clinical practice. The area under the ROC curve (figure 5) was 0.716 (95% CI 0.669 to 0.763), indicating that the model had moderate predictive ability. The DCA (figure 6) showed that when the threshold probability of the nomogram lay between 0.44 and 0.63, there was a net clinical benefit for the patients.

Figure 4

Nomogram calibration curve. The dotted line indicates perfect prediction by an ideal model. The red line depicts the performance of the model.

Figure 5

The area under the receiver operating characteristic (ROC) curve of the nomogram.

Figure 6

Decision curve analysis for the nomogram. The y-axis shows the net benefit and the x-axis shows the corresponding risk threshold. The green line represents the assumption that all patients have good adherence. The blue line represents the assumption that none of the patients adhere to treatment. The red line represents the nomogram. The decision curve indicated that if the threshold probability lies between 0 and 1.0, then using the nomogram to predict adherence provides more benefit than the Intervention-all-patients scheme or the Intervention-none scheme.

Discussion

Medication compliance among patients with mental disorders is related to disease prognosis, recovery and outcome. Studies have shown that good patient compliance can reduce the recurrence rate of mental illnesses.14 15 However, a 6-month follow-up study found that non-adherence to treatment among patients with schizophrenia was as high as 58.2%.16 Moreover, the Clinical Antipsychotic Trials of Intervention Effectiveness study found that among 1493 patients with schizophrenia participating in multisite clinical trials, 74% discontinued antipsychotic treatment after 18 months.17 Treatment adherence is also closely related to the recurrence of schizophrenia. Indeed, one study reported that compared with continuous medication, the risk of recurrence and hospitalisation after discontinuation nearly doubles, even if the medication is stopped for only 1–10 days.18 A retrospective study by Zipursky et al 19 reported that 77% of patients with schizophrenia relapsed within 1 year after discontinuation of medication, and the risk of relapse increased to a staggering 90% after 2 years. In contrast, the estimated relapse rate for patients who continued medication was only 3%. Therefore, the early identification and improvement of patient compliance are of major clinical significance. In this study, we analysed the clinical data of 432 patients with mental disorders available in the Dryad digital repository. The results showed that approximately 38.7% of these patients had poor compliance, which is consistent with previously reported overall average compliance rates for patients with mental disorders.16 20 This implies that clinical psychiatrists should focus on improving patient compliance. On the one hand, they should formulate relevant intervention measures for the risk factors that affect compliance. On the other hand, they should integrate these factors and identify patients with poor compliance to allow for targeted intervention, including clinical–community and clinical–family linkage, to reduce the recurrence rate of the respective disease.

In this study, using LASSO regression analysis, we screened out four factors related to patient compliance, namely, the DAI-10 score, BPRS score, number of hospitalizations in 1 year and history of long-acting injectable medication. DAI-10 is a survey of attitudes towards antipsychotic medication that incorporates 10 items, including ‘taking medication is my own choice’ and ‘medication makes me feel tired and sluggish’.21 The DAI-10 score is the main indicator of patient compliance, with higher scores reflecting better adherence to medication. Higher scores also indicate that a patient has a greater acceptance of and a clear attitude towards a given medication, which can positively influence their decision to continue treatment. The BPRS score22 is used in clinical settings to assess the severity of a patient’s illness. The higher the score, the more severe the illness, with studies having shown that a BPRS score >35 is a risk factor for poor compliance.23 It has also been reported that the extent of a patient’s insight into their illness and the prescribed treatment is related to their medication compliance.24 The more hospitalisations a patient has in 1 year, the worse their medication compliance, and the more severe their illness is likely to be. This is in line with the results of surveys of medication compliance in patients with mental disorders in Jiangmen and Shenzhen Baoan.25 26 However, our results indicated that the compliance of patients who have previously used long-acting injectable drugs is worse than that of patients who have not used them, which seems to contradict the conclusions of real-world studies. The use of long-acting injectable drugs started relatively late in China, and many relevant references support their use in improving patient adherence to treatment.27 28

In Italy, the adoption of long-acting injectable antipsychotics has been supported by various studies showing their effectiveness in improving medication adherence and reducing hospitalisation rates. A meta-analysis in the USA showed that, compared with oral antipsychotics, the use of long-acting injectable antipsychotic drugs is associated with reduced hospitalisation and emergency admission rates and increased medication compliance.29 For patients in the initial stages of a mental illness, the continuous use of long-acting injectables can relieve symptoms faster than oral medication.30 Moreover, a real-world retrospective cohort study of antipsychotic drugs involving patient data from two hospitals in China and the Japan Medical Data Centre31 found that both persistence and compliance were better among patients taking long-acting injectable antipsychotics (paliperidone palmitate once-monthly injection) than among those on oral second-generation antipsychotics. While this study highlighted regional differences, similar patterns can be expected in Italy due to similar healthcare structures and insurance coverage for long-acting intramuscular (LAI). In Italy, LAI is generally covered by the national healthcare system, which could potentially lead to better persistence and compliance compared with regions where such coverage is not available. Furthermore, in some low-income countries, the use of LAI drugs among patients with mental disorders is not standard practice. LAI antipsychotics are often used by patients who cannot tolerate oral medication, resulting in poor compliance.32 Understanding these differences is crucial for interpreting our study’s findings in the context of Italy, where healthcare policies and practices support the use of LAI, potentially leading to better patient outcomes. Specifically, an open-label, randomised controlled trial (RCT) conducted by Weiden and others showed no significant difference in the level of compliance between patients using long-acting injectable antipsychotics and those taking oral antipsychotics.33 The inconsistencies in the reported compliance rates may be explained by the fact that RCTs34 strictly limit the severity of the investigated disease, the associated complications and concomitant medication use, among other factors. Patients often display a strong willingness to cooperate and treatment motivation, and their cognitive function is relatively unaffected, thereby reducing the importance of treatment compliance. At the same time, when compared with a normal medical environment, participation in an RCT can draw more attention to the patient. RCTs usually also provide free treatment and drugs, reimburse travel costs, and remind patients of their return visits on time. In real-world studies,35 unravelling the use of medication by subjects is complex. Patients choose treatment measures based on their own conditions and wishes, and concomitant medication use and medication conditions are not controlled. Accordingly, the extrapolation of RCT results to a real-world setting has certain limitations.36 This means that in studies more oriented towards actual clinical situations, the issue of treatment compliance in patients with schizophrenia remains of critical importance.

In this study, based on the identified risk factors, a nomogram model was established to predict medication compliance in patients with mental disorders. The model was based on clinical interview data and did not involve additional medical costs, making it simple and easy to use. The C-index37 is commonly employed to evaluate the accuracy of a model. The C-index of this model was 0.709>0.7, indicative of moderate accuracy, and the predicted values of the calibration curve were in good agreement with the actual values. The area under the ROC curve of the model was 0.716>0.6, indicating that it had moderate predictive power. Additionally, we found that at threshold probabilities between 44% and 63%, using the nomograph to predict medication compliance in patients with mental disorders had high net benefit value and, to some extent, clinical predictive utility. In clinical practice, the use of nomograms proves to be straightforward and convenient, which not only facilitates their application in community primary care settings but also enables preliminary risk prediction for individuals without a medical history. For instance, consider a 55-year-old female patient diagnosed with schizophrenia who has a preceding BPRS score of 50, two hospital admissions in the past year, no history of long-acting injectable drug usage and a DAI-10 score of 10. Her corresponding score would approach 190, correlating with an estimated treatment adherence probability of approximately 82%. Moreover, for first-time patients, the long-term treatment effect and prognosis are unknown. At this point, our nomogram can predict individual treatment compliance and can supplement the patient’s diagnosis and treatment plan. Additionally, the nomogram can be used to provide targeted disease education for individual patients, thus improving long-term treatment effects.

This study had some limitations, including in data collection and methodology. First, the data were sourced from only one public dataset, which limited its diversity and representativeness, potentially making the results non-generalisable to other patient populations with mental disorders. Second, the use of retrospective data analysis prevented the establishment of causality. Additionally, other factors that may affect medication compliance, such as psychosocial factors and different treatment regimens, were not considered. Furthermore, the differential impacts of disease type on compliance were not adequately assessed.

To improve the model’s accuracy, we plan to take the following steps: first, we will expand our dataset by collecting more data to enhance the model’s training effectiveness and diversity. Second, we will optimise feature selection by incorporating more clinically significant features and considering factors like psychosocial influences and treatment regimens. Finally, we will explore the use of more complex models, such as ensemble learning methods, to improve predictive performance. We believe these improvements will significantly enhance the model’s prediction accuracy and clinical utility.