ObjectiveA simulation study was used to generate the multivariate normal distribution data with a residual effect based on series of N-of-1 trials. The statistical performance of paired t-test, mixed effect model and Bayesian mixed effect model were compared.MethodsThree-cycles N-of-1 trials were set, and the participants were randomly assigned to 2 different treatments in each cycle. The simulation study included the following procedures: producing six-dimensional normal distribution data, randomly allocating intervention methods and patients, adding residual effects, constructing and evaluating 3 models, and setting the parameters. The sample sizes were set as 3, 5, 8 and 10, and the correlation coefficients among different times were set as 0.0, 0.5 and 0.8. Different proportions of residual effects for the 2 groups were set. Type I error, power, mean error (ME), and mean square error (MSE) were used to compare the 3 models.ResultsWhen there was no residual effect in the 2 groups, type I errors of 3 models were approximately 0.05, and their MEs were approximately 0. Paired t-test had the highest power and the lowest MSE. When the residual effect existed in the 2 groups, the type I error of paired t-test increased, and its estimated value deviated from the true value (ME≠0). Type I errors of the mixed effect model and Bayesian mixed-effect model were approximately 0.05, and they had the same power. The estimated values of the two models were close to the true value (ME was approximately 0).ConclusionsWhen there is no residual effect (0% vs. 0%), paired t-test is suitable for data analysis of N-of-1 trials. When there is a residual effect, the mixed effect model and Bayesian mixed-effect model are suitable for data analysis of N-of-1 trials.
ObjectiveWhen using multi-center data to construct clinical prediction models, the independence assumption of data will be violated, and there is an obvious clustering effect among research objects. In order to fully consider the clustering effect, this study intends to compare the model performance of the random intercept logistic regression model (RI) and the fixed effects model (FEM) considering the clustering effect with the standard logistic regression model (SLR) and the random forest algorithm (RF) without considering the clustering effect under different scenarios. MethodsIn the process of forecasting model establishment, the prediction performance of different models at the center level was simulated when there were different degrees of clustering effects, including the difference of discrimination and calibration in different scenarios, and the change trend of this difference at different event rates was compared. ResultsAt the center level, different models, except RF, showed little difference in the discrimination of different scenarios under the clustering effect, and the mean of their C-index changed very little. When using multi-center highly clustered data for forecasting, the marginal forecasts (M.RI, SLR and RF) had calibrated intercepts slightly less than 0 compared with the conditional forecasts, which overestimated the average probability of prediction. RF performed well in intercept calibration under the condition of multi-center and large samples, which also reflected the advantage of machine learning algorithm for processing large sample data. When there were few multiple patients in the center, the FEM made conditional predictions, the calibrated intercept was greater than 0, and the predicted mean probability was underestimated. In addition, when the multi-center large sample data were used to develop the prediction model, the slopes of the three conditional forecasts (FEM, A.RI, C.RI) were well calibrated, while the calibrated slopes of the marginal forecasts (M.RI and SLR) were greater than 1, which led to the problem of underfitting, and the underfitting problem became more prominent with the increase in the central aggregation effect. In particular, when there were few centers and few patients, overfitting of the data could mask the difference in calibration performance between marginal and conditional forecasts. Finally, the lower the event rate the central clustering effect at the central level had a more pronounced impact on the forecasting performance of the different models. ConclusionThe highly clustered multi-center data are used to construct the model and apply it to the prediction in a specific environment. RI and FEM can be selected for conditional prediction when the number of centers is small or the difference between centers is large due to different incidence rates. When the number of hearts is large and the sample size is large, RI can be selected for conditional prediction or RF for edge prediction.
ObjectiveTo explore the method of constructing time-dependent variables of clinical prognostic model, and to combine marginal structure model with clinical prognostic model to provide a more accurate tool for individualized prognostic assessment of patients. MethodsThrough data simulation, a training dataset with sample size of 7 000 and a validation dataset with sample size of 3 000 were constructed, and the predictive efficacy of ignoring treatment model, baseline no-treatment model, baseline treatment prediction model and marginal structure prediction model were respectively compared under different follow-up times and different situations. ResultsAt 2 follow-up time points, there was no significant difference between the marginal structure prediction model and the baseline treatment prediction model, but they were higher than the neglected treatment model and the baseline no treatment model. At 5 follow-up time points, the prediction ability of the marginal structure prediction model was significantly higher than that of the other three prediction models. ConclusionIn the case of time-dependent treatment in the observational cohort, the change of treatment after baseline should be considered when constructing the clinical prognosis model, otherwise the prediction accuracy of the prognosis model will be reduced.