Aggregating published prediction models with individual participant data: a comparison of different approaches

  title={Aggregating published prediction models with individual participant data: a comparison of different approaches},
  author={Thomas P. A. Debray and Hendrik Koffijberg and Yvonne Vergouwe and Karel G. M. Moons and Ewout W.  Steyerberg},
  journal={Statistics in Medicine},
During the recent decades, interest in prediction models has substantially increased, but approaches to synthesize evidence from previously developed models have failed to keep pace. This causes researchers to ignore potentially useful past evidence when developing a novel prediction model with individual participant data (IPD) from their population of interest. We aimed to evaluate approaches to aggregate previously published prediction models with new data. We consider the situation that… 

Meta‐analysis and aggregation of multiple published prediction models

Model aggregation is a promising strategy when several prediction models are available from the literature and a validation dataset is at hand, and results in equivalent performance when validation datasets are relatively large.

A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta‐analysis

This framework allows the development, implementation, implementation in new individuals, and evaluation of a single, integrated prediction model from an IPD-MA in order to achieve improved model performance and generalizability.

Developing more generalizable prediction models from pooled studies and large clustered data sets

The methodology enables the assessment of heterogeneity of prediction model performance during model development in multiple or clustered data sets, thereby informing researchers on predictor selection to improve the generalizability to different settings and populations, and reduce the need for model tailoring.

Assessment of heterogeneity in an individual participant data meta‐analysis of prediction models: An overview and illustration

Approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources are proposed and illustrated in a case study with patients suffering from traumatic brain injury.

Evidence synthesis for prognosis and prediction : application, methodology and use of individual participant data

This thesis investigates the application and development of evidence synthesis methods for prognosis research, with particular attention given to improving individualised predictions from prognostic models developed and/or validated using metaanalysis techniques.

A multiple‐model generalisation of updating clinical prediction models

The proposed “hybrid method” re‐calibrates multiple CPMs using stacked regression while concurrently revising specific covariates using individual participant data (IPD) under a penalised likelihood, thereby incorporating IPD, existing research, and prior (clinical) knowledge into the modelling strategy.

Clinical prediction in defined populations: a simulation study investigating when and how to aggregate existing models

Investigation of the impact of between-population-heterogeneity and sample size on aggregating existing CPMs in a defined population, compared with developing a model de novo demonstrates a pragmatic approach to contextualising C PMs to defined populations.

Synthesis of clinical prediction models under different sets of covariates with one individual patient data

BackgroundRecently, increased development of clinical prediction models has been reported in the medical literature. However, evidence synthesis methodologies for these prediction models have not

Incorporating published univariable associations in diagnostic and prognostic modeling

The proposed method to aggregate previously published univariable associations with individual participant data in the construction of a novel prediction models outperforms established approaches and is especially worthwhile when relatively limited individual participantData are available.

Avenues for Further Research

This chapter presents an overview of the recent statistical methods for diagnostic meta-analysis and suggests a few directions for future research, including the robustness of model misspecifications and the identifiability of models and the assumption of conditional independence in the absence of a gold standard.



A simple method to adjust clinical prediction models to local circumstances

It is shown how a simple updating method may suffice to update a clinical prediction model when a previously developed model is validated and the performance is poor, and may yield better risk estimates.

Prognostic models based on literature and individual patient data in logistic regression analysis.

It is concluded that prognostic models may benefit substantially from explicit incorporation of literature data, as distinguished in calibration and discrimination, when compared to models including shrunk or penalized estimates.

Clinical Prediction Models: A Practical Approach to Development, Validation and Updating

Suggested that could be demonstrated a live birth and data and demonstrated that were excluded, and developed and could be appropriate aac evidence.

Impact of approximating or ignoring within‐study covariances in multivariate meta‐analyses

The impact of errors in approximating within-study covariances on the parameters of multivariate models in a simulation study found that treatment effect and heterogeneity estimates were not strongly affected by inaccurate approximations, but estimates of the correlation between outcomes were sometimes highly biased.

Augmenting Data With Published Results in Bayesian Linear Regression

A Bayesian model in which a linear regression analysis on current data is augmented with the reported regression coefficients of previous studies to show that augmenting data with results from previous studies is viable and yields significant improvements in the parameter estimation.

Advanced methods in meta‐analysis: multivariate approach and meta‐regression

This tutorial on advanced statistical methods for meta‐analysis can be seen as a sequel to the recent Tutorial in Biostatistics on meta‐analysis by Normand, which focused on elementary methods.

Bivariate random-effects meta-analysis and the estimation of between-study correlation

Maximum likelihood estimation of a general normal model and a generalised model for bivariate random-effects meta-analysis (BRMA), which highlights some of these benefits in both a normal and generalised modelling framework, and examines the estimation of between-study correlation to aid practitioners.