A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta‐analysis

@article{Debray2013AFF,
  title={A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta‐analysis},
  author={Thomas P. A. Debray and Karel G. M. Moons and Ikhlaaq Ahmed and Hendrik Koffijberg and Richard D. Riley},
  journal={Statistics in Medicine},
  year={2013},
  volume={32}
}
The use of individual participant data (IPD) from multiple studies is an increasingly popular approach when developing a multivariable risk prediction model. Corresponding datasets, however, typically differ in important aspects, such as baseline risk. This has driven the adoption of meta‐analytical approaches for appropriately dealing with heterogeneity between study populations. Although these approaches provide an averaged prediction model across all studies, little guidance exists about how… 
View on Wiley
ncbi.nlm.nih.gov
154 Citations
Highly Influential Citations
5
Background Citations
32
Methods Citations
43
Results Citations
2

154 Citations

Developing and validating risk prediction models in an individual participant data meta-analysis

An IPD meta-analysis offers unique opportunities for risk prediction research by allowing separate model intercept terms for each study (population) to improve generalisability, and by using ‘internal-external cross-validation’ to simultaneously develop and validate their model.

Meta‐analysis and aggregation of multiple published prediction models

Model aggregation is a promising strategy when several prediction models are available from the literature and a validation dataset is at hand, and results in equivalent performance when validation datasets are relatively large.

Individual participant data meta‐analysis for external validation, recalibration, and updating of a flexible parametric prognostic model

IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibrations strategy to achieve acceptable external model performance without discarding existing model information.

Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model

Meta-analysis of risk prediction studies

A systematic review of published breast cancer models is performed, and IPD is obtained from an article which uses parathyroid hormone (PTH) assay (a continuous variable) to predict postoperative hypocalcaemia after thyroidectomy, and results indicate the approach performs well, though further research is required.

Evidence synthesis for prognosis and prediction : application, methodology and use of individual participant data

This thesis investigates the application and development of evidence synthesis methods for prognosis research, with particular attention given to improving individualised predictions from prognostic models developed and/or validated using metaanalysis techniques.

Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores

This work applies a network meta-analytic methodology to externally validate and concurrently compare the prognostic properties of clinical scores, and indicates that the scores with the best discriminative properties to predict 3 year mortality in patients with COPD are ADO and eBODE.

Developing more generalizable prediction models from pooled studies and large clustered data sets

The methodology enables the assessment of heterogeneity of prediction model performance during model development in multiple or clustered data sets, thereby informing researchers on predictor selection to improve the generalizability to different settings and populations, and reduce the need for model tailoring.

Assessment of heterogeneity in an individual participant data meta‐analysis of prediction models: An overview and illustration

Approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources are proposed and illustrated in a case study with patients suffering from traumatic brain injury.

A review of statistical updating methods for clinical prediction models

A clinical prediction model is a tool for predicting healthcare outcomes, usually within a specific population and context. A common approach is to develop a new clinical prediction model for each…
...

References

SHOWING 1-10 OF 74 REFERENCES

Aggregating published prediction models with individual participant data: a comparison of different approaches

It is illustrated how the generally unrealistic assumption of consistency in the availability of evidence across included studies can be relaxed and yielded prediction models with an improved discrimination and calibration in a vast majority of scenarios and result in equivalent performance in a small minority of situations.

Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker

This first article focuses on the different aspects of model development studies, from design to reporting, how to estimate a model's predictive performance and the potential optimism in these estimates using internal validation techniques, and how to quantify the added or incremental value of new predictors or biomarkers to existing predictors.

Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data.

Risk prediction models: II. External validation, model updating, and impact assessment

An overview is provided of the consecutive steps for the assessment of the model's predictive performance in new individuals, how to adjust or update existing models to local circumstances or with new predictors, and how to investigate the impact of the uptake of prediction models on clinical decision-making and patient outcomes (impact studies).

Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer

It is shown that a three-group prognostic classification scheme based on either score produces well-separated survival curves for each of the data sets, despite identifiable heterogeneity among the baseline distribution functions and to a lesser extent among the prognostic indexes for the individual studies.

Individual patient data meta-analysis of survival data using Poisson regression models

The approach provides a highly flexible and computationally efficient framework, available in all standard statistical software, to the investigation of not only heterogeneity, but the presence of non-proportional hazards and treatment effect modifiers.

A simple method to adjust clinical prediction models to local circumstances

It is shown how a simple updating method may suffice to update a clinical prediction model when a previously developed model is validated and the performance is poor, and may yield better risk estimates.

Heterogeneity of the baseline risk within patient populations of clinical trials: a proposed evaluation algorithm.

An evaluation algorithm for systematic assessment of the observed heterogeneity in disease risk within trial populations is presented and may be used for comparing diverse trials and used prospectively for designing future trials, as shown in simulations.

Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve.

  • N. Cook
  • Medicine
    Clinical chemistry
  • 2008
Evaluation of prognostic models should not rely solely on the ROC curve, but should assess both discrimination and calibration, to aid in comparing the clinical impact of two models on risk for the individual, as well as the population.

Clinical Prediction Models: A Practical Approach to Development, Validation and Updating

Suggested that could be demonstrated a live birth and data and demonstrated that were excluded, and developed and could be appropriate aac evidence.
...