Skip to content
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Menu
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Menu
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Publication Details
AFRICAN RESEARCH NEXUS
SHINING A SPOTLIGHT ON AFRICAN RESEARCH
mathematics
Calibrating machine learning approaches for probability estimation: A comprehensive comparison
Statistics in Medicine, Volume 42, No. 29, Year 2023
Notification
URL copied to clipboard!
Description
Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies. © 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
Authors & Co-Authors
Ojeda, Francisco Miguel
Germany, Hamburg
Universitätsklinikum Hamburg-eppendorf
Blankenberg, Stefan S.
Germany, Hamburg
Universitätsklinikum Hamburg-eppendorf
Germany, Berlin
Deutsches Zentrum Für Herz-kreislauf-forschung E. V.
Weimar, Christian
Germany, Duisburg
Universität Duisburg-essen
Schmid, Matthias C.
Germany, Bonn
Universität Bonn
Ziegler, Andreas E.
Germany, Hamburg
Universitätsklinikum Hamburg-eppendorf
Switzerland, Davos
Cardio-care
South Africa, Durban
University of Kwazulu-natal
Switzerland, Lausanne
Sib Swiss Institute of Bioinformatics
Statistics
Authors: 5
Affiliations: 7
Identifiers
Doi:
10.1002/sim.9921
ISSN:
02776715
Research Areas
Health System And Policy
Study Design
Cross Sectional Study