AutoScore: An Interpretable Machine Learning-Based Automatic Clinical Score Generator

Author

Feng Xie, Yilin Ning, Han Yuan, Mingxuan Liu, Seyed Ehsan Saffari, Siqi Li, Bibhas Chakraborty, Nan Liu

Published

2023-02-06

AutoScore introduction

AutoScore is a novel machine learning framework to automate the development of interpretable clinical scoring models. AutoScore consists of six modules: 1) variable ranking with machine learning, 2) variable transformation, 3) score derivation, 4) model selection, 5) domain knowledge-based score fine-tuning, and 6) performance evaluation. The original AutoScore structure is elaborated in this article and its flowchart is shown in the following figure. AutoScore was originally designed for binary outcomes and later extended to survival outcomes and ordinal outcomes. AutoScore could seamlessly generate risk scores using a parsimonious set of variables for different types of clinical outcomes, which can be easily implemented and validated in clinical practice. Moreover, it enables users to build transparent and interpretable clinical scores quickly in a straightforward manner. Please visit GitHub page for source code.

Usage

The five pipeline functions constitute the 5-step process for generating point-based clinical scores for binary (Chapter 4), survival (Chapter 5) and ordinal (Chapter 6) outcomes.

This 5-step process gives users the flexibility of customization (e.g., determining the final list of variables according to the parsimony plot, and fine-tuning the cutoffs in variable transformation). Please follow the step-by-step instructions (in following chapters) to build your own scores.

  • STEP(i): AutoScore_rank()or AutoScore_rank_Survival() or AutoScore_rank_Ordinal() - Rank variables with machine learning (AutoScore Module 1)
  • STEP(ii): AutoScore_parsimony() or AutoScore_parsimony_Survival() or AutoScore_parsimony_Ordinal() - Select the best model with parsimony plot (AutoScore Modules 2+3+4)
  • STEP(iii): AutoScore_weighting() or AutoScore_weighting_Survival() or AutoScore_weighting_Ordinal() - Generate the initial score with the final list of variables (Re-run AutoScore Modules 2+3)
  • STEP(iv): AutoScore_fine_tuning() or AutoScore_fine_tuning_Survival() or AutoScore_fine_tuning_Ordinal() - Fine-tune the score by revising cut_vec with domain knowledge (AutoScore Module 5)
  • STEP(v): AutoScore_testing() or AutoScore_testing_Survival() or AutoScore_testing_Ordinal() - Evaluate the final score with ROC analysis (AutoScore Module 6)

We also include several functions in the package, which could help with data analysis and result reporting. As demonstrated in Chapter 3, these functions are compute_descriptive_table() for generating the table of descriptive analysis for your dataset, compute_uni_variable_table() or compute_uni_variable_table_survival() or compute_uni_variable_table_ordinal() for creating the table of univariable analysis for your dataset, and compute_multi_variable_table() for generating the table of multivariable analysis for your dataset.

AutoScore package installation

Install from GitHub or CRAN：

# From Github
install.packages("devtools")
library(devtools)
install_github(repo = "nliulab/AutoScore", build_vignettes = TRUE)

# From CRAN (recommended)
install.packages("AutoScore")

Load AutoScore package:

library(AutoScore)

Citation

Core paper

Method extensions

Contact