Background
Methods
Data description
Database full name | Database short name | Country | Data type | Population size | Date range |
---|---|---|---|---|---|
IBM MarketScan® Commercial Claims and Encounters Database | CCAE | USA | Claims | 157 m | 2000–2021 |
IBM MarketScan® Multi-State Medicaid Database | MDCD | USA | Claims | 33 m | 2006–2021 |
IBM MarketScan® Medicare Supplemental Database | MDCR | USA | Claims | 10 m | 2000–2021 |
IQVIA Disease Analyser Germany EMR | IQVIA Germany | Germany | EHR | 31 m | 2011–2021 |
Candidate predictors
Handling of missing data
Statistical analysis methods
Outcome of interest | CCAE | MDCD | MDCR | IQVIA Germany |
---|---|---|---|---|
Acute myocardial infarction | 221.4 | 60.5 | ||
Alopecia | 191.8 | 205.8 | 203.6 | |
Constipation | 43.6 | 18.6 | 16.3 | 85.3 |
Delirium | 245.3 | 84.5 | ||
Diarrhea | 28.9 | 16.8 | 16.7 | |
Fracture | 174.3 | 104.7 | 34.1 | 185.8 |
Gastrointestinal hemorrhage | 209.5 | 83.1 | 44.6 | |
Hyponatremia | 157.7 | 85.8 | 32.4 | |
Hypotension | 131.6 | 47.6 | 21.5 | 178.3 |
Hypothyroidism | 73.1 | 76.6 | 31.9 | 158.9 |
Insomnia | 19.5 | 12.7 | 17.9 | 57.2 |
Ischemic stroke inpatient | 101.1 | |||
Nausea | 20.6 | 8.6 | 15.7 | 60.5 |
Open-angle glaucoma | 194.0 | |||
Seizure | 180.5 | 71.4 | 90.8 | |
Suicide and ideation | 49.7 | 19.2 | 164.6 | |
Tinnitus | 158.0 | 174.8 | 83.4 | 152.3 |
Ventricular arrhythmia and sudden cardiac death inpatient | 220.7 | 90.9 | ||
Vertigo | 167.5 | 202.4 | 71.9 | 204.5 |
Model evaluation
Results
Outcome of interest | Classifier | CCAE | MDCD | MDCR | IQVIA Germany |
---|---|---|---|---|---|
Acute myocardial infarction | Lasso | 0.86 (0.82–0.89) | 0.71 (0.69–0.73) | ||
Random forest | 0.87 (0.84–0.90) | 0.69 (0.66–0.71) | |||
XGBoost | 0.87 (0.85–0.90) | 0.71 (0.69–0.73) | |||
Alopecia | Lasso | 0.61 (0.57–0.66) | 0.69 (0.65–0.73) | 0.69 (0.65–0.73) | |
Random forest | 0.58 (0.53–0.63) | 0.65 (0.61–0.70) | 0.68 (0.64–0.72) | ||
XGBoost | 0.64 (0.59–0.68) | 0.68 (0.64–0.73) | 0.68 (0.64–0.72) | ||
Constipation | Lasso | 0.67 (0.64–0.69) | 0.65 (0.63–0.66) | 0.66 (0.65–0.68) | 0.80 (0.78–0.83) |
Random forest | 0.66 (0.64–0.69) | 0.64 (0.62–0.66) | 0.64 (0.63–0.66) | 0.81 (0.79–0.83) | |
XGBoost | 0.67 (0.65–0.69) | 0.65 (0.63–0.66) | 0.66 (0.65–0.68) | 0.80 (0.77–0.83) | |
Delirium | Lasso | 0.79 (0.75–0.84) | 0.75 (0.72–0.78) | ||
Random forest | 0.80 (0.76–0.84) | 0.73 (0.70–0.76) | |||
XGBoost | 0.80 (0.75–0.84) | 0.74 (0.71–0.77) | |||
Diarrhea | Lasso | 0.65 (0.63–0.67) | 0.67 (0.66–0.69) | 0.64 (0.62–0.65) | |
Random forest | 0.64 (0.62–0.66) | 0.67 (0.65–0.69) | 0.62 (0.61–0.64) | ||
XGBoost | 0.63 (0.61–0.66) | 0.67 (0.66–0.69) | 0.63 (0.61–0.65) | ||
Fracture | Lasso | 0.61 (0.56–0.66) | 0.70 (0.67–0.74) | 0.67 (0.65–0.70) | 0.82 (0.78–0.86) |
Random forest | 0.61 (0.56–0.65) | 0.66 (0.63–0.70) | 0.65 (0.63–0.67) | 0.80 (0.77–0.84) | |
XGBoost | 0.62 (0.57–0.67) | 0.69 (0.65–0.72) | 0.67 (0.65–0.69) | 0.82 (0.79–0.86) | |
Gastrointestinal hemorrhage | Lasso | 0.73 (0.67–0.78) | 0.74 (0.71–0.77) | 0.73 (0.71–0.76) | |
Random forest | 0.72 (0.67–0.77) | 0.75 (0.72–0.78) | 0.72 (0.70–0.74) | ||
XGBoost | 0.70 (0.65–0.75) | 0.74 (0.71–0.77) | 0.72 (0.70–0.75) | ||
Hyponatremia | Lasso | 0.74 (0.69–0.78) | 0.84 (0.81–0.86) | 0.66 (0.64–0.68) | |
Random forest | 0.73 (0.68–0.77) | 0.83 (0.80–0.85) | 0.64 (0.62–0.66) | ||
XGBoost | 0.74 (0.70–0.78) | 0.84 (0.81–0.86) | 0.66 (0.64–0.68) | ||
Hypotension | Lasso | 0.74 (0.70–0.78) | 0.75 (0.73–0.77) | 0.72 (0.71–0.74) | 0.71 (0.66–0.75) |
Random forest | 0.74 (0.70–0.78) | 0.74 (0.72–0.77) | 0.71 (0.70–0.73) | 0.71 (0.67–0.75) | |
XGBoost | 0.74 (0.71–0.78) | 0.75 (0.73–0.78) | 0.72 (0.70–0.74) | 0.71 (0.67–0.75) | |
Hypothyroidism | Lasso | 0.80 (0.78–0.83) | 0.76 (0.72–0.79) | 0.83 (0.81–0.85) | 0.86 (0.82–0.89) |
Random forest | 0.79 (0.76–0.82) | 0.74 (0.71–0.78) | 0.82 (0.80–0.84) | 0.87 (0.84–0.90) | |
XGBoost | 0.80 (0.77–0.83) | 0.75 (0.72–0.78) | 0.83 (0.81–0.85) | 0.86 (0.82–0.89) | |
Insomnia | Lasso | 0.64 (0.62–0.66) | 0.61 (0.60–0.63) | 0.67 (0.65–0.69) | 0.60 (0.57–0.63) |
Random forest | 0.62 (0.61–0.64) | 0.60 (0.58–0.61) | 0.66 (0.64–0.67) | 0.58 (0.55–0.60) | |
XGBoost | 0.64 (0.62–0.66) | 0.61 (0.60–0.63) | 0.67 (0.65–0.69) | 0.59 (0.56–0.62) | |
Ischemic stroke inpatient | Lasso | 0.79 (0.76–0.82) | |||
Random forest | 0.76 (0.73–0.79) | ||||
XGBoost | 0.78 (0.75–0.81) | ||||
Nausea | Lasso | 0.67 (0.66–0.69) | 0.66 (0.65–0.68) | 0.66 (0.64–0.68) | 0.75 (0.73–0.77) |
Random forest | 0.65 (0.64–0.67) | 0.65 (0.64–0.66) | 0.64 (0.63–0.66) | 0.75 (0.72–0.77) | |
XGBoost | 0.66 (0.65–0.68) | 0.66 (0.65–0.67) | 0.66 (0.64–0.68) | 0.75 (0.73–0.77) | |
Open-angle glaucoma | Lasso | 0.76 (0.71–0.82) | |||
Random forest | 0.77 (0.72–0.82) | ||||
XGBoost | 0.79 (0.75–0.84) | ||||
Seizure | Lasso | 0.75 (0.70–0.79) | 0.74 (0.71–0.77) | 0.74 (0.70–0.77) | |
Random forest | 0.73 (0.69–0.78) | 0.71 (0.68–0.74) | 0.73 (0.70–0.77) | ||
XGBoost | 0.72 (0.67–0.76) | 0.73 (0.70–0.76) | 0.73 (0.69–0.76) | ||
Suicide and ideation | Lasso | 0.79 (0.77–0.81) | 0.76 (0.74–0.77) | 0.73 (0.69–0.77) | |
Random forest | 0.75 (0.73–0.77) | 0.72 (0.71–0.74) | 0.64 (0.59–0.68) | ||
XGBoost | 0.79 (0.77–0.81) | 0.75 (0.74–0.77) | 0.71 (0.67–0.75) | ||
Tinnitus | Lasso | 0.66 (0.62–0.70) | 0.69 (0.64–0.74) | 0.60 (0.56–0.63) | 0.60 (0.56–0.65) |
Random forest | 0.64 (0.60–0.68) | 0.71 (0.67–0.76) | 0.58 (0.55–0.62) | 0.62 (0.58–0.66) | |
XGBoost | 0.66 (0.62–0.70) | 0.69 (0.65–0.74) | 0.59 (0.55–0.62) | 0.60 (0.55–0.65) | |
Ventricular arrhythmia and sudden cardiac death inpatient | Lasso | 0.83 (0.79–0.87) | 0.77 (0.74–0.79) | ||
Random forest | 0.84 (0.81–0.87) | 0.76 (0.73–0.79) | |||
XGBoost | 0.83 (0.79–0.87) | 0.77 (0.74–0.80) | |||
Vertigo | Lasso | 0.65 (0.61–0.70) | 0.72 (0.67–0.76) | 0.62 (0.59–0.65) | 0.63 (0.57–0.68) |
Random forest | 0.63 (0.58–0.68) | 0.70 (0.66–0.74) | 0.59 (0.55–0.62) | 0.65 (0.60–0.70) | |
XGBoost | 0.63 (0.58–0.67) | 0.71 (0.66–0.75) | 0.60 (0.57–0.64) | 0.63 (0.59–0.68) |
Database | Number of prediction tasks | Lasso | Random forest | XGBoost | All classifiers |
---|---|---|---|---|---|
CCAE | 14 | − 0.0025 (0.0073) | 0.0001 (0.0106) | 0 (0.0067) | − 0.0004 (0.0076) |
MDCD | 17 | − 0.0004 (0.0044) | 0 (0.0062) | 0 (0.0071) | 0 (0.0068) |
MDCR | 19 | 0.0000 (0.0052) | 0.0037 (0.0143) | 0 (0.0057) | 0 (0.0075) |
IQVIA Germany | 8 | − 0.0011 (0.0048) | 0.0012 (0.0095) | − 0.0045 (0.0204) | − 0.0010 (0.0098) |
All databases | 58 | − 0.0004 (0.0053) | 0.0008 (0.0099) | 0 (0.0074) | 0 (0.0081) |