AI/ML models to predict cardiovascular related health outcomes
iAdeptive Technologies was privileged to engage in the Veterans Cardiac Health and AI Model Predictions (V-CHAMPS) Challenge, an initiative jointly hosted by the Veterans Health Administration (VHA) Innovation Ecosystem (IE), the Digital Health Center of Excellence (DHCoE) at the US Food and Drug Administration (FDA), the FDA Office of Digital Transformation (ODT) via precisionFDA, and the UK Medicines and Healthcare products Regulatory Agency (MHRA).
In our participation, we leveraged deep learning methodologies employing neural networks and gradient boosted tree models to predict health outcomes associated with cardiovascular conditions. This collaborative effort was a significant stride toward advancing our understanding of cardiac health within the Veterans community and the broader healthcare landscape.
Scope
- AI Product Development
- AI Strategy Consulting
- Data management & Intelligence
Objective
Prioritizing heart health, especially among Veterans with conditions like diabetes, spinal cord injuries, and PTSD, is crucial. Cardiovascular disease encompasses various heart and blood vessel conditions, from hypertension to heart attacks and strokes. It’s a leading cause of hospitalization in the VA healthcare system and a major source of disability.
To enhance our grasp of cardiovascular health, the VHA IE and partners organized the V-CHAMPS competition. This initiative leveraged AI and machine learning to predict two vital health outcomes:
- Mortality Model: Predicting the likelihood of patient mortality during hospital admission.
- Readmissions Model: Predicting the likelihood of patient readmission within 90 days.
Additionally, the competition sought insights from non-traditional factors, making V-CHAMPS a comprehensive effort in advancing cardiovascular health research.
The —
Solution
Approach
01. Data
The challenge organizers provided synthetic data generated from veteran health records, creating a data lake mirroring the patient population while ensuring anonymity. This dataset included diverse health records like demographics, inpatient/outpatient admissions, lab results, diagnoses, procedures, medications, vital measurements, and more. Various data transformations were applied to shape the modeling dataset, including merges, searches, counts, and intricate text data embeddings, as elaborated in the mortality model section.
02. Exploratory Factor Analysis (EFA)
EFA was employed to cluster correlated conditions, generating 30 factor scores for the readmissions model. In the ultimate model, the three most predictive factors were identified: factor 2 (linked to homelessness, substance dependence, and unemployment), factor 15 (associated with influenza and tumor screenings), and factor 17 (correlated with nausea, abdominal pain, and dehydration).
03. Mortality model
The patient mortality model is a dual-input feed-forward neural network that incorporates both numeric and text data. It shares most input variables with the readmissions model but benefits from an expanded feature space and text embeddings. Unlike the readmissions model’s factor-based approach, the mortality model retains raw patient condition data without additional dimensionality reduction. To account for the diversity of procedures, text data on a patient’s procedures within a year prior to admission is included. This text data undergoes text vectorization and embedding. The model combines outputs from numeric input and text embedding layers in a final prediction layer. Training utilizes an Adams optimizer with L2 regularization to prevent overfitting.
04. Readmissions model
The readmissions model employed the XGBoost library to construct a classification model based on gradient boosted trees. The modeling variables for the readmissions model encompassed factors generated through exploratory factor analysis, patient age, medications, an array of lab results, and vital sign measurements. Hyperparameter tuning for the model was performed through 5-fold cross-validation to ensure robust outcomes.
Results
Mortality model
The patient mortality model exemplifies the efficacy of neural networks combined with mixed input data in predictive modeling. This model achieved an impressive area under the receiver operating characteristic curve (AUROC) of 0.82 and an overall accuracy of 92%. Figure 1 visually represents the AUROC curve for the mortality model. These successful modeling outcomes underscore the capacity of deep learning to harness extensive, diverse data sources for enhanced predictive performance. Techniques like text embedding prove invaluable in extracting insights from unstructured data, a task that conventional algorithms often struggle with.
Readmissions model
The patient readmission model underscores the intricate challenges inherent to machine learning in the clinical healthcare domain. In comparison to the mortality model, this model exhibited suboptimal performance, with an AUROC of 0.63 and an overall accuracy of 62%. While several factors contribute to model performance, this scenario spotlights the complexities of constructing a machine learning model for an outcome with multifaceted causal mechanisms. Unlike cases where death during admission can often be causally linked to various biological, operational, or medical markers, patient readmission can arise from a much broader array of causes, including accidents and events that may not be readily measurable based solely on a patient’s prior admission data.
Benefits
AI/ML models offer numerous advantages within clinical settings, as outlined below. While the V-CHAMPS competition primarily addressed cardiovascular health, it’s important to note that similar benefits in terms of enhancing care quality, efficiency, and cost-effectiveness can be extended to a wide range of health conditions and diseases.
- Clinical practice, research, and public health policy benefit from screening at-risk populations and estimating future health outcomes.
- Clinical trials emphasize the importance of risk-treatment interactions in clinical practice.
- Risk predictions can serve as a basis for prioritizing patients for various management procedures
- Early identification of high-risk patients enhances healthcare cost-effectiveness.
Key Facts
Conclusion
AI/ML and deep learning techniques empower the utilization of extensive electronic healthcare data for the prediction of patient outcomes.
Health outcomes, intricate and challenging to model through conventional statistical approaches, can benefit significantly from AI/ML and deep learning techniques. These advanced methods harness extensive electronic healthcare data to forecast patient outcomes, offering valuable applications in clinical and research domains. AI/ML holds promise for addressing various healthcare challenges.
