AI can predict if you'll die soon – but we've no idea how it works
Artificial intelligence can predict a person’s chances of dying within a year by looking at heart test results – even when they look normal to doctors. How it does so is a mystery.
Brandon Fornwalt at healthcare provider Geisinger in Pennsylvania, US and colleagues tasked an AI with examining 1.77 million electrocardiogram (ECG) results from nearly 400,000 people to predict who was at a higher risk of dying within the next year.
An ECG records the electrical activity of the heart. Its pattern changes in cardiac conditions including heart attacks and atrial fibrillation.
The team trained two versions of the AI: in one, the algorithm was only given the raw ECG data, which measures voltage over time. In the other, it was fed ECG data in combination with patient age and sex.
They measured the AI’s performance using a metric known as AUC, which measures how well a model distinguishes between two groups of people – in this case, patients who died within a year and those who survived. The AI consistently scored above 0.85, where a perfect score is 1 and a score of 0.5 indicates no distinction between the two groups.
The AUCs for risk scoring models currently used by doctors range between 0.65 and 0.8, says Fornwalt.
For comparison, the researchers also created an algorithm based on ECG features that doctors currently measure, such as certain patterns from the recordings.
“No matter what, the voltage-based model was always better than any model you could build out of things that we already measure from an ECG,” says Fornwalt.
The AI accurately predicted risk of death even in people deemed by cardiologists to have a normal ECG. Three cardiologists who separately reviewed normal-looking ECGs weren’t able to pick up the risk patterns that the AI detected.
“That finding suggests that the model is seeing things that humans probably can’t see, or at least that we just ignore and think are normal,” says Fornwalt.
“AI can potentially teach us things that we’ve been maybe misinterpreting for decades,” he says.
It’s still unclear what patterns the AI is picking up, which makes some physicians reluctant to use such algorithms.
This research was based on historical data, and it will be important to demonstrate in clinical studies that such an algorithm improves patient outcomes, says collaborator Christopher Haggerty.
The research will be presented at the American Heart Association’s Scientific Sessions in Dallas on November 16.