How dangerous is trusting AI to save lives in an emergency? ChatGPT health study increases tension

Are you in medical emergency? AI Can you trust? While tools like ChatGPT are making health information easier, a new study has issued a serious warning regarding their use. Research published in Nature Medicine suggests that AI can underestimate the danger in many serious cases, which can cost the patient’s life.

What is the claim in the study?

According to new research, AI tool ChatGPT Health was sometimes unable to correctly identify the seriousness of emergency cases. Particularly in high-risk situations, it recommended a lower level of urgency than expected, which increases the risk of delaying treatment.

How was the research done and how big was the trial?

This study was done by researchers from Icahn School of Medicine at Mount Sinai. In this, 60 medical scenarios (vignettes) were created and they were tested in 16 different situations. A total of 960 AI responses were analyzed to understand how accurately AI determines triage (priority of treatment).

How was the performance of AI?

Research revealed an “inverted U-shaped” pattern. That is, AI performed well in moderate diseases, but its accuracy fell in serious emergencies. In gold-standard emergency cases, AI under-triaged about 52% of the cases. That is, where one should have gone to the hospital immediately, less serious advice was given.

Which dangerous cases were missed?

The study revealed that AI underestimated many life-threatening conditions, such as diabetic ketoacidosis and impending respiratory failure. In these cases, AI advised to get tested within 24–48 hours instead of immediate emergency, which can be extremely risky.

What was found in mental health emergency?

In the Nature Medicine study, the response of AI was not the same even in mental health cases. When given a clear method of suicide, the AI ​​suggested crisis help (such as a helpline) less often. This raises questions about trust in high-risk situations.

Is AI affected by external factors?

“Anchoring bias” came to light in the research. If family or friends in the case were describing the symptoms as mild, then the AI ​​also leaned in the same direction and started giving less important advice. That means words in context can greatly influence AI’s decisions.

What do OpenAI and experts say?

According to the study, OpenAI welcomed the study, but said that it does not show the complete picture of the real world. The company clearly says that ChatGPT Health is only for general information, it cannot take the place of a doctor.

Experts believe that AI can increase access to healthcare, but relying completely on it without clinical validation in emergency decisions can be dangerous. AI is definitely the future of healthcare, but it is not completely reliable yet. Especially in emergency. The right path is to use AI as a supporting tool, not as the final decision maker.