A recent study published in JAMA Pediatrics has raised concerns about the use of artificial intelligence chatbots in diagnosing medical conditions in children. The study found that one AI system, ChatGPT, had only a 17% accuracy rate in diagnosing childhood diseases, leading researchers to conclude that the clinical experience of pediatricians is irreplaceable.
The study was conducted at Cohen Children’s Medical Center in New York, where researchers tested ChatGPT’s ability to diagnose 100 real children’s health cases. The results were less than promising, with the AI system only getting the correct diagnosis in 17 cases, while being completely wrong in 72 cases.
One of the main issues identified by researchers was ChatGPT’s tendency to provide overly generalized diagnoses and a lack of connection between symptoms and medical conditions. In one instance, the chatbot misdiagnosed a child with autism as having scurvy, which is a rare condition typically caused by a severe lack of vitamin C.
The findings have led researchers to emphasize the importance of training and tuning AI systems with specialized medical data to improve diagnostic accuracy. They believe that with better training, chatbots could become more reliable in diagnosing pediatric diseases.
Despite the low effectiveness of AI chatbots in diagnosing children’s illnesses, the integration of artificial intelligence into healthcare is still seen as imminent. AI technology is advancing rapidly and is being used for a wide range of applications, including analyzing large volumes of medical data, developing algorithms for more accurate diagnoses, personalizing treatments for patients, and automating administrative tasks to improve the efficiency of health services.
However, it is clear from the recent study that when it comes to diagnosing childhood diseases, the clinical experience of pediatricians is essential. The study has highlighted the limitations of current AI systems in this area and has emphasized the need for further research and development to improve the reliability of chatbots in diagnosing pediatric illnesses.