by Ruggiero Corcella
A study conducted in Spain shows that Ai’s tool answered the 50 most frequently asked questions on the topic with 85% accuracy. For experts ChatGPT is a «reliable source of non-technical information for the public»
On the contribution (detrimental and heavy) of social media in spreading fake news, about Covid and other epidemics and disasters, rivers of ink have been spilled. The latest systematic review by the World Health Organization across four studies it analyzed the rate of health misinformation on social media, finding that it reached up to 51% in posts associated with vaccines, up to 28.8% in posts associated with Covid, and up to 60% in posts related to pandemics. Among YouTube videos on emerging infectious diseases, 20-30% were found to contain inaccurate or misleading information.
The “infodemic” has certainly been favored by the use of artificial intelligence tools for the dissemination of information without scientific foundation. But Ai itself has been used to expose fake news and, ultimately, ChatGPT could also help increase the spread of vaccinations by debunking myths about vaccine safety.
Fifty questions
At least this is what the authors of a study published in the journal Human Vaccines and Immunotherapeutics say. Researchers asked the artificial intelligence (AI) chatbot the 50 most frequently asked questions about the Covid-19 vaccine. They included questions based on myths and false stories such as the vaccine causing Long Covid. The results show that ChatGPT scored an average of nine out of 10 points in accuracy. According to the study, however, there would be some gaps in the information provided.
Based on these results, the experts who conducted the study from the GenPoB research group based at the Instituto de Investigación Sanitaria (IDIS) – Hospital Clinico Universitario de Santiago de Compostela, state that the AI tool is a “reliable source of non-technical information for the public’, especially for people without specialist scientific knowledge.
Sometimes, however, ChatGPT «changes its mind»
However, the findings highlight some concerns about the technology such as ChatGPT changing its responses in certain situations. “Overall, ChatGPT constructs a narrative in line with the available scientific evidence, debunking the myths circulating on social media,” says lead author Antonio Salas, who in addition to leading the GenPoB research group, is also a professor at the Faculty of Medicine, University of Santiago de Compostela, Spain.
“Doing so potentially facilitates an increase in vaccinations. ChatGPT can detect forged questions related to vaccines and vaccinations. The language used by this AI is not too technical and therefore easily understandable to the public but without losing scientific rigour”. With an important addition: «We recognize that the current version of ChatGPT cannot replace an expert or scientific evidence. But the findings suggest it could be a reliable source of information for the public.”
Hesitancy to get vaccinated, threat to global health
In 2019, The World Health Organization (WHO) has listed vaccine hesitancy among the top 10 threats to global health. During the pandemic, misinformation spread via social media has helped fuel public mistrust of the Covid-19 vaccination. Among the authors of this study are those of the Hospital Clinico Universitario de Santiago, which WHO designated a Collaborating Center for Vaccine Safety in 2018.
Researchers at the Center explored vaccine safety myths and medical situations that are incorrectly believed to be reasons not to get vaccinated. These misplaced concerns contribute to vaccine hesitancy.
The topics addressed
The study authors set out to test ChatGPT’s ability to correctly interpret facts and share accurate information on Covid vaccine safety in line with current scientific evidence. ChatGPT allows people to have human-like conversations and interactions with a virtual assistant. The technology is very intuitive and makes it accessible to a large population. However, many governments are concerned about the potential fraudulent use of ChatGPT in educational settings such as universities. The study was designed to challenge the chatbot by asking it the most frequently asked questions received from the WHO Collaborating Center in Santiago.
The questions concerned three topics. The first is about misconceptions about safety, such as the one that the vaccine caused Long Covid. Then there were the false contraindications: medical situations where the vaccine is safe to use, such as in breastfeeding women. Questions also concerned real contraindications – a health condition for which the vaccine should not be used – and cases where doctors need to take precautions, such as a patient with inflammation of the heart muscle.
Accurate answers
Next, the experts analyzed the responses and then evaluated them for truthfulness and accuracy against current scientific evidence and recommendations from WHO and other international agencies. The authors say this is important because the algorithms created by social media and Internet search engines are often based on an individual’s habitual preferences. This could lead to “partial or wrong answers,” they add.
The results showed that most of the questions were answered correctly with an average score of nine out of 10, defined as ‘excellent’ or ‘good’. Responses to the three question topics were on average 85.5% or 14.5% accurate but with gaps in the information provided. ChatGPT provided correct answers to questions arising from real vaccine myths and to those considered in clinical recommendations guidelines as true or false contraindications. However, the research team highlights the disadvantages of ChatGPT in providing vaccine information.
It limits the ChatGPT
Professor Salas, a specialist in human genetics concludes: «ChatGPT provides different answers if the question is repeated “with a few seconds delay”. Another concern we’ve encountered is that this AI tool, in its current form, could also be trained to provide answers that don’t align with scientific evidence.”
“You can “torture” the system into giving the desired answer. This also applies to other contexts other than vaccines. For example, it might be possible to align the chatbot with absurd narratives such as the flat earth theory, deny climate change or oppose the theory of evolution, just to give a few examples.
However, it is important to note that these responses are not the default behavior of ChatGPT. Therefore, the results we have obtained regarding the safety of vaccines can probably be extrapolated to many other myths and pseudoscience,” concludes Salas.
September 4, 2023 (change September 4, 2023 | 14:48)
© breaking latest news