UCL scientists’ new tool detects risk of online nutrition misinformation

Researchers from the University College London (UCL), UK, have developed a Diet-Nutrition Misinformation Risk Assessment Tool (Diet-MisRAT) to detect and evaluate online nutrition misinformation’s potential harm. 

The team says this tool is the “first of its kind” as it differs from other misinformation-detecting tools, solely detecting if the content is true or false.

This tool identifies dangerously misleading content, such as missing context, half-truths, and over-hyped claims. Rather than a true or false claim, it estimates the risk level of content on a scale.

“When it comes to diet and nutrition, misinformation often operates through selective framing that masks potential health risks. Harmful, misleading content tends to fly under fact-checkers’ radars and escape meaningful oversight until high-profile cases make the headlines,” says lead author and developer Alex Ruani, a doctoral researcher at UCL.

The developers say that true or false models often fail to capture contextual and cumulative ways, and misleading health information can influence decision-making and behaviors. This is especially important for vulnerable groups such as adolescents.

Nutrition misinformation

Health misinformation spread online creates a major public health threat, according to the WHO. These can have disastrous, and sometimes fatal, outcomes.

The research team points to restrictive diets and fasting, and also consuming dietary supplements without medical advice, which are significant contributors to drug-induced liver injuries in the US.

Health misinformation spread online creates a major public health threat, according to the WHO.A previous national survey has found that nearly half of US citizens rely on unaccredited sources, social media, and AI-generated recommendations for nutrition advice rather than trained professionals. It flagged that consumers struggle to differentiate reliable data from misinformation.

The new paper has been published in Scientific Reports. The Diet-MisRAT is a rule-based content analysis model using the WHO’s approach to assessing hazardous exposures from digital environments.

The tool looks at four different dimensions: Inaccuracy — if the facts are wrong, Incompleteness — if important information is missing, Deceptiveness — if the content is framed in a misleading way, and Health Harm — if the information could lead to dangerous behavior.

It starts by analyzing online content from sources such as social media and news articles, before prompting structured questions such as whether the content poses a risk, is exaggerated, or contradicts science.

The answers given provide a score, and the higher it is, the higher the risk that the content is harmful. These scores can then rank the content based on risk classification.

The researchers say the Diet-MisRAT offers a scalable, graded alternative to binary detection and that domain-calibrated risk stratification could guide proportionate interventions in content oversight, regulation, education, misinformation inoculation, and infodemic mitigation.

Verified results

The results from the tool were tested and calibrated through five verification rounds and included judgment from over 60 specialists in public health, dietetics, and nutrition.

“It is essential to include specialist expertise when assessing misinformation risk. Our tool was calibrated and validated with feedback from nearly 60 subject-matter experts. This helps ensure that assessments of potential harm reflect appropriate professional judgment,” says co-author and professor Anastasia Kalea at UCL’s Division of Medicine.

The study stresses that misinformation has been implicated in decisions to abandon life-saving treatments.The study stresses that misinformation has also been implicated in decisions to abandon life-saving treatments, such as patients with curable cancers opting for unproven dietary alternatives, an approach linked to two-fold higher mortality rates.

If further exemplifies: “Emerging clinical reports add to the evidence of misinformation’s dangers. One involved a man with alarming cholesterol-induced skin lesions from a carnivore diet, a trend amplified among manosphere subcultures.”

“Another described hazardous metallic layering in a man’s colon from ingesting colloidal silver drops touted in some naturopathic circles. A recent fatality involved an adolescent girl who died after adhering to a water-only fasting regime she reportedly discovered online.”

The study taps into an ongoing debate about online misinformation and how public authorities and policymakers should address this problem. The tool’s risk assessment criteria can be taught and applied for educational and professional training, say the authors.

“In public health, we assess exposure to risk factors. We believe misleading health information should be treated in the same way. Some misinformation can lead to serious harm, so mitigation strategies should be proportionate to the level of risk. The more severe the potential harm, the stronger the response should be,” says Ruani.

“When AI chatbots speak confidently, users may assume their advice is safe. If we can properly measure how misleading a piece of advice is and how much harm it may pose, we can build stronger safeguards into models and AI agents before deployment rather than reacting after harm occurs.”

A previous study on nutrition misinformation found that following the health advice of 53 social media “super-spreaders” could put up to 24 million people at risk of serious health consequences.