AI-powered app for precise nutrition measurement helps consumers maintain Mediterranean diet

12 Oct 2022 --- Researchers in Switzerland are developing an artificial intelligence (AI) and algorithm-powered app to calculate the true nutritional value of meals made in the style of the Mediterranean diet (MD).

According to the study, the inability of most apps to correctly calculate the nutritional value of the MD is a main reason for consumers ultimately rejecting calorie intake and meal-tracking apps. This new app could make it easier for people to adhere to MD guidelines, the researchers indicate.

In essence, the researchers may have solved three problems. The first is the correct evaluation of the nutritional content of MD meals, the second is accuracy in scoring MD adherence (MDA) and the third is using pictures from a smartphone to both identify and recognize the portions and sizes of foods in an image or video. 

“Adhering to the MD has been proven to be beneficial against non-communicable diseases such as cardiovascular diseases, type 2 diabetes mellitus and cancer,” say the study authors. “Therefore, to calculate the MDA score, an accurate and objective method of dietary assessment is of utmost importance.”

“To our knowledge, this is the first end-to-end fully automatic system that jointly recognizes the food items and serving sizes from a single image, calculates a compliance with a healthy diet score and provides feedback and personalized suggestions to the user.”

The images and food items were broken down into 31 different dietary categories.Intelligent estimations
According to the study, published in Nature, creating a smartphone app that could accurately identify food items, estimate likely ingredients (based on its knowledge of the MD) and the size of the portions and then calculate the nutritional value, all based on a picture of the meal to be consumed, presented significant challenges.

The researchers utilized the Gofood system developed by the University of Bern in Switzerland. The system utilized Red Green Blue-Depth images to perform food segmentation, food recognition and food volume estimation. The AI’s algorithm uses two images or a short video to create a virtual 3D image of the meal, estimate the volumes and then assign nutritional value. 

To perform this accurately, the researchers collected 11,024 images and then had 10 “annotators” – people who annotated the nutritional label values of each food item – prescribe specific annotations on food type, category and nutritional value. To ensure precision, 9,888 of the images were evaluated by at least four annotators. 

The images and food items were then further broken down into 31 different dietary categories like vegetables, fish and types of dairy – all based on expert dietician input.

“The estimations of Gofood were compared to those of dietitians, and the system outperformed the dietitians in a database consisting of European meals and had a similar performance in a fast-food dataset,” underscore the authors.

Future implementation
To test the results of the AI-powered smartphone app, 24 participants (21 men and 3 women) with a mean age of 46.9 volunteered to record their daily meals for four weeks using the app. The app’s accuracy was then tested against that of the known nutritional content. 

The app’s accuracy was then tested against that of the known nutritional content.After collecting all of the data for a week, the app generated a weekly MDA score report for the participants that focused on four main aspects of the MD. It reminded the participants of the key aspects of the MD, displayed a color-coded score of their MDAs, showed traffic light-style symbols for food categories (green = good, yellow = needs more, red = not eaten) and then provided them with detailed recommendations for improving their MDA scores.

“Adherence to MD includes frequent consumption of vegetables, fruits, nuts, cereal, legumes, and olive oil and a lower intake of eggs, red or processed meat and sweets,” the authors note. 

At the end of the four weeks, 19 out of the 24 participants had improved their MDA scores. Though some found the weekly report to be hard to understand, 20 out of 24 of the participants stated that they would continue using the app, and 23 out of 24 said that the app was easy to use. 

Regarding our future steps, we firstly intend to conduct a clinical trial with a larger number of participants coming from different countries and for a longer study period to investigate the effects of the system on the MDA and the BMI of the participants,” the authors conclude. 

“For this reason, additional data to train the food recognition and serving size estimation network will be acquired that cover a wider range of cuisines, while the network will also be able to learn based on the users’ new data.”

Edited by William Bradford Nichols