Purdue University Graduate School
PurdueThesis_Runyu_clean.pdf (21.43 MB)

Learning Based Food Image Analysis - Detection, Recognition and Segmentation

Download (21.43 MB)
posted on 2023-04-25, 12:09 authored by Runyu MaoRunyu Mao

Advances of mobile and wearable technologies have enabled a wide range of new methods for dietary assessment and monitoring, such as active or passive capturing of food images of an eating scene. Compared to traditional methods, these approaches are less burdensome and can reduce biased measurements. Food image analysis, consists of food region detection, food category classification, and food segmentation, can benefit subsequent nutrient analysis. However, due to the different complexity levels of food images and inter-class similarity of food categories, it is challenging for an image-based food analysis system to achieve high performance outside of a lab setting.

In this thesis, we investigate four research topics related to image-based dietary assess- ment: (1) construction of the VIPER-FoodNet dataset, (2) food recognition, (3) nutrient integrated hierarchy food classification, and (4) weakly supervised segmentation. For topic (1), we developed a learning-based method to automatically remove non-food images for dataset construction. For topic (2), we proposed a novel two-step food recognition system that consists of food localization and hierarchical food classification. For topic (3), we de- veloped a cross-domain food classification framework that integrates nutrition information to help the classification system make better mistakes. Finally, for topic (4), a weakly- supervised segmentation system is developed which only requires image-level supervision during training.

In addition, we developed a high-quality Photoplethysmography (PPG) signal selection method for a wearable device when subjects are undergoing daily life activities, which could be used to inform the health status of the individual.


Degree Type

  • Doctor of Philosophy


  • Electrical and Computer Engineering

Campus location

  • West Lafayette

Advisor/Supervisor/Committee Chair

Fengqing M. Zhu

Additional Committee Member 2

Amy R. Reibman

Additional Committee Member 3

Jan P. Allebach

Additional Committee Member 4

Heather A. Eicher-Miller

Usage metrics



    Ref. manager