Video processing for safe food handling

Most foodborne illnesses result from inappropriate food handling practices. One proven practice to reduce pathogens is to perform effective hand-hygiene before all stages of food handling. In food handling, there exist steps to achieve good manufacturing practices (GMPs). Traditionally, the assessment of food handling quality would require hiring a food expert for audit, which is expensive in cost. Recently, recognizing activities in videos becomes a rapidly growing field with wide-ranging applications. In this presentation, we propose to approach the assessment of hand-hygiene quality, which is a crucial step in food handling, with video analytic methods: action recognition and action detection algorithms. Our approaches focus on hand-hygiene activities with different requirements include camera views and scenario variations.

For hand-hygiene with egocentric video data, we create a two-stage system to localize and recognize all the hand-hygiene actions in each untrimmed video. This involves applying a low-cost hand mask and motion histogram features to localize the temporal regions of hand-hygiene actions. For hand-hygiene with multi-camera view video data, we design a system processes untrimmed video from both egocentric and third-person cameras, and each hand-hygiene action is recognized with its “expert” camera view. For hand-hygiene across different scenarios, we propose a multi-modality framework to recognize hand-hygiene actions in untrimmed video sequences. We use modalities such as RGB, optical flow, hand segmentation mask, and human skeleton joint modalities to construct individual CNN and apply a hierarchical method to recognize hand-hygiene action