Current: Research -- Highlighted work -- Gait recognition

Skeleton-based gait recognition with flash lidar

In this project, we used the data collected by a single flash lidar camera for the task of gait recognition. A flash lidar provides real-time range (depth) and intensity data, using eye-safe laser. With an extensive working range, due to the high irradiance power of pulsed laser with respect to the background, the performance of a flash lidar is not degraded in the outdoor environments. With these properties, flash lidar provides new opportunities for applications such as gait recognition and activity recognition. However, the data provided by a flash lidar is plagued with noise, artifacts, and even missing measurements that challenges the existing analysis solution.

For gait recognition, we take a model-based approach by hiring a pre-trained pose detector [1]. By using camera properties and range data provided by flash lidar, we project the detected skeletons into real-world frame of reference. We address the problem of missing and noisy measurements by presenting a robust filtering mechanism that can successfully correct and recover the noisy and missing skeleton measurements. Furthermore, through an extensive set of experiments, we show that considering robust statistics along with traditional feature moments can be a better representative of the motion dynamics when extensive data correction is employed. For applications where data elimination is not an issue, we also present outlier removal for vector-based features.

The following videos exhibit the skeleton joints of a flash lidar data
Before GliarPoly filtering (left) and after GlidarPoly filtering (right).
beforeCorrection beforeCorrection
[1] Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7291-7299).
Related paper(s):
GlidarCo: gait recognition by 3D skeleton estimation and biometric feature correction of flash lidar data

A flash lidar can provide new opportunities for gait recognition; however, the lower quality and noisy imaging process of flash lidar negatively affects the performance of conventional solutions including the state-of-the-art skeleton-based systems. We present a filtering mechanism that corrects noisy and missing skeleton joint measurements to improve gait recognition ...( See more about the project)