Research on Three-Dimensional Pose Reconstruction of Experimental Mice with Depth Information Fusion

Le résumé fourni par la source

Abstract The 2D video analysis technology has propelled research in rodent behavioral studies towards automated and high-fidelity analysis of behavioral data. Precise posture detection serves as a crucial prerequisite for behavior quantification. However, 2D video analysis falls short in determining the three-dimensional pose of mice. Existing methods for three-dimensional pose reconstruction often rely on multi-view cameras, which are costly and lack real-time capabilities. Therefore, this paper introduces a method that integrates depth information for three-dimensional pose reconstruction of experimental mice. The approach involves constructing a skeletal model of experimental mouse poses based on multiple keypoints. Utilizing a single low-cost depth camera, the Intel RealSense D455, data is collected from a top-down perspective as RGB-D data. To address the keypoint detection challenge, this study designs and improves the YOLOv7 network model. Omni-dimensional dynamic convolutions and coordinate attention mechanisms are respectively introduced into the feature extraction and fusion layers to enhance detection accuracy. Ghost convolutions are incorporated into the feature extraction layer to achieve a lightweight network design. Through ablation experiments, this network model achieves high precision and real-time keypoint detection performance on RGB images. The network model attains an average PCK (Probability of Correct Keypoint) of 97.71$\%$ for ten mouse keypoints, with a real-time frame rate of 70.47 frames. Subsequently, the depth information provided by the depth camera is optimized. Two-dimensional keypoint coordinates are employed as indices to retrieve corresponding depth values, completing the three-dimensional pose reconstruction. The reconstructed three-dimensional poses are then utilized for the analysis of mouse behavior in the vertical dimension during open field experiments. This research method offers a more comprehensive and accurate representation of experimental mouse motion, posture, and behavior, thereby contributing to a more detailed and profound understanding of behavioral studies.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.