The normal and disordered people balance ability classification is a key premise for rehabilitation training. This paper proposes a multi-barycentric area model (MBAM), which can be applied for accurate video analysis based classification. First, we have invited fifty-three subjects to wear an HTC (High Tech Computer Corporation) VIVE (Very Immersive Virtual Experience) helmet and to walk ten meters while seeing a virtual environment. The subjects’ motion behaviors are collected as our balance ability classification dataset. Secondly, we use background differential algorithm and bilateral filtering as the preprocessing to alleviate the video noise and motion blur. Inspired by the balance principle of a tumbler, we introduce a MBAM model to describe the body balancing condition by computing the gravity center of a triangle area, which is surrounded by the upper, middle and lower parts of the human body. Finally, we can obtain the projection coordinates according to the center of gravity of the triangle, and get the roadmap of the subjects by connecting those projection coordinates. In the experiments, we adopt four kinds of metrics (the MBAM, the area variance, the roadmap and the walking speed) innumerical analysis to verify the effect of the proposed method. Experimental results show that the proposed method can obtain a more accurate classification for human balance ability. The proposed research may provide potential theoretical support for the clinical diagnosis and treatment for balance dysfunction patients.