The external acceleration of a fast-moving body induces uncertainty in attitude determination based on inertial measurement unit (IMU) signals and thus, frequently degrades the determination accuracy. Although previous works adopt acceleration-compensating mechanisms to deal with this problem, they cannot completely eliminate the uncertainty as they are, inherently, approaches to an underdetermined problem. This paper presents a novel constraint-augmented Kalman filter (KF) that eliminates the acceleration-induced uncertainty for a robust IMU-based attitude determination when IMU is attached to a constrained link. Particularly, this research deals with an acceleration-level kinematic constraint derived on the basis of a ball joint. Experimental results demonstrate the superiority of the proposed constrained KF over the conventional unconstrained KF: The average accuracy improved by 1.88° with a maximum of 4.18°. More importantly, whereas the accuracy of conventional KF is dependent to some extent on test acceleration conditions, that of the proposed KF is independent of these conditions. Due to the robustness of the proposed KF, it may be applied when accurate attitude estimation is needed regardless of dynamic conditions.