Vitamin B12 (B12) is involved as a cofactor in the synthesis of myelin. A lack of B12 impairs peripheral nerve production, which can contribute to sarcopenia. In this cross-sectional study, we aimed to investigate the relationship between B12 insufficiency and sarcopenia in community-dwelling older Korean adults. A total of 2325 (1112 men; 1213 women) adults aged 70–84 years were recruited. The tools used for sarcopenia were based on the Asian Working Group for Sarcopenia (AWGS) guidelines. Individuals with low appendicular skeletal muscle mass index (ASMI) (<7.0 kg/m2 for men; <5.4 kg/m2 for women) and low hand grip strength (HGS) (<28 kg for men; <18 kg for women) were defined as the sarcopenia group. Among this group, those who showed low physical performance (≤9 points on the Short Physical Performance Battery (SPPB)) were defined as the severe sarcopenia group. B12 concentrations were classified into insufficient (<350 pg/mL) and sufficient (≥350 pg/mL). Univariate and multivariate logistic regression analyses were used to evaluate the relationship between sarcopenia and B12 levels. Low ASMI showed a high incidence in the B12-insufficient group. However, HGS, SPPB, and the severity of sarcopenia showed no correlation with B12. Further, insufficient B12 may affect muscle quantity rather than muscle strength or physical performance.