Mobility management is a transportation policy aiming to change travel behavior from car use to sustainable transportation modes while increasing people’s physical activity. Providing pedometers and visualizing step counts, popular interventions in public health practice, may constitute a mobility management program. However, the ease of modal shifts and changeability of walking habits differ across neighborhood environments. Using questionnaire data from 2023 middle-aged and older participants from Yokohama, Japan, in May 2017, this study examined (1) the relationship between the physical and social environments of Yokohama Walking Point Program participants who volunteered to use free pedometers and their modal shifts from cars to walking and public transport, and (2) whether participants’ modal shifts were associated with increases in step counts. Multivariate categorical regression analyses identified the frequency of greetings and conversations with neighbors as well as health motivation as important explanatory variables in both analyses. Participants living in neighborhoods far from railway stations and in neighborhoods with a high bus stop density tended to shift to walking and public transport, a modal shift that was highly associated with increased step counts. These results suggest that mobility management should be promoted in collaboration with public health and city planning professionals.