Traffic signs play an important role in traffic management systems. A variety of studies have focused on drivers’ comprehension of traffic signs. However, the travel safety of prospective users, which has been rarely mentioned in previous studies, has attracted considerable attention from relevant departments in China. With the growth of international and interregional travel demand, traffic signs should be designed more universally to reduce the potential risks to drivers. To identify key factors that improve prospective users’ sign comprehension, this study investigated eight factors that may affect users’ performance regarding sign guessing. Two hundred and one Chinese students, all of whom intended to be drivers and none of whom had experience with daily driving after obtaining a license or visits to Germany, guessed the meanings and rated the sign features of 54 signs. We investigated the effects of selected user factors on their sign guessing performance. Additionally, the contributions of four cognitive design features to the guessability of traffic signs were examined. Based on an analysis of the relationships between the cognitive features and the guessability score of signs, the contributions of four sign features to the guessability of traffic signs were examined. Moreover, by exploring Chinese users’ differences in guessing performance between Chinese signs and German signs, cultural issues in sign design were identified. The results showed that vehicle ownership and attention to traffic signs exerted a significant influence on guessing performance. As expected, driver’s license training and the number of years in college were dominant factors for guessing performance. With regard to design features, semantic distance and confidence in guessing were two dominant factors for the guessability of signs. We suggest improving the design of signs by including vivid, universal symbols. Thus, we provide several suggestions for designing more user-friendly signs.