With the rapid development of railway construction and the massive exploitation of mineral resources, many railway projects have had to cross mining areas and their caverns. However, the settlement of the ground surface may cause severe damage to human-built structures and lead to the loss of human lives. The research on ground deformation monitoring over caverns is undoubtedly important and has a guiding role in railway design. Settlement observation points were set up around the mine, establishing a ground subsidence monitoring level network that has been in operation for 11 years. The ground settlement and lateral displacement along the designed railway were studied. A finite element model was established to predict the long-term ground settlements over the mined-out region induced by designed railway embankment construction and train operation. The results show that the predicted ground settlement induced by railway embankment construction is smaller than the ground settlement induced by the mined-out cavity. One train pass-by has an insignificant impact on the safety of train operation. However, when the number of train pass-bys increases to 10,000,000 times and 20,000,000 times, the cumulative deformations of the ground at different depths are quite large, which may affect the safety of the railway operation. Thus, it is necessary to deal with settlement issues when designing railway construction.