Air pollution connected to smelter activity can significantly deteriorate the quality of soil due to the precipitation of rain or simple deposition of the air particulates into the ground. Hence, in this study, we focused on the analysis of the soil which can inform us about the general state of the environment in the area and the possible health hazard for humans. If the top layer of the soil is in bad condition, it can indicate that air pollution in the area is also not in good condition, and the lives of the inhabitants of these areas can be at serious risk. To comprehensively identify the level of contamination in the soils from the areas of Polish smelters, studies of the concentration of potentially toxic elements (PTEs) in the soil were conducted. On the basis of the obtained results, health risk assessment was performed to verify the possible influence on human health. The results showed that the non-carcinogenic risk existed only for Oława, while the possibility of the carcinogenic risk occurred in all of the studied places. The outcome is very disturbing and certain steps must be undertaken in order to protect the lives of the inhabitants. Additionally, in order to assess the suitability of soils for the cultivation of edible plants, phytotoxicity tests were conducted. The obtained results revealed that in all the studied areas, a visible inhibition of roots and shoots can be observed. The conducted study indicated the most polluted regions and the possible health hazard, and it can provide the general information about the impact of smelters on the environment.