Precise PM2.5 concentration forecasting is significant to environmental management and human health. Researchers currently add various parameters to deep learning models for PM2.5 concentration forecasting, but most of them ignore the problem of PM2.5 concentration diffusion. To address this issue, a deep learning model-based PM2.5 concentration forecasting method considering the diffusion process is proposed in this paper. We designed a spatial diffuser to express the diffusion process of gaseous pollutants; that is, the concentration of PM2.5 in four surrounding directions was taken as the explanatory variable. The information from the target and associated stations was then employed as inputs and fed into the model, together with meteorological features and other pollutant parameters. The hourly data from 1 January 2019 to 31 December 2019, and the central area of the Yangtze River Delta, were used to conduct the experiment. The results showed that the forecasting performance of the method we proposed is superior to that of ignoring diffusion, with an average RMSE = 8.247 μg/m3 and average R2 = 0.922 in three different deep learning models, RNN, LSTM, and GRU, in which RMSE decreased by 10.52% and R2 increased by 2.22%. Our PM2.5 concentration forecasting method, which was based on an understanding of basic physical laws and conformed to the characteristics of data-driven models, achieved excellent performance.