Effective and rapid assessment of pork freshness is significant for monitoring pork quality. However, a traditional sensory evaluation method is subjective and physicochemical analysis is time-consuming. In this study, the near-infrared spectroscopy (NIRS) technique, a fast and non-destructive analysis method, is employed to determine pork freshness. Considering that commonly used statistical modeling methods require preprocessing data for satisfactory performance, this paper presents a one-dimensional squeeze-and-excitation residual network (1D-SE-ResNet) to construct the complex relationship between pork freshness and NIRS. The developed model enhances the one-dimensional residual network (1D-ResNet) with squeeze-and-excitation (SE) blocks. As a deep learning model, the proposed method is capable of extracting features from the input spectra automatically and can be used as an end-to-end model to simplify the modeling process. A comparison between the proposed method and five popular classification models indicates that the 1D-SE-ResNet achieves the best performance, with a classification accuracy of 93.72%. The research demonstrates that the NIRS analysis technique based on deep learning provides a promising tool for pork freshness detection and therefore is helpful for ensuring food safety.