The observed radiation data from the second-generation Hyperspectral Infrared Atmospheric Sounder (HIRAS-II) on the Fengyun-3E (FY-3E) satellite contain useful vertical atmosphere information which can distinguish and retrieve vertical profiles of atmospheric gas components including ozone (O3), carbon monoxide (CO), and methane (CH4). This paper utilizes FY-3E/HIRAS-II observational data to optimize each gas channel using the improved Optimal Sensitivity Profile method (OSP) channel algorithm and establishes a typical convolutional neural network model (CNN) and a representative U-shaped network model (UNET) with deep features and shallow feature links to perform atmospheric profile retrieval calculations of O3, CO, and CH4. We chose the clear sky data of the Indian and its southern seas in December 2021 and January 2022, with reanalysis data from European Center for Medium-Range Weather Forecasts Reanalysis v5 (ERA5) and European Center for Medium-Range Weather Forecasts Atmospheric Composition Reanalysis v4 (EAC4) serving as the reference values. The retrieval outcomes were then compared against advanced numerical forecast models including the Whole Atmosphere Community Climate Model (WACCM), Global Forecast System (GFS), and satellite products from an Atmospheric Infrared Sounder (AIRS) and Infrared Atmospheric Sounding Interferometer (IASI). Experimental results show that the generalization ability and retrieval accuracy of CNN are slightly higher compared with UNET. For O3 profile retrieval, the mean percentage error (MPE) of the whole layers for CNN and UNET data in relation to ERA5 data was less than 8%, while the root-mean-square error (RMSE) was below 1.5 × 10−7 kg/kg; for CH4 profile retrieval, the MPE of the whole layers for CNN and UNET data in relation to EAC4 data was less than 0.7%, while the RMSE was below 1.5 × 10−8 kg/kg. The retrieval of O3 and CH4 are resulted in a significant improvement compared to the forecast data and satellite products in most pressure levels; for CO profile retrieval, the MPE of the whole layers for CNN and UNET data in relation to EAC4 data was less than 11%, while the RMSE was below 4 × 10−8 kg/kg. The error of the CO retrieval results was higher than that of the forecast data at the pressure level of 200~500 hPa and lower than that of similar satellite products with most pressure levels. The experiments indicated that the neural network method effectively determines the atmospheric gas profiles using infrared hyperspectral data, exhibiting a positive performance in accuracy and retrieval speed.