In the field of multi-view SAR target location, the greater the difference in viewing angles, the higher the target location accuracy. However, this makes it difficult to match the same target between the SAR images with different viewing angles. The performance of traditional SAR image-matching algorithms will deteriorate or even fail to match the images correctly when the viewing angle is gradually increased. To solve this problem, a fast SAR matching method for the SAR images with large viewing angles based on inertial navigation information and neighborhood structure consensus (ININSC) is proposed in this paper. In this algorithm, the key targets are detected in the SAR images by using the maximum connected domain algorithm and the K-means clustering algorithm, and the connected domain centroid of the target is taken as a feature point. The approximate position of the key targets after the viewing angle change is found through inertial navigation information, and then accurate and fast matching is achieved by using the consensus of the neighborhood topological structure of the key targets. The measured data sets demonstrate that compared with traditional SAR image-matching algorithms, the proposed ININSC algorithm solves such a problem that SAR images cannot be accurately matched under the differences at large viewing angles and has better robustness and timeliness.