In signal array processing, high-resolution direction-of-arrival (DOA) estimation algorithms work well on the assumption that the system models are perfect. However, in practicality, there are imperfect system models in which sensor gain-and-phase errors are considered. In this paper, we propose a novel framework that can effectively solve direction-of-arrival estimation tasks in the presence of sensor gain-and-phase errors. In contrast to existing approaches based on phase retrieval, our method eliminates gain errors by using the compensated covariance matrix. Meanwhile, we propose a data preprocessing method by taking only one column of the compensated covariance matrix without losing any magnitude information. Additionally, the phase retrieval problem is formed by the proposed data preprocessing method. Furthermore, the phase retrieval problem is solved by the recently proposed sparse feasible point pursuit algorithm, and DOA estimates are obtained. To prevent the model from ambiguities, we employ the known DOA to place reference sources. Numerical results show that the proposed scheme achieves better performance compared to state-of-the-art approaches.