This study provides an empirical analysis of household technical efficiency and its determinant factors (especially conservation payments) in the context of the Grain for Green program. On the basis of a sample of 225 farm households on the Loess Plateau in 2007, we estimate household technical efficiency using the data envelopment analysis method. In addition to a traditional ordinary least square (OLS) analysis, quantile regression (QR) analysis is also deployed to explore the possible heterogeneous effects of conservation payments and other variables on the technical efficiency across the quantiles. The results suggest that when off-farm activities are taken into account, households have considerable potential for improving their technical efficiency; OLS analysis shows that conservation payments decrease household efficiency, and the QR analysis suggests that the negative impact is significant only for higher performance households; The presence of children, access of households to leased land markets, credit markets, and extension services all show heterogeneous impacts on household efficiency. On the basis of the findings of the study, policies suggestions to improve the program’s effectiveness are provided.