Extracting farmland from high spatial resolution remote sensing images is a basic task for agricultural information management. According to Tobler’s first law of geography, closer objects have a stronger relation. Meanwhile, due to the scale effect, there are differences on both spatial and attribute scales among different kinds of objects. Thus, it is not appropriate to segment images with unique or fixed parameters for different kinds of objects. In view of this, this paper presents a stratified object-based farmland extraction method, which includes two key processes: one is image region division on a rough scale and the other is scale parameter pre-estimation within local regions. Firstly, the image in RGB color space is converted into HSV color space, and then the texture features of the hue layer are calculated using the grey level co-occurrence matrix method. Thus, the whole image can be divided into different regions based on the texture features, such as the mean and homogeneity. Secondly, within local regions, the optimal spatial scale segmentation parameter was pre-estimated by average local variance and its first-order and second-order rate of change. The optimal attribute scale segmentation parameter can be estimated based on the histogram of local variance. Through stratified regionalization and local segmentation parameters estimation, fine farmland segmentation can be achieved. GF-2 and Quickbird images were used in this paper, and mean-shift and multi-resolution segmentation algorithms were applied as examples to verify the validity of the proposed method. The experimental results have shown that the stratified processing method can release under-segmentation and over-segmentation phenomena to a certain extent, which ultimately benefits the accurate farmland information extraction.