Automatic ground filtering is an essential step for Digital Elevation Model (DEM) generation, which has significant application value. However, extraction and classification of ground points from the Light Detection and Ranging (LiDAR) data, especially in multitudinous terrain situations, is a challenging task because it is difficult to determine the set of optimal parameters for removing various non-ground features. In this paper, a new ground filtering technique based on an improved Ball Pivot Algorithm (BPA) is proposed. At the beginning, the LiDAR point cloud dataset was divided into different subsets based on the 2D regular grid. The lowest point in each grid was selected as the seed point to build a single-layer surface. After that, the improved BPA was executed to remove points on the higher location. Then, the rest of the points were calculated and selected as a new seed point according to the spatial relationship with the initial surface. Finally, non-ground points were filtered by means of improved BPA traversing all the grids. Our experimental results on the Benchmark dataset provided by the International Society for Photogrammetry and Remote Sensing (ISPRS) Working Group III/3 showed high accuracy (with a mean kappa coefficient over 80%) in terms of completeness, correctness, and quality for DEM generation. The experimental results demonstrated the proposed method is robust to various terrain situations, as it is more effective and feasible for ground filtering.