Entropy, Vol. 25, Pages 891: Improving Semantic Segmentation via Decoupled Body and Edge Information (Entropy)

In this paper, we propose a method that uses the idea of decoupling and unites edge information for semantic segmentation. We build a new dual-stream CNN architecture that fully considers the interaction between the body and the edge of the object, and our method significantly improves the segmentation performance of small objects and object boundaries. The dual-stream CNN architecture mainly consists of a body-stream module and an edge-stream module, which process the feature map of the segmented object into two parts with low coupling: body features and edge features. The body stream warps the image features by learning the flow-field offset, warps the body pixels toward object inner parts, completes the generation of the body features, and enhances the object’s inner consistency. In the generation of edge features, the current state-of-the-art model processes information such as color, shape, and texture under a single network, which will ignore the recognition of important information. Our method separates the edge-processing branch in the network, i.e., the edge stream. The edge stream processes information in parallel with the body stream and effectively eliminates the noise of useless information by introducing a non-edge suppression layer to emphasize the importance of edge information. We validate our method on the large-scale public dataset Cityscapes, and our method greatly improves the segmentation performance of hard-to-segment objects and achieves state-of-the-art result. Notably, the method in this paper can achieve 82.6% mIoU on the Cityscapes with only fine-annotated data.