Background: Superficial perivascular dermatitis, an important type of inflammatory dermatosis, comprises various skin diseases, which are difficult to distinguish by clinical manifestations and need pathological imaging observation. Coupled with its complex pathological characteristics, the subtype classification depends to a great extent on dermatopathologists. There is an urgent need to develop an efficient approach to recognize the pathological characteristics and classify the subtypes of superficial perivascular dermatitis.Methods: 3,954 pathological images (4 × and 10 ×) of three subtypes—psoriasiform, spongiotic and interface—of superficial perivascular dermatitis were captured from 327 cases diagnosed both clinically and pathologically. The control group comprised 1,337 pathological images of 85 normal skin tissue slides taken from the edge of benign epidermal cysts. First, senior dermatologists and dermatopathologists followed the structure–pattern analysis method to label the pathological characteristics that significantly contribute to classifying different subtypes on 4 × and 10 × images. A cascaded deep learning algorithm framework was then proposed to establish pixel-level pathological characteristics` masks and classify the subtypes by supervised learning.Results: 13 different pathological characteristics were recognized, and the accuracy of subtype classification was 85.24%. In contrast, the accuracy of the subtype classification model without recognition was 71.35%.Conclusion: Our cascaded deep learning model used small samples to deliver efficient recognition of pathological characteristics and subtype classification simultaneously. Moreover, the proposed method could be applied to both microscopic images and digital scanned images.