Text-aware small sample remote sensing image classification based on contrastive learning

Abstract

In view of the fact that existing methods mainly use a single modality of remote sensing images to solve the problem of low similarity of the same category, a remote sensing image classification method based on multimodal learning is proposed. Firstly, the spatial features of the image are corrected and the image encoder is pre-trained using contrastive learning to generate image features, and the text encoder is used to generate text features. Secondly, a feature decoder is introduced to obtain text-aware visual features, and a new attention mechanism method is proposed in the feature fusion stage. Then, a new image encoder is designed to improve the
classification accuracy. Finally, the similarity between the support set and the query set is calculated to further predict the category. Experiments are conducted on the NWPU-RESISC45, AID, and UC Merced datasets. Its 5-way 5-shot accuracy reaches 86.46%、85.89%and 80.32%outperforms the existing small-sample remote sensing image classification methods, respectively.