Image aesthetics assessment using composite features from transformer and CNN

As a popular research problem in computational aesthetics, image aesthetic assessment has many important applications in image editing, retrieval, and recommendation. However, the existing mainstream CNN-based image aesthetic assessment methods are difficult to obtain the global aesthetic attributes of images well. To this end, we propose a two-stream image aesthetic assessment model that couples Transformer and CNN features. We use the traditional CNN network to extract the image’s local aesthetic feature in the first stream, apply the superpixel algorithm to segment the image, and then feed the segmented image region into the Transformer network to learn the image’s aesthetic global features in the second stream. Finally, the features learned by Transformer and CNN are fused to achieve the image aesthetic assessment. The experimental results on the AVA dataset show that our proposed method can obtain local and global aesthetic information on images, which enables the model to learn richer aesthetic information, and the combination of whole and part is more in line with human aesthetic characteristics. Our proposed model achieves an accuracy of 84.5% in the classification task, achieving optimal performance compared to existing methods and good performance in the other two tasks (Score Regression and Distribution).