Top

Published in:

2019 | OriginalPaper | Chapter

Personalised Aesthetics with Residual Adapters

Authors : Carlos Rodríguez-Pardo, Hakan Bilen

Published in: Pattern Recognition and Image Analysis

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The use of computational methods to evaluate aesthetics in photography has gained interest in recent years due to the popularization of convolutional neural networks and the availability of new annotated datasets. Most studies in this area have focused on designing models that do not take into account individual preferences for the prediction of the aesthetic value of pictures. We propose a model based on residual learning that is capable of learning subjective, user-specific preferences over aesthetics in photography, while surpassing the state-of-the-art methods and keeping a limited number of user-specific parameters in the model. Our model can also be used for picture enhancement, and it is suitable for content-based or hybrid recommender systems in which the amount of computational resources is limited.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Non-destructively Prediction of Quality Parameters of Dry-Cured Iberian Ham by Applying Computer Vision and Low-Field MRI

next chapter An Improvement for Capsule Networks Using Depthwise Separable Convolution

Please visit https://github.com/crp94/Personalised-aesthetic-assessment-using-residual-adapters for our implementation in PyTorch.

Alif, M.A.R., Ahmed, S., Hasan, M.A.: Isolated Bangla handwritten character recognition with convolutional neural network. In: 2017 20th International Conference of Computer and Information Technology (ICCIT), pp. 1–6. IEEE, December 2017. http://ieeexplore.ieee.org/document/8281823/

Shaji, A.: Understanding aesthetics with deep learning (2016). https://devblogs.nvidia.com/understanding-aesthetics-deep-learning/

Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the International Conference on Multimedia - MM 2010, p. 271. ACM Press, New York (2010). http://dl.acm.org/citation.cfm?doid=1873951.1873990

Bianco, S., Celona, L., Napoletano, P., Schettini, R.: Predicting image aesthetics with deep learning. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 117–125. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48680-2_11CrossRef

Browniee, J.: A gentle introduction to transfer learning for deep learning (2017). https://machinelearningmastery.com/transfer-learning-for-deep-learning/

Chen, Y.L., Huang, T.W., Chang, K.H., Tsai, Y.C., Chen, H.T., Chen, B.Y.: Quantitative analysis of automatic image cropping algorithms: a dataset and comparative study. In: Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV, pp. 226–234 (2017). https://arxiv.org/pdf/1701.01480.pdf

Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_23CrossRef

Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: an exposition (2008). http://riemann.ist.psu.edu/

Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Signal Process. Mag. 34(4), 80–106 (2017). https://arxiv.org/pdf/1610.00838.pdfCrossRef

10.

Denzler, J., Rodner, E., Simon, M.: Convolutional neural networks as a computational model for the underlying processes of aesthetics perception. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 871–887. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_60CrossRef

11.

Hayn-Leichsenring, G.U., Lehmann, T., Redies, C.: Subjective ratings of beauty and aesthetics: correlations with statistical image properties in Western oil paintings (2017)

12.

Hong, L., Doumith, A.S., Davison, B.D.: Co-factorization machines. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining - WSDM 2013, p. 557. ACM Press, New York (2013). http://dl.acm.org/citation.cfm?doid=2433396.2433467

13.

Isinkaye, F., Folajimi, Y., Ojokoh, B.: Recommendation systems: principles, methods and evaluation. Egypt. Inform. J. 16(3), 261–273 (2015). https://www.sciencedirect.com/science/article/pii/S1110866515000341CrossRef

14.

Jiang, W., Loui, A.C., Cerosaletti, C.D.: Automatic aesthetic value assessment in photographic images. In: 2010 International Conference on Multimedia and Expo, pp. 920–925. IEEE, July 2010. http://ieeexplore.ieee.org/document/5582588/

15.

Jin, X., Zhao, M., Chen, X., Zhao, Q., Zhu, S.-C.: Learning artistic lighting template from portrait photographs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 101–114. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_8CrossRef

16.

Joshi, D., et al.: Aesthetics and emotions in images. IEEE Signal Process. Mag. 28(5), 94–115 (2011). http://ieeexplore.ieee.org/document/5999579/CrossRef

17.

Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation (2016). https://arxiv.org/pdf/1606.01621.pdf

18.

Leder, H., Belke, B., Oeberst, A., Augustin, D.: A model of aesthetic appreciation and aesthetic judgments. Br. J. Psychol. 95(4), 489–508 (2010). https://doi.org/10.1348/0007126042369811CrossRef

19.

Liu, Y., Zhang, Y.M., Zhang, X.Y., Liu, C.L.: Adaptive spatial pooling for image classification. Pattern Recogn. 55(C), 58–67 (2016). https://linkinghub.elsevier.com/retrieve/pii/S0031320316000510

20.

Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the ACM International Conference on Multimedia - MM 2014, pp. 457–466 (2014). http://dl.acm.org/citation.cfm?doid=2647868.2654927

21.

Luo, Y., Tang, X.: Photo and video quality evaluation: focusing on the subject. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 386–399. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88690-7_29CrossRef

22.

Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. http://refbase.cvc.uab.es/files/MMP2012a.pdf

23.

Niu, Y., Liu, F.: What makes a professional video? A computational aesthetics approach. IEEE Trans. Circuits Syst. Video Technol. 22(7), 1037–1049 (2012). http://ieeexplore.ieee.org/document/6162974/CrossRef

24.

O’Donovan, P., Agarwala, A., Hertzmann, A.: Collaborative filtering of color aesthetics. In: Proceedings of Workshop Computational Aesthetics - CAe 2014, pp. 33–40. ACM Press, New York (2014). http://dl.acm.org/citation.cfm?doid=2630099.2630100

25.

Rebuffi, S.A., Bilen, H., Vedaldi, A.: Learning multiple visual domains with residual adapters (2017). http://arxiv.org/abs/1705.08045

26.

Rebuffi, S.A., Bilen, H., Vedaldi, A.: Efficient parametrization of multi-domain deep neural networks (2018). http://arxiv.org/abs/1803.10082

27.

Ren, J., Shen, X., Lin, Z., Mech, R., Foran, D.J.: Personalized image aesthetics. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 638–647 (2017)

28.

Rothe, R., Timofte, R., Van Gool, L.: Some like it hot-visual guidance for preference prediction. Technical report (2016). http://hotornot.com

29.

Schafer, J.B., Frankowski, D., Herlocker, J., Sen, S.: Collaborative filtering recommender systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web. LNCS, vol. 4321, pp. 291–324. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72079-9_9CrossRef

30.

de Stoutz, E., Ignatov, A., Kobyshev, N., Timofte, R., Van Gool, L.: Fast perceptual image enhancement, December 2018. http://arxiv.org/abs/1812.11852

31.

Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimed. 15(8), 1930–1943 (2013). http://ieeexplore.ieee.org/document/6544270/CrossRef

32.

Vogel, D., Khan, S.S.: Evaluating visual aesthetics in photographic portraiture. In: Proceedings of Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, p. 128 (2012). https://dl.acm.org/citation.cfm?id=2328898

33.

Wang, W., Shen, J.: Deep cropping via attention box prediction and aesthetics assessment. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 2205–2213 (2017)

34.

Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 1, pp. 419–426. IEEE (2006). http://ieeexplore.ieee.org/document/1640788/

35.

Yu, W., Chen, X.: Aesthetic-based clothing recommendation 2 (2018)

Title: Personalised Aesthetics with Residual Adapters
Authors: Carlos Rodríguez-Pardo
Hakan Bilen
Publisher: Springer International Publishing
Book: Pattern Recognition and Image Analysis
Print ISBN: 978-3-030-31331-9

Electronic ISBN: 978-3-030-31332-6

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-31332-6_44

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner