Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

Published in: Business & Information Systems Engineering 2/2020

31-01-2019 | Research Paper

A Reinforcement Learning Based Model for Adaptive Service Quality Management in E-Commerce Websites

Authors: Hoda Ghavamipoor, S. Alireza Hashemi Golpayegani

Published in: Business & Information Systems Engineering | Issue 2/2020

Login to get access
share
SHARE

Abstract

Providing high-quality service to all users is a difficult and inefficient strategy for e-commerce providers, especially when Web servers experience overload conditions that cause increased response time and request rejections, leading to user frustration and reduced revenue. In an e-commerce system, customer Web sessions have differing values for service providers. These tend to: give preference to customer Web sessions that are likely to bring more profit by providing better service quality. This paper proposes a reinforcement-learning based adaptive e-commerce system model that adapts the service quality level for different Web sessions within the customer’s navigation in order to maximize total profit. The e-commerce system is considered as an electronic supply chain which includes a network of basic e- providers used to supply e-commerce services for end customers. The learner agent noted as e-commerce supply chain manager (ECSCM) agent allocates a service quality level to the customer’s request based on his/her navigation pattern in the e-commerce Website and selects an optimized combination of service providers to respond to the customer’s request. To evaluate the proposed model, a multi agent framework composed of three agent types, the ECSCM agent, customer agent (buyer/browser) and service provider agent, is employed. Experimental results show that the proposed model improves total profits through cost reduction and revenue enhancement simultaneously and encourages customers to purchase from the Website through service quality adaptation.
Literature
go back to reference Aciar S, Zhang D, Simoff S, Debenham J (2007) Informed recommender: basing recommendations on consumer product reviews. IEEE Intell Syst 22(3):39–47 CrossRef Aciar S, Zhang D, Simoff S, Debenham J (2007) Informed recommender: basing recommendations on consumer product reviews. IEEE Intell Syst 22(3):39–47 CrossRef
go back to reference Al-Masri E, Mahmoud QH (2007) QoS-based discovery and ranking of web services. In: 16th International conference on computer communications and networks. IEEE, pp 529–534 Al-Masri E, Mahmoud QH (2007) QoS-based discovery and ranking of web services. In: 16th International conference on computer communications and networks. IEEE, pp 529–534
go back to reference Bathumalai G (2008) Self-adapting websites: mining user access logs. Dissertation, Robert Gordon University, Aberdeen Bathumalai G (2008) Self-adapting websites: mining user access logs. Dissertation, Robert Gordon University, Aberdeen
go back to reference Belk M, Fidas C, Germanakos P, Samaras G (2015) Do human cognitive differences in information processing affect preference and performance of CAPTCHA? Int J Hum Comput Stud 84:1–18 CrossRef Belk M, Fidas C, Germanakos P, Samaras G (2015) Do human cognitive differences in information processing affect preference and performance of CAPTCHA? Int J Hum Comput Stud 84:1–18 CrossRef
go back to reference Bellifemine F, Poggi A, Rimassa G (2001) JADE: a FIPA2000 compliant agent development environment. In: 5th International conference on autonomous agents. ACM, pp 216–217 Bellifemine F, Poggi A, Rimassa G (2001) JADE: a FIPA2000 compliant agent development environment. In: 5th International conference on autonomous agents. ACM, pp 216–217
go back to reference Bhatti N, Friedrich R (1999) Web server support for tiered services. IEEE Netw 13(5):64–71 CrossRef Bhatti N, Friedrich R (1999) Web server support for tiered services. IEEE Netw 13(5):64–71 CrossRef
go back to reference Brusilovsky P, Kobsa A, Nejdl W (2007) The adaptive web. Lecture notes in computer science, vol 4321. Springer, Cham, pp 325–341 Brusilovsky P, Kobsa A, Nejdl W (2007) The adaptive web. Lecture notes in computer science, vol 4321. Springer, Cham, pp 325–341
go back to reference Chan NN, Gaaloul W, Tata S (2012) A recommender system based on historical usage data for web service discovery. Serv Oriented Comput Appl 6(1):51–63 CrossRef Chan NN, Gaaloul W, Tata S (2012) A recommender system based on historical usage data for web service discovery. Serv Oriented Comput Appl 6(1):51–63 CrossRef
go back to reference Chen M, Ryu YU (2013) Facilitating effective user navigation through website structure improvement. IEEE Trans Knowl Data Eng 25(3):571–588 CrossRef Chen M, Ryu YU (2013) Facilitating effective user navigation through website structure improvement. IEEE Trans Knowl Data Eng 25(3):571–588 CrossRef
go back to reference Chen CC, Huang TC, Park JJ, Yen NY (2015) Real-time smartphone sensing and recommendations towards context-awareness shopping. Multimed Syst 21(1):61–72 CrossRef Chen CC, Huang TC, Park JJ, Yen NY (2015) Real-time smartphone sensing and recommendations towards context-awareness shopping. Multimed Syst 21(1):61–72 CrossRef
go back to reference Ewing JM, Menascé DA (2009) Business-oriented autonomic load balancing for multi-tiered Web sites. In: IEEE international symposium on modeling, analysis and simulation of computer and telecommunication systems. IEEE, pp 1–10 Ewing JM, Menascé DA (2009) Business-oriented autonomic load balancing for multi-tiered Web sites. In: IEEE international symposium on modeling, analysis and simulation of computer and telecommunication systems. IEEE, pp 1–10
go back to reference Ghavamipoor H, Golpayegani SAH (2013) Comparing and applying the approach of supply chain in electronic services management. Int J Comput Inf Technol 1(2):118–136 Ghavamipoor H, Golpayegani SAH (2013) Comparing and applying the approach of supply chain in electronic services management. Int J Comput Inf Technol 1(2):118–136
go back to reference Ghavamipoor H, Golpayegani SAH (2016) QoS-aware provider selection in e-services supply chain. In: 8th International conference on information and knowledge technology. IEEE, pp 258–262 Ghavamipoor H, Golpayegani SAH (2016) QoS-aware provider selection in e-services supply chain. In: 8th International conference on information and knowledge technology. IEEE, pp 258–262
go back to reference Ghavamipoor H, Golpayegani SAH (2017) A QoS sensitive model for e-commerce customer behavior. J Res Interact Mark 11(4):380–397 CrossRef Ghavamipoor H, Golpayegani SAH (2017) A QoS sensitive model for e-commerce customer behavior. J Res Interact Mark 11(4):380–397 CrossRef
go back to reference Harini N, Padmanabhan TR (2013) Admission control and request scheduling for secured-concurrent-available architecture. Int J Comput Appl 63(6):24–30 Harini N, Padmanabhan TR (2013) Admission control and request scheduling for secured-concurrent-available architecture. Int J Comput Appl 63(6):24–30
go back to reference Hong J, Suh EH, Kim J, Kim S (2009) Context-aware system for proactive personalized service based on context history. Expert Syst Appl 36(4):7448–7457 CrossRef Hong J, Suh EH, Kim J, Kim S (2009) Context-aware system for proactive personalized service based on context history. Expert Syst Appl 36(4):7448–7457 CrossRef
go back to reference Lakshmi MS, Kumar SP, Janardhan M, Gayathri K (2017) Machine learning methods for refining SLA based admission control and resource allocation in cloud computing. Int J Adv Res Comput Sci 8(9):834–840 CrossRef Lakshmi MS, Kumar SP, Janardhan M, Gayathri K (2017) Machine learning methods for refining SLA based admission control and resource allocation in cloud computing. Int J Adv Res Comput Sci 8(9):834–840 CrossRef
go back to reference Larisa G, Mariia S, Andriy R (2014) Control strategy of the input stream on the online charging system in peak load moments. In: 24th International crimean conference microwave and telecommunication technology. IEEE, pp 312–313 Larisa G, Mariia S, Andriy R (2014) Control strategy of the input stream on the online charging system in peak load moments. In: 24th International crimean conference microwave and telecommunication technology. IEEE, pp 312–313
go back to reference Lee Y, Kozar KA (2006) Investigating the effect of website quality on e-business success: an analytic hierarchy process (AHP) approach. Decis Support Syst 42(3):1383–1401 CrossRef Lee Y, Kozar KA (2006) Investigating the effect of website quality on e-business success: an analytic hierarchy process (AHP) approach. Decis Support Syst 42(3):1383–1401 CrossRef
go back to reference Li K, Jamin S (2000) A measurement-based admission-controlled web server. In: Proceedings of the 19th annual joint conference of the IEEE computer and communications societies, vol 2, pp 651–659 Li K, Jamin S (2000) A measurement-based admission-controlled web server. In: Proceedings of the 19th annual joint conference of the IEEE computer and communications societies, vol 2, pp 651–659
go back to reference Li YM, Wu CT, Lai CY (2013) A social recommender mechanism for e-commerce: combining similarity, trust, and relationship. Decis Support Syst 55(3):740–752 CrossRef Li YM, Wu CT, Lai CY (2013) A social recommender mechanism for e-commerce: combining similarity, trust, and relationship. Decis Support Syst 55(3):740–752 CrossRef
go back to reference Lin HF (2007) The impact of Website quality dimensions on customer satisfaction in the B2C e-commerce context. Total Qual Manag Bus Excell 18(4):363–378 CrossRef Lin HF (2007) The impact of Website quality dimensions on customer satisfaction in the B2C e-commerce context. Total Qual Manag Bus Excell 18(4):363–378 CrossRef
go back to reference Mark K, Csaba L (2007) Analyzing customer behavior model graph (CBMG) using Markov chains. In: 11th International conference on intelligent engineering systems. IEEE, pp 71–76 Mark K, Csaba L (2007) Analyzing customer behavior model graph (CBMG) using Markov chains. In: 11th International conference on intelligent engineering systems. IEEE, pp 71–76
go back to reference Menascé DA (2002) QoS issues in web services. IEEE Internet Comput 6(6):72–75 CrossRef Menascé DA (2002) QoS issues in web services. IEEE Internet Comput 6(6):72–75 CrossRef
go back to reference Poggi N (2014) AUGURES: profit-aware web infrastructure management. Doctoral thesis, Polytechnic University of Catalonia Poggi N (2014) AUGURES: profit-aware web infrastructure management. Doctoral thesis, Polytechnic University of Catalonia
go back to reference Poggi N, Carrera D, Gavalda R, Ayguadé E (2011) Non-intrusive estimation of QoS degradation impact on e-commerce user satisfaction. In: 10th IEEE international symposium on network computing and applications. IEEE, pp 179–186 Poggi N, Carrera D, Gavalda R, Ayguadé E (2011) Non-intrusive estimation of QoS degradation impact on e-commerce user satisfaction. In: 10th IEEE international symposium on network computing and applications. IEEE, pp 179–186
go back to reference Raufi B, Georgieva J, Luma A, Ismaili F, Zenuni X (2010) An adaptation algorithm for adaptive Web based systems based on link structure and document similarity. In: 9th WSEAS international conference on telecommunications and informatics. World Scientific and Engineering Academy and Society, pp 29–34 Raufi B, Georgieva J, Luma A, Ismaili F, Zenuni X (2010) An adaptation algorithm for adaptive Web based systems based on link structure and document similarity. In: 9th WSEAS international conference on telecommunications and informatics. World Scientific and Engineering Academy and Society, pp 29–34
go back to reference Rosaci D, Sarné GM (2012) A multi-agent recommender system for supporting device adaptivity in e-commerce. J Intell Inf Syst 38(2):393–418 CrossRef Rosaci D, Sarné GM (2012) A multi-agent recommender system for supporting device adaptivity in e-commerce. J Intell Inf Syst 38(2):393–418 CrossRef
go back to reference Sarwar B et al (2001) Item-based collaborative filtering recommendation algorithms. In: 10th international conference on world wide web. ACM, pp 285–295 Sarwar B et al (2001) Item-based collaborative filtering recommendation algorithms. In: 10th international conference on world wide web. ACM, pp 285–295
go back to reference Schafer JB et al (2007) Collaborative filtering recommender systems. The adaptive web. Springer, Heidelberg, pp 291–324 Schafer JB et al (2007) Collaborative filtering recommender systems. The adaptive web. Springer, Heidelberg, pp 291–324
go back to reference Suchacka G, Borzemski L (2013) Web server support for e-customer loyalty through QoS differentiation. In: Nguyen NT (eds) Transactions on computational collective intelligence XII. Lecture Notes in Computer Science. vol. 8240. Springer, Heidelberg Suchacka G, Borzemski L (2013) Web server support for e-customer loyalty through QoS differentiation. In: Nguyen NT (eds) Transactions on computational collective intelligence XII. Lecture Notes in Computer Science. vol. 8240. Springer, Heidelberg
go back to reference Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
go back to reference Totok A, Karamcheti V (2010) RDRP: reward-driven request prioritization for e-commerce web sites. Electron Commer Res Appl 9(6):549–561 CrossRef Totok A, Karamcheti V (2010) RDRP: reward-driven request prioritization for e-commerce web sites. Electron Commer Res Appl 9(6):549–561 CrossRef
go back to reference Urgaonkar B (2005) Dynamic resource management in internet hosting platforms. Doctoral dissertation, University of Massachusetts Amherst Urgaonkar B (2005) Dynamic resource management in internet hosting platforms. Doctoral dissertation, University of Massachusetts Amherst
go back to reference Urgaonkar B, Shenoy P (2005) Cataclysm: policing extreme overloads in internet applications. In: 14th International conference on world wide web. ACM, pp 740–749 Urgaonkar B, Shenoy P (2005) Cataclysm: policing extreme overloads in internet applications. In: 14th International conference on world wide web. ACM, pp 740–749
go back to reference Weiss G (1999) Multiagent systems: a modern approach to distributed artificial intelligence. MIT Press, Cambridge Weiss G (1999) Multiagent systems: a modern approach to distributed artificial intelligence. MIT Press, Cambridge
go back to reference Yin PY, Guo YM (2013) Optimization of multi-criteria website structure based on enhanced tabu search and web usage mining. Appl Math Comput 219(24):11082–11095 Yin PY, Guo YM (2013) Optimization of multi-criteria website structure based on enhanced tabu search and web usage mining. Appl Math Comput 219(24):11082–11095
go back to reference Yue C, Wang H (2007) Profit-aware admission control for overload protection in e-commerce web sites. In: 15th IEEE international workshop on quality of service. IEEE, pp 188–193 Yue C, Wang H (2007) Profit-aware admission control for overload protection in e-commerce web sites. In: 15th IEEE international workshop on quality of service. IEEE, pp 188–193
go back to reference Zatwarnicki K, Zatwarnicka A (2014) The cluster-based time-aware web system. In: International conference on computer networks. Springer, pp 37–46 Zatwarnicki K, Zatwarnicka A (2014) The cluster-based time-aware web system. In: International conference on computer networks. Springer, pp 37–46
go back to reference Zheng Z, Zhang Y, Lyu MR (2014) Investigating QoS of real-world web services. IEEE Trans Serv Comput 7(1):32–39 CrossRef Zheng Z, Zhang Y, Lyu MR (2014) Investigating QoS of real-world web services. IEEE Trans Serv Comput 7(1):32–39 CrossRef
Metadata
Title
A Reinforcement Learning Based Model for Adaptive Service Quality Management in E-Commerce Websites
Authors
Hoda Ghavamipoor
S. Alireza Hashemi Golpayegani
Publication date
31-01-2019
Publisher
Springer Fachmedien Wiesbaden
Published in
Business & Information Systems Engineering / Issue 2/2020
Print ISSN: 2363-7005
Electronic ISSN: 1867-0202
DOI
https://doi.org/10.1007/s12599-019-00583-6

Other articles of this Issue 2/2020

Business & Information Systems Engineering 2/2020 Go to the issue

Premium Partner