Top

Published in:

2024 | OriginalPaper | Chapter

A Case for Business Process-Specific Foundation Models

Authors : Yara Rizk, Praveen Venkateswaran, Vatche Isahagian, Austin Narcomey, Vinod Muthusamy

Published in: Business Process Management Workshops

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The inception of large language models has helped advance the state-of-the-art on numerous natural language tasks. This has also opened the door for the development of foundation models for other domains and data modalities (e.g., images and code). In this paper, we argue that business process data has unique characteristics that warrant the creation of a new class of foundation models to handle tasks like activity prediction, process optimization, and decision making. These models should also tackle the challenges of applying AI to business processes which include data scarcity, multi-modal representations, domain specific terminology, and privacy concerns. To support our claim, we show the effectiveness of few-shot learning and transfer learning in next activity prediction, crucial properties for the success of foundation models.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter An Experiment on Transfer Learning for Suffix Prediction on Event Logs

next chapter Using Reinforcement Learning to Optimize Responses in Care Processes: A Case Study on Aggression Incidents

https://www.gartner.com/smarterwithgartner/the-disruptive-power-of-artificial-intelligence.

https://www.marketwatch.com/press-release/business-process-management-market-size-growth-with-top-leading-players-growth-key-factors-global-trends-industry-share-and-forecast-2022-2031-2022-08-18.

https://www.gartner.com/en/newsroom/press-releases/2021-09-29-gartner-finds-33-percent-of-technology-providers-plan-to-invest-1-million-or-more-in-ai-within-two-years.

To avoid confusion with business process tasks or activities, we will use “downstream tasks” to refer to foundation model specific prediction tasks.

Van der Aalst, W.M., Bichler, M., Heinzl, A.: Robotic process automation (2018)

Alsentzer, E., et al.: Publicly available clinical BERT embeddings. arXiv preprint arXiv:1904.03323 (2019)

Arlbjørn, J.S., Haug, A.: Business Process Optimization. Academica (2010)

Bach, S.H., et al.: PromptSource: an integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279 (2022)

Bernhart, W., Winterhoff, M.: Autonomous driving: disruptive innovation that promises to change the automotive industry as we know it. In: Langheim, J. (ed.) Energy Consumption and Autonomous Driving. LNM, pp. 3–10. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-19818-7_1CrossRef

Blodgett, S.L., Madaio, M.: Risks of AI foundation models in education. arXiv preprint arXiv:2110.10024 (2021)

Bommasani, R., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)

Camargo, M., Dumas, M., González-Rojas, O.: Learning accurate LSTM models of business processes. In: Hildebrandt, T., van Dongen, B.F., Röglinger, M., Mendling, J. (eds.) BPM 2019. LNCS, vol. 11675, pp. 286–302. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26619-6_19CrossRef

10.

Chakraborti, T., et al.: From robotic process automation to intelligent process automation. In: Asatiani, A., et al. (eds.) BPM 2020. LNBIP, vol. 393, pp. 215–228. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58779-6_15CrossRef

11.

Chen, H., Fang, X., Fang, H.: Multi-task prediction method of business process based on BERT and transfer learning. Knowl.-Based Syst. 254, 109603 (2022)CrossRef

12.

Chui, M., Henke, N., Miremadi, M.: Most of AI business uses will be in two areas. Harv. Bus. Rev. 20 (2018)

13.

van Dongen, B.B.: BPI challenge 2015 (2015)

14.

van Dongen, B., Borchert, F.F.: BPI challenge 2018 (2018)

15.

Dunzer, S., Stierle, M., Matzner, M., Baier, S.: Conformance checking: a state-of-the-art literature review. In: 11th International Conference on Subject-Oriented Business Process Management (2019)

16.

Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)

17.

Geyer-Klingeberg, J., Nakladal, J., Baldauf, F., Veit, F.: Process mining and robotic process automation: a perfect match. In: BPM (2018)

18.

Grosskopf, A., Decker, G., Weske, M.: The Process: Business Process Modeling Using BPMN. Meghan Kiffer Press (2009)

19.

Huo, S., Völzer, H., Reddy, P., Agarwal, P., Isahagian, V., Muthusamy, V.: Graph autoencoders for business process anomaly detection. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 417–433. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_26CrossRef

20.

Jia, C., et al.: Scaling up visual and vision-language representation learning with noisy text supervision. In: ICML, pp. 4904–4916. PMLR (2021)

21.

Kecht, C., Egger, A., Kratsch, W., Röglinger, M.: Quantifying chatbots ability to learn business processes. Inf. Syst. 113, 102176 (2023)

22.

Kenton, J.D., Chang, M.W., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the NAACL-HLT (2019)

23.

Kratsch, W., Manderscheid, J., Röglinger, M., Seyfried, J.: Machine learning in business process monitoring: a comparison of deep learning and classical approaches used for outcome prediction. Bus. Inf. Syst. Eng. 63, 261–276 (2021)CrossRef

24.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021)

25.

Maaradji, A., Dumas, M., La Rosa, M., Ostovar, A.: Fast and accurate business process drift detection. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 406–422. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23063-4_27CrossRef

26.

Maaradji, A., Dumas, M., La Rosa, M., Ostovar, A.: Detecting sudden and gradual drifts in business processes from execution traces. IEEE Trans. Knowl. Data Eng. 29(10), 2140–2154 (2017)CrossRef

27.

McKendrick, J.: AI adoption skyrocketed over the last 18 months. Harv. Bus. Rev. (2021)

28.

Mehdiyev, N., Evermann, J., Fettke, P.: A novel business process prediction model using a deep learning method. Bus. Inf. Syst. Eng. 62, 143–157 (2020)CrossRef

29.

Mendling, J.: Advancing business process science via the co-evolution of substantive and methodological knowledge. In: Di Ciccio, C., Dijkman, R., del Río Ortega, A., Rinderle-Ma, S. (eds.) BPM 2022. LNCS, vol. 13420, pp. 3–18. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16103-2_1CrossRef

30.

Min, S., et al.: Rethinking the role of demonstrations: what makes in-context learning work? arXiv preprint arXiv:2202.12837 (2022)

31.

Nguyen, P., Isahagian, V., Muthusamy, V., Slominski, A.: Summarizing process traces for analysis tasks: an intuitive and user-controlled approach. In: International Joint Conference on Artificial Intelligence (2022)

32.

Park, G., Song, M.: Predicting performances in business processes using deep neural networks. Decis. Support Syst. 129, 113191 (2020)CrossRef

33.

Pettey, C., van der Meulen, R.: Gartner says global artificial intelligence business value to reach \$1.2 trillion in 2018 (2018)

34.

Poesia, G., et al.: Synchromesh: reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 (2022)

35.

Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

36.

Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML, pp. 8748–8763. PMLR (2021)

37.

Rama-Maneiro, E., Vidal, J., Lama, M.: Deep learning for predictive business process monitoring: review and benchmark. IEEE Trans. Serv. Comput. (2021)

38.

Rizk, Y., Venkateswaran, P., Isahagian, V., Muthusamy, V., Talamadupula, K.: Can you teach robotic process automation bots new tricks? In: Marrella, A., et al. (eds.) BPM 2022. LNBIP, vol. 459, pp. 246–259. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16168-1_16CrossRef

39.

Schneider, F.: How users reciprocate to Alexa. In: Stephanidis, C., Antona, M., Ntoa, S. (eds.) HCII 2020. CCIS, vol. 1293, pp. 376–383. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60700-5_48CrossRef

40.

Senderovich, A., Di Francescomarino, C., Ghidini, C., Jorbina, K., Maggi, F.M.: Intra and inter-case features in predictive process monitoring: a tale of two dimensions. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNCS, vol. 10445, pp. 306–323. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65000-5_18CrossRef

41.

Tian, X., Pavur, R., Han, H., Zhang, L.: A machine learning-based human resources recruitment system for business process management: using LSA, BERT and SVM. Bus. Process. Manag. J. 29(1), 202–222 (2022)CrossRef

42.

Van Der Aalst, W.: Process mining: overview and opportunities. ACM Trans. Manage. Inf. Syst. 3(2), 1–17 (2012)CrossRef

43.

Venkateswaran, P., Isahagian, V., Muthusamy, V., Venkatasubramanian, N.: FedGen: generalizable federated learning for sequential data. In: IEEE International Conference on Cloud Computing (2023)

44.

Venkateswaran, P., Muthusamy, V., Isahagian, V., Venkatasubramanian, N.: Environment agnostic invariant risk minimization for classification of sequential datasets. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1615–1624 (2021)

45.

Venkateswaran, P., Muthusamy, V., Isahagian, V., Venkatasubramanian, N.: Robust and generalizable predictive models for business processes. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 105–122. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_9CrossRef

46.

Wang, W., Bao, H., Dong, L., Wei, F.: VLMo: unified vision-language pre-training with mixture-of-modality-experts. arXiv preprint arXiv:2111.02358 (2021)

47.

Wang, W., et al.: Image as a foreign language: BEiT pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208.10442 (2022)

48.

Weske, Mathias: Business process management methodology. In: Weske, M. (ed.) Business Process Management, pp. 373–388. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28616-2_8CrossRef

49.

White, S.A.: Introduction to BPMN. IBM Coop. 2 (2004)

50.

Wiggins, W.F., Tejani, A.S.: On the opportunities and risks of foundation models for natural language processing in radiology. Radiol.: Artif. Intell. 4(4), e220119 (2022)

Title: A Case for Business Process-Specific Foundation Models
Authors: Yara Rizk
Praveen Venkateswaran
Vatche Isahagian
Austin Narcomey
Vinod Muthusamy
Publisher: Springer Nature Switzerland
Book: Business Process Management Workshops
Print ISBN: 978-3-031-50973-5

Electronic ISBN: 978-3-031-50974-2

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-3-031-50974-2_4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner