Top

Computing

Published in:

14-11-2017

Mechanisms for provenance collection in scientific workflow systems

Authors: Mehdi Sarikhani, Andrew Wendelborn

Published in: Computing | Issue 5/2018

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Scientific workflow management systems run scientific experiments. They manage sequences of complex process transformations and collect provenance information at various levels of abstraction. Collected provenance information from scientific experiments documents how experimental results are derived from input values along with experimental parameters and workflow configurations. Provenance greatly enhances usability and acceptance of workflow systems among scientists, because provenance allows workflow systems to capture process configuration and behaviour at different levels of detail. On this basis, a sufficient level of collected provenance information enables scientists to validate their hypotheses and make a workflow reproducible. Currently SWfMS’s do not use a standard or portable provenance model for either capturing, storing, querying or representing model. There are a variety of design issues in provenance models and mechanisms in workflow system, owing to the variation of design dimensions in workflow architectures. Given this variety, it seems desirable to classify provenance mechanisms in workflow systems. We aim to survey provenance collection mechanisms, that are either a part of scientific workflow system, or of a software infrastructure that supports collection mechanisms in a scientific workflow system. In this paper, firstly, we identify and define a set of design dimensions and conventions for provenance collection mechanisms in the context of working on scientific workflow systems. After this, we survey a set of scientific workflow projects based on our design dimensions with an emphasis on provenance collection mechanisms. Then, those conventions are used in order to evaluate a number of existing provenance collection mechanisms, presented at the end of this paper. This survey provides an understanding of primary design issues for provenance collection mechanisms along with a set of desirable design dimensions.

next article Load-aware multicast routing in multi-radio wireless mesh networks using FCA-CMAC neural network

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Lin C, Lu S, Fei X, Chebotko A, Pai D, Lai Z, Fotouhi F, Hua J (2009) A reference architecture for scientific workflow management systems and the VIEW SOA solution. IEEE Trans Serv Comput 2(1):79–92. https://doi.org/10.1109/TSC.2009.4 CrossRef

Ranno F, Shrivastava S (1999) A review of distributed workflow management systems. In: The international joint conference on Work activities coordination and collaboration (WACC99), San Francisco, California

Wu Q, Zhu M, Gu Y, Brown P, Lu X, Lin W, Liu Y (2012) A distributed workflow management system with case study of real-life scientific applications on Grids. J Grid Comput 10(3):367–393. https://doi.org/10.1007/s10723-012-9222-7 CrossRef

Miller JA, Sheth AP, Kochut KJ, Wang X (1996) CORBA-based run-time architectures for workflow management systems. J Database Manag (JDM) 7(1):16–27. https://doi.org/10.4018/jdm.1996010102 CrossRef

Li H, Yang Y, Shi M (2003) Key issues and experiences in development of distributed workflow management systems. In: Zhou X, Orlowska M, Zhang Y (eds) Web technologies and applications, vol 2642. Lecture notes in computer science. Springer, Berlin, pp 507–512. https://doi.org/10.1007/3-540-36901-5_51

Görlach K, Sonntag M, Karastoyanova D, Leymann F, Reiter M (2011) Conventional workflow technology for scientific simulation. In: Yang X, Wang L, Jie W (eds) Guide to e-science. Computer communications and networks. Springer, London, pp 323–352. https://doi.org/10.1007/978-0-85729-439-5_12

Hahn C, Horn S, Jablonski S, Lay R, Neeb J, Schamburger R, Schlundt M Taxonomy of distribution concepts for workflow management. University Erlangen-Nürnberg

Simmhan YL, Plale B, Gannon D (2005) A survey of data provenance techniques, vol 47405. Indiana University, Bloomington

Freire J, Koop D, Santos E, Silva CT (2008) Provenance for computational tasks: a survey. Comput Sci Eng 10(3):11–21. https://doi.org/10.1109/MCSE.2008.79 CrossRef

10.

Lee B, Awad A, Awad M (2015) Towards secure provenance in the cloud: a survey. In: 2015 IEEE/ACM 8th international conference on utility and cloud computing (UCC), 7–10 Dec 2015, pp 577-582. https://doi.org/10.1109/UCC.2015.102

11.

Tao L, Ling L, Xiaolong Z, Kai X, Chao Y (2014) ProvenanceLens: service provenance management in the cloud. In: 2014 international conference on collaborative computing: networking, applications and worksharing (CollaborateCom), 22–25 Oct 2014, pp 275-284

12.

Chen P, Plale BA (2015) Big data provenance analysis and visualization. In: 15th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGrid), 4–7 May 2015, pp 797-800. https://doi.org/10.1109/CCGrid.2015.85

13.

Agrawal R, Imran A, Seay C, Walker J (2014) A layer based architecture for provenance in big data. In: IEEE international conference on big data (Big Data), 27–30 Oct. 2014, pp 1-7. https://doi.org/10.1109/BigData.2014.7004468

14.

Tan YS, Ko RKL, Holmes G (2013) Security and data accountability in distributed systems: a provenance survey. In: 2013 IEEE 10th international conference on high performance computing and communications & 2013 IEEE international conference on embedded and ubiquitous computing (HPCC_EUC), 13–15 Nov 2013, pp 1571–1578. https://doi.org/10.1109/HPCC.and.EUC.2013.221

15.

Moreau L, Kwasnikowska N, Van den Bussche J (2009) The foundations of the open provenance model. http://eprints.soton.ac.uk/id/eprint/267282

16.

Davidson SB, Boulakia SC, Eyal A, Ludäscher B, McPhillips TM, Bowers S, Anand MK, Freire J (2007) Provenance in scientific workflow systems. IEEE Data Eng Bull 30(4):44–50

17.

Amsterdamer Y, Davidson SB, Deutch D, Milo T, Stoyanovich J, Tannen V (2011) Putting lipstick on pig: enabling database-style workflow provenance. Very Large Data Base (VLDB) Endow 5(4):346–357

18.

Bowers S (2012) Scientific workflow, provenance, and data modeling challenges and approaches. J Data Sem 1(1):19–30. https://doi.org/10.1007/s13740-012-0004-y CrossRef

19.

Stamatogiannakis M, Groth P, Bos H (2015) Looking inside the black-box: capturing data provenance using dynamic instrumentation. In: Ludäscher B, Plale B (eds) Provenance and annotation of data and processes, vol 8628. Lecture notes in computer science. Springer, Switzerland, pp 155–167. https://doi.org/10.1007/978-3-319-16462-5_12

20.

Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report Ver. 2:3

21.

Keele S (2007) Guidelines for performing systematic literature reviews in software engineering. In: Technical report, Ver. 2.3 EBSE Technical Report. EBSE

22.

Daneva M, Damian D, Marchetto A, Pastor O (2014) Empirical research methodologies and studies in requirements engineering: how far did we come? J Syst Softw 95:1–9. https://doi.org/10.1016/j.jss.2014.06.035 CrossRef

23.

Kitchenham B, Pretorius R, Budgen D, Pearl Brereton O, Turner M, Niazi M, Linkman S (2010) Systematic literature reviews in software engineering: a tertiary study. Inf Softw Technol 52(8):792–805. https://doi.org/10.1016/j.infsof.2010.03.006 CrossRef

24.

Burnham JF (2006) Scopus database: a review. Biomed Digit Libr 3(1):1CrossRef

25.

Zahedi M, Shahin M, Babar MA (2015) A systematic review of knowledge sharing challenges and practices in global software development. Int J Inf Manag (submiited to)

26.

Shahin M, Liang P, Babar MA (2014) A systematic review of software architecture visualization techniques. J Syst Softw 94:161–185. https://doi.org/10.1016/j.jss.2014.03.071 CrossRef

27.

Zhang H, Babar MA, Tell P (2011) Identifying relevant studies in software engineering. Inf Softw Technol 53(6):625–637. https://doi.org/10.1016/j.infsof.2010.12.010 CrossRef

28.

Chen L, Babar MA, Zhang H (2010) Towards an evidence-based understanding of electronic data sources. Paper presented at the. Proceedings of the 14th international conference on evaluation and assessment in software engineering, UK

29.

Scheidegger C, Koop D, Santos E, Vo H, Callahan S, Freire J, Silva C (2008) Tackling the provenance challenge one layer at a time. Concurr Comput Pract Exp 20(5):473–483. https://doi.org/10.1002/cpe.1237 CrossRef

30.

Moreau L, Clifford B, Freire J, Futrelle J, Gil Y, Groth P, Kwasnikowska N, Miles S, Missier P, Myers J, Plale B, Simmhan Y, Stephan E, den Bussche JV (2011) The open provenance model core specification (v1.1). Future Gener Comput Syst 27(6):743–756. https://doi.org/10.1016/J.Future.2010.07.005 CrossRef

31.

Groth P, Miles S, Missier P, Moreau L (2009) A proposal for handling collections in the open provenance model. http://mailman.ecs.soton.ac.uk/pipermail/provenance-challenge-ipaw-info/attachments/20090605/85b3e182/attachment-0001.pdf

32.

Groth P, Moreau L (2013) PROV-Overview. W3C. http://www.w3.org/TR/prov-overview/. Accessed 30 April 2013

33.

Garijo D, Gil Y (2012) The OPMW ontology. http://www.opmw.org/model/OPMW_20121009/

34.

Simmhan Y, Groth P, Moreau L (2011) Special section: the third provenance challenge on using the open provenance model for interoperability. Future Gener Comput Syst 27(6):737–742CrossRef

35.

Crawl D, Wang J, Altintas I (2011) Provenance for MapReduce-based data-intensive workflows. In: 6th workshop on workflows in support of large-scale science, Seattle, Washington, ACM, pp 21–30

36.

Cruz SMS, Paulino CE, Oliveira Dd, Campos MLM, Mattoso M (2011) Capturing distributed provenance metadata from cloud-based scientific workflows. J Inf Data Manag 2(1):43–50

37.

Muniswamy-Reddy K-K, Macko P, Seltzer MI (2010) Provenance for the cloud. In: the 8th USENIX conference on file and storage technologies, San Jose, California, USENIX Association, 1855526, pp 15–14

38.

Muniswamy-Reddy K-K, Macko P, Seltzer MI (2009) Making a cloud provenance-aware. In: Workshop on the theory and practice of provenance, San Francisco, California, USENIX Association

39.

Marinho A, Murta L, Werner C, Braganholo V, Cruz SMS, Ogasawara E, Mattoso M (2012) ProvManager: a provenance management system for scientific workflows. Concurr Comput Pract Exp 24(13):1513–1530. https://doi.org/10.1002/cpe.1870 CrossRef

40.

Chapman A, Blaustein BT, Seligman L, Allen MD (2011) PLUS: a provenance manager for integrated information. In: IEEE international conference on information reuse and integration (IRI), 3–5 August 2011, pp 269–275. https://doi.org/10.1109/IRI.2011.6009558

41.

Buchert T, Nussbaum L, Gustedt J (2015) Towards complete tracking of provenance in experimental distributed systems research. In: Hunold S, Costan A, Giménez D et al (eds) Euro-Par 2015: parallel processing workshops: Euro-Par 2015 international workshops, Vienna, Austria, 24–25 August 2015, Revised Selected Papers. Springer, Cham, pp 604-616. https://doi.org/10.1007/978-3-319-27308-2_49

42.

Carata L, Akoush S, Balakrishnan N, Bytheway T, Sohan R, Seltzer M, Hopper A (2014) A primer on provenance. Commun ACM 57(5):52–60. https://doi.org/10.1145/2596628 CrossRef

43.

Cruz SMS, Campos MLM, Mattoso M (2009) Towards a taxonomy of provenance in scientific workflow management systems. In: IEEE congress on services, Los Angeles, California, 6–10 July 2009. IEEE, pp 259–266. https://doi.org/10.1109/SERVICES-I.2009.18

44.

Glavic B, Dittrich KR (2007) Data provenance: a categorization of existing approaches. In: Conference on Datenbanksysteme in Buisness, Technologie und Web (BTW), Aachen, Germany, vol 12, pp 227–241

45.

Simmhan YL, Plale B, Gannon D (2005) A survey of data provenance in e-science. ACM Sigmod Rec 34(3):31–36. https://doi.org/10.1145/1084805.1084812 CrossRef

46.

Sarikhani M (2015) An adaptive provenance collection architecture in scientific workflow systems. Ph.D. Thesis, The University of Adelaide, Adelaide, Australia

47.

Lee EA, Parks TM (1995) Dataflow process networks. Proc IEEE 83(5):773–801. https://doi.org/10.1109/5.381846 CrossRef

48.

Brooks C, Lee EA, Liu X, Neuendorffer S, Zhao Y, Zheng H (2008) Heterogeneous concurrent modeling and design in Java (volume 3: Ptolemy ii domains). EECS Department, University of California, Berkley, California

49.

Muniswamy-Reddy K-K (2010) Foundations for provenance-aware systems. Harvard University, Cambridge

50.

Anand MK (2010) Managing scientific workflow provenance. Univeristy of California Davis, Davis

51.

Bowers S, McPhillips TM, Ludäscher B (2008) Provenance in collection-oriented scientific workflows. Concurr Comput Pract Exp 20(5):519–529. https://doi.org/10.1002/cpe.1226 CrossRef

52.

Moreau L, Freire J, Futrelle J, McGrath R, Myers J, Paulson P (2008) The open provenance model: an overview. In: Freire J, Koop D, Moreau L (eds) Provenance and annotation of data and processes, vol 5272. Lecture notes in computer science. Springer, Berlin, Germany, pp 323–326. https://doi.org/10.1007/978-3-540-89965-5_31

53.

Sonntag M, Karastoyanova D, Deelman E (2010) Bridging the gap between business and scientific workflows: humans in the loop of scientific workflows. In: Sixth international conference on e-science (e-science 2010), Brisbane, Queensland, Australia, pp 206–213. IEEE. https://doi.org/10.1109/eScience.2010.12

54.

Ludäscher B, Weske M, McPhillips T, Bowers S (2009) Scientific workflows: business as usual? In: Dayal U, Eder J, Koehler J, Reijers H (eds) Business process management, vol 5701. Lecture notes in computer science. Springer, Berlin, Germany, pp 31–47. https://doi.org/10.1007/978-3-642-03848-8_4

55.

Barga R, Gannon D (2007) Scientific versus business workflows. In: Taylor I, Deelman E, Gannon D, Shields M (eds) Workflows for e-science. Springer, London, pp 9–16. https://doi.org/10.1007/978-1-84628-757-2_2

56.

Andrews T, Curbera F, Dholakia H, Goland Y, Klein J, Leymann F, Liu K, Roller D, Smith D, Thatte S (2003) Business process execution language for web services. version

57.

Juric MB, Mathew B, Sarang PG (2006) Business process execution language for web services: an architect and developer’s guide to orchestrating web services using BPEL4WS. Packt Publishing Ltd, Birmingham

58.

Altintas I, Barney O, Jaeger-Frank E (2006) Provenance collection support in the Kepler scientific workflow system. In: Moreau L, Foster I (eds) Provenance and annotation of data, vol 4145. Lecture notes in computer science. Springer, Berlin, Germany, pp 118–132. https://doi.org/10.1007/11890850_14

59.

Deelman E, Gannon D, Shields M, Taylor I (2009) Workflows and e-science: an overview of workflow system features and capabilities. Future Gener Comput Syst 25(5):528–540. https://doi.org/10.1016/j.future.2008.06.012 CrossRef

60.

Mattoso M, Werner C, Travassos GH, Braganholo V, Ogasawara E, Oliveira D, Cruz SMS, Martinho W, Murta L (2010) Towards supporting the life cycle of large scale scientific experiments. Int J Bus Process Integr Manag 5(1):79–92CrossRef

61.

Stevens R, Zhao J, Goble C (2007) Using provenance to manage knowledge of in silico experiments. Briefings Bioinform 8(3):183–194. https://doi.org/10.1093/bib/bbm015 CrossRef

62.

Cruz SMS, Barros PM, Bisch PM, Campos MLM, Mattoso M (2008) Provenance services for distributed workflows. In: 8th IEEE international symposium on cluster computing and the grid (CCGRID), Lyon, France, 19–22 May 2008. IEEE, pp 526–533. https://doi.org/10.1109/CCGRID.2008.73

63.

Belhajjame K, Wolstencroft K, Corcho O, Oinn T, Tanoh F, William A, Goble C (2008) Metadata management in the Taverna workflow system. In: 8th IEEE international symposium on cluster computing and the grid, CCGRID’08. IEEE, pp 651–656

64.

Davidson SB, Freire J (2008) Provenance and scientific workflows: challenges and opportunities. In: ACM SIGMOD international conference on management of data, Vancouver, Canada. ACM, 1376772, pp 1345–1350. https://doi.org/10.1145/1376616.1376772

65.

Lim C, Lu S, Chebotko A, Fotouhi F (2010) Prospective and retrospective provenance collection in scientific workflow environments. In: International Conference on Services Computing (SCC), Miami, Florida. IEEE, pp 449–456. https://doi.org/10.1109/SCC.2010.18

66.

McPhillips T, Bowers S, Belhajjame K, Ludascher B (2015) Retrospective provenance without a runtime provenance recorder. Paper presented at the proceedings of the 7th USENIX conference on theory and practice of provenance, Edinburgh, Scotland

67.

Marinho A, Werner C, Cruz S, Mattoso M, Braganholo V, Murta L (2009) A strategy for provenance gathering in distributed scientific workflows. In: International conference on services computing (SCC), Bangalore, India. IEEE, pp 344–347. https://doi.org/10.1109/SERVICES-I.2009.53

68.

Groth PT (2005) On the record: provenance in large scale, open distributed systems. A mini-thesis for transfer from M.Phil. to Ph.D., University of Southampton, Southampton, England

69.

Spillane RP, Sears R, Yalamanchili C, Gaikwad S, Chinni M, Zadok E (2009) Story book: an efficient extensible provenance framework. In: Theory and practice of provenance (TaPP’09), San Francisco, California

70.

Vahdat A, Anderson TE (1998) Transparent result caching. In: USENIX annual technical conference, New Orleans, Louisiana

71.

Malik T, Gehani A, Tariq D, Zaffar F (2013) Sketching distributed data provenance. In: Liu Q, Bai Q, Giugni S, Williamson D, Taylor J (eds) Data provenance and data management in eScience. Studies in computational intelligence. Springer, Berlin, Germany, pp 85–107. https://doi.org/10.1007/978-3-642-29931-5_4

72.

Malik T, Nistor L, Gehani A (2010) Tracking and sketching distributed data provenance. In: Sixth international conference on e-science (e-Science 2010), Brisbane, Queensland, Australia. IEEE, pp 190–197. https://doi.org/10.1109/eScience.2010.51

73.

Gehani A, Tariq D (2012) SPADE: support for provenance auditing in distributed environments. In: Narasimhan P, Triantafillou P (eds) Middleware 2012, vol 7662. Lecture notes in computer science. Springer, Berlin, Germany, pp 101–120. https://doi.org/10.1007/978-3-642-35170-9_6

74.

Widom J (2005) Trio: a system for integrated management of data, accuracy, and lineage. In: Conference on innovative data systems research (CIDR), Asilomar, California

75.

Ikeda R, Widom J (2010) Panda: a system for provenance and data. IEEE Data Eng Bull 33(3):42–49

76.

Foster IT, Vöckler J-S, Wilde M, Zhao Y (2003) The virtual data grid: a new model and architecture for data-intensive collaboration. In: Conference on innovative data systems research (CIDR), Asilomar, California, pp 18–29

77.

Cao B, Plale B, Subramanian G, Robertson E, Simmhan Y (2009) Provenance information model of karma version 3. In: International conference on services computing (SCC), Bangalore, India. IEEE, pp 348–351. https://doi.org/10.1109/SERVICES-I.2009.54

78.

Simmhan YL, Plale B, Gannon D (2008) Karma2: provenance management for data-driven workflows. Int J Web Serv Res 5(2):1–22CrossRef

79.

Lanter DP (1990) Lineage in gis: The problem and a solution. In: National center for geographic information and analysis (NCGIA), Santa Barbara, California

80.

Hasan R, Sion R, Winslett M (2009) The case of the fake Picasso: preventing history forgery with secure provenance. Paper presented at the proccedings of the 7th conference on file and storage technologies, San Francisco, California

81.

Asghar MR, Ion M, Russello G, Crispo B (2012) Securing data provenance in the cloud. In: Camenisch J, Kesdogan D (eds) Open problems in network security: IFIP WG 11.4 international workshop, iNetSec 2011, Lucerne, Switzerland, June 9, 2011, Revised Selected Papers. Springer, Berlin, pp 145–160. https://doi.org/10.1007/978-3-642-27585-2_12

82.

Murta L, Braganholo V, Chirigati F, Koop D, Freire J (2015) noWorkflow: capturing and analyzing provenance of scripts. In: Ludäscher B, Plale B (eds) Provenance and annotation of data and processes: 5th international provenance and annotation workshop, IPAW 2014, Cologne, Germany, 9–13 June 2014. Revised selected papers. Springer International Publishing, Cham, pp 71–83. https://doi.org/10.1007/978-3-319-16462-5_6

83.

McPhillips T, Song T, Kolisnik T, Aulenbach S, Belhajjame K, Bocinsky K, Cao Y, Chirigati F, Dey S, Freire J (2015) YesWorkflow: a user-oriented, language-independent tool for recovering workflow information from scripts. arXiv preprint arXiv:1502.02403

84.

Reynolds P, Killian C, Wiener JL, Mogul JC, Shah MA, Vahdat A (2006) Pip: detecting the unexpected in distributed systems. Paper presented at the proceedings of the 3rd conference on networked systems design & implementation—volume 3, San Jose, CA

85.

Singh A, Maniatis P, Roscoe T, Druschel P (2006) Using queries for distributed monitoring and forensics. Paper presented at the Proceedings of the 1st ACM SIGOPS/EuroSys European conference on computer systems 2006, Leuven, Belgium

86.

Ruth P, Xu D, Bhargava B, Regnier F (2004) E-notebook middleware for accountability and reputation based trust in distributed data sharing communities. In: Jensen C, Poslad S, Dimitrakos T (eds) Trust management: second international conference, iTrust 2004, Oxford, UK, March 29–April 1, 2004. Proceedings. Springer, Berlin, pp 161–175. https://doi.org/10.1007/978-3-540-24747-0_13

87.

Marathe AP (2001) Tracing lineage of array data. J Intell Inf Syst 17(2–3):193–214. https://doi.org/10.1023/A:1012857830230 CrossRefMATH

88.

Otero C (2012) Software engineering design: theory and practice, 1st edn. CRC Press, Boca Raton

89.

Aktas MS, Plale B, Leake D, Mukhi NK (2013) Unmanaged workflows: their provenance and use. In: Liu Q, Bai Q, Giugni S, Williamson D, Taylor J (eds) Data provenance and data management in eScience. Studies in computational intelligence. Springer, Berlin, Germany, pp 59–81. https://doi.org/10.1007/978-3-642-29931-5_3

90.

De Nies T, Coppens S, Van Deursen D, Mannens E, Van de Walle R (2012) Automatic discovery of high-level provenance using semantic similarity. In: Groth P, Frew J (eds) Provenance and annotation of data and processes. Lecture notes in computer science. Springer, Berlin, pp 97–110. https://doi.org/10.1007/978-3-642-34222-6_8

91.

Magliacane S (2012) Reconstructing provenance. In: Cudré-Mauroux P, Heflin J, Sirin E et al (eds) The semantic web (ISWC), vol 7650. Lecture notes in computer science. Springer, Berlin, pp 399–406. https://doi.org/10.1007/978-3-642-35173-0_29

92.

Tariq D, Ali M, Gehani A (2012) Towards automated collection of application-level data provenance. In: Theory and practice of provenance (TaPP’12), Boston, Massachusetts. USENIX Association

93.

Web-Page (2013) getting started with kepler provenance 2.4. https://code.kepler-project.org/code/kepler/trunk/modules/provenance/docs/provenance.pdf

94.

Simmhan Y, Barga R, Van Ingen C, Lazowska E, Szalay A (2009) Building the trident scientific workflow workbench for data management in the cloud. In: 3rd international conference on advanced engineering computing and applications in sciences (ADVCOMP), pp 41–50. https://doi.org/10.1109/ADVCOMP.2009.14

95.

Barga R, Simmhan Y, Withana EC, Sahoo S, Jackson J, Araujo N (2010) Provenance for scientific workflows towards reproducible research. IEEE Data Eng Bull 33:50–59

96.

Barga R, Jackson J, Araujo N, Guo D, Gautam N, Simmhan Y (2008) The trident scientific workflow workbench. In: IEEE fourth international conference on eScience (eScience ’08), Indianapolis, Indiana, pp 317–318. https://doi.org/10.1109/eScience.2008.126

97.

Freeman E, Robson E, Bates B, Sierra K (2004) Head first design patterns. O’Reilly Media, Inc., Sebastopol, CA, USA

98.

Forman IR, Forman N (2004) Java reflection in action. Manning Publications Co., Greenwich, CT, USA

99.

Maes P (1987) Concepts and experiments in computational reflection. In: Meyrowitz N (ed) Object-oriented programming systems, languages and applications (OOPSLA), Orlando, Florida, USA. ACM, 38821, pp 147–155. https://doi.org/10.1145/38765.38821

100.

Oliva A, Garcia IC, Buzato LE (1998) The reflective architecture of Guaraná. State University of Campinas, Sao Paulo

101.

Corradi A, Lodolo E, Monti S, Pasini S (2009) Dynamic reconfiguration of middleware for ubiquitous computing. In: the 3rd international workshop on Adaptive and dependable mobile ubiquitous systems, London, UK. ACM, pp 7–12

102.

Smith BC (1984) Reflection and semantics in Lisp. In: The 11th ACM SIGACT-SIGPLAN symposium on principles of programming languages (POPL84), Salt Lake City, Utah, USA. ACM, 800513, pp 23–35. https://doi.org/10.1145/800017.800513

103.

Coulson G (2001) What is reflective middleware. IEEE Distrib Syst Online 2(8):165–169

104.

Barbosa R, Pinho LM (2004) Monitoring of real time systems: a case for reflection. Polytechnic Institute of Porto, Porto

105.

McKinley PK, Sadjadi SM, Kasten EP, Cheng BHC (2004) Composing adaptive software. Computer 37(7):56–64. https://doi.org/10.1109/MC.2004.48 CrossRef

106.

Aksit M, Choukair Z (2003) Dynamic, adaptive and reconfigurable systems overview and prospective vision. In: the 23rd international conference on distributed computing systems workshops, 19–22 May 2003. IEEE, pp 84–89. https://doi.org/10.1109/ICDCSW.2003.1203537

107.

Webb D, Wendelborn A (2003) The PAGIS grid application environment. In: Sloot PA, Abramson D, Bogdanov A, Gorbachev Y, Dongarra J, Zomaya A (eds) Computational science—ICCS 2003, vol 2659. Lecture notes in computer science. Springer, Berlin, pp 1113–1122. https://doi.org/10.1007/3-540-44863-2_110

108.

Lopes CV (2002) Aspect-oriented programming: an historical perspective (what’s in a name?). University of California, Irvine

109.

Pawlak R, Seinturier L, Retaillé J-P, Younessi H (2005) Foundations of AOP for J2EE Development. Apress. https://doi.org/10.1007/978-1-4302-0063-5

110.

Elrad T, Aksit M, Kiczales G, Lieberherr KJ, Ossher H (2001) Discussing aspects of AOP. Commun ACM 44(10):33–38CrossRef

111.

Web-Page Provenance Aware Service Oriented Architecture (PASOA) (2014). http://twiki.pasoa.ecs.soton.ac.uk/bin/view/PASOA/WebHome. Accessed 10 July 2014

112.

Ding L, Michaelis J, McCusker J, McGuinness DL (2011) Linked provenance data: a semantic web-based approach to interoperable workflow traces. Future Gener Comput Syst 27(6):797–805. https://doi.org/10.1016/j.future.2010.011 CrossRef

113.

Moreau L, Ludäscher B, Altintas I, Barga RS, Bowers S, Callahan S, Chin G, Clifford B, Cohen S, Cohen-Boulakia S, Davidson S, Deelman E, Digiampietri L, Foster I, Freire J, Frew J, Futrelle J, Gibson T, Gil Y, Goble C, Golbeck J, Groth P, Holland DA, Jiang S, Kim J, Koop D, Krenek A, McPhillips T, Mehta G, Miles S, Metzger D, Munroe S, Myers J, Plale B, Podhorszki N, Ratnakar V, Santos E, Scheidegger C, Schuchardt K, Seltzer M, Simmhan YL, Silva C, Slaughter P, Stephan E, Stevens R, Turi D, Vo H, Wilde M, Zhao J, Zhao Y (2008) Special issue: the first provenance challenge. Concurr Comput Pract Exp 20(5):409–418. https://doi.org/10.1002/cpe.1233 CrossRef

114.

Web-Page (2010) The fourth and last provenance challenge. http://twiki.ipaw.info/bin/view/Challenge/FourthProvenanceChallenge

115.

Web-Page Provenance Challenge Wik. http://twiki.ipaw.info/bin/view/Challenge/WebHome. Accessed November 2008

116.

Garijo D, Gil Y (2012) The OPMW ontology. http://www.opmw.org/model/OPMW_20121009/

117.

Garijo D, Gil Y (2012) Towards open publication of reusable scientific workflows: abstractions, standards and linked data. http://www.isi.edu/~gil/papers/garijo-gil-opmw12.pdf

118.

Altintas I, Berkley C, Jaeger E, Jones M, Ludascher B, Mock S (2004) Kepler: an extensible system for design and execution of scientific workflows. In: 16th international conference on scientific and statistical database management, 21–23 June 2004. IEEE, pp 423–424. https://doi.org/10.1109/SSDM.2004.1311241

119.

Kim J, Deelman E, Gil Y, Mehta G, Ratnakar V (2008) Provenance trails in the wings/pegasus system. Concurr Comput Pract Exp 20(5):587–597. https://doi.org/10.1002/cpe.1228 CrossRef

120.

Web-Page Pegasus workflow management system. http://pegasus.isi.edu/. Accessed 10 March 2015

121.

Lin C, Lu S, Lai Z, Chebotko A, Fei X, Hua J, Fotouhi F (2008) Service-oriented architecture for VIEW: a visual scientific workflow management system. In: IEEE international conference on services computing (SCC’08), Honolulu, Hawaii. IEEE, pp 335–342

122.

Simmhan Y, Plale B, Gannon D, Marru S (2006) Performance evaluation of the karma provenance framework for scientific workflows. In: Moreau L, Foster I (eds) Provenance and annotation of data, vol 4145. Lecture notes in computer science. Springer, Berlin, pp 222–236. https://doi.org/10.1007/11890850_23

123.

Wolstencroft K, Haines R, Fellows D, Williams A, Withers D, Owen S, Soiland-Reyes S, Dunlop I, Nenadic A, Fisher P (2013) The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucl Acids Res 41(W1):W557–W561CrossRef

124.

Oinn T, Greenwood M, Addis M, Alpdemir MN, Ferris J, Glover K, Goble C, Goderis A, Hull D, Marvin D (2006) Taverna: lessons in creating a workflow environment for the life sciences. Concurr Comput Pract Exp 18(10):1067–1100CrossRef

125.

Marinho A, Murta L, Werner C, Braganholo V, Ogasawara E, Cruz SMS, Mattoso M (2010) Integrating provenance data from distributed workflow systems with ProvManager. In: Provenance and annotation of data and processes. Springer, pp 286–288

126.

Marinho A, Murta L, Werner C, Braganholo V, Cruz SMS, Ogasawara E, Mattoso M (2010) Managing provenance in scientific workflows with ProvManager. In: International workshop on challenges in e-Science (CIS2010), Petrópolis, Rio de Janeiro, Brazil, pp 17–24

127.

Olston C, Reed B, Srivastava U, Kumar R, Tomkins A (2008) Pig latin: a not-so-foreign language for data processing. Paper presented at the proceedings of the 2008 ACM SIGMOD international conference on management of data, Vancouver, Canada

128.

Green TJ, Karvounarakis G, Tannen V (2007) Provenance semirings. Paper presented at the proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on principles of database systems, Beijing, China

129.

Dey S, Belhajjame K, Koop D, Raul M, Ludascher B (2015) Linking prospective and retrospective provenance in scripts. Paper presented at the proceedings of the 7th USENIX conference on theory and practice of provenance, Edinburgh, Scotland

130.

Kim J, Deelman E, Gil Y, Mehta G, Ratnakar V (2008) Provenance trails in the wings/pegasus system. Concurr Comput Pract Exper 20(5):587–597. https://doi.org/10.1002/cpe.1228 CrossRef

131.

Williams DN, Bremer T, Doutriaux C, Patchett J, Williams S, Shipman G, Miller R, Pugmire DR, Smith B, Steed C, Bethel EW, Childs H, Krishnan H, Prabhat P, Wehner M, Silva CT, Santos E, Koop D, Ellqvist T, Poco J, Geveci B, Chaudhary A, Bauer A, Pletzer A, Kindig D, Potter GL, Maxwell TP (2013) Ultrascale visualization of climate data. Computer 46(9):68–76. https://doi.org/10.1109/MC.2013.119 CrossRef

132.

Web-Page vistrails. http://www.vistrails.org/index.php/Main_Page. Accessed 20 March 2015

133.

Silva CT, Freire J, Callahan SP (2007) Provenance for visualizations: reproducibility and beyond. Comput Sci Eng 9(5):82–89. https://doi.org/10.1109/MCSE.2007.106 CrossRef

134.

Hey AJG, Tansley S, Tolle KM (2009) The fourth paradigm: data-intensive scientific discovery, 1st edn. Microsoft Research Redmond, Washangton

135.

Delaney J, Heath G, Chave A, Howe B, Kirkham H (2000) NEPTUNE: real-time ocean and earth sciences at the scale of a tectonic plate. Oceanography 13(2):71–79CrossRef

Title: Mechanisms for provenance collection in scientific workflow systems
Authors: Mehdi Sarikhani
Andrew Wendelborn
Publication date: 14-11-2017
Publisher: Springer Vienna
Published in: Computing / Issue 5/2018
Print ISSN: 0010-485X
Electronic ISSN: 1436-5057
DOI: https://doi.org/10.1007/s00607-017-0578-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Premium Partner