Skip to main content

2020 | OriginalPaper | Buchkapitel

5. Systemic Evaluation Approach to Meet the Challenges of Complexity

verfasst von : Mika Nieminen, Kirsi Hyytinen, Vesa Salminen, Sampsa Ruutu

Erschienen in: Society as an Interaction Space

Verlag: Springer Nature Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Traditional linear evaluation approaches are not able to address the dynamic interrelationships and feedback mechanisms involved in the increasingly complex social environment. To meet the challenges of complexity, new evaluation approaches are required. This chapter contributes to this discussion by suggesting a new integrative evaluation approach which combines foresight, multi-criteria evaluation and system dynamic modelling into the evaluation process. The developed methodology is applied in the evaluation of the Finnish Innovation Fund, Sitra.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
3
Sitra law (only in Finnish) Laki Suomen itsenäisyyden juhlarahastosta. 24.8.1990/717 http://​www.​finlex.​fi/​fi/​laki/​ajantasa/​1990/​19900717
 
5
The more detailed results are published in the evaluation report (executive summary is available in English; the whole report is available only in Finnish) and publicly available at https://​media.​sitra.​fi/​2017/​11/​29120141/​Selvityksia127.​pdf.
 
6
Before formulating the comprehensive system dynamic model, two narrower models were created to describe the dynamics of the two selected case areas. These models helped to create the comprehensive model.
 
Literatur
Zurück zum Zitat Autio, E., & Thomas, L. D. W. (2013). Innovation ecosystems. In M. Dodgson, D. Gann, & N. Phillips (Eds.), Oxford handbook of innovation management (pp. 204–288). Oxford: Oxford University Press. Autio, E., & Thomas, L. D. W. (2013). Innovation ecosystems. In M. Dodgson, D. Gann, & N. Phillips (Eds.), Oxford handbook of innovation management (pp. 204–288). Oxford: Oxford University Press.
Zurück zum Zitat Cabrera, D., Colosi, L., & Lobdell, C. (2008). Systems thinking. Evaluation and Program Planning, 31, 299–310.CrossRef Cabrera, D., Colosi, L., & Lobdell, C. (2008). Systems thinking. Evaluation and Program Planning, 31, 299–310.CrossRef
Zurück zum Zitat Castells, M. (1996). The rise of the network society, the information age: Economy, society and culture (Vol. I). Cambridge and Oxford: Blackwell. Castells, M. (1996). The rise of the network society, the information age: Economy, society and culture (Vol. I). Cambridge and Oxford: Blackwell.
Zurück zum Zitat Chen, H. T. (2005). Theory driven evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 415–419). Thousand Oaks: Sage. Chen, H. T. (2005). Theory driven evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 415–419). Thousand Oaks: Sage.
Zurück zum Zitat Cozzens, S., & Melkers, J. (1997). Use and usefulness of performance measurement in state science and technology programs. Policy Studies Journal, 25(3), 425–435.CrossRef Cozzens, S., & Melkers, J. (1997). Use and usefulness of performance measurement in state science and technology programs. Policy Studies Journal, 25(3), 425–435.CrossRef
Zurück zum Zitat Denzin, N. (2006). Sociological methods: A sourcebook (5th ed.). Piscataway: Aldine Transaction. Denzin, N. (2006). Sociological methods: A sourcebook (5th ed.). Piscataway: Aldine Transaction.
Zurück zum Zitat Djellal, F., & Gallouj, F. (2013). The productivity challenge in services: Measurement and strategic perspectives. The Service Industries Journal, 33(3–4), 282–299.CrossRef Djellal, F., & Gallouj, F. (2013). The productivity challenge in services: Measurement and strategic perspectives. The Service Industries Journal, 33(3–4), 282–299.CrossRef
Zurück zum Zitat Dyehouse, M., Bennett, D., Harbor, J., Childress, A., & Dark, M. (2009). A comparison of linear and systems thinking approaches for program evaluation illustrated using Indiana Interdisciplinary GK-12. Evaluation and Program Planning, 32, 187–196.CrossRef Dyehouse, M., Bennett, D., Harbor, J., Childress, A., & Dark, M. (2009). A comparison of linear and systems thinking approaches for program evaluation illustrated using Indiana Interdisciplinary GK-12. Evaluation and Program Planning, 32, 187–196.CrossRef
Zurück zum Zitat Edquist, C. (2005). Reflections on the systems of innovation approach. Science and Public Policy, 31(6), 485–489.CrossRef Edquist, C. (2005). Reflections on the systems of innovation approach. Science and Public Policy, 31(6), 485–489.CrossRef
Zurück zum Zitat Elzen, B., Geels, F. W., & Green, K. (2004). System innovation and the transition to sustainability. Cheltenham and Northampton: Edward Elgar.CrossRef Elzen, B., Geels, F. W., & Green, K. (2004). System innovation and the transition to sustainability. Cheltenham and Northampton: Edward Elgar.CrossRef
Zurück zum Zitat Eoyang, G., & Holladay, R. (2013). Adaptive action: Leveraging uncertainty in your organization. Stanford: Stanford University Press. Eoyang, G., & Holladay, R. (2013). Adaptive action: Leveraging uncertainty in your organization. Stanford: Stanford University Press.
Zurück zum Zitat Forrester, J. W. (2007). System dynamics—The next fifty years. System Dynamics Review, 23(2/3), 359–370.CrossRef Forrester, J. W. (2007). System dynamics—The next fifty years. System Dynamics Review, 23(2/3), 359–370.CrossRef
Zurück zum Zitat Forss, K., Marra, M., & Schwartz, R. (2011). Evaluating the complex: Attribution, contribution, and beyond. London: Transaction Publishers. Forss, K., Marra, M., & Schwartz, R. (2011). Evaluating the complex: Attribution, contribution, and beyond. London: Transaction Publishers.
Zurück zum Zitat Freeman, C., & Louca, F. (2002). As time goes by: From the industrial revolutions to the information revolution. Oxford: Oxford University Press.CrossRef Freeman, C., & Louca, F. (2002). As time goes by: From the industrial revolutions to the information revolution. Oxford: Oxford University Press.CrossRef
Zurück zum Zitat Funnel, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Jossey-Bass. Funnel, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Jossey-Bass.
Zurück zum Zitat Geels, F. W. (2002). Technological transitions as evolutionary reconfiguration processes: A multi-level perspective and a case-study. Research Policy, 31, 1257–1274.CrossRef Geels, F. W. (2002). Technological transitions as evolutionary reconfiguration processes: A multi-level perspective and a case-study. Research Policy, 31, 1257–1274.CrossRef
Zurück zum Zitat Geels, F. W. (2004). From sectoral systems of innovation to socio-technical systems. Insights about dynamics and change from sociology and institutional theory. Research Policy, 33, 897–920.CrossRef Geels, F. W. (2004). From sectoral systems of innovation to socio-technical systems. Insights about dynamics and change from sociology and institutional theory. Research Policy, 33, 897–920.CrossRef
Zurück zum Zitat Geels, F. W. (2005). System innovation and the transition to sustainability. Theory, evidence and policy. Cheltenham: Edward Elgar. Geels, F. W. (2005). System innovation and the transition to sustainability. Theory, evidence and policy. Cheltenham: Edward Elgar.
Zurück zum Zitat Geels, F. W., & Kemp, R. (2007). Dynamics in socio-technical systems: Typology of change processes and contrasting case studies. Technology in Society, 29(4), 441–455.CrossRef Geels, F. W., & Kemp, R. (2007). Dynamics in socio-technical systems: Typology of change processes and contrasting case studies. Technology in Society, 29(4), 441–455.CrossRef
Zurück zum Zitat Geels, F. W., & Schot, J. (2007). Typology of sociotechnical transition pathways. Research Policy, 36, 399–417.CrossRef Geels, F. W., & Schot, J. (2007). Typology of sociotechnical transition pathways. Research Policy, 36, 399–417.CrossRef
Zurück zum Zitat Hansson, F. (2006). Organisational use of evaluation. Evaluation, 12(2), 159–178.CrossRef Hansson, F. (2006). Organisational use of evaluation. Evaluation, 12(2), 159–178.CrossRef
Zurück zum Zitat Hargreaves, M. B., & Podems, D. (2012). Advancing systems thinking in evaluation: A review of four publications. American Journal of Evaluation, 33, 462–470.CrossRef Hargreaves, M. B., & Podems, D. (2012). Advancing systems thinking in evaluation: A review of four publications. American Journal of Evaluation, 33, 462–470.CrossRef
Zurück zum Zitat Holland, J. H. (1995). Hidden order: How adaptation builds complexity. New York: Basic Books. Holland, J. H. (1995). Hidden order: How adaptation builds complexity. New York: Basic Books.
Zurück zum Zitat Hyytinen, K., Ruutu, S., Nieminen, M., Gallouj, F., & Toivonen, M. (2014). A system dynamic and multi-criteria evaluation of innovations in environmental services. Economics and Policy of Energy and the Environment, 3, 29–52. Hyytinen, K., Ruutu, S., Nieminen, M., Gallouj, F., & Toivonen, M. (2014). A system dynamic and multi-criteria evaluation of innovations in environmental services. Economics and Policy of Energy and the Environment, 3, 29–52.
Zurück zum Zitat Kemp, R., Rip, A., & Schot, J. (2001). Constructing transition paths through the management of niche. In R. Garud & P. Karnoe (Eds.), Path dependency and creation (pp. 269–299). Mahwah: Lawrence Erlbaum. Kemp, R., Rip, A., & Schot, J. (2001). Constructing transition paths through the management of niche. In R. Garud & P. Karnoe (Eds.), Path dependency and creation (pp. 269–299). Mahwah: Lawrence Erlbaum.
Zurück zum Zitat Kemp, R., & Rotmans, J. (2004). Managing the transition to sustainable mobility. In B. Elzen, F. Geels, & K. Green (Eds.), System innovation and the transition to sustainability: Theory, evidence and policy (pp. 137–167). Cheltenham: Edgar Elgar. Kemp, R., & Rotmans, J. (2004). Managing the transition to sustainable mobility. In B. Elzen, F. Geels, & K. Green (Eds.), System innovation and the transition to sustainability: Theory, evidence and policy (pp. 137–167). Cheltenham: Edgar Elgar.
Zurück zum Zitat Kemp, R., Schot, J., & Hoogma, R. (1998). Regime shifts to sustainability through processes of niche formation. The approach of strategic niche management. Technology Analysis and Strategic Management, 10(2), 175–195.CrossRef Kemp, R., Schot, J., & Hoogma, R. (1998). Regime shifts to sustainability through processes of niche formation. The approach of strategic niche management. Technology Analysis and Strategic Management, 10(2), 175–195.CrossRef
Zurück zum Zitat Kivisaari, S., Lovio, R., & Väyrynen, E. (2004). Managing experiments for transition: Examples of societal embedding in energy and health care sectors. In B. Elzen, F. W. Geels, & K. Green (Eds.), System innovation and the transition to sustainability. Theory, evidence and policy (pp. 223–250). Cheltenham: Edward Elgar. Kivisaari, S., Lovio, R., & Väyrynen, E. (2004). Managing experiments for transition: Examples of societal embedding in energy and health care sectors. In B. Elzen, F. W. Geels, & K. Green (Eds.), System innovation and the transition to sustainability. Theory, evidence and policy (pp. 223–250). Cheltenham: Edward Elgar.
Zurück zum Zitat Martin, B. R. (2010). The origins of the concept of ‘foresight’ in science and technology: An insider’s perspective. Technological Forecasting and Social Change, 77, 1438–1447.CrossRef Martin, B. R. (2010). The origins of the concept of ‘foresight’ in science and technology: An insider’s perspective. Technological Forecasting and Social Change, 77, 1438–1447.CrossRef
Zurück zum Zitat Martin, B. R., & Irvine, J. (1989). Research foresight: Priority-setting in science. London and New York: Pinter Publishers. Martin, B. R., & Irvine, J. (1989). Research foresight: Priority-setting in science. London and New York: Pinter Publishers.
Zurück zum Zitat Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18, 270–280.CrossRef Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18, 270–280.CrossRef
Zurück zum Zitat Meadows, D. H. (2008). Thinking in systems. A primer. Chelsea Green, White River Junction. Meadows, D. H. (2008). Thinking in systems. A primer. Chelsea Green, White River Junction.
Zurück zum Zitat Merril, J. A., Deegan, M., Wilson, R., Kaushal, R., & Frederiks, K. (2013). A system dynamics evaluation model: Implementation of health information exchange for public health reporting. Journal of the American Medical Informatics Association, 20, 131–138.CrossRef Merril, J. A., Deegan, M., Wilson, R., Kaushal, R., & Frederiks, K. (2013). A system dynamics evaluation model: Implementation of health information exchange for public health reporting. Journal of the American Medical Informatics Association, 20, 131–138.CrossRef
Zurück zum Zitat Miles, I. (2013). Interactive impacts—Foresight as a product, service and coproduction process. In D. Dirk Meissner, L. Gokhberg, & A. Sokolov (Eds.), Technology and innovation policy for the future: Potentials and limits of foresight studies (pp. 63–82). Berlin: Springer.CrossRef Miles, I. (2013). Interactive impacts—Foresight as a product, service and coproduction process. In D. Dirk Meissner, L. Gokhberg, & A. Sokolov (Eds.), Technology and innovation policy for the future: Potentials and limits of foresight studies (pp. 63–82). Berlin: Springer.CrossRef
Zurück zum Zitat Mitleton-Kelly, E. (Ed.). (2007). Complex systems and evolutionary perspectives on organizations. The application of complexity theory to organizations. Bingley, UK: Emerald. Mitleton-Kelly, E. (Ed.). (2007). Complex systems and evolutionary perspectives on organizations. The application of complexity theory to organizations. Bingley, UK: Emerald.
Zurück zum Zitat Nieminen, M., & Hyytinen, K. (2015). Future-oriented impact assessment: Supporting strategic decision-making in complex socio-technical environments. Evaluation, 21(4), 448–461.CrossRef Nieminen, M., & Hyytinen, K. (2015). Future-oriented impact assessment: Supporting strategic decision-making in complex socio-technical environments. Evaluation, 21(4), 448–461.CrossRef
Zurück zum Zitat Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford. Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford.
Zurück zum Zitat Rip, A., & Kemp, R. (1998). Technological change. In S. Rayner & E. J. Malone (Eds.), Human choice and climate change: Vol. 2. Resources and technology (pp. 327–399). Battelle Press: Columbus. Rip, A., & Kemp, R. (1998). Technological change. In S. Rayner & E. J. Malone (Eds.), Human choice and climate change: Vol. 2. Resources and technology (pp. 327–399). Battelle Press: Columbus.
Zurück zum Zitat Rip, A. (2003). Societal challenges for R&D evaluation. In P. Shapira & S. Kuhlmann (Eds.), Learning from science and technology policy evaluation: Experiences from the United States and Europe (pp. 35–59). Cheltenham: Edward Elgar. Rip, A. (2003). Societal challenges for R&D evaluation. In P. Shapira & S. Kuhlmann (Eds.), Learning from science and technology policy evaluation: Experiences from the United States and Europe (pp. 35–59). Cheltenham: Edward Elgar.
Zurück zum Zitat Ritala, P., Armila, L., & Blomqvist, K.-M. (2009). Innovation orchestration capability—Defining the organizational and individual level determinants. International Journal of Innovation Management, 13(4), 569–591.CrossRef Ritala, P., Armila, L., & Blomqvist, K.-M. (2009). Innovation orchestration capability—Defining the organizational and individual level determinants. International Journal of Innovation Management, 13(4), 569–591.CrossRef
Zurück zum Zitat Rittel, H., & Webber Melvin, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.CrossRef Rittel, H., & Webber Melvin, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.CrossRef
Zurück zum Zitat Rotmans, J., & Loorbach, D. (2009). Complexity and transition management. Journal of Industrial Ecology, 13(2), 184–196.CrossRef Rotmans, J., & Loorbach, D. (2009). Complexity and transition management. Journal of Industrial Ecology, 13(2), 184–196.CrossRef
Zurück zum Zitat Sterman, J. D. (2001). System dynamics modeling: Tools for learning in a complex world. California Management Review, 43(4), 8–25.CrossRef Sterman, J. D. (2001). System dynamics modeling: Tools for learning in a complex world. California Management Review, 43(4), 8–25.CrossRef
Zurück zum Zitat Tait, J., & Williams, R. (1999). Policy approaches to research and development: foresight, framework and competitiveness. Science and Public Policy, 26(2), 101–112.CrossRef Tait, J., & Williams, R. (1999). Policy approaches to research and development: foresight, framework and competitiveness. Science and Public Policy, 26(2), 101–112.CrossRef
Zurück zum Zitat van der Knaap, P. (2006). Performance evaluation and performance management. Overcoming the downsides of policy objectives and performance indicators. Evaluation, 12(3), 278–293.CrossRef van der Knaap, P. (2006). Performance evaluation and performance management. Overcoming the downsides of policy objectives and performance indicators. Evaluation, 12(3), 278–293.CrossRef
Zurück zum Zitat Williams, B., & Hummelbrunner, R. (2011). Systems concepts in action. A practitioner’s Toolkit. Stanford: Stanford University Press. Williams, B., & Hummelbrunner, R. (2011). Systems concepts in action. A practitioner’s Toolkit. Stanford: Stanford University Press.
Zurück zum Zitat Williams, B., & Imam, I. (Eds.). (2007). Systems concepts in evaluation: An expert anthology. Point Reyes: Edge Press of Inverness. Williams, B., & Imam, I. (Eds.). (2007). Systems concepts in evaluation: An expert anthology. Point Reyes: Edge Press of Inverness.
Zurück zum Zitat Windrum, P., & García-Goñi, M. (2008). A neo-Schumpeterian model of health services innovation. Research Policy, 37(4), 649–672.CrossRef Windrum, P., & García-Goñi, M. (2008). A neo-Schumpeterian model of health services innovation. Research Policy, 37(4), 649–672.CrossRef
Metadaten
Titel
Systemic Evaluation Approach to Meet the Challenges of Complexity
verfasst von
Mika Nieminen
Kirsi Hyytinen
Vesa Salminen
Sampsa Ruutu
Copyright-Jahr
2020
Verlag
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-15-0069-5_5