Stacie Petter
Deepak Khazanchi
Skip Abstract Section
Abstract
Patterns were originally developed in the field of architecture as a mechanism for communicating good solutions to recurring classes of problems. Since then, many researchers and practitioners have created patterns to describe effective solutions to problems associated with disparate areas such as virtual project management, human-computer interaction, software development and engineering, and design science research. We believe that the development of patterns is a design science activity in which an artifact (i.e., a pattern) is created to communicate about and improve upon the current state-of-practice. Design science research has two critical components, creation and evaluation of an artifact. While many patterns have been created, few, if any, have been evaluated in this sense. In this paper, we propose a framework to evaluate patterns in any domain and provide examples of how to use the evaluation framework. This process of evaluation could help researchers refine extant patterns and improve the possibility of creating sustainable best practices for a given domain. We believe this evaluation framework begins an important dialogue related to the evaluation of patterns as artifacts of design science research. We draw upon the literature associated with patterns, evaluation of design science research, and research methods to develop this framework for evaluating patterns in a more consistent and rigorous manner.
- Alexander, C. 1964. Notes on the Synthesis of Form, Cambridge: Harvard University Press.Google Scholar
- Alexander, C. 1979. The Timeless Way of Building, Oxford University Press.Google Scholar
- Alexander, C. 1999. "The Origins of Pattern Theory: The Future of the Theory, and the Generation of a Living World," IEEE Software (16:5), pp 71--82. Google ScholarDigital Library
- Alexander, C., Ishikawa, S., and Silverstein, M. 1977. A Pattern Language, New York: Oxford University Press.Google Scholar
- Appleton, B. 2000. "Patterns and Software: Essential Concepts and Terminology. " Feb. 14 (available online at http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html)Google Scholar
- Austin, R.D., and Westerman, G. 2002. "Destiny WebSolutions, Inc.," Unpbulished Manuscript, Boston.Google Scholar
- Baskerville, R., Pries-Heje, J., and Venable, J. 2007. "Soft Design Science Research: Extending the Boundaries of Evaluation in Design Science Research," 2nd International Conference on Design Science Research in Information Systems & Technology, Pasadena, CA.Google Scholar
- Baskerville, R., Pries--Heje, J., and Venable, J. 2008. "Evaluation Risks in Design Science Research: A Framework," International Conference on Design Science Research in Information Systems and Technology, Atlanta, GA.Google Scholar
- Beck, K., Crocker, R., Meszaros, G., Vlissides, J., Coplien, J.O., Dominick, L., and Paulisch, F. 1996. "Industrial Experience with Design Patterns," International Conference on Software Engineering, Berlin, Germany, pp. 103--114. Google ScholarDigital Library
- Beck, K., and Cunningham, W. 1987. "Using Pattern Languages for Object-Oriented Programs," OOPSLA Workshop on the Specification and Design for Object-Oriented Programming, Orlando, FL.Google Scholar
- Brown, W., Malveau, R., McCormick, H., Mowbray, T., and Thomas, S.W. 1998. "The Software Patterns Criteria: Proposed Definitions for Evaluating Software Pattern Quality," June 10 (available online at http://www.antipatterns.com/whatisapattern/).Google Scholar
- Campbell, D.T., and Fiske, D.W. 1959. "Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin (56:2), pp 81--105.Google Scholar
- Cook, T.D., and Campbell, D.T. 1979. Quasi-Experimentation: Design & Analysis Issues for Field Settings, Chicago, IL: Rand McNally College Publishing Company.Google Scholar
- Coplien, J. O. 2007. "A Pattern Definition: Software Patterns," (available online at http://hillside.net/patterns/definition.html).Google Scholar
- Deardon, A., Finlay, J., Allgar, L., and McManus, B. 2002. "Evaluating Pattern Languages in Participatory Design," Conference on Human Factors in Computing Systems, Minneapolis, MN, pp. 664--665. Google ScholarDigital Library
- Fernandez, A., Holmer, T., Rubart, J., and Schuemmer, T. 2002. "Three Groupware Patterns from the Activity Awareness Family," Seventh European Conference on Pattern Languages of Programs, EuroPLoP, Irsee, Germany.Google Scholar
- Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1995. Design Patterns: Elements of Reusable Object-Oriented Systems, Reading, MA: Addison-Wesley. Google ScholarDigital Library
- Gregor, S., and Jones, D. 2007. "The Anatomy of a Design Theory," Journal of the Association for Information Systems (8:5), pp 312--335.Google Scholar
- Hevner, A., March, S., Park, J., and Ram, S. 2004. "Design Science in Information Systems Research," MIS Quarterly (28:1), pp 75--105. Google ScholarDigital Library
- Homsky, O. "More Patterns for Group Leadership," in: Eighth European Conference on Pattern Languages of Programs, EuroPLoP, Irsee, Germany, 2003.Google Scholar
- Iivari, J. 2007. "A Paradigmatic Analysis of Information Systems as a Design Science," Scandinavian Journal of Information Systems (19:2).Google Scholar
- Khazanchi, D. 1996. "A Philosophical Framework for the Validation of Information Systems Concepts," Americas Conference on Information Systems, Phoenix, AZ, pp. 755--757.Google Scholar
- Khazanchi, D., and Zigurs, I. 2005. Patterns of Effective Management of Virtual Projects: An Exploratory Study, (Newtown Square, PA: Project Management Institute.Google Scholar
- Khazanchi, D., and Zigurs, I. 2006. "Patterns for Effective Management of Virtual Projects: Theory and Evidence," International Journal of e-Collaboration (2:3), pp 25--48.Google ScholarCross Ref
- Khazanchi, D., and Zigurs, I. 2007. "A Systematic Method for Discovering Effective Patterns of Virtual Project Management," Unpublished Working Paper, Social Science Review Network.Google Scholar
- Klein, H.K., and Myers, M.D. 1999. "A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems," MIS Quarterly (23:1), pp 67--93. Google ScholarDigital Library
- Lea, D. 1994. "Christopher Alexander: An Introduction for Object-Oriented Designers," ACM Software Engineering Notes (19:1), pp. 39--46. Google ScholarDigital Library
- March, S.T., and Smith, G.F. 1995. "Design and Natural Science Research on Information Technology," Decision Support Systems (15:4), pp 251--266. Google ScholarDigital Library
- Maznevski, M.L., and Chudoba, K.M. 2000. "Bridging Space over Time: Global Virtual Team Dynamics and Effectiveness," Organization Science (11:5), pp. 473--492. Google ScholarDigital Library
- Meszaros, G., and Doble, J. 1996. "A Pattern Language for Pattern Writing," Pattern Languages of Programs Conference, Allerton Park, IL. Google ScholarDigital Library
- Nunamaker, J.F., Chen, M., and Purdin, T.D.M. 1990--1991. "Systems Development in Information Systems Research," Journal of Management Information Systems (7:3), pp 89--106. Google ScholarDigital Library
- Prechelt, L., Unger-Lamprecht, B., Philippsen, M., and Tichy, W.F. 2002. "Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance," IEEE Transactions on Software Engeineering (28:6), pp 595--606. Google ScholarDigital Library
- Pries-Heje, J., Baskerville, R., and Venable, J. 2008. "Strategies for Design Science Research Evaluation," European Conference on Information Systems, Galway, Ireland.Google Scholar
- Robson, C. 2002. Real World Research, (2nd ed.) Malden, MA: Blackwell Publishers.Google Scholar
- Rossiter, J.R. 2002. "The C-OAR-SE Procedure for Scale Development in Marketing," International Journal of Research in Marketing (19:4), pp 305--335.Google ScholarCross Ref
- Schuemmer, T. 2003. "Evolving a Group Pattern Language," Fifth International Workshop on Collaborative Editing, Helsinki, Finland.Google Scholar
- Sproull, N.L. 1995. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Sciences, (2nd ed.) London: The Scarecrow Press, Inc.Google Scholar
- Straub, D.W. 1989. "Validating Instruments in MIS Research," MIS Quarterly, (13:2), pp 146--169.Google Scholar
- Straub, D.W., Boudreau, M.-C., and Gefen, D. 2004. "Validation Guidelines for IS Positivist Research," Communications of the Association for Information Systems (13:Article 24), pp 380--427.Google Scholar
- Tidwell, J. 2005. Designing Interfaces: Patterns for Effective Interaction Design, Sebastopal, CA: O'Reilly Media. Google ScholarDigital Library
- Trochim, W. 2000. The Research Methods Knowledge Base, (2 ed.) Cincinnati, OH: Atomic Dog Publishing.Google Scholar
- Vaishnavi, V., and Kuechler, W. 2008. "Design Research in Information Systems," AISNet (available online at http://ais.affiniscape.com/displaycommon.cfm?an=1&subarticlenbr=279).Google Scholar
- Vaishnavi, V.K., and Kuechler, W. 2007. Improving and Innovating Information & Communication Technology: Design Science Research Methods and Patterns, Taylor & Francis. Google ScholarDigital Library
- Venable, J. 2006a. "A Framework for Design Science Research Activities," Information Resource Management Association Conference, Washington D.C., pp. 184--187.Google Scholar
- Venable, J. 2006b. "The Role of Theory and Theorizing in Design Science Research," First International Conference on Design Science Research in Information Systems & Technology, Claremont, CA.Google Scholar
- Walls, J.G., Widmeyer, G.R., and El Sawy, O.A. 1992. "Building an Information System Design Theory for Vigilant EIS," Information Systems Research (3:1), pp 36--59.Google Scholar
- Wania, C.E., and Atwood, M.E. 2009. "Pattern Languages in the Wild: Exploring Pattern Languages in the Laboratory and in the Real World," Design Science Research in Information Systems and Technologies, Philadelphia, PA. Google ScholarDigital Library
- Weick, K.E. 1989. "Theory Construction as Disciplined Imagination," Academy of Management Review (14:4), pp. 516--531.Google ScholarCross Ref
- Yoshioka, T., Herman, G., Yates, J., and Olikowski, W. 2001. "Genre Taxonomy: A Knowledge Repository of Communicative Actions," ACM Transactions on Information Systems (19:4), pp 431--456. Google ScholarDigital Library
Recommendations
Understanding the Power of Abstraction in Patterns
This unusual experience report about mining and documenting patterns describes lessons learned from writing a set of patterns. Many times, pattern stories report on an author's expertise and creation of a set of patterns and their application in that ...
A Framework of Patterns Applicability in Software Development
COMPSACW '14: Proceedings of the 2014 IEEE 38th International Computer Software and Applications Conference WorkshopsPatterns and pattern languages are widely used nowadays. The need to apply patterns in the right sequence is very important, but how to apply them using many pattern languages or pattern catalogue is sometimes very difficult. After examining the current ...
Design science research evaluation
DESRIST'12: Proceedings of the 7th international conference on Design Science Research in Information Systems: advances in theory and practiceThe consensus view is that the rigorous evaluation of design science (DS) artifacts is essential. There are many types of DS artifacts and many forms of evaluation; what is missing is guidance for how to perform the evaluation, more specifically, what ...
Comments