skip to main content
10.1145/2950290.2950315acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Efficiency of projectional editing: a controlled experiment

Published:01 November 2016Publication History

ABSTRACT

Projectional editors are editors where a user's editing actions directly change the abstract syntax tree without using a parser. They promise essentially unrestricted language com position as well as flexible notations, which supports aligning languages with their respective domain and constitutes an essential ingredient of model-driven development. Such editors have existed since the 1980s and gained widespread attention with the Intentional Programming paradigm, which used projectional editing at its core. However, despite the benefits, programming still mainly relies on editing textual code, where projectional editors imply a very different -- typically perceived as worse -- editing experience, often seen as the main challenge prohibiting their widespread adoption. We present an experiment of code-editing activities in a projectional editor, conducted with 19 graduate computer-science students and industrial developers. We investigate the effects of projectional editing on editing efficiency, editing strategies, and error rates -- each of which we also compare to conventional, parser-based editing. We observe that editing is efficient for basic-editing tasks, but that editing strategies and typical errors differ. More complex tasks require substantial experience and a better understanding of the abstract-syntax-tree structure -- then, projectional editing is also efficient. We also witness a tradeoff between fewer typing mistakes and an increased complexity of code editing.

References

  1. CamStudio Desktop Screen Recorder. http://sourceforge.net/projects/camstudio.Google ScholarGoogle Scholar
  2. Code Orchestra IDE. http://codeorchestra.com/ide.Google ScholarGoogle Scholar
  3. Concrete. http://concrete-editor.org.Google ScholarGoogle Scholar
  4. Jetbrains MPS. http://www.jetbrains.com/mps.Google ScholarGoogle Scholar
  5. mbeddr. http://mbeddr.com.Google ScholarGoogle Scholar
  6. mbeddr Graphical State Machines. http://mbeddr.com/2015/03/05/graphicalSM.html.Google ScholarGoogle Scholar
  7. MPS Tutorial. http://www.jetbrains.com/mps/docs/tutorial.html.Google ScholarGoogle Scholar
  8. Online Appendix. http://gsd.uwaterloo.ca/projectional-workbenches.Google ScholarGoogle Scholar
  9. Whole Platform. http://whole.sourceforge.net.Google ScholarGoogle Scholar
  10. T. W. Anderson and J. D. Finn. The New Statistical Analysis of Data. Springer, 1996.Google ScholarGoogle ScholarCross RefCross Ref
  11. Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological), 57(1):289–300, 1995.Google ScholarGoogle ScholarCross RefCross Ref
  12. A. F. Blackwell. First steps in programming: A rationale for attention investment models. In Proc. HCC, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. M. Broy, S. Kirstan, H. Krcmar, and B. Schätz. What is the Benefit of a Model-Based Design of Embedded Software Systems in the Car Industry? In J. Rech and C. Bunse, editors, Emerging Technologies for the Evolution and Maintenance of Software Models. IGI Global, 2011.Google ScholarGoogle Scholar
  14. M. Christerson and H. Kolk. Domain expert DSLs, 2009. talk at QCon London 2009, available at http://www.infoq.com/presentations/ DSL-Magnus-Christerson-Henk-Kolk.Google ScholarGoogle Scholar
  15. T. Clark. A Declarative Approach to Heterogeneous Multi-Mode Modelling Languages. In Proc. GEMOC, 2014.Google ScholarGoogle Scholar
  16. N. Cliff. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3):494, 1993.Google ScholarGoogle ScholarCross RefCross Ref
  17. J. O. Coplien. Multi-paradigm Design for C++. Addison-Wesley, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. K. Czarnecki and U. W. Eisenecker. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. S. Erdweg, P. G. Giarrusso, and T. Rendel. Language Composition Untangled. In Proc. LDTA, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. Erdweg, T. van der Storm, M. Völter, M. Boersma, R. Bosman, W. R. Cook, A. Gerritsen, A. Hulshout, S. Kelly, A. Loh, et al. The State of the Art in Language Workbenches. In Proc. SLE, 2013.Google ScholarGoogle Scholar
  21. M. Fowler. Language Workbenches: The Killer-App for Domain Specific Languages? http://www.martinfowler. com/articles/languageWorkbench.html, 2005.Google ScholarGoogle Scholar
  22. J. Gray and G. Karsai. An Examination of DSLs for Concisely Representing Model Traversals and Transformations. In Proc. HICSS, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Höst, B. Regnell, and C. Wohlin. Using Students As Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empirical Softw. Engg., 5(3):201–214, Nov. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Hutchinson, J. Whittle, M. Rouncefield, and S. Kristoffersen. Empirical Assessment of MDE in Industry. In Proc. ICSE, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. R. B. Kieburtz, L. McKinney, J. M. Bell, J. Hook, A. Kotov, J. Lewis, D. P. Oliva, T. Sheard, I. Smith, and L. Walton. A Software Engineering Experiment in Software Component Generation. In Proc. ICSE, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. J. Ko, H. H. Aung, and B. A. Myers. Design Requirements for More Flexible Structured Editors from a Study of Programmers’ Text Editing. In Proc. CHI EA, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A. Kuhn, G. C. Murphy, and C. A. Thompson. An Exploratory Study of Forces and Frictions Affecting Large-Scale Model-Driven Development. In Proc. MODELS. 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. C. M. Lewis. How programming environment shapes perception, learning and goals: Logo vs. scratch. In Proc. SIGCSE, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. G. Liebel, N. Marko, M. Tichy, A. Leitner, and J. Hansson. Assessing the state-of-practice of model-based engineering in the embedded systems domain. In Proc. MODELS, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  30. P. Liggesmeyer and M. Trapp. Trends in Embedded Software Engineering. IEEE Softw., 26(3):19–25, May 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Lillack, T. Berger, and R. Hebig. Experiences from reengineering and modularizing a legacy software generator with a projectional language workbench. In Proc. SPLC. 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. R. Medina-Mora and P. H. Feiler. An Incremental Programming Environment. IEEE Trans. Softw. Eng., 7(5):472–482, Sept. 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. R. C. Miller and B. A. Myers. Multiple Selections in Smart Text Editing. In Proc. IUI, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Z. Molotnikov, M. Völter, and D. Ratiu. Automated domain-specific C verification with mbeddr. In Proc. ASE, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. D. Notkin. The GANDALF Project. J. Syst. Softw., 5(2):91–105, May 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. F. Poller and S. K. Garter. A Comparative Study of Moded and Modeless Text Editing by Experienced Editor Users. In Proc. CHI, 1983. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. S. W. Porter. Design of a Syntax Directed Editor for PSDL (Prototype Systems Design Language). Master’s thesis, Naval Postgraduate School, Monterey, CA, USA, 1988.Google ScholarGoogle Scholar
  38. D. Ratiu, B. Schaetz, M. Voelter, and B. Kolb. Language engineering as an enabler for incrementally defined formal analyses. In Proc. FormSERA, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. T. W. Reps and T. Teitelbaum. The Synthesizer Generator. In Proc. SDE, 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. P. Runeson. Using Students as Experiment Subjects—An Analysis on Graduate and Freshmen Student Data. In Proc. EASE, 2003.Google ScholarGoogle Scholar
  41. B. Selic. The Pragmatics of Model-Driven Development. IEEE Softw., 20(5):19–25, Sept. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. W. Shadish, T. Cook, and D. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2002.Google ScholarGoogle Scholar
  43. M. Simi and F. Campagne. Composable Languages for Bioinformatics: The NYoSh Experiment. PeerJ, 2:e241, 2014.Google ScholarGoogle Scholar
  44. C. Simonyi. The death of computer languages, the birth of intentional programming. In Proc. NATO Science Committee Conference, 1995.Google ScholarGoogle Scholar
  45. C. Simonyi, M. Christerson, and S. Clifford. Intentional Software. In Proc. OOPSLA, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. A. Strauss and J. Corbin. Open Coding. Basics of Qualitative Research: Grounded Theory Procedures and Techniques, 2:101–121, 1990.Google ScholarGoogle Scholar
  47. M. W. Van Someren, Y. F. Barnard, J. A. Sandberg, et al. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. Academic Press London, 1994.Google ScholarGoogle Scholar
  48. E. Visser. WebDSL: A Case Study in Domain-Specific Language Engineering. In R. Lämmel, J. Visser, and J. a. Saraiva, editors, Generative and Transformational Techniques in Software Engineering II, pages 291–373. Springer, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. M. Voelter. mbeddr C User Guide. Itemis AG. http://mbeddr.com/userguide/UserGuideExport.html.Google ScholarGoogle Scholar
  50. M. Voelter. Language and IDE Modularization and Composition with MPS. In GTTSE, LNCS. Springer, 2011.Google ScholarGoogle Scholar
  51. M. Voelter and S. Lisson. Supporting Diverse Notations in MPS’ Projectional Editor. 2014.Google ScholarGoogle Scholar
  52. M. Voelter, D. Ratiu, B. Kolb, and B. Schätz. mbeddr: Instantiating a Language Workbench in the Embedded Software Domain. Autom. Softw. Eng., 20(3):339–390, 2013.Google ScholarGoogle ScholarCross RefCross Ref
  53. M. Voelter, D. Ratiu, B. Schaetz, and B. Kolb. mbeddr: an extensible c-based programming language and ide for embedded systems. In Proceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity, pages 121–140. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. M. Voelter, D. Ratiu, and F. Tomassetti. Requirements as First-Class Citizens. In Proc. ACES-MB, 2013.Google ScholarGoogle Scholar
  55. M. Voelter, J. Siegmund, T. Berger, and B. Kolb. Towards User-Friendly Projectional Editors. In Proc. SLE, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  56. M. Voelter, A. van Deursen, B. Kolb, and S. Eberle. Using C Language Extensions for Developing Embedded Software: A Case Study. In Proc. OOPSLA, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. M. Voelter, A. van Deursen, B. Kolb, and S. Eberle. Using c language extensions for developing embedded software: A case study. In Proc. OOPSLA, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Efficiency of projectional editing: a controlled experiment

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
      November 2016
      1156 pages
      ISBN:9781450342186
      DOI:10.1145/2950290

      Copyright © 2016 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 November 2016

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate17of128submissions,13%

      Upcoming Conference

      FSE '24

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader