Skip to main content

Über dieses Buch

This book highlights the latest research in pen and touch, its current use in STEM classrooms, sketching and haptics technologies. Computer and educational scientists from academia and industry presented their research at the Conference on Pen and Touch Technology on Education (CPTTE) 2017 on the advancement of digital ink technology and its applications for college and K-12 classrooms. This book is the synthesis of the presented results and the ideas generated from conference discussions.

This volume contains seven parts; exploring topics like sketching forensics, teaching STEM, sketch recognition applications, creating a learning environment with sketching, teaching to sketch, and haptics. The book focuses on intelligent systems using digital ink that enable pen and touch interaction that teach and inspire students. Inspiring Students through Digital Ink is a must-read for anyone wanting to improve today’s student experiences and apply innovative approaches in the classroom. Also highlighted are current and future directions in pen and touch research.



Introductions and Welcome


Chapter 1. Inspiring Students Through Digital Ink: An Introduction

Derived from contributions to the Conference on Pen and Touch Technology in Education (CPTTE) in 2017, this edited volume highlights recent developments for pen and tablet research within the domains of computer science, education, and outreach. This book will be of great interest for researchers and educators at every level. The book includes seven sections and explores topics on sketching forensics in education, sketch recognition and teaching STEM through sketch, teaching sketching skills, creating your learning environment with sketching, and forward directions with haptics. In this book, you will find new ideas and future perspectives in digital learning.
Anna Stepanova, Manoj Prasad, Tracy Hammond

Sketching Forensics in Education: What We Can Learn from How You Sketch


Chapter 2. The Quantified Student: Using Smartpens and Data Mining to Understand Student Learning and Achievement

Dr. Thomas Stahovich gave a keynote in the morning of the first day of the conference. He showed how researchers have long sought to understand the effects of study skills on academic achievement but found no consistent relationship between them. Dr. Stahovich explained that this is due, in part, to the use of research methods that rely on surveys and students’ self-reports of study habits. In his work, he overcomes this limitation by using smartpens and an instrumented document viewer to objectively measure studying. This combination of technology provides a fine-grained view of the learning process not available with conventional assessment methods and enables the use of data mining to examine the relationship between studying and achievement. In his talk he presented novel data mining techniques, as well as the results of several studies that reveal new insights about the relationship between traditional learning activities—completing homework, taking lecture notes, and reading—and performance in introductory engineering courses. Finally, Dr. Stahovich discussed interventions that are based on these insights and are designed to improve student engagement and increase academic achievement. This chapter provides an edited transcription of that keynote. Thank you to David Hoeft for videotaping the sessions.
Thomas Stahovich

Teaching STEM Through Sketch


Chapter 3. Repairing Misconceptions and Inadequate Preparation in Upper-Level Undergraduate Engineering: The KACIE Model and Pen-Based Computing

Engineering educators frequently discover that students coming into upper-level courses lack the foundational knowledge necessary for course success, and frequently entertain misconceptions common to entry-level engineering students. This chapter reports on an NSF-funded initiative that approaches this difficulty with a comprehensive portfolio of research-based strategies for students to acquire, manipulate, express, and revise conceptual systems around engineering. These strategies referred to as protocols, entail a labor-intensive, methodical, and stepwise approach to working with upper-level students with less course preparation than they should have. The overall approach is referred to as “Knowledge and Curriculum Integration Ecosystem (KACIE)” protocol, formerly known as “Tailored Instruction and Engineered Delivery Using Protocols (TIDE UP)”. One aspect of the strategy portfolio involves the use of pen-based computing. The chapter reviews statistically significant results from this approach as a prelude to ongoing research to assess the efficacy of the digital ink component of the project (Hamilton et al. Inspiring students through digital ink: impact of pen and touch technology on education. Springer, 2019).
John Solomon, Eric Hamilton, Chitra Nayak

Chapter 4. Sub-Saharan Teacher STEM Professional Development with Digital Ink: Follow-Up (Sub-Saharan Pen-Based Computing Follow-Up)

This chapter reports on research activities originally reported at the 2015 WIPTTE conference, the last one held prior to the conference name change. The chapter, entitled “A Model and Research Agenda for Teacher and Student Collaboration Using Tablets in Digital Media Making: Results from Sub-Saharan Workshops” reported on a series of science, technology, engineering, and mathematics (STEM) projects funded by the Kenyan Ministry of Education, Microsoft Research, and the National Science Foundation. The research has continued since then, with additional support from the Namibian Ministry of Education, Arts and Culture, the US State Department’s Fulbright Program, and new, multiyear funding from NSF. The chapter describes the evolution of the project and five themes emerging from it, in addition to describing a new four-year effort related to the original paper. The original paper focused on activities in Kenya, Ghana, Uganda, and Namibia. This chapter centers on follow-up activities in Namibia.
Eric Hamilton, Loide Kapenda, Helena Miranda, Elizabeth Ngololo

Sketch Recognition in STEM


Chapter 5. Qualitative Assessment of CogSketch Worksheets in Introductory Geoscience Laboratory Courses: An Argument for Expanded Use

The use of computer-assisted instruction to tutor students in the geosciences is not common, in part because practical programs capable of assessing sketches relevant to introductory geoscience subjects are relatively new. One such program, CogSketch, is an open-domain sketch understanding system that is capable of tutoring students on sketch-based exercises and grading their work. This chapter reports on the mixed use of paper-based exercises and tablet PCs equipped with CogSketch worksheets in introductory geoscience laboratory courses. Potential benefits of using CogSketch worksheets include increased group focus and reduced barriers to starting laboratory exercises. Grading efficiency was not affected. However, increasing the proportion of CogSketch worksheets used during lab sessions will improve grading efficiency and amplify benefits in classroom social dynamics and teaching efficiency that may lead to learning gains.
Wayne T. Marko

Chapter 6. Flow2Code: Transforming Hand-Drawn Flowcharts into Executable Code to Enhance Learning

Flowcharts serve as a great tool for new students learning programming logic in a language-independent fashion. However, current software tools for creating flowcharts on the computer have various shortcomings. UI-based solutions make creating flowcharts difficult through inefficient drag-and-drop menus. Sketching-based solutions take a more intuitive approach but don’t help the students beyond having pseudocode to start with. Flow2Code, on the other hand, enables students to draw flowcharts and translate them into code in a helpful way of bridging the gap between the two languages. Flow2Code can identify and interpret flowcharts drawn on paper using an algorithm for recognizing various shapes used in flowcharts. It fills a gap in the current solutions by converting the flowcharts into executable code and by having an intuitive and interactive interface where the users can edit both their flowchart and its resulting code.
Samantha Ray, Jorge Iván Herrera-Cámara, Matthew Runyon, Tracy Hammond

Chapter 7. FreeStyle: A Sketch-Based Wireframing Tool

User interfaces have classically involved WIMP (windows, icons, mouse, and pointing) paradigm. Current wireframing tools in the early stages of web interface design focus on using drag-and-drop features, that is, selecting elements from a menu and inserting them in the interface. Most designers prefer to sketch early interface ideas on paper. A system that allows them to draw sketches would make the process more intuitive, thus going beyond traditional methods of website interface designing. We have developed an interactive tool called FreeStyle that allows designers the ability to quickly sketch a web-based interface electronically. The designer can draw web-based HTML elements on the canvas and in response, FreeStyle transforms the sketch into an interface. Additionally, we also provide the functionality of downloading the code developed from the sketch.
Savinay Narendra, Sheelabhadra Dey, Josiah Coad, Seth Polsley, Tracy Hammond

Teaching Sketching Skills


Chapter 8. Improvement in Freehand Sketching Application for Spatial Visualization Training

Spatial visualization is the cognitive ability of mentally representing and manipulating two-dimensional and three-dimensional figures. This is a learnable cognitive skill that has been correlated with increased GPA’s and retention in science, technology, engineering, and math (STEM) disciplines. Traditional spatial visualization training includes freehand sketching assignments, which require human grading. A spatial visualization training application (app) was developed in which students freehand sketch on a touchscreen and an automatic grading algorithm provides immediate feedback. In spring 2014, the app was used with iPads in a one-unit spatial visualization course where students took pre- and post-spatial visualization assessment tests. In 2014, 46% of the students who scored low on the pre-assessment had a significant improvement of 10% or more on the post-assessment. This chapter describes how the app was modified to increase student gains: the user interface and assignments were modified, taking advantage of color and cues that are not present in the traditional paper-based sketching assignments; and gamification was added to motivate increased persistence. The course was taught with the newly modified app in 2017, during which 82% of incoming students with low spatial skills showed significant improvement. The increase from 46 to 82% is attributed to the capability of pen and touch technology to be adapted effectively for educational purposes.
Elizabeth Cowan, Nathan Delson, Bob Mihelich, Lelli Van Den Einde

Chapter 9. Teaching Engineering Students Freehand Sketching with an Intelligent Tutoring System

Sketching is an essential skill in engineering design. However, the instruction of sketching in engineering curriculum has greatly diminished in recent decades. Previous studies have shown that teaching an Industrial Design-inspired version of free hand sketching is effective in engineering courses, but engineering professors are often untrained to teach this method effectively. This chapter studies the effect of supplementing instructor-taught sketching with an online sketching tutor that utilizes artificial intelligence to provide human-like feedback to user sketches. To measure the impact of the introduction of this program, the authors compared students who used the online tutor against students who only completed paper-based sketching homework using pre- and post-course spatial visualization evaluations and sketching quizzes. The results show that the students using the online tutor significantly improve their spatial visualization skills at a level equivalent to students who had more practice with pen-and-paper sketching.
Ethan Hilton, Blake Williford, Wayne Li, Tracy Hammond, Julie Linsey

Creating Your Learning Environment with Sketching


Chapter 10. Personalizing Student Learning and Enhancing Peer-to-Peer Networks Through the Pen-Based MyEduDecks Application

MyEduDecks is a pen-based flashcard application that has been in development at South Fayette High School since 2013. The project initially began as a South Fayette student project under the supervision of computer science professor Ananda Gunawardena of Carnegie Mellon University. The MyEduDecks team has extended the first iteration of the application to meet the user needs at South Fayette. This application intends to personalize student’s learning and build collaborative networks in the classroom by leveraging pen-based technologies and their benefits. The application encourages the user to use a stylus to input questions and answers in an electronic flashcard format. MyEduDecks uses the Microsoft Ink API to integrate digital ink into the app. This research chapter highlights our application, its continued development, impact on a K-12 learning institution, and both qualitative and quantitative data analyzing the impact of an application reliant on pen and touch technology.
Parv Shrivastava, Prateek Jukalkar, Aileen Owens

Chapter 11. iCanTrace: Avatar Personalization Through Selfie Sketches

This chapter introduces a novel system that allows users to generate customized cartoon avatars through a sketching interface. The rise of social media and personalized gaming has given a need for personalized virtual appearances. Avatars, self-curated and customized images to represent oneself, have become a common means of expressing oneself in these new media. Avatar creation platforms face the challenges of granting user significant control over the avatar creation and of encumbering the user with too many choices in their avatar customization. This chapter demonstrates a sketch-guided avatar customization system and its potential to simplify the avatar creation process.
Alexander Berman, Leela Krishna Chaitanya Gottumukkala, Zepeng Huo, Seth Polsley, Francis Quek, Tracy Hammond

Forward Directions with Haptics


Chapter 12. Touching with Feeling

Dr. Edward Colgate gave a keynote on morning of the second day of the conference. He spoke about haptics and their vital role in everyday life. He showed that, on contrary, haptics are decidedly less important when interfacing to the digital world. Our touch screens treat the finger as a pointer and more or less ignore the thousands of mechanoreceptors within, not to mention the somatosensory cortex. This is an odd state of affairs. Dr. Colgate explained that part of the reason is technological: how do you go beyond simple vibrations and create versatile, programmable haptic experiences? Research in his lab has focused on solving this question for better than a decade now. In his talk, he explained the rapidly expanding capabilities of “surface haptic” technologies. Dr. Colgate also showed how limited are the data on haptics used to create great experiences, be they for education, entertainment, or elsewhere. That turns out to be a much more difficult problem. He described some incipient efforts and offered some thoughts for the future. This chapter provides a lightly edited transcription of that keynote.
J. Edward Colgate
Weitere Informationen