Suporn Pongnumkul
ABSTRACT
Video tutorials provide a convenient means for novices to learn new software applications. Unfortunately, staying in sync with a video while trying to use the target application at the same time requires users to repeatedly switch from the application to the video to pause or scrub backwards to replay missed steps. We present Pause-and-Play, a system that helps users work along with existing video tutorials. Pause-and-Play detects important events in the video and links them with corresponding events in the target application as the user tries to replicate the depicted procedure. This linking allows our system to automatically pause and play the video to stay in sync with the user. Pause-and-Play also supports convenient video navigation controls that are accessible from within the target application and allow the user to easily replay portions of the video without switching focus out of the application. Finally, since our system uses computer vision to detect events in existing videos and leverages application scripting APIs to obtain real time usage traces, our approach is largely independent of the specific target application and does not require access or modifications to application source code. We have implemented Pause-and-Play for two target applications, Google SketchUp and Adobe Photoshop, and we report on a user study that shows our system improves the user experience of working with video tutorials.
Supplemental Material
- L. Bergman, V. Castelli, T. Lau, and D. Oblinger. Docwizards: a system for authoring follow-me documentation wizards. In Proc. ACM UIST, pages 191--200, 2005. Google Scholar
Digital Library
- K.-Y. Cheng, S.-J. Luo, B.-Y. Chen, and H.-H. Chu. Smartplayer: user-centric video fast-forwarding. In Proc. SIGCHI, pages 789--798, 2009. Google ScholarDigital Library
- M. Dixon and J. Fogarty. Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. In Proc. SIGCHI, pages 1525--1534, 2010. Google ScholarDigital Library
- S. Gilbert, S. B. Blessing, and S. Kodavali. The extensible problem-specific tutor (xpst): Evaluation of an api for tutoring on existing interfaces. In Proc. the conf. Artificial Intelligence in Education, pages 707--709, 2009. Google ScholarDigital Library
- F. Grabler, M. Agrawala, W. Li, M. Dontcheva, and T. Igarashi. Generating photo manipulation tutorials by demonstration. In Proc. ACM SIGGRAPH, pages 1--9, 2009. Google ScholarDigital Library
- T. Grossman and G. Fitzmaurice. Toolclips: an investigation of contextual video assistance for functionality understanding. In Proc. SIGCHI, pages 1515--1524, 2010. Google ScholarDigital Library
- T. Grossman, J. Matejka, and G. Fitzmaurice. Chronicle: capture, exploration, and playback of document workflow histories. In Proc. ACM UIST, pages 143--152, 2010. Google ScholarDigital Library
- S. M. Harrison. A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface. In Computer Human Interaction, pages 82--89, 1995. Google ScholarDigital Library
- C. Hategekimana, S. Gilbert, and S. Blessing. Effectiveness of using an intelligent tutoring system to train users on off-the-shelf software. In Proc. Society for Info. Tech. and Teacher Education Int'l Conf., AACE, 2008.Google Scholar
- C. Kelleher and R. Pausch. Stencils-based tutorials: design and evaluation. In Proc. SIGCHI, pages 541--550, 2005. Google ScholarDigital Library
- J. Matejka, T. Grossman, and G. Fitzmaurice. Ambient help. In Proc. SIGCHI, pages 2751--2760, 2011. Google ScholarDigital Library
- S. Palmiter and J. Elkerton. An evaluation of animated demonstrations of learning computer-based tasks. In Proc SIGCHI, pages 257--263, 1991. Google ScholarDigital Library
- N. Petrovic, N. Jojic, and T. S. Huang. Adaptive video fast forward. Multimedia Tools Appl., 26:327--344, 2005. Google ScholarDigital Library
- S. Pongnumkul, J. Wang, G. Ramos, and M. Cohen. Content-aware dynamic timeline for video browsing. In Proc. ACM UIST, pages 139--142, 2010. Google ScholarDigital Library
- B. Shneiderman. Direct manipulation: A step beyond programming languages. Computer, 16(8):57--69, 1983. Google ScholarDigital Library
- S. L. Su. Enhanced Visual Authoring Using Operation History. PhD thesis, Massachusetts Institute of Technology, Boston, Massachusetts, 2009. Google ScholarDigital Library
- T. Yeh, T.-H. Chang, and R. C. Miller. Sikuli: using gui screenshots for search and automation. In Proc. ACM UIST, pages 183--192, 2009. Google ScholarDigital Library
Index Terms
- Pause-and-play: automatically linking screencast video tutorials with applications
Recommendations
MixT: automatic generation of step-by-step mixed media tutorials
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technologyUsers of complex software applications often learn concepts and skills through step-by-step tutorials. Today, these tutorials are published in two dominant forms: static tutorials composed of images and text that are easy to scan, but cannot effectively ...
MixT: automatic generation of step-by-step mixed media tutorials
CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing SystemsAs software interfaces become more complicated, users rely on tutorials to learn, creating an increasing demand for effective tutorials. Existing tutorials, however, are limited in their presentation: Static step-by-step tutorials are easy to scan but ...
VT-revolution: interactive programming tutorials made possible
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringProgramming video tutorials showcase programming tasks and associated workflows. Although video tutorials are easy to create, it is often difficult to explore the captured workflows and interact with the programs in the videos. In this work, we propose ...
Comments