Many online educational materials for STEM are now produced with
Microsoft PowerPoint (PPT); however, many of them are not necessarily accessible for print-disabled students. By making use of Infty software, our new add-on allows users to add easily alt text/aloud reading with a TTS voice to any technical part such as math expressions included in PPT slides. An accessible MP4 video for STEM education also can be produced efficiently. The video contents produced in this manner are preliminarily evaluated by comparison with another accessible content: “ChattyBook" (audio-embedded HTML5).
1 Introduction
All of us are still in the coronavirus pandemic that is a rare crisis in modern history. Fortunately, however, digitization provides us many powerful tools to get past this serious situation. For instance, digitized content is actively used in online/distance education everywhere in the world.
Many online educational materials for sighted students are produced with Microsoft PowerPoint (PPT). They are provided as lecture videos in MP4 and/or PDF. Otherwise, the PPT slides are displayed directly on a computer screen in remote lectures. In an inclusive class, if the lecture were face-to-face, a teacher could explain directly any part of the PPT or PPT-originated educational materials as necessary even if they were not necessarily accessible. On the other hand, in case of distance education, print-disabled students should access the entire content for themselves at home.
Anzeige
Currently, various method to make PPT presentations accessible are suggested by Microsoft or other organizations [1]. Certainly, in terms of non-technical contents, we actually can produce accessible PPT contents in a certain level, based on those methods. However, as far as STEM (science, technology, engineering and math) contents are concerned, there is still a serious difficulty. In many cases, print-disabled students hardly access important elements such as math formulas, diagrams and tables included in STEM educational materials created with PPT.
Math accessibility has been remarkably improved for the past more-than-20 years. For instance, “Digital Accessible Information System” (DAISY), which is an international standard of accessible digital books, has adopted MathML to represent mathematical formulas (DAISY Consortium n.d.), and some DAISY/ accessible EPUB3 players can read out mathematical formulas in MathML with a text-to-speech (TTS) voice. In terms of web accessibility, recently, MathJax is widely used to represent MathML formulas on the web. the current version of JAWS, the most popular screen reader in Windows OS, can read out such mathematical contents to a certain extent (MathJax n.d.; Freedom Scientific, Inc. n.d.). Unfortunately, however, you do not have such a good tool to make PowerPoint or PowerPoint-originated STEM contents be accessible for the present.
As was mentioned, in terms of non-technical content, we can produce accessible educational materials with PowerPoint [1]. Print-disabled students can read them with ordinary assistive tools such as a screen reader, a Braille translator, etc. However, as far as a technical part such as a math expression is concerned, embedding an alternative text (alt text) is usually only effective countermeasures to make it accessible. Accessible math editors such as EquatIO [2] can help you with preparing an alt text for a math expression in a PPT slide, but embedding such alt texts in the slide is not easy and requires to teachers a lot of extra time and efforts. Furthermore, if the content were changed, some alt texts should be revised as well, but all the corrections process should be done manually.
Another solution to make PPT contents accessible is to add audio (aloud reading) information to them as audio objects. You could add the explanation in a recorded voice to each slide and export the result to a MP4 video. However, this process should require also a lot of extra manual work.
Anzeige
For more-than-twenty years, our group has been developing assistive tools for print-disabled people to access STEM contents.“ChattyInfty" [3], which is our accessible math-document editor, has a function of reading out a content including math formulas with a TTS voice. It also has a function of exporting a document in other accessible formats such as multimedia DAISY, audio-embedded EPUB3, etc. “InftyReader" [3], which is our OCR (optical character recognition) software for STEM contents, can recognize properly a document in print or PDF and convert the result into ChattyInfty document or other accessible formats. It is expected that they should be useful to make PowerPoint STEM contents accessible; however, the collaboration of PowerPoint with Infty software has not yet been realized.
Here, we show a new efficient method to make PPT or PPT-originated STEM educational content be accessible by making use of Infty software [3]. Using its functions, our new add-on for PowerPoint allows you to embed easily an alt text and aloud reading with a TTS voice to any technical part such as math expressions included in PPT slides. An accessible MP4 video for STEM education also can be produced efficiently. The video contents produced in this manner are preliminarily evaluated by comparison with another accessible content: “ChattyBook" (audio-embedded HTML5).
2 Method
As was mentioned, we have recently developed a PowerPoint add-on with VBA (Visual Basic for Application) to realize a systematic/efficient-processing mechanism for embedding alt texts or/and aloud reading in PPT contents [4]. This new add-on requires that computer OS is Microsoft Windows 8.1 or later and that PowerPoint, ChattyInfty and InftyReader are all installed and work properly.
The add-on provides you with three ways to embed the alt text or/and the MP3 audio file (aloud reading) for each math part in PPT slides.
(1)
Converting a math part in an existing PPT slide into the alt text and the MP3 audio file and embedding. By copying a selected math part in the PPT slide, it is stored on the Windows clipboard as an image. The add-on uses VBA shell functions to recognize the image automatically with InftyReader, its command-line application and to paste the result into ChattyInfty in an editable form. If necessary, the result can be corrected in an intuitive manner, and then, it can be converted manually into its word description (alt text) or/and MP3 audio file (aloud reading) with a TTS voice. In the next step, the add-on converts the generated audio file into a PPT audio object and embeds it at the background of the original math part in the slide. The generated text is treated as an alt text for this audio object and also embedded. In addition, the math part in ChattyInfty format is added to the background as well.
(2)
Converting a math part in PDF into the alt text or/and the MP3 audio file. If a PDF or other printed source of a PPT slide is available, you may copy a selected math part from it with a Snapshot function in Acrobat Reader. The add-on also can generate the alt text or/and the MP3 audio file in the same manner as is described (1).
(3)
Newly creating a math formula together with its alt text or/and MP3 audio file. When editing a PPT slide, the add-on allows you to start up ChattyInfty to create newly a math formula. You can insert it into the slide together with its alt text or/and MP3 audio file.
In addition, the add-on also allows you to revise/correct easily the alt text or the MP3 audio file that is already inserted in the slide. By clicking the math part, at the background of which the audio object is embedded, ChattyInfty opens automatically to display its ChattyInfty form, and you can revise/correct it as you like. The previous audio object will be replaced automatically with the revised one.
In summary, using the add-on, you can make PPT STEM contents accessible systematically/efficiently in the following workflow (Fig. 1).
Fig. 1.
An overview of a systematic/efficient-processing mechanism for adding alt text or/and audio information (aloud reading) in PPT contents.
×
From the viewpoint of software GUI (graphic user interface), by installing the add-on, two new items appear in the PowerPoint ribbon menu: “Add Recognized Result as" and “Add New Math Item".
Using “Add Recognized Result as", you can select a math part in a PPT slide and paste it into ChattyInfty. As was mentioned, its snapshot is automatically recognized by InftyReader, and the result is pasted into ChattyInfty in an editable form. After correcting recognition errors (if necessary), the math formula is converted into its word description (alt text) or/and aloud reading with a TTS engine. You can add that alt text and/or aloud reading at the indicated position in the original Slide.
“Add New Math Item" starts up ChattyInfty automatically. As was also mentioned, if a math formula is not written yet in the PPT slide, you may create the necessary formula directly in ChattyInfty (which is easily done in an intuitive manner) and insert it into the PPT slide together with alt text and/or aloud reading.
By making use of this method, teachers can easily/efficiently make math parts in PPT slides be accessible. Furthermore, ChattyInfty can read out entire STEM contents and convert that into a MP3 audio file. If a lecture script for PPT slides were prepared in ChattyInfty, you could add the speech presentation generated by ChattyInfty to each slide and convert PPT into a MP4 video, in which all the content including technical parts would be read out properly with a TTS voice.
3 Preliminary Evaluation Report
We actually produce accessible MP4 videos for online education in this manner at Nihon University. Due to the COVID-19 pandemic, in Nihon University, many STEM courses have been given online in 2020 and 2021. In a class entitled “Applied Information Science, Laboratory II", one of the authors, Komada taught about twenty students how to treat statistical data with the software “R."
In 2020, he prepared just PowerPoint slides for each lecture and provided students with those slides in PDF form. In addition, he added the speech presentation in recorded voice to the slides and convert that PPT into a MP4 video. Those materials were provided through a learning management system (LMS) in Nihon University. However, he realized that this method required to him a lot of extra time and efforts. Thus, our team has decided to develop the new method described in the previous section.
In 2021, by making use of the new method, Komada produced a MP4 video in the following steps.
(1)
Convert the PPT slides prepared in 2020 into ChattyInfty; in addition, create/input directly new STEM contents in ChattyInfty or convert existing PDF STEM contents including math formulas into ChattyInfty (also with the math-OCR system, InftyReader).
(2)
Prepare a lecture script for the PPT slides by editing the ChattyInfty content.
(3)
Generate aloud reading of the lecture script in MP3 with ChattyInfty and embed the MP3 files into the slides.
(4)
Convert the PPT slides into a MP4 video.
We realized that this way is certainly more efficient than the previous. In addition, if necessary, you can modify the accessible MP4 video quite easily.
To compare this video with other accessible format, we also prepared the a “ChattyBook" content [5] for the same class. ChattyBook is an accessible web content: audio-embedded HTML5 with JAVA script which has the same operability and functionality as multimedia DAISY.
We issued a questionnaire to the students to ask their impressions on those two-type accessible contents. The rate of the students who judged the content “easy to understand" or “rather easy to understand" are 84% for the MP4 video and 91% for the ChattyBook content in total, respectively.
4 Conclusion and Future Tasks
Here, we show a new efficient method to make PPT or PPT-originated STEM educational content be accessible by making use of Infty software. Using its functions, our new add-on for PowerPoint allows you to embed easily an alt text and aloud reading with a TTS voice to any technical part such as math expressions included in PPT slides. An accessible MP4 video for STEM education also can be produced efficiently.
We intend to work on tasks listed below in the near future. Some of them should be completed by the ICCHP 2022 conference.
(1)
Realizing more-automatic processing in the workflow For the present, there are some manual steps such as selecting each math part in our workflow of adding alt text or/and aloud-reading. It requires you a certain skill to carry out. It should be improved so that the process is done automatically as well as possible.
(2)
Making the add-on multilingual The add-on has been developed for Japanese in first stage. It is implemented by making use of ChattyInfty/InftyReader as PowerPoint VBA. Several other local languages such as English, Vietnamese, French, etc. are available in ChattyInfty/InftyReader, and it is possible to localize the add-on by writing down VBA in each local language. In ICCHP 2022, at least, we will develop its English version and demonstrate how it works.
(3)
Seamless transformation between PowerPoint and ChattyInfty formats We intend to realize more-direct transformation from PPT to ChattyInfty and vice versa. If this function were realized, it would be possible to convert a PPT content (via ChattyInfty) into various other accessible formats such as multimedia DAISY, accessible EPUB3 or audio-embedded HTML5 (ChattyBook). A ChattyInfty content could be also converted into accessible PowerPoint directly. It should give a more efficient way to produce accessible materials for inclusive math education.
(4)
More thorough evaluation for the PPT add-on. We intend to conduct more thorough evaluation test to understand remaining tasks in the PPT add-on, which we should work on. We will provide math teachers with its trial version to ask them to test it on their online STEM lectures. Based on its feedback, we intend to polish up it and to release its official version in the near future.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
