Top

Published in:

2022 | OriginalPaper | Chapter

Developing a Corpus of Hierarchically Classified STEM Images for Accessibility Purposes

Authors : Theodora Antonakopoulou, Paraskevi Riga, Georgios Kouroupetroglou

Published in: Computers Helping People with Special Needs

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Even though considerable efforts have been made to provide effective image descriptions for digital accessibility, a large portion of STEM images, especially complex STEM images, nowadays remains inaccessible to people with visual disability. The quality of alt text is much more critical for university STEM textbooks as image descriptions must be accurate and detailed but not tire out the reader. This work aims to develop a large corpus of hierarchically classified STEM images from university textbooks which later will be used for developing appropriate guidelines for meaningful non-automatic high-quality alt-text image descriptions with the purpose of accessibility in mind. We present first our approach for the creation of the corpus with the STEM images. Our corpus at the current stage includes 8.859 STEM images from 82 textbooks in the domains of Mathematics, Biology, Computer Science, Chemistry, Physics and Geology. Then, we describe the methodology we followed for the classification of the images in the corpus and in particular, the way for the creation of the five categories and twenty-four subcategories, as well as the manner of the assignment of images to categories.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Designing an Inclusive and Accessible Mathematical Learning Environment Based on a Theorem Prover

next chapter Effective Non-visual Access to Diagrams via an Augmented Natural Language Interface

Farhadi, A., et al.: Every picture tells a story: generating sentences from images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 15–29. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_2CrossRef

Bernardi, R., et al.: Automatic description generation from images. J. Artif. Intell. Res. 55, 409–442 (2016). https://doi.org/10.1613/jair.4900CrossRef

Shahira, K.C., Lijiya, A.: Document image classification: towards assisting visually impaired. In TENCON 2019 – 2019 IEEE Region 10 Conference (TENCON), pp. 852–857 (2019). https://doi.org/10.1109/TENCON.2019.8929594

Kavasidis, I., et al.: A saliency-based convolutional neural network for table and chart detection in digitized documents. In: International Conference on Image Analysis and Processing, 2019, pp. 292–302. Springer (2019)

Savva, M., Kong, N., Chhajta, A., Fei-Fei, L., Agrawala, M., Heer, J.: ReVision: automated classification, analysis and redesign of chart images. In: 24th Annual ACM Symposium on User Interface Software and Technology, pp. 393–402 (2011). https://doi.org/10.1145/2047196.2047247

Amara, J., Kaur, P., Owonibi, M., Bouaziz, B.: Convolutional neural network based chart image classification. In: 25th Int. Conf. Central Europe on Computer Graphics, Visualization and Computer Vision, pp. 83–88 (2017)

Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: F. Pereira, F., C.J.C. Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-yMathSciNetCrossRef

10.

Bigham, J.: Increasing web accessibility by automatically judging alternative text quality. In: 12th International Conf. Intelligent User Interfaces, pp. 349–352 (2007). https://doi.org/10.1145/1216295.1216364

11.

Tang, L., Carter, J.A.: Communicating image content. In: The Human Factors and Ergonomics Society 55th Annual Meeting, 2011, pp. 495–499 (2011). https://doi.org/10.1177/1071181311551102

12.

Nengroo, A.S., Kuppusamy, K.S.: Accessible images (AIMS): a model to build self-describing images for assisting screen reader users. Univ. Access Inf. Soc. 17(3), 607–619 (2017). https://doi.org/10.1007/s10209-017-0607-zCrossRef

13.

NCAM: Project History and Research Methodology. http://ncamftp.wgbh.org/ncam-old-site/experience_learn/educational_media/stemdx/research.html. Accessed 1 Feb 2022

14.

DIAGRAM CENTER: Accessible Images Sample Book. http://diagramcenter.org/accessible-image-sample-book.html. Accessed 18 Dec 2021

15.

Shahira, K.C., Lijiya, S.: Towards assisting the visually impaired: a review on techniques for decoding the visual data from chart images. IEEE Access 9, 52926–52943 (2021). https://doi.org/10.1109/ACCESS.2021.3069205CrossRef

16.

Kahou, S.E., Adam, A., Michalski, V., Kadar, A., Trischler, A., Bengio, Y.: Figureqa: an annotated figure dataset for visual reasoning. arXiv Preprint arXiv:1710.07300 (2017). https://doi.org/10.48550/arXiv.1710.07300

17.

Chaudhry, R., Shekhar, S., Gupta, U., Maneriker, P., Bansal, P., Joshi, A.: Leaf-qa: locate, encode attend for figure question answering. In: The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 3501–3510 (2020). https://doi.org/10.1109/WACV45572.2020.9093269

18.

Kafle, K., Price, B., Cohen, S., Kanan, C.: Dvqa: understanding data visualizations via question answering. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 5648–5656 (2018). https://doi.org/10.1109/CVPR.2018.00592

19.

Methani, N., Ganguly, P., Khapra, M.M., Kumar, P.: PlotQA: reasoning over scientific plots. In: IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 1516–1525 (2020). https://doi.org/10.1109/WACV45572.2020.9093523

20.

Siegel, N., Horvitz, Z., Levin, R., Divvala, S., Farhadi, A.: FigureSeer: Parsing Result-Figures in Research Papers. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 664–680. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_41CrossRef

21.

Clark, C.A., Divvala, S.: Looking beyond text: extracting figures, tables and captions from computer science papers. In: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015). http://ai2-website.s3.amazonaws.com/publications/clark_divvala.pdf

Title: Developing a Corpus of Hierarchically Classified STEM Images for Accessibility Purposes
Authors: Theodora Antonakopoulou
Paraskevi Riga
Georgios Kouroupetroglou
Publisher: Springer International Publishing
Book: Computers Helping People with Special Needs
Print ISBN: 978-3-031-08647-2

Electronic ISBN: 978-3-031-08648-9

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-031-08648-9_8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"