Analysis of the accessibility of selected massive open online courses (MOOCs) for users with disabilities

The development of technologies, including medical technologies, medical care and a general increase in prosperity contribute to a greater detection of diseases and the effectiveness of their treatment. Modern medical technologies give us the possibility to save lives of patients after serious injuries and extend lives of people suffering from degenerative diseases, what influences the increase in the number of persons with special needs [1‐5]. WHO reports that over a billion people in the world are affected by disability, which constitutes about 15% of the society [6]. In order to improve the quality of life of the persons with impairments it is crucial to introduce the concept of accessibility to information and communications technologies (ICTs) [7]. The goal of accessibility in ICT is to ensure that all electronic information, resources and services are accessible to people using alternative ways of human-computer interaction [8]. In the context of digital accessibility, one can also come across the relatively “young” term “users with special needs”. This concept means that a person who, due to internal or external characteristics, or due to the circumstances in which one is, must take additional measures to overcome barriers so that one can use digital resources on an equal basis with others. This approach eliminates all divisions into groups in terms of legal requirements, suggesting the need to develop facilities for all persons in need (Design for All) [9].

Massive open online course (MOOC) is a course available to a large number of participants that has online access via the Web. The name MOOC defines three features of the course. Massiveness suggests the appropriate number of participants, often referred to as the level of 2000. Openness is defined as accessibility to a huge number of participants around the world, regardless of their location, age, income, ideology or level of education, without any prerequisites or access fees. The third feature, ’online’, is the availability of courses from anywhere in the world via an Internet connection to ensure synchronous and asynchronous interaction between participants [10]. The method of learning described above, apart from many advantages, has a number of drawbacks. The most frequently raised comments concern the lack of feedback, low level of interaction with the teacher or a large percentage of people who did not complete the courses, but also the quality of the materials created. There is also a lot of talk about possible barriers–linguistic, economic or technical [11‐13]. These issues are even more visible in case of users with disability. The aim of this study is to verify compliance with the criteria for the accessibility of massive online open courses (MOOCs) on the Coursera platform for users with disabilities, with particular emphasis on people with visual and motor dysfunction. The review was carried out against the guidelines developed by the W3C consortium, in particular the web content accessibility guidelines (WCAG) 2.1.

Massive open online course (MOOC) services offer people all over the world access to education regardless of their age, gender, culture, location and physical fitness. Using new technologies followed by new functionalities makes MOOCs more attractive and accessible to the students [14]. According to the research performed by the Open University in UK [15], one out of seven users of the MOOC courses is impaired. This means that online learning makes education more reachable to people who have limited mobility because of their disability [16]. The application of WCAG principles also brings wide benefits to the elderly [17]. It was found that participation of people over the age of 66 in some courses is over 15%, and even around 40% of participants are over the age of 56. Taking into consideration the variety of MOOC users, it is very important to ensure high level of the accessibility of MOOC platforms as it has a significant impact on the attitude and motivation of the students [18, 19]. Research by Sanchez-Gordon et al. from 2013 [20] included analysis of five selected courses that were tested for compliance with WCAG 2.0. In their research from 2016 they analyzed compliance of the EdX platform with accessibility guidelines and provided the authors of the course content with a set of recommendations about improving the quality of the courses [21]. Subsequent studies were described in an article by Al-Mouh et al. [22] and concerned the analysis of the availability of selected courses by two blind people and a specialist in the field of accessibility using the blindfold. As in the previous case, the tests were based on WCAG 2.0. In the same year, the results of their research were published by M. Bohnsack and S. Puhl. They assessed the accessibility of five platforms offering MOOCs (Coursera, Udacity, edX, OpenCourseWorld, Iversity), in which the blind users participated [23]. The results of similar studies were published in 2017 by Królak et al. [24]. The research included seven users with vision impairments, all of whom used screen readers, and two additional people using Braille-based technologies. Each participant completed the 4-week course and after the completion submitted one’s comments. Analysis of three MOOC platforms, edX, Coursera and Khan Academy was presented in [31]. This research also concentrated on learners with visual disability and was composed of user studies with three volunteers and heuristic evaluation performed by five experts and based on WCAG 2.1 recommendation. The main problems identified were frequent lack of alternative text for multimedia content, hard to access drop-down menus and no bypass buttons allowing to avoid listening to repetitive information, located on the page, read by screen readers. Coursera, Canvas and edX platforms were also evaluated in [34] using Universal Design for Learning (UDL) framework. The results showed that the UDL principles related to providing multiple means of engagement and multiple means of representation were partly achieved in Canvas and edX and largely achieved in Coursera. The principle concerning providing multiple means of action and expression was only partially achieved in all the three platforms. The main problem related to UDL guidelines in examined MOOCs was the lack of variable methods for response and navigation. All of the described studies relate to the courses offered, among others, by the Coursera platform, indicating significant accessibility issues. In recent studies, the accessibility of MOOCs has also been examined from the perspective of providers and stakeholders [32, 33]. The results show that MOOC providers are mostly aware of the users with special needs; however, their priority is legislation over the accessibility preferences of the disabled learners. Identified problems that were pointed out include the use of third-party software and technologies that often do not comply with accessibility requirements and lack of information collected by the providers about the MOOC users and their accessibility needs.

In the literature, we can find several approaches to testing the accessibility of MOOC platforms. A classification into five groups was presented in [25]: (1) user-based empirical studies, (2) heuristic evaluations, (3) online surveys, (4) accessibility in terms of legal framework, and (5) MOOC in terms of teaching accessibility. Another method is automatic accessibility audit using online tools, such as AChecker of WAVE, which is often reported in the literature [25‐27]. Research studies using user-based approach usually include small-size testing groups representing only one type of disability [22, 23], while the heuristic approach tends to combine automatic tools and expert evaluation [20, 21, 23, 29, 30]. The heuristic approach should include various aspects of accessibility evaluations, as described in [28], including: (1) technical accessibility evaluation, (2) user experience (UX) evaluation, (3) quality evaluation, and (4) learning design evaluation. In the presented study the heuristic evaluation is proposed, being a combination of used-based evaluation and automatic audit. Technical accessibility is verified in addition to user experience tests with specific scenario performed by persons with different disability types.

Coursera is a global learning platform that was set up in 2012. It offers over 7000 free and paid courses that are delivered not only in English but also in many other languages. Due to its popularity Coursera has been assessed for its accessiblity by many researchers since 2013 [20, 22‐24, 30, 31, 34]. It gives a possibility to verify how the quality of the courses, their accessibility and conformance with WCAG principles change over the years. For these reasons Coursera was chosen as a test platform in the presented study. The research problem was defined as follows:

RP: Are MOOCs on the Coursera platform accessible to users with disabilities?

In order to answer the above research question, the following research hypothesis was formulated:

RH: Massive online open courses on the coursera platform meet the criteria of the web content accessibility guidelines (WCAG) 2.1.

The results of the research conducted are presented in two parts: in the first subsection the results obtained with the use of automated tools are described, and the next subsection presents the comments of the participants of the qualitative study, in accordance with the principles of the web content accessibility guidelines (WCAG) 2.1.

Study participants’ reports revealed many significant inconsistencies with web content accessibility guidelines (WCAG) 2.1, both at the A and AA levels. The most important of them are:The main problems identified in the study conducted by Sanchez-Gordon et al. [20] were defined as gaps in descriptions of alternative images, poorly visible focus on the page and ambiguous links, which was confirmed by the participants of this study. Bohnsack and Puhl [23] used Internet Explorer 8, which was recognized by the Coursera platform as out of date with an update recommendation. For technical reasons, this issue was not possible to verify in the presented study. The next results of the Coursera accessibility analysis were published in 2017 by Królak et al. [24]. The discussion in this paper included the most important results obtained and presented in the article written by Al-Mouh et al. [22], which were mostly confirmed in the presented study. The participants of the research conducted in [24] noted the lack of labels for buttons, deficiencies in alternative texts for graphic elements, incorrect hierarchy of header levels, problems with the use of the page using the keyboard, including controls in tests, and slow operation of the web browser in combination with the screen reader application. All these issues were confirmed by the participants of the presented research study. Some of the errors mentioned are now on a smaller scale than in previous studies. The problem with automatic multimedia playback was completely fixed, resigning from this solution, page titles and link intelligibility were also improved. Obtained results give partial answer to the research question stated, show that MOOCs on the Coursera platform are to some extent accessible and allow learners with disabilities complete the course. The accessibility problems reported in previous studies are constantly being removed or reduced. The research hypothesis of the presented research, stating that Massive online open courses on the Coursera platform meet the criteria of the Web Content Accessibility Guidelines 2.1. was partially confirmed.

The purpose of this study was to identify problems and non-compliance with the web content accessibility guidelines (WCAG) 2.1. An informal analysis of data obtained on the basis of the reports sent by the participants of the study was carried out. Significant non-compliances with the guidelines have been demonstrated, mainly from the perceivable and operable principle groups. The inconsistencies listed in the Discussion are significant and should be considered as a priority by all MOOC providers and developers of the content of e-Learning services. Comparing results of the studies on MOOCs accessibility since 2014, it can be observed that the platforms have improved their functionalities and are more user friendly for learners with special needs.

Future works should include an extended form of research, i.e., accessibility verification by automatic tools, full expert research and usability tests with the participation of users with disabilities using specific scenarios. To achieve reliable heuristic evaluation it is recommended to enroll accessibility experts representing each disability group. The next stage of research should take into account a more diverse research group, including people using Braille displays, additional assistive technologies, including, for example, people operating devices using the eyeball movement or users with color vision impairments. It is also worth testing a wider range of devices such as tablets or smartphones, operating systems such as Linux, Android, iOS or macOS, and applications and services such as VoiceOver or TalkBack. The inclusion of more MOOCs and the extension of the research to other platforms providing online courses can also provide meaningful information.