What Factors Motivate Culturally Deaf People to Want Assistive Technologies?

2.1 Acceptance and Rejection of Assistive Technologies

The World Health Organization defines assistive technologies (ATs) as “any product, instrument, equipment or technology adapted or specially designed for improving the functioning of a disabled person” [37]. ATs can have many benefits for users, including improving access, increasing independence, and easing communication and social interactions [7, 13]. However, there are barriers to their acceptance and use. ATs can be expensive, uncomfortable, or require a long acclimatization period or multiple adjustments [26, 29]. They may be considered aesthetically unappealing, especially if they “appear medical” [1, 26]. ATs can also make previously invisible disabilities visible, “outing” the user without their control, which can be validating; or can cause fear of social stigma ("self-stigma"), social anxiety, and social stigma [7, 13, 26, 29]. Networked and AI-based ATs, such as fall detectors, also introduce concerns about privacy, surveillance, and control of data [1].

Assistive technologies intended for DHH people can be divided into three categories: assistive listening devices, sound notification systems, and intermediate communication devices [10]. Assistive listening devices include technologies such as cochlear implants and hearing aids, which face resistance from multiple sub-groups for multiple reasons [27]. Some key examples include:

As surgical implants, cochlear implants seem more contentious than hearing aids which can be removed at will [7].

Sound notification systems include flashing smoke alarms, baby cry monitors, shake ’n’ wake alarms and door bell alert systems. Adoption and acceptance of non-sound-based AT devices for DHH people has not been well studied to date. When reviewing sound notification alarm systems intended to wake sleeping DHH people, Smedberg et al. found that of the 35 DHH people surveyed, the most popular system, a fixed strobe light alarm, was in use by less than 40% of respondents [33].

Intermediate communication devices are systems that assist communication between a Deaf person and a hearing person. Examples of this are sign language processing systems and speech-to-text systems. Sign language processing systems include sign recognition, via video or gloves, which translate signing (or more commonly, transliterate signed spelling) to speech or text, and automatic translation of spoken/written language to sign language via avatars or generated video [34]. These technologies have received mixed reviews, with the most egregious being branded “disability dongles” [11] by Deaf communities [19, 36]. Speech-to-text systems can be seen when using applications such as Zoom or Microsoft Teams. There are also several speech-to-text applications which are web-based or can be downloaded onto mobile devices. Automatic speech to text currently lacks accuracy, especially for domain-specific words, or in environments with multiple speakers [12]. Readability of displayed text can affect comprehension [3]. Text-based assistive technologies may not be desirable for signing Deaf people due to either preference, or “literacy concerns" [15]. These concerns act as barriers to adoption, although Butler et al.[6] found that DHH students were more accepting of ASR in conjunction with other assistive technologies

Historically, the development of ATs has focused on functionality alone [26]. Designing ATs in ways which consider aesthetic appeal, users’ self-expression and importantly identity, especially along the dimensions of ability/disability and culture, may support adoption of ATs [26, 29]. This principle can be seen in the attempts of some people with hearing loss to "hack" their hearing loss, by developing strategies and technologies which support their AT use, replace ATs which did not meet their needs, or alter ATs to better suit their identity and aesthetics [29, 30].

2.2 Personal Assistants for Deaf and Hard of Hearing People

Voice-activated personal assistants are not currently accessible to many DHH people. Automatic speech recognition systems are often not trained on datasets including deaf voices, which can vary from hearing people’s voices, and from other deaf voices [14]. This means that, even for DHH speakers whose speech is considered “good" by hearing listeners, automatic speech recognition produces a high error rate and unpredictable results [14, 16, 32]. Further, for culturally Deaf people, speaking may mean using their second language. Existing research conducted in the USA and Japan has found that DHH people would be interested in a personal assistant they could sign to [16, 17, 21, 35] and signing Deaf participants were interested in a personal assistant which could sign back to them [16, 21].

Our proposed AI AT takes the form of an Auslan-capable personal assistant, which Deaf people can sign to in Auslan, and receive responses in Auslan via a photorealistic avatar [38].

4.1 Linguistic and Cultural Alignment

The most significant motivational factor that emerged was how a sign language-capable AI AT would be linguistically aligned to the Deaf community. All participants (n=12) said they would be interested in an AI AT which recognised Auslan commands and responded in Auslan. All 12 participants also said if they had such a device, as long as it had features they personally desired and signed fluently, they would use it every day.

The consensus among participants was that it would be faster and easier to communicate with an Auslan-capable AT, compared with voice-activated AI technologies. This suggests linguistic alignment can provide a level of comfort for Deaf people that hearing ATs do not. One participant also commented that an Auslan-capable AI AT would give him more scope to choose when he would wear his hearing ATs, because our AT would give him more control and more options, thereby supporting his use of both types of AT.

Four participants said they had used voice-activated personal assistants. Three of those four said they had difficulties interacting with them because there is low accuracy for understanding their “Deaf voice”, which reflects [14]’s findings.

It’s important to note that although most participants suggested the device ought to only communicate in Auslan (n=7), there were suggestions among other participants that linguistic alignment with the Deaf community is broader. For example, people in the community who are Hard of Hearing or are still learning Auslan may require English subtitles. There are some Deaf people who still use Signed English⁵. For those who have experienced language deprivation, there could also be pictures and visual cues to facilitate understanding. One participant suggested including Braille to support Deafblind⁶ users.

Four participants also wanted the proposed AI AT to align with Deaf culture. One participant stated that, "You want it to behave like a Deaf person. You want it to include Deaf culture and how we communicate... You want the avatar to incorporate Deaf culture. How we sign, how we structure things. We want it to be the same," (Jarvis1). In addition to Deaf-like signing and language use, participants wanted interactions with the device to align with Deaf culture, such as suggested “wake-up" triggers mimicking interactions between Deaf people (i.e. an attention getting wave and use of a sign name) One participant noted that the data used to train the AI system must be collected from Deaf people.

4.2 Equality

Another significant motivating factor which emerged was equality for the Deaf community (n=8). Several participants stated that having an AI AT which Deaf people can communicate with in their first language would mean they are equal to hearing people, who already have access to voice-activated personal assistant technologies. For instance, one participant stated that, "I would say I would feel... more equal to hearing people. Because hearing people can speak and use [personal assistants]. I could have access [to a device] that I can sign to and ‘communicate’ with," (P3), while another reported a desire to "catch up a bit" (P14) with hearing people. Another participant stated that, "technology is often designed for the hearing world. This forced me to adjust to ‘match’ with the technology," (TAtkinson). In contrast, we are proposing an AT designed from a Deaf-first perspective, which may be able to fill access gaps; for example, four participants stated they would be able to receive news updates and emergency announcements straight away without any barriers. Some participants compared this to hearing people’s use of the radio, stating they benefit from immediate access to news and information in convenient situations, where Deaf people cannot.

4.3 Accessibility and Inclusion

Another motivational factor pointed out by participants is the potential for accessibility and inclusion (n=6). The idea of an AT which would support the whole household was appealing. Three participants pointed out that there are opportunities for accessibility and inclusion in mixed family households if the device were to be bilingual (Auslan and English), with three participants also suggesting it could be a learning tool for people not fluent in Auslan. When asked if they might use a personal assistant device for something that hearing people would not, one participant responded: “I think it would be amazing to have an Auslan only Alexa, but what if there are families with Deaf kids and hearing parents? Some CODAs are bilingual, some [deaf people] focus on English and speech. So, I think having both Auslan and English would be good.” (P6).

Two participants also brought attention to the racial, ethnic, ability, gender and sexual diversity within the Deaf community and noted that ATs should recognise and accommodate this diversity. For our AI AT, this would be through a customisable appearance and personality, and ability to recognise and produce diverse signing variants.

4.4 Independence

Five participants indicated independence was a motivating factor. Two participants mentioned they must often ask friends or family for up-to-date information. One participant expressed that, “Deaf people often felt that they ask hearing people for assistance. In the past we had no advanced technology, we had to ask for help... But now we are now becoming more independent because of better technology” (TAtkinson).

4.5 Improving Life

In addition to supporting their independence, as discussed above, five participants identified that our proposed AI AT would improve their lives in the following areas; convenience (n=2; e.g. “Most important - convenience and worthwhile to use this device that make my life easier", DeafGavin), empowerment and confidence (n=1; e.g. “Benefit of having a device, giving a commands will help to increase my confidence.... We sometimes have lots of barriers, e.g. low esteem, no confidence or nervous and shy. Being placid, that device is not a real person, so this will give us the opportunity to talk with the device, asking questions. This will help us to improve our confidence.", Captain Awesome), psychological wellbeing and companionship (n=3, e.g. “If Alexa understand Auslan.... So I can feel relieved, good for my psychological well-being and Deaf culture", TAtkinson), memory aid (n=2, e.g. “I don’t have a great memory for everything, like cooking.... If I use this device, when I ask something, it will pop up on the screen... It will show me ready how to cook them", Captain Awesome) and health education (n=2, e.g. “I think health should be included.... I have Apple watch, they tell me about exercise or about my heart beating for fitness.", P32).

4.6 Aesthetic Appeal

For our proposed AI AT, participants commented on two aesthetic aspects: the design of the physical device, and the design and actuation of the Auslan-signing avatar. Three participants commented on the appeal of some aspect of the physical device (“We can put it in the house with pride. It need to be a good design looking nice.” P7). Five participants commented specifically on the aesthetics of the avatar (“You don’t want it to look ugly or weird” P3), with four wanting some degree of customisability. Participants also reported on the importance of fluency and fluidity of signing, as noted in Section 4.1 above.

4.7 Technology Appeal

Three participants were motivated by the technological aspects of the personal assistant. This fell into two categories: three participants were excited by the potential for a smart home and control of interconnected devices; and one was excited for a new technology interaction style.

4.8 Barriers

The most common area of concern among participants was privacy and security (n=10). Seven participants stated they were concerned about the security of information that may be stored in the device. Six participants were concerned that having a device with a camera in their home would enable people to hack into it and observe them. One participant wondered where the Auslan data would be stored.

Five participants noted that a barrier to adoption could be the high cost of personal assistant devices. It was noted that assistive technologies can often be expensive. One participant was concerned that, if it would require an internet connection data, this would represent an ongoing cost. This also aligns with findings from the literature that cost is a barrier to AT adoption. We have not yet addressed this in our project, as we are still in a very early stage of design and prototype development, but it will be important for us to consider throughout the life cycle of the project.

Participants were also concerned about the device’s ability to understand Auslan (n=4), reinforcing the importance of linguistic alignment. This concern was present given the complexity of the features in Auslan including facial expression, body movement and non-lexical signs. Three participants expressed concern about the device replacing Auslan Interpreters. Given these concerns, one participant raised there may be challenges with Deaf community acceptance.

5.1 ATs “fixing" Deaf people

We have identified linguistic and cultural alignment as the most important motivating factor for Deaf people’s adoption of ATs because, historically, assistive technology has been rejected by Deaf people when they perceive it as being intended to “fix” them (i.e. make them hearing) [8]. This has resulted in resistance towards the adoption of “hearing" ATs. The Deaf community is proud of their culture and language. ATs which are not seen as mutually exclusive with that culture and language are likely to see greater levels of acceptance. Involving users of an AT in the design of that AT, such as through a participatory design approach, is more likely to result in an AT that the users want to engage with, because it will ensure it aligns with their identity, e.g. [26].

5.2 Visibility of AT or Disability

While none of our participants addressed the issue of visibility directly, the fact that cultural alignment, and greater visibility of their Deaf culture, are motivating factors suggests visibility of culturally aligned ATs is a benefit, not a barrier. The only concern participants raised is whether those ATs would be accepted by the wider Deaf community.

5.3 Acclimatization and Comfort

An interactive sign language AT could have a shorter acclimatization period for Deaf people than hearing-based ATs, as it uses the native language of the Deaf community, making it comfortable and natural to use [26, 29]. Ensuring the device includes cultural norms from the Deaf community will further ease acclimatization. In the case of our sign language smart home assistant, this means we must consider how well the device recognises Auslan signs, and if there are unnatural limitations on the complexity of commands the device can recognise (which parallels problems with early commercial voice-activated personal assistants [25]); as well as how acceptable the avatar which will provide signed responses is to Deaf people as individuals and as a community. Artificial limitations on language use would introduce an acclimatization period which may impact satisfaction and adoption rates. Like ATs, signing avatars also have a long history of low acceptance among Deaf communities, in part because they have also not been linguistically and culturally aligned. Key considerations for signing avatars include accuracy of signing; appropriate use of facial expressions, mouthings and mouth gestures; appropriate timing for manual and non-manual sign components; and smoothness or naturalness of movements [24]. If these elements are not sufficiently considered in the creation of a sign language avatar, any AT which relies on that avatar will likely face rejection by the Deaf community.

Our proposed AI AT would further supports comfort by providing participants with greater control over their use of ATs in general, as noted by the one participant who felt it would allow him to choose when he wanted to wear his hearing aid, and when he wanted a signing-first environment.

5.4 Social Stigma

The only social stigma our participants were concerned about was whether or not our proposed device would be accepted within the Deaf community. As mentioned above, linguistic and cultural fit, accuracy and robustness of sign language recognition, and acceptability of the signing avatar were seen as key features of the device itself. The “bigger picture" of how the sign language smart home assistant would fit into Deaf culture – and in particular, whether or not it would compete with sign language interpreters – was also a concern.

5.5 Losing Independence

While many assistive technologies aim to provide independence by augmenting users’ abilities, the fact that they are perceived as indicating a loss of independence [1] shows developers of ATs must consider how to design and market ATs in ways that meet prospective users’ true needs and desires, such as by taking a participatory design approach. Participants saw our prospective AI AT as a way to gain independence through access to information and control over technologies without the need to rely on hearing people, which motivated their desire to engage with it.

5.6 Aesthetics

Our findings confirm the importance of ATs being aesthetically pleasing.

5.7 Privacy and Security

Our participants’ comments align with findings from the literature that networked and/or AI-based ATs raise concerns about privacy and data security. We do not have a unique solution to this concern, but recommend developers of such ATs prioritize data security in prototyping and production, and communicate transparently with users about where data is stored and processed, and who (if anyone) is able to access user data.

5.8 Cost

Our findings confirm the cost of ATs can be a barrier to adoption. This is a difficult issue to address in a capitalist system, as factors which contribute to the costs of ATs are the high levels of investment and research required to develop high quality ATs, compared to relatively small target user groups. The development of high tech ATs, such as networked and/or AI-based ATs, are also likely to have both relatively high upfront costs (for the purchase of a technology), and ongoing costs (due to the need for a network connection) as a requirement of the technology.

6.1 Limitations and Future Work

The limitations of this paper include sample bias and the umbrella project’s early design phase. People recruited to participate are at least a little interested in this prospective technology, or they would not have wanted to be interviewed. All participants were recruited in Queensland and northern New South Wales, so the perspectives of the entire Australian Deaf community have not been captured. A broader survey or interview program could find more nuanced results.

These interviews were conducted as part of the design phase of the umbrella project, so there is lots of hope about the potential of a shiny new technology. Future work, once development of a robust prototype is completed, would involve studies to examine actual use and adoption of the technology.

The umbrella project to which this work belongs continues to use a participatory design approach to designing and developing an Auslan-capable personal assistant, with native signers (Deaf and CODA) involved in both the core research team and as participants in data gathering and co-design activities.