Introduction
Research on “assistive technologies” (AT), especially “assistive apps,” mainly deals with the concept of “accessibility” [cf., e.g., 1] by analyzing the corresponding design choices, interfacing options, practicability, or presumable physical or technological barriers. In the best case, such apps can be “enabler[s] of participation” [2: p. 437], assisting a growing number of users in actively participating in social, cultural, or professional activities and contexts. In the worst case—for example, in the case of impractical designs, overly complicated interfaces, or malfunctioning (and often expensive) software—however, these apps might be considered as hindering their users from profiting from contemporary media technologies. Consequently, these technologies can be regarded as exclusive to able-bodied persons. In other words, AT help provide particular persons “the life they would like to lead” [2: p. 437]. This conceptualization of “assisting” or “enabling” technology [3‐5] bears two significant problems in terms of the concept of “participation.” With such a modeling of “handicapped” users, the difference between being “abled” and “disabled” is reinscribed and requires these users be first “excluded” in order to be then “included” by the assistance of the respective media technology. Consequently, although such apps obviously do have a wide range of practical everyday benefits for some users, the preconditioned distinction between those to be assisted and the technology that provides assistance needs to be considered in the sense of a technological inscription of the ableism-disability-divide. Yet, how can we conceptualize constellations between assistive apps or digital devices and users that do not prescribe to this dualism in advance, focusing instead on the mediated processes that generate such differences?
In this paper, we are interested in the mutual relationships between users and their socio-technical and media environments. Thus, in our examples, we focus on specific bodily practices and social, cultural, and medical discourses alongside technical operations exploring the co-constitutional processes between users with visual and hearing disabilities and assistive app technologies. Our approach is based on Uexküll’s [6] concept of the reciprocal relationship between environment and organism and on recent adaptions of Gibson’s [7] concept of “affordances,” revised by the outcomes of Actor-Network Theory (ANT) and Science & Technology Studies (STS). The concept of “affordances” appears to be especially fitting here, since it draws attention to the mutual interplay of users, technologies, and environment.1 Based on this theoretical perspective, assistive practices are comprised of the heterogeneous relationships and distribution of agencies [8] between users/subjects and their media-specific, socio-technical, and material environment. Hence, instead of arguing for a singular excluding or disabling technology or body, we posit that technologies, bodies, senses, and identities are reciprocally shaped and produced in the process of dis-/abling practices [9] that enact, relate, mediate, and translate situated socio-technical arrangements [10]. The following analysis from media studies’ point of view can be understood as an important step and contribution towards a perspective that empirically accounts for broader daily relational affordances in practice [11].
Advertisement
Users, Affordances, and Techno-Sensory Environments
To discuss the concept of user-environment relationality, we will now turn to the work of James Gibson. In coining the concept of “affordances,” Gibson’s key premise is that there is a mutual connection between the perceiving organism and its (perceived) environment [cf. 7: p. 7–15]. According to this ecological perspective, “[t]he words animal [or living being] and environment make an inseparable pair. Each term implies the other” [7: p. 8].2 According to Gibson, “[t]he affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill” [7: p. 127] Thus, “affordances are triggered by ‘the particular ways in which an actor, or set of actors, perceives and uses [an] object’” [12: p. 1230]. Affordances then are specific potentialities for actions that arise from the ordering and structuring of the environment perceived by the respective organism, whereas perception is conditioned by the sensory system of the organism, relying on the makeup of the environment. Consequently, there is no perception without the perceived or vice versa—or, as we would like to add, perceiving technologies. An object’s possible usages are not intrinsic characteristics but are generated in relation to the perceiver as well as its sensory, bodily, and cognitive conditions. Affordances, then, are bound to individual and situated processes of reciprocal adaptation and the negotiation of agencies.
Gibson’s ecological perspective was very influential but, as Ingold [13] and others [14: pp. 311–313] have argued, it is partially contradictory. The concept appears to reproduce the problem of the “reconciliation of the relational understanding of the environment with an older and more conventional view that posits the environment as a set of objective conditions that exist independently and in advance of the creatures that come to inhabit it, and to which they must perforce adapt” [13: p. 78].
The contradiction here is, that, instead of radically clinging to the reciprocal mutuality of environment and organism, Gibson proposes a causal “before-after”-situation, by introducing the concept of the “niche” as a specific environmental situation to which the organism must adapt [13: p. 78]. Hence, he argues that while the organism relies on the objects/inhabitants of its environment, “the environment does not depend on the organism for its existence” [7: p. 129]. However, our premise is not to conceptualize the relationship between (dis)ability and (assistive) technology as a causal relationship. Gibson’s concept of the “niche” implies that the organism is supposed to adapt to the environment. If we take this concept as an analogy for the relationship between users, digital media, and technology, it implies that the given technology is the “normal” or “normalizing” environment to which the user should adapt in order to function. We would then have to argue that technology is a highly normative socio-technical practice, since the user would be forced to “accept” the necessity of adapting to a technological setup in order to be considered or function as “normal.”
To address this problem, Ingold turns to von Uexküll’s concept of Umwelt3 [6] and its respective relationship to living organisms. Uexküll and Gibson both propose an “immediate coupling of perception and action,” which, in conventional cognitive psychology or behavioral theory, does not follow a Cartesian dualism of the “thinking mind and the executing body” [13: p. 77]. Like Gibson, Uexküll [6] describes the organism’s close, mutual, and unique relationship to its environments and sets out to understand “how the world exists for the animal, given its own particular morphology, sensibilities and action potentials” [13: p. 79]. However, contrary to Gibson, Uexküll does not locate the potential for action in the objects of the organisms’ environment, locating the concept of Umwelt as “just the opposite: it is on the side of the organism pointing towards the environment” [13: p. 80]. In other words: “Thus, far from fitting into a given corner of the world (a niche), it is the animal that fits the world to itself by ascribing functional qualities to the things it encounters and thereby integrating them into a coherent system of its own” [13: p. 80]. This claim fits our conceptualization of a mutual relationship between human subjects and technological objects, like a smartphone app, as it allows for dismissing an a priori separation of the two and renders dis-/ability as a process rather than preexisting, fixed constants.
Advertisement
In our view, the controversy about the relation of living beings, affordances, and Umwelt raises the question of whether agency or the potential for action is provided by and attributed to humans by their (objective) environment or whether in turn the set of environmental objects conditions and “affords” the actions of the human beings related to it. To further extend this thought, one has to acknowledge that speaking of contemporary environments requires taking into account that relations and agencies are already technologically framed in significant ways [15: p. 2]. As more and more digital technologies are imbricated into environments, mediating human perception on a daily basis, other affordances, potentialities, and sensory constellations emerge. Whereas Ingold argues for dismantling the precise border between organism and environment by stressing that living organisms are in themselves entanglements of diverse relations and by introducing the term “meshwork” [cf. 13], our interest lies in the significance of AT and the ways they shape current relations between material worlds, discourses, mutual agencies, subjects/users, and the senses. In an ANT or STS perspective represented by theorists such as Bruno Latour, Madeleine Akrich, Annemarie Mol, or Michel Callon, the question of where to “locate” the origin of agency itself needs to be considered by the same problematic premise, namely the constitutional a priori difference between human action (or agency) and non-human action (or agency). According to ANT, this difference (and a rat-tail of further differences) should not be understood as the starting point of description (investigation), but rather as a posteriori “effect,” where coming-into-being is the actual question. While ANT research mostly succeeds in accomplishing the description of such processes that affect these kinds of differences, it usually produces a blind spot with regard to the role perception or sensory activities play. Basically, ANT allows for describing sensory activities and their relations in socio-technical arrangements: ANT is certainly compatible to sensory practices and their effects that produce differences, sensory practices are, for the most part, not researched at all. This might be the case, because ANT research usually works with so-called accounts—descriptions or representations produced by actors that were enrolled in such actor-networks or by researchers who locate themselves in an assumed “outside” of the actor-networks. However, representation is at any time a translation of perceptual modes and a product of the practices that engender perception. In this respect, in order to analyze sensory practices, the conditions for sensory perception and its effect on the production of representation need to be considered as well.
In the following, we will argue that practices of hearing and seeing as well as the notion of dis-/ability are not prerequisites for the use of the digital devices. Instead, these practices and notions emerge as an effect of the specific socio-technical arrangements and networked entanglements of bodies, technologies, and sensory practices. We take recent assistive hearing technology and apps for the blind as examples for our conceptual inquiry as they are often reduced to their functionality in technically substituting “missing” or “lost” sensory abilities. That is, AT are often thought of as bridging elements (intermediaries that have the same in- and output) [8] between a pregiven and often inaccessible environment and its users. Contrary to such technological and socio-medical approaches, we understand mobile AT as part of a complex interplay of heterogeneous entities enacting a techno-reality in practice where environments, users, senses, and their micropractices [16‐18], first and foremost, are coming into being.
Techno-Mediation or: New Standards for Smart Hearing
Hearing aids have come a long way in recent times. Nowadays, ears seem to be an increasingly popular place for wearable digital assistive technology and, as a consequence, a highly competitive market [19, 20]. While one of the most important goals of the hearing aid industry was to make hearing aids “invisible” by continuously miniaturizing the devices [21: p. 36], nowadays hearing technologies are becoming ever present in everyday life: Simultaneously introduced by Apple and in a blogpost for wearable technologies in April 2014, Internet platform product designer and wireless application specialist Nick Hunn [20, 22] describes “hearables,” a hybrid of the terms of “wearable” and “headphone,” as mostly wireless, often ear-mounted or in-ear wearable computer interfaces that not only promise to restore “normal” hearing but enable a form of hearing that is ubiquitous, connectable, increasingly adapted and adaptive, and strongly personalized—one that is (simply) a better form of hearing.4
Combining major assets of wearable technology with the basic principle of audio-based information services, hearables follow a holistic approach by targeting the combination of communication, information, and hearing technologies in one single device, discreetly concealed in a small earbud. Recent developments pertaining to this type of modern digital hearing assistant offer additional, required or simply desired, features such as full connectivity to all technological devices incorporated into modern life (i.e., speech translator, fitness coach, mobile sound streaming, personalized hearing favorites, online chats with audiologists, noise reduction, or biometric and behavorial data, etc.). A survey carried out for ReSound “smart connectivity” showed this to be a top priority, especially for male users associating this form of advanced technology with coolness.5 Coolness, on the one hand, might serve as a selling argument in an able-bodied society. Technological connectivity, however, does not automatically create social connectedness or acceptance. As van Dijck has shown, connectivity rather aims at providing (pre-)engineered solutions quickly developing into valuable revenue [24: p. 4].
While hearables may be desirable for some, for others, especially those with hearing issues or hearing loss, they may be more akin to a smart hearing aid. According to the Irish hearing loss blogger Geoffrey Cooling, hearables will “have a dramatic effect on the cultural acceptability of hearing aids” [25]. Despite this growing “cultural [and technical, B.O.] convergence between ‘mainstream’ and ‘assistive’ technologies” [26: p. 134], differences between assistive hearing technology and hearing assistants still remain: in most countries, hearing aids—in contrast to both personal sound amplification systems (PSAP) and hearables—are considered medical devices. They are, therefore, partly covered by health services. However, many persons with hearing issues, before seeking medical assistance and being diagnosed as hard of hearing, may prefer to try the less stigmatized and more stylish devices that, nowadays and in almost any public space, have become almost “normal.” It is therefore not surprising that, in 2017, the Senate of the USA passed legislation to allow over-the-counter devices to be marketed and sold as hearing aids without requiring a prescription. This legislation, on the one hand, could save US consumers, users, and patients a lot of money on costly medical devices that are mostly not covered by Medicare or health insurance; on the other hand, it confirms existing and encourages future cooperation between traditional headphone makers and hearing technology companies and reinforces the “steady increase in circuit complexity” [21: p. 24].
In the following, we would like to investigate, in particular, how digital hearing technology conditions personal and environmentally adapted hearing practices while, recursively, technology itself is structured by the users’ hearing needs or wishes as well as by different auditory situations and environments. The concept of “affordances” thereby allows “to avoid positioning technology as determinative of particular socio-cultural outcomes, or situating the use of technology as completely socially constructed” [27: p. 2]. Moreover, Gibson’s theory makes space for “a third way between the (constructivist) emphasis on the shaping power of human agency and the (realist) emphasis on the constraining power of technical capacities” [28: p. 444]. In this mutual interplay, digital hearing devices do not function as mere tools, amplifiers, or restorative prostheses, but as mediators [8: p. 39] in operational processes in which bodily, cultural discourses and technical operations are meant not to artificially (re-)produce a preexisting “natural” or “normal” hearing but to enact new forms (and norms) of techno-mediated hearing experiences. The definition of hearing ability is thus to be described in relation to its socio-technical production [29]. We would like to ask in which ways the apparently infinite possibilities of customizable techno-mediated hearing experiences afford, on the one hand, new practices of hearing and acting, while demanding, on the other hand, a growing (self-)discipline and environmental control [30, 31] to “successfully” cope with the demands of the digital devices? With focus on mutual agencies of hearables, hearing practices, cultural discourses, and technical operations [cf. 15], we will demonstrate how hearing technology, in its becoming, mobilizes a new economy of hearing, acting, and being [32, 33] at the interface of dis-/ability and media-technological studies [34, 35].
The Becoming of Hearing Technology
The “technological trajectory” [36: p. 284], from ear trumpets dating back to the seventeenth century (often made of sheet metal, silver, wood, snail shells, or animal horns), through to twentieth century hearing aids, coming in an at least controversial medical beige, to today’s digital hearing technology cannot be solely reduced to simple technological progress. It is an effect of situated configurations of heterogeneous actors such as sensory experiences, affordances, practices, digital technology, discourses, and cultural contexts [cf. 37].6 Analogue hearing aids amplify all sounds (speech, noise, music, etc.) in the same way—with some using a microchip to allow for programmed settings for various listening environments. Conversely, digital hearing technology converts sound waves into digital signals to produce an exact duplication of each sound. Computer chips that analyze speech and other environmental sounds allow for more complex processing of sound during the amplification process and offer a greater flexibility in programming, promising that the sound they transmit can be matched to the needs of a specific pattern of hearing loss or easily adapted to a particular hearing situation. By providing multiple program memories and optional features, digital hearing technology is intended to afford assistance to all hearers (with or without hearing issues) in changing environments. As already mentioned above, advanced hearing technology—whether assistive or not—not only “adjusts” the relation between hearers and hearing environments but it also configures a steadily growing number of interfaces in which hearers, specific hearing situations, social behavior, individual activities, and personal lifestyles emerge as connectable actors, building highly complex socio-technical relations in which human and non-human actors become available for each other [38: p. 12].
A short advertising clip by ReSound LINX2 illustrates the differences of rediscovering hearing in terms of smart, “effortless,” “personalized,” and “enhanced hearing comfort” in a likewise better sounding world [39]7: The smooth transitions, from close-ups on ears, mouths, and hearing aids at the beginning through medium shots situating hearing aid wearers in their close surroundings to long or extreme long shots translating the (re-) conquered space, focus on the almost compelling relationship between hearing with (more) ease, increased connectivity, hearing enhancement, and the personalized location assistant automatically adjusting the preferences for favorite environments. Furthermore, the clip visualizes the complex entanglements and affordances between technology, sociality, and the environment by inserting small pictograms connecting persons, objects, and specific environments using dashed lines. The sophisticated audio-visual setting, however, does not distinctly portray the hearing ability of the users of this brand new technology (deaf, hard of hearing, or “normal” hearers) or whether the hearing aids shown at the beginning serve as medical or consumer technology.8 This undecidability does not seem surprising at first glance, “since all technologies are assistive” [19: p. 4].9
Media like digital hearing aids tend to exist in our formulation merely as invisible windows, meaning that we mainly focus on what they do (for us) instead listening to their own “evocative” potential [41]. The merging of assistive hearing technology and consumer electronics aims not only at restoring (normal) hearing, but it also operationalizes specific forms (that become new norms!) of healthy, individually customized, and, most importantly, constantly better hearing. The intimate connection to near-body technologies like in-ear devices, smartphones, or the Internet of Things exceeds mere functionality or assistive appropriation of a technical object [42‐44]. Digital hearing technologies converge into the fabric of our everyday life until they are indistinguishable from it and become part of our regime of subjectivation [45: p. 36] [46‐49]. Personal audio assistants like sporting headphones, wireless earbuds, or a Bluetooth headset worn in, on, or over the ears have become almost obligatory for joggers, young people streaming music, business travelers, or commuters catching news or making urgent phone calls while waiting for or passing time on trains and busses. Media sociologist David Beer proposes a closer look at the material experiences and dimension of attachment, suggesting to “think about mobile media as objects that are intimately incorporated into routine bodily practices” and explore the “material attachments to the mobile media themselves, the devices, as they are used to generate and negotiate cogito” [50: p. 362, [51]. The instrumental power of the object, according to Sherry Turkle, thus becomes less important compared to an understanding of the “affordances” [7, 22, 52] of an “evocative object” as a “companion in life experience” [41: p. 66]. When we hear (and speak) through digital devices, we also live in them, “[we] live our lives in the middle of things” [41: p. 6]. “Listening to these things” [43: p. 6] can help us explore the manifold relationships “flow[ing] constantly between persons and things” [43: p. 6]; however, it also feeds a complex production system that is constantly in search for new trends.
From Self-Technology to Governance Technologies?
As shown above, the production of smart hearing has to be considered as a technical, cultural, and material practice that emerges in and through specific configurations with corresponding interrelated affordances, apparatuses, symbols, producers, and forms of knowledge. Due to its recursive relation to technology, hearing, based on the collected data, experiences, and usages of the users or patients, keeps constantly changing. Free choice of preferences and growing personalization do not subvert a production system that immediately captures new needs or wishes, constantly assisted by more or less voluntary users/hearers producing data for free [53]. The promised (and promising) diversity of individual hearing experiences and dis-/abilities thus turns out to be a set of commercializable technological fixes or preferences to which hearers voluntarily subject themselves as a result of a self-forming discipline [54].
In a ReSound LINX2 video testimonial, Steve DeLuca, hearing aid user for over 20 years, talks about his experiences with the new device. Before wearing the ReSound hearing aid, he had to deliberately focus on what his wife said in order to hear, and now, “it’s just second nature” [55: 00:00:50]. The connectivity of the device enables him, as a user, to adapt settings on the Apple Watch, which is less “awkward” because it does not make him seem like checking Facebook or other social media while being in a professional meeting or conversation. The most amazing feature of the new hearing aid is, according to de Luca, the possibility to block sounds like disturbing wind noise outside or background conversations in restaurants. The hearing aid thus not only enables (“normal”) hearing and specific affordance potentialities but it also enhances a few specific situations by automatically eliminating acoustic interferences or irritations are. Digital hearing technology like LINX2 shapes new practices of (better) hearing and in a certain sense reciprocally determines the perceived meaning of hearing loss: “I never realized how much of a hearing loss I had before I put these hearing aids in.” [00:02:42–00:02:46] That said, hearing loss is not just a medical diagnosis as it directly depends on the hearing devices and the act of purchasing objects [56: p. 95] through which hearing is produced [57]. In another testimonial, a passionate rower tells us that with the ReSound App he is enabled to control the annoying wind noise when he is on the water, just like the ardent cyclist in the advertising who, after visiting a café, adjusts the app to his individually programmed bicycle-mode with wind reduction then turns on his personalized “home-hearing” just as he arrives home [58]. In other words, the digital affordance of controlling different auditory environments (or “Umwelten”) produces what, when, and how the user hears. Smart hearing technologies, thus, are not only things we hear with [57], they are, with a slight modification to Turkle’s quote, “things we [have to, B.O.] think [and work, B.O.] with” [41] in an “environmental […] infrastructure of power” [59: p. 41], becoming visible and/or hearable in and through the complex arrangements of smart hearing (self-)technologies.
With the (more or less) free choice of different and—i.e., with the ReSound LINX2 geotagging-app—environmentally adapted hearing programs, the idea of a so-called normal or natural hearing becomes one possibility among others like, perhaps, the aforementioned home-hearing program. Hearing, then, needs to be considered a process that is produced in relation to the situated setup of the corresponding socio-technical environment. In contrast to the definition of medical hearing devices that, mostly, aim at technically reproducing a presupposed “normal” hearing and understanding, consumer (audio) electronics apparently follow another “script” [60: p. 208]: this script frames a—more or less—co-operational space in which the data required by producers to produce new forms of (better) hearing, which is then realized in a self-customizable hearing technology with applications specifically designed for use by a single self-responsible individual and principally outside healthcare facilities, is constantly provided by users who send data and information on their individual hearing experiences, needs, and wishes to audiologists, doctors, hearing aid producers, or third parties.
Restorative hearing aids have been transformed into a fashionable lifestyle product by increasingly smarter technology which, while promoting a new hearing diversity, mediates the growing demands of our contemporary meritocracy and (re-)introduces new hearing norms that are determined by technological and industrial standards. Hearables thus seem to blur this difference by appealing to all kinds of hearers or hearing, but as hearing is never enough, the new digital technologies and resulting environmental relationships transform their wearers into prognostic subjects [61] that cannot live up to the utopian promises of commercials, developers, and medical discourses. The aim is neither normalization nor pure enhancement: hybrid hearing technologies, like hearables with applications designed for use by a single self-responsible individual (principally outside healthcare facilities, sending information to audiologists, doctors, hearing aid producers, or third parties), introduce new and dynamic norms of hearing against the backdrop of a hearing practice reinforcing “a duty of self-care” [cf. [62: p. 15]. This practice is inscribed in the process of an “economization of the social” [63], in which both health and individual success depend on an optimal applicability of the self through technological interconnection.
In this perspective, it is not (hearing) disability that is the “most convenient Other to the smooth functioning of contemporary corporatized media” [35: 366]. Techno-mediated hearing, whether assistive or not, rather addresses the general possibility of manufacturing hearing in different personalized ways—this is at least what is promised. But what does this mean to persons with hearing issues? Confronted with a kind of not just self-, but environmentally controlled hearing flex-ability that constantly adapts to and is adapted by different actors of the socio-technical hearing event, hearing impairment and disability seem, at first sight, to vanish and re-emerge as an effect of a specific situated techno-sensory arrangement. The abovementioned blurring of boundaries between everyday digital hearing assistants and assistive hearing technology, however, reappears when it comes to the question of health insurance coverage. Scientific journalist and cyborg Frank Swain has further concerns about the assistive potential of the new hearing technology [64]. Most of the developers and designers, he says, have never used assistive hearing technology that is used for the whole day, a practice that not only challenges the battery as a destabilizing material factor in the interaction but also causes ear infection problems. Fully occluded headphones are great for listening to music, because all you hear is the music, but they are terrible for anything except listening to the device [65], emphasis B.O.]. Manufacturing hearing, on the one hand, enables individually hearing practices; on the other hand, not only the access to the newest technology is economically conditioned but also the ambivalence of making hearing a commodity sheds new light on the question of dis-/ability.
Mobile App-Practices, Sonification, and Blind Users
Assistive Apps for the Blind as Constellations of Distributed Agency
Public transport systems and buildings are still often described as lacking features that allow accessible use by blind persons. Resulting from the setup of urban infrastructures, mobility training is thus essential for blind persons.10 Hence, practices like the use of the long cane and guide dogs are widespread among people with visual disabilities [67]. However, as smartphones and other mobile devices are now widely available (and more financially affordable), blind persons also begin to establish relations with digital technologies regarding their mobilities [68: p. 90]. This is possible primarily due to the introduction of mobile technologies and software such as VoiceOver (iOS) or TalkBack (Android), which allow for maneuvering the device using of oral speech and/or haptic input.11 How non-visual practices with digital devices are enacted on a daily basis is however an ongoing process from which multifaceted dis-/abilities arise.
In recent years, blindness is being manufactured and addressed socio-technically in novel ways as a variety of apps became available in Google’s Playstore and Apple’s Appstore that specifically target people with visual disabilities as their users. That also means that—concerning wayfinding practices and mobility—forms of direct touch, indirect touch (through the long cane), or cognitive mapping of spaces (i.e., step counting) are complemented by digital practices involving smartphones and apps [18]. As Wong puts it, “people’s movements and mobility strategies are configured and reconfigured through their everyday engagements with technology and digital information.” [68: p. 90].
Consequently, we have to go beyond Paul Rodaway’s approach in his work on Sensuous Geographies [70]. According to him, sensory experiences are crucial when it comes to daily practices and the respective geographies that are related to them. Rodaway, following the ecological approach of Gibson, reflects on the sensuous geographies of people with visual disabilities, emphasizing the significance of sound and acoustic spaces [18]:
“[…] though sound does not provide a continuous or reliable source of environmental information, auditory experience can give the blind a wider geography of spatial dimensions and relationships, an acoustic space, and at particular moments offer vivid evocations of place character […]. Free from the continuous flood of visual information, the blind show a clearer understanding of acoustic space. […]” [70: p. 104].
Framed in this way, “vision loss” [71: p. 99] turns into something different, becomes a resource and is rather positively connoted, for acoustic geographies might be perceived better when not distracted by visual information. The situated makeup of the acoustic space perceived through the auditory system of the respective blind person then both enables and “disables” a relational set of “acoustic affordances” [72] that allows for the distribution of “blind agencies.” Yet, it is not only important to bear in mind that a “blind style of perception,” as Saerberg puts it [73: p. 25], relies on a multiplicity of sensory information. We also have to take into account that these specific “ambient” sounds and blind listeners are reciprocally formed and—additionally—shaped in their relation to each other by digital technologies, as will be argued in the following.
When considering apps for blind persons, several aspects are important to their performance with relational technologies. Geotagging in electronic maps and GPS location services are important elements in enabling applications such as Blindsquare [74]. The annotation of images and the development of automatic object recognition, face recognition, and complex sceneries are other facets of this economic area where artificial intelligence will play an important role. SeeingAI, Envision AI, or Aipoly and other apps are examples that provide a range of different functions and multiple services.12 While all of the mentioned apps rely on the translation of visual information into verbal (synthetic) speech, some applications are heading in a different direction by using sonification, that is “the usage of sound to represent scientific data” [77: p. 249; [78, 79]. In these cases, there is no verbal information about the object or surrounding area to be identified. Rather, abstract sounds are produced to deliver information about the needed item or surrounding area.13 Thus, such an app technology acts as a “mediator” [8: p. 39], a relational link between user and environment that translates visual “input” into “auditory” output and reciprocally mediates and transforms the users’ perception and their interconnected environment. These techno-sensory mediation processes ultimately both enact affordances and their respective agencies. In this respect, though functioning as a way of opening up potentialities by mediating a range of possible actions, a media constellation like this also generates social and material constraints, “that is what it is materially and socially [im]possible to do with it” [81: p. 187].
In what follows, we present an analysis of Camassia, a recent sonification app that has been available since 2018. This example will be analyzed to demonstrate how users and environment are reciprocally and simultaneously enacted in specific situations. By doing so, we aim to transgress an understanding of such applications as merely functional articulations between inaccessible environments and pregiven blind persons. App technologies are not mere instruments or “channels” that convey information—in the Latourian sense: they cannot be considered “intermediaries” [8: p. 39]—but instead are interconnected nodes in ecological constellations that generate and transform information and agencies.
“An intermediary, in my vocabulary, is what transports meaning or force without transformation: defining its inputs is enough to define its outputs. […] Mediators transform, translate, distort, and modify the meaning or the elements they are supposed to carry. […] No matter how apparently simple a mediator may look, it may become complex; it may lead in multiple directions which will modify all the contradictory accounts attributed to its role.” [8: p. 39].
By proposing to develop a concept of a techno-sensorium (according to Elizabeth Stephens) or “more-than-human sensorium” [82], in which heterogeneous elements collaborate to enact both the environment and respective users, we question how app technologies figure as relational technologies that manufacture blindness and enact visual dis-/abilities. To explore this problematic, the following section traces various processes of translations, processes, and mutual configurations and conceptualizes the app as a mediator that coordinates the production of blind users, a certain acoustic space and the sensory relations between them. The aim is to critically re-situate the borders between a sensing human subject and a perceived environment [83] and supposed preceding fixations of dis-/abilities. This reconsideration allows taking into account an app-user-configuration where agency is distributed and approached from a non-anthropocentric stance [84].
Camassia and Resonating Relationships
The app Camassia is one of the several applications designed for the support of wayfinding and mobility practices among blind persons. It hence addresses the targeted group directly and inscribes the user as having a visual disability and, at the same time, being a competent smartphone user. In particular, as the app is currently only available at the Apple App Store, the application presumes the user owns an iPhone. Perhaps, this is not a surprise because Apple embraced accessibility features as an economic factor and people with disabilities as a relevant target group quite early in comparison with other technology and software companies. Consequently, Camassia users need to meet a specific “obligatory passage point” [85]: “Obligatory passage points serve to establish the identities of the actors in relation to the network, and thus serve to assess their indispensability to the network. Unless actors pass the obligatory passage point, they are not enrolled in this network” [86: p. 175], Hence, in order to enroll in the actor-network of this app and generate the desired affordances and sensory configurations, the users are subject to the economic, social, and technological demands that Apple prescribes.
Regarding its main function, the app allows the processing of visual information and produces auditory feedback to assist the user in finding a desired pathway. Applying a low-latency approach, the app enables real-time scanning of chosen surroundings and provides “immediate feedback” [87: p. 12]. It was released in 2018 by iXpoint and developed by this software company together with students of the Department of Informatics at the Karlsruhe Institute of Technology (KIT) [88]. The idea of the application is to provide an assistive system for the blind and visually impaired that allows independent movement in urban areas while not requiring mobile internet connection, GPS navigation, complex sensor systems, or electronic maps. The main point concerning Camassia is that it relies on the low color saturation that paths have in relation to surrounding areas. In the case of paths in parks and pavements surrounded by grass, the app presented the best results.14 Thus, instead of identifying obstacles, the app recognizes free space as a possible way.15 In this case, “locatability” [12] is highly dependent on and relational to the respective socio-material environments and the geographical actions and agencies the users can undertake: affordances are thus generated in relation to the infrastructural Umwelt and the bodily, sensory, and technical configurations of their users.
To process visual information, the app uses the smartphone camera that captures images in the direction the device is pointed and the person is walking. Developers programmed the app’s algorithm in a way that it processes 30 pictures per second, which have been stabilized by the device’s motion sensor [87: p. 14]; more precisely, a shaking smartphone dangling from the user’s neck will probably not interfere with the output. While the company’s demonstration shows the use of the app in a vertical format [cf. 89], user reviews recommend the horizontal format in order to use a wider angle and capture a broader space in front of the walking person [90]. This shows that smart technologies in no way inherit “universal characteristics” such as “portability” and “availability” that Schrock [12] proposes. Rather, in order to be portable and available, the smart device needs to be handled according to specific bodily techniques in relation to the user’s perception of holding the device in a particular manner. Portability and availability are hence processually and relationally afforded. Another aspect concerns image cropping as only the lower two-thirds of the captured surroundings on the image are used by the algorithm. That is, only a limited extract of the available information is used by the app to compute the probability of the path. Thus, the user’s perception and related affordances are restrained by the technological condition of this media constellation.
As reviewers suggest [90, 91], the app is relatively easy to use and access. However, it requires blind persons to adopt a different way of listening (and hearing) as it is necessary for them to learn to understand the signals the app produces as a translation providing information about the area the person is walking through. The auditory soundscape offered by Camassia radically differs from spatial listening, tactile exploration with a long cane, or digital haptic feedback [92]. Yet, in the twentieth century, one encounters a series of attempts to construct systems that translate written text into sound [93] and other forms of sonification specifically developed for blind persons [94]. Moreover, a variety of listening modes can be distinguished regarding sonification [77], which constitutes an important facet in scientific knowledge production. Specific sonic skills [78]—both bodily and sensory techniques—are thus required and have to be learned for understanding data that was transformed into sound patterns according to particular software programs and processed across a number of different devices (i.e., smartphone and headphones in this case). That is, affordances, auditory perceptions, modes of listening, and a host of material practices are inextricably entangled. Concerning the translation of a given park area with an earth path into sound through Camassia, one might presume that in order to perceive the respective sound pattern one has to be aware of which sound is an indication of the path and which is not. As a result of the testing phase, training material and sound samples were added to the final version of the app that is now available [87: p. 16]. Together with the aspects on affording portability and availability, this also includes a particular “learning process,” the sensory information the app provides is interpreted and operated with according to an individual “habitus” [cf. [81: p. 187]. These affordances are produced in the course of “practicing” and “learning” with the app and consequently, according to Pickering, “such practice consists in the reciprocal tuning of human and material agency, tuning that can itself reconfigure human intentions” [95: p. 21], actions, and perceptions.
We will consider now in more detail how the app’s algorithm processes visual input. After initializing and calibrating, Camassia transforms the received visual input according to a scale of 24 halftones, stereo sound, variable volume, and tone pitch. The surroundings are thus enacted as an abstract, atonal, and arrhythmic acoustic space: lower sounds signal that there is a path on the right side, higher sounds indicate a way on the left, and a smooth pulsating tone notifies the user of a free way ahead of him/her. As the company’s demonstration video shows [89], when walking on a certain path, a regular pulsating tone is produced by pointing the device to the right or left, and lower or higher tones are superimposed over this rhythmic sound to indicate when the user approaches another area with a different surface. The video suggests that auditory and visual can be compared on an activated smartphone screen. However, such a comparison is not possible for blind users and thus a different learning process arises when using the accompanying training material and app in situ. An important aspect in this respect is how users listen to the technologically produced soundscape of Camassia. One way is to use the device’s monophonic loudspeaker [87]. Users can also connect headphones to their smartphone as the app provides a stereo panorama. As headphones might decouple the user from other sounds in the surrounding environment (other people, cars etc., see [90]), persons testing the app preferred bone conduction models [87].
The app is advertised as an assistive technology that provides support for the independent mobility of blind people. As is the case with most digital wayfinding practices (use of Blindsquare or other apps, cf. [18]), such applications can be considered as complementary elements in a diverse range of different mobility practices, where various senses, the long cane, or guide dogs are at play [67]. What is of interest here is that one might go beyond an approach, which emphasizes the question of accessibility or the aspect of assisted mobility in order to raise the question of a techno-sensorium, which is produced in a socio-technical arrangement of users, devices, senses, and surroundings. Hence, we argue for reflecting upon technologies of sonification [77] as a way of (medially) producing both different environments and users. While visually captured surrounding areas are translated into stereophonic acoustic events by Camassia, users are enacted as blind listeners who (learn to) navigate through electronic soundscapes—even if they complement their practices with a long cane or other sensory input [90]. To put it differently, one may propose that, in collaboration with a smartphone, headphones, and a blind person, Camassia produces a techno-sensorium where agency is distributed to human and non-human elements, that is, to a heterogeneous host of actors. We are confronted with a situation where one is able to observe that agency “is not a basic human capacity, not a precondition of the social; it is a relational, ever-changing outcome of its enactment” [84: p. 4].16 Consequently, the app-user-environment-hybrid enacted by Camassia consists of a (temporarily) continuous stream of electronically generated sounds that result from the contact of captured camera images with the app’s algorithm. Here, the sound of this contact provides the conditions of possibilities for a resonating relationship between a blind user and an acoustic environment—both of which emerge simultaneously and are situated in a specific moment.
Yet, this brief discussion of the Camassia app is not able to fully analyze the impact of this sonification application and its economic aspects or ambiguous effects on manufacturing the senses within socio-technical arrangements. Furthermore, there are problematic issues in this techno-sensory framework that are worth mentioning. For instance, the continuous sonic space created by the app is more likely to be enacted in parks, that is, in a specific part of the city’s landscape. Also, it adds “different sensory inputs” [68: p. 91] to actual traffic noises whereby sonic scenarios rendered sensible, i.e., through distorted echoes elicited by footsteps or clicking of the cane [73]: p. 22], become more complex. Future research therefore still has to show whether and how such sonically generated affordances and sensory agencies can potentially be “intimately incorporated into routine bodily practices” [50]: p. 362] of mobile media and digital wayfinding practices.
Conclusion
At its most general, assistive technology is meant to balance a technological-engineering interface with a modern understanding of dis-/ability [96]. Viewing AT as a “special equipment” is a barrier to access and to overcoming socially, culturally, or technologically created obstacles. Concerning digital practices on hearing and seeing, we therefore propose an understanding of “dismediation” [35: p. 365]. This term coined by Mills and Sterne refers to the entanglements of dis-/ability and media technology, based on the conviction that the affordances generated in such environments are not only related to portability, availability, locatability, and multimediality, as Schrock has put it [12]—but, to the same extent, to the material makeup (weight and size of smartphone, form, haptic cues, etc.), the representational design (software design, “intuitive” symbols, etc.), the users’ perceptual and bodily conditions (digital literacy, functioning sensory organs, motor skills, self-monitoring, etc.), and the techno-economic conditions. Furthermore, as we have shown, each media environment is in a sense unique and situational in accordance with its relation to the user’s sensory and bodily functions. When it comes to novel forms of manufacturing life, bodies, senses, or technology, neither (bodily) inclusion and exclusion nor an invariant location or environment are prerequisites of assistive technologies. They are, however, instead (re)produced, enacted, and conditioned [33, 97] in the corresponding digital practices and related environments. Involved in such practices are “complex synergistic and recursive engagements and intra-actions between the sensory affordances and capacities of human bodies and computer hardware and software” [82: p. 191]. “[P]ermanence is relative” [7: p. 13] or a matter of relationality in this kind of dynamic, processual ecology.
Acknowledgments
The authors would like to thank the anonymous reviewers and the editors of this special issue for providing constructive comments that helped to improve this paper.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that there is no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.