How Physical–Digital Experiences are Designed

Humans are embodied beings and experience the world through bodily interactions with other social actors or artefacts within a specific spatiotemporal context (Merleau-Ponty 1962). Experiences are a constant in people’s lives, in both intentional (e.g., going to the movies) and incidental ways (e.g., hearing a street musician en passant), and they draw on rich sensory stimulation: the more senses are engaged and the more coherent this stimulation is, the more effective, meaningful, and memorable an experience becomes (Pine and Gilmore 1998).

The human desire for new experiences and rich sensory stimulation gave rise to what is known as “the experience economy”, in which commercial providers create and sell engaging experiences to consumers (Pine and Gilmore 1998). Initially, providers simply enhanced ordinary services or products with experiential components (e.g., the integration of pleasant scents or interactive displays in stores), but eventually, they began to create experiential consumer offerings–curated multidimensional events staged to engage participants in a hedonic, sense-stimulating way (Chandra Kruse and Drechsler 2022; Lorenz et al. 2024). These offerings target either single or multiple participants who engage in the experience for hedonic pleasure, enjoyment, and stimulation, rather than to achieve functional goals.

Similar to how digital technologies have transformed other commercial offerings over the past few decades (e.g., Barrett et al. 2015; Yoo 2010), experiential consumer offerings have also started to feature digital components. Many museum tours now feature interactive digital content on touchscreens (Khalil et al. 2024), and sports events now often feature immersive systems technology to increase vividness, interactivity, and telepresence for spectators (Kim and Ko 2019). The COVID-19 pandemic further accelerated the digitalization of experiences and resulted in a significant increase in experiential consumer offerings, such as virtual wine tastings, online yoga classes, or virtual museum visits, from both established (e.g., Faber-Castell) or born-digital (e.g., Masterclass) providers.

Immersive systems technologies have gained importance for experiential-offering providers because they allow designing experiences that cross traditional boundaries of the physical realm (Pine and Korn 2011) – for instance, those imposed by space–time linearity or physical embodiment (Cavusoglu et al. 2019; Wang et al. 2022). A prominent example is the Modigliani VR: The Ochre Atelier experience at the Tate Museum in London, an experience that integrates virtual reality components into the museum visit by enabling participants to step beyond the confines of the physical exhibition space and travel back in time to explore the renowned painter’s studio.

Immersive systems technologies “purposefully change or enhance the user’s perception of reality” (Cavusoglu et al. 2019, p. 680), by using digital audiovisual information and communication technologies to create a sense of immersion, that is, the illusion of a “real experience” for users through the merging of physical and digital components in a way that facilitates absorption, presence, and a sense of “being there” (Dincelli and Yayla 2022; Slater and Wilbur 1997). They do so by using digitally produced cues to stimulate a comprehensive and enveloping “immersive” experience for the user. To that end, digitally produced cues often minimize or substitute stimuli from the physical world (Bowman and McMahan 2007) such as when systems like the Oculus Quest, HTC Vive Cosmos, Microsoft HoloLens 2, or Apple Vision Pro build on cameras and sensors that integrate the virtual experience within the confines of the physical realm or represent the physical environment virtually. A key design question is thus whether the digital cues that contribute to an immersive experience must be synthetic or if they can strategically involve coherent stimuli from the physical realm.

Immersive systems technologies are typically classified as virtual reality (VR) or augmented reality (AR), even though these distinctions are not categorical (Flavián et al. 2019; Pine and Korn 2011). VR describes a “set of technologies that enable people to immersively experience a world beyond reality” (Berg and Vance 2017, p. 1) and that surround the user in a computed world mediated by VR headsets. Unlike VR, AR does not completely replace physical reality visually but allows users to “see the real world, with virtual artefacts superimposed upon or composited with the real world” (Azuma 1997, p. 356). This is done in real-time, for example, through head-mounted displays (HMDs), tablet computers, or smartphones.

Notwithstanding the astonishing technical advances in emerging immersive systems technology such as the Oculus Rift, the Teslasuit or the Apple Vision Pro, a paradoxical trend has emerged in practice: On one hand, leapfrogging innovations in hardware and software are broadly marketed and hailed for their ability to “disrupt” our experience of reality and blur the boundaries between the physical and virtual realms. On the other hand, we are simultaneously witnessing a turn toward more physicality-oriented experiences. Despite the technological and economic advances, along with the proliferation of VR and AR hardware and software enabling all-digital experiences, such offerings have largely failed to resonate. For example, the metaverse, hailed as a revolutionary new, entirely digital experience with virtual offices, virtual social spaces, and virtual cities, arguably failed to attract notable momentum. Decentraland, one of the largest and most relevant Metaverse platforms, reportedly has only 38 active daily users, while one of the main user-engagement features of Meta’s flagship product Horizon Worlds failed to produce any noticeable revenue (Horwitz et al. 2022).

Instead, in practice, many experience-economy providers are creating physical–digital experiential consumer offerings that integrate characteristics of both physical and virtual realms and that range in their relative emphasis from physicality-centric experiences over integrated experiences to digitality-centric experiences. Figure 1 visually positions different physical–digital experiences on this continuum, and Table 1 differentiates them by the focus on the senses that are incorporated in the design of different experiences. For example, the National Gallery in London’s Art of London AR Gallery places artworks (i.e., by Van Gogh and Tracey Emin) directly onto the streets of London via a mobile app, with the physical environment remaining central as the app overlays static visuals onto specific real-world locations (Fowler 2021). In contrast, the VOID offers a fully integrated multisensory theme park experience, combining themed physical spaces, such as rooms with tactile walls and objects, with immersive VR environments, like those resembling Star Wars or Ghostbusters. For instance, touching a real-world wall corresponds to a virtual castle wall (Porter 2024). On the digitality-centric end, the Beat Saber VR game immerses players in a vibrant virtual environment where they slice neon cubes with virtual lightsabers, synchronizing their movements to the rhythm of the music. The game is designed within a confined geospatial radius to ensure safe movement within the physical realm. Finally, Van Gogh: The immersive experience describes an integrated physical–digital exhibition experience in which digital components, such as animated drawings, are placed in specially sourced locations, such as a ninteenth-century stable building with exposed brickwork and timber beams (in the London location).

Full size image

We are interested specifically in integrated physical–digital experiential consumer offerings, in which both virtual and physical realms and their components are not simply considered peripheral landscapes but instead constitute a joint nexus of physical and digital components (Goebeler et al. 2024) that must be purposefully fused into an experience that exceeds what either realm can offer alone. Consider a VR wine tasting or an AR historic city tour. A VR wine tasting requires the presence of the actual wine to provide a complete sensory experience that delivers on the key degustation experience. Similarly, an AR historic city tour relies on the actual facades and ambient sensory cues of the physical realm as a backdrop and reference point for the AR overlays. Both types of experiential consumer offerings require the integration of physical sensations and digital components in their design to create a truly sensational experience for participants. But doing so requires dedicated design strategies to integrate activities or experiences that span both realms, such as redirection techniques “that enable walking over large distances in the virtual world while remaining within a relatively small space in the real world” (Steinicke et al. 2009, p. 2). This is especially relevant in cases where a virtual experience is situated in a confined venue, such as a VR escape room (Sun et al. 2018).

However, while the use of integrated physical–digital experiential consumer offerings in areas such as tourism (Yung and Khoo-Lattimore 2019), human–computer interaction (Yang et al. 2019), healthcare (Fertleman et al. 2018) or consumer offerings (Flavián et al. 2019) are well-known by now, how designers can integrate, synthesize, combine, or otherwise fuse experiential components from physical and virtual realms to create such experiences is not yet well enough understood. We lack detailed design insights into how providers can effectively create and integrate physical and virtual activities for participants across both realms to craft experiences that neither realm alone can fully provide. For example, studies such as those exploring aroma-content congruence by pairing scent with VR (Flavián et al. 2021) or the synthetic production of physical stimuli such as wind and heat to accompany a VR experience (Ranasinghe et al. 2017) have found a generally positive effect on the user experience when orchestrating multisensory stimulation in immersive systems-mediated experiences (Melo et al. 2022). However, they fall short in explaining the choices designers should make when integrating multisensory cues into immersive experiences (Marto et al. 2022). Likewise, even though a surge of research is noticeable about immersive systems technology in contexts such as education and learning (Suh and Lee 2005), e-commerce (Hsiao et al. 2024; Peukert et al. 2019), collaboration and co-creation (Asatiani et al. 2021), gaming (Pamuru et al. 2021), communication (Teng et al. 2023), healthcare (Zahedi et al. 2016), or sports events (Westmattelmann et al. 2021), this research focuses almost entirely on downstream impacts from the use of immersive systems technology, not on their design. Moreover, despite the growth of the experience economy, research on either the design or use of immersive systems technology for experiential offerings such as VR applications in museums, AR integration in tourism experiences, AR-supported business models in tourism (Cranmer et al. 2021), or experiential, escapist motifs in such technology use (Haj-Bolouri 2023) is progressing only slowly.

This lack of design insights is problematic. At any point in a designed experience, even when users are immersed in a VR experience that provides significant audiovisual shielding from the physical realm (Peukert et al. 2019), they remain embodied beings who perceive sensory cues from the physical realm – such as temperature, auditory signals, or scents – surrounding their bodies. They are also always still subject to physical risks or obstacles (Cavusoglu et al. 2019). Even mental processes, such as attitude and intention formation, are deeply rooted in what users perceive through modalities beyond the audiovisual capabilities that a virtual realm can provide (Cerulo 2019). Thus, the blind spot regarding the role of physical components and their fusion with digital components into an experience nexus is problematic because there is a lack of empirically sound guidance for providers and designers on how to create meaningful fusions of the two realms to create valuable experiential consumer offerings. Our study addresses this problem.

Congruent with our ambition to provide design insights about how virtual and physical realms can be artificially created and fused in the design of experiential consumer offerings, we draw on ideas of soma design (Höök 2018) and designing user activities (Kaptelinin and Nardi 2007). Soma design emphasizes the bodily and sensory quality of human experiences (Shusterman 2012) and thus focuses on designing the experiential, felt, bodily, and aesthetic aspects of technology (Höök 2018). Soma design encourages designers to integrate bodily activities, such as movement, gestures, and embodied interaction, into a user experience (Höök et al. 2021). It highlights that experiences in the virtual or physical realm should be designed to enable users to experience and perform activities⁴ specific to that realm that are aimed at achieving immediate goals that contribute to the overall outcomes of the designed experience (Kaptelinin and Nardi 2007) and ensure that a designed experience is embodied, purposeful, and seamless (Bødker et al. 2024). In other words, soma design stresses that activities must be designed such that users experience continuity in what they do with their bodies to achieve a goal, regardless of the realm in which they occur. Creating a physical–digital experience thus means designing how users can act through and with technology (Bertelsen and Bødker 2000). In virtual realms, for example, designing activities might involve creating the possibility for a user to manipulate virtual objects or interact with virtual actors. In physical realms, designing activities might involve crafting haptic objects that users can handle to trigger events.

Our research is a form of design archaeology (Chandra Kruse et al. 2019; Recker 2021). The key idea behind design archaeology lies in observing implemented IS artefacts to generate design knowledge at different levels of abstraction – for instance, as design principles or a design theory (Peffers et al. 2007). Design archaeology builds on the idea that the extraction of design-relevant knowledge does not strictly necessitate the construction of an artefact (Maedche et al. 2021). Instead, it stresses that design knowledge contributions can also arise from observation, when researchers analyze IS artefacts in situ to understand the underlying design rationales to derive prescriptive and abstract design insights or carve out potentials for improving the implemented artefact (e.g., Molla et al. 2024; Recker 2021).

Like historical archaeology, design archaeology involves analyzing an existing in situ artefact to retrospectively excavate the rationales and activities that informed its design, implementation, and use (Chandra Kruse et al. 2019). Ideally, this analysis involves exploring the design choices that formed the basis for the existing system but also personally engaging with the artifact under study (Recker 2021). Applied to our case, this means that we wanted to understand how different commercial providers created integrated physical–digital experiences (the design artefact) that feature naturally occurring and artificially produced sensory components and designed activities within both physical and virtual realms. We aimed to generate broadly applicable and abstract prescriptive design knowledge that could either be applied to the design of other types of physical–digital experiences beyond those we studied or to improve the design of those we did study.

Therefore, we conducted a multiple case study with four providers of physical–digital experiential consumer offerings in Germany (Table 2). We used two case selection criteria: First, we selected providers that offered physical–digital experiential consumer offerings that could be considered integrated (as per the continuum in Fig. 1). Thus, we chose providers of experiences in which the virtual realm (created through immersive systems technology) had to play a significant role in the overall experience designed but where the experience also prominently featured distinct physical realm components (e.g., a city center and its scenery in a city tour). This criterion was essential to ensure that for these experiences, the physical realm was not merely a vessel for a largely virtual experience but instead an integral part of the thematic “nexus” (Goebeler et al. 2024) of the experience itself. Second, we selected providers that offered physical–digital experiential offerings with different experiential themes (e.g., sensory involvement and indulgence in the VR wine tasting, nostalgic exploration in the VR time travel experience) and that varied in their use of immersive systems technology mediation (both concerning hardware and software components), so that our dataset had sufficient variance in the configuration of physical and digital components in both physical and virtual realms.

Our data collection proceeded in two waves in parallel with our iterative process of data analysis (Urquhart 2012) in 2023 and 2024. Our sources are summarized in Table 3. Our primary data source was semi-structured interviews with key informants involved in the design of the physical–digital experiential consumer offerings (e.g., project leaders, company directors, technology specialists, or experience designers). The number of informants varied from case to case, as the size and organizational responsibility for conception and design differed by case (e.g., a one-person designer and venture owner in WineCo compared to a team of historians, developers, and narrators jointly responsible for the design in TimeCo). All semi-structured interviews were conducted face-to-face (in person during site visits or via videoconferencing technology) in German and professionally transcribed and translated into English (120 pages in total). The interview protocol is found in the Appendix (available online via http://link.springer.com).

In our initial interviews in 2023, we focused on gathering information about how the offerings were designed, focusing on design choices in the virtual and physical realms and the means used to integrate the two. After analyzing this data initially, we then conducted a second round of interviews to consolidate the emergent themes (Urquhart et al. 2010). For example, during the initial data analysis, we quickly realized that designers created activities and components not only in the virtual realm (e.g., evoking actual repositioning or grasping through virtual triggers), but also in the physical realm (e.g., adding scents or actual wine to create olfactory stimulation). Therefore, in the subsequent data collection, we decided to place greater emphasis on the design choices made between semantically matched key activities in both realms, such as designing forms of navigation that apply to both virtual and bodily modes of locomotion, and how they were integrated. In addition, we used the second wave of interviews to deepen our understanding of the properties that were germane to the integrated experiences that emerged from the merging of the two realms in the offering. We interviewed the same informants in each round because our screening of the other roles indicated that they also remained central sources of insight for the emerging themes (Charmaz 2006).

A second data source in our design archaeology was participant observation data, which we gathered through direct engagement with the designed artefacts – the physical–digital experiential consumer offerings. Two members of our research team participated in each of the physical–digital experiences, adhering to the observational researcher role to “derive prescriptive knowledge from observing and analyzing existing artifacts in use” (Maedche et al. 2021, p. 361). This participation enabled us to engage directly with the artefacts under analysis, studying their in-situ implementation and adopting the perspective of typical users – namely, the consumers of these commercially offered experiences. Through firsthand observation, we uncovered how activities and sensory stimuli across both realms were designed and integrated. For instance, we observed how the olfactory stimulus of a toffee scent was synthetically integrated and synchronized with the audiovisual digital content of a carnival parade scene, where sweets were thrown from a truck. Similarly, we experienced how social interactions spanned both realms, featuring the same character to create a cohesive narrative.

During our participant observations, we focused specifically on sensory activities implied by the design of the experiences, such as smelling, seeing, grasping, and moving. We specifically considered the sensory stimuli that could be derived from these activities, whether naturally occurring (e.g., the audiovisual soundscape of a city center) or synthetic yet physically introduced (e.g., ambient smells, music). For example, we observed that church music was integrated into one of the city tours at a location where a church remains visible in the present cityscape, while computed visual overlays of a historic bustling street scene unfolded through the AR glasses in front of the church. Immediately after participating in the experiences, we met to discuss impressions and documented the experiences systematically in 23 pages of memos. These memos were then used to triangulate our observational insights with the providers’ perspectives obtained from interview data.

Finally, as a complementary source of data, we also conducted several informal, unrecorded interviews during our site visits or via telephone (Table 3), which were captured in notes and served as member-check-ins (Lim 2025). We used them to probe for specific details about certain design choices (e.g., the choice of immersive systems technology and the reasons for allowing virtual and real human interactions), confirm our understanding of certain rationales and design choices, and (towards the end) corroborate the plausibility of our emergent conceptualizations.

Data collection and analysis were conducted in iterations (Urquhart et al. 2010). Our data analysis was primarily inductive and adhered to established guidelines for grounded theorizing (Charmaz 2006; Gioia et al. 2013; Urquhart et al. 2010). We began by developing case narratives (Schwandt et al. 2007) to share knowledge within our research team about each provider organization, including details such as their location, industry, and the experiential offerings they designed (e.g., the immersive systems technology used, the experiential theme, and physical venue characteristics). We then openly coded our data with an initial focus on understanding: (1) the design choices and underlying rationales for designing and embedding virtual experiences within the existing physical realm, supported by immersive systems technology; and (2) the intended properties (design goals) of the resulting physical–digital experience.

Our open coding yielded a comprehensive set of descriptive codes that shed light on the design choices and underlying rationales behind providers’ incorporation of immersive systems technology into their physical–digital offerings to embed them within the surrounding physical environments, such as city centers or indoor venues.

Initial focus: providers as physical–digital integrators. Through these initial analyses, we realized that the designers did not act solely or dominantly as virtual realm creators; instead, they acted as integrators of components they designed in both physical and virtual realms. This highlighted to us that to these providers, the physical realm was not merely a “starting point” or “vessel” (Goebeler et al. 2024). We recognized that the experience designers made intentional and coordinated design choices within and across both realms, carefully creating and arranging components from each to form semantically matched pairs of activities – such as actions or tasks like gesturing, interacting, or opportunities for sensory perception – that engaged participants’ bodies and supported the specific experiential goals envisioned by the providers.

Unpacking the design of experiential activities. Building on this realization, we adopted a designing perspective informed by theories of soma design (Höök 2018; Höök et al. 2021) and the design of user activities (Kaptelinin and Nardi 2007) and focused on activities as the core units of design. In this context, digital technologies – specifically immersive systems technologies – served a mediating role in activity design as tools that shape and facilitate users’ actions and interactions while contributing to the attainment of specific goals. Consequently, our realization that the overarching design goal was to create cohesive physical–digital experiences that align with the designers’ clear experiential goals (e.g., allowing participants to stroll through the present-day city center while experiencing the evolution of a historic German city from the 1900s to today) prompted us to conduct a second wave of follow-up interviews, which we open coded aligned with our emergent focus on physical–digital activity design (Gioia et al. 2013). These interviews acknowledged the dual focus on both physical and virtual activity design and the intentional integration of these designed activities across these realms. To ensure a solid foundation for our theorizing, we consistently compared new data points with existing ones across cases and informants (Charmaz 2006).

First, we examined which activities providers designed in both realms, for which we triangulated observation protocols from our own participant observations and the interview data. We distinguished activities based on whether they were created through physical design choices, digital design choices, or a combination of both. For example, consider how the insertion of a computer-generated Madonna figure on a pedestal in a city center street, viewed through AR goggles, prompted participants to walk around it and direct their gaze in a specific direction. As participants changed their position, the Madonna adapted its virtual-visual properties in response to their perspective. This examples, along with similar instances absorbed in the open codes, illustrates how participants could trigger algorithmic processes, such as through bodily movement in the physical realm, that controlled or manipulated audiovisual properties (e.g., the shapes of virtual artefacts or sounds) within the virtual realm, which we subsumed under the second-order category algorithmic handling (Gioia et al. 2013).

Second, we investigated how these activities were distributed across physical and virtual realms. We noticed that designed activities in one realm always featured a corresponding activity in the other realm. For example, alongside the algorithmic handling of the computer-generated Madonna figure, we observed instructed haptic handling, such as swirling a wine glass in WineCo or carrying a hat box from one physical room to another and handing it over to a different actor in the TimeCo case. These examples illustrate how key designed activities occurred in both realms as semantically matched pairs of actions, rather than as disconnected or unrelated tasks. All experiences featured designed operations in both physical and virtual realms that together formed an integrative designed activity that jointly created a cohesive physical–digital experience in line with the providers’ experiential design goals. For example, the design goal of ArCo and TabCo was to emotionally and compellingly connect participants to the city’s history, highlight its architectural evolution, and seamlessly emphasize the appeal of the present-day cityscape. In the WineCo case, the provider aimed to make the wine tasting experience more engaging, accessible, and memorable by using VR to visualize aromas that are simultaneously transmitted during the physical degustation process, helping participants more effectively identify and remember specific flavor profiles.

Having identified these pairs of designed activities that were semantically matched yet distributed across physical and virtual realms, and through triangulation with our observational data (Urquhart 2012), we then grouped the different coded activities into two sets of four abstract, summative, second-order categories that differentiated between virtual and physical realm activities (Gioia et al. 2013). Each is rooted in design choices specific to its respective realm. We found that we could categorize all different designed activities as instances of four semantically matched types of actions: detecting, handling, navigating, and interacting. For example, we categorized activities for the perception of multisensory cues, such as the haptic experience of touching a bronze statue during the TabCo city tour, as multisensory detecting. Similarly, we referred to the corresponding designed activities that allowed participants to perceive synthetic audiovisual cues, often highlighting key experiential themes or phases, as audiovisual detecting. Examples include digitally produced street music and audiovisual dialogues of virtual figures that carried the story forward.

Deriving the characteristics of the integrated physical–digital experience. To identify the distinct characteristics of the physical–digital experience that emerged because of these integrated activities, we drew on the core idea from user activity design theory (Kaptelinin and Nardi 2007) that “actions are goal-directed processes that must be undertaken to fulfill the object. They are conscious […], and different actions may be undertaken to meet the same goal” (Nardi 1996, p. 71). Building on this idea and aligning it with themes identified in our data regarding the designers’ goals of creating a coherent physical–digital experience, we found that these goals were closely tied to the design of specific activities and their integration across realms. For example, one provider explained that the opportunity for participants to interact with a costumed historical character embodied by a human actor was designed to be later reinforced by a virtual interaction with the same character in the VR phase of the experience. This continuity was implied through shared features such as similar appearance, name, and clothing. The activities of scripted and naturalistic interacting (identified during the second stage of data analysis) were designed by providers to serve as a narrative anchor, bridging and connecting the two realms.

We triangulated these insights with our participant observations to evaluate whether these design goals effectively resulted in unique qualities of the physical–digital experience. These observations illuminated the properties that emerged from the participant-centric engagement in both physical and virtual activities of an offering. In the given example, the two of us who participated in the experience noted that interacting with the same character across both realms successfully integrated experiences in both realms, providing a coherent narrative thread, where components in the virtual realm had a physical analog to seamlessly integrate both realms. Consequently, we organized this instance, along with similar cases and the, under the dimension of cross-realm anchoring (Gioia et al. 2013). This analytical approach, which integrated observational and interview data, finally enabled us to inductively derive three new theoretical attributes as aggregate dimensions (Gioia et al. 2013) of integrated physical–digital experiences, which we defined as sensory enrichment, spatiotemporal and social recombination, and cross-realm anchoring. These dimensions capture the defining characteristics of a physical–digital experience.

Linking the designed activities to the integrated physical–digital experience. Finally, we analyzed how the emergent properties of the physical–digital experience were connected to the activities that were designed across the physical and virtual realms. We conceptualized this connection through two interdependent processes that we termed localizing and de-localizing. Through the analysis of interview and observational data, we noticed that all the physical manifestations of the four key categories of designed activities effectively grounded the experience in physical reality, by leveraging tangible components such as holistic sensory stimuli for multisensory detecting, perceptible surroundings for bodily navigating, human actors for naturalistic interacting, and physical properties and objects for haptic handling. These designed activities thereby ensured that a physical–digital experience always featured a physically real, embodied connection to the immediate environment (the location). Thus, we captured this process as the act of localizing an experience in the spatiotemporal setting of the experience. In contrast, we noticed that all the digital manifestations of the four key categories of designed activities effectively broke the spatiotemporal boundaries of the physical environment of the experience. They allowed users to explore simulated dimensions that went beyond what was temporally or spatially possible in the physical setting of the experience. The designed virtual activities provided strategically produced synthetic cues for audiovisual detecting, fantastical or distant sceneries or worlds for expanded navigating, virtual characters from other eras or mythical creatures for scripted interacting, and mutable audiovisual components or features for algorithmic handling. In effect, these activities de-localized the experience; they literally took the user into activities that were far away, temporally distant, or otherwise not possible in their concrete physical setting.

We finalized our analytical process by creating a data structure (Gioia et al. 2013) as a traceable audit trail of the progression from raw data and codes to more abstract categories and concepts (Fig. 2).

Full size image

In integrated physical–digital experiential consumer offerings, the experience unfolds both in a physical realm, a tailored part of the naturally occurring reality without digital mediation, and in a virtual realm, a computational reality that exists only in digitally embodied form. Both realms can manifest in different ways. The physical realm of experiential consumer offerings can take the form of a natural realm, a subset of a physical landscape composed of naturally occurring sensory cues around the participant. It may also take the form of a curated realm, which includes both naturally occurring and intentionally introduced or orchestrated sensory cues. Similarly, the virtual realm can take the shape of computed landscapes beyond the spatiotemporal reach of the participant, such as distant lands or historical sceneries (a distant realm), or even the form of computed imaginary landscapes (e.g., worlds with wizards and dragons) that have no real-world equivalents or precedents (a fantastic realm). For example, in our cases, the distant realm manifested in digital overlays of historic facades in both AR historic city tour cases (TabCo and ArCo), while the fantastic realm–among others–was instantiated as floating aroma visualizations (e.g., floating fruits) in the VR wine tasting (WineCo). A natural cityscape served as the natural realm in both AR historic city tours. Each room of TimeCo was curated to depict different themes (e.g., of a millinery, cinema, and cable car replica), representing the curated realm.

Through our data analysis, we uncovered that each of these realms was intentionally designed to facilitate distinct yet semantically matched activities that contribute to participation in the overall experience. These activities were either materially designed (in physical realms) or created through immersive systems technology (in virtual realms). Table 4 summarizes these “pairs of activities” we identified and provides illustrative evidence from selected cases.

Multisensory and audiovisual detecting. The first set of designed activities we uncovered was related to the quantity and quality of sensory cues each realm provided for participants to discern and experience. The physical realm across the experiences we studied was in all cases intentionally designed to facilitate multisensory detecting. It offered participants opportunities to engage in actions that enabled them to perceive a wide range of spatial, interpersonal, and artefact-related sensory cues. For example, TimeCo added smells as olfactory stimuli or fabrics for tactile interaction to their physical–digital time travel experience, WineCo provided carefully curated wines for participants to smell and taste, and participants in TabCo or ArCo city tours could observe pedestrians and the cityscape, feel the outdoor temperature, and touch specific object surfaces (e.g., of a bronze statue).

The matched designed activities in the virtual realm can be described as forms of audiovisual detecting, that is, activities for participants to identify specific, computed cues that stimulate auditory and visual senses. These synthetic cues, like high-fidelity sounds or hyper-realistic computer-generated landscapes, enabled well-defined yet limited sensory experiences. For instance, in TimeCo, a well-known carnival song had been integrated as part of the VR tram ride’s grand finale, creating an auditory highlight. Similarly, synthetic soundscapes, like a cheering crowd during a celebration scene in TabCo, or floating aroma symbols in the WineCo experience, had been designed to amplify sensory engagement within the virtual realm and emphasize the particular phase of the experience. The infusion of olfactory stimulation paired with a digital audiovisual carnival scene by TimeCo is illustrated with the subsequent quote:

Another sense we introduce is the sense of smell. We have paired the smell of toffee with the carnival scene where sweets are thrown.–CEO of TimeCo

Bodily and expanded navigating. The second set of activities we identified focused on how an embodied person navigated through a physical or virtual realm. In the physical realm, we observed that navigation activities were tailored to the natural or curated spatiotemporal boundaries (e.g., walls in a physical venue room, streets, and rows of buildings in a city center), enabling a natural, embodied form of locomotion (e.g., walking, running, jumping), consistent with real-world physical experiences. Thus, all cases included designed activities that were about bodily navigating. For example, in the ArCo case, participants were allowed to walk through the city center within physical spatiotemporal boundaries, such as walls and busy streets, while aligning with the tour route’s key sights (e.g., a renowned theater or a church). Conversely, the corresponding designed activity of expanded navigating in the virtual realm was designed to enable participants to audiovisually explore spaces, which were either physically inaccessible or entirely imaginary within distant or fantastic realms, outside regular spatiotemporal boundaries (e.g., events occurring far earlier or later than the natural progression of time, such as a 1920s carnival parade), or which involved forms of locomotion typically not available to humans (e.g., flying, teleporting). In the TabCo case, participants could visually explore a brewery cellar – normally inaccessible to the public – via its detailed virtual representation on a tablet PC. In the ArCo case, participants could walk through a virtual portal into the entrance hall of a historic mansion. However, in the physical realm, just a few meters behind the portal, a storefront display in the city center would block physical navigation in that area or cause participants to walk into the store. To prevent this, specific coordination mechanisms were incorporated to map bodily and virtual movement to avoid such situations, as the CEO explained:

People sometimes placed the scenes in completely absurd locations, leading to crazy consequences. At times, you had to position scenes through a shop window, only to realize that you couldn’t actually go through the window. There was also the funny side effect that people would enter these shops and practically walk into the scene inside. –CEO of ArCo

Naturalistic and scripted interacting. The third set of designed activities focused on the modes of social interaction facilitated in both the physical and virtual realms. In all four experiences we studied, participants interacted with both human actors and virtual characters in ways that either aligned with an experiential narrative (e.g., introducing a historical character) or served functional purposes (e.g., guiding the tour or assisting with technical issues). In the physical realm, we observed that by employing human guides and/or facilitating conversations among participants, providers designed forms of naturalistic interacting that allowed participants to engage multimodally with human actors in both scripted and unscripted ways. For instance, in the WineCo case, a human tasting leader was employed to engage with participants, sharing sommelier knowledge, asking about the aromas they perceived, and handing wine bottles and glasses. While these interactions followed the experience’s theme and were part of the scripted narrative, they could occasionally be spontaneous when participants asked follow-up questions. In contrast, in the virtual realm, interactions with virtual characters were designed to foster scripted interacting, where participants engaged in algorithmically pre-programmed brief interactions that were triggered by audiovisual cues or gestures. These interactions were generally nonverbal and varied in interactivity. For example, in TimeCo, when participants turned around in the tram they had been sitting in during the VR ride, virtual characters adjusted their gaze to face them directly. Two explanations illustrate these design choices:

We have incorporated triggers which, given your high status, allow you to experience unique interactions from your position within a beautiful baroque carriage. Specifically, when you gaze at certain characters for an extended period, they will actually bow. I believe these are the primary triggers designed to enhance engagement and introduce more interactive components. – CEO of TimeCo

There was a conscious decision behind this. When you have an experience like this you still need some form of guidance from a human entity. Whether that guidance is remote or delivered through a recorded voiceover, I don’t find it as valuable. However, for a wine tasting, especially given the novelty involved, it was incredibly important to have someone present – like myself – or other wine and gastronomy professionals who could also provide presentations. – CEO of WineCo

Haptic and algorithmic handling. A fourth set of designed activities allowed participants to interactively manipulate physical or digital components within both physical and virtual realms. In the physical realm, providers created activities for haptic handling, meaning the possibility for participants to physically grasp, reshape, or reposition tangible artefacts with physical and spatiotemporal attributes such as shape, weight, size, or surface conditions. For example, in the WineCo case, participants physically swirled a wine glass to explore the sensory cues, such as the intensity of the wine’s aroma. In the TimeCo case, a newspaper was handed to participants at the start of the experience, serving as a tangible link to the historical story that would unfold in the subsequent experience. In the virtual realm, the equivalent activities manifested as a form of algorithmic handling, where participants used bodily cues (such as gaze or touch) to trigger algorithmic procedures that manipulated audiovisual features of the virtual experience. For example, triggered by the directed gaze of participants in the ArCo city tour, a virtual Madonna statue changed its visual properties according to the positioning and direction of the participants’ look, as the CEO explained:

And then you can virtually pick up the artefacts in the scenes and move them around or something. Or the projection of the golden Madonna, as I said, has a crown. The crown is already there. And the idea was that you could then put the crown on the Madonna. – CEO of ArCo

While our data analysis allowed us to identify four pairs of activities designed in either the physical or virtual realm, it also became clear that the experiential consumer offerings included activities that spanned both realms and drew on components of both. Providers recognized the functional need to combine and orchestrate activities across the physical and virtual realms to integrate them, overcome the experiential limitations of each realm, and familiarize participants with immersive systems by maintaining references to the familiar physical realm. This created a unique experience that extended what could be done and experienced in the physical and virtual realms in isolation.

Even the GPS tracking was based on the same idea. Originally it was going to be an automatic control, meaning that when you arrived at the location, the scene would automatically start at the right point without any user control. However, we realized that because the pedestrian zone is so busy and constantly changing, with construction sites and installations, if we had fixed it to a specific point - let’s say there was a weekly market there and something was being displayed - it wouldn’t have worked. So the next logical idea was to allow users to set these scenes themselves through gestures. – CEO of ArCo

The result of these efforts is what we call a physical–digital experience that integrates constituents from both the physical and virtual realms and allows participants to be digitally immersed while also being physically embodied. One executive stated this intent as follows:

The main motivations were to offer a guided tour that is not only analogue, but also uses digital components to make things visible that you can no longer see in the real world, not only in analogue form, but also through digital media. In other words, looking into the past to create a context for the here and now. To make visible things that would otherwise remain hidden because the door is locked. To show things you can’t get into, or scenes that take place at another time of year. – CEO of TabCo

This quote illustrates how providers integrated activities across different realms. Each experience allowed natural movement, through the city center (natural navigating), exploration of a brewery cellar (unbounded navigating), or walking through a historic marketplace. Participants encountered multisensory stimuli from the natural realm (e.g., seeing and touching a bronze statue, experiencing the cityscape) (multisensory detecting), alongside emphasized cues (e.g., cheering in a virtual carnival projection) (audiovisual detecting) via a tablet PC. A human host provided guidance (naturalistic interacting), while gestures or repositioning caused objects to change shape or perspective (e.g., statue projections, passing carriages) (algorithmic handling).

We identified three key characteristics of the resulting physical–digital experience that characterize it as an integrated experience: spatiotemporal and social recombination, sensory enrichment, and cross-realm anchoring. Spatiotemporal and social recombination describes how the physical–digital experience provided participants with a new layer of meaning in the experience that was designed by recombining experiential components from the physical or virtual realms across spatial, temporal, or social dimensions. In our cases, these components could be of spatiotemporal (e.g., the historical and contemporary cityscapes were visually compared) or social nature (e.g., the exact historical figure appeared in both realms), and they were recombined through narrative and spatiotemporal connections. For example, during the AR city tours of ArCo and TabCo, the projection of authentic historic facades onto their contemporary counterparts in the cityscape at precise locations allowed participants to engage in a direct visual comparison of the present with times long past. This feature allowed participants to learn about architectural evolution and broke the linearity of time, enabling participants to experience past events in distant realms while underscoring the constant reference to the here and now. Consider how the designer of TabCo expressed spatiotemporal and social recombination:

They can see a scene of what the old town looked like in the 19th century or what the old town looks like when we have the Rose Monday parade. Otherwise you’re standing in the square and there’s a monument and there’s a city hall and there’s people walking around and there’s nothing else. And so you can experience this particularity of a Rose Monday parade that you can only experience if you are there, even in the summer, at a different time of year. – CEO of TabCo

Moreover, the integration of holograms of historic figures, represented by celebrities, into the otherwise static cityscape introduced a synthetic social interaction, as indicated by an informant from ArCo:

These are some scenes, but it’s these interactive scenes - for example in the pub, where two people talk to each other the whole time and you still have the opportunity to look around – that really captivate people. And these scenes wouldn’t be as successful without the actors. – project lead at ArCo

The second salient characteristic of integrated physical–digital experiences was their sensory enrichment, which describes the multisensory richness participants could experience when presented with orchestrated interdependent physical and digital sensory cues. Sensory enrichment occurred when physical sensory cues intensified or supplemented corresponding digital sensory cues, and vice versa. In the case of ArCo, for example, participants could listen to festive street music and watch a carnival in the virtual realm, while still being able to discern the background noise of a bustling city from the physical realm. In the case of TimeCo, participants could catch the sweet smell of toffee in an old theater while watching a black-and-white movie premiere. Virtual characters on the screen seemed to inhabit the theater alongside them, providing an immersive sensation of shared spaces. In one scene, a carnival truck could be seen and heard, where people were throwing sweets, and this digital sensory cue was supplemented by the infused physical toffee smell. Sensory enrichment was thus key, as explained by the main designer of TimeCo:

Adding scent during the VR ride and consciously triggering certain moments during the ride is another aspect. When I drive past Cologne Central Station, the train comes in, and suddenly there is a bit of steam, or at the end, when the carnival candies are thrown, I have the smell of toffee candy. I believe that this adds another level of immersion to the experience. – CEO of TimeCo

The third salient characteristic of the physical–digital experience we uncovered was cross-realm anchoring, which describes how activities in one realm functioned as physical, conceptual, and social anchors in the opposing realm. For example, in the case of ArCo, AR-mediated navigation cues, such as digital arrows and distance visualizations, assisted participants in navigating through the physical city layout and alerted them to physical obstacles, such as intersections. Here, the digital symbols drew on physical geolocation points tracked through sensors for precision. In this case, cross-realm anchoring was designed to ensure a seamless and safe exploration of the urban environment. In the TimeCo case, cross-realm anchoring manifested in the form of an artefact, a hat. Participants could touch, wear, and hand a physical hat to a human actor in the replicated 1920s millinery. The same hat was displayed multiple times throughout the experience, in both physical and digital form, thereby establishing a point of cross-reference between the realms. Similarly, we also discerned social references, such as when human actors and virtual characters reappeared throughout the experience in both realms. As the designer put it:

We have brought a real actress into these three dimensions. And that was also a need for us to say that’s there and that’s what a physical hat store looked like back then. And it has been reconstructed and it has also been deliberately reconstructed together with a traditional hat-making family from Cologne. And then we just said, okay, in the third part we’ll transfer the figure into the virtual world, where it already has a task that we encounter again and again and so on. And that was the transition. Of course, there is a difference between a digital character and a real character: it’s a bit of a transition between a virtuality and a reality, a bit of a smooth transition. And it helps me a bit to build this bridge into this world. –CEO at TimeCo

Finally, in cases like ArCo or WineCo, the presence of human hosts facilitated cross-realm anchoring. They served as social reference points that helped participants navigate the virtual realm or assist when technological errors disrupted the experience.

Figure 3 shows the conceptual model we developed to organize and abstract the insights we gained about the design of activities in both physical and virtual realms and how these can be integrated to create a coherent physical–digital experience. The model presents two sets of designed activities in physical and virtual realms and shows how their integration creates a fusion of the two realms through localizing and de-localizing each other. This fusion produces an intentionally integrated physical–digital experience through spatiotemporal and social recombination, sensory enrichment, and cross-realm referencing. These three attributes are what characterize physical–digital experiences as cutting across both physical and virtual realm constituents to allow participants to be both digitally immersed and physically embodied.

Full size image

Key to our explanation of the design of integrated physical–digital experiences is how activities are designed within both the physical and virtual realms so that they constitute different yet semantically aligned operations that together overcome the experiential limitations of each operation alone. The key mechanism to this integrated fusion that produces the integrated experience is an interdependent cycle of localizing and de-localizing experiential activities. Localizing involves grounding experiential activities in the physical realm by leveraging tangible components such as multisensory input, perceptible features of the immediate physical surroundings, and social interactions with other human beings, creating a concrete and embodied experience. This spatiotemporal grounding ensures participants remain aware of their physical presence and can engage naturally with their environment. Components of the physical realm, such as participants’ physical position and peripheral cues like smells, temperature, and breezes, help anchor the experience by maintaining a connection to the real-world context. De-localizing, on the other hand, utilizes the virtual realm’s capabilities to transcend the physical and spatiotemporal boundaries of an experience, thereby opening new simulated dimensions that extend beyond reality. This process breaks traditional limits such as geography, time linearity, bodily limitations, and physical laws, enabling users to experience the seemingly or physically impossible – such as flying, meeting mythical creatures, or traveling through time and distant places. These new possibilities augment the experience in ways that the physical world cannot.

Our model also explains that four pairs of activities in physical–digital experiences correspond to four categories of experiential action. First, in experiences, participants can detect audiovisual cues in the virtual realm or discern full-blown multisensory cues in the physical realm. Both kinds of cues are designed as deliberate components of an experiential consumer offering. Second, experiences revolve around interacting with social and non-animate surroundings (Merleau-Ponty 1962; Yoo 2010), whether naturalistic or scripted. Third, almost every experience, whether digitally mediated or physical in nature, involves some form of locomotion, or the illusion thereof, whether bounded within the possibilities of the human body in space and time or expanded altogether. Fourth, most experiences that involve physical or digital artefacts require handling, either repositioning or reshaping tangible artefacts through haptic interaction or their audiovisual properties, executed through algorithmic logic.

The fusion of these pairs of activities for detecting, interacting, navigating, and handling, through a mutually constitutive cycle of localizing and de-localizing, gives rise to the characteristics – spatiotemporal and social recombination, sensory enrichment, and cross-realm anchoring – that together coin the complexified and multilayered nature of a physical–digital experience. For example, the time travel experience offered by TimeCo began in a curated physical realm, where participants could discern a wide range of multisensory cues – they caught the sweet smell of toffee, sat in a digital movie theater that showed a spectator’s perspective of the main characters and initiated the time travel experience on a movie screen, felt the softness of fabrics, and stepped into a 1920s millinery shop surrounded by jazz on vintage vinyl (a form of spatiotemporal and social recombination). In this room, the experience crossed realms: digital representations of a historical figure appeared on a digital screen and interacted with the human actress in the physical realm to create a smooth and frictionless transition between realms (an instance of cross-realm anchoring). As the experience progressed, participants sat down in a physical cable car for a virtual tram ride through historic Cologne, instantly whisked back a century in time. During the ride, participants not only recognized high-fidelity visuals (e.g., historic facades, virtual characters roaming the streets) or sounds, but they also felt the tram’s speed and the breeze on their faces and clothes thanks to strategically placed air movers (an example of sensory enrichment). Taken together, the physical–digital experience exceeded any experience that could be provided by the physical or virtual realm alone. At all times, whether walking through the physical venue rooms or experiencing the VR tram ride, the physical embodiment of the participants localized the experience in the here and now, maintaining their receptivity to physical sensations and the ability for haptic interactions, while the virtual renditions of 1920s Cologne in the VR tram ride de-localized the experience by breaking the experiential limitation of the physical realm to experience landscapes beyond the spatiotemporal reach of individuals.

Our findings contribute new insights to the literature on immersive systems. We add new knowledge about how designing activities, such as movement, gestures, and embodied interactions, can collectively shape integrated physical–digital experiential consumer offerings that span across physical and virtual realms, combining components and enabling sensory stimulation that neither realm can provide on its own. Our analysis highlights the importance of integrating designed activities within both physical and virtual realms through a dynamic process of localizing and de-localizing that anchors an experience in tangible, site-specific contexts to acknowledging that human beings are perpetually embodied, experiencing the world through bodily movement and physical presence whilst also liberating participants from physical and spatial constraints by incorporating the boundless possibilities of the virtual realm. The result is a multifaceted, holistic sensory physical–digital experience that integrates the strengths of both realms.

Our study also adds to the broader conversation about the role of physicality for the digital innovation of offerings such as products or services (e.g., Recker et al. 2024; Von Briel et al. 2018; Wang et al. 2022). We contribute new knowledge about the experiential aspect of physicality as the spatiotemporal context in which individuals experience sensory, social, and spatial cues that trigger or bind bodily experiences. This consideration of how the integration of the physical and virtual realms creates a physical–digital experience – achieving meaning and significance that neither realm could achieve on its own – is our key insight because it highlights that physicality is not a countermeasure to the immersive potential of digital innovations, but rather a catalyst. When thoughtfully integrated and synchronized with a designed virtual realm, physicality can extend the limits of both realms by allowing for recombination, cross-realm referencing, and sensory enrichment of experiential components in either realm.

Finally, by highlighting the importance of cross-realm referencing, our study draws attention to neglect. For example, sensory cues like sounds that are not aligned with the experiential theme (e.g., a cellphone ringing during a historical city tour) (Oh et al. 2019) or locomotion in one realm that is not mapped to properties of the other realm can disrupt the experience and lead to unintended consequences (e.g., participants bumping into a storefront during a city tour) (Steinicke et al. 2009). Failing to account for the limitations or boundaries of one realm can cause serious harm (Chesler 2024). In experiences set in the natural realm, this insight has led designers to create activities that promote cross-realm anchoring and warn participants of physical risks, such as busy streets (as seen in our ArCo case), within the virtual realm through audiovisual cues or notifications. Thereby, our work highlights not only the benefits of accounting for both the physical and digital realms in experience design (Melo et al. 2022; Wang et al. 2022), such as increases in immersion by adding physical sensations to a virtual experience (Harley et al. 2018), but it also highlights the potential for severe distractions or risks that can emerge from a lack of cross-realm referencing.

Different avenues for future research exist that could build on our findings. These are summarized in Table 5. The first key opportunity to progress from our work lies in evaluating the qualities of an integrated physical–digital experience. Our work suggests that integrated physical–digital experiences are characterized by three core attributes that designers adopt as their design goals. However, our work does not make evaluative claims, so we recommend that future research incorporate participants’ perspectives to assess the quality and impact of experience design that follows the design goals and choices we uncovered. This could involve measuring perceived value, quality, and levels of immersion. For example, evaluating immersion among participants is particularly relevant, as this cognitive concept has been shown to drive satisfaction and joy in experiential contexts (Agarwal and Karahanna 2000; Peukert et al. 2019). Similarly, evaluating the perceived value of experiences can provide insights not only into participation outcomes but also into potential entrepreneurial benefits for companies (Lorenz et al. 2024). Linking such insights to the design choices made across the offerings could provide valuable information, as shown in other contexts (e.g., Hsiao et al. 2024; D. Liu et al. 2017; Y. Liu et al. 2019. For example, we suggest that blending activities across physical and virtual realms is advantageous. However, future studies can help specify the mechanisms of blending and identify which of the two realms should serve as the anchor under specific conditions. Additionally, they can investigate the effects (e.g., in terms of levels of immersion) of an integrated versus an unintegrated experience. Future studies can also explore how the blending of designed activities may differ between VR and AR.

Further empirical work (using classical design science or design archaeology approaches) would also be needed to formulate prescriptive, context-sensitive guidelines for when which physical–digital design configuration is indicated and which conditionalities (e.g., type of experiential offering, physical realm characteristics) need to be considered (Gregor et al. 2020; Peffers et al. 2007). Future research could benefit from a more granular analysis of physical–digital experiential consumer offerings, for example, by examining the relative weighting of each realm in shaping the experience and allowing statements about how to prioritize the design of which activity by localizing and de-localizing. This approach could yield more nuanced design insights and offer a deeper understanding of how different realms contribute to the overall immersive experience (Steffen et al. 2019).

Finally, we advise research to build on our observations that physicality (embodied by the physical realm and its constituents) serves as an anchor and lever for digital innovation and that human beings are always embodied, sensually experiencing entities (Chughtai 2021). This implies studying the roles that physicality can play in digital innovation even more systematically (Goebeler et al. 2024) and with a specific focus on commercial offerings in the experience economy (Pine and Gilmore 1998).

Practically, we can distill preliminary design advice from our findings that could be useful for experience-economy providers. First, designers should thoroughly explore and understand the realms they are working with (Wang et al. 2022). Our analysis identifies four key realms: natural and curated physical realms, as well as distant and fantastical virtual realms. This understanding enables a clear distinction between native components – those inherently present in their natural environment – and designed components, which are the outcomes of physical or computational manipulation. These diverse components collectively contribute to the user experience and can be seamlessly integrated into activities that can be crafted within and across both physical and virtual realms.

Second, designers can adopt the three characteristics of the integrated physical–digital experience we identified as their design goals. For example, to achieve sensory enrichment, we recommend that providers evaluate which stages or components are most critical to their experience and, based on this analysis, enhance those sensory cues by either intensifying or supplementing them with other sensory cues (e.g., pairing natural soundscapes in a city tour with synthetic street music) (Lorenz et al. 2024). Doing so could also include adding tactile artefacts, incorporating scents, or even introducing taste cues. In this way, providers can mitigate some of the financial costs of digitally produced immersion and compensate by leveraging the physical activities available (here, multisensory detecting).

Third, designers should think about semantically matching pairs of activities when considering which components to design in physical and digital realms. Our study suggests that activities can be specifically designed in one realm to be semantically matched to a set of activities in the other realm, and that blending the sets across realms can integrate the physical–digital experience. Designers can intentionally create activities in one realm to either localize or de-localize activities in the other. Poor semantically matched pairs of activities may lead to issues in localizing and de-localizing, which, in turn, affect the cross-realm referencing in the designed physical–digital experience. These issues may even lead to safety problems, such as when participants walk toward a tangible barrier in the physical realm (e.g., a wall) while seeing a vineyard in the virtual realm. An example of a semantically matched pair of activities is the synergy between haptic and algorithmic handling routines, which incorporates not only the act of grasping a box in the physical realm but also coordinates an audiovisual response in the virtual environment to enhance the experience. In addition, designers can structure interactions in a variety of ways: they can script interactions with historical or mystical figures to follow a storyline (scripted interacting), or they can design more organic interactions that allow for flexible, realistic engagement (naturalistic interacting), which may be appropriate when the experience requires constant supervision of the use of the technology (e.g., for older groups of participants) or the experience itself (e.g., as a sommelier in a VR wine tasting).

Fourth, our findings provide some insights about how providers can achieve a coherent physical–digital experience cost-effectively. Particularly in cases where the physical realm plays a significant role in the experience (e.g., city tours where the status quo of the site is a central part of the experience), it is important not to focus solely on the creation of highly immersive digital components enabled by sophisticated and costly immersive systems technology (both hardware and software) (Wang et al. 2022). Rather, it is advisable to consider more mundane applications where immersive systems enhance the experience by overcoming some of the limitations of the physical realm and revealing some of the characteristics of a physical–digital experience. An example of this is ArCo’s city tour, where more mundane digital overlays are projected onto the present cityscape via a hand-held tablet PC, but show scenes from long ago, allowing experiential layering without the investment of a full-blown VR experience.

Our study is not without limitations. First, we caution that the ability to generalize from our research is limited. The transferability of the findings from our study is, amongst others, constrained by the limited heterogeneity of experiential consumer offerings (experiential consumer offerings cover a wide range of types, such as adventure activities, aesthetic cultural experiences, or meditation workshops). Furthermore, all of our cases were located in Germany, and we were not able to cover the broad range of potential immersive systems configurations that could be used for such offerings (e.g., Dincelli and Yayla 2022). Still, all our cases faced a similar challenge and common goal: how to design activities that leverage properties of the virtual realm (produced by immersive systems) and merge them with those created in the physical realm to provide an experience that crosses what participants would be able to perceive or do in either realm alone. Thus, our abstracted theoretical concepts may be useful for analysis and explanation in other settings that involve alternative experiential themes, different forms of immersive systems mediation, or different geographic locations. It is important to note, however, that their usefulness remains an empirical question.

Second, we sampled multiple cases and collected data until we felt saturated with the information that additional data slices could provide (Charmaz 2006; Urquhart 2012). Nevertheless, our data, while drawn from multiple sources and rich in information, could be expanded, and our analyses, while traceable and transparent, could be done differently.

Third, other types of cases might exhibit different configurations of designed activities, which could manifest in different ways. For example, movement might be restricted even in the virtual realm, such as in a virtual room that can only be explored through small head movements, thereby lacking the potential for expanded navigating and its role in de-localizing the experience. Alternatively, certain designed activities may be entirely absent, such as in cases where a physical–virtual experience does not require any form of social interaction, neither naturalistic nor scripted. Further, the advent of AI-coordinated social interactions in virtual realms with virtual characters, powered by advanced large language models (LLMs), could delimit the activity of scripted interacting in the virtual realm. The first indicator of this shift is in cutting-edge avatars, whose responsiveness and reactivity go beyond the narrow scripts present in the cases studied (NTT Data 2024).

Finally, we propose four design implications based on the insights from our empirical study linked with our understanding of prior works. Despite their empirical and theoretical grounding, these design implications can benefit from future validation in terms of proof of concept – the extent to which they are applicable in designing new physical–digital experiences – or proof of value – whether and how much these guidelines help to improve the design process and outcome (Nunamaker et al. 2015).