Technological mediation and 3D visualizations in construction engineering practice

The digitization of three-dimensional (3D) building data through remote-sensing technologies, Building Information Modeling, and Augmented Reality has come to play a significant role in construction engineering practice (Lu et al. 2015). Building Information Modeling (BIM) is a key digital technology that serves as a software platform representing a digital model in which generated 3D data and other information about a project can be stored (Garbett et al. 2021). BIM models can be filled with 3D data from remote-sensing technologies that monitor a building or parts of a building during its whole life-cycle (Davtalab et al. 2018).

Remote-sensing technologies being rapidly adopted by the construction sector include three-dimensional (3D) laser scanning and ground penetrating radar technologies. Laser scanning technology (LST) has the ability to develop a 3D geometry of building and infrastructure objects—such as a building—through point clouds obtained from laser scans (Omar and Nehdi 2016). Ground penetrating radar (GPR) is another inspection technique capable of detecting subsurface properties through electromagnetic waves (Benedetto et al. 2016). By generating digital 3D building data in construction projects (O’Keeffe and Bosche 2015), these technologies support construction practitioners by mapping existing site conditions and enhance project control by providing means to monitor progress. LST and GPR necessarily use algorithms to automatically process the collected input and shorten the modeling process necessary to generate 3D images. By generating images of building objects or aspects of these objects that would be largely imperceptible to the human eye, LST and GPR extend human optical measurement capabilities. Nevertheless, these technologies need to be carefully calibrated, and deliver data that still require filtering and interpretation to support the decision-maker. Practitioners face several challenges during its use, because the interpretation of images is not straightforward and depends on training and expertise (Benedetto and Pajewski 2015).

BIM as a digital platform is also the basis for the use of 3D visualization devices on construction sites (Garbett et al. 2021; Williams et al. 2015). BIM-based 3D visualizations are used on construction sites by technologies that ‘augment’, or overlay, the real world with BIM data that seem to co-exist with the real world. Augmented Reality (AR) technologies add digital elements or objects to the user’s surroundings (Liberati 2018). These digital elements and objects are superimposed on the everyday world that serves as the background. The digital objects are part of the visualization represented by a mobile AR device (Li et al. 2018). Mobile AR devices are portable displays and include smartphones, tablets, and wearable Head-Mounted Displays (HMDs). These allow users to interact with data from both actual and virtual building objects and to monitor construction progress by comparing the as-planned and as-built status (Dunston and Shin 2009). Mobile AR devices rely solely on data uploaded into the system, which makes its visualization features only as ‘good’ as the information they are based on. AR devices may invoke a ‘false sense of security’ in people if they do not have a full understanding of the technological working principles, and consequently misinterpret the technique.

Misunderstandings about how 3D images and visualizations are generated may lead to poor decision-making. Providing insights into the ways in which remote-sensing technologies transform data input into a BIM and how BIM-based 3D images and visualizations are used on construction sites will contribute to more realistic expectations about what these technologies can do. Studying this transformation from data input to visible output is also the focus of the philosophy of technological mediation. (Fried 2020; Friis 2017; Ihde 2009; Rosenberger 2013; Verbeek 2015). The central idea of this philosophy is that technologies mediate the relationship between humans and the world they experience (Coeckelbergh 2020; Ihde 1990; Verbeek 2006). The objective of this study is to explore the form of technological mediation that the generation and use of 3D images and visualizations provide between a human and objects, or aspects of these objects, which would otherwise be largely imperceptible to professionals in construction practice. Two major questions are how are 3D images and visualizations generated, and how do they interact with their users? The scientific contribution of this study is in connecting the generation and use of 3D images and visualizations on construction sites to this technological mediation perspective. As a first study in the literature, we adopt concepts of ‘responsive digital materials’ and their ‘technological intentionality’ as lenses to study how 3D images and visualizations are generated in construction engineering practice. Their use in this practice is analyzed by adopting the lens of ‘double mediation’ of ‘augmentation relationships’ of Rosenberger and Verbeek (2015). In an augmentation relationship, a technology directs a user’s intention in different directions.

In this study, we first elaborate on the concept of technological mediation: its philosophical background and Ihde’s typology that describes different ‘actant roles’ or ways that technologies mediate one’s perception of the world. We discuss the transformation of data inputs into outputs in terms of ‘responsive digital materials’ and their ‘technological intentionality’. Second, the generation of 3D images and visualizations in construction practice is described through a review of the literature and empirical studies on LST and GPR using algorithms that process the collected input necessary to create these images. We also review empirical studies to explore the use of 3D images and visualizations augmenting the construction worker’s view with representations and instructions to support on-site construction processes. Subsequently, we analyze the kind of technological mediation that the generation of digital 3D images and visualizations provides in construction practice through applying the concepts of responsive digital materials and technological intentionality. The use of digital 3D images and visualizations is analyzed in terms of different ‘actant’ roles and double mediation of augmentation relationships. Finally, conclusions are drawn.

In this section, the background to the mediation approach is discussed first. Second, the use of data is discussed by elaborating on different ‘actant’ roles that technologies may play in shaping the thinking and acting of individuals. Third, we discuss the generation of 3D images by conceiving technologies and their inbuilt selectivities as responsive digital materials.

In the introduction section, it is explained why this study is undertaken, followed by introducing the basic theoretical concepts in the subsequent section. In this section, the two following steps of this research are introduced: (1) a review describing GPR and LSR prototypes and empirical studies on the generation and use of 3D images and visualizations on construction sites; (2) a discussion on the kind of technological mediation that both technologies provide in construction practice through applying the concepts of technological selectivities, digital materials, and double mediation. Table 2 shows the research design.

First, we review studies that elaborate on the way the acquisition and processing of data and data modeling take place using GPR and LST. That is, how GPR and LST receive ‘input’ and transform this into visible ‘output’. Both LST and GPR are detection techniques that ‘translate’ the input they collect into 3D images that humans are able to perceive and understand. These images mediate one’s experience of the world—of an otherwise invisible object or invisible aspects of an object (Friis 2015). With its emphasis on human practices and experiences, postphenomenology views empirical studies (work of others or self-conducted studies) as the basis for philosophical reflection (Rosenberger and Verbeek 2015).

The exploration, of generating 3D images through GPR in construction practice, was drawn from an ongoing empirical program carried out at the authors’ research institute. Part of this program was the review of empirical studies on developing methods that could identify subsurface infrastructure. Exploring the generation of 3D images by LST in construction practice was part of another research project, also carried out at the authors’ research institute. Part of this research project was the review of empirical studies on developing and implementing algorithms that could process point cloud data, collected through internal and external scanning of a building, into 3D coordinates. The focus was on imperceptible geometric details of the visible surface of a building. Empirical studies on the use of 3D images and visualizations of the existing buildings and on-site subsurface infrastructure were also reviewed. Mobile AR devices can orient and display 3D virtual models over a camera image by adding 3D information to visual models. In the empirical studies reviewed, the focus is also on mobile AR devices that augment a construction worker’s view by adding assembly instructions, such as Head-Mounted Displays (HMDs). 3D images and visualizations can be integrated in an HMD to enhance the user’s view on the site.

Second, conceiving LST and GPR technologies as responsive digital materials initiates a postphenomenological analysis on how technological intentionality makes certain aspects of reality accessible to humans through data processing. The focus is on “the specific way in which a specific technology can be directed at a specific aspect of reality” (Verbeek 2008) [p. 393]. This directedness is programmed into the technology by its developers. The directedness of the sensing apparatus of each of the technologies reviewed, or their built-in selectivities (Ihde 2015) [p. xv], is elaborated with descriptions of how the sensing apparatus is configured to acquire input in certain ways (Ihde 1990; Wiltse 2014). Both LST and GPR receive ‘input’ and transform this into visible ‘output’. Using the concepts of built-in selectivities and responsive digital materials, we analyze how through data processing, LST and GPR generate images that will cause a user to experience a certain, amplified, aspect of reality, while the experience of other aspects of reality is simultaneously reduced (Ihde 1990).

The use of 3D visualizations on construction sites is analyzed by applying the concept of double mediation of augmentation relationships. We elaborate on this by analyzing how mobile AR devices have the ability to direct a user’s intention in different directions (i.e., the mediation effect of technology). The taxonomy in Table 1 served as a framework to identify the phenomenological relationships analyzed. By understanding the different mediating or ‘actant’ roles of mobile AR devices, in an augmentation relationship, we provide insights into what these technologies actually do in construction practice.

Technologies such as GPR and LST and Building Information Modeling (BIM) significantly overlap, since GPR and LST are about the generation of 3D visualizations and BIM is often about their use on construction sites. The generation of 3D images and visualizations through GPR and LST is described first. Second, their use on construction sites is described.

We first analyze the generation of 3D images and visualizations through GPR and LST by conceiving remote-sensing technologies and their inbuilt selectivities as responsive digital materials with substrates and traces. Second, the use of resulting 3D BIM images and visualizations establishing various mediation relationships and playing different actant roles is analyzed. It is argued that there are double mediation relationships when using 3D BIM images on construction sites. In other words, using 3D BIM images is not exclusively bound to one type of mediated relation.

By generating 3D images and visualizations of building objects, or aspects of these objects, sensing technologies like GPR and LST extend human optical measurement capabilities. Nevertheless, users still face several challenges when interpreting these 3D visualizations because of the inbuilt selectivities of GPR and LST. A GPR—after inadequate processing, poor data collection, or due to the technological limitations—may be showing a ‘clean’ underground radargram without any hyperbolic signals signifying the existence of buried objects. This could lead to false-negative conclusions from a user that a scanned location is free of any utilities, while in fact, there might be a pipe cable on site. Further limitations are that a GPR generates images based on reflected signals, which makes it hard to ‘measure’ the exact geometry (Jaw and Hashim, 2013) of a utility line. Due to non-homogeneous properties of soil and the many possible underground network layouts, it is also challenging to identify multiple, co-located cables that lay closely alongside one another in the subsurface from radargrams.

Similarly, when applying LST, data collection may contain errors due to a surface’s shape, color, and conditions of the object scanned (Anil et al. 2013). Occlusions and cluttered images may also ‘hide’ relevant information from the modeler, and eventually feed incomplete information to the decision-maker. Incomplete data collection through LST, because certain locations on the construction site are inaccessible for scanning, may hence result in modeling errors and wrong interpretations of the images provided.

The algorithms that pre- and post-process collected input from techniques such as GPR and LST function as mediators through their technological selectivities. These selectivities determine possible differences between generated 3D visualizations and the ‘real’ properties of the physical objects that are not visible to the human eye. These differences attribute to the so-called epistemic uncertainties. Epistemic uncertainty derives from the lack of knowledge of a phenomenon, process, or system (Basu 2017). In the case of GPR and LST, it occurs either from transforming inputs into outputs inside these technologies or from external factors generating 3D visualizations. Verbeek (2008) claims that inbuilt selectivities are not necessarily directed at accurately representing the world but, rather, constructs a certain way of seeing the world. In providing 3D images of objects, GPR and LST technologies, as digital material, respond to the input in line with how they have been designed and programmed.

Next to processing and selectivities, human intentionality comes into play when users ‘read’ and interpret the 3D images provided (Rosenberger and Verbeek 2015). These 3D images amplify a person’s experience of a certain aspect of reality, while simultaneously reduce experiences of other aspects of reality (Ihde 1990). Reduction occurs, because the representation in a 3D BIM model or image is ‘reduced’ to views that can be expressed using the three-dimensional objects and properties available (Turk 2001). Through this reduction and amplification when using GPR and LST, a “blindness” may be created for other aspects of reality. Users limit their view to that, which can be represented in the 3D image or visualization as a result from technological selectivities as programmed and designed. As a result, users may interpret 3D images and visualizations in ways that match modeled reality and possibly disregard information that does not match this reality. When this results in 3D models that miss information about locations of utility lines, or geometric shapes of scanners facilities, this could result in incorrect interpretations of the images provided, and eventually lead to design errors and rework during construction.

Construction professionals frequently are not conscious of selectivities and intentionalities, and consequently consider the tool either as useful under all possible conditions, or not suitable at all. Failing to observe the conditions under which technologies mediate input into output, and the possibilities and limitations that processing algorithms provide, has eventually an impact on the end-user assessment of a technology. Ignoring this mediating role of the LST and GPR technologies in construction practice may result in faulty expectations about what these technologies can do. We thus argue for a need to be critical in assessing the output provided by these technologies. We posit that, when potential adopters acknowledge the mediating role of a technology, they can develop an appropriate set of decision criteria and applications contexts in which a technology needs to function.

It is shown that 3D images and visualizations may play different ‘actant’ roles (Latour 1992) at the same time. In the double-mediated augmentation and engaged relationships analyzed, one can speak of a dynamic back-and-forth movement between the hermeneutic and alterity relations on the one side and the background relation on the other. When the focus is on the virtual representations or instructions provided, human intentionality is directed to the technology and users’ relation with the project environment is in the background. Conversely, when the user’s attention is directed to the physical objects external to the augmentation, the virtual representations or instructions move to the background of one’s perceptions.

Through this augmented double mediation, 3D images and visualizations through mobile AR will generally reduce the susceptibility to mistakes in interpreting plans or designs. More specifically, it reduces the user’s ‘cognitive load’ by rendering a selected portion of a 3D model spatially on the user’s view (Dunston and Shin 2009). Double mediation through a mobile AR device influences the ‘risk awareness’ of engineers and contractors. It allows practitioners to answer questions such as ‘where can we dig safely?’ and ‘where do we need to execute work with extra care given the uncertainty boundaries and located existing infrastructures?’.

In sum, on one hand, technological intentionality through inbuilt selectivities may attribute to the so-called epistemic uncertainties. As a result from these selectivities, human “blindness” may be created, because users limit their view to that, which can be represented in the 3D image or visualization and disregard information that does not match this representation. On the other hand, the dynamic back-and-forth movement between the different relations through AR-enabled double mediations may lower the level of epistemic uncertainties. Compared to the typically employed printed maps or paper instructions, AR-enabled double mediation enhances context and situational awareness, and may reduce possible differences between interpretations of virtual representations and properties of physical objects observed.

This study’s objective was to explore the form of technological mediation that the generation and use of 3D images and visualizations provide between a user and objects, or aspects of these objects, which would otherwise be largely imperceptible to professionals in construction practice.

Generating digital 3D images and visualizations in construction practice was described through studying data acquisition, processing, and modeling using GPR and LST in construction practice. It was described how GPR and LST receive ‘input’ and transform this into visible ‘output’. We analyzed GPR and LST in terms of responsive digital materials and inbuilt selectivities. As such, both technologies are a ‘substrate’: the interface between objects in the physical world and their captured digital representation. A user of GPR or LST interprets images or ‘traces’ made visible by the technology itself. We conceptualized the digital material mediation of GPR and LST as human → ([trace | substrate] → world). We explained how the properties of physical objects could only be revealed by making the technological intentionality of GPR and LST accessible to human intentionality through ‘reduction’ and ‘amplification’ of the data collected.

We also explored the use of 3D images and visualizations augmenting the construction worker’s view with representations and instructions. Using the four human–technology–world relations introduced by Ihde (1990), it was found that mobile AR devices can, depending on use practices or the context of use, establish a range of mediation relationships between the project environment or construction site and those who experience it. ‘Non-interactive’ mobile AR fits with Verbeek and Rosenberger’s augmentation relation, and includes both embodiment and hermeneutic relationships as one not only perceives the construction site through a mobile AR device but also interprets the digital representation of the site revealed by such a device. The ‘interactive’ relation of engagement includes, in addition to an embodiment relation with the mobile AR device itself, also an alterity relation where interaction takes place with its users through virtual instructions. From this, we concluded that Verbeek’s concept of double mediations may create more realistic insights into, and realistic expectations of, what technologies actually do. On one hand, inbuilt selectivities may attribute to the so-called epistemic uncertainties and human “blindness”. On the other hand, the dynamic back-and-forth movement between the different relations through AR-enabled double mediations may lower these uncertainties. It is essential that the end-user understands this relationship to successfully align the features of a technology to support their project’s goals.

These reflections contribute to the construction and engineering practice by showing that generating 3D images and visualizations through technological prototypes, systems, and algorithms involve intentionality from both the technology (or its developer) and its user. Understanding this complex interplay may eventually help in developing and tailoring better technologies, and may increase our understanding of how and why technologies are either adopted successfully or not. Neglecting this mediating role could result in misinterpretations or overly optimistic expectations about what these technologies can do.

The augmentation relation indicates that users treat the technology as a ‘quasi-me’, but that it is essential that users know the constraints and limitations of projected project representations. An engaged relationship shows that augmentation, through virtual instructions, may create an ‘intelligent’ project environment that is explicitly used to influence human actions and to produce particular project outcomes. These conclusions shape various questions for the design and uptake of mobile AR in construction practice. Fundamental questions include what information should be presented, how, and what should algorithms do with project elements that are not shown but are necessary for decisions to be made? We therefore encourage a closer analysis of technology–human interaction through additional field studies, replacing a technology-push approach with a more iterative development and adoption cycles with reflection on the technologies’ mediated character. The framework we employed is promising for future studies in this area as it enables a deeper analysis of how new digital technologies that are entering the construction arena, such as IoT and cyber-physical systems, may steer user’s assessments, decision-making, experiences, and expectations.