Skip to main content
Erschienen in: Computer Supported Cooperative Work (CSCW) 6/2010

Open Access 01.12.2010

The Role of Integration in Health-Based Information Infrastructures

verfasst von: Gunnar Ellingsen, Kristoffer Røed

Erschienen in: Computer Supported Cooperative Work (CSCW) | Ausgabe 6/2010

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we contribute with empirical insight into the complexity of establishing and sustaining integration between different information infrastructures in health care. An overall concern is to elaborate on how, despite many obstacles, the integration effort moves forward. We see this as a collective achievement, where users have an essential role in terms of mobilizing and coordinating the other actors as well as maintaining the integration. These activities are not limited to a specific project; they emerge from and are part of day-to-day practice. Empirically, we focus on a large integration initiative between the laboratory systems at the University Hospital of Northern Norway and the electronic patient records used by general practitioners in the Northern health region. Together with the vendor, Well Diagnostics, the hospital initiated a project aimed at establishing a new laboratory requisition system that enabled GPs to send requisitions electronically to the hospital laboratories. Theoretically, we draw on the concept of information infrastructures, and supplement this with Actor Network Theory.

1 Introduction

An information infrastructure perspective has increasingly gained relevance in Western healthcare. Several countries have started to target their efforts towards interaction and integration between different institutions in the healthcare sector (Chantler et al. 2006; Ministry of Health and Care Services 2008; OUS 2009; Jones et al. 2008). The motivation is associated with improved efficiency and quality, as well as better exploitation of substantial investments made in the healthcare sector over several years (Chantler et al. 2006; Chiasson et al. 2007; LeRouge et al. 2007).
National health authorities have launched several measures to improve the information flow between healthcare organizations with the aim of establishing an interconnected healthcare information infrastructure. In Norway, there are ongoing projects on electronic prescription (KITH 2004), message-based interaction (KITH 2008), and electronic core records (KITH 2006). The Norwegian health regions are also preparing portal systems enabling integrated access for health staff to clinical information from heterogeneous information sources (OUS 2009). Similar initiatives are taking place in other Western regions, such as in England (Chantler et al. 2006), Scotland (Jones et al. 2008) and the United States (Singer 2009).
So far, the hoped-for benefits of large-scale integration remain visions rather than reality. Many projects do not deliver as promised (Ministry of Health and Care Services 2008; Auditor General 2008; Cross 2006). The problem appears to be composite, suggesting a variety of explanations. However, it is increasingly evident that the initiatives and efforts involve far more than a technical integration between different systems.
In this paper, we contribute with empirical insight into the complexity of establishing and sustaining integration between different information infrastructures in health care. An overall concern is to elaborate on how, despite many obstacles, the integration effort moves forward. We see this as a collective achievement, where users have an essential role in terms of mobilizing and coordinating the other actors as well as maintaining the integration. These activities are not limited to a specific project, but emerge from and are part of daily practice.
Empirically, we focus on a large integration effort between the laboratory systems at the University Hospital of Northern Norway (UNN) and the electronic patient records (EPRs) used by general practitioners (GPs) in the northern health region of Norway. Together with the vendor Well Diagnostics, the hospital initiated a project with the aim of establishing a new laboratory requisition system that enabled GPs to send requisitions electronically to the hospital laboratories. Today, the vendor has sold the system quite successfully to many hospitals in the south of Norway, but the actual implementation at UNN has proven to be far slower than expected. Hence, we examine this situation more closely and pose our research question as follows:
What are the nature, challenges and consequences of infrastructural integration in health care?
Specifically, our discussion proceeds along the following three dimensions: Firstly, we elaborate on how, when and by whom integration is performed and negotiated. Secondly, we explore how existing technology choices and strategy shape the integration and how this is handled. Thirdly, we discuss the opportunities and conditions for organizational redesign of integrated portfolios. Finally, we discuss how information is interpreted and translated as it moves in integrated infrastructures, and how this must be managed by the users.
Theoretically, we draw on the concept of information infrastructures (Hanseth and Lyytinen 2004; Bowker and Star 1999; Star and Ruhleder 1996; Hanseth and Lundberg 2001). We supplement this with Actor Network Theory (ANT) (Latour 1987; 2005; Law 1987).
The rest of the paper is organized as follows: in the theoretical section we outline our perspective on integration between information infrastructures. In the following section we describe and reflect on methodological issues. The subsequent section contains the case, where we elaborate on the project history as well as the work practice in the three different laboratories involved in the study. This is followed by the discussion and conclusion.

2 Theory

The notion of information infrastructure invites a large-scale inter-organizational perspective on information systems (Hanseth and Lyytinen 2004; Bowker and Star 1999; Star and Ruhleder 1996; Hanseth and Lundberg 2001). In the Western healthcare sector, such a perspective is increasingly more relevant, as different parts of the sector have invested heavily in information systems over the years (Chantler et al. 2006; Chiasson et al. 2007; LeRouge et al. 2007). In 2002, IT expenditures in the UK’s National Health Service (NHS) were estimated at £18 billion over a 10-year period—4% of the total NHS budget (Chantler et al. 2006). In the United States, a $20 billion cash inflow is aimed at digitizing the country’s health care system (Singer 2009).
However, while different areas of the health sector have well-developed support in terms of information infrastructure, the large-scale effects of the investments have so far lagged behind expectations (Cross 2006; Ministry of Health and Care Services 2008; Auditor General 2008), as many of these effects are associated with increased information flows across different healthcare organizations. It is no surprise, then, that many national measures have been implemented with the aim of establishing large-scale integration. Some examples are electronic prescription (KITH 2004), national core records (Jones et al. 2008) and large-scale integration in the NHS (Chantler et al. 2006; Wainwright and Waring 2004).
However, many integration projects in healthcare have had a purely technical focus, where integration has basically meant transmitting electronic messages through exchange standards such as EDIFACT or HL7 (Kalra 2006, p. 136). Unfortunately, targeting only the technical challenges has led to more failures than success stories (see, for instance, Berg 2001; Giuse and Kuhn 2003; Ellingsen and Monteiro 2006). A sobering example from the Norwegian healthcare sector is the first version of the nationally run, large-scale electronic prescription project, which crashed dramatically when a pilot system was launched in 2009. “A living hell”, one of the GP pilot users commented, and the implementation was postponed for one year.
Nonetheless, in the context of health care, the notion of integration has a highly ambiguous meaning. From a patient’s perspective, integration “marks the political and ideological commitment towards integrated care, i.e. service integration as experienced by individual patients” (Ellingsen and Monteiro 2006, p. 444). From a managerial or strategic perspective, integration is supposed to streamline and improve the medical processes across different organizational units to promote conformity with best practices (Wainwright and Waring 2004, p. 335). Finally, from the perspective of the individual physician, an integrated system is supposed to ensure easy access to data from multiple information sources (Tsiknakis et al. 2002), thus providing a complete picture of the patient’s medical history. Based on this, we do not adhere to a strict definition of integration, but note that in different ways the notion of integration is deeply embedded in complex organizational issues in healthcare organizations. Consequently, a fully integrated or seamless information infrastructure does not imply responding to the need of just one perspective, but involves paying attention to a multitude of practices, stakeholders and goals.
Since various organizations deal with completely different infrastructures, we need to consider exactly how these infrastructures are integrated. For example, this does not involve just one infrastructural hospital-based EPR delivered by one vendor, but many EPRs residing in several organizations, many of them with different vendors and technologies. In this regard, Bowker and Star (1999) introduce the notion of boundary infrastructures to emphasize the different practices and systems constituting a large-scale infrastructure:
“What we gain with the concept of boundary infrastructure over the more traditional unitary vision of infrastructures is the explicit recognition of the differing constitution of information objects within the diverse communities of practice that share a given infrastructure” (Bowker and Star 1999, p. 314).
Related to this, ANT is frequently used to conceptualize the relationship between technology and people (Latour 1987, 2005; Law 1987). Here, a key factor is how technological components and people together constitute actors in a network. We appreciate the way ANT gives us the opportunity to focus on the bits and pieces that shape as well as constitute an information infrastructure (Monteiro 2000). In addition, ANT provides us with a mechanism for looking at integrations as a dynamic process of negotiation. This contrasts with the more traditional view on how a great deal of information is shared in an inter-organizational healthcare context—apparently straightforward through standardized interfaces and messages (see, for instance, Grimson et al. 2000; Xu et al. 2000; KITH 2008). The ANT approach enables us to focus on how different sets of actors are involved in deciding how, where and when to integrate. Traditionally, integration is associated with vendors and how well they collaborate with each other. However, we believe that users also have a key role to play in mobilizing and coordinating the actors involved (including vendors) as well as maintaining the integrations in practice. The users’ influence may typically increase when integration gains momentum in practice, enabling them to put more collective pressure on the vendors to achieve specific functionalities (Johannesen and Ellingsen 2009). In ANT terms, this enables us to see the act of integration as an ongoing distributed activity, which is not solely associated with only a few actors and a few systems in a particular project period. Integration is rather something that emerges from practice as a collective achievement accomplished by different actors at different times.
One of the key actors in information infrastructure terms is the installed base—the existing portfolio of information systems. Depending on the size of the installed base, it will exercise various influences on the potential for integration. On the one side, the installed base can serve as a platform to build on, or to develop further, hence exploiting a technology that is already working well. On the other, the installed base may represent a problem due to its size and the way it limits action, for instance by being locked into a specific technology. Since integration is about increasing the size of an installed base, users may experience less flexibility as a result of integration (Elbanna 2007). Boudreau and Robey (2005, p. 5) make the general argument that when “technological artifacts become more tightly integrated into larger systems or networks, a narrower range of enactment may be expected from users”.
Hanseth and Lyytinen (2004) elaborate on how “gateways” may be a fruitful strategy to interconnect heterogeneous infrastructures, ensuring that different infrastructures can reside side by side without complex mutual alignment processes. A gateway is mostly promoted as a technical device:
“Designers should use gateways to connect together different regions of any infrastructure which run different versions of the same standard (14), or between different layers (15) or between related vertical infrastructures” (ibid, p. 225)
While we recognize the value of such a strategy, we are not completely convinced that this concept fully addresses the practical complexities that are involved in the establishment of fully integrated information infrastructures. We are more inclined to believe that interconnecting different but well-functioning infrastructures may require considerable adaptations, negotiations and manual maintenance. However, rather than regarding this as a shortcoming, we believe that this hidden work and workarounds (Gasser 1986; Grudin 1989; Pollock 2005) represent a condition for maintaining integration and seamlessness in infrastructures.
Still, inter-organizational integration carries the “promise” of large-scale design or redesign of the workflow (Davenport 1993). It is assumed that processes identified as bottlenecks may easily be replaced or reconfigured independently of organizational boundaries (Ashkenas et al. 2002). In healthcare, some of the envisioned effects of integrating different infrastructures into a coherent whole are typically associated with improved efficiency and quality. Some examples are clinical pathways and lean strategies (Trägårdh and Lindberg 2004). In comparison, the information infrastructure literature reflects a much more cautious attitude to the issue. One reason for this is that information infrastructures support different user groups, which often have different interests, goals and strategies that need attention. To emphasize how information infrastructure design projects entail various degrees of complexity, Star and Ruhleder (1996) identified three levels (or “orders”) of issues to be addressed in their study of the development of a large system for a geographically dispersed community of geneticists. The first order is related to concrete issues around implementation and use of the new system, typically involving “finding out about it, figuring out how to install it, and making different pieces of software work together” (ibid., p 118). The second order centres on unintended effects of the new infrastructure. Finally, the third order is inherently political and involved permanent disputes between the stakeholders (ibid., p. 118). While challenges on the first and the second level can largely be dealt with by adding more resources, challenges on the third level are not easily resolved due to the involvement of conflicting interests. Nonetheless, even at the risk of invoking third order issues, we argue that sometimes an inter-organizational redesign strategy may be fruitful to achieve larger organizational effects. However, a crucial question is who should be in charge of such a redesign. Basically, the information infrastructure literature emphasizes that such measures should be anchored in or should emerge from the practices themselves, both to avoid failure and to establish legitimacy.
Looking more closely at the information that is transported across different information infrastructures, we imagine the information as relatively standardized and stable or as an immutable mobile (Latour 1987). For instance, what is sent as a laboratory requisition remains the same when it is received and processed in the laboratory.
However, we challenge the apprehension of stable information objects in interconnected large-scale infrastructures. Information is shared across many contexts, and needs to be adapted to particular settings. By applying the notion of translation rather than transmission, Winthereik and Vikkelsø (2005) underscore how the recipient of a discharge letter plays several roles and how different users adapt the letter to their own context. They provide an example of how a GP modified the discharge letter by highlighting different sections to emphasize important points. Green was used to mark the reason for hospitalization and red was used to mark medications prescribed for the patient (ibid, p. 56). For information infrastructures, this may ultimately imply that some of the original meaning of the information gets lost on the way. This may have consequences for a large infrastructural workflow, where some objects are shared on a broad scale, and where end-to-end control is desired. Typically, we may imagine situations where GPs want to inquire about the status of a laboratory requisition. The question then becomes: In what shape is the requisition when different parts of the workflow engage in communication through it? What are its current properties? How can the integration be maintained, and by whom, under such circumstances?

3 Method

The University Hospital of North Norway (UNN) is the largest hospital in northern Norway, with some 5000 employees and 600 beds. It is run and owned by the Northern Norway Regional Health Authority, which in turn is owned by the Norwegian Ministry of Health and Care Services. In total, seven medical laboratories are located at UNN. Our field study has centred on three of the laboratories: the Medical Biochemistry, the Microbiology and the Pathology laboratories. We have chosen these laboratories carefully, because they have fundamentally different infrastructures and work practice. The Medical Biochemistry laboratory is largely automated, with a large portfolio of integrated analysis equipment. The work tasks at the Microbiology laboratory are far more complex and resource demanding. Finally, the routines in the Pathology laboratory are highly dependent on a paper-based workflow. In addition, we have focused on implementation and use in the GP practices. Approximately 220 GP practices located in North Norway submit laboratory requisitions to the laboratories every day. The research setting also includes the local vendor Well Diagnostics, a company that develops software supporting cooperation in the health care sector.
The study is based on an interpretative research tradition (Walsham 1995; Klein and Myers 1999), where reality is socially constructed among the participants. Our study is largely set in an ethnographic tradition, which is a useful method to gather an in-depth understanding of the people, the organization, and the broader context within their work (Klein and Myers 1999; Forsythe 1999; Harper 2000).
The empirical observations were conducted at several sites by the second author. In the summer of 2006 he spent two months as a summer employee at Well Diagnostics, making it possible to gather valuable background information about the product Well Interactor. The observations at the laboratories and in the GP practices were made from January 2006 to October 2009. At the laboratories, he observed the work of receiving requisitions as well as the procedures related to assessing and preparing samples for analysis.
During the observations, the second author was able to ask detailed questions about what was going on and to make appointments for interviews. The latter were particularly important to make it possible to talk to physicians who were otherwise difficult to get hold of. He always took field notes during the observations, acknowledging Eisenhardt’s (1989) point that that ‘one key to useful field notes is to write down whatever impressions occur’ (ibid, p. 539). Afterwards he transcribed the notes, adding questions and ideas to the transcriptions. The different contexts and the extent of the participant observation are outlined in Table 1.
Table 1
Observations of pre-analytic work related to the three laboratories.
Observation of work
Hours of observation
Well Diagnostics
300 (Summer job)
Medical Biochemistry laboratory
160
Microbiology laboratory
125
Pathology laboratory
105
General practitioners
5
In total, 32 tape-recorded semi-structured and unstructured interviews were conducted with various personnel: GPs, laboratory technologists, assistants, medical secretaries, managers, executive personnel, physicians and systems developers. The second author conducted 23 interviews, the first author conducted five, and the authors conducted a further four interviews together. On average, all tape-recorded interviews lasted from 30 to 60 min. The tape recordings were primarily transcribed shortly after the interviews.
The interviews were open-ended, and no detailed or well-defined interview guide was used. Instead, the authors prepared themselves with an indexed list of key themes, which initially were fairly broad. One such theme was to obtain insight into the work situations observed when electronic requisitions were received. Another theme was to identify challenges experienced by the users in the particular project. However, based on the data from the interviews and the observations, we narrowed down our research themes. For example, from the outset when we focused on how the Medical Biochemistry laboratory infrastructure was integrated with the systems in the GP practices, we increasingly realized that the interplay between the three different laboratories was a key condition for a well-functioning workflow. We changed our data collection strategy accordingly. This implied that after our initial data collection at the Medical Biochemistry laboratory, we started to collect data more systematically at the Microbiology laboratory, and lastly at the Pathology laboratory. In this process, we had to identify new informants in the other two laboratories. Although it was a cumbersome process to cover the three largest laboratories in the hospital, we found it relatively easy to establish new contacts. We were able to utilize the network of laboratory users (Latour 1987) who were involved in the overall integration effort in various ways.
In addition to the interviews and the observations of work, both authors have participated in several project meetings as well as having access to a variety of project documents. The data collection process was supplemented with conversation, telephone calls and exchange of e-mails with various hospital personnel, which both authors undertook over several years.
The collected data were thoroughly discussed between the authors, in terms of both data collection strategy and potential analytical points. As authors, we realized very early on that the setting was complex, especially since many different laboratories were involved. We have therefore tried to convey some key characteristics and particularities from each of the contexts, and also emphasized how the users in these contexts interpreted the integration effort differently (Klein and Myers 1999; Forsythe 1999). In the light of this, we have strived to provide relatively detailed insight, so-called “thick description” (Walsham 1995), to convey our findings and the different perspectives to a broader audience.
We are fully aware of the criticism that ethnographic studies do not allow extrapolation of specific findings to a broader perspective (Klein and Myers 1999). Still, we do not agree that we cannot generalize from ethnographic studies. According to Walsham (1995), interpretive research can be generalized in four ways: the development of concepts, the generation of theory, the drawing of specific implications and the contribution of rich insight. The interpretive study might support the design process of information systems, as it may tease out some specific implications (Walsham 1995). Our study reflects aspects of these concepts, as it contributes to a richer insight both into the integration of complex infrastructures and into some specific implications.

4 GiLab—the electronic laboratory requisition project

For several years, the University Hospital of North Norway (UNN) had planned to improve the quality and efficiency of its pre-analytic laboratory services. Investigations completed in 2002 (Haaheim et al. 2002) had indicated that there were problems related to logistics, resources, quality and existing infrastructures. Many of the existing work procedures were manual, redundant, and resource demanding. The fact that the incoming requisitions were paper-based was considered a major cause of the problem.
In addition, the various laboratories mainly operated independently of each other. They had different ways of receiving and handling samples from the 220 GP practices in the northern region of Norway. This was clearly reflected in the way that each of the laboratories had different laboratory systems supplied by different vendors. Medical Biochemistry had used its laboratory system DIPS Lab for five years, and the system handled approximately 80% of the hospital’s laboratory production.
When DIPS Lab was implemented five years ago, the management at UNN had strongly encouraged the Microbiology laboratory to use DIPS Lab as well, to reduce the number of laboratory systems. The laboratory had categorically rejected this, and instead acquired the system SafirLIS Deltrix (SAFIR). Together with the vendor, they had invested considerable resources in developing the system to conform to the needs of the laboratory. Finally, the Pathology laboratory had been using the system SymPathy since 1997. The use of different systems meant that the content of the same requisition form had to be entered in the laboratory systems more than once if sample material was addressed to more than one laboratory. In 2002, this work was estimated to total approximately 1350 h a year (Haaheim et al. 2002).
A measure for dealing with the problems outlined above was to establish electronic laboratory requisitions from the GP practices. In 2006, UNN and Well Diagnostics established a project named GiLab. The primary goal was to develop a new system, named Well Interactor, which gave GPs the opportunity to request laboratory services electronically. The GPs used an EPR supplied by the vendor Profdoc, which had approximately 80% of the market in the primary care sector. The integrated portfolio was also supposed to ensure that requisitions could easily be tracked down anywhere in the workflow and was supposed to find missing samples, indicate which laboratories were involved, and show the status of the analyses for any given requisition.
During the first month of the system development phase, the project management team at UNN invited local GPs, physicians from UNN, laboratory technologists, and the vendor to participate and exchange opinions in workshops and meetings. The different actors regarded this process as a good initiative to ensure that users’ opinions were taken into account. Generally, the participants were enthusiastic about the potential benefits of the new system. In particular, it was pointed out that the software had to be easy to use for the GPs, as they were the primary users of the requisition procedures. This meant that whenever the GPs wanted to make a request, they should be able to activate Well Interactor directly from their existing EPR.
Rather than using various types of paper requisitions for ordering laboratory services, the vendor designed the system in a way that allowed all service providers (the laboratories) to store their analysis repertoire in an electronic module called service provider profiles (SPPs). It was then each hospital’s own responsibility to produce the list of all the analyses offered to the GPs. This design strategy enabled hospital staff to update the services whenever they needed to, for instance if they needed to add or delete analyses. The module was published on the network, enabling the GPs to download changes, and to keep up to date about the services provided. For integration purposes, the vendor used the new national standard KITH XML defined by KITH, the Norwegian Centre for Informatics in Health and Social Care, which is responsible for development of IT standards for healthcare in Norway.
The initiative involving electronic requisitions also created the opportunity to automate the process of preparing the sample tubes received for analysis. Therefore, the project purchased an automatic distribution robot that could take over much of the manual preparation work.
The implementation time schedule varied across the various laboratories. Since DIPS Lab accounted for the largest laboratory volume, the project management at UNN had decided to start with the Medical Biochemistry laboratory and expected to start piloting here in September 2006. The implementation of the Microbiology laboratory and the Pathology laboratory was scheduled for the beginning of 2007.
On time, the Medical Biochemistry laboratory received its first e-requisitions in September 2006. Some months later, the number had increased to approximately 3000 requisitions from four GP practices. During the first six months of 2009, the hospital had received 14 200 e-requisitions, now accounting for 30% of all external requisitions received by the laboratory. Twelve GP practices with 54 GPs had the ability to transfer e-requisitions. The project team concluded that the e-requisitions had resulted in major quality improvements so far, because the number of errors made in filling in requisition forms had dramatically decreased.
At the same time, Well Diagnostics had managed to promote and sell Well Interactor to nine other hospitals in Norway (Johannesen and Ellingsen 2009). The largest of these was Ahus, a large hospital in the south that bought Well Interactor in June 2006.
Still, the implementation of Well Interactor in the northern region did not progress as fast as expected. Both the Microbiology and the Pathology laboratories at UNN were affected by major delays. The project was formally closed at the end of 2007, but the integration activities continued in the laboratories. While the Microbiology laboratory received its first e-requisitions in November 2008, only one GP practice with six GPs has so far been included. And the Pathology laboratory could not receive any e-requisitions, due to integration problems.
The process was slowed down even more in early 2009, because the Regional Health Authority wanted to call for tenders for electronic requisitions that encompassed all the 11 hospitals in the northern health region. Since the other 10 smaller hospitals had not taken part in the actual development, international European Union regulations applied, and an international invitation to tender had to be issued. While positive to the product, the authority has put the expansion on hold until the decision is made during 2010. In this situation, the laboratories have been allowed to continue further deployment of the product on a limited scale only.

5 Three different laboratories

In this section, we focus more specifically on each of the three laboratories at UNN and in particular we outline the differences between the laboratories. While all of the laboratories aimed for a technical integration through the use of Well Interactor, they had completely different ambitions with the integrated portfolio. In various ways, they saw the potential for fulfilling ambitions which otherwise would be impossible. Hence, we describe how these ambitions entailed different meanings of integration and how the technical integration was just the first step in ensuring an integrated information infrastructure. The Medical Biochemistry laboratory worked hard to achieve full automation of their laboratory infrastructure through a combined use of Well Interactor and other medical equipment. The Microbiology laboratory worked according to a completely different pattern, aiming for greater control of the laboratory’s resources, and saw integration as a means to this end. And lastly, as the Pathology laboratory was highly dependent on many paper-based routines in the laboratory work, its management regarded Well Interactor as a facilitator to introduce an electronic workflow in the laboratory.

5.1 The Medical Biochemistry laboratory—an automated process

The Medical Biochemistry laboratory conducted nearly 2 million analyses a year. Hence, both for GPs and for the staff at this laboratory, it was extremely important that ordering, analysing and dispatching the medical biochemistry results took place as smoothly and efficiently as possible. The actual analyses were fairly simple and clear cut: the GP ordered a given analysis and the laboratory provided the result. The sample materials were usually blood (99%), serum, and plasma, and were primarily analysed by one of the laboratory’s 30 analysis machines. These machines were integrated with the laboratory system DIPS Lab through several locally developed interface programs. The paper-based forms received were scanned using optical character recognition technology. Consequently, the laboratory was already highly automated.
The project team experienced that it was not straightforward to integrate Well Interactor with DIPS Lab, due to incompatible interfaces between the systems. Well Interactor was developed using the new national standard KITH XML, while DIPS Lab used another standard. DIPS ASA was adapting its software to KITH XML, but according to the IT staff at the hospital, this process had been delayed. In the meantime, an alternative strategy had to be found, and a couple of the superusers at the laboratory developed a type of ad hoc gateway application. The application converted the XML-based requisitions in Well Interactor to the format used by DIPS Lab, enabling electronic input of the requisitions in DIPS Lab.
Integration with the EPR in the GP practices also turned out to be more complicated than initially expected. The existing message standards for electronic requisitions did not fully cover the requirements for the project. Therefore, Well Diagnostics and Profdoc had to develop an API (application programming interface) that enabled the systems to communicate. Well Interactor facilitated these negotiations by offering Profdoc a stake in Well Interactor. This meant that for each hospital that bought the product, Profdoc earned some money. Later, after taking Well Interactor into use, the GPs requested Profdoc to make changes to the GPs’ EPR as well, allowing the Interactor module screen to appear as part of the regular referral window in the EPR.
The project team realized that constructing the analysis repertoire (SPP) was not straightforward either, since each GP had his or her own opinions about what the profiles should look like. On an overall level, it had been challenging for the hospital to create a layout that all of the GPs could agree on. Some GPs argued that the list should look like an exact copy of the paper-based requisition, as they had a mental picture of it. Others complained that the list was not sufficiently intuitive and that it was too long. For instance, choosing and navigating analyses from a huge list was seen as challenge. One of the GPs complained:
“Now we have got something with a lot of possibilities, but it takes longer than it should (…) we rather need something simple and easy. Most of the time we know exactly what we want to order, and then it is preferable to just push a button to accomplish the ordering process” (GP)
However, the major bottleneck in receiving sample tubes from the GPs had still not been dealt with. When the sample tubes were received, they had to be “prepared” manually for the analyses. Approximately 80–90% of the samples in the (primary) tubes received from the GPs had to be distributed into one or several new (secondary) tubes tailored to the analysis machines. Together with this, a new barcode printed out from DIPS Lab was glued to the secondary tubes, a routine normally referred to as relabelling. Upon reading the barcode, the analysis machines could access the interface programs, requesting instructions for analysis of the samples. The work involved in this process was quite cumbersome, as one manager explained:
“There has been a lot of sick leave due to strain on the neck and wrists. It is very monotonous work to distribute as many as 500 samples a day”
As the project management team saw it, receiving requisitions electronically through Well Interactor created the opportunity to eliminate this cumbersome process. Hence, with the support of the hospital’s top-level leadership, the project management decided to acquire a robot that could automatically distribute and relabel the sample tubes. A prerequisite for the implementation of a robot was that the laboratory received sample tubes from the GP practices that were already marked with a barcode (including a laboratory number). This condition was fulfilled with the use of Well Interactor. Accordingly, when these samples were received at the laboratory, the robot could simply read the barcodes and automatically distribute and relabel the sample tubes. After an extensive investigation process, the hospital acquired and installed the RSD 800A from Roche in the spring of 2007. The robot had a capacity of 800 distributions per hour. It could automatically cut the cap of the primary tubes, centrifuge the material and distribute it into secondary tubes, and subsequently put on new caps. It could also scan the barcodes on the primary tubes and glue an identical barcode on the secondary tubes.
However, integrating the robot with DIPS Lab proved more difficult than expected, as this was dependent on using a unique laboratory number throughout the whole workflow from the GPs to the laboratory. The reason for this was that the robot could not translate between external and internal laboratory numbers. It could only “copy” the barcode (and thus the laboratory number) from the incoming tubes onto the secondary tubes. To solve this, at an early stage of the project the hospital had requested DIPS ASA to develop a service in DIPS Lab that could export DIPS laboratory numbers to the GP practices. Unfortunately, DIPS ASA was delayed with this development due to other pressing matters.
In this situation, the superusers in the laboratory increasingly argued that relying on the robot would create more dependency on the GP practices and reduce the flexibility in some of their work routines. For instance, under certain circumstances the laboratory numbers generated in-house needed to be supplemented with checksums and special instructions to the analysis machines. This could not be performed as long as the barcodes were all set in the GP practices. Consequently, despite electronic receipt of requisitions, the manual distribution and relabelling process continued as before. The distribution robot was never put into action.
In sum, the major concern for the Medical Biochemistry laboratory was to ensure that Well Interactor was integrated seamlessly with the existing large-scale portfolio of automated analysis machines and the DIPS laboratory system. As the laboratory had a very high throughput of analyses, it was considered crucial that Well Interactor could contribute to improved efficiency. The combined use of Well Interactor and the distribution robot was regarded as a key factor in this regard. Unfortunately, this was not realized due to the difficulty of establishing a unique laboratory number throughout the whole workflow.

5.2 The Microbiology laboratory—controlling resources

While the Medical Biochemistry laboratory was pursuing increased automation of the work processes, the Microbiology laboratory saw another potential for electronic requisitions. As the analyses performed at the laboratory were fairly resource demanding and expensive, Well Interactor could be used for resource control. By imposing several conditions on the electronic ordering process, the laboratory could ensure the relevance of the requested analyses. In this regard it was extremely important that the requesting GPs had provided additional information about clinical information, material (synovial fluid, urine, plasma, etc.) and location in the body. Based on the clinical information, the microbiologist could decide how to proceed, for instance by rejecting the requisition when there was no justification for the test, or by adding analyses not initially requested by the GP. However, a lack of this information was the rule rather than the exception.
The laboratory’s effort to reduce the number of irrelevant analyses had so far been fairly successful. The staff regularly informed the GPs through circular letters what kind of information was needed as well as underscoring what conditions had to be fulfilled before a requisition could be processed. In this way, the laboratory’s analysis volume had been reduced for the third consecutive year. To continue this effort, the laboratory considered Well Interactor an important tool because the conditions could be implemented in the system as input validation. Several options were discussed. The system could check the validity of the combination of provided material and requested analyses. If a malaria investigation was requested, the system might ask when and where the patient had been travelling. If a Clostridium analysis was ordered, the laboratory needed to know whether the patient was on antibiotics. If not, the requisition should be rejected on a routine basis. The laboratory also consulted the vendor about the possibility of presenting GPs with the costs of particularly expensive and resource-demanding analyses.
The presence of sufficient clinical information would also enable the microbiologists to better assess the requisition and potentially add new analyses. The reason for this was that often it was not obvious to the GPs what kinds of analyses were needed. Instead, the GP presented a problem that in turn would indicate a variety of microbiological analyses. In fact, as many as half of the virology analyses were ordered by the microbiologists themselves. For instance, if the GP wrote ‘hepatitis?’ in the requisition, a variety of analyses had to be performed to cover all the relevant possibilities. In addition, if the microbiologists discovered something in the process, further supplementary analyses were often requested. Consequently, it was extremely important that the GPs had added clinical information in order to narrow down an investigation strategy. Consider the following quote:
“If the GP sends a sample and only requests a HIV test, then we do a HIV test. However if the GP also informs us that the patient has had fever and swollen lymphatic nodes for a week, then we can add 3–4 other possibilities which may also be causing the problem.” (Chief physician, Microbiology).
However, the potential to use Well Interactor in this way has so far not been realized. The only implemented requirement is that the GP has to fill in the field for clinical information. To define fairly strict rules was not seen as an easy task, because “it is not possible to define 100% how things should be”, as one of the laboratory technologists put it. In addition, the idea of highlighting the costs of expensive analyses was dismissed, as the GPs were not interested in this kind of control. Despite these challenges, the laboratory regards placing conditions on the requisition procedure as part of their ongoing strategy.
In contrast to the Medical Biochemistry laboratory, the Microbiology laboratory had no problems in using laboratory numbers generated in the GP practices. Exporting laboratory numbers to the GP practices was in line with the successful integration that had already been achieved between SAFIR and the in-house blood-bank system. SAFIR had exported series of laboratory numbers to this system, and was now receiving a high volume of electronic requisitions to analyse blood from blood donors. These blood samples were brought up to the Microbiology laboratory and could be input into the analysis process without any relabelling of barcodes.
Still, the use of these laboratory numbers brought its own problems. The way the laboratory numbers were generated in SAFIR meant that within a few years the sequence would be repeated. According to the chief laboratory technologist, this was “a serious error that urgently needed to be dealt with”. Expansion of the system implementation was put on hold while the laboratory requested the vendor to add a “date” field together with the laboratory number (to ensure uniqueness) before more GP practices were included. As a result, the diffusion process was halted, pending further development from the vendor of SAFIR.
Moreover, the design of the SPPs (analysis lists) was far from straightforward; according to one of the GPs, it was ‘far more difficult than the Medical Biochemistry SPP’. In addition, the laboratory had organized the list according to the laboratory’s structure (sections, desks, etc.), a structure that did not fit the GPs’ work practice. For instance, in the list of respiratory diseases bacteriology had been separated from virology, forcing the GPs to fill in clinical information in several places:
“Whenever we are confronted with respiratory diseases, for instance nasopharynx, we want to check whether there are bacteria and viruses in the same sample. Then we just want to check it off and accomplish it in one operation” (GP).
The struggle to reach agreement had resulted in much discussion between the laboratory and the GPs to find a compromise on the different needs:
“We have discovered that the way we have structured the SPPs is far from optimal. While seemingly good in theory, it appeared to be a bottleneck for the GPs, as it was resource-demanding to search for the analyses.” (Laboratory technologist, Microbiology laboratory)
Overall, the Microbiology laboratory regarded Well Interactor (and thus integration) as a way to limit unnecessary access to the laboratory’s scarce and expensive resources. By imposing several conditions on the electronic ordering process, the laboratory could ensure that the GPs provided sufficient clinical information. However, the laboratory has a way to go to achieve this. In addition, the design of the analysis repertoire revealed completely different perspectives and needs in the laboratory and in the GPs’ work practice. This entailed extensive negotiations and compromises between the stakeholders.

5.3 The Pathology laboratory—a paper-based workflow

The Pathology laboratory received approximately 25,000 histology and 28,000 cytology requisitions a year. Cytology samples were normally transported on standardized microscopy plates, while histology sample materials included everything from small parts of the skin to larger organs, thus requiring different types of packaging.
The Pathology laboratory was highly reliant on manual and paper-based routines. A motivation for taking part in the project was the wish to initiate a full-scale transformation from the laboratory’s paper-based work processes to electronic ones. As a part of the process, the laboratory had started to plan an application for funding to the Regional Health Authority in order to initiate a major modernization process for the laboratory.
The project had planned to integrate the laboratory system SymPathy with Well Interactor in early 2007 (nearly three years ago), but it faced several setbacks and delays. The vendor of SymPathy, TietoEnator, had started to adapt some of its software and in this process was engaged in discussions with the laboratory. When the integration was almost complete (still in 2007), the laboratory needed assistance from the IT department to provide infrastructural support and testing. Unfortunately, at this stage the IT department could not provide such assistance, due to other pressing demands. When the IT department could finally offer its support, TietoEnator had reallocated its development resources elsewhere. “This is in a sense our daily reality” as the superuser at the laboratory expressed it. Later on, the new national XML standard for data exchange from KITH was completed, and TietoEnator decided to postpone the delivery of the integration functionality until it could comply with this standard. This was scheduled to take place at the turn of the year 2009/2010.
We will now elaborate on the current workflow in the laboratory for histology samples with the purpose of highlighting some of the complexities the project will face in interconnecting two completely different infrastructures: the state-of-the art Well Interactor and the more “old-fashioned” SymPathy.
Every morning, the assistants from the Pathology laboratory opened and sorted the parcels sent from the GPs. They counted the number of samples bundled with each paper requisition and wrote the number on the paper requisition. In this way, it was clear how many barcode labels would be needed during the registration process. If there was a risk of leakage from the material, an X was written on the requisition to alert the staff of this. After the samples and the paper requisitions had been unpacked, the information on the requisition had to be entered in the SymPathy software. In this process, various patient details (names, addresses, general practitioner ID, number of sample tubes, etc.) were entered manually one by one. For each registered requisition, the secretaries printed out the necessary number of SymPathy barcodes, which were glued to the original sample containers and the original paper requisition. A handheld barcode reader could then be used to scan the barcode label on the paper requisition to provide instant access to the SymPathy record, which now served as a copy of the original paper requisition. The barcode contained a SymPathy laboratory number composed of the current year, the sample type, and a random number.
After 20–25 requisitions had been registered, the pile of paper requisitions was bulk-scanned into SymPathy, since the paper requisitions contained handwritten text and drawings that illustrated the location on the body from which the sample material had been extracted. As mistakes sometimes occurred during the scanning of multiple requisitions in a single process, the staff had to check that the information had been correctly linked to the original requisition. When this had been done, the paper requisitions and the sample material were transferred to another location of the laboratory, called the macro room.
In the macro room, the laboratory technologists started out with the pile of paper requisitions. They used a handheld scanner to read the barcode on each paper requisition to access the associated SymPathy record. For each of the requisitions, the laboratory technologists prepared several 3 × 4 × 1 cm plastic boxes, all labelled with the correct barcode number, thus connecting the boxes with the correct requisition. After this, the plastic boxes and the original sample materials were classified on the basis of the sample materials and then investigated by the pathologists. The samples were examined in sequence. The staff identified and “cut out” parts they suspected to include pathology, put these into the various plastic boxes and encased them in liquid myelin. Information about the number of plastic boxes and which samples had been placed on various blocks was handwritten on the paper requisitions. In addition, drawings were frequently added to the paper requisitions:
“We make drawings to show where the cross sections are cut and how we handle and colour the samples before we cut in it (…) this is very important.” (Pathologist)
During the process, a description of the material‘s type, size and shape was dictated into a dictation machine. The pile of paper requisitions and tapes was then brought back to the secretaries, who scanned the drawings and entered the dictated information into SymPathy. However, as the actual workflow required the use of papers, the secretaries had to print the transcripts and any new drawings, and put this new paper sheet together with the original paper requisition. In the meantime, the sample material was put through a dehydrating process in the laboratory.
In the subsequent steps of the analysis process, the laboratory technologists prepared the biopsies encased in paraffin for microscopy analysis. This included slicing very thin parts of the material and putting it on a microscopy plate, followed by a process to stain the material. Here, too, the pile of paper requisitions was used to organize the work and to access the associated SymPathy record in due order. Later, after the analysis process had been completed and the results had been given, the final task was to store the microscopy plate according to the specific SymPathy laboratory number. It was stored for at least 10 years.
A key point in this section has been to illustrate the challenges of integrating a “modern” (Well Interactor) and an “old” (SymPathy) infrastructure. A purely technical integration between these two systems will not result in significant organizational benefits for the laboratory, but will need to be combined with a systematic adaptation of the SymPathy system as well as the current work practice. The Pathology laboratory realized this and considered Well Interactor a stepping stone for initiating a full transition from paper-based work processes to electronic ones.

6 Discussion

In this section, we discuss four themes related to the integration of health-based information infrastructures. First, we discuss integration as a collective and distributed activity, emphasizing the role of the users. Second, we discuss how the installed base prohibited a common integration strategy and how this was dealt with in the laboratories involved. Third, we focus on the potential for inter-organizational redesign and associated strategies for managing this. Lastly, we discuss how information sent between practices required interpretation and translation to be rendered useful.

6.1 Integration as a collective achievement

Drawing on ANT (Latour 1987; 2005), we spell out the integration effort as a complex socio-technical network involving many heterogeneous actors, who had to “play along” in concert in order to fulfil the envisioned integrations. As each of them could effectively terminate or substantially delay the project, this called for a cautious approach. We analyse how the participants dealt with the coordination challenges, not in a centralized way, but rather through a carefully distributed activity consisting of negotiations and improvisation. Over time, many of the different user groups increasingly undertook an essential role, as heterogeneous engineers (Law 1987) in this process.
In this study, the project management team at UNN, five different vendors, users in three laboratories and many users in the GP practice needed to coordinate their activities. The national centre KITH and the Regional Health Authority also influenced the project. This made centralized coordination nearly impossible. A general problem related to the vendors’ involvement was that each of them targeted their efforts towards different market segments. Hence, it was difficult to mobilize (Callon 1986) them into projects representing other vendors’ market domains. Well Diagnostics experienced this with regard to Profdoc, whose user base consisted solely of GPs. Well Diagnostics knew that unless Profdoc’s ordinary GP users asked for the functionality, it would probably not be given priority. And initially, there were no users. However, the issue was settled through direct negotiations between the two, as Well Diagnostics offered Profdoc a stake in Well Interactor.
The lack of developed standards for electronic requisitions also threatened the integration, thus making KITH a gatekeeper in the process (Latour 1987). While this halted progress in the Pathology laboratory, the other actors found ways to deal with this on an ad hoc basis. A typical example is how Well Diagnostics negotiated with Profdoc on how to exchange microbiology requisitions instead of waiting for the formal standard. In addition, competent users contributed at crucial moments. As DIPS Lab lacked the new KITH XML standard, some of the superusers in the laboratory improvised (Orlikowski 1996) and developed a piece of software, ensuring that the received requisitions were imported into DIPS Lab.
In fact, the various user groups became increasingly important in coordinating the different activities. A telling illustration is the Pathology laboratory’s efforts to coordinate TietoEnator, Well Diagnostics, the hospital’s IT department and itself into a concerted integration test activity. The users’ role should not really come as a surprise, since the users were the parties capable of putting pressure on the vendors, particularly when the integration became embedded in practice. For instance, it was the Microbiology laboratory that put pressure on the vendor of the SAFIR system to fix the problem related to the problem of regenerated laboratory numbers. The users’ increasing influence is perhaps best expressed by the way the GPs appreciated Well Interactor so much that they forced Profdoc to make changes to their EPRs beyond what had initially been agreed on.
Increasingly, Well Diagnostics realized the advantage of letting the laboratory users take care of direct negotiations with the GPs, since there were many “voices” among the GPs as well as some fundamental differences between the laboratories. As a result, the vendor developed Well Interactor in a way that made it easy for the laboratory staff to define the form and the content of the laboratory services together with the GPs. This was also very useful for Well Diagnostics, as they could target their resources towards other parts of the project. Yet another positive effect of such a configurable technology was that the services could be modified throughout the design and project phase; it was possible to experiment with the service layout after the services were put into daily production. This was a crucial point, both because the GPs were far from satisfied with how the services were presented initially and because this was a process that continued after the project period ended.
The transition to live production implied even more responsibility for the laboratory users. Now Well Diagnostics had completed their development task and the project management at UNN was not operative anymore. Thus the laboratory staff was left with the sole responsibility for enrolling and supporting more GP practices. Hence, when the recent call for tenders for a new common regional requisition system halted further expansion, it was the laboratories that engaged in direct negotiations with the Regional Health Authority on how, and under which conditions, to proceed. Although there were strict EU regulations in play, the laboratories were allowed to continue distributing Well Interactor to a few more GP practices, pending the Health Authority’s overall decision.

6.2 The installed base shaping the integration

Several studies indicate how the installed base both constrains and facilitates action (Bowker and Star 1999; Hanseth and Lyytinen 2004). We discuss the influence of the installed base in the context of how the existing laboratory numbers were embedded in organizational issues in each of the laboratories. This prohibited a common integration strategy, necessary for tracking down requisitions across the different laboratory systems, hence representing a third-order issue (Star and Ruhleder 1996, p. 118). We discuss the content of this, and elaborate on how this paved the way for emerging gateway strategies, not technical ones as described by Hanseth and Lyytinen (2004), but rather practice-oriented, performed manually by the laboratory staff. In this way, highly invisible work (Gasser 1986; Grudin 1989; Pollock 2005) emerged as a condition for integration and seamlessness in the laboratory infrastructures.
A crucial point for each of the laboratories was compatibility with the existing portfolios, and they were therefore reluctant to make substantial changes to their laboratory numbers in order to adapt to the integration. The Microbiology laboratory’s strategy for integration with other systems was based on the use of a common laboratory number throughout the whole analysis process. The laboratory exported series of available SAFIR laboratory numbers to the surrounding systems. As this was an integration strategy that was well-established with respect to the blood bank, it was quite obvious that such a strategy should be used with respect to the GPs as well. Accordingly, if every laboratory followed the same strategy, it would be easy to search for requisitions independent of a specific laboratory system.
However, in terms of a common strategy for all the laboratories, this was problematic due to the different roles of the laboratory numbers. In the Pathology laboratory, the SymPathy laboratory number took on a particular meaning, as it provided information about the sample type, the current year, and long-term archival of the samples. In the Medical Biochemistry laboratory, the DIPS Lab laboratory numbers were used to integrate DIPS Lab with the portfolio of 30 analysis machines as well as the hospital’s electronic patient record.
Hence, to maintain compatibility with its existing portfolio, the Medical Biochemistry laboratory rejected an integration strategy where the laboratory numbers remained the same throughout the workflow. As they saw it, this would place considerable constraints on their practice. Instead, the laboratory’s staff preferred to continue manually relabelling the received sample tubes with in-house DIPS laboratory numbers because they needed flexibility when dealing with specific instructions to the analysis machines. For this reason, the distribution robot was never put into production, as it presupposed that the laboratory numbers generated in the GP practice should also be used throughout the laboratory’s production. However, in information infrastructure terms, the decision to continue relabelling meant maintaining a work-based gateway functionality that enabled mapping between the different infrastructural portfolios in the GP practice and in the Medical Biochemistry laboratory.
A similar situation will probably arise for the paper-based Pathology laboratory. While we cannot state precisely how the paper-based infrastructure might be connected to Well Interactor, we can indicate it fairly closely. So far, the suggestion has been to print out the electronic requisitions received, then scan them into the SymPathy system as a paper requisition and assign them a SymPathy laboratory number during the process. In this sense, the laboratory staff must fill in the gaps (Gasser 1986; Berg 1999) between two different infrastructures: the electronic Well Interactor and the paper-based infrastructure supporting the laboratory.
While the different laboratories may successfully integrate their portfolios with the EPR systems in the GP practice, the outcome is lack of integration with each other, as is now the case with DIPS Lab and SAFIR. As the goal of the project was easy tracking of requisitions across different laboratory systems, this was still an unsolved problem. However, by identifying the requisitions where the lack of integration emerged as a major problem, the laboratory’s staff managed to carve out a solution. An example of “problematic” requisitions was regular maternity clinic check-ups, which were analysed at several laboratories and which needed a coordinated follow-up. To overcome the challenge, the laboratory staff started to enter these requisitions into DIPS Lab when they were received in the laboratories, assigning them a DIPS laboratory number and then performing the requisitions for the other two laboratories from DIPS Lab. In practice, then, the initial registration process in DIPS Lab represented yet another gateway between DIPS Lab and the other laboratory systems.

6.3 Restructuring work along the integrated workflow

From a managerial point of view, the notion of integration often comes with high expectations of large-scale organizational effects (see Davenport 1993; Ashkenas et al. 2002), which may even be taken for granted. While agreeing on a great potential for integrated practices, we do not believe that the organizational effect of integration is simply a given. In contrast, we see technical integration as a basis for organizational re-design. However, such efforts need to be approached carefully and need to be anchored in or to emerge from the practices themselves in order to avoid failure. In addition, since the interconnected practices are often very different, it is crucial that there are benefits for each of them (Latour 2005; Berg 2001).
The effort to redesign the workflow was an essential part of this project. The project management, which had its mandate from the CEO level at UNN, saw the potential of relocating pre-analytic work related to the Biomedical Chemistry laboratory to the GP practice. The way to achieve this was to export series of DIPS laboratory numbers to the GP systems and let the staff in the GP practice glue proper DIPS barcodes on the sample tubes. When tubes were received in the Medical Biochemistry laboratory, the distribution robot from Roche was supposed to automatically redistribute the samples. Hence, we may easily see the project management’s acquisition of the distribution robot as part of a strategy to streamline a cumbersome distribution and relabelling process. Now we know that this strategy failed, but what seems rather surprising is that a similar redesign strategy proved more successful in the Microbiology laboratory. The laboratory managed fairly successfully to redistribute some of its pre-analytic work to the GP practice by exporting series of SAFIR laboratory numbers to the GP systems. But the laboratory staff had even higher ambitions: in accordance with the strategy they had used for many years, they saw the potential to impose a set of specific conditions on the GP in the ordering process to ensure that the GP did not waste the laboratories’ scarce resources. With such measures, they tried to impose long-distance control (Latour 1987) beyond their own local system. While the idea of presenting the cost was abandoned, the strategy of implementing input validation is still alive and well.
In sum, the planned change in the Medical Biochemistry laboratory failed, while the change proved more successful in the Microbiology laboratory. However, a crucial difference between the two initiatives was that the redesign planned for the Medical Biochemistry laboratory was initiated top-down by project management, while the redesign in the Microbiology laboratory was initiated from within the laboratory itself. The Microbiology laboratory had reduced its analysis volume for the third consecutive year, and saw Well Interactor as conforming to or as supporting these measures. While some of these measures have taken a long time to implement, they are far from being abandoned; instead, they have been placed on hold pending a well-functioning technology and distribution to more GP practices. The integrated portfolio may then serve as a foundation for a stepwise implementation of more conditions and input validation in the requisition process as these measures emerge from the laboratory’s primary needs.
The planned redesign of the Pathology laboratory was strongly influenced by its position as the “least developed” laboratory in terms of digitization. Accordingly, we should not be surprised that this laboratory’s major aim was to use the integration project as a stepping stone for a full-blown modernization of its facilities. The Pathology laboratory wanted a full-scale transformation of its paper-based work processes into electronic ones. It is premature to say whether such reorganization will also involve the GPs. However, strategies similar to those of the other two laboratories may easily be the result.

6.4 Translation and interpretation between the practices

One of the ideas behind an integrated infrastructure is that it offers easier access to information across different systems supporting different contexts. For GPs, this may amount to requiring the status of the requisition at different stages of the analysis process. For some processes, such as medical biochemistry analysis, this is fairly straightforward. However, if we expand our perspective to include more complex practices, such as those of the Microbiology and the Pathology laboratories, a messier picture emerges. Here, information needs to be translated and interpreted between the different practices in order to be rendered useful (Latour 1987; Winthereik and Vikkelsø 2005). This not only makes “free sharing” of information illusory, but also illustrates how users need to engage with and work with the data to prepare for the next step in the workflow.
The Microbiological requisition is a striking illustration in this regard. When received in the laboratory, the requisitions were thoroughly assessed by the staff. Based on these assessments, more specific analyses or analysis packages might be ordered, each of which ended up in a result. In addition, the accompanying clinical information would possibly indicate to the microbiologists that the analysis process should take a completely different direction:
“Sometimes I know that what the GPs are requesting is completely meaningless given the situation [clinical information] that is described by the GP.” (Physician, Microbiology laboratory)
Hence, the original problem stated by the GP may have translated into something that is not immediately understood outside the laboratory. At least, if a GP requested information about the status, a microbiologist would be required to provide a detailed explanation. Basically, this is exactly what the microbiologists do for many microbiology results. The microbiologists often contacted the GP to explain what the results really indicated (possibly something different from what the GP had considered when ordering the tests), how to interpret the results, and what the implications were for the treatment of the patient.
We provide interpretation on how to understand the result and how it should be used (…) we may say that ‘these findings mean this’, or we can say ‘we don’t know exactly what this means, but we recommend some further examinations’ or that one has to wait for a month, take a new test and compare. In other instances, we go very far in instructing the physicians in how to treat the patient to prevent them from doing anything foolish” (Chief physician, Medical Biochemistry laboratory)
The complexity associated with the Pathology laboratory is relatively similar to that of the Microbiology laboratory, but there is another issue related to the translation between the electronic and the paper-based requisition. Although it is possible to interconnect the pathology system SymPathy with Well Interactor, the system is still dependent on a paper-based workflow inside the Pathology laboratory. Here, the paper-based requisition accumulates a great deal of information. This information is annotated by hand in the laboratory’s macro room, either as text or as drawings, and plays an important role in the laboratory’s workflow. Hence, when SymPathy in integrated into the shared infrastructure, the information contained in the paper-based requisition will not immediately be accessible to the GPs.

7 Conclusion

In this study we have aimed to provide insight into the complexity of establishing sustainable integration between different information infrastructures. The formal project period has officially ended and in terms of a project perspective, the outcome has been only partly successful. Not all of the integrations, developments and expansions have been completed, and the Pathology laboratory is still on hold in terms of incorporation in the electronic infrastructure. However, from the perspective of the users’ practice and their continuous initiatives, the outcome looks far more interesting and promising. Well Interactor has now gained a foothold in the laboratory practices and the integrated infrastructure is evolving, not as a part of a project, but with a strong basis in the laboratory’s daily work, allowing for a careful and stepwise approach. An example is the newly started process of entering details of maternity clinic check-ups in DIPS Lab in order to provide an overview of the requisitions.
To support such a process, we believe that it is of utmost importance that the technology put into use in integrated portfolios has a certain level of flexibility, allowing not only for different use associated with one specific system, but for experimentation related to the integrated systems. As we have discussed, such capabilities were built into Well Interactor. This made it possible for each of the laboratories to negotiate their services, not only in the system’s design phase, but also after it was put into production.
We have presented a case with many actors, indicating the need for a great deal of coordination among them. The project management undertook some of this, but it was definitely not possible to have general control over all the actors, since the organizations involved were varied and autonomous, with different priorities. Instead, the coordination occurred in a more distributed way: vendors to vendors, laboratory staff to GPs, users to vendors, and so on. This coordination process is continuing even after the completion of the project, and is now increasingly driven forward by the users. We believe that this illustrates the limitations of a centralized approach to coordination. In addition, the process must take the autonomy of the different participants into account: it is important that the benefits for each participant are obvious in order to motivate them to take part in integration efforts.
A key issue in large-scale integrations is how well the different vendors collaborate in order to establish the envisioned integrations. After all, the vendors know the technology which is the basis for the integrations. However, the collaboration between the vendors is not automatically given. It may be given higher or lower priority or even completely ignored. A key issue is that the vendors are reluctant to participate unless there are benefits for them as well, which are basically connected to their market segments. This means that, in essence, while the vendors control the technology, the users are the major force capable of influencing the vendors. Accordingly, when the user base in integration projects is increasing, the potential for the users to exercise influence on the vendors is also increasing. The implication of this is that it may be beneficial to channel resources (money, personnel, etc.) to the users and make sure to allocate sufficient time for their activities that are related to integration. In turn, the users may choose to channel some of these resources to the vendors for specific assignments.
Still, integration efforts imply a dilemma: the stability associated with the installed base versus the expected organization redesign. We have shown that the installed bases shape integration strategies in different ways, possibly impeding substantial changes to existing portfolios. While workarounds using various gateway strategies are possible, integration involves expectations of organizational effects and redesign. Hence there is an inherent tension between stability and change. We believe that the solution to this may be approached in several steps. Integrations through gateways should be regarded as opportunities or as an enabling information infrastructure on which an organizational redesign can build. However, as we have pointed out earlier, organizational redesign comes with substantial risk, making it crucial that it is initiated from within the implicated practice.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://​creativecommons.​org/​licenses/​by-nc/​2.​0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Literatur
Zurück zum Zitat Ashkenas, R., Ulrich, D., Jick, T., & Steve, K. (2002). The boundaryless organization. Breaking the chains of organizational structure. San Francisco: Jossey-Bass. Wiley. Ashkenas, R., Ulrich, D., Jick, T., & Steve, K. (2002). The boundaryless organization. Breaking the chains of organizational structure. San Francisco: Jossey-Bass. Wiley.
Zurück zum Zitat Auditor General (2008). Riksrevisjonens undersøkelse om IKT i sykehus og elektronisk samhandling i helsetjenesten [The Office of the Auditor General of Norway’s evaluation on ICT in hospitals and electronic interaction in the health service]. Document no. 3:7 (2007–2008), Riksrevisjonen, Oslo. Auditor General (2008). Riksrevisjonens undersøkelse om IKT i sykehus og elektronisk samhandling i helsetjenesten [The Office of the Auditor General of Norway’s evaluation on ICT in hospitals and electronic interaction in the health service]. Document no. 3:7 (2007–2008), Riksrevisjonen, Oslo.
Zurück zum Zitat Berg, M. (1999). Accumulating and coordinating: occasions for information technologies in medical work. Computer Supported Cooperative Work, 8(4), 373–401.CrossRef Berg, M. (1999). Accumulating and coordinating: occasions for information technologies in medical work. Computer Supported Cooperative Work, 8(4), 373–401.CrossRef
Zurück zum Zitat Berg, M. (2001). Implementing information systems in health care organizations: myths and challenges. International Journal of Medical Informatics, 64, 143–156.CrossRef Berg, M. (2001). Implementing information systems in health care organizations: myths and challenges. International Journal of Medical Informatics, 64, 143–156.CrossRef
Zurück zum Zitat Boudreau, M. C., & Robey, D. (2005). Enacting integrated information technology: a human agency perspective. Organization Science, 16(1), 3–18.CrossRef Boudreau, M. C., & Robey, D. (2005). Enacting integrated information technology: a human agency perspective. Organization Science, 16(1), 3–18.CrossRef
Zurück zum Zitat Bowker, G. C., & Star, S. L. (1999). Sorting things out: Classification and its consequences. Cambridge: The MIT. Bowker, G. C., & Star, S. L. (1999). Sorting things out: Classification and its consequences. Cambridge: The MIT.
Zurück zum Zitat Callon, M. (1986). Some elements of a sociology of translation: domestication of the scallops and the fishermen of St Brieuc Bay. In J. Law (Ed.), Power, action, and belief: A new sociology of knowledge? (pp. 196–223). Routledge: London. Callon, M. (1986). Some elements of a sociology of translation: domestication of the scallops and the fishermen of St Brieuc Bay. In J. Law (Ed.), Power, action, and belief: A new sociology of knowledge? (pp. 196–223). Routledge: London.
Zurück zum Zitat Chantler, C., Clarke, T., & Granger, R. (2006). Information technology in the English National Health Service. Journal of the American Medical Association, 296(18), 2255–2258.CrossRef Chantler, C., Clarke, T., & Granger, R. (2006). Information technology in the English National Health Service. Journal of the American Medical Association, 296(18), 2255–2258.CrossRef
Zurück zum Zitat Chiasson, M., Reddy, M., Kaplan, B., & Davidson, E. (2007). Expanding multi-disciplinary approaches to healthcare information technologies: what does information systems offer medical informatics? International Journal of Medical Informatics, 76, 89–97.CrossRef Chiasson, M., Reddy, M., Kaplan, B., & Davidson, E. (2007). Expanding multi-disciplinary approaches to healthcare information technologies: what does information systems offer medical informatics? International Journal of Medical Informatics, 76, 89–97.CrossRef
Zurück zum Zitat Cross, M. (2006). Will connecting for health deliver its promises? BMJ, 332, 599–601.CrossRef Cross, M. (2006). Will connecting for health deliver its promises? BMJ, 332, 599–601.CrossRef
Zurück zum Zitat Davenport, T. (1993). Process innovation: Reengineering work through information technology. Boston: Harvard Business School Press. Davenport, T. (1993). Process innovation: Reengineering work through information technology. Boston: Harvard Business School Press.
Zurück zum Zitat Eisenhardt, K. H. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.CrossRef Eisenhardt, K. H. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.CrossRef
Zurück zum Zitat Elbanna, A. R. (2007). Implementing an integrated system in a socially dis-integrated enterprise: a critical view of ERP enabled integration. Information Technology & People, 20(2), 121–139.CrossRef Elbanna, A. R. (2007). Implementing an integrated system in a socially dis-integrated enterprise: a critical view of ERP enabled integration. Information Technology & People, 20(2), 121–139.CrossRef
Zurück zum Zitat Ellingsen, G., & Monteiro, E. (2006). Seamless integration: standardisation across multiple local settings. Journal of Computer Supported Cooperative Work, 15(5–6), 443–466. Ellingsen, G., & Monteiro, E. (2006). Seamless integration: standardisation across multiple local settings. Journal of Computer Supported Cooperative Work, 15(5–6), 443–466.
Zurück zum Zitat Forsythe, D. E. (1999). It's just a matter of common sense: ethnography as invisible work. Journal of Computer Supported Cooperative Work, 8, 127–145.CrossRef Forsythe, D. E. (1999). It's just a matter of common sense: ethnography as invisible work. Journal of Computer Supported Cooperative Work, 8, 127–145.CrossRef
Zurück zum Zitat Gasser, L. (1986). The integration of computing and routine work. ACM Transactions on Office Information Systems, 4(3), 205–225.CrossRef Gasser, L. (1986). The integration of computing and routine work. ACM Transactions on Office Information Systems, 4(3), 205–225.CrossRef
Zurück zum Zitat Giuse, D. A., & Kuhn, K. A. (2003). Health information system challenges: the Heidelberg conference and the future. International Journal of Medical Informatics, 69, 105–114.CrossRef Giuse, D. A., & Kuhn, K. A. (2003). Health information system challenges: the Heidelberg conference and the future. International Journal of Medical Informatics, 69, 105–114.CrossRef
Zurück zum Zitat Grimson, J., Grimson, W., & Hasselbring, W. (2000). The SI challenge in health care. Communications of the ACM, 43, 48–55.CrossRef Grimson, J., Grimson, W., & Hasselbring, W. (2000). The SI challenge in health care. Communications of the ACM, 43, 48–55.CrossRef
Zurück zum Zitat Grudin, J. (1989). Why groupware applications fail: problems in design and evaluation. Office: Technology and People, 4(3), 245–264. Grudin, J. (1989). Why groupware applications fail: problems in design and evaluation. Office: Technology and People, 4(3), 245–264.
Zurück zum Zitat Haaheim, H., Halvorsen, Å., Steigen, S. E., Nygård, B., Johnsen, G., Lien, A.-K., Carlson, I. (2002). Laboratorieprosjektet: Prøver og prøveflyt. En MOR-Produksjon. [The laboratory project: samples and sample flows, internal project report]. University Hospital North-Norway, Tromsø, Norway. Haaheim, H., Halvorsen, Å., Steigen, S. E., Nygård, B., Johnsen, G., Lien, A.-K., Carlson, I. (2002). Laboratorieprosjektet: Prøver og prøveflyt. En MOR-Produksjon. [The laboratory project: samples and sample flows, internal project report]. University Hospital North-Norway, Tromsø, Norway.
Zurück zum Zitat Hanseth, O., & Lundberg, N. (2001). Designing work oriented infrastructures. Journal of Computer Supported Cooperative Work, 10(3–4), 347–372.CrossRef Hanseth, O., & Lundberg, N. (2001). Designing work oriented infrastructures. Journal of Computer Supported Cooperative Work, 10(3–4), 347–372.CrossRef
Zurück zum Zitat Hanseth, O., & Lyytinen, K. (2004). Theorizing about the design of information infrastructures: design kernel theories and principles. Sprouts: Working Papers on Information Environments, Systems and Organizations, 4, 207–241. Hanseth, O., & Lyytinen, K. (2004). Theorizing about the design of information infrastructures: design kernel theories and principles. Sprouts: Working Papers on Information Environments, Systems and Organizations, 4, 207–241.
Zurück zum Zitat Harper, R. H. R. (2000). The organisation in ethnography: a discussion of ethnographic fieldwork programs in CSCW. Journal of Computer Supported Cooperative Work, 9, 563–581.CrossRef Harper, R. H. R. (2000). The organisation in ethnography: a discussion of ethnographic fieldwork programs in CSCW. Journal of Computer Supported Cooperative Work, 9, 563–581.CrossRef
Zurück zum Zitat Johannesen, L. K., & Ellingsen, G. (2009). Integration and generification—agile software development in the healthcare market. Journal of Computer Supported Cooperative Work, 18, 607–634.CrossRef Johannesen, L. K., & Ellingsen, G. (2009). Integration and generification—agile software development in the healthcare market. Journal of Computer Supported Cooperative Work, 18, 607–634.CrossRef
Zurück zum Zitat Jones, T., Dobrev, A., Stroetmann, K. A., Cameron, J., & Morris, L. (2008). Report on the socio-economic impact of NHS Scotland’s emergency care summary- final draft. European Commission Information Society and Media, 10, 1–53. Jones, T., Dobrev, A., Stroetmann, K. A., Cameron, J., & Morris, L. (2008). Report on the socio-economic impact of NHS Scotland’s emergency care summary- final draft. European Commission Information Society and Media, 10, 1–53.
Zurück zum Zitat Kalra, D. (2006). Electronic health record standards. In R. Haux & C. Kulikowski (Eds.), IMIA yearbook of medical informatics 2006 (pp. 136–144). Stuttgart: International Medical Informatics Association and Schattauer. Kalra, D. (2006). Electronic health record standards. In R. Haux & C. Kulikowski (Eds.), IMIA yearbook of medical informatics 2006 (pp. 136–144). Stuttgart: International Medical Informatics Association and Schattauer.
Zurück zum Zitat KITH (2004). Elektronisk resept. Informasjonsmodell og meldingsbeskrivelse XML. Versjon 0.9. [Electronic prescriptions. Information model and messaging description XML]. KITH Rapport R12/04, pp. 1–33. KITH (2004). Elektronisk resept. Informasjonsmodell og meldingsbeskrivelse XML. Versjon 0.9. [Electronic prescriptions. Information model and messaging description XML]. KITH Rapport R12/04, pp. 1–33.
Zurück zum Zitat KITH (2006). Fyrtårn Trondheim—Samtykkebasert kjernejournal: Kravspesifikasjon sentral løsning. Versjon 0.6 [The "Beacon” Project in Trondheim—consent-based core records: Requirements specification central solution], KITH rapport 31/05, pp. 1–18. KITH (2006). Fyrtårn Trondheim—Samtykkebasert kjernejournal: Kravspesifikasjon sentral løsning. Versjon 0.6 [The "Beacon” Project in Trondheim—consent-based core records: Requirements specification central solution], KITH rapport 31/05, pp. 1–18.
Zurück zum Zitat KITH (2008). ELIN-K Prosjektet. Elektronisk informasjonsutveksling I pleie og omsorgstjenesten i kommunene. Sammen skaper vi en enklere og tryggere hverdag for helsearbeidere og pasienter. [The ELIN-K Project. Electronic information exchange in municipal nursing and care services] KITH rapport 06/08, pp. 1–32. KITH (2008). ELIN-K Prosjektet. Elektronisk informasjonsutveksling I pleie og omsorgstjenesten i kommunene. Sammen skaper vi en enklere og tryggere hverdag for helsearbeidere og pasienter. [The ELIN-K Project. Electronic information exchange in municipal nursing and care services] KITH rapport 06/08, pp. 1–32.
Zurück zum Zitat Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67–94.CrossRef Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67–94.CrossRef
Zurück zum Zitat Latour, B. (1987). Science in action. Cambridge: Harvard University Press. Latour, B. (1987). Science in action. Cambridge: Harvard University Press.
Zurück zum Zitat Latour, B. (2005). Reassembling the social. An introduction to actor-network-theory. Oxford University Press. Latour, B. (2005). Reassembling the social. An introduction to actor-network-theory. Oxford University Press.
Zurück zum Zitat Law, J. (1987). Technology and heterogeneous engineering: The case of Portuguese expansion. In W. E. Bijker, T. P. Hughes, T. Pinch (eds.). The social construction of technological systems (pp. 111–134). Massachusetts Institute of Technology, The MIT Press. Law, J. (1987). Technology and heterogeneous engineering: The case of Portuguese expansion. In W. E. Bijker, T. P. Hughes, T. Pinch (eds.). The social construction of technological systems (pp. 111–134). Massachusetts Institute of Technology, The MIT Press.
Zurück zum Zitat LeRouge, C., Mantzana, V., & Vance Wilson, E. (2007). Healthcare information systems research, revelations and visions. European Journal of Information Systems, 16(6), 669–671.CrossRef LeRouge, C., Mantzana, V., & Vance Wilson, E. (2007). Healthcare information systems research, revelations and visions. European Journal of Information Systems, 16(6), 669–671.CrossRef
Zurück zum Zitat Ministry of Health and Care Services. (2008). National strategy for electronic interaction in health and community care (2008–2013). Oslo: Norwegian Ministry of Health and Care Services. Ministry of Health and Care Services. (2008). National strategy for electronic interaction in health and community care (2008–2013). Oslo: Norwegian Ministry of Health and Care Services.
Zurück zum Zitat Monteiro, E. (2000). Actor-network theory and information infrastructure. In C. U. Ciborra (Ed.), From control to drift: The dynamics of corporate information infrastructures. Oxford: Oxford University Press. Monteiro, E. (2000). Actor-network theory and information infrastructure. In C. U. Ciborra (Ed.), From control to drift: The dynamics of corporate information infrastructures. Oxford: Oxford University Press.
Zurück zum Zitat Orlikowski, W. J. (1996). Improvising organizational transformation over time: a situated change perspective. Information Systems Research, 7(1), 63–92.CrossRef Orlikowski, W. J. (1996). Improvising organizational transformation over time: a situated change perspective. Information Systems Research, 7(1), 63–92.CrossRef
Zurück zum Zitat OUS. (2009). Project report for common clinical information platform. Oslo: Oslo University Hospital. OUS. (2009). Project report for common clinical information platform. Oslo: Oslo University Hospital.
Zurück zum Zitat Pollock, N. (2005). When is a work-around? Conflict and negotiation in computer systems development. Science, Technology, & Human Values, 30(4), 496–514.CrossRef Pollock, N. (2005). When is a work-around? Conflict and negotiation in computer systems development. Science, Technology, & Human Values, 30(4), 496–514.CrossRef
Zurück zum Zitat Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: design and access for large information spaces. Information Systems Research, 7(1), 111–134.CrossRef Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: design and access for large information spaces. Information Systems Research, 7(1), 111–134.CrossRef
Zurück zum Zitat Trägårdh, B., & Lindberg, K. (2004). Curing a meagre health care system by lean methods—translating ’chains of care’ in the Swedish health care sector. The International Journal of Health Planning and Management, 19(4), 383–398.CrossRef Trägårdh, B., & Lindberg, K. (2004). Curing a meagre health care system by lean methods—translating ’chains of care’ in the Swedish health care sector. The International Journal of Health Planning and Management, 19(4), 383–398.CrossRef
Zurück zum Zitat Tsiknakis, M., Katehakis, D. G., & Orphanoudakis, S. C. (2002). An open, component-based information infrastructure for integrated health information networks. International Journal of Medical Informatics, 68, 3–26.CrossRef Tsiknakis, M., Katehakis, D. G., & Orphanoudakis, S. C. (2002). An open, component-based information infrastructure for integrated health information networks. International Journal of Medical Informatics, 68, 3–26.CrossRef
Zurück zum Zitat Walsham, G. (1995). Interpretive case studies in IS research: nature and method. European Journal of Information systems, 4, 74–81.CrossRef Walsham, G. (1995). Interpretive case studies in IS research: nature and method. European Journal of Information systems, 4, 74–81.CrossRef
Zurück zum Zitat Wainwright, D., & Waring, T. (2004). Three domains for implementing integrated information systems: redressing the balance between technology strategic and organisational analysis. International Journal of Information Management, 24, 329–346.CrossRef Wainwright, D., & Waring, T. (2004). Three domains for implementing integrated information systems: redressing the balance between technology strategic and organisational analysis. International Journal of Information Management, 24, 329–346.CrossRef
Zurück zum Zitat Winthereik, B. R., & Vikkelsø, S. (2005). ICT and integrated care: some dilemmas of standardising inter-organisational communication. Computer Supported Cooperative Work, 14(1), 43–67.CrossRef Winthereik, B. R., & Vikkelsø, S. (2005). ICT and integrated care: some dilemmas of standardising inter-organisational communication. Computer Supported Cooperative Work, 14(1), 43–67.CrossRef
Zurück zum Zitat Xu, Y., Sauquet, D., Zapletal, E., Lemaitre, D., & Degoulet, P. (2000). Integration of medical applications: the ‘Mediator service’ of the SynEx platform. International Journal of Medical Informatics, 58–59, 157–166.CrossRef Xu, Y., Sauquet, D., Zapletal, E., Lemaitre, D., & Degoulet, P. (2000). Integration of medical applications: the ‘Mediator service’ of the SynEx platform. International Journal of Medical Informatics, 58–59, 157–166.CrossRef
Metadaten
Titel
The Role of Integration in Health-Based Information Infrastructures
verfasst von
Gunnar Ellingsen
Kristoffer Røed
Publikationsdatum
01.12.2010
Verlag
Springer Netherlands
Erschienen in
Computer Supported Cooperative Work (CSCW) / Ausgabe 6/2010
Print ISSN: 0925-9724
Elektronische ISSN: 1573-7551
DOI
https://doi.org/10.1007/s10606-010-9122-y

Weitere Artikel der Ausgabe 6/2010

Computer Supported Cooperative Work (CSCW) 6/2010 Zur Ausgabe

Premium Partner